WO2021197158A1 - 一种图像处理方法和图像处理装置 - Google Patents

一种图像处理方法和图像处理装置 Download PDF

Info

Publication number
WO2021197158A1
WO2021197158A1 PCT/CN2021/082657 CN2021082657W WO2021197158A1 WO 2021197158 A1 WO2021197158 A1 WO 2021197158A1 CN 2021082657 W CN2021082657 W CN 2021082657W WO 2021197158 A1 WO2021197158 A1 WO 2021197158A1
Authority
WO
WIPO (PCT)
Prior art keywords
coefficient
transform coefficient
transform
decoded
image
Prior art date
Application number
PCT/CN2021/082657
Other languages
English (en)
French (fr)
Inventor
杨海涛
张恋
马祥
Original Assignee
华为技术有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 华为技术有限公司 filed Critical 华为技术有限公司
Priority to EP21779779.4A priority Critical patent/EP4117289A4/en
Publication of WO2021197158A1 publication Critical patent/WO2021197158A1/zh
Priority to US17/954,861 priority patent/US12022078B2/en

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/102—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or selection affected or controlled by the adaptive coding
    • H04N19/13—Adaptive entropy coding, e.g. adaptive variable length coding [AVLC] or context adaptive binary arithmetic coding [CABAC]
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/169—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding
    • H04N19/17—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being an image region, e.g. an object
    • H04N19/176—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being an image region, e.g. an object the region being a block, e.g. a macroblock
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N1/00—Scanning, transmission or reproduction of documents or the like, e.g. facsimile transmission; Details thereof
    • H04N1/41—Bandwidth or redundancy reduction
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/102—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or selection affected or controlled by the adaptive coding
    • H04N19/129—Scanning of coding units, e.g. zig-zag scan of transform coefficients or flexible macroblock ordering [FMO]
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/169—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding
    • H04N19/17—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being an image region, e.g. an object
    • H04N19/172—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being an image region, e.g. an object the region being a picture, frame or field
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/169—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding
    • H04N19/18—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being a set of transform coefficients
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/42—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals characterised by implementation details or hardware specially adapted for video compression or decompression, e.g. dedicated software implementation
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/60—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using transform coding
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/60—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using transform coding
    • H04N19/63—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using transform coding using sub-band based transform, e.g. wavelets
    • H04N19/64—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using transform coding using sub-band based transform, e.g. wavelets characterised by ordering of coefficients or of bits for transmission
    • H04N19/645—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using transform coding using sub-band based transform, e.g. wavelets characterised by ordering of coefficients or of bits for transmission by grouping of coefficients into blocks after the transform
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/90—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using coding techniques not provided for in groups H04N19/10-H04N19/85, e.g. fractals
    • H04N19/91—Entropy coding, e.g. variable length coding [VLC] or arithmetic coding

Definitions

  • This application relates to image processing neighborhoods, and in particular to an image processing method and image processing device.
  • image storage and transmission in electronic devices have become an indispensable processing method, and image compression is becoming more and more important in saving transmission bandwidth and storage devices.
  • JPEG Joint Photographic Experts Group
  • JPEG2000 JPEG2000
  • H.264 intra-frame coding H.265 intra-frame coding
  • H.265 intra-frame coding JPEG standard
  • JPEG standard is widely used in hardware devices such as video surveillance cameras, and the generated image files can also be decoded by electronic devices for playback.
  • the encoder first transforms the image block, and the method used for the transformation includes discrete cosine transform (DCT).
  • DCT discrete cosine transform
  • the low-frequency components of the image block are concentrated in the upper left corner of the image, and the high-frequency components are distributed in the lower right corner of the image.
  • the coefficient in the first row and first column of the image block is the direct coefficient (DC) coefficient. That is, the average value of the image block, and the other 63 coefficients in the image block are alternating coefficient (AC) coefficients.
  • DC coefficients and AC coefficients are quantized to obtain the quantized DC coefficients and AC coefficients.
  • the quantized DC coefficient and AC coefficient are compressed.
  • the decoding end decodes, inversely quantizes, and inversely transforms the compressed image data to obtain reconstructed image blocks.
  • the compressed file generated by the above image compression method consumes a large amount of storage resources of the decoding end, which increases the storage resource overhead of the decoding end.
  • the embodiments of the present application provide an image processing method and an image processing device, which are used to improve image compression efficiency and reduce the storage cost of image coding information.
  • an embodiment of the present application provides an image processing method, including: acquiring at least two transform coefficient blocks, each of the at least two transform coefficient blocks includes: at least two transform coefficients, wherein, Each transform coefficient in a transform coefficient block corresponds to a different frequency point; the transform coefficients in the at least two transform coefficient blocks corresponding to one frequency point are determined according to a preset frequency point scanning order; according to the preset coefficient scanning order Encoding each transform coefficient corresponding to the one frequency point to obtain image encoding information.
  • the frequency point scanning order can be preset or configured, and a frequency point is determined based on the frequency point scanning order.
  • the frequency point can be any frequency point, and each frequency point corresponds to at least two transform coefficient blocks.
  • One transform coefficient in each transform coefficient block in each transform coefficient block that is, one frequency point corresponds to multiple transform coefficients, and these multiple transform coefficients come from different transform coefficient blocks.
  • the coefficient scanning order can be preset or configured, based on the coefficient scanning order Encoding each transform coefficient corresponding to a frequency point can finally obtain image encoding information.
  • the image coding information can be obtained only by coding according to the coefficient scanning order. Therefore, coding can be realized for multiple transform coefficients corresponding to the same frequency point.
  • the image compression efficiency is improved, and the image encoding information in the embodiment of the application is obtained by encoding the transform coefficients corresponding to the same frequency points in at least two transform coefficient blocks according to the coefficient scanning order, and the file of the encoded image encoding information is reduced, Reduce the storage cost of image coding information, and ultimately save storage resources.
  • the determining the transform coefficients in the at least two transform coefficient blocks corresponding to one frequency point according to a preset frequency point scanning order includes: from the at least two transform coefficient blocks Obtain at least two two-dimensional subband coefficient arrays, where one two-dimensional subband coefficient array includes transform coefficients corresponding to the same frequency points in the at least two transform coefficient blocks, and the different two-dimensional subband coefficient arrays include transforms The coefficients correspond to different frequency points; the first two-dimensional sub-band coefficient array of the at least two two-dimensional sub-band coefficient arrays is determined according to a preset frequency point scanning order; Encoding each transform coefficient corresponding to the one frequency point includes: determining the first transform coefficient in the first two-dimensional subband coefficient array according to a preset coefficient scanning order; and according to the context area corresponding to the first transform coefficient Entropy coding is performed on the first transform coefficient.
  • the encoding end can determine the first transform coefficient in the first two-dimensional subband coefficient array according to the preset coefficient scanning order, and the first transform coefficient can be any transform in the first two-dimensional subband coefficient array.
  • the encoding end may respectively perform lossless encoding on each transform coefficient in the first two-dimensional subband coefficient array, for example, may perform entropy encoding on the first transform coefficient according to the context region corresponding to the first transform coefficient.
  • image encoding information can be obtained, and image compression processing can be implemented.
  • the scanning order of the three-dimensional sub-band coefficients is the order in which the three-dimensional sub-band coefficient array is scanned.
  • the encoding end may respectively perform entropy encoding on each transform coefficient in the first two-dimensional subband coefficient array according to a preset coefficient scanning order.
  • image encoding information can be obtained, and image compression processing can be implemented.
  • the determining the transform coefficients in the at least two transform coefficient blocks corresponding to one frequency point according to a preset frequency point scanning order includes: comparing all the transform coefficients in the at least two transform coefficient blocks corresponding to a frequency point according to a preset frequency point scanning order.
  • the at least two transform coefficient blocks are sorted to obtain at least two sorted transform coefficient blocks; multiple two-dimensional subband coefficient arrays corresponding to different frequency points are obtained from the at least two sorted transform coefficient blocks;
  • the multiple two-dimensional sub-band coefficient arrays corresponding to the different frequency points are combined to obtain a three-dimensional sub-band coefficient array;
  • the first two-dimensional sub-band coefficient array is determined from the three-dimensional sub-band coefficient array according to a preset scanning order of the three-dimensional sub-band coefficients.
  • a subband coefficient array; said encoding each transform coefficient corresponding to the one frequency point according to a preset coefficient scanning order includes: determining the first two-dimensional subband coefficient array in the first two-dimensional subband coefficient array according to the preset coefficient scanning order A transform coefficient; performing entropy coding on the first transform coefficient according to the context region corresponding to the first transform coefficient.
  • the encoding end may respectively perform entropy encoding on each transform coefficient in the first two-dimensional subband coefficient array according to a preset coefficient scanning order.
  • image encoding information can be obtained, and image compression processing can be implemented.
  • the scanning order of the three-dimensional subband coefficients includes: a preset change order of the horizontal position index; or, a preset change order of the vertical position index; or, a preset frequency index The order of change.
  • each transform coefficient in the three-dimensional subband coefficient array has a horizontal position index, a vertical position index, and a frequency index.
  • each transform coefficient can be expressed as C(bx, by, fs), where , Bx is the horizontal position index, by is the vertical position index, and fs is the frequency index corresponding to the frequency point.
  • the three-dimensional sub-band coefficient scanning order is the order in which the three-dimensional sub-band coefficient array is scanned.
  • the scanning order of the three-dimensional sub-band coefficients determines the multiple two-dimensional sub-band coefficient arrays in the three-dimensional sub-band coefficient array according to the change order of the horizontal position index from large to small (or from small to large).
  • a transform coefficient in the two-dimensional sub-band coefficient array is determined according to the aforementioned coefficient scanning order, that is, the three-dimensional sub-band coefficient scanning order can be used to indicate the corresponding index to different horizontal positions in the three-dimensional sub-band coefficient array.
  • the two-dimensional subband coefficient array is encoded.
  • the determining the transform coefficients in the at least two transform coefficient blocks corresponding to one frequency point according to a preset frequency point scanning order includes: determining all the transform coefficients according to a preset frequency point scanning order.
  • a first two-dimensional subband coefficient array composed of transform coefficients corresponding to the same frequency point in the at least two transform coefficient blocks; said encoding each transform coefficient corresponding to the one frequency point according to a preset coefficient scanning order,
  • the method includes: determining the first transform coefficient in the first two-dimensional subband coefficient array according to a preset coefficient scanning order; and performing entropy coding on the first transform coefficient according to a context region corresponding to the first transform coefficient.
  • the encoding end does not need to combine a three-dimensional subband coefficient array.
  • the encoding end may respectively encode each transform coefficient in the first two-dimensional sub-band coefficient array according to a preset coefficient scanning order. For example, the encoding end determines the first transform coefficient in the first two-dimensional subband coefficient array according to a preset coefficient scanning order, and performs entropy encoding on the first transform coefficient according to the context region corresponding to the first transform coefficient.
  • image encoding information can be obtained, and image compression processing can be implemented.
  • the determining the transform coefficients in the at least two transform coefficient blocks corresponding to one frequency point according to a preset frequency point scanning order includes: determining one according to a preset frequency point scanning order The transform coefficients in the at least two transform coefficient blocks included in the frequency subband correspond to the same frequency point; the transform coefficients in the at least two transform coefficient blocks included in the one frequency subband correspond to the same frequency point; Encoding the transform coefficients corresponding to the one frequency point in the coefficient scanning order includes: determining the first transform coefficient included in the one frequency subband according to a preset coefficient scanning order; The context area performs entropy coding on the first transform coefficient.
  • the first transform coefficient may be any transform coefficient of a plurality of transform coefficients included in a frequency subband, and the first transform coefficient is entropy-coded according to the context region corresponding to the first transform coefficient.
  • image encoding information can be obtained, and image compression processing can be implemented.
  • the performing entropy coding on the first transform coefficient according to the context region corresponding to the first transform coefficient includes: obtaining the context region corresponding to the first transform coefficient.
  • the context information of the first transform coefficient wherein the context information of the first transform coefficient includes: the encoded transform coefficient in the context area corresponding to the first transform coefficient;
  • the first transform coefficient is estimated to obtain probability distribution information of the first transform coefficient; and the first transform coefficient is arithmetic-coded according to the probability distribution information of the first transform coefficient.
  • the encoding end performs a probability estimation on the first transform coefficient according to the context information of the first transform coefficient to obtain the probability distribution of the first transform coefficient, and then the encoding end performs a probability estimation on the first transform coefficient according to the probability distribution of the first transform coefficient. Perform arithmetic coding to obtain image coding information corresponding to the first transform coefficient.
  • the context area corresponding to the first transform coefficient includes at least one of the following: a spatial neighborhood of the first transform coefficient, a subband neighborhood of the first transform coefficient, and The three-dimensional neighborhood of the first transform coefficient.
  • the performing entropy coding on the first transform coefficient according to the context region corresponding to the first transform coefficient includes: when the first transform coefficient is a plurality of transform coefficients, from Acquire the context information of the multiple transform coefficients in the context regions corresponding to the multiple transform coefficients, where the context information of the multiple transform coefficients includes: coded transforms in the context regions corresponding to the multiple transform coefficients Coefficient; according to the context information of the multiple transformation coefficients, the multiple transformation coefficients are estimated in parallel to obtain the probability distribution information of the multiple transformation coefficients; according to the probability distribution information of the multiple transformation coefficients, the The multiple transform coefficients are arithmetic coding in parallel.
  • the encoding end can obtain multiple transform coefficients, and the encoding end performs entropy encoding on multiple transform coefficients in parallel according to the context information of the multiple transform coefficients.
  • the image encoding efficiency can be improved.
  • the encoder can also perform probability estimation on the probability distributions of multiple transform coefficients in parallel, that is, obtain the probability distributions of multiple transform coefficients at a time, and then perform parallel analysis on multiple transform coefficients. Perform arithmetic coding to improve coding efficiency.
  • the context regions corresponding to the multiple transform coefficients include at least one of the following: spatial neighborhoods of the multiple transform coefficients, subband neighborhoods of the multiple transform coefficients, and A three-dimensional neighborhood of multiple transform coefficients.
  • the determining the transform coefficients in the at least two transform coefficient blocks corresponding to one frequency point according to a preset frequency point scanning order includes: from the at least two transform coefficient blocks Obtain at least two two-dimensional subband coefficient arrays, where one two-dimensional subband coefficient array includes transform coefficients corresponding to the same frequency points in the at least two transform coefficient blocks, and the different two-dimensional subband coefficient arrays include transforms The coefficients correspond to different frequency points; the at least two sub-band images corresponding to the at least two two-dimensional sub-band coefficient arrays are determined according to the preset frequency point scanning order; the one frequency is determined according to the preset coefficient scanning order Encoding the transform coefficients corresponding to the points includes: encoding the transform coefficients in the at least two sub-band images according to a preset coefficient scanning order.
  • the encoding end after the encoding end obtains at least two two-dimensional sub-band coefficient arrays, the encoding end adopts the method of lossless encoding of the sub-band images, and uses at least two two-dimensional sub-band coefficient arrays as the at least two sub-band images. , Perform image lossless encoding on the transform coefficients in at least two sub-band images according to the preset frequency point scanning order.
  • the encoding end can encode the two-dimensional sub-band coefficient array as a sub-band image during T-shirt lossless encoding, which simplifies The realization process of lossless coding.
  • the acquiring at least two transform coefficient blocks includes: acquiring transform coefficient encoding information of at least two image blocks; performing entropy decoding on the transform coefficient encoding information to obtain the at least two transform coefficient blocks.
  • the transform coefficient encoding information may be transform coefficient encoding information corresponding to at least two image blocks.
  • the at least two image blocks may be image blocks in a JPEG file, and the image processing device may use a Huffman decoding method to Entropy decoding is performed on transform coefficient coding information of all image blocks in the JPEG file to obtain at least two transform coefficient blocks corresponding to at least two image blocks.
  • the transform coefficient encoding information is entropy-decoded to obtain at least two transform coefficient blocks, and the transform coefficient blocks are encoded to obtain image encoding information and realize image compression processing.
  • the acquiring at least two transform coefficient blocks includes: dividing an original image area into at least two image blocks, and the original image area includes: a pixel image area or a residual image area;
  • the at least two image blocks are transformed and quantized to obtain the at least two transform coefficient blocks.
  • a transform coefficient block can be generated by transforming and quantizing an image block, so that the transform coefficient block can be coded, image coding information can be obtained, and image compression processing can be realized.
  • an embodiment of the present application also provides an image processing method, including: acquiring image encoding information; determining a transform coefficient to be decoded corresponding to a frequency point in the image encoding information according to a preset frequency point scanning order; In the coefficient scanning order, the transform coefficient to be decoded corresponding to the one frequency point is decoded to obtain at least two transform coefficient blocks, wherein each transform coefficient block in the at least two transform coefficient blocks includes: at least two transform coefficient blocks; Each transform coefficient in a transform coefficient block corresponds to a different frequency point.
  • the frequency point scanning order can be preset or configured, and the transform coefficient to be decoded corresponding to a frequency point in the image encoding information is determined based on the frequency point scanning order.
  • the frequency point can be any frequency point, which can be pre-determined.
  • the coefficient scanning order is set, and the transform coefficient to be decoded corresponding to at least one frequency point is decoded based on the coefficient scanning order to obtain at least two transform coefficient blocks, each of the at least two transform coefficient blocks includes: at least two Transform coefficients. Each transform coefficient in a transform coefficient block corresponds to a different frequency point.
  • lossless decoding is performed on the transform coefficients to be decoded corresponding to the same frequency point in the image encoding information, and at least two transform coefficient blocks can be obtained by decoding in accordance with the coefficient scanning order. Therefore, for multiple transform coefficient blocks corresponding to the same frequency point.
  • the transform coefficients can be decoded, and the image encoding information in the embodiment of the application is obtained by encoding the transform coefficients corresponding to the same frequency points in at least two transform coefficient blocks according to the coefficient scanning order, and the file of the encoded image encoding information is reduced. Reduce the storage cost of image coding information, and ultimately save storage resources.
  • the determining the transform coefficient to be decoded corresponding to a frequency point in the image encoding information according to a preset frequency point scanning order includes: determining the image according to a preset frequency point scanning order
  • the first to-be-decoded two-dimensional sub-band coefficient array in the encoded information includes: the to-be-decoded transform coefficient corresponding to a frequency point in the image encoding information
  • Decoding the transform coefficient to be decoded corresponding to the one frequency point in the coefficient scanning order includes: determining the first transform coefficient to be decoded in the first two-dimensional subband coefficient array to be decoded according to a preset coefficient scanning order; Entropy decoding the first transform coefficient to be decoded according to the context region corresponding to the first transform coefficient to be decoded.
  • the decoding end adopts the image decoding method corresponding to the encoding end. If the encoding end executes the encoding methods shown in steps C1 to C4, the image encoding information received by the decoding end may include at least two to-be-decoded images. Two-dimensional array of subband coefficients.
  • the frequency point scanning order is the scanning order of the two-dimensional subband coefficient array to be decoded
  • the first two-dimensional subband coefficient array to be decoded is obtained from at least two two-dimensional subband coefficient arrays to be decoded according to the preset frequency point scanning order
  • the first two-dimensional subband coefficient array to be decoded includes a plurality of transform coefficients to be decoded.
  • the determining the transform coefficient to be decoded corresponding to a frequency point in the image encoding information according to a preset frequency point scanning order includes: determining the image according to a preset frequency point scanning order
  • the to-be-decoded three-dimensional sub-band coefficient array in the encoded information, the to-be-decoded three-dimensional sub-band coefficient array includes: the to-be-decoded transform coefficients corresponding to all frequency points in the image encoding information; according to the preset scanning order of the three-dimensional sub-band coefficients Determine a first to-be-decoded two-dimensional sub-band coefficient array in the to-be-decoded three-dimensional sub-band coefficient array, where the first to-be-decoded two-dimensional sub-band coefficient array includes: the to-be-decoded corresponding to a frequency point in the image encoding information Transform coefficients; said decoding the transform coefficients to be decoded corresponding to the one frequency point according to the preset coefficient scanning order, including:
  • the decoding end adopts the image decoding method corresponding to the encoding end.
  • the coefficient scanning order preset by the decoding end performs the second two-dimensional subband coefficient array. Lossless decoding to obtain the at least two transform coefficient blocks.
  • the scanning order of the three-dimensional subband coefficients includes: a preset change order of the horizontal position index; or, a preset change order of the vertical position index; or, a preset frequency index The order of change.
  • the determining the transform coefficient to be decoded corresponding to a frequency point in the image encoding information according to a preset frequency point scanning order includes: determining the image according to a preset frequency point scanning order The to-be-decoded transform coefficients included in one frequency sub-band in the encoded information correspond to the same frequency point; the one frequency point corresponding to the one frequency point in the preset coefficient scanning order
  • the decoding of the transform coefficient to be decoded includes: determining the first transform coefficient to be decoded in the one frequency subband according to a preset coefficient scanning order; The transform coefficients to be decoded are subjected to entropy decoding.
  • the decoding end obtains image encoding information corresponding to multiple frequency subbands, and the decoding end adopts a method of lossless decoding of the frequency subbands, using at least two two-dimensional subband coefficient arrays as at least two frequency subbands, Perform lossless decoding on the image encoding information corresponding to at least two frequency subbands according to the preset coefficient scanning order.
  • the decoding end can decode the two-dimensional subband coefficient array as a frequency subband during lossless decoding, which simplifies the lossless decoding process. Implementation process.
  • the entropy decoding of the first transform coefficient to be decoded according to the context region corresponding to the first transform coefficient to be decoded includes: Acquire the context information of the first transform coefficient to be decoded in the context area, wherein the context area corresponding to the first transform coefficient to be decoded includes: the decoded transform coefficient in the context area corresponding to the first transform coefficient to be decoded Estimate the first transform coefficient to be decoded according to the context information of the first transform coefficient to be decoded to obtain the probability distribution information of the first transform coefficient to be decoded; The probability distribution information performs arithmetic decoding on the first transform coefficient to be decoded to obtain a decoded first transform coefficient, and the decoded first transform coefficient belongs to the at least two transform coefficient blocks.
  • the decoder after the decoder obtains the context information of the first transform coefficient, the decoder performs entropy decoding on the first transform coefficient according to the context information of the first transform coefficient to obtain the decoded first transform coefficient.
  • the entropy decoding performed by the decoding end may include probability estimation and arithmetic decoding. Specifically, the decoder performs a probability estimation on the first transform coefficient according to the context information of the first transform coefficient to obtain the probability distribution of the first transform coefficient, and then the decoder performs arithmetic on the first transform coefficient according to the probability distribution of the first transform coefficient Decoding to obtain image decoding information corresponding to the first transform coefficient.
  • the context area corresponding to the first transform coefficient to be decoded includes at least one of the following: a spatial neighborhood of the first transform coefficient to be decoded, and a sub-region of the first transform coefficient to be decoded Band neighborhood, the three-dimensional neighborhood of the first transform coefficient to be decoded.
  • the performing entropy decoding on the first transform coefficient to be decoded according to the context region corresponding to the first transform coefficient to be decoded includes: when the first transform coefficient to be decoded is multiple
  • the context information of the plurality of transform coefficients to be decoded is obtained from the context regions corresponding to the plurality of transform coefficients to be decoded
  • the plurality of context information to be decoded includes: The decoded transform coefficients in the context area corresponding to the transform coefficients to be decoded; the plurality of transform coefficients to be decoded are estimated in parallel according to the context information of the transform coefficients to be decoded, to obtain the first to-be-decoded transform coefficients
  • the probability distribution information of the transform coefficients according to the probability distribution information of the plurality of transform coefficients to be decoded, arithmetic decoding is performed on the plurality of transform coefficients to be decoded in parallel to obtain the de
  • the context regions corresponding to the plurality of transform coefficients to be decoded include at least one of the following: spatial neighborhoods of the plurality of transform coefficients to be decoded, and sub-regions of the plurality of transform coefficients to be decoded With neighborhood, the three-dimensional neighborhood of the plurality of transform coefficients to be decoded.
  • the determining the transform coefficient to be decoded corresponding to a frequency point in the image encoding information according to a preset frequency point scanning order includes: determining the image according to a preset frequency point scanning order
  • the first sub-band image in the encoded information, the first sub-band image includes: a transform coefficient to be decoded corresponding to a frequency point in the image encoding information
  • Decoding the to-be-decoded transform coefficients corresponding to the points includes: decoding the to-be-decoded coefficients of the first subband image according to a preset coefficient scanning order to obtain the at least two transform coefficient blocks.
  • the decoding end uses lossless decoding for sub-band images, and after obtaining multiple transform coefficients, the decoding end uses lossless decoding of the sub-band images, using at least two two-dimensional sub-band coefficient arrays as at least two sub-bands.
  • Image according to the preset coefficient scanning order, the image encoding information corresponding to at least two sub-band images is decoded losslessly.
  • the decoding end can decode the two-dimensional sub-band coefficient array as a sub-band image during lossless decoding, which simplifies the lossless decoding The realization process.
  • the method further includes: performing entropy coding on the at least two transform coefficient blocks to obtain transform coefficient coding information;
  • the coefficient encoding information generates at least two image blocks; the original image area is restored according to the at least two image blocks, and the original image area includes: a pixel image area or a residual image area.
  • the decoding end adopts the image decoding method corresponding to the encoding end. If the encoding end obtains the transform coefficient block by entropy decoding the transform coefficient coding information, and encodes the transform coefficient block, the decoding end can perform at least two Entropy coding is performed on each transform coefficient block to obtain transform coefficient coding information.
  • at least two image blocks are generated according to the transform coefficient encoding information, and the original image area is restored according to the at least two image blocks, so that the image decompression process can be completed.
  • the method further includes: performing inverse quantization and inverse transform on the at least two transform coefficient blocks to obtain at least two image blocks;
  • the original image area is restored according to the at least two image blocks, and the original image area includes: a pixel image area or a residual image area.
  • the decoding end after the decoding end generates at least two transform coefficient blocks, the decoding end generates at least two image blocks according to the at least two transform coefficient blocks.
  • the decoding end can process at least two transform coefficient blocks by means of inverse transformation and inverse quantization, so as to generate at least two image blocks.
  • an embodiment of the present application provides an image processing device, including: a transceiver module, configured to obtain at least two transform coefficient blocks, each of the at least two transform coefficient blocks includes: at least two Transform coefficients, wherein each transform coefficient in one transform coefficient block corresponds to different frequency points; the processing module is configured to determine the at least two transform coefficient blocks corresponding to one frequency point according to a preset frequency point scanning order Transform coefficients; according to a preset coefficient scanning order, each transform coefficient corresponding to the one frequency point is encoded to obtain image encoding information.
  • the frequency point scanning order can be preset or configured, and a frequency point is determined based on the frequency point scanning order.
  • the frequency point can be any frequency point, and each frequency point corresponds to at least two transform coefficient blocks.
  • a transform coefficient in each transform coefficient block in each transform coefficient block, that is, a frequency point corresponds to multiple transform coefficients. These multiple transform coefficients are derived from different transform coefficient blocks.
  • the coefficient scanning order can be preset or configured based on the coefficient scanning order Encoding each transform coefficient corresponding to a frequency point can finally obtain image encoding information.
  • the image coding information can be obtained only by coding according to the coefficient scanning order. Therefore, coding can be realized for multiple transform coefficients corresponding to the same frequency point.
  • the image compression efficiency is improved, and the image encoding information in the embodiment of the application is obtained by encoding the transform coefficients corresponding to the same frequency points in at least two transform coefficient blocks according to the coefficient scanning order, and the file of the encoded image encoding information is reduced, Reduce the storage cost of image coding information, and ultimately save storage resources.
  • the processing module is configured to obtain at least two two-dimensional sub-band coefficient arrays from the at least two transform coefficient blocks, wherein one two-dimensional sub-band coefficient array includes the at least two transform coefficient blocks.
  • the transform coefficients corresponding to the same frequency points in the two transform coefficient blocks, and the transform coefficients included in the different two-dimensional subband coefficient arrays correspond to different frequency points; the at least two two-dimensional frequency points are determined according to a preset scanning order of frequency points.
  • the first two-dimensional sub-band coefficient array in the sub-band coefficient array; the first transform coefficient in the first two-dimensional sub-band coefficient array is determined according to a preset coefficient scanning order; according to the context corresponding to the first transform coefficient
  • the region performs entropy coding on the first transform coefficient.
  • the processing module is configured to obtain at least two two-dimensional sub-band coefficient arrays from the at least two transform coefficient blocks, wherein one two-dimensional sub-band coefficient array includes the at least two transform coefficient blocks.
  • the transform coefficients corresponding to the same frequency points in the two transform coefficient blocks, the transform coefficients included in the different two-dimensional subband coefficient arrays correspond to different frequency points;
  • the sub-band coefficient arrays are combined to obtain a three-dimensional sub-band coefficient array; the first two-dimensional sub-band coefficient array in the three-dimensional sub-band coefficient array is determined according to a preset scanning order of three-dimensional sub-band coefficients; and the first two-dimensional sub-band coefficient array is scanned according to the preset coefficients.
  • the first transform coefficient in the first two-dimensional subband coefficient array is determined in order; and the first transform coefficient is entropy-encoded according to the context region corresponding to the first transform coefficient.
  • the processing module is configured to sort the at least two transform coefficient blocks according to a preset frequency point scanning order to obtain the sorted at least two transform coefficient blocks; Obtain multiple two-dimensional subband coefficient arrays corresponding to different frequency points from the sorted at least two transform coefficient blocks; combine the multiple two-dimensional subband coefficient arrays corresponding to the different frequency points to obtain three-dimensional subband coefficients Array; determine the first two-dimensional sub-band coefficient array from the three-dimensional sub-band coefficient array according to the preset three-dimensional sub-band coefficient scanning order; determine the first two-dimensional sub-band coefficient array in the first two-dimensional sub-band coefficient array according to the preset coefficient scanning order A first transform coefficient; performing entropy coding on the first transform coefficient according to the context region corresponding to the first transform coefficient.
  • the scanning order of the three-dimensional subband coefficients includes: a preset change order of the horizontal position index; or, a preset change order of the vertical position index; or, a preset frequency index The order of change.
  • the processing module is configured to determine, according to a preset frequency point scanning order, a first two-dimensional subband composed of transform coefficients corresponding to the same frequency point in the at least two transform coefficient blocks Coefficient array; determine the first transform coefficient in the first two-dimensional subband coefficient array according to a preset coefficient scanning order; perform entropy coding on the first transform coefficient according to the context area corresponding to the first transform coefficient.
  • the processing module is configured to determine the transform coefficients in the at least two transform coefficient blocks included in one frequency subband according to a preset frequency point scanning order, and the one frequency subband The included transform coefficients in the at least two transform coefficient blocks correspond to the same frequency point; the first transform coefficient included in the one frequency subband is determined according to the preset coefficient scanning order; the first transform coefficient included in the one frequency subband is determined according to the corresponding first transform coefficient The context area of performs entropy coding on the first transform coefficient.
  • the processing module is configured to obtain context information of the first transform coefficient from a context region corresponding to the first transform coefficient, wherein the context information of the first transform coefficient
  • the method includes: the encoded transform coefficient in the context area corresponding to the first transform coefficient; estimating the first transform coefficient according to the context information of the first transform coefficient to obtain the probability distribution of the first transform coefficient Information; Perform arithmetic coding on the first transform coefficient according to the probability distribution information of the first transform coefficient.
  • the context area corresponding to the first transform coefficient includes at least one of the following: a spatial neighborhood of the first transform coefficient, a subband neighborhood of the first transform coefficient, and The three-dimensional neighborhood of the first transform coefficient.
  • the processing module is configured to, when the first transform coefficient is a plurality of transform coefficients, obtain the information of the plurality of transform coefficients from the context regions corresponding to the plurality of transform coefficients.
  • Context information wherein the context information of the plurality of transform coefficients includes: encoded transform coefficients in the context regions corresponding to the plurality of transform coefficients; Estimation is performed in parallel to obtain the probability distribution information of the plurality of transform coefficients; and the arithmetic coding is performed on the plurality of transform coefficients in parallel according to the probability distribution information of the plurality of transform coefficients.
  • the context regions corresponding to the multiple transform coefficients include at least one of the following: spatial neighborhoods of the multiple transform coefficients, subband neighborhoods of the multiple transform coefficients, and A three-dimensional neighborhood of multiple transform coefficients.
  • the processing module is configured to obtain at least two two-dimensional sub-band coefficient arrays from the at least two transform coefficient blocks, wherein one two-dimensional sub-band coefficient array includes the at least two transform coefficient blocks.
  • the transform coefficients corresponding to the same frequency points in the two transform coefficient blocks, and the transform coefficients included in the different two-dimensional subband coefficient arrays correspond to different frequency points; the at least two two-dimensional frequency points are determined according to a preset scanning order of frequency points.
  • At least two sub-band images corresponding to the sub-band coefficient array encoding the transform coefficients in the at least two sub-band images according to a preset coefficient scanning order.
  • the transceiver module is configured to obtain transform coefficient encoding information of at least two image blocks; and perform entropy decoding on the transform coefficient encoding information to obtain the at least two transform coefficient blocks.
  • the transceiver module is configured to divide the original image area into at least two image blocks, and the original image area includes: a pixel image area or a residual image area; The image block is transformed and quantized to obtain the at least two transform coefficient blocks.
  • the component modules of the image processing device can also execute the steps described in the foregoing first aspect and various possible implementation manners.
  • an embodiment of the present application further provides an image processing device, including: a transceiver module for acquiring image encoding information; a processing module for determining a frequency in the image encoding information according to a preset frequency point scanning order Point corresponding to the transform coefficient to be decoded; according to a preset coefficient scanning order, the transform coefficient to be decoded corresponding to the one frequency point is decoded to obtain at least two transform coefficient blocks, wherein the at least two transform coefficient blocks are Each transform coefficient block of includes: at least two transform coefficients, and each transform coefficient in a transform coefficient block corresponds to a different frequency point.
  • the processing module is configured to determine the first to-be-decoded two-dimensional subband coefficient array in the image encoding information according to a preset frequency point scanning order, and the first to-be-decoded two-dimensional subband coefficient array
  • the multi-dimensional subband coefficient array includes: a transform coefficient to be decoded corresponding to a frequency point in the image encoding information; the first transform coefficient to be decoded in the first two-dimensional subband coefficient array to be decoded is determined according to a preset coefficient scanning order Coefficients; perform entropy decoding on the first transform coefficient to be decoded according to the context region corresponding to the first transform coefficient to be decoded.
  • the processing module is configured to determine the three-dimensional subband coefficient array to be decoded in the image encoding information according to a preset frequency point scanning order, and the three-dimensional subband coefficient array to be decoded includes : The transform coefficients to be decoded corresponding to all frequency points in the image encoding information; the first two-dimensional subband coefficient array to be decoded in the three-dimensional subband coefficient array to be decoded is determined according to the preset scanning order of the three-dimensional subband coefficients
  • the first two-dimensional subband coefficient array to be decoded includes: a transform coefficient to be decoded corresponding to a frequency point in the image encoding information; and the first two-dimensional subband coefficient to be decoded is determined according to a preset coefficient scanning order
  • the first transform coefficient to be decoded in the array; the first transform coefficient to be decoded is entropy-decoded according to the context region corresponding to the first transform coefficient to be decoded.
  • the scanning order of the three-dimensional subband coefficients includes: a preset change order of the horizontal position index; or, a preset change order of the vertical position index; or, a preset frequency index The order of change.
  • the processing module is configured to determine the transform coefficient to be decoded included in a frequency subband in the image encoding information according to a preset frequency point scanning order, and the one frequency subband includes The transform coefficients to be decoded correspond to the same frequency point; the first transform coefficient to be decoded in the one frequency subband is determined according to a preset coefficient scanning order; the first transform coefficient to be decoded corresponds to the context region corresponding to the Entropy decoding is performed on the first transform coefficient to be decoded.
  • the processing module is configured to obtain context information of the first transform coefficient to be decoded from a context region corresponding to the first transform coefficient to be decoded, wherein the first transform coefficient to be decoded is
  • the context area corresponding to the decoded transform coefficient includes: the decoded transform coefficient in the context area corresponding to the first transform coefficient to be decoded; and the first transform coefficient to be decoded is performed according to the context information of the first transform coefficient to be decoded Estimate to obtain the probability distribution information of the first transform coefficient to be decoded; and perform arithmetic decoding on the first transform coefficient to be decoded according to the probability distribution information of the first transform coefficient to be decoded to obtain the decoded first Transform coefficient, the decoded first transform coefficient belongs to the at least two transform coefficient blocks.
  • the context area corresponding to the first transform coefficient to be decoded includes at least one of the following: a spatial neighborhood of the first transform coefficient to be decoded, and a sub-region of the first transform coefficient to be decoded Band neighborhood, the three-dimensional neighborhood of the first transform coefficient to be decoded.
  • the processing module is configured to, when the first to-be-decoded transform coefficient is a plurality of to-be-decoded transform coefficients, obtain all the data from the context regions corresponding to the plurality of to-be-decoded transform coefficients.
  • the context information of the plurality of transform coefficients to be decoded includes: decoded transform coefficients in the context regions corresponding to the plurality of transform coefficients to be decoded; according to the plurality of transform coefficients to be decoded
  • the context information of the transform coefficient estimates the plurality of transform coefficients to be decoded in parallel to obtain the probability distribution information of the first transform coefficient to be decoded;
  • a plurality of transform coefficients to be decoded are arithmetically decoded in parallel to obtain a decoded first transform coefficient, and the decoded first transform coefficient belongs to the at least two transform coefficient blocks.
  • the context regions corresponding to the plurality of transform coefficients to be decoded include at least one of the following: spatial neighborhoods of the plurality of transform coefficients to be decoded, and sub-regions of the plurality of transform coefficients to be decoded With neighborhood, the three-dimensional neighborhood of the plurality of transform coefficients to be decoded.
  • the processing module is configured to determine a first sub-band image in the image encoding information according to a preset frequency point scanning order, and the first sub-band image includes: the image The to-be-decoded transform coefficient corresponding to a frequency point in the encoded information is decoded according to a preset coefficient scanning order to decode the to-be-decoded coefficient of the first subband image to obtain the at least two transform coefficient blocks.
  • the processing module is further configured to perform entropy coding on the at least two transform coefficient blocks after obtaining at least two transform coefficient blocks to obtain transform coefficient coding information;
  • the coefficient encoding information generates at least two image blocks; the original image area is restored according to the at least two image blocks, and the original image area includes: a pixel image area or a residual image area.
  • the processing module is further configured to perform inverse quantization and inverse transformation on the at least two transform coefficient blocks after obtaining at least two transform coefficient blocks to obtain at least two image blocks;
  • the original image area is restored according to the at least two image blocks, and the original image area includes: a pixel image area or a residual image area.
  • the component modules of the image processing device can also execute the steps described in the foregoing second aspect and various possible implementations.
  • steps described in the foregoing second aspect and various possible implementations please refer to the foregoing description of the second aspect and various possible implementations. instruction of.
  • the embodiments of the present application provide a computer-readable storage medium that stores instructions in the computer-readable storage medium, which when run on a computer, causes the computer to execute the above-mentioned first or second aspect. The method described.
  • the embodiments of the present application provide a computer program product containing instructions, which when run on a computer, cause the computer to execute the method described in the first aspect or the second aspect.
  • an embodiment of the present application provides an image processing device.
  • the image processing device may include entities such as an image encoding device, an image decoding device, or a chip, and the image processing device includes a processor.
  • the image processing may further include a memory; the memory is used to store instructions; the processor is used to execute the instructions in the memory, so that the image processing apparatus executes any one of the foregoing first aspect or second aspect The method described in the item.
  • an embodiment of the present application provides a computer-readable storage medium that stores instructions in the computer-readable storage medium, which when run on a computer, causes the computer to execute the above-mentioned first or second aspect. The method described.
  • the embodiments of the present application provide a computer program product containing instructions, which when run on a computer, cause the computer to execute the method described in the first aspect or the second aspect.
  • this application provides a chip system that includes a processor for supporting the image encoding device or the image decoding device to implement the functions involved in the first or second aspect, for example, sending or processing The data and/or information involved in the above methods.
  • the chip system further includes a memory, and the memory is used to store program instructions and data necessary for the image encoding device or the image decoding device.
  • the chip system can be composed of chips, and can also include chips and other discrete devices.
  • FIG. 1 is a schematic diagram of the composition structure of an image processing system provided by an embodiment of the application
  • FIG. 2 is a schematic diagram of the image encoding device and the image decoding device provided by an embodiment of the application applied to terminal equipment;
  • FIG. 3 is a schematic diagram of an interaction flow between an image encoding device and an image decoding device provided by an embodiment of the application;
  • FIG. 4 is a schematic diagram of an image coding process provided by an embodiment of the application.
  • FIG. 5 is a schematic diagram of acquiring a transform coefficient block provided by an embodiment of the application.
  • FIG. 6 is a schematic diagram of obtaining a two-dimensional subband coefficient array provided by an embodiment of the application.
  • FIG. 7 is a schematic diagram of a transform coefficient block provided by an embodiment of the application.
  • FIG. 8 is a schematic diagram of an image decoding process provided by an embodiment of this application.
  • FIG. 9 is a schematic diagram of the correspondence between a two-dimensional subband coefficient array and a transform coefficient block provided by an embodiment of the application.
  • FIG. 10 is a schematic diagram of an image encoding process provided by an embodiment of this application.
  • Fig. 11a is a schematic diagram of a three-dimensional subband coefficient array provided by an embodiment of the application.
  • Fig. 11b is a schematic diagram of a three-dimensional neighborhood of transform coefficients provided by an embodiment of the application.
  • FIG. 12 is a schematic diagram of an image decoding process provided by an embodiment of this application.
  • FIG. 13 is a schematic diagram of an image coding process provided by an embodiment of this application.
  • FIG. 14 is a schematic diagram of an image coding process provided by an embodiment of the application.
  • FIG. 15 is a schematic diagram of an image coding process provided by an embodiment of this application.
  • FIG. 16 is a schematic diagram of an image decoding process provided by an embodiment of this application.
  • FIG. 17 is a schematic diagram of the composition structure of an image encoding device provided by an embodiment of the application.
  • FIG. 18 is a schematic diagram of the composition structure of an image decoding device provided by an embodiment of the application.
  • FIG. 19 is a schematic diagram of the composition structure of another image encoding device provided by an embodiment of this application.
  • FIG. 20 is a schematic diagram of the composition structure of another image decoding device provided by an embodiment of the application.
  • the embodiments of the present application provide an image processing method and an image processing device, which are used to improve image compression efficiency and reduce the storage cost of image coding information.
  • the technical solutions of the embodiments of the present application can be applied to various image processing systems.
  • FIG. 1 it is a schematic diagram of the composition structure of the image processing system provided by the embodiments of the present application.
  • the image processing system 100 may include: an image encoding device 101 and an image decoding device 102.
  • the image encoding device 101 may be configured to generate image encoding information according to at least two transform coefficient blocks.
  • the at least two transform coefficient blocks may be generated according to at least two image blocks corresponding to the original image area.
  • the original image area may be the entire image or a rectangular image area, or an image area of other shapes, which is not limited this time.
  • the original image area includes: a pixel image area or a residual image area.
  • the image encoding information can be an image encoding code stream (also called a compressed code stream), and then the image encoding information can be transmitted to the image decoding device 102 through the image transmission channel, and the image decoding device 102 can receive the image encoding information, and then The image decoding device 102 can decode the image encoding information to obtain at least two transform coefficient blocks. For example, at least two transform coefficient blocks can also be used to restore the original image.
  • the image encoding device can be applied to various terminal equipment that requires image communication, wireless equipment that needs image transcoding, and core network equipment.
  • the image encoding device may be the above-mentioned terminal device or wireless device. Or the image encoder of the core network equipment.
  • the image decoding device can be applied to various terminal devices that require image communication, wireless devices that require image transcoding, and core network devices.
  • the image decoding device can be the above-mentioned terminal device or wireless device or core network device. Image decoder.
  • the image encoding device is specifically an image encoder
  • the image decoding device is specifically an image decoder.
  • the terminal device may include an image encoder and a channel encoder, or the terminal device may include an image decoder and a channel decoder.
  • the channel encoder is used for channel encoding the image signal
  • the channel decoder is used for channel decoding the image signal
  • the image encoder is used for encoding the original image area.
  • the image encoder has a lossless encoding function.
  • the image decoder is used to decode image coding information, for example, the image decoder has a lossless decoding function.
  • lossless coding methods that can be used include entropy coding, Huffman coding, and the like.
  • lossless decoding methods that can be used include entropy decoding, Huffman decoding, and so on.
  • the first terminal device 20 may include: an image encoder 201 and a channel encoder 202.
  • the second terminal device 21 may include: an image decoder 211 and a channel decoder 212.
  • the first terminal device 20 is connected to a wireless or wired first network communication device 22, the first network communication device 22 is connected to a wireless or wired second network communication device 23 through a digital channel, and the second terminal device 21 is connected to wireless or wired The second network communication device 23.
  • the above-mentioned wireless or wired network communication equipment may generally refer to signal transmission equipment, such as communication base stations, data exchange equipment, and the like.
  • the terminal device as the transmitting end performs image coding on the collected image signal, and then performs channel coding, and then transmits it in a digital channel through a wireless network or a core network.
  • the terminal device as the receiving end performs channel decoding according to the received signal to obtain the image signal encoding code stream, and then the image signal is restored through image decoding, and the receiving end terminal device performs playback.
  • the embodiments of the present application provide encoding for transform coefficient blocks corresponding to the original image area, which improves image encoding performance, ultimately saves storage resources, and further improves image encoding efficiency. For example, current compressed files generated by entropy encoding consume a large amount of storage resources on the server side, and the embodiments of the present application can be applied to efficient transcoding of entropy encoded images. The embodiments of the present application can also be applied to the server to transcode compressed files generated by entropy coding, improve image coding performance, and ultimately save storage resources.
  • FIG. 3 it is a schematic diagram of an interaction flow between the image encoding device and the image decoding device in this embodiment of the application, in which the following steps 301 to 303 can be executed by the image encoding device (hereinafter referred to as the encoding end),
  • the following steps 311 to 313 may be executed by an image decoding device (hereinafter referred to as a decoder for short), and mainly include the following processes:
  • each transform coefficient block in the at least two transform coefficient blocks includes: at least two transform coefficients, wherein each transform coefficient in one transform coefficient block corresponds to a different frequency point.
  • each transform coefficient block in the at least two transform coefficient blocks includes: at least two transform coefficients, and each transform coefficient in one transform coefficient block corresponds to a different frequency point, that is, for each transform coefficient block, the transform coefficient All transform coefficients in a block have different frequency points, and each transform coefficient includes frequency point information.
  • each transform coefficient may also include position information.
  • the frequency point of a transformation coefficient can be expressed, fx and fy indicate the two-dimensional frequency coordinate value corresponding to the frequency point, the position information of a transformation coefficient can be expressed by (bx, by), bx and by indicate the two-dimensional transformation coefficient
  • the position coordinates are taken, and each transform coefficient in the transform coefficient block can be represented by C_(bx,by) ⁇ (fx,fy).
  • obtaining at least two transform coefficient blocks in step 301 includes:
  • the original image area includes: pixel image area or residual image area;
  • A2. Transform and quantize at least two image blocks to obtain at least two transform coefficient blocks.
  • the original image area (or simply the original image) may be the entire image or a rectangular image area, or an image area of other shapes.
  • the original image area is the image to be encoded (or called to be processed).
  • the encoding end divides the image or image area to be processed into at least two image blocks of the same preset size.
  • the size of the sub-image blocks of the preset size may be 4 ⁇ 4, 8 ⁇ 8, 16 ⁇ 16, 32 ⁇ 32 , 64 ⁇ 64, 128 ⁇ 128, 256 ⁇ 256, etc.
  • the image block is obtained by dividing the original image, or the image block is transmitted to the encoding end through an external device of the encoding end.
  • the image block may also be called a sub-image, or a sub-image block.
  • the image data included in the image block may include a luminance component, a chrominance component, an R component, a G component, a B component, and so on.
  • each image block may be transformed and quantized to obtain at least two transform coefficient blocks.
  • the encoding end obtains at least two image blocks
  • the encoding end obtains at least two transform coefficient blocks according to the at least two image blocks, where each image block can generate one transform coefficient block, and the image block and the transform coefficient block are the same.
  • the transform coefficient block is a data block composed of multiple transform coefficients.
  • the transform coefficient block can be obtained by performing discrete cosine transform (DCT) on the image block.
  • DCT discrete cosine transform
  • a transform coefficient block can be generated by transforming and quantizing an image block, so that the transform coefficient block can be coded, image coding information can be obtained, and image compression processing can be realized.
  • obtaining at least two transform coefficient blocks in step 301 includes:
  • the transform coefficient encoding information may be transform coefficient encoding information corresponding to at least two image blocks.
  • the at least two image blocks may be image blocks in a JPEG file
  • the image processing device may use a Huffman decoding method to decode the information in the JPEG file.
  • Entropy decoding is performed on the transform coefficient coding information of all the image blocks of, to obtain at least two transform coefficient blocks corresponding to the at least two image blocks.
  • the transform coefficient encoding information is entropy-decoded to obtain at least two transform coefficient blocks, and the transform coefficient blocks are encoded to obtain image encoding information and realize image compression processing.
  • the frequency point scanning order may be preset or pre-configured in the encoding end.
  • the frequency point scanning order refers to the sequence of scanning frequency points.
  • Each transform coefficient in at least two transform coefficient blocks The block includes: at least two transform coefficients.
  • Each transform coefficient in a transform coefficient block corresponds to different frequency points. Therefore, a frequency point scanning order can be set for all frequency points, and a frequency point is determined by the frequency point scanning order. In the same way, different frequency points can be determined in sequence through the frequency point scanning order.
  • the encoding process for one frequency point is taken as an example.
  • the encoding process for other frequency points can be based on step 302. And step 303 and the detailed procedures included in step 302 and step 303 are implemented.
  • the at least two transform coefficient blocks may be N transform coefficient blocks, and the transform coefficient in the at least two transform coefficient blocks corresponding to one frequency point may be one frequency point corresponding to each of the N transform coefficient blocks.
  • the transform coefficient in the at least two transform coefficient blocks corresponding to one frequency point may be one frequency point corresponding to each of the N transform coefficient blocks.
  • one frequency point corresponds to N transform coefficients.
  • the N transform coefficients are derived from each of the N transform coefficient blocks.
  • the value of N can be positive An integer, for example, N is an integer greater than or equal to 2.
  • the coefficient scanning order may be preset or pre-configured in the encoding end.
  • the coefficient scanning order refers to the scanning order of transform coefficients.
  • a transform corresponding to a frequency point is determined according to the preset coefficient scanning times.
  • the determined transform coefficient is then encoded.
  • other transform coefficients corresponding to a frequency point can be determined in sequence, and then the determined transform coefficients can be encoded in sequence, and finally the image can be encoded. .
  • multiple transform coefficients corresponding to the same frequency point in the at least two transform coefficient blocks may be encoded.
  • lossless coding methods that can be used include entropy coding, Huffman coding, and the like. It is not limited that the entropy coding in the embodiment of the present application uses a deep network, and the structure of the deep network is not limited.
  • the next frequency point is determined according to the preset frequency point scanning order, and then the multiple transform coefficients corresponding to the next frequency point are encoded, according to the frequency point scanning order All frequency points are acquired in turn, and after the lossless encoding of the transform coefficients corresponding to all frequency points is completed, the encoding end can generate image encoding information. After the encoding end generates the image encoding information, the encoding end can write the image encoding information into the code stream, and then the encoding end sends the code stream to the decoding end.
  • the frequency point scanning order is the sequence of selecting the frequency points during encoding, and the frequency point scanning order is referred to simply as "preset order" or “preset order one" in the subsequent embodiments.
  • the frequency point scanning order can be the processing order obtained by zig-zag scanning, or other scanning methods can be used to obtain the processing order, such as progressive scanning, column-by-column scanning, diagonal scanning, Horizontal reverse scanning and vertical reverse scanning, etc.
  • progressive scanning refers to continuous scanning from the first row of the image to the last row
  • column-wise scanning refers to continuous scanning from the first column of the image to the last column.
  • Diagonal scanning refers to the continuous scanning from the first line of the image to the last line in a diagonal manner.
  • Horizontal reverse scanning refers to continuous scanning from the last line of the image in the horizontal direction to the first line
  • vertical reverse scanning refers to continuous scanning from the last line of the image to the first line in the vertical direction.
  • the coefficient scanning order is the order in which transform coefficients are selected during encoding, and the coefficient scanning order is simply referred to as "preset order" or "preset order two" in the subsequent embodiments.
  • preset order or "preset order two" in the subsequent embodiments.
  • There are many ways to implement the coefficient scanning order for example, it can be the processing order obtained by zig-zag scanning, or other scanning methods can be used to obtain the processing order, such as progressive scanning, column-by-column scanning, diagonal scanning, and horizontal scanning. Reverse scanning and vertical reverse scanning, etc.
  • the frequency point scanning order and the coefficient scanning order are the same image scanning order; or, the frequency point scanning order and the coefficient scanning order are different image scanning orders, which are not limited here.
  • step 302 determines the transform coefficients in the at least two transform coefficient blocks corresponding to one frequency point according to a preset frequency point scanning order, including:
  • Step 303 encodes each transform coefficient corresponding to a frequency point according to the preset coefficient scanning order, including:
  • the encoding end may arrange at least two transform coefficient blocks according to different frequency points, so as to obtain at least two two-dimensional subband coefficient arrays.
  • the same two-dimensional subband coefficient array includes the transform coefficients in the transform coefficient block corresponding to the same frequency point.
  • Different two-dimensional subband coefficient arrays have different frequency points. That is, the encoding end can assign at least two corresponding frequency points to the same frequency point.
  • the transform coefficients in the transform coefficient block belong to a two-dimensional sub-band coefficient array. Therefore, as many frequency points exist, the two-dimensional sub-band coefficient array corresponding to the multiple frequency points can be obtained.
  • An example is as follows: Obtain the transform coefficient value of the designated frequency point in the 8x8 transform coefficient block in the image to form a two-dimensional subband coefficient array of the frequency point. After obtaining the two-dimensional sub-band coefficient array obtained by the transform coefficient values of the frequency points in all 8 ⁇ 8 transform coefficient blocks in the image, similarly, the two-dimensional sub-band coefficient array of the remaining frequency points is obtained.
  • An 8x8 transform coefficient block has a total of 64 frequency points, so 64 two-dimensional subband coefficient arrays of the current coded image can be obtained.
  • the encoding end combines at least two transform coefficient blocks according to different frequency points to combine the transform coefficients to obtain at least two two-dimensional subband coefficient arrays.
  • Obtain a first two-dimensional sub-band coefficient array in the array the first two-dimensional sub-band coefficient array is a two-dimensional sub-band coefficient array of at least two two-dimensional sub-band coefficient arrays, and the first two-dimensional sub-band coefficient array is A two-dimensional sub-band coefficient array obtained according to the frequency point scanning order, the first two-dimensional sub-band coefficient array includes a plurality of transform coefficients, and the encoding end may determine the first two-dimensional sub-band coefficient array according to a preset coefficient scanning order
  • the first transform coefficient in the first transform coefficient, the first transform coefficient can be any transform coefficient in the first two-dimensional sub-band coefficient array, and the encoding end can respectively perform lossless encoding on each transform coefficient in the first two-dimensional sub-band coefficient array
  • entropy coding may be performed on the first transform coefficient according to the context region corresponding to
  • the frequency point scanning order may be the order of scanning the two-dimensional subband coefficient array
  • the coefficient scanning order is the scanning order of the transform coefficients in the two-dimensional subband coefficient array.
  • the frequency point scanning order is simply referred to as "preset order one" in the subsequent embodiments
  • the coefficient scanning order is referred to as "preset order two" in the subsequent embodiments.
  • step 302 determines the transform coefficients in the at least two transform coefficient blocks corresponding to one frequency point according to a preset frequency point scanning order, including:
  • D1 obtain at least two two-dimensional subband coefficient arrays from at least two transform coefficient blocks, where one two-dimensional subband coefficient array includes transform coefficients corresponding to the same frequency points in at least two transform coefficient blocks, and two different two The transform coefficients included in the dimensional subband coefficient array correspond to different frequency points;
  • D3. Determine the first two-dimensional sub-band coefficient array in the three-dimensional sub-band coefficient array according to the preset scanning order of the three-dimensional sub-band coefficients.
  • Step 303 encodes each transform coefficient corresponding to a frequency point according to the preset coefficient scanning order, including:
  • D5. Perform entropy coding on the first transform coefficient according to the context region corresponding to the first transform coefficient.
  • the difference between the scenes in steps D1 to D5 and the scenes in steps C1 to C4 is that a three-dimensional subband coefficient array needs to be generated in the scenes from steps D1 to D5, and for the first two-dimensional field in the three-dimensional subband coefficient array The first transform coefficient in the coefficient array is coded, and the three-dimensional subband coefficient array is not generated in the scene of steps C1 to C4.
  • steps D1 to D5 will be described next.
  • the encoding end After generating at least two two-dimensional sub-band coefficient arrays, the encoding end combines the at least two two-dimensional sub-band coefficient arrays according to the preset frequency point scanning order to obtain a three-dimensional sub-band coefficient array, that is, one frequency point corresponds to one In the two-dimensional subband coefficient array, the two-dimensional subband coefficient arrays corresponding to all frequency points can be combined together according to the preset frequency point scanning order to obtain a three-dimensional subband coefficient array.
  • the three-dimensional sub-band coefficient array includes multiple two-dimensional sub-band coefficient arrays.
  • the encoding end first determines a two-dimensional sub-band coefficient array from the three-dimensional sub-band coefficient array according to the preset scanning order of the three-dimensional sub-band coefficients, for example, determines the first A two-dimensional sub-band coefficient array, and the first two-dimensional sub-band coefficient array is a two-dimensional sub-band coefficient array in the three-dimensional sub-band coefficient array.
  • the three-dimensional sub-band coefficient scanning order is the order in which the three-dimensional sub-band coefficient array is scanned.
  • the encoding end may respectively perform entropy encoding on each transform coefficient in the first two-dimensional subband coefficient array according to a preset coefficient scanning order.
  • image encoding information can be obtained, and image compression processing can be implemented.
  • step 302 determines the transform coefficients in the at least two transform coefficient blocks corresponding to one frequency point according to a preset frequency point scanning order, and corresponds to one frequency point according to the preset coefficient scanning order
  • Each transform coefficient of is coded, including:
  • E4. Determine the first two-dimensional sub-band coefficient array from the three-dimensional sub-band coefficient array according to the preset scanning order of the three-dimensional sub-band coefficients.
  • Step 303 encodes each transform coefficient corresponding to a frequency point according to the preset coefficient scanning order, including:
  • E5. Determine the first transform coefficient in the first two-dimensional subband coefficient array according to the preset coefficient scanning order
  • the difference between the scenes of steps E1 to E6 and the scenes of steps D1 to D5 is that the way of generating the three-dimensional subband coefficient array in the scenes of steps E1 to E6 is the same as that of generating the three-dimensional subband coefficient array in the steps D1 to D5.
  • the way is not the same. That is, in the scenes of steps E1 to E6, at least two transform coefficient blocks are sorted first, and then multiple two-dimensional subband coefficient arrays corresponding to different frequency points are determined according to different frequency points, and multiple two-dimensional subband coefficient arrays corresponding to different frequency points are determined.
  • the sub-band coefficient arrays are combined to obtain a three-dimensional sub-band coefficient array.
  • steps E1 to E6 will be described next.
  • the three-dimensional sub-band coefficient array includes multiple two-dimensional sub-band coefficient arrays.
  • the scanning order with coefficients determines a two-dimensional sub-band coefficient array from the three-dimensional sub-band coefficient array. For example, the first two-dimensional sub-band coefficient array is determined, and the first two-dimensional sub-band coefficient array is one of the three-dimensional sub-band coefficient arrays. Two-dimensional array of subband coefficients. Among them, the scanning order of the three-dimensional sub-band coefficients is the order in which the three-dimensional sub-band coefficient array is scanned.
  • the encoding end may respectively perform entropy encoding on each transform coefficient in the first two-dimensional subband coefficient array according to a preset coefficient scanning order.
  • image encoding information can be obtained, and image compression processing can be implemented.
  • the encoding end sorts each transform coefficient in the transform coefficient block (referred to as the coefficient block in the following) according to the frequency point scanning order, and then obtains the coefficients of the same frequency point and combines them to obtain a three-dimensional subband coefficient array.
  • the transform coefficient block may be an 8 ⁇ 8 transform coefficient block.
  • Obtain the transform coefficient value of the specified frequency point in the 8x8 transform coefficient block in the image form a two-dimensional subband coefficient array of the frequency point, and combine the two-dimensional subband coefficient array corresponding to multiple frequency points to obtain a three-dimensional subband coefficient array .
  • the encoding end may also perform encoding processing on each transform coefficient in the three-dimensional sub-band coefficient array. For details, refer to the example of the encoding process in the subsequent embodiments.
  • the scanning order of the three-dimensional subband coefficients includes:
  • each transform coefficient in the three-dimensional subband coefficient array has a horizontal position index, a vertical position index, and a frequency index.
  • each transform coefficient can be expressed as C(bx, by, fs), where bx is The horizontal position index, by is the vertical position index, and fs is the frequency index corresponding to the frequency point.
  • the three-dimensional sub-band coefficient scanning order is the order in which the three-dimensional sub-band coefficient array is scanned.
  • the scanning order of the three-dimensional sub-band coefficients determines the multiple two-dimensional sub-band coefficient arrays in the three-dimensional sub-band coefficient array according to the change order of the horizontal position index from large to small (or from small to large).
  • a transform coefficient in the two-dimensional sub-band coefficient array is determined according to the aforementioned coefficient scanning order, that is, the three-dimensional sub-band coefficient scanning order can be used to indicate the corresponding index to different horizontal positions in the three-dimensional sub-band coefficient array.
  • the two-dimensional subband coefficient array is encoded.
  • the scanning order of the three-dimensional sub-band coefficients is determined from large to small (or from small to large) in the order of vertical position index change, and a two-dimensional sub-band coefficient array is determined every time.
  • a transform coefficient in the two-dimensional sub-band coefficient array is determined according to the aforementioned coefficient scanning order, that is, the three-dimensional sub-band coefficient scanning order can be used to indicate the sequence of different vertical sub-band coefficients in the three-dimensional sub-band coefficient array.
  • the two-dimensional subband coefficient array corresponding to the position index is coded.
  • the scanning order of the three-dimensional sub-band coefficients determines the multiple two-dimensional sub-band coefficient arrays in the three-dimensional sub-band coefficient array according to the change order of the frequency index from large to small (or from small to large).
  • a transform coefficient in the two-dimensional sub-band coefficient array is determined according to the aforementioned coefficient scanning order, that is, the three-dimensional sub-band coefficient scanning order can be used to indicate the sequence corresponding to different frequency indexes in the three-dimensional sub-band coefficient array.
  • a two-dimensional array of subband coefficients is coded.
  • the specific implementation manner of the scanning order of the three-dimensional subband coefficients can be determined according to the application scenario, which is not limited this time.
  • M is the maximum number of image blocks
  • W represents the width value of the current image
  • H represents the height value of the current image
  • subW represents the width value of the image block in the current image
  • subH represents the height value of the image block in the current image.
  • (fx, fy) represents the frequency point of each transform coefficient in the transform coefficient block
  • the preset coefficient scanning order may be zig-zag scanning order, line-by-line scanning order, column-by-column scanning order, and so on.
  • step 302 determines the transform coefficients in the at least two transform coefficient blocks corresponding to one frequency point according to a preset frequency point scanning order, including:
  • F1 Determine a first two-dimensional subband coefficient array formed by transform coefficients corresponding to the same frequency points in at least two transform coefficient blocks according to a preset frequency point scanning order.
  • Step 303 encodes each transform coefficient corresponding to a frequency point according to the preset coefficient scanning order, including:
  • F3. Perform entropy coding on the first transform coefficient according to the context region corresponding to the first transform coefficient.
  • the encoding end may also obtain the first two-dimensional structure composed of transform coefficients corresponding to the same frequency point from the at least two transform coefficient blocks according to a preset frequency point scanning order. Subband coefficient array. The encoding end does not need to combine a three-dimensional subband coefficient array. After the encoding end determines the first two-dimensional sub-band coefficient array, the encoding end may respectively encode each transform coefficient in the first two-dimensional sub-band coefficient array according to a preset coefficient scanning order.
  • the encoding end determines the first transform coefficient in the first two-dimensional subband coefficient array according to a preset coefficient scanning order, and performs entropy encoding on the first transform coefficient according to the context region corresponding to the first transform coefficient.
  • image encoding information can be obtained, and image compression processing can be implemented.
  • the encoding end first scans each transform coefficient in at least two transform coefficient blocks according to the frequency point scanning order, for example, determines a frequency point according to the frequency point scanning order (the frequency index of the frequency point is fs), Obtain one transform coefficient in one transform coefficient block respectively corresponding to the designated frequency index fs in at least two transform coefficient blocks to form a two-dimensional subband coefficient array of one frequency point. After a two-dimensional sub-band coefficient array is determined, each transform coefficient in the two-dimensional sub-band coefficient array can be processed according to a preset coefficient scanning order. A transform coefficient is coded to obtain image coding information.
  • step 302 determines the transform coefficients in the at least two transform coefficient blocks corresponding to one frequency point according to a preset frequency point scanning order, including:
  • G1 Determine the transform coefficients in at least two transform coefficient blocks included in one frequency subband according to a preset frequency point scanning order, and the transform coefficients in at least two transform coefficient blocks included in one frequency subband correspond to the same frequency point .
  • Step 303 encodes each transform coefficient corresponding to a frequency point according to the preset coefficient scanning order, including:
  • G2. Determine the first transform coefficient included in a frequency subband according to a preset coefficient scanning order
  • the encoding end needs to generate a two-dimensional sub-band coefficient array or a three-dimensional sub-band coefficient array.
  • the encoding end may also use not to generate a two-dimensional sub-band coefficient array or a three-dimensional sub-band coefficient.
  • the encoder determines the transform coefficients in at least two transform coefficient blocks included in one frequency subband according to a preset frequency point scanning order, and the transform coefficients in at least two transform coefficient blocks included in one frequency subband correspond to For the same frequency point, the encoding end respectively encodes the transform coefficients in at least two transform coefficient blocks corresponding to one frequency subband according to the preset coefficient scanning order, for example, according to the preset coefficient scanning order, it is determined that a frequency subband includes The first transform coefficient, the first transform coefficient may be any transform coefficient of a plurality of transform coefficients included in a frequency subband, and the first transform coefficient is entropy-encoded according to the context region corresponding to the first transform coefficient.
  • image encoding information can be obtained, and image compression processing can be implemented.
  • the specific process of performing entropy coding on the aforementioned first transform coefficient is explained by taking the encoding end to describe the encoding process of other transform coefficients in the first two-dimensional subband coefficient array or a frequency subband.
  • the encoding process of other included transform coefficients can all be implemented according to the entropy encoding process of the first transform coefficient.
  • performing entropy coding on the first transform coefficient according to the context region corresponding to the first transform coefficient in the foregoing steps C4, D5, E6, F3, and G3 includes:
  • H1 Acquire the context information of the first transform coefficient from the context area corresponding to the first transform coefficient, where the context information of the first transform coefficient includes: coded transform coefficients in the context area corresponding to the first transform coefficient.
  • the context area corresponding to the first transform coefficient may be preset.
  • the context area corresponding to the first transform coefficient is determined according to the current horizontal position index, vertical position index, and frequency index of the first transform coefficient.
  • the encoded transform coefficient is a transform coefficient that has been entropy-encoded before the first transform coefficient is entropy-encoded. Therefore, the first transform coefficient is obtained from the context area corresponding to the first transform coefficient. Context information of a transform coefficient.
  • the context area corresponding to the first transform coefficient includes at least one of the following: a spatial neighborhood of the first transform coefficient, a subband neighborhood of the first transform coefficient, and a three-dimensional neighborhood of the first transform coefficient .
  • the context area corresponding to the first transform coefficient has multiple implementation manners, for example, it may include the spatial neighborhood of the first transform coefficient, or the subband neighborhood of the first transform coefficient, or the three-dimensional neighborhood of the first transform coefficient.
  • the spatial neighborhood of the first transform coefficient refers to an adjacent area in the same horizontal space or vertical space as the first transform coefficient
  • the subband neighborhood of the first transform coefficient refers to the same sub-band as the first transform coefficient.
  • the three-dimensional neighborhood of the adjacent area of the belt refers to an adjacent area that is in the same three-dimensional space as the first transform coefficient.
  • Encoded transform coefficients can be included in the spatial neighborhood of the first transform coefficient, or the subband neighborhood of the first transform coefficient, or the three-dimensional neighborhood of the first transform coefficient, so that the encoding end can obtain the first transform coefficient Contextual information.
  • H2 Estimate the first transform coefficient according to the context information of the first transform coefficient to obtain probability distribution information of the first transform coefficient.
  • the encoding end after the encoding end obtains the context information of the first transform coefficient, the encoding end performs entropy encoding on the first transform coefficient according to the context information of the first transform coefficient to obtain the image encoding corresponding to the first transform coefficient information.
  • the entropy encoding process performed by the encoding end may include probability estimation and arithmetic encoding. Specifically, the encoding end performs probability estimation on the first transform coefficient according to the context information of the first transform coefficient to obtain the probability distribution of the first transform coefficient, and then the encoding end performs arithmetic on the first transform coefficient according to the probability distribution of the first transform coefficient Encoding to obtain image encoding information corresponding to the first transform coefficient.
  • each transform coefficient in each two-dimensional sub-band coefficient array is processed according to the coefficient scanning order.
  • the preset context area may include the spatial neighborhood and/or subband neighborhood of the coefficient.
  • the method for estimating the probability distribution of the current coefficient includes: the probability estimation network directly estimates the probability of the value within the value range according to the dynamic value range of the coefficient to obtain a probability vector, which includes the value range The probability value of each value within, where the sum of all probability values is 1.
  • the method of estimating the probability distribution of the current coefficient includes: using a probability distribution model to obtain a probability vector.
  • a probability distribution model For example, a Gaussian single model (GSM) or a Gaussian mixture model (GMM) is used for modeling, a probability estimation network is used to estimate the model parameters, and the model parameters are substituted into the probability distribution model to obtain a probability vector.
  • GSM Gaussian single model
  • GMM Gaussian mixture model
  • the context information is input to the probability estimation network, and the probability estimation of the first transform coefficient is performed to obtain the probability distribution of the first transform coefficient, using The probability distribution performs arithmetic coding on the current first transform coefficient to obtain coding information of the first transform coefficient.
  • the probability distribution estimation methods used in the probability estimation network include probability distribution estimation methods based on deep learning networks, such as recurrent neural network (RNN) and pixel-by-pixel convolutional neural network (PixelCNN).
  • the specific process of performing entropy coding on the aforementioned first transform coefficient is explained by taking the encoding end to describe the encoding process of other transform coefficients in the first two-dimensional subband coefficient array or a frequency subband.
  • the encoding process of other included transform coefficients can all be implemented according to the entropy encoding process of the first transform coefficient.
  • performing entropy coding on the first transform coefficient according to the context region corresponding to the first transform coefficient in the foregoing steps C4, D5, E6, F3, and G3 includes:
  • the first transform coefficient is multiple transform coefficients
  • step I1 to step I3 are for the probability distribution that the first transform coefficient is a plurality of transform coefficients. At this time, it can be performed in parallel. Estimation and parallel arithmetic coding.
  • the encoding end can obtain multiple transform coefficients, and the encoding end performs entropy encoding on the multiple transform coefficients in parallel according to the context information of the multiple transform coefficients, and the image encoding efficiency can be improved through parallel entropy encoding.
  • the encoder when obtaining the probability distribution of coefficients, can also perform probability estimation on the probability distributions of multiple transform coefficients in parallel, that is, obtain the probability distributions of multiple transform coefficients at a time, and then perform parallel analysis on multiple transform coefficients. Perform arithmetic coding.
  • the context regions corresponding to the multiple transform coefficients include at least one of the following: spatial neighborhoods of multiple transform coefficients, subband neighborhoods of multiple transform coefficients, and three-dimensional neighborhoods of multiple transform coefficients .
  • the context regions corresponding to multiple transform coefficients have multiple implementation manners, for example, it may include a spatial neighborhood of multiple transform coefficients, or a subband neighborhood of multiple transform coefficients, or a three-dimensional neighborhood of multiple transform coefficients.
  • the spatial neighborhood of multiple transform coefficients refers to the adjacent area that is in the same horizontal space or vertical space with multiple transform coefficients
  • the sub-band neighborhood of multiple transform coefficients refers to the adjacent regions that are in the same sub-band with multiple transform coefficients.
  • the three-dimensional neighborhood of the adjacent area of the belt refers to an adjacent area that is in the same three-dimensional space as the first transform coefficient.
  • Encoded transform coefficients can be included in the spatial neighborhood of multiple transform coefficients, or the subband neighborhood of multiple transform coefficients, or the three-dimensional neighborhood of multiple transform coefficients, so that the encoder can obtain multiple transform coefficients Contextual information.
  • step 302 determines the transform coefficients in the at least two transform coefficient blocks corresponding to one frequency point according to a preset frequency point scanning order, including:
  • Step 303 encodes each transform coefficient corresponding to a frequency point according to the preset coefficient scanning order, including:
  • J3. Encode the transform coefficients in the at least two sub-band images according to the preset coefficient scanning order.
  • the encoding end of the encoding end may arrange at least two transform coefficient blocks according to different frequency points, so as to obtain at least two two-dimensional subband coefficient arrays.
  • the same two-dimensional subband coefficient array includes the transform coefficients in the transform coefficient block corresponding to the same frequency point.
  • Different two-dimensional subband coefficient arrays have different frequency points. That is, the encoding end can assign at least two corresponding frequency points to the same frequency point.
  • the transform coefficients in the transform coefficient block belong to a two-dimensional sub-band coefficient array. Therefore, as many frequency points exist, the two-dimensional sub-band coefficient array corresponding to the multiple frequency points can be obtained.
  • the encoding end After the encoding end obtains at least two two-dimensional sub-band coefficient arrays, the encoding end adopts the method of lossless encoding of the sub-band images, and uses at least two two-dimensional sub-band coefficient arrays as the at least two sub-band images according to the preset
  • the frequency point scanning sequence performs image lossless encoding on the transform coefficients in at least two sub-band images.
  • the encoding end can encode the two-dimensional sub-band coefficient array as a sub-band image during T-shirt lossless encoding, which simplifies the implementation process of lossless encoding. .
  • At least two transform coefficient blocks can be obtained in the embodiment of the present application, so that the transform coefficients corresponding to the same frequency points in the at least two transform coefficient blocks need only be encoded according to the frequency point scanning order.
  • the image encoding information can be obtained, so multiple transform coefficients corresponding to the same frequency point can be encoded at the same time, which improves image compression efficiency, reduces the storage cost of image encoding information, and ultimately saves storage resources.
  • the encoding end can send it to the decoding end through the image transmission channel, for example, the encoding end sends a compressed code stream, and the compressed code stream includes image encoding information.
  • the image encoding information is obtained by encoding at least two transform coefficient blocks by the encoding end, and the decoding end needs to use an image processing method corresponding to the encoding end for decoding. Specifically, the decoding end executes subsequent steps 311 to 313.
  • the decoding end first obtains image coding information from the encoding end. For example, the decoding end receives the compressed code stream from the encoding end, and then parses the compressed code stream to obtain image coding information.
  • each transform coefficient block in the at least two transform coefficient blocks includes: at least two transform coefficients, and each transform coefficient in one transform coefficient block corresponds to a different frequency point.
  • the transform coefficients to be decoded corresponding to the same frequency point in the image encoding information are decoded, where the transform coefficients to be decoded may also be referred to as transform coefficients to be decoded.
  • the available decoding methods include entropy decoding, Huffman decoding, and the like.
  • the decoder can generate at least two transform coefficient blocks, and each transform coefficient block of the at least two transform coefficient blocks includes: at least two transform coefficients , Where each transform coefficient in a transform coefficient block corresponds to a different frequency point.
  • transform coefficient block frequency point scanning order and coefficient scanning order, please refer to the example in the aforementioned encoding terminal for details.
  • step 312 determines the image encoding information according to the preset frequency point scanning order
  • the transform coefficient to be decoded corresponding to a frequency point in includes:
  • the first to-be-decoded two-dimensional sub-band coefficient array includes: the to-be-decoded corresponding to a frequency point in the image encoding information Transformation coefficient.
  • Step 313 decodes the transform coefficient to be decoded corresponding to a frequency point according to the preset coefficient scanning order, including:
  • K2. Determine the first to-be-decoded transform coefficient in the first to-be-decoded two-dimensional subband coefficient array according to the preset coefficient scanning order
  • the decoding end adopts the image decoding method corresponding to the encoding end. If the encoding end executes the encoding methods shown in steps C1 to C4, the image encoding information received by the decoding end may include at least two two-dimensional images to be decoded. Subband coefficient array.
  • the frequency point scanning order is the scanning order of the two-dimensional subband coefficient array to be decoded
  • the first two-dimensional subband coefficient array to be decoded is obtained from at least two two-dimensional subband coefficient arrays to be decoded according to the preset frequency point scanning order
  • the first two-dimensional subband coefficient array to be decoded includes a plurality of transform coefficients to be decoded.
  • step 312 determines the image encoding information according to the preset frequency point scanning order
  • the transform coefficient to be decoded corresponding to a frequency point in includes:
  • the three-dimensional subband coefficient array to be decoded includes: the transform coefficients to be decoded corresponding to all the frequency points in the image encoding information;
  • Step 313 decodes the transform coefficient to be decoded corresponding to a frequency point according to the preset coefficient scanning order, including:
  • the decoding end adopts an image decoding method corresponding to the encoding end, and the image encoding information received by the decoding end may include a three-dimensional subband coefficient array to be decoded.
  • the decoding end can obtain at least two two-dimensional sub-band coefficient arrays from the three-dimensional sub-band coefficient array to be decoded according to the preset scanning order of the three-dimensional sub-band coefficients.
  • the same two-dimensional subband coefficient array includes transform coefficients in at least two transform coefficient blocks corresponding to the same frequency point, and different two-dimensional subband coefficient arrays have different frequency points.
  • the three-dimensional sub-band coefficient array includes multiple two-dimensional sub-band coefficient arrays
  • the decoding end first determines a two-dimensional sub-band coefficient array from the three-dimensional sub-band coefficient array according to the preset scanning order of the three-dimensional sub-band coefficients, for example, A first two-dimensional sub-band coefficient array is obtained, and the first two-dimensional sub-band coefficient array is a two-dimensional sub-band coefficient array in the three-dimensional sub-band coefficient array.
  • the three-dimensional sub-band coefficient scanning order is the order in which the three-dimensional sub-band coefficient array is scanned.
  • the decoding end adopts the image decoding method corresponding to the encoding end.
  • the coefficient scanning order preset by the decoding end performs lossless decoding on the second two-dimensional sub-band coefficient array to obtain The at least two transform coefficient blocks.
  • lossless decoding may include entropy decoding.
  • the scanning order of the three-dimensional subband coefficients includes:
  • step 312 determines the transform coefficient to be decoded corresponding to a frequency point in the image coding information according to the preset frequency point scanning order, including:
  • M1 Determine the to-be-decoded transform coefficients included in one frequency sub-band in the image encoding information according to a preset frequency point scanning order, and the to-be-decoded transform coefficients included in one frequency sub-band correspond to the same frequency point.
  • Step 313 decodes the transform coefficient to be decoded corresponding to a frequency point according to the preset coefficient scanning order, including:
  • the decoding end adopts the image decoding method corresponding to the encoding end, and the encoding end can encode the two-dimensional subband coefficient array as a frequency subband during encoding, and the image encoding information received by the decoding end may include multiple The transform coefficients to be decoded corresponding to each frequency subband.
  • the decoding end obtains a transform coefficient to be decoded corresponding to a frequency subband from the image coding information according to the frequency point scanning order.
  • the decoding end may adopt a method of not generating the sub-band coefficient array, for example, neither generating the two-dimensional sub-band coefficient array nor generating the three-dimensional sub-band coefficient array.
  • the decoding end first determines the first frequency subband, and the transform coefficients in the at least two transform coefficient blocks corresponding to the first frequency subband correspond to the same frequency point.
  • the decoding end performs lossless decoding on the image coding information corresponding to the first frequency subband according to the preset coefficient scanning order, and performs lossless decoding on the image coding information corresponding to the multiple frequency subbands in this way to obtain multiple transformations. coefficient.
  • the decoding end obtains image coding information corresponding to multiple frequency subbands, and the decoding end adopts a method of lossless decoding on the frequency subbands, and uses at least two two-dimensional subband coefficient arrays as at least two frequency subbands, according to the prediction
  • the set coefficient scanning order performs lossless decoding on the image encoding information corresponding to at least two frequency subbands.
  • the decoding end can decode the two-dimensional subband coefficient array as a frequency subband during lossless decoding, which simplifies the implementation process of lossless decoding. .
  • performing entropy decoding on the first transform coefficient to be decoded according to the context region corresponding to the first transform coefficient to be decoded in the foregoing steps K3, L4, and M4 includes:
  • the lossless decoding based on the decoding end may be entropy decoding.
  • the entropy decoding process of the first transform coefficient will be described.
  • the decoding process may follow the first The entropy decoding process of transform coefficients is realized.
  • the decoder after the decoder obtains the context information of the first transform coefficient, the decoder performs entropy decoding on the first transform coefficient according to the context information of the first transform coefficient to obtain the decoded first transform coefficient.
  • the entropy decoding performed by the decoding end may include probability estimation and arithmetic decoding. Specifically, the decoder performs a probability estimation on the first transform coefficient according to the context information of the first transform coefficient to obtain the probability distribution of the first transform coefficient, and then the decoder performs arithmetic on the first transform coefficient according to the probability distribution of the first transform coefficient Decoding to obtain image decoding information corresponding to the first transform coefficient.
  • each symbol is decoded using the same processing order as the encoding end.
  • the decoded symbol in the preset context area of the symbol to be decoded is input into the probability estimation network as the context, and the probability distribution of the current symbol is estimated, and the probability distribution is used for the current symbol.
  • the lossless decoding based on the decoding end can be entropy decoding.
  • the entropy decoding process of the first transform coefficient is described.
  • the decoding process can be based on the first transform coefficient.
  • the entropy decoding process is implemented.
  • the decoder after the decoder obtains the context information of the first transform coefficient, the decoder performs entropy decoding on the first transform coefficient according to the context information of the first transform coefficient to obtain the decoded first transform coefficient.
  • the entropy decoding performed by the decoding end may include probability estimation and arithmetic decoding. Specifically, the decoder performs a probability estimation on the first transform coefficient according to the context information of the first transform coefficient to obtain the probability distribution of the first transform coefficient, and then the decoder performs arithmetic on the first transform coefficient according to the probability distribution of the first transform coefficient Decoding to obtain image decoding information corresponding to the first transform coefficient.
  • each symbol is decoded using the same processing order as the encoding end.
  • the decoded symbol in the preset context area of the symbol to be decoded is input into the probability estimation network as the context, and the probability distribution of the current symbol is estimated, and the probability distribution is used for the current symbol.
  • the context area corresponding to the first transform coefficient to be decoded includes at least one of the following: the spatial neighborhood of the first transform coefficient to be decoded, the subband neighborhood of the first transform coefficient to be decoded, and the first The three-dimensional neighborhood of the transform coefficient to be decoded.
  • performing entropy decoding on the first transform coefficient to be decoded according to the context region corresponding to the first transform coefficient to be decoded in the foregoing steps K3, L4, and M4 includes:
  • the first transform coefficient to be decoded is a plurality of transform coefficients to be decoded
  • the decoding end adopts an image decoding method corresponding to the encoding end, and the encoding end can perform encoding in parallel during encoding, and then through parallel decoding at the decoding end, the image decoding efficiency can be improved.
  • the context regions corresponding to the multiple transform coefficients to be decoded include at least one of the following: multiple spatial neighborhoods of the transform coefficients to be decoded, multiple subband neighborhoods of the transform coefficients to be decoded, and multiple The three-dimensional neighborhood of the transform coefficient to be decoded.
  • step 312 determines the image encoding information according to the preset frequency point scanning order
  • the transform coefficient to be decoded corresponding to a frequency point in includes:
  • the first subband image includes: a transform coefficient to be decoded corresponding to a frequency point in the image encoding information.
  • Step 313 decodes the transform coefficient to be decoded corresponding to a frequency point according to the preset coefficient scanning order, including:
  • the decoding end adopts the image decoding method corresponding to the encoding end, and the encoding end can encode the two-dimensional subband coefficient array as a subband image during encoding, and the image encoding information received by the decoding end may include The transform coefficients to be decoded corresponding to multiple sub-band images.
  • the decoding end obtains the to-be-decoded transform coefficient corresponding to the first sub-band image from the image encoding information according to the frequency point scanning order.
  • the decoding end adopts lossless decoding for the sub-band image, and after obtaining multiple transform coefficients, the decoding end adopts the method of lossless decoding of the sub-band image, using at least two two-dimensional sub-band coefficient arrays as at least two sub-band images, according to the preset
  • the coefficient scanning order of performs lossless decoding on the image encoding information corresponding to at least two subband images, and the decoding end can decode the two-dimensional subband coefficient array as a subband image during lossless decoding, which simplifies the implementation process of lossless decoding.
  • the method provided in the embodiments of the present application further includes:
  • R1 perform entropy coding on at least two transform coefficient blocks to obtain transform coefficient coding information
  • R2 generate at least two image blocks according to the transform coefficient coding information
  • the original image area is restored according to at least two image blocks, the original image area includes: an original pixel image area or a residual image area.
  • the decoding end adopts an image decoding method corresponding to the encoding end. If the encoding end obtains a transform coefficient block by entropy decoding the transform coefficient encoding information, and encodes the transform coefficient block, the decoding end can perform at least two transform coefficients. Entropy coding is performed on the block to obtain transform coefficient coding information. In the embodiment of the present application, at least two image blocks are generated according to the transform coefficient encoding information, and then the original image area is restored according to the at least two image blocks, so that the image decompression process can be completed.
  • the method provided in the embodiments of the present application further includes:
  • the original image area is restored according to at least two image blocks, where the original image area includes: an original pixel image area or a residual image area.
  • the decoding end after the decoding end generates at least two transform coefficient blocks, the decoding end generates at least two image blocks according to the at least two transform coefficient blocks.
  • the decoding end can process at least two transform coefficient blocks by means of inverse transformation and inverse quantization, so as to generate at least two image blocks.
  • the decoding end restores the original image according to the at least two image blocks, and multiple image blocks may be spliced to obtain the original image.
  • the transform coefficients to be decoded corresponding to the same frequency points in the image encoding information are decoded, and at least two are obtained. Since each transform coefficient in a transform coefficient block corresponds to a different frequency point, for the decoding of image coding information, different transform coefficients corresponding to the same frequency point can be obtained, which improves the efficiency of image decompression and reduces the image
  • the storage cost of encoded information ultimately saves storage resources.
  • the embodiment of the present application proposes a block transform and coefficient coding scheme in video image compression.
  • the embodiment of the present application improves image coding performance, ultimately saves storage resources, and improves the coding efficiency of video images.
  • compressed files generated by entropy encoding consume a large amount of storage resources on the server side.
  • the embodiments of this application can be applied to the process of high-efficiency and lossless transcoding of entropy-encoded images. Lossless transcoding of compressed files improves image encoding performance and ultimately saves storage resources.
  • the image processing method provided by the embodiments of this application will be described in detail through five different embodiments.
  • the image encoding process is introduced from the encoding end. After the image encoding code stream is generated at the encoding end, the encoding end Send to the decoding end. After the decoding end receives the image encoding bitstream, the decoding end decodes the image encoding bitstream to restore the original image.
  • This embodiment relates to an image coding scheme.
  • a transform coefficient block (referred to as a coefficient block) is obtained, and then the coefficient block is used for lossless coding to obtain coded image data.
  • This embodiment provides the entire encoding process, which may include, for example, transforming and quantizing the image area, and then performing lossless encoding on the quantized coefficients to obtain encoded image data.
  • the image coding process mainly includes:
  • Step 401 Obtain a transform coefficient block.
  • Method 1 Divide the image area to be processed into image blocks of preset size, and transform and quantize each image block to obtain a transform coefficient block.
  • the transform coefficient block may specifically be a transform coefficient array C(bx, by, fx ,fy).
  • the size of the sub-image blocks of the preset size can be 4x4, 8x8, 16x16, 32x32, 64x64, 128x128, 256x256, and so on.
  • FIG. 5 it is a schematic diagram of obtaining a transform coefficient block provided by an embodiment of this application.
  • DCT transformation is performed on the sub-image block of the preset size to obtain the transform coefficient block.
  • quantize the transform coefficient block to obtain a quantized coefficient array C (bx, by, fx, fy), and each transform coefficient in the coefficient array can be C_(bx, by) ⁇ (fx, fy) Express.
  • (bx, by) represents the horizontal and vertical position index of the coefficient block
  • M W/subW
  • N H/subH
  • M and N are the width and height of the image in the unit of sub-image block
  • W and H represent the width and height of the current image
  • subW and subH represent the width and height of the sub-image block in the current image.
  • the size of the sub-image block of the preset size is taken as an example of 8x8, that is, the width and height of the sub-image block are both 8, that is, the values of subW and subH are both 8, and will not be described in detail later.
  • Method 2 Entropy decoding the entropy-encoded image data to obtain transform coefficients (referred to as coefficients for short) of 8x8 blocks in the image.
  • Each transform coefficient of the coefficient block is recorded as C_(bx,by ) ⁇ (fx,fy).
  • (bx, by) represents the horizontal and vertical position index of the coefficient block
  • M W/subW
  • N H/subH
  • M and N are the image width and height in units of sub-image blocks
  • W and H respectively represent the width and height of the current image
  • subW and subH represent the width and height of the sub-image blocks in the current image, respectively.
  • the width and height of all sub-image blocks in the JPEG file are both 8, that is, the values of subW and subH are both 8, which will not be described in detail later.
  • the technical solutions described in the embodiments of the present application can also be applied to the case where the values of subW and subH are not 8, for example, subW is not equal to subH.
  • Step 402 Obtain the transform coefficient value of the designated frequency point in the 8x8 transform coefficient block (8x8 coefficient block for short) in the image to form a two-dimensional subband coefficient array of the frequency point.
  • FIG. 6 it is a schematic diagram of obtaining a two-dimensional subband coefficient array provided by an embodiment of this application. Obtain the transform coefficient value of the designated frequency point (fx, fy) in the 8x8 transform coefficient block in the image, and obtain the two-dimensional subband coefficient array C ⁇ (fx, fy)(bx, by) of the frequency point.
  • An 8x8 transform coefficient block has a total of 64 frequency points, so 64 two-dimensional subband coefficient arrays of the current coded image can be obtained.
  • Step 403 Process each two-dimensional sub-band coefficient array according to the preset order 1.
  • process each transform coefficient according to the preset order two When transform coefficients are selected, the processed coefficients in the preset spatial neighborhood and subband neighborhood are selected as the context estimation to obtain the probability distribution of the current coefficient, and the probability distribution is used to arithmetic coding the current coefficient to obtain the coding information of the current coefficient.
  • processing each two-dimensional subband coefficient array according to the preset order one refers to determining each two-dimensional subband coefficient array according to the preset order one
  • processing each transform coefficient according to the preset order two refers to presetting.
  • Step 403.1 Process each two-dimensional subband coefficient array in a preset order.
  • FIG. 7 it is a schematic diagram of a transform coefficient block provided by an embodiment of this application.
  • any 8x8 transform coefficient block in the image scan the transform coefficients of the specified frequency point (fx, fy) in the coefficient block according to the preset order, and then specify the two-dimensional subdivision of the frequency point (fx, fy)
  • the index fs of the coefficient array is the same as the scanning order of the frequency point in the coefficient block, and the two-dimensional subband coefficient array C ⁇ (fx, fy)(bx, by) corresponding to fs is processed in the order of the magnitude of fs.
  • fs is an integer, and the value range is 0,...,subW ⁇ subH-1.
  • Step 403.2 Process each transform coefficient in each two-dimensional subband coefficient array.
  • each transform coefficient in each two-dimensional sub-band coefficient array is processed according to the preset order two.
  • Select the processed coefficient in the preset context area as the context and input the context information into the probability estimation network to estimate the probability distribution of the current coefficient to obtain the probability distribution of the current coefficient, and use the probability distribution to perform arithmetic coding on the current coefficient Get the coding information of the current coefficient.
  • the preset context area may include the spatial neighborhood and/or subband neighborhood of the coefficient, or the three-dimensional neighborhood defined in the subsequent embodiments.
  • the probability distribution of the coefficients can also be estimated in parallel, that is, the probability distributions of multiple coefficients are obtained at one time, and then each transform coefficient is losslessly performed according to the preset order 2. Encoding, or lossless encoding of multiple coefficients in parallel.
  • Method 1 The probability estimation network directly estimates the probability of the value within the value range according to the dynamic value range of the coefficient, and obtains a probability vector, which includes The probability value of each value in the value range, where the sum of all probability values is 1.
  • Method 2 The probability distribution model can also be used to obtain the probability vector. For example, a single Gaussian model or a mixed Gaussian model is used for modeling, the probability estimation network is used to estimate the model parameters, and the model parameters are substituted into the probability distribution model to obtain the probability vector.
  • the context information is input to the probability estimation network, the probability is estimated on the coefficient to obtain the probability distribution of the coefficient, and the current coefficient is losslessly encoded using the probability distribution to obtain the Coding information of the coefficient.
  • the probability distribution estimation method used in the probability estimation network may include a probability distribution estimation method based on a deep learning network, such as a recurrent neural network and a convolutional neural network.
  • the coefficient at the position (bx1, by1, fs1) is in the subband neighborhood of the coefficient.
  • M1, M2, N1, N2, and N3 are integers greater than or equal to 0, and the values can be equal or unequal to each other, which is not limited here, for example, the values can be 1, 2, 3, 4, or 5. .
  • the preset order 1 and the preset order 2 can be the processing order obtained by zig-zag (zig-zag or Z-character) scanning, and other scanning methods can also be used to obtain the processing order, such as row-by-row scanning and column-by-column scanning. , Diagonal scanning, horizontal reverse scanning and vertical reverse scanning, etc.
  • progressive scanning refers to continuous scanning from the first row of the image to the last row
  • column-wise scanning refers to continuous scanning from the first column of the image to the last column.
  • Step 404 Obtain the transcoded output file.
  • FIG. 8 it is a schematic diagram of an image decoding process provided by an embodiment of this application. mainly includes:
  • Step 801 Obtain encoding information after entropy encoding, and perform entropy decoding on the encoded information to obtain a coefficient array.
  • each symbol is decoded using the same processing order as the encoding end. That is, first process each encoded two-dimensional sub-band coefficient array according to the preset order 1. When processing each encoded two-dimensional sub-band coefficient array, process each two-dimensional sub-band coefficient array according to the preset order 2. Each of the encoded coefficients.
  • the coded coefficients can be understood as symbols.
  • the decoded symbol in the preset context area of the symbol to be decoded is input into the probability estimation network as the context, and the probability distribution of the current symbol is estimated, and the probability distribution is used for the current symbol.
  • Decode the transform coefficient of the current symbol After decoding each symbol, an 8x8 transform coefficient block or coefficient array C (bx, by, fx, fy) of multiple images is obtained.
  • Step 802 Obtain decoded image data.
  • Method one decoding corresponding to method one in step 401: performing inverse transformation and inverse quantization on the coefficient array C (bx, by, fx, fy) to obtain decoded image data.
  • Method two decoding corresponding to method two in step 401: performing entropy coding on an 8 ⁇ 8 transform coefficient block to obtain entropy-coded image data.
  • the input image block is divided into image blocks of a preset size, and then transformed and quantized to obtain a transform coefficient array.
  • Each transform coefficient in the transform coefficient array is processed in a preset order, and when arithmetic coding is performed on one of the coefficients, the processed coefficient in the neighborhood of the coefficient is used as the context to estimate the probability distribution of the coefficient value.
  • the embodiments of the present application can increase the compression rate of images and reduce storage resources on the server side.
  • This embodiment relates to an image coding scheme.
  • a transform coefficient block (referred to as a coefficient block) is obtained, and then the coefficient block is used for lossless coding to obtain coded image data.
  • This embodiment provides the entire encoding process, which may include, for example, transforming and quantizing the image area, and then performing lossless encoding on the quantized coefficients to obtain encoded image data.
  • the image coding process mainly includes:
  • Step 901 Obtain a transform coefficient block.
  • This step is the same as step 401 in the first embodiment.
  • Step 902 Obtain the transform coefficient value of the designated frequency point in the 8x8 transform coefficient block in the image to form a two-dimensional subband coefficient array of the frequency point.
  • This step is the same as step 402 in the first embodiment.
  • Step 903 Arrange the two-dimensional sub-band coefficient array in a preset order to obtain a three-dimensional sub-band coefficient array.
  • any 8x8 transform coefficient block in the image and sort the transform coefficients of the specified frequency point (fx, fy) in the coefficient block according to the preset order, then the two-dimensional of the specified frequency point (fx, fy)
  • the sort index of the subband coefficient array is the same as the sort index in the coefficient block, so as to determine the frequency of the two-dimensional subband coefficient array C ⁇ (fx,fy)(bx,by) at the specified frequency point (fx, fy) Index fs.
  • the two-dimensional subband coefficient array C ⁇ (fx, fy)(bx, by) is combined to obtain the three-dimensional subband coefficient array C(bx, by, fs) as shown in FIG. 8.
  • the one-dimensional frequency index fs is an integer in 0,..., subW ⁇ subH-1. It can be understood that the frequency index fs is an index after sorting the two-dimensional subband coefficient array in a preset order.
  • the preset order one can be the arrangement order obtained by zig-zag scanning, and other scanning methods can also be used to obtain the preset order, such as: progressive scanning, column-by-column scanning and diagonal scanning.
  • progressive scanning refers to continuous scanning from the first row of the image to the last row
  • column-wise scanning refers to continuous scanning from the first column of the image to the last column.
  • FIG. 11a it is a schematic diagram of a three-dimensional subband coefficient array provided by an embodiment of this application.
  • the three-dimensional subband coefficient array C (bx, by, fs) may include data in three dimensions: bx, by, and fs.
  • step 901 and step 902 in this embodiment are the same as step 401 and step 402 in the first embodiment. The difference is that in the second embodiment, a step of combining the two-dimensional sub-band coefficient array into a three-dimensional sub-band coefficient array according to a preset order is added.
  • Step 904 Process each transform coefficient in the three-dimensional subband coefficient array. When processing each coefficient, use the processed coefficient in the three-dimensional neighborhood of the coefficient as a context estimation to obtain the probability distribution of the current coefficient, and use the probability distribution Perform arithmetic coding on the current coefficient to obtain the coding information of the current coefficient.
  • FIG. 11b it is a schematic diagram of the three-dimensional transform coefficients provided by the embodiment of this application.
  • One square filled with black in the figure represents the currently processed transform coefficient
  • the square filled with horizontal lines represents the uncoded transform coefficient
  • the white square represents the encoded transform coefficient. Then the three-dimensional neighborhood of the currently processed transform coefficient can be Including the coded transform coefficients.
  • Step 9041 set the processing order of each transform coefficient in the three-dimensional subband coefficient array C (bx, by, fs).
  • one of the following methods can be used to set the processing order of each transform coefficient, and the first method is the best method.
  • the preset order two can use zig-zag scanning, line-by-line scan, and column-by-column scanning, etc.
  • the processing order is the same as the "preset order one" in step three, and will not be repeated here.
  • Step 9042 obtain the probability distribution of the coefficient.
  • the coefficient in the three-dimensional neighborhood of the coefficient can be obtained according to the following method: if the coordinates of the coefficient are (bx0, by0, fs0), if the coordinate position (bx1, by1, fs1) satisfies bx0-M1 ⁇ bx1 ⁇ bx0+N1 , And by0-M2 ⁇ by1 ⁇ by0+N2, and fs0-N3 ⁇ fs1 ⁇ fs0, then the coefficient at (bx1, by1, fs1) is in the three-dimensional neighborhood of the coefficient, where M1, M2, N1, N2 and N3 It is an integer greater than or equal to 0, and the values can be equal or unequal to each other, which is not limited here.
  • the values can be 1, 2, 3, 4, or 5. Therefore, the processed coefficient in the three-dimensional neighborhood of the coefficient also includes the processed coefficient in the two-dimensional neighborhood of the coefficient, which is not limited here.
  • the probability distribution of the coefficients can also be estimated in parallel, that is, the probability distributions of multiple coefficients are obtained at one time, and then each transform coefficient is losslessly performed according to the preset order 2. Encoding, or lossless encoding of multiple coefficients in parallel.
  • the context information is input to the probability estimation network, the probability is estimated on the coefficient to obtain the probability distribution of the coefficient, and the current coefficient is losslessly encoded using the probability distribution to obtain the coefficient Encoding information.
  • the probability distribution estimation methods used in the probability estimation network include probability distribution estimation methods based on deep learning networks, such as recurrent neural networks and pixel-by-pixel convolutional neural networks.
  • Method 1 The probability estimation network directly estimates the probability of the value within the value range according to the dynamic value range of the coefficient, and obtains a probability vector, which includes The probability value of each value in the value range, where the sum of all probability values is 1.
  • Method 2 The probability distribution model can also be used to obtain the probability vector. For example, a single Gaussian model or a mixed Gaussian model is used for modeling, the probability estimation network is used to estimate the model parameters, and the model parameters are substituted into the probability distribution model to obtain the probability vector.
  • Step 905 Obtain the transcoded output file.
  • FIG. 12 it is a schematic diagram of an image decoding process provided by an embodiment of this application. mainly includes:
  • Step 1201 Obtain encoding information after entropy encoding, and perform entropy decoding on the encoded information to obtain decoded transform coefficients.
  • the same processing order refers to first determining a frequency point in the image encoding information according to the frequency point scanning order
  • the coefficient to be decoded corresponding to a frequency point is decoded according to the coefficient scanning order.
  • input the symbol to be decoded into the same probability estimation network as the encoding end use the processed symbol in the three-dimensional neighborhood of the symbol as the context estimation to obtain the probability distribution of the current symbol, and use the probability The distribution decodes the current symbol to obtain the transform coefficient of the current symbol.
  • a transform coefficient block or coefficient array C (bx, by, fx, fy) is obtained.
  • Step 1203 Obtain decoded image data.
  • Method one decoding corresponding to method one in step 401: performing inverse transformation and inverse quantization on the coefficient array C (bx, by, fx, fy) to obtain decoded image data.
  • Method two decoding corresponding to method two in step 401: performing entropy coding on an 8 ⁇ 8 transform coefficient block to obtain entropy-coded image data.
  • this embodiment combines the transform coefficients according to frequency components to obtain a three-dimensional coefficient array C (bx, by, fs), and when performing arithmetic coding on one of the coefficients, the coefficient neighborhood is used.
  • the processed coefficient is used as the context to estimate the probability distribution of the coefficient value.
  • the embodiments of the present application can increase the compression rate of images and reduce storage resources on the server side.
  • This embodiment relates to an image coding scheme.
  • a transform coefficient block (referred to as a coefficient block) is obtained, and then the coefficient block is used for lossless coding to obtain coded image data.
  • This embodiment provides the entire encoding process, which may include, for example, transforming and quantizing the image area, and then performing lossless encoding on the quantized coefficients to obtain encoded image data.
  • This embodiment uses the method in step 401 of the first embodiment to obtain the transform coefficient array C (bx, by, fx, fy), and there are the following two coding methods:
  • Method 1 Firstly, each transform coefficient in the coefficient block is sorted according to the preset order 1, and then the coefficients of the same frequency point are obtained and combined to obtain a three-dimensional sub-band coefficient array (referred to as a three-dimensional coefficient array).
  • the image coding process mainly includes:
  • Step 1301 Obtain a transform coefficient block.
  • This step is the same as step 401 in the first embodiment.
  • Step 1302 Arrange each transform coefficient in the 8 ⁇ 8 transform coefficient block according to the preset order to obtain the sorted 8 ⁇ 8 transform coefficient block.
  • Step 1303 Obtain the transform coefficient value of the designated frequency point in the 8x8 transform coefficient block in the image, form a two-dimensional subband coefficient array of the frequency point, and combine the two-dimensional subband coefficient array to obtain a three-dimensional subband coefficient array.
  • Step 1304 Process each transform coefficient in the three-dimensional subband coefficient array. When processing each coefficient, use the processed coefficient in the three-dimensional neighborhood of the coefficient as a context estimation to obtain the probability distribution of the current coefficient, and use the probability distribution Perform arithmetic coding on the current coefficient to obtain the coding information of the current coefficient. Specifically, it is the same as step 904 in the second embodiment.
  • Method 2 The implementation of this method is similar to that of Example 1, and it does not need to be combined into a three-dimensional subband coefficient array.
  • each two-dimensional sub-band coefficient array Process each two-dimensional sub-band coefficient array according to fs.
  • the processing order when processing each transform coefficient in this embodiment is different from the processing order in the first and second embodiments.
  • FIG. 12 it is a schematic diagram of an image decoding process provided by an embodiment of this application. mainly includes:
  • Step 1201 Obtain encoding information after entropy encoding, and perform entropy decoding on the encoded information to obtain decoded transform coefficients.
  • the same processing order refers to first determining a frequency point in the image encoding information according to the frequency point scanning order
  • the coefficient to be decoded corresponding to a frequency point is decoded according to the coefficient scanning order.
  • input the symbol to be decoded into the same probability estimation network as the encoding end use the processed symbol in the three-dimensional neighborhood of the symbol as the context estimation to obtain the probability distribution of the current symbol, and use the probability The distribution decodes the current symbol to obtain the transform coefficient of the current symbol.
  • a transform coefficient block or coefficient array C (bx, by, fx, fy) is obtained.
  • Step 1202 Obtain decoded image data.
  • Method one decoding corresponding to method one in step 401: performing inverse transformation and inverse quantization on the coefficient array C (bx, by, fx, fy) to obtain decoded image data.
  • Method two decoding corresponding to method two in step 401: performing entropy coding on an 8 ⁇ 8 transform coefficient block to obtain entropy-coded image data.
  • this embodiment combines transform coefficients according to frequency components to obtain a three-dimensional coefficient array C (bx, by, fs), and when performing arithmetic coding on one of the coefficients, the coefficient neighborhood is used
  • the processed coefficient is used as the context to estimate the probability distribution of the coefficient value.
  • the embodiments of the present application can increase the compression rate of images and reduce storage resources on the server side.
  • This embodiment relates to an image coding scheme.
  • a transform coefficient block (referred to as a coefficient block) is obtained, and then the coefficient block is used for lossless coding to obtain coded image data.
  • This embodiment provides the entire encoding process, which may include, for example, transforming and quantizing the image area, and then performing lossless encoding on the quantized coefficients to obtain encoded image data.
  • the image coding process mainly includes:
  • Step 1401 Obtain a block of transform coefficients.
  • This step is the same as step 401 in the first embodiment.
  • Step 1402 Obtain the transform coefficient value of the designated frequency point in the 8x8 transform coefficient block in the image to form a two-dimensional subband coefficient array of the frequency point.
  • step 1401 and step 1402 in this embodiment are the same as step 401 and step 402 in the first embodiment.
  • the difference is that in the entropy coding in the fourth embodiment, the two-dimensional subband coefficient array is used as a subband image for coding.
  • Step 1403 Use the two-dimensional subband coefficient array corresponding to the frequency point as a subband image, and use an image lossless coding method for coding when processing a coefficient.
  • the two-dimensional subband coefficient array C ⁇ (fx, fy)(bx, by) corresponding to the designated frequency point is taken as a subband image, the subband images are processed according to a preset order, and the subband image processing order index is set as fs.
  • the existing lossless image processing method is used for encoding. For example, lossless coding modes in standard schemes such as H.264 and H.265, such as arithmetic coding methods.
  • the method for obtaining the processing order index fs includes: first obtaining any 8x8 transform coefficient block in the image, and sorting the transform coefficients of the specified frequency points (fx, fy) in the coefficient block according to the preset order, then specifying
  • the processing order index fs of the two-dimensional subband coefficient array of the frequency point (fx, fy) is the same as the order index in the coefficient block.
  • the preset sequence has the same meaning as in the first embodiment, and will not be repeated here.
  • the two-dimensional subband coefficient array is used as a subband image for coding, which is simpler to implement.
  • This embodiment relates to an image coding scheme.
  • a transform coefficient block (referred to as a coefficient block) is obtained, and then the coefficient block is used for lossless coding to obtain coded image data.
  • This embodiment provides the entire encoding process, which may include, for example, transforming and quantizing the image area, and then performing lossless encoding on the quantized coefficients to obtain encoded image data.
  • the image coding process mainly includes:
  • Step 1501 Obtain a block of transform coefficients.
  • This step is the same as step 401 in the first embodiment.
  • Step 1501 Divide the image area to be processed into image blocks of a preset size, and perform transformation and quantization on each image block to obtain a transform coefficient array.
  • the image or image area to be processed is divided into sub-image blocks of the same preset size.
  • the size of the sub-image blocks of the preset size may be 4x4, 8x8, 16x16, 32x32, 64x64, 128x128, 256x256, and so on.
  • DCT transformation is performed on a sub-image block of a preset size to obtain a transform coefficient block.
  • the transform coefficient block is quantized to obtain a quantized transform coefficient array C (bx, by, fx, fy), and each transform coefficient in the coefficient array can be C_(bx, by) ⁇ (fx, fy )Express.
  • (bx, by) represents the horizontal and vertical position index of the coefficient block
  • M W/subW
  • N H/subH
  • M and N are the width and height of the image in sub-blocks
  • W and H represent the width and height of the current image
  • subW and subH represent the width and height of the sub-image block in the current image.
  • the size of the sub-image block of the preset size is taken as an example of 8x8, that is, the width and height of the sub-image block are both 8, that is, the values of subW and subH are both 8, and will not be described in detail later.
  • the difference between the fifth embodiment and the first to fourth embodiments is that, when processing each transform coefficient, there is no action of constructing a two-dimensional subband coefficient array, nor is there an action of constructing a three-dimensional coefficient array.
  • the coefficient block is processed in accordance with the preset order 1 and the preset order 2, which is simpler in implementation and lower in complexity.
  • step 1501 may be replaced with the following step: entropy decoding the entropy-encoded image data to obtain transform coefficients of 8 ⁇ 8 blocks in the image.
  • Each transform coefficient of the coefficient block is recorded as C_( bx,by) ⁇ (fx,fy).
  • (bx, by) represents the horizontal and vertical position index of the coefficient block
  • M W/subW
  • N H/subH
  • M and N are the width and height of the image in sub-blocks
  • W and H represent the width and height of the current image
  • subW and subH represent the width and height of the sub-image block in the current image.
  • the width and height of all sub-image blocks in the JPEG file are both 8, that is, the values of subW and subH are both 8, which will not be described in detail later.
  • the technical solutions described in the embodiments of the present application can also be applied to the case where the values of subW and subH are not 8, and even subW is not equal to subH.
  • Step 1502 Process each frequency subband of the coefficient block of the current image according to the preset order one, and process the transform coefficients in each frequency subband according to the preset order two.
  • processing each transform coefficient select the processed coefficient in the preset context area as the context estimation to obtain the probability distribution of the current coefficient, and use the probability distribution to perform arithmetic coding on the current coefficient to obtain the coding information of the current coefficient.
  • the aforementioned step 1502 can be replaced with the following steps: according to the preset order one, the transform coefficient corresponding to a frequency point in the coefficient block of the current image is processed, and the transformation coefficient corresponding to each frequency point is processed according to the preset order two. Transformation coefficient.
  • the transform coefficient corresponding to a frequency point in the coefficient block of the current image is processed, and the transformation coefficient corresponding to each frequency point is processed according to the preset order two. Transformation coefficient.
  • the frequency subband is a set of coefficients C ⁇ (fx, fy)(bx, by) at a specified frequency point (fx, fy) in a transform coefficient block of 8x8, and there are 64 frequency subbands in this embodiment.
  • First obtain any 8x8 transform coefficient block in the image scan the coefficients of the specified frequency point (fx, fy) in the coefficient block according to the preset order, then specify the frequency subband of the frequency point (fx, fy)
  • the index fs is the same as the scanning order of the frequency point in the coefficient block, and the frequency subbands corresponding to fs are processed in the order of the magnitude of fs.
  • fs is an integer, and the value range is 0,...,subW ⁇ subH-1.
  • each transform coefficient in the frequency subband is processed according to the preset order two.
  • select the processed coefficient in the preset context area as the context and input the context information into the probability estimation network to estimate the probability distribution of the coefficient to obtain the probability distribution of the current coefficient and use the probability
  • the distribution performs arithmetic coding on the current coefficient to obtain the coding information of the current coefficient.
  • the preset context area may include the spatial neighborhood and/or subband neighborhood of the coefficient, or the three-dimensional neighborhood defined in the second embodiment.
  • Method 1 The probability estimation network directly estimates the probability of the value within the value range according to the dynamic value range of the coefficient, and obtains a probability vector, which includes The probability value of each value in the value range, where the sum of all probability values is 1.
  • Method 2 The probability distribution model can also be used to obtain the probability vector. For example, a single Gaussian model or a mixed Gaussian model is used for modeling, the probability estimation network is used to estimate the model parameters, and the model parameters are substituted into the probability distribution model to obtain the probability vector.
  • the context information is input to the probability estimation network, the probability is estimated on the coefficient to obtain the probability distribution of the coefficient, and the current coefficient is losslessly encoded using the probability distribution to obtain the coefficient Encoding information.
  • the probability distribution estimation methods used in probability estimation networks include probability distribution estimation methods based on deep learning networks, such as recurrent neural networks and convolutional neural networks.
  • the coefficient at the position (bx1, by1, fs1) is in the subband neighborhood of the coefficient.
  • the values of integers equal to 0 can be equal or unequal to each other, and are not limited here. For example, they are 1, 2, 3, 4, or 5.
  • the preset order 1 and the preset order 2 can be the processing order obtained by zig-zag scanning, and other scanning methods can also be used to obtain the processing order, such as: progressive scanning, column-by-column scanning, diagonal scanning , Horizontal reverse scanning and vertical reverse scanning, etc.
  • progressive scanning refers to continuous scanning from the first row of the image to the last row
  • column-wise scanning refers to continuous scanning from the first column of the image to the last column.
  • Step 1503 Obtain the transcoded output file.
  • FIG. 16 it is a schematic diagram of an image decoding process provided by an embodiment of this application. mainly includes:
  • Step 1601 Obtain encoding information after entropy encoding, and perform entropy decoding on each symbol in the encoding information to obtain decoded transform coefficients.
  • Entropy decoding is performed on the encoded information after entropy encoding, and each symbol is decoded using the same processing order as the encoding end. First, obtain each frequency subband according to the index of the frequency subband or the preset order one, and then process the symbols in each frequency subband according to the preset order two. When processing each symbol, the decoded symbol in the preset context area of the symbol is input into the probability estimation network as the context to obtain the probability distribution of the current symbol, and use the probability distribution to decode the current symbol to obtain The transform coefficient of the current symbol. Entropy decoding is performed on each symbol output by the encoding end in turn to obtain the transform coefficient of each symbol, thereby obtaining the coefficient block of the image.
  • Step 1602 Obtain decoded image data.
  • Method one decoding corresponding to method one in step 401: performing inverse transformation and inverse quantization on the coefficient block to obtain decoded image data.
  • Method two decoding corresponding to method two in step 401: performing entropy coding on an 8 ⁇ 8 transform coefficient block to obtain entropy-coded image data.
  • the embodiment of the application relates to block transformation and coefficient coding in video image compression.
  • the transformation coefficients are combined according to designated frequency points to obtain a three-dimensional coefficient array C (bx, by, fs), and the coefficient sub-image corresponding to each frequency index fs is lossless coding.
  • the transform coefficients are combined according to the specified frequency points to obtain the three-dimensional coefficient array C (bx, by, fs), and when arithmetic coding is performed on one of the coefficients, the coefficients processed in the three-dimensional neighborhood of the coefficient are used as the context to estimate the probability of the coefficient value distributed.
  • each transform coefficient in each two-dimensional subband coefficient array When processing each transform coefficient in each two-dimensional subband coefficient array, follow the preset order two Process each transform coefficient in each two-dimensional subband coefficient array, and use the processed coefficient in the coefficient neighborhood as the context to estimate the probability distribution of the coefficient value. Obtain the two-dimensional subband coefficient array C ⁇ (fx,fy)(bx,by) at the specified frequency point. The two-dimensional sub-band coefficient array corresponding to the frequency point is taken as a sub-band image, and each sub-band image is processed according to a preset order when processing each sub-band image. When processing each transform coefficient of the sub-band image, the image lossless coding method is used for coding.
  • the frequency subband is a set of coefficients C ⁇ (fx, fy)(bx, by) at a specified frequency point (fx, fy) in the 8x8 transform coefficient block.
  • the image area to be processed is divided into image blocks of a preset size, and each image block is transformed and quantized to obtain a transform coefficient array C (bx, by, fx, fy) . Then, entropy coding is performed on each of the coefficients in a preset order, and the preset order will give priority to processing the coefficients of all coefficient blocks corresponding to each frequency index.
  • the processed coefficient in the neighboring area is selected as the context estimation to obtain the probability distribution of the current coefficient, and the current coefficient is followed based on the probability distribution Encoding operation.
  • the embodiments of the present application can further reduce the size of compressed files and reduce the storage cost of video image files on the server side.
  • an image encoding device 1700 may include: a transceiver module 1701 and a processing module 1702, where:
  • the transceiver module is configured to obtain at least two transform coefficient blocks, each of the at least two transform coefficient blocks includes: at least two transform coefficients, wherein each transform coefficient in one transform coefficient block corresponds to a different Frequency point;
  • the processing module is configured to determine the transform coefficients in the at least two transform coefficient blocks corresponding to one frequency point according to a preset frequency point scanning order; and to perform processing on each transform coefficient corresponding to the one frequency point according to the preset coefficient scanning order Perform encoding to obtain image encoding information.
  • the frequency point scanning order can be preset or configured, and a frequency point is determined based on the frequency point scanning order.
  • the frequency point can be any frequency point, and each frequency point corresponds to at least two transform coefficient blocks.
  • a transform coefficient in each transform coefficient block in each transform coefficient block, that is, a frequency point corresponds to multiple transform coefficients. These multiple transform coefficients are derived from different transform coefficient blocks.
  • the coefficient scanning order can be preset or configured based on the coefficient scanning order Encoding each transform coefficient corresponding to a frequency point can finally obtain image encoding information.
  • the image coding information can be obtained only by coding according to the coefficient scanning order. Therefore, coding can be realized for multiple transform coefficients corresponding to the same frequency point.
  • the image compression efficiency is improved, and the image encoding information in the embodiment of the application is obtained by encoding the transform coefficients corresponding to the same frequency points in at least two transform coefficient blocks according to the coefficient scanning order, and the file of the encoded image encoding information is reduced, Reduce the storage cost of image coding information, and ultimately save storage resources.
  • the processing module is configured to obtain at least two two-dimensional sub-band coefficient arrays from the at least two transform coefficient blocks, wherein one two-dimensional sub-band coefficient array includes the at least two transform coefficient blocks.
  • the transform coefficients corresponding to the same frequency points in the two transform coefficient blocks, and the transform coefficients included in the different two-dimensional subband coefficient arrays correspond to different frequency points; the at least two two-dimensional frequency points are determined according to a preset scanning order of frequency points.
  • the first two-dimensional sub-band coefficient array in the sub-band coefficient array; the first transform coefficient in the first two-dimensional sub-band coefficient array is determined according to a preset coefficient scanning order; according to the context corresponding to the first transform coefficient
  • the region performs entropy coding on the first transform coefficient.
  • the processing module is configured to obtain at least two two-dimensional sub-band coefficient arrays from the at least two transform coefficient blocks, wherein one two-dimensional sub-band coefficient array includes the at least two transform coefficient blocks.
  • the transform coefficients corresponding to the same frequency points in the two transform coefficient blocks, the transform coefficients included in the different two-dimensional subband coefficient arrays correspond to different frequency points;
  • the sub-band coefficient arrays are combined to obtain a three-dimensional sub-band coefficient array; the first two-dimensional sub-band coefficient array in the three-dimensional sub-band coefficient array is determined according to a preset scanning order of three-dimensional sub-band coefficients; and the first two-dimensional sub-band coefficient array is scanned according to the preset coefficients.
  • the first transform coefficient in the first two-dimensional subband coefficient array is determined in order; and the first transform coefficient is entropy-encoded according to the context region corresponding to the first transform coefficient.
  • the processing module is configured to sort the at least two transform coefficient blocks according to a preset frequency point scanning order to obtain the sorted at least two transform coefficient blocks; Obtain multiple two-dimensional subband coefficient arrays corresponding to different frequency points from the sorted at least two transform coefficient blocks; combine the multiple two-dimensional subband coefficient arrays corresponding to the different frequency points to obtain three-dimensional subband coefficients Array; determine the first two-dimensional sub-band coefficient array from the three-dimensional sub-band coefficient array according to the preset three-dimensional sub-band coefficient scanning order; determine the first two-dimensional sub-band coefficient array in the first two-dimensional sub-band coefficient array according to the preset coefficient scanning order A first transform coefficient; performing entropy coding on the first transform coefficient according to the context region corresponding to the first transform coefficient.
  • the scanning order of the three-dimensional subband coefficients includes: a preset change order of the horizontal position index; or, a preset change order of the vertical position index; or, a preset frequency index The order of change.
  • the processing module is configured to determine, according to a preset frequency point scanning order, a first two-dimensional subband composed of transform coefficients corresponding to the same frequency point in the at least two transform coefficient blocks Coefficient array; determine the first transform coefficient in the first two-dimensional subband coefficient array according to a preset coefficient scanning order; perform entropy coding on the first transform coefficient according to the context area corresponding to the first transform coefficient.
  • the processing module is configured to determine the transform coefficients in the at least two transform coefficient blocks included in one frequency subband according to a preset frequency point scanning order, and the one frequency subband The included transform coefficients in the at least two transform coefficient blocks correspond to the same frequency point; the first transform coefficient included in the one frequency subband is determined according to the preset coefficient scanning order; the first transform coefficient included in the one frequency subband is determined according to the corresponding first transform coefficient The context area of performs entropy coding on the first transform coefficient.
  • the processing module is configured to obtain context information of the first transform coefficient from a context region corresponding to the first transform coefficient, wherein the context information of the first transform coefficient
  • the method includes: the encoded transform coefficient in the context area corresponding to the first transform coefficient; estimating the first transform coefficient according to the context information of the first transform coefficient to obtain the probability distribution of the first transform coefficient Information; Perform arithmetic coding on the first transform coefficient according to the probability distribution information of the first transform coefficient.
  • the context area corresponding to the first transform coefficient includes at least one of the following: a spatial neighborhood of the first transform coefficient, a subband neighborhood of the first transform coefficient, and The three-dimensional neighborhood of the first transform coefficient.
  • the processing module is configured to, when the first transform coefficient is a plurality of transform coefficients, obtain the information of the plurality of transform coefficients from the context regions corresponding to the plurality of transform coefficients.
  • Context information wherein the context information of the plurality of transform coefficients includes: encoded transform coefficients in the context regions corresponding to the plurality of transform coefficients; Estimation is performed in parallel to obtain the probability distribution information of the plurality of transform coefficients; and the arithmetic coding is performed on the plurality of transform coefficients in parallel according to the probability distribution information of the plurality of transform coefficients.
  • the context regions corresponding to the multiple transform coefficients include at least one of the following: spatial neighborhoods of the multiple transform coefficients, subband neighborhoods of the multiple transform coefficients, and A three-dimensional neighborhood of multiple transform coefficients.
  • the processing module is configured to obtain at least two two-dimensional sub-band coefficient arrays from the at least two transform coefficient blocks, wherein one two-dimensional sub-band coefficient array includes the at least two transform coefficient blocks.
  • the transform coefficients corresponding to the same frequency points in the two transform coefficient blocks, and the transform coefficients included in the different two-dimensional subband coefficient arrays correspond to different frequency points; the at least two two-dimensional frequency points are determined according to a preset scanning order of frequency points.
  • At least two sub-band images corresponding to the sub-band coefficient array encoding the transform coefficients in the at least two sub-band images according to a preset coefficient scanning order.
  • the transceiver module is configured to obtain transform coefficient encoding information of at least two image blocks; and perform entropy decoding on the transform coefficient encoding information to obtain the at least two transform coefficient blocks.
  • the transceiver module is configured to divide the original image area into at least two image blocks, and the original image area includes: a pixel image area or a residual image area; The image block is transformed and quantized to obtain the at least two transform coefficient blocks.
  • the component modules of the image processing device can also execute the steps described in the foregoing first aspect and various possible implementation manners.
  • an image decoding apparatus 1800 may include: a transceiver module 1801 and a processing module 1802, where:
  • the transceiver module is used to obtain image coding information
  • the processing module is configured to determine the transform coefficient to be decoded corresponding to a frequency point in the image encoding information according to the preset frequency point scanning order; perform the transform coefficient to be decoded corresponding to the one frequency point according to the preset coefficient scanning order Decoding to obtain at least two transform coefficient blocks, wherein each transform coefficient block in the at least two transform coefficient blocks includes: at least two transform coefficients, and each transform coefficient in one transform coefficient block corresponds to a different frequency point.
  • the processing module is configured to determine the first to-be-decoded two-dimensional subband coefficient array in the image encoding information according to a preset frequency point scanning order, and the first to-be-decoded two-dimensional subband coefficient array
  • the multi-dimensional subband coefficient array includes: a transform coefficient to be decoded corresponding to a frequency point in the image encoding information; the first transform coefficient to be decoded in the first two-dimensional subband coefficient array to be decoded is determined according to a preset coefficient scanning order Coefficients; perform entropy decoding on the first transform coefficient to be decoded according to the context region corresponding to the first transform coefficient to be decoded.
  • the processing module is configured to determine the three-dimensional subband coefficient array to be decoded in the image encoding information according to a preset frequency point scanning order, and the three-dimensional subband coefficient array to be decoded includes : The transform coefficients to be decoded corresponding to all frequency points in the image encoding information; the first two-dimensional subband coefficient array to be decoded in the three-dimensional subband coefficient array to be decoded is determined according to the preset scanning order of the three-dimensional subband coefficients
  • the first two-dimensional subband coefficient array to be decoded includes: a transform coefficient to be decoded corresponding to a frequency point in the image encoding information; and the first two-dimensional subband coefficient to be decoded is determined according to a preset coefficient scanning order
  • the first transform coefficient to be decoded in the array; the first transform coefficient to be decoded is entropy-decoded according to the context region corresponding to the first transform coefficient to be decoded.
  • the scanning order of the three-dimensional subband coefficients includes: a preset change order of the horizontal position index; or, a preset change order of the vertical position index; or, a preset frequency index The order of change.
  • the processing module is configured to determine the transform coefficient to be decoded included in a frequency subband in the image encoding information according to a preset frequency point scanning order, and the one frequency subband includes The transform coefficients to be decoded correspond to the same frequency point; the first transform coefficient to be decoded in the one frequency subband is determined according to a preset coefficient scanning order; the first transform coefficient to be decoded corresponds to the context region corresponding to the Entropy decoding is performed on the first transform coefficient to be decoded.
  • the processing module is configured to obtain context information of the first transform coefficient to be decoded from a context region corresponding to the first transform coefficient to be decoded, wherein the first transform coefficient to be decoded is
  • the context area corresponding to the decoded transform coefficient includes: the decoded transform coefficient in the context area corresponding to the first transform coefficient to be decoded; and the first transform coefficient to be decoded is performed according to the context information of the first transform coefficient to be decoded Estimate to obtain the probability distribution information of the first transform coefficient to be decoded; and perform arithmetic decoding on the first transform coefficient to be decoded according to the probability distribution information of the first transform coefficient to be decoded to obtain the decoded first Transform coefficient, the decoded first transform coefficient belongs to the at least two transform coefficient blocks.
  • the context area corresponding to the first transform coefficient to be decoded includes at least one of the following: a spatial neighborhood of the first transform coefficient to be decoded, and a sub-region of the first transform coefficient to be decoded Band neighborhood, the three-dimensional neighborhood of the first transform coefficient to be decoded.
  • the processing module is configured to, when the first to-be-decoded transform coefficient is a plurality of to-be-decoded transform coefficients, obtain all the data from the context regions corresponding to the plurality of to-be-decoded transform coefficients.
  • the context information of the plurality of transform coefficients to be decoded includes: decoded transform coefficients in the context regions corresponding to the plurality of transform coefficients to be decoded; according to the plurality of transform coefficients to be decoded
  • the context information of the transform coefficient estimates the plurality of transform coefficients to be decoded in parallel to obtain the probability distribution information of the first transform coefficient to be decoded;
  • a plurality of transform coefficients to be decoded are arithmetically decoded in parallel to obtain a decoded first transform coefficient, and the decoded first transform coefficient belongs to the at least two transform coefficient blocks.
  • the context regions corresponding to the plurality of transform coefficients to be decoded include at least one of the following: spatial neighborhoods of the plurality of transform coefficients to be decoded, and sub-regions of the plurality of transform coefficients to be decoded With neighborhood, the three-dimensional neighborhood of the plurality of transform coefficients to be decoded.
  • the processing module is configured to determine a first sub-band image in the image encoding information according to a preset frequency point scanning order, and the first sub-band image includes: the image The to-be-decoded transform coefficient corresponding to a frequency point in the encoded information is decoded according to a preset coefficient scanning order to decode the to-be-decoded coefficient of the first subband image to obtain the at least two transform coefficient blocks.
  • the processing module is further configured to perform entropy coding on the at least two transform coefficient blocks after obtaining at least two transform coefficient blocks to obtain transform coefficient coding information;
  • the coefficient encoding information generates at least two image blocks; the original image area is restored according to the at least two image blocks, and the original image area includes: a pixel image area or a residual image area.
  • the processing module is further configured to perform inverse quantization and inverse transformation on the at least two transform coefficient blocks after obtaining at least two transform coefficient blocks to obtain at least two image blocks;
  • the original image area is restored according to the at least two image blocks, and the original image area includes: a pixel image area or a residual image area.
  • An embodiment of the present application also provides a computer storage medium, wherein the computer storage medium stores a program, and the program executes some or all of the steps recorded in the above method embodiments.
  • the image encoding device 1900 includes:
  • the receiver 1901, the transmitter 1902, the processor 1903, and the memory 1904 may be one or more, and one processor is taken as an example in FIG. 19).
  • the receiver 1901, the transmitter 1902, the processor 1903, and the memory 1904 may be connected by a bus or in other ways. In FIG. 19, a bus connection is taken as an example.
  • the memory 1904 may include a read-only memory and a random access memory, and provides instructions and data to the processor 1903. A part of the memory 1904 may also include a non-volatile random access memory (NVRAM).
  • NVRAM non-volatile random access memory
  • the memory 1904 stores an operating system and operating instructions, executable modules or data structures, or a subset of them, or an extended set of them.
  • the operating instructions may include various operating instructions for implementing various operations.
  • the operating system may include various system programs for implementing various basic services and processing hardware-based tasks.
  • the processor 1903 controls the operation of the image encoding device, and the processor 1903 may also be referred to as a central processing unit (CPU).
  • the various components of the image encoding device are coupled together through a bus system, where the bus system may include a power bus, a control bus, and a status signal bus in addition to a data bus.
  • bus system may include a power bus, a control bus, and a status signal bus in addition to a data bus.
  • various buses are referred to as bus systems in the figure.
  • the method disclosed in the foregoing embodiment of the present application may be applied to the processor 1903 or implemented by the processor 1903.
  • the processor 1903 may be an integrated circuit chip with signal processing capabilities. In the implementation process, the steps of the foregoing method may be completed by an integrated logic circuit of hardware in the processor 1903 or instructions in the form of software.
  • the aforementioned processor 1903 may be a general-purpose processor, a digital signal processing (digital signal processing, DSP), an application specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or Other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components.
  • DSP digital signal processing
  • ASIC application specific integrated circuit
  • FPGA field-programmable gate array
  • Other programmable logic devices discrete gates or transistor logic devices, discrete hardware components.
  • the general-purpose processor may be a microprocessor or the processor may also be any conventional processor or the like.
  • the steps of the method disclosed in the embodiments of the present application may be directly embodied as being executed and completed by a hardware decoding processor, or executed and completed by a combination of hardware and software modules in the decoding processor.
  • the software module can be located in a mature storage medium in this neighborhood, such as random access memory, flash memory, read-only memory, programmable read-only memory, or electrically erasable programmable memory, registers.
  • the storage medium is located in the memory 1904, and the processor 1903 reads the information in the memory 1904, and completes the steps of the foregoing method in combination with its hardware.
  • the receiver 1901 can be used to receive input digital or character information, and generate signal input related to the relevant settings and function control of the image encoding device.
  • the transmitter 1902 can include display devices such as a display screen, and the transmitter 1902 can be used to output through an external interface Number or character information.
  • the processor 1903 is configured to execute the image processing method executed by the image encoding device shown in FIG. 4 of the foregoing embodiment.
  • the image decoding device 2000 includes:
  • the receiver 2001, the transmitter 2002, the processor 2003, and the memory 2004 (the number of processors 2003 in the image decoding device 2000 may be one or more, and one processor is taken as an example in FIG. 20).
  • the receiver 2001, the transmitter 2002, the processor 2003, and the memory 2004 may be connected by a bus or in other ways. In FIG. 20, the connection by a bus is taken as an example.
  • the memory 2004 may include a read-only memory and a random access memory, and provides instructions and data to the processor 2003. A part of the memory 2004 may also include NVRAM.
  • the memory 2004 stores an operating system and operating instructions, executable modules or data structures, or a subset of them, or an extended set of them.
  • the operating instructions may include various operating instructions for implementing various operations.
  • the operating system may include various system programs for implementing various basic services and processing hardware-based tasks.
  • the processor 2003 controls the operation of the image decoding device, and the processor 2003 may also be referred to as a CPU.
  • the various components of the image decoding device are coupled together through a bus system.
  • the bus system may also include a power bus, a control bus, and a status signal bus.
  • various buses are referred to as bus systems in the figure.
  • the method disclosed in the above embodiments of the present application may be applied to the processor 2003 or implemented by the processor 2003.
  • the processor 2003 may be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the above method can be completed by an integrated logic circuit of hardware in the processor 2003 or instructions in the form of software.
  • the aforementioned processor 2003 may be a general-purpose processor, DSP, ASIC, FPGA or other programmable logic device, discrete gate or transistor logic device, or discrete hardware component.
  • the methods, steps, and logical block diagrams disclosed in the embodiments of the present application can be implemented or executed.
  • the general-purpose processor may be a microprocessor or the processor may also be any conventional processor or the like.
  • the steps of the method disclosed in the embodiments of the present application may be directly embodied as being executed and completed by a hardware decoding processor, or executed and completed by a combination of hardware and software modules in the decoding processor.
  • the software module can be located in a mature storage medium in this neighborhood, such as random access memory, flash memory, read-only memory, programmable read-only memory, or electrically erasable programmable memory, registers.
  • the storage medium is located in the memory 2004, and the processor 2003 reads the information in the memory 2004, and completes the steps of the above method in combination with its hardware.
  • the processor 2003 is configured to execute the image processing method executed by the image decoding device shown in FIG. 4 of the foregoing embodiment.
  • the chip when the image processing device or the image processing device is a chip in the terminal, the chip includes: a processing unit and a communication unit.
  • the processing unit may be, for example, a processor, and the communication unit may be, for example, Input/output interface, pin or circuit, etc.
  • the processing unit can execute the computer-executable instructions stored in the storage unit, so that the chip in the terminal executes the wireless communication method of any one of the above-mentioned first aspects.
  • the storage unit is a storage unit in the chip, such as a register, a cache, etc., and the storage unit may also be a storage unit in the terminal located outside the chip, such as a read-only memory (read-only memory). -only memory, ROM) or other types of static storage devices that can store static information and instructions, random access memory (RAM), etc.
  • the processor mentioned in any one of the foregoing may be a general-purpose central processing unit, a microprocessor, an ASIC, or one or more integrated circuits used to control the execution of the program of the first aspect or the second aspect.
  • the device embodiments described above are only illustrative, and the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physically separate.
  • the physical unit can be located in one place or distributed across multiple network units. Some or all of the modules may be selected according to actual needs to achieve the objectives of the solutions of the embodiments.
  • the connection relationship between the modules indicates that they have a communication connection between them, which may be specifically implemented as one or more communication buses or signal lines.
  • this application can be implemented by software plus necessary general hardware. Of course, it can also be implemented by dedicated hardware including dedicated integrated circuits, dedicated CPUs, and dedicated memories. , Dedicated components, etc. to achieve. Under normal circumstances, all functions completed by computer programs can be easily implemented with corresponding hardware. Moreover, the specific hardware structures used to achieve the same function can also be diverse, such as analog circuits, digital circuits or special-purpose circuits. Circuit etc. However, for this application, software program implementation is a better implementation in more cases. Based on this understanding, the technical solution of this application essentially or the part that contributes to the prior art can be embodied in the form of a software product.
  • the computer software product is stored in a readable storage medium, such as a computer floppy disk. , U disk, mobile hard disk, ROM, RAM, magnetic disk or optical disk, etc., including several instructions to make a computer device (which can be a personal computer, server, or network device, etc.) execute the methods described in each embodiment of this application .
  • the computer program product includes one or more computer instructions.
  • the computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices.
  • the computer instructions may be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another computer-readable storage medium.
  • the computer instructions may be transmitted from a website, computer, server, or data center. Transmission to another website, computer, server or data center via wired (such as coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (such as infrared, wireless, microwave, etc.).
  • wired such as coaxial cable, optical fiber, digital subscriber line (DSL)
  • wireless such as infrared, wireless, microwave, etc.
  • the computer-readable storage medium may be any available medium that can be stored by a computer or a data storage device such as a server or a data center integrated with one or more available media.
  • the usable medium may be a magnetic medium (for example, a floppy disk, a hard disk, and a magnetic tape), an optical medium (for example, a DVD), or a semiconductor medium (for example, a solid state disk (SSD)).

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Compression Or Coding Systems Of Tv Signals (AREA)
  • Compression Of Band Width Or Redundancy In Fax (AREA)

Abstract

本申请实施例公开了一种图像处理方法和图像处理装置，适用于视频图像的处理场景，用于提高图像压缩效率，降低图像编码信息的存储成本。本申请实施例提供一种图像处理方法，包括：获取至少两个变换系数块，所述至少两个变换系数块中的每个变换系数块包括：至少两个变换系数，其中，一个变换系数块中的各个变换系数对应于不同的频率点；按照预设的频率点扫描次序确定一个频率点对应的所述至少两个变换系数块中的变换系数；按照预设的系数扫描次序对所述一个频率点对应的各个变换系数进行编码，以得到图像编码信息。

Description

一种图像处理方法和图像处理装置
本申请要求于2020年3月31日提交中国专利局、申请号为202010246679.8、发明名称为“一种图像处理方法和图像处理装置”的中国专利申请的优先权，其全部内容通过引用结合在本申请中。
技术领域
本申请涉及图像处理邻域，尤其涉及一种图像处理方法和图像处理装置。
背景技术
目前，电子设备中的图像存储以及传输成为必不可少的处理方式，图像压缩在节省传输宽带和存储设备上愈发重要。
应用于图像压缩的国际标准有多种，例如联合图像专家组织(joint photographic experts group，JPEG)、JPEG2000、H.264帧内编码、H.265帧内编码等。例如，JPEG标准广泛应用于视频监控摄像头等硬件设备中，产生的图像文件也能够被电子设备解码后进行播放。
例如，在基于JPEG的图像压缩方案中，编码端首先对图像块进行变换，变换使用的方法包括离散余弦变换(discrete cosine transform，DCT)。图像块经过DCT后，图像块的低频分量都集中在图像的左上角，高频分量分布在图像的右下角，图像块中第一行第一列的系数为直流(direct coefficient，DC)系数，即图像块的平均值，图像块中的其它63个系数是交流(alternate coefficient，AC)系数。然后对DC系数和AC系数进行量化，得到量化后的DC系数和AC系数。接下来对量化后的DC系数和AC系数进行压缩。编码端输出的图像压缩数据被传输至解码端之后，解码端将压缩图像数据进行解码、反量化、反变换后，可以得到重构出的图像块。
对于上述的图像压缩方法，存在图像压缩效率低的问题。另外，上述图像压缩方法产生的压缩文件消耗大量解码端的存储资源，增加了解码端的存储资源开销。
发明内容
本申请实施例提供了一种图像处理方法和图像处理装置，用于提高图像压缩效率，降低图像编码信息的存储成本。
为解决上述技术问题，本申请实施例提供以下技术方案：
第一方面，本申请实施例提供一种图像处理方法，包括：获取至少两个变换系数块，所述至少两个变换系数块中的每个变换系数块包括：至少两个变换系数，其中，一个变换系数块中的各个变换系数对应于不同的频率点；按照预设的频率点扫描次序确定一个频率点对应的所述至少两个变换系数块中的变换系数；按照预设的系数扫描次序对所述一个频率点对应的各个变换系数进行编码，以得到图像编码信息。在本申请实施例中，可以预先设置或者配置频率点扫描次序，基于该频率点扫描次序确定一个频率点，该频率点可以是任意一个频率点，每一个频率点对应于至少两个变换系数块中每个变换系数块中的一个变换系数，即一个频率点对应于多个变换系数，这多个变换系数来源于不同的变换系数块， 可以预先设置或者配置系数扫描次序，基于该系数扫描次序对一个频率点对应的各个变换系数进行编码，最终可以得到图像编码信息。本申请实施例中针对至少两个变换系数块中相同频率点对应的变换系数，只需要按照系数扫描次序进行编码就可以得到图像编码信息，因此针对相同频率点对应的多个变换系数可以实现编码，提高图像压缩效率，且本申请实施例中图像编码信息是按照系数扫描次序对至少两个变换系数块中相同频率点对应的变换系数进行编码得到，编码后的图像编码信息的文件减小，降低图像编码信息的存储成本，最终节省存储资源。
在一种可能的实现方式中，所述按照预设的频率点扫描次序确定一个频率点对应的所述至少两个变换系数块中的变换系数，包括：从所述至少两个变换系数块中获取至少两个二维子带系数阵列，其中，一个二维子带系数阵列包括所述至少两个变换系数块中相同频率点对应的变换系数，不相同的二维子带系数阵列包括的变换系数对应于不同的频率点；按照预设的频率点扫描次序确定所述至少两个二维子带系数阵列中的第一二维子带系数阵列；所述按照预设的系数扫描次序对所述一个频率点对应的各个变换系数进行编码，包括：按照预设的系数扫描次序确定所述第一二维子带系数阵列中的第一变换系数；根据所述第一变换系数对应的上下文区域对所述第一变换系数进行熵编码。在该方案中，编码端可以按照预设的系数扫描次序确定第一二维子带系数阵列中的第一变换系数，第一变换系数可以是第一二维子带系数阵列中的任意一个变换系数，编码端可以分别对第一二维子带系数阵列中的每个变换系数进行无损编码，例如可以对根据第一变换系数对应的上下文区域对第一变换系数进行熵编码。本申请实施例中通过对二维子带系数阵列中的变换系数进行编码，可以得到图像编码信息，实现对图像的压缩处理。
在一种可能的实现方式中，所述按照预设的频率点扫描次序确定一个频率点对应的所述至少两个变换系数块中的变换系数，包括：从所述至少两个变换系数块中获取至少两个二维子带系数阵列，其中，一个二维子带系数阵列包括所述至少两个变换系数块中相同频率点对应的变换系数，不相同的二维子带系数阵列包括的变换系数对应于不同的频率点；按照预设的频率点扫描次序对所述至少两个二维子带系数阵列进行组合，以得到三维子带系数阵列；按照预设的三维子带系数扫描次序确定所述三维子带系数阵列中的第一二维子带系数阵列；所述按照预设的系数扫描次序对所述一个频率点对应的各个变换系数进行编码，包括：按照预设的系数扫描次序确定所述第一二维子带系数阵列中第一变换系数；根据所述第一变换系数对应的上下文区域对所述第一变换系数进行熵编码。在该方案中，三维子带系数扫描次序是对三维子带系数阵列进行扫描的次序。编码端在确定出第一二维子带系数阵列之后，编码端可以按照预设的系数扫描次序分别对第一二维子带系数阵列中的每个变换系数进行熵编码。本申请实施例中通过对二维子带系数阵列中的变换系数进行编码，可以得到图像编码信息，实现对图像的压缩处理。
在一种可能的实现方式中，所述按照预设的频率点扫描次序确定一个频率点对应的所述至少两个变换系数块中的变换系数，包括：按照预设的频率点扫描次序对所述至少两个变换系数块进行排序，得到排序后的至少两个变换系数块；从所述排序后的至少两个变换系数块中获取不同频率点对应的多个二维子带系数阵列；对所述不同频率点对应的多个二维子带系数阵列进行组合，以得到三维子带系数阵列；按照预设的三维子带系数扫描次序 从所述三维子带系数阵列中确定第一二维子带系数阵列；所述按照预设的系数扫描次序对所述一个频率点对应的各个变换系数进行编码，包括：按照预设的系数扫描次序确定所述第一二维子带系数阵列中第一变换系数；根据所述第一变换系数对应的上下文区域对所述第一变换系数进行熵编码。在该方案中，编码端在确定出第一二维子带系数阵列之后，编码端可以按照预设的系数扫描次序分别对第一二维子带系数阵列中的每个变换系数进行熵编码。本申请实施例中通过对二维子带系数阵列中的变换系数进行编码，可以得到图像编码信息，实现对图像的压缩处理。
在一种可能的实现方式中，所述三维子带系数扫描次序，包括：预设的水平位置索引的变化次序；或者，预设的竖直位置索引的变化次序；或者，预设的频率索引的变化次序。在该方案中，三维子带系数阵列中的每个变换系数具有一个水平位置索引、一个竖直位置索引和一个频率索引，例如每个变换系数可以表示为C(bx,by,fs)，其中，bx是水平位置索引，by是竖直位置索引，fs是频率点对应的频率索引。三维子带系数扫描次序是对三维子带系数阵列进行扫描的次序。例如，三维子带系数扫描次序以水平位置索引的变化次序从大到小(或者从小到大)确定三维子带系数阵列中的多个二维子带系数阵列，在每确定出一个二维子带系数阵列之后，再按照前述的系数扫描次序确定该二维子带系数阵列中的一个变换系数，即三维子带系数扫描次序可以用于指示依次对三维子带系数阵列中不同水平位置索引对应的二维子带系数阵列进行编码。
在一种可能的实现方式中，所述按照预设的频率点扫描次序确定一个频率点对应的所述至少两个变换系数块中的变换系数，包括：按照预设的频率点扫描次序确定所述至少两个变换系数块中由相同频率点对应的变换系数构成的第一二维子带系数阵列；所述按照预设的系数扫描次序对所述一个频率点对应的各个变换系数进行编码，包括：按照预设的系数扫描次序确定所述第一二维子带系数阵列中的第一变换系数；根据所述第一变换系数对应的上下文区域对所述第一变换系数进行熵编码。在该方案中，编码端无需组合出三维子带系数阵列。编码端在确定出第一二维子带系数阵列之后，编码端可以按照预设的系数扫描次序分别对第一二维子带系数阵列中的每个变换系数进行编码。例如，编码端按照预设的系数扫描次序确定第一二维子带系数阵列中的第一变换系数，根据第一变换系数对应的上下文区域对第一变换系数进行熵编码。本申请实施例中通过对二维子带系数阵列中的变换系数进行编码，可以得到图像编码信息，实现对图像的压缩处理。
在一种可能的实现方式中，所述按照预设的频率点扫描次序确定一个频率点对应的所述至少两个变换系数块中的变换系数，包括：按照预设的频率点扫描次序确定一个频率子带包括的所述至少两个变换系数块中的变换系数，所述一个频率子带包括的所述至少两个变换系数块中的变换系数对应于相同的频率点；所述按照预设的系数扫描次序对所述一个频率点对应的各个变换系数进行编码，包括：按照预设的系数扫描次序确定所述一个频率子带包括的第一变换系数；根据所述第一变换系数对应的上下文区域对所述第一变换系数进行熵编码。在该方案中，第一变换系数可以是一个频率子带包括的多个变换系数中的任意一个变换系数，根据第一变换系数对应的上下文区域对第一变换系数进行熵编码。本申请实施例中通过对频率子带中的变换系数进行编码，可以得到图像编码信息，实现对图像的压缩处理。
在一种可能的实现方式中，所述根据所述第一变换系数对应的上下文区域对所述第一变换系数进行熵编码，包括：从所述第一变换系数对应的上下文区域中获取所述第一变换系数的上下文信息，其中，所述第一变换系数的上下文信息包括：所述第一变换系数对应的上下文区域中的已编码变换系数；根据所述第一变换系数的上下文信息对所述第一变换系数进行估计，以得到所述第一变换系数的概率分布信息；根据所述第一变换系数的概率分布信息对所述第一变换系数进行算术编码。在该方案中，编码端根据第一变换系数的上下文信息对第一变换系数进行概率估计，以得到第一变换系数的概率分布，然后编码端根据第一变换系数的概率分布对第一变换系数进行算术编码，以得到第一变换系数对应的图像编码信息。
在一种可能的实现方式中，所述第一变换系数对应的上下文区域包括如下至少一种：所述第一变换系数的空间邻域，所述第一变换系数的子带邻域，所述第一变换系数的三维邻域。
在一种可能的实现方式中，所述根据所述第一变换系数对应的上下文区域对所述第一变换系数进行熵编码，包括：当所述第一变换系数为多个变换系数时，从所述多个变换系数对应的上下文区域中获取所述多个变换系数的上下文信息，其中，所述多个变换系数的上下文信息包括：所述多个变换系数对应的上下文区域中的已编码变换系数；根据所述多个变换系数的上下文信息对所述多个变换系数并行的进行估计，以得到所述多个变换系数的概率分布信息；根据所述多个变换系数的概率分布信息对所述多个变换系数并行的进行算术编码。在该方案中，编码端可以获取到多个的变换系数，编码端根据多个的变换系数的上下文信息并行的对多个的变换系数进行熵编码，通过并行的熵编码，可以提高图像编码效率。举例说明如下，编码端在获取系数的概率分布时，也可以对多个变换系数的概率分布并行的进行概率估计，即一次获取多个变换系数的概率分布，然后再按对多个变换系数并行的进行算术编码，提高编码效率。
在一种可能的实现方式中，所述多个变换系数对应的上下文区域包括如下至少一种：所述多个变换系数的空间邻域，所述多个变换系数的子带邻域，所述多个变换系数的三维邻域。
在一种可能的实现方式中，所述按照预设的频率点扫描次序确定一个频率点对应的所述至少两个变换系数块中的变换系数，包括：从所述至少两个变换系数块中获取至少两个二维子带系数阵列，其中，一个二维子带系数阵列包括所述至少两个变换系数块中相同频率点对应的变换系数，不相同的二维子带系数阵列包括的变换系数对应于不同的频率点；按照预设的频率点扫描次序确定所述至少两个二维子带系数阵列对应的至少两个子带图像；所述按照预设的系数扫描次序对所述一个频率点对应的各个变换系数进行编码，包括：按照预设的系数扫描次序对所述至少两个子带图像中的变换系数进行编码。在该方案中，编码端在获取到至少两个二维子带系数阵列之后，编码端采用对子带图像进行无损编码的方式，以至少两个二维子带系数阵列作为至少两个子带图像，按照预设的频率点扫描次序对至少两个子带图像中的变换系数进行图像无损编码，编码端在T恤无损编码时可以将二维子带系数阵列作为一个子带图像进行编码，简化了无损编码的实现流程。
在一种可能的实现方式中，所述获取至少两个变换系数块，包括：获取至少两个图像 块的变换系数编码信息；对所述变换系数编码信息进行熵解码，以得到所述至少两个变换系数块。在该方案中，变换系数编码信息可以是至少两个图像块对应的变换系数编码信息，例如，至少两个图像块可以是JPEG文件中的图像块，图像处理装置可以使用哈夫曼解码方法对JPEG文件中的所有图像块的变换系数编码信息进行熵解码，得到至少两个图像块对应的至少两个变换系数块。本申请实施例中通过对变换系数编码信息进行熵解码，以得到至少两个变换系数块，对该变换系数块进行编码，可以得到图像编码信息，实现对图像的压缩处理。
在一种可能的实现方式中，所述获取至少两个变换系数块，包括：将原始图像区域划分为至少两个图像块，所述原始图像区域包括：像素图像区域或者残差图像区域；对所述至少两个图像块进行变换和量化，以得到所述至少两个变换系数块。在该方案中，通过对图像块的变换和量化可以生成变换系数块，从而可以对该变换系数块进行编码，可以得到图像编码信息，实现对图像的压缩处理。
第二方面，本申请实施例还提供一种图像处理方法，包括：获取图像编码信息；按照预设的频率点扫描次序确定所述图像编码信息中一个频率点对应的待解码变换系数；按照预设的系数扫描次序对所述一个频率点对应的待解码变换系数进行解码，以得到至少两个变换系数块，其中，所述至少两个变换系数块中的每个变换系数块包括：至少两个变换系数，一个变换系数块中的各个变换系数对应于不同的频率点。在本申请实施例中，可以预先设置或者配置频率点扫描次序，基于该频率点扫描次序确定图像编码信息中一个频率点对应的待解码变换系数，该频率点可以是任意一个频率点，可以预先设置系数扫描次序，基于系数扫描次序至少对一个频率点对应的待解码变换系数进行解码，以得到至少两个变换系数块，至少两个变换系数块中的每个变换系数块包括：至少两个变换系数，一个变换系数块中的各个变换系数对应于不同的频率点。本申请实施例中针对图像编码信息中相同频率点对应的待解码变换系数进行无损解码，只需要按照系数扫描次序进行解码就可以得到至少两个变换系数块，因此针对相同频率点对应的多个变换系数可以实现解码，且本申请实施例中图像编码信息是按照系数扫描次序对至少两个变换系数块中相同频率点对应的变换系数进行编码得到，编码后的图像编码信息的文件减小，降低图像编码信息的存储成本，最终节省存储资源。
在一种可能的实现方式中，所述按照预设的频率点扫描次序确定所述图像编码信息中一个频率点对应的待解码变换系数，包括：按照预设的频率点扫描次序确定所述图像编码信息中的第一待解码二维子带系数阵列，所述第一待解码二维子带系数阵列包括：所述图像编码信息中一个频率点对应的待解码变换系数；所述按照预设的系数扫描次序对所述一个频率点对应的待解码变换系数进行解码，包括：按照预设的系数扫描次序确定所述第一待解码二维子带系数阵列中的第一待解码变换系数；根据所述第一待解码变换系数对应的上下文区域对所述第一待解码变换系数进行熵解码。在该方案中，解码端采用与编码端相对应的图像解码方式，若编码端执行步骤C1至C4所示的编码方法，则在解码端收到的图像编码信息中可以包括至少两个待解码二维子带系数阵列。频率点扫描次序是待解码二维子带系数阵列的扫描次序，按照预设的频率点扫描次序从至少两个待解码二维子带系数阵列中获取第一待解码二维子带系数阵列，该第一待解码二维子带系数阵列中包括多个待解 码变换系数。
在一种可能的实现方式中，所述按照预设的频率点扫描次序确定所述图像编码信息中一个频率点对应的待解码变换系数，包括：按照预设的频率点扫描次序确定所述图像编码信息中的待解码三维子带系数阵列，所述待解码三维子带系数阵列包括：所述图像编码信息中所有频率点分别对应的待解码变换系数；按照预设的三维子带系数扫描次序确定所述待解码三维子带系数阵列中的第一待解码二维子带系数阵列，所述第一待解码二维子带系数阵列包括：所述图像编码信息中一个频率点对应的待解码变换系数；所述按照预设的系数扫描次序对所述一个频率点对应的待解码变换系数进行解码，包括：按照预设的系数扫描次序确定所述第一待解码二维子带系数阵列中的第一待解码变换系数；根据所述第一待解码变换系数对应的上下文区域对所述第一待解码变换系数进行熵解码。在该方案中，解码端采用与编码端相对应的图像解码方式，在编码端采用生成三维子带系数阵列的情况下，解码端预设的系数扫描次序对第二二维子带系数阵列进行无损解码，以得到所述至少两个变换系数块。
在一种可能的实现方式中，所述三维子带系数扫描次序，包括：预设的水平位置索引的变化次序；或者，预设的竖直位置索引的变化次序；或者，预设的频率索引的变化次序。
在一种可能的实现方式中，所述按照预设的频率点扫描次序确定所述图像编码信息中一个频率点对应的待解码变换系数，包括：按照预设的频率点扫描次序确定所述图像编码信息中一个频率子带包括的待解码变换系数，所述一个频率子带包括的待解码变换系数对应于相同的频率点；所述按照预设的系数扫描次序对所述一个频率点对应的待解码变换系数进行解码，包括：按照预设的系数扫描次序确定所述一个频率子带中的第一待解码变换系数；根据所述第一待解码变换系数对应的上下文区域对所述第一待解码变换系数进行熵解码。在该方案中，解码端获取多个频率子带对应的图像编码信息，解码端采用对频率子带进行无损解码的方式，以至少两个二维子带系数阵列作为至少两个频率子带，按照预设的系数扫描次序对至少两个频率子带对应的图像编码信息进行无损解码，解码端在无损解码时可以将二维子带系数阵列作为一个频率子带进行解码，简化了无损解码的实现流程。
在一种可能的实现方式中，所述根据所述第一待解码变换系数对应的上下文区域对所述第一待解码变换系数进行熵解码，包括：从所述第一待解码变换系数对应的上下文区域中获取所述第一待解码变换系数的上下文信息，其中，所述第一待解码变换系数对应的上下文区域包括：所述第一待解码变换系数对应的上下文区域中的已解码变换系数；根据所述第一待解码变换系数的上下文信息对所述第一待解码变换系数进行估计，以得到所述第一待解码变换系数的概率分布信息；根据所述第一待解码变换系数的概率分布信息对所述第一待解码变换系数进行算术解码，以得到解码后的第一变换系数，所述解码后的第一变换系数属于所述至少两个变换系数块。在该方案中，解码端在获取到第一变换系数的上下文信息之后，解码端根据第一变换系数的上下文信息对第一变换系数进行熵解码，以得到解码后的第一变换系数。解码端执行的熵解码可以包括概率估计和算术解码。具体的，解码端根据第一变换系数的上下文信息对第一变换系数进行概率估计，以得到第一变换系数的概率分布，然后解码端根据第一变换系数的概率分布对第一变换系数进行算术解码，以得到第一变换系数对应的图像解码信息。
在一种可能的实现方式中，所述第一待解码变换系数对应的上下文区域包括如下至少一种：所述第一待解码变换系数的空间邻域，所述第一待解码变换系数的子带邻域，所述第一待解码变换系数的三维邻域。
在一种可能的实现方式中，所述根据所述第一待解码变换系数对应的上下文区域对所述第一待解码变换系数进行熵解码，包括：当所述第一待解码变换系数为多个待解码变换系数时，从所述多个待解码变换系数对应的上下文区域中获取所述多个待解码变换系数的上下文信息，其中，所述多个待解码的上下文信息包括：所述多个待解码变换系数对应的上下文区域中的已解码变换系数；根据所述多个待解码变换系数的上下文信息对所述多个待解码变换系数并行的进行估计，以得到所述第一待解码变换系数的概率分布信息；根据所述多个待解码变换系数的概率分布信息对所述多个待解码变换系数并行的进行算术解码，以得到解码后的第一变换系数，所述解码后的第一变换系数属于所述至少两个变换系数块。在该方案中，解码端采用与编码端相对应的图像解码方式，编码端在编码时可以并行的进行编码，则在解码端通过并行的解码，可以提高图像解码效率。
在一种可能的实现方式中，所述多个待解码变换系数对应的上下文区域包括如下至少一种：所述多个待解码变换系数的空间邻域，所述多个待解码变换系数的子带邻域，所述多个待解码变换系数的三维邻域。
在一种可能的实现方式中，所述按照预设的频率点扫描次序确定所述图像编码信息中一个频率点对应的待解码变换系数，包括：按照预设的频率点扫描次序确定所述图像编码信息中的第一子带图像，所述第一子带图像包括：所述图像编码信息中的一个频率点对应的待解码变换系数；所述按照预设的系数扫描次序对所述一个频率点对应的待解码变换系数进行解码，包括：按照预设的系数扫描次序对所述第一子带图像的待解码系数进行解码，以得到所述至少两个变换系数块。在该方案中，解码端采用针对子带图像无损解码，得到多个变换系数之后，解码端采用对子带图像进行无损解码的方式，以至少两个二维子带系数阵列作为至少两个子带图像，按照预设的系数扫描次序对至少两个子带图像对应的图像编码信息进行无损解码，解码端在无损解码时可以将二维子带系数阵列作为一个子带图像进行解码，简化了无损解码的实现流程。
在一种可能的实现方式中，所述得到至少两个变换系数块之后，所述方法还包括：对所述至少两个变换系数块进行熵编码，以得到变换系数编码信息；根据所述变换系数编码信息生成至少两个图像块；根据所述至少两个图像块还原出原始图像区域，所述原始图像区域包括：像素图像区域或者残差图像区域。在该方案中，解码端采用与编码端相对应的图像解码方式，若编码端通过对变换系数编码信息进行熵解码得到变换系数块，并对变换系数块进行编码，则解码端可以对至少两个变换系数块进行熵编码，以得到变换系数编码信息。本申请实施例中根据变换系数编码信息生成至少两个图像块，再根据至少两个图像块还原出原始图像区域，从而可以完成图像的解压缩处理。
在一种可能的实现方式中，所述得到至少两个变换系数块之后，所述方法还包括：对所述至少两个变换系数块进行反量化和反变换，以得到至少两个图像块；根据所述至少两个图像块还原出原始图像区域，所述原始图像区域包括：像素图像区域或者残差图像区域。在该方案中，解码端在生成至少两个变换系数块之后，解码端根据该至少两个变换系数块 生成至少两个图像块。其中，解码端可以通过反变换和反量化的方式对至少两个变换系数块进行处理，从而可以生成至少两个图像块。
第三方面，本申请实施例提供一种图像处理装置，包括：收发模块，用于获取至少两个变换系数块，所述至少两个变换系数块中的每个变换系数块包括：至少两个变换系数，其中，一个变换系数块中的各个变换系数对应于不同的频率点；处理模块，用于按照预设的频率点扫描次序确定一个频率点对应的所述至少两个变换系数块中的变换系数；按照预设的系数扫描次序对所述一个频率点对应的各个变换系数进行编码，以得到图像编码信息。在本申请实施例中，可以预先设置或者配置频率点扫描次序，基于该频率点扫描次序确定一个频率点，该频率点可以是任意一个频率点，每一个频率点对应于至少两个变换系数块中每个变换系数块中的一个变换系数，即一个频率点对应于多个变换系数，这多个变换系数来源于不同的变换系数块，可以预先设置或者配置系数扫描次序，基于该系数扫描次序对一个频率点对应的各个变换系数进行编码，最终可以得到图像编码信息。本申请实施例中针对至少两个变换系数块中相同频率点对应的变换系数，只需要按照系数扫描次序进行编码就可以得到图像编码信息，因此针对相同频率点对应的多个变换系数可以实现编码，提高图像压缩效率，且本申请实施例中图像编码信息是按照系数扫描次序对至少两个变换系数块中相同频率点对应的变换系数进行编码得到，编码后的图像编码信息的文件减小，降低图像编码信息的存储成本，最终节省存储资源。
在一种可能的实现方式中，所述处理模块，用于从所述至少两个变换系数块中获取至少两个二维子带系数阵列，其中，一个二维子带系数阵列包括所述至少两个变换系数块中相同频率点对应的变换系数，不相同的二维子带系数阵列包括的变换系数对应于不同的频率点；按照预设的频率点扫描次序确定所述至少两个二维子带系数阵列中的第一二维子带系数阵列；按照预设的系数扫描次序确定所述第一二维子带系数阵列中的第一变换系数；根据所述第一变换系数对应的上下文区域对所述第一变换系数进行熵编码。
在一种可能的实现方式中，所述处理模块，用于从所述至少两个变换系数块中获取至少两个二维子带系数阵列，其中，一个二维子带系数阵列包括所述至少两个变换系数块中相同频率点对应的变换系数，不相同的二维子带系数阵列包括的变换系数对应于不同的频率点；按照预设的频率点扫描次序对所述至少两个二维子带系数阵列进行组合，以得到三维子带系数阵列；按照预设的三维子带系数扫描次序确定所述三维子带系数阵列中的第一二维子带系数阵列；按照预设的系数扫描次序确定所述第一二维子带系数阵列中第一变换系数；根据所述第一变换系数对应的上下文区域对所述第一变换系数进行熵编码。
在一种可能的实现方式中，所述处理模块，用于按照预设的频率点扫描次序对所述至少两个变换系数块进行排序，得到排序后的至少两个变换系数块；从所述排序后的至少两个变换系数块中获取不同频率点对应的多个二维子带系数阵列；对所述不同频率点对应的多个二维子带系数阵列进行组合，以得到三维子带系数阵列；按照预设的三维子带系数扫描次序从所述三维子带系数阵列中确定第一二维子带系数阵列；按照预设的系数扫描次序确定所述第一二维子带系数阵列中第一变换系数；根据所述第一变换系数对应的上下文区域对所述第一变换系数进行熵编码。
在一种可能的实现方式中，所述三维子带系数扫描次序，包括：预设的水平位置索引 的变化次序；或者，预设的竖直位置索引的变化次序；或者，预设的频率索引的变化次序。
在一种可能的实现方式中，所述处理模块，用于按照预设的频率点扫描次序确定所述至少两个变换系数块中由相同频率点对应的变换系数构成的第一二维子带系数阵列；按照预设的系数扫描次序确定所述第一二维子带系数阵列中的第一变换系数；根据所述第一变换系数对应的上下文区域对所述第一变换系数进行熵编码。
在一种可能的实现方式中，所述处理模块，用于按照预设的频率点扫描次序确定一个频率子带包括的所述至少两个变换系数块中的变换系数，所述一个频率子带包括的所述至少两个变换系数块中的变换系数对应于相同的频率点；按照预设的系数扫描次序确定所述一个频率子带包括的第一变换系数；根据所述第一变换系数对应的上下文区域对所述第一变换系数进行熵编码。
在一种可能的实现方式中，所述处理模块，用于从所述第一变换系数对应的上下文区域中获取所述第一变换系数的上下文信息，其中，所述第一变换系数的上下文信息包括：所述第一变换系数对应的上下文区域中的已编码变换系数；根据所述第一变换系数的上下文信息对所述第一变换系数进行估计，以得到所述第一变换系数的概率分布信息；根据所述第一变换系数的概率分布信息对所述第一变换系数进行算术编码。
在一种可能的实现方式中，所述第一变换系数对应的上下文区域包括如下至少一种：所述第一变换系数的空间邻域，所述第一变换系数的子带邻域，所述第一变换系数的三维邻域。
在一种可能的实现方式中，所述处理模块，用于当所述第一变换系数为多个变换系数时，从所述多个变换系数对应的上下文区域中获取所述多个变换系数的上下文信息，其中，所述多个变换系数的上下文信息包括：所述多个变换系数对应的上下文区域中的已编码变换系数；根据所述多个变换系数的上下文信息对所述多个变换系数并行的进行估计，以得到所述多个变换系数的概率分布信息；根据所述多个变换系数的概率分布信息对所述多个变换系数并行的进行算术编码。
在一种可能的实现方式中，所述多个变换系数对应的上下文区域包括如下至少一种：所述多个变换系数的空间邻域，所述多个变换系数的子带邻域，所述多个变换系数的三维邻域。
在一种可能的实现方式中，所述处理模块，用于从所述至少两个变换系数块中获取至少两个二维子带系数阵列，其中，一个二维子带系数阵列包括所述至少两个变换系数块中相同频率点对应的变换系数，不相同的二维子带系数阵列包括的变换系数对应于不同的频率点；按照预设的频率点扫描次序确定所述至少两个二维子带系数阵列对应的至少两个子带图像；按照预设的系数扫描次序对所述至少两个子带图像中的变换系数进行编码。
在一种可能的实现方式中，所述收发模块，用于获取至少两个图像块的变换系数编码信息；对所述变换系数编码信息进行熵解码，以得到所述至少两个变换系数块。
在一种可能的实现方式中，所述收发模块，用于将原始图像区域划分为至少两个图像块，所述原始图像区域包括：像素图像区域或者残差图像区域；对所述至少两个图像块进行变换和量化，以得到所述至少两个变换系数块。
在本申请的第三方面中，图像处理装置的组成模块还可以执行前述第一方面以及各种 可能的实现方式中所描述的步骤，详见前述对第一方面以及各种可能的实现方式中的说明。
第四方面，本申请实施例还提供一种图像处理装置，包括：收发模块，用于获取图像编码信息；处理模块，用于按照预设的频率点扫描次序确定所述图像编码信息中一个频率点对应的待解码变换系数；按照预设的系数扫描次序对所述一个频率点对应的待解码变换系数进行解码，以得到至少两个变换系数块，其中，所述至少两个变换系数块中的每个变换系数块包括：至少两个变换系数，一个变换系数块中的各个变换系数对应于不同的频率点。
在一种可能的实现方式中，所述处理模块，用于按照预设的频率点扫描次序确定所述图像编码信息中的第一待解码二维子带系数阵列，所述第一待解码二维子带系数阵列包括：所述图像编码信息中一个频率点对应的待解码变换系数；按照预设的系数扫描次序确定所述第一待解码二维子带系数阵列中的第一待解码变换系数；根据所述第一待解码变换系数对应的上下文区域对所述第一待解码变换系数进行熵解码。
在一种可能的实现方式中，所述处理模块，用于按照预设的频率点扫描次序确定所述图像编码信息中的待解码三维子带系数阵列，所述待解码三维子带系数阵列包括：所述图像编码信息中所有频率点分别对应的待解码变换系数；按照预设的三维子带系数扫描次序确定所述待解码三维子带系数阵列中的第一待解码二维子带系数阵列，所述第一待解码二维子带系数阵列包括：所述图像编码信息中一个频率点对应的待解码变换系数；按照预设的系数扫描次序确定所述第一待解码二维子带系数阵列中的第一待解码变换系数；根据所述第一待解码变换系数对应的上下文区域对所述第一待解码变换系数进行熵解码。
在一种可能的实现方式中，所述三维子带系数扫描次序，包括：预设的水平位置索引的变化次序；或者，预设的竖直位置索引的变化次序；或者，预设的频率索引的变化次序。
在一种可能的实现方式中，所述处理模块，用于按照预设的频率点扫描次序确定所述图像编码信息中一个频率子带包括的待解码变换系数，所述一个频率子带包括的待解码变换系数对应于相同的频率点；按照预设的系数扫描次序确定所述一个频率子带中的第一待解码变换系数；根据所述第一待解码变换系数对应的上下文区域对所述第一待解码变换系数进行熵解码。
在一种可能的实现方式中，所述处理模块，用于从所述第一待解码变换系数对应的上下文区域中获取所述第一待解码变换系数的上下文信息，其中，所述第一待解码变换系数对应的上下文区域包括：所述第一待解码变换系数对应的上下文区域中的已解码变换系数；根据所述第一待解码变换系数的上下文信息对所述第一待解码变换系数进行估计，以得到所述第一待解码变换系数的概率分布信息；根据所述第一待解码变换系数的概率分布信息对所述第一待解码变换系数进行算术解码，以得到解码后的第一变换系数，所述解码后的第一变换系数属于所述至少两个变换系数块。
在一种可能的实现方式中，所述第一待解码变换系数对应的上下文区域包括如下至少一种：所述第一待解码变换系数的空间邻域，所述第一待解码变换系数的子带邻域，所述第一待解码变换系数的三维邻域。
在一种可能的实现方式中，所述处理模块，用于当所述第一待解码变换系数为多个待解码变换系数时，从所述多个待解码变换系数对应的上下文区域中获取所述多个待解码变 换系数的上下文信息，其中，所述多个待解码的上下文信息包括：所述多个待解码变换系数对应的上下文区域中的已解码变换系数；根据所述多个待解码变换系数的上下文信息对所述多个待解码变换系数并行的进行估计，以得到所述第一待解码变换系数的概率分布信息；根据所述多个待解码变换系数的概率分布信息对所述多个待解码变换系数并行的进行算术解码，以得到解码后的第一变换系数，所述解码后的第一变换系数属于所述至少两个变换系数块。
在一种可能的实现方式中，所述多个待解码变换系数对应的上下文区域包括如下至少一种：所述多个待解码变换系数的空间邻域，所述多个待解码变换系数的子带邻域，所述多个待解码变换系数的三维邻域。
在一种可能的实现方式中，所述处理模块，用于按照预设的频率点扫描次序确定所述图像编码信息中的第一子带图像，所述第一子带图像包括：所述图像编码信息中的一个频率点对应的待解码变换系数；按照预设的系数扫描次序对所述第一子带图像的待解码系数进行解码，以得到所述至少两个变换系数块。
在一种可能的实现方式中，所述处理模块，还用于得到至少两个变换系数块之后，对所述至少两个变换系数块进行熵编码，以得到变换系数编码信息；根据所述变换系数编码信息生成至少两个图像块；根据所述至少两个图像块还原出原始图像区域，所述原始图像区域包括：像素图像区域或者残差图像区域。
在一种可能的实现方式中，所述处理模块，还用于得到至少两个变换系数块之后，对所述至少两个变换系数块进行反量化和反变换，以得到至少两个图像块；根据所述至少两个图像块还原出原始图像区域，所述原始图像区域包括：像素图像区域或者残差图像区域。
在本申请的第四方面中，图像处理装置的组成模块还可以执行前述第二方面以及各种可能的实现方式中所描述的步骤，详见前述对第二方面以及各种可能的实现方式中的说明。
第五方面，本申请实施例提供了一种计算机可读存储介质，所述计算机可读存储介质中存储有指令，当其在计算机上运行时，使得计算机执行上述第一方面或第二方面所述的方法。
第六方面，本申请实施例提供了一种包含指令的计算机程序产品，当其在计算机上运行时，使得计算机执行上述第一方面或第二方面所述的方法。
第七方面，本申请实施例提供一种图像处理装置，该图像处理装置可以包括图像编码装置或者图像解码装置或者芯片等实体，所述图像处理装置包括：处理器。该图像处理还可以包括存储器；所述存储器用于存储指令；所述处理器用于执行所述存储器中的所述指令，使得所述图像处理装置执行如前述第一方面或第二方面中任一项所述的方法。
第八方面，本申请实施例提供了一种计算机可读存储介质，所述计算机可读存储介质中存储有指令，当其在计算机上运行时，使得计算机执行上述第一方面或第二方面所述的方法。
第九方面，本申请实施例提供了一种包含指令的计算机程序产品，当其在计算机上运行时，使得计算机执行上述第一方面或第二方面所述的方法。
第十方面，本申请提供了一种芯片系统，该芯片系统包括处理器，用于支持图像编码装置或者图像解码装置实现上述第一方面或第二方面中所涉及的功能，例如，发送或处理 上述方法中所涉及的数据和/或信息。在一种可能的设计中，所述芯片系统还包括存储器，所述存储器，用于保存图像编码装置或者图像解码装置必要的程序指令和数据。该芯片系统，可以由芯片构成，也可以包括芯片和其他分立器件。
附图说明
图1为本申请实施例提供的图像处理系统的组成结构示意图；
图2为本申请实施例提供的图像编码装置和图像解码装置应用于终端设备的示意图；
图3为本申请实施例提供的图像编码装置和图像解码装置之间的一种交互流程示意图；
图4为本申请实施例提供的一种图像编码流程的示意图；
图5为本申请实施例提供的获取变换系数块的示意图；
图6为本申请实施例提供的获取二维子带系数阵列的示意图；
图7为本申请实施例提供的变换系数块的示意图；
图8为本申请实施例提供的一种图像解码流程的示意图；
图9为本申请实施例提供的二维子带系数阵列和变换系数块之间的对应关系的示意图；
图10为本申请实施例提供的一种图像编码流程的示意图；
图11a为本申请实施例提供的三维子带系数阵列的示意图；
图11b为本申请实施例提供的变换系数的三维邻域的示意图；
图12为本申请实施例提供的一种图像解码流程的示意图；
图13为本申请实施例提供的一种图像编码流程的示意图；
图14为本申请实施例提供的一种图像编码流程的示意图；
图15为本申请实施例提供的一种图像编码流程的示意图；
图16为本申请实施例提供的一种图像解码流程的示意图；
图17为本申请实施例提供的一种图像编码装置的组成结构示意图；
图18为本申请实施例提供的一种图像解码装置的组成结构示意图；
图19为本申请实施例提供的另一种图像编码装置的组成结构示意图；
图20为本申请实施例提供的另一种图像解码装置的组成结构示意图。
具体实施方式
本申请实施例提供了一种图像处理方法和图像处理装置，用于提高图像压缩效率，降低图像编码信息的存储成本。
下面结合附图，对本申请的实施例进行描述。
本申请的说明书和权利要求书及上述附图中的术语“第一”、“第二”等是用于区别类似的对象，而不必用于描述特定的顺序或先后次序。应该理解这样使用的术语在适当情况下可以互换，这仅仅是描述本申请的实施例中对相同属性的对象在描述时所采用的区分方式。此外，术语“包括”和“具有”以及他们的任何变形，意图在于覆盖不排他的包含，以便包含一系列单元的过程、方法、系统、产品或设备不必限于那些单元，而是可包括没 有清楚地列出的或对于这些过程、方法、产品或设备固有的其它单元。
本申请实施例的技术方案可以应用于各种的图像处理系统，如图1所示，为本申请实施例提供的图像处理系统的组成结构示意图。图像处理系统100可以包括：图像编码装置101和图像解码装置102。其中，图像编码装置101可用于根据至少两个变换系数块生成图像编码信息，例如，至少两个变换系数块可以根据原始图像区域对应的至少两个图像块生成。其中，原始图像区域可以是整幅图像或一个矩形图像区域，或者其它形状的图像区域，此次不做限定，例如原始图像区域包括：像素图像区域或者残差图像区域。另外，图像编码信息可以是图像编码码流(也可以称为压缩码流)，然后该图像编码信息可以通过图像传输通道传输给图像解码装置102，图像解码装置102可以接收到图像编码信息，然后图像解码装置102可以对该图像编码信息进行解码，得到至少两个变换系数块。例如，至少两个变换系数块还可以用于还原出原始图像。
在本申请的实施例中，该图像编码装置可以应用于各种有图像通信需要的终端设备、有图像转码需要的无线设备与核心网设备，例如图像编码装置可以是上述终端设备或者无线设备或者核心网设备的图像编码器。同样的，该图像解码装置可以应用于各种有图像通信需要的终端设备、有图像转码需要的无线设备与核心网设备，例如图像解码装置可以是上述终端设备或者无线设备或者核心网设备的图像解码器。
例如，图像编码装置具体图像编码器，图像解码装置具体为图像解码器。终端设备可以包括：图像编码器和信道编码器，或者终端设备可以包括图像解码器和信道解码器。具体的，信道编码器用于对图像信号进行信道编码，信道解码器用于对图像信号进行信道解码，图像编码器用于对原始图像区域进行编码，例如图像编码器具有无损编码功能。图像解码器用于对图像编码信息进行解码，例如图像解码器具有无损解码功能。例如可使用的无损编码方法包括熵编码、哈夫曼(Huffman)编码等。例如可使用的无损解码方法包括熵解码、哈夫曼解码等。
如图2所示，为本申请实施例提供的图像编码装置和图像解码装置应用于终端设备的示意图。在第一终端设备20中可以包括：图像编码器201和信道编码器202。在第二终端设备21中可以包括：图像解码器211和信道解码器212。第一终端设备20连接无线或者有线的第一网络通信设备22，第一网络通信设备22和无线或者有线的第二网络通信设备23之间通过数字信道连接，第二终端设备21连接无线或者有线的第二网络通信设备23。其中，上述无线或者有线的网络通信设备可以泛指信号传输设备，例如通信基站，数据交换设备等。
在图像通信中，作为发送端的终端设备对采集到的图像信号进行图像编码，再进行信道编码后，通过无线网络或者核心网进行在数字信道中传输。而作为接收端的终端设备根据接收到的信号进行信道解码，以得到图像信号编码码流，然后经过图像解码恢复出图像信号，由接收端的终端设备进回放。
本申请实施例提供针对原始图像区域对应的变换系数块的编码，提高图像编码性能，最终节省存储资源，进一步提高图像的编码效率。例如，目前的熵编码产生的压缩文件消耗大量服务器侧的存储资源，本申请实施例可应用于熵编码图像的高效转码。本申请实施例还可以应用于服务器端，对熵编码产生的压缩文件进行转码，提高图像编码性能，最终 节省存储资源。
接下来基于前述的系统架构以及图像编码装置和图像解码装置，对本申请实施例提供的图像处理方法进行说明。如图3所示，为本申请实施例中图像编码装置和图像解码装置之间的一种交互流程示意图，其中，下述步骤301至步骤303可以由图像编码装置(如下简称编码端)执行，下述步骤311至步骤313可以由图像解码装置(如下简称解码端)执行，主要包括如下过程：
301、获取至少两个变换系数块，至少两个变换系数块中的每个变换系数块包括：至少两个变换系数，其中，一个变换系数块中的各个变换系数对应于不同的频率点。
其中，至少两个变换系数块中的每个变换系数块包括：至少两个变换系数，一个变换系数块中的各个变换系数对应于不同的频率点，即对于每一个变换系数块，该变换系数块中的所有变换系数都具有不同的频率点，对于每个变换系数都包括频率点信息，每个变换系数除了包括频率点，还可以包括位置信息。
对于一个变换系数的频率点可以用表示，fx和fy表示频率点对应的二维频率坐标取值，一个变换系数的位置信息可以用(bx,by)表示，bx和by表示变换系数的二维位置坐标取值，变换系数块中的每个变换系数可以用C_(bx,by)^(fx,fy)表示。另外，还可以为不同的频率点设置频率索引，例如频率索引为。
在本申请的一些实施例中，步骤301获取至少两个变换系数块，包括：
A1、将原始图像区域划分为至少两个图像块，原始图像区域包括：像素图像区域或者残差图像区域；
A2、对至少两个图像块进行变换和量化，以得到至少两个变换系数块。
其中，原始图像区域(也可以简称为原始图像)可以是整幅图像或一个矩形图像区域，或者其它形状的图像区域。该原始图像区域是待编码(或者称为待处理)的图像。编码端将待处理图像或者图像区域划分为相同预置大小的至少两个图像块，例如，预置大小的子图像块的尺寸可以是4×4、8×8、16×16、32×32、64×64、128×128和256×256等。本申请实施例中图像块是通过对原始图像进行划分得到的，或者该图像块是通过编码端的外接设备传输给编码端得到。其中，图像块也可以称为子图像，或者子图像块等。
需要说明的是，本申请实施例中，图像块包括的图像数据可以包括亮度分量、色度分量，R分量、G分量、B分量等。
在本申请实施例中，编码端将待处理的原始图像区域划分为预置大小的图像块之后，可以对每个图像块进行变换和量化，以得到至少两个变换系数块。例如，编码端获取到至少两个图像块之后，编码端根据至少两个图像块获取至少两个变换系数块，其中，每个图像块可以生成一个变换系数块，图像块和变换系数块是一一对应的。变换系数块是多个变换系数组成的数据块，例如变换系数块可以通过对图像块进行离散余弦变换(discrete cosine transform，DCT)得到。例如，在DCT变换中，f_m表示的是某个频率点(例如频率点m)对应的变换系数，k＝0，…，n-1，k表示的是像素点的位置，xk表示k位置的像素值，
Figure PCTCN2021082657-appb-000001
本申请实施例中通过对图像块的变换和量化可以生成变换系数块，从而可以对该变换系数块进行编码，可以得到图像编码信息，实现对图像的压缩处理。
在本申请的一些实施例中，步骤301获取至少两个变换系数块，包括：
B1、获取至少两个图像块的变换系数编码信息；
B2、对变换系数编码信息进行熵解码，以得到至少两个变换系数块。
其中，变换系数编码信息可以是至少两个图像块对应的变换系数编码信息，例如，至少两个图像块可以是JPEG文件中的图像块，图像处理装置可以使用哈夫曼解码方法对JPEG文件中的所有图像块的变换系数编码信息进行熵解码，得到至少两个图像块对应的至少两个变换系数块。本申请实施例中通过对变换系数编码信息进行熵解码，以得到至少两个变换系数块，对该变换系数块进行编码，可以得到图像编码信息，实现对图像的压缩处理。
302、按照预设的频率点扫描次序确定一个频率点对应的至少两个变换系数块中的变换系数。
303、按照预设的系数扫描次序对一个频率点对应的各个变换系数进行编码，以得到图像编码信息。
在本申请实施例中，频率点扫描次序可以是在编码端中预设的或者预先配置的，频率点扫描次序是指扫描频率点的先后顺序，至少两个变换系数块中的每个变换系数块包括：至少两个变换系数，其中，一个变换系数块中的各个变换系数对应于不同的频率点，因此针对所有的频率点可以设置频率点扫描次序，通过该频率点扫描次序确定出一个频率点，同样的通过该频率点扫描次序可以依次确定出不同的频率点，步骤302和步骤303中以针对一个频率点的编码处理流程为例，对于其他频率点的编码处理流程都可以基于步骤302和步骤303以及针对步骤302和步骤303包括的详细流程来实现。
本申请实施例中，至少两个变换系数块可以是N个变换系数块，一个频率点对应的至少两个变换系数块中的变换系数可以是一个频率点对应于N个变换系数块中的每一个变换系数块中的一个变换系数，则一个频率点对应于N个变换系数，这N个变换系数分别来源于N个变换系数块中的每一个变换系数块中，N的取值可以为正整数，例如N为大于或等于2的整数。
在本申请实施例中，系数扫描次序可以是在编码端中预设的或者预先配置的，系数扫描次序是指变换系数的扫描次序，按照预设的系数扫描次确定一个频率点对应的一个变换系数，再对确定出的该变换系数进行编码，基于步骤302中的系数扫描次序可以依次确定出一个频率点对应的其他变换系数，再对依次确定出的变换系数进行编码，最终可以图像编码信息。
在本申请实施例中，编码端获取到至少两个变换系数块之后，对于至少两个变换系数块中相同频率点对应的多个变换系数可以进行编码。例如可使用的无损编码方法包括熵编码、哈夫曼(Huffman)编码等。不限定的是，本申请实施例中熵编码采用的是深度网络，对于深度网络的结构不做限定。
在某一个频率点对应的所有变换系数编码完成之后，按照预设的频率点扫描次序确定下一个频率点，再针对该下一个频率点对应的多个变换系数进行编码，按照该频率点扫描次序依次获取到所有的频率点，当所有频率点对应的变换系数都完成无损编码之后，编码端可以生成图像编码信息。编码端生成图像编码信息之后，编码端可以将图像编码信息写入码流，然后编码端向解码端发送该码流。
其中，频率点扫描次序是编码时选择频率点的先后顺序，频率点扫描次序在后续实施例中简称为“预置次序”或者“预置次序一”。频率点扫描次序有多种实现方式，例如可以是按照之字(zig-zag)扫描得到的处理次序，还可以使用其他扫描方法得到处理次序，例如逐行扫描、逐列扫描、对角扫描、水平反向扫描和垂直反向扫描等。其中逐行扫描是指从图像的第一行一直连续扫描到最后一行，逐列扫描是指从图像的第一列一直连续扫描到最后一列。对角扫描是指从图像的第一行按照对角方式连续扫描到最后一行。水平反向扫描是指从图像的最后一行按照水平方向一直连续扫描到第一行，垂直反向扫描是指从图像的最后一行按照垂直方向一直连续扫描到第一行。系数扫描次序是编码时选择变换系数的先后顺序，系数扫描次序在后续实施例中简称为“预置次序”或者“预置次序二”。系数扫描次序有多种实现方式，例如可以是按照之字(zig-zag)扫描得到的处理次序，还可以使用其他扫描方法得到处理次序，例如逐行扫描、逐列扫描、对角扫描、水平反向扫描和垂直反向扫描等。
其中，频率点扫描次序和系数扫描次序为相同的图像扫描次序；或者，频率点扫描次序和系数扫描次序为不相同的图像扫描次序，此处不做限定。
举例说明如下，先按照频率点扫描次序获取频率点(fx＝0,fy＝0)对应的所有变换系数，接下来按照系数扫描次序获取频率点(fx＝0,fy＝0)对应的一个变换系数，对该变换系数进行编码，然后按照系数扫描次序获取频率点(fx＝0,fy＝0)对应的下一个变换系数，对该下一个变换系数进行编码，直至按照系数扫描次序编码完频率点(fx＝0,fy＝0)对应的所有变换系数。对频率点(fx＝0,fy＝0)对应的所有变换系数编码完成之后，接下来再获取频率点(fx＝0,fy＝1)对应的所有变换系数，对频率点(fx＝0,fy＝1)对应的所有变换系数进行编码，按照频率点扫描次序依次确定出所有频率点对应的变换系数进行编码，当所有频率点对应的变换系数都完成编码之后，编码端可以输出图像编码信息。
在本申请的一些实施例中，步骤302按照预设的频率点扫描次序确定一个频率点对应的所述至少两个变换系数块中的变换系数，包括：
C1、从至少两个变换系数块中获取至少两个二维子带系数阵列，其中，一个二维子带系数阵列包括至少两个变换系数块中相同频率点对应的变换系数，不相同的二维子带系数阵列包括的变换系数对应于不同的频率点。
C2、按照预设的频率点扫描次序确定至少两个二维子带系数阵列中的第一二维子带系数阵列。
步骤303按照预设的系数扫描次序对一个频率点对应的各个变换系数进行编码，包括：
C3、按照预设的系数扫描次序确定第一二维子带系数阵列中的第一变换系数；
C4、根据第一变换系数对应的上下文区域对第一变换系数进行熵编码。
其中，编码端可以根据频率点的不同对至少两个变换系数块进行排列，从而得到至少两个二维子带系数阵列。同一个二维子带系数阵列中包括相同频率点对应的变换系数块中的变换系数，不同的二维子带系数阵列具有不同的频率点，即编码端可以将相同频率点对应的至少两个变换系数块中的变换系数归属到一个二维子带系数阵列中，因此存在多少个频率点，就可以得到多个频率点对应的二维子带系数阵列。
举例说明如下，获取图像中8x8的变换系数块中指定频率点的变换系数值，构成该频 率点的二维子带系数阵列。在获取图像中所有8x8的变换系数块中频率点的变换系数值得到的二维子带系数阵列之后，同样的，获取剩余的频率点的二维子带系数阵列。一个8x8的变换系数块共有64个频率点，因此可获得当前编码图像的64个二维子带系数阵列。
编码端将至少两个变换系数块按照不同的频率点对变换系数进行组合，可以得到为至少两个二维子带系数阵列，按照预设的频率点扫描次序从至少两个二维子带系数阵列中获取第一二维子带系数阵列，第一二维子带系数阵列为至少两个二维子带系数阵列中的一个二维子带系数阵列，该第一二维子带系数阵列是按照频率点扫描次序得到的一个二维子带系数阵列，该第一二维子带系数阵列中包括多个变换系数，编码端可以按照预设的系数扫描次序确定第一二维子带系数阵列中的第一变换系数，第一变换系数可以是第一二维子带系数阵列中的任意一个变换系数，编码端可以分别对第一二维子带系数阵列中的每个变换系数进行无损编码，例如可以对根据第一变换系数对应的上下文区域对第一变换系数进行熵编码。本申请实施例中通过对二维子带系数阵列中的变换系数进行编码，可以得到图像编码信息，实现对图像的压缩处理。
其中，频率点扫描次序可以是扫描二维子带系数阵列的先后次序，系数扫描次序是二维子带系数阵列中的变换系数的扫描次序。频率点扫描次序在后续实施例中简称为“预置次序一”，系数扫描次序在后续实施例中简称为“预置次序二”。
在本申请的一些实施例中，步骤302按照预设的频率点扫描次序确定一个频率点对应的所述至少两个变换系数块中的变换系数，包括：
D1、从至少两个变换系数块中获取至少两个二维子带系数阵列，其中，一个二维子带系数阵列包括至少两个变换系数块中相同频率点对应的变换系数，不相同的二维子带系数阵列包括的变换系数对应于不同的频率点；
D2、按照预设的频率点扫描次序对至少两个二维子带系数阵列进行组合，以得到三维子带系数阵列；
D3、按照预设的三维子带系数扫描次序确定三维子带系数阵列中的第一二维子带系数阵列。
步骤303按照预设的系数扫描次序对一个频率点对应的各个变换系数进行编码，包括：
D4、按照预设的系数扫描次序确定第一二维子带系数阵列中第一变换系数；
D5、根据第一变换系数对应的上下文区域对第一变换系数进行熵编码。
其中，步骤D1至D5的场景与前述步骤C1至C4的场景的区别在于，在步骤D1至D5的场景中需要生成三维子带系数阵列，且针对三维子带系数阵列中的第一二维字段系数阵列中的第一变换系数进行编码，而步骤C1至C4的场景中没有生成三维子带系数阵列。
具体的，接下来对步骤D1至D5进行说明。编码端在生成至少两个二维子带系数阵列之后，按照预设的频率点扫描次序对至少两个二维子带系数阵列进行组合，以得到三维子带系数阵列，即一个频率点对应一个二维子带系数阵列，按照预设的频率点扫描次序可以将所有频率点分别对应的二维子带系数阵列组合在一起，得到三维子带系数阵列。
三维子带系数阵列中包括多个二维子带系数阵列，编码端首先根据预设的三维子带系数扫描次序从三维子带系数阵列中确定出一个二维子带系数阵列，例如确定出第一二维子带系数阵列，第一二维子带系数阵列为三维子带系数阵列中的一个二维子带系数阵列。其 中，三维子带系数扫描次序是对三维子带系数阵列进行扫描的次序。编码端在确定出第一二维子带系数阵列之后，编码端可以按照预设的系数扫描次序分别对第一二维子带系数阵列中的每个变换系数进行熵编码。本申请实施例中通过对二维子带系数阵列中的变换系数进行编码，可以得到图像编码信息，实现对图像的压缩处理。
在本申请的一些实施例中，步骤302按照预设的频率点扫描次序确定一个频率点对应的所述至少两个变换系数块中的变换系数，按照预设的系数扫描次序对一个频率点对应的各个变换系数进行编码，包括：
E1、按照预设的频率点扫描次序对至少两个变换系数块进行排序，得到排序后的至少两个变换系数块；
E2、从排序后的至少两个变换系数块中获取不同频率点对应的多个二维子带系数阵列；
E3、对不同频率点对应的多个二维子带系数阵列进行组合，以得到三维子带系数阵列；
E4、按照预设的三维子带系数扫描次序从三维子带系数阵列中确定第一二维子带系数阵列。
步骤303按照预设的系数扫描次序对一个频率点对应的各个变换系数进行编码，包括：
E5、按照预设的系数扫描次序确定第一二维子带系数阵列中第一变换系数；
E6、根据第一变换系数对应的上下文区域对第一变换系数进行熵编码。
其中，步骤E1至E6的场景与前述步骤D1至D5的场景的区别在于，在步骤E1至E6的场景中生成三维子带系数阵列的方式与前述步骤D1至D5中生成三维子带系数阵列的方式不相同。即步骤E1至E6的场景中先对至少两个变换系数块进行排序，然后按照不同的频率点确定不同频率点对应的多个二维子带系数阵列，对不同频率点对应的多个二维子带系数阵列进行组合，以得到三维子带系数阵列。
具体的，接下来对步骤E1至E6进行说明。编码端获取到多个二维子带系数阵列之后，进行组合，以得到三维子带系数阵列，三维子带系数阵列中包括多个二维子带系数阵列，编码端首先根据预设的三维子带系数扫描次序从三维子带系数阵列中确定出一个二维子带系数阵列，例如确定出第一二维子带系数阵列，第一二维子带系数阵列为三维子带系数阵列中的一个二维子带系数阵列。其中，三维子带系数扫描次序是对三维子带系数阵列进行扫描的次序。编码端在确定出第一二维子带系数阵列之后，编码端可以按照预设的系数扫描次序分别对第一二维子带系数阵列中的每个变换系数进行熵编码。本申请实施例中通过对二维子带系数阵列中的变换系数进行编码，可以得到图像编码信息，实现对图像的压缩处理。
举例说明如下，编码端按频率点扫描次序分别对变换系数块(后续简称为系数块)中的每个变换系数进行排序，然后再获取相同频率点的系数进行组合，得到三维子带系数阵列。例如变换系数块可以是8x8的变换系数块。按照频率点扫描次序排列8x8的变换系数块中的每个变换系数，得到排序后的8x8的变换系数块。获取图像中8x8的变换系数块中指定频率点的变换系数值，构成该频率点的二维子带系数阵列，将多个频率点对应的二维子带系数阵列组合，得到三维子带系数阵列。在获取到三维子带系数阵列之后，编码端还可以对三维子带系数阵列中的每个变换系数进行编码处理，详见后续实施例中对编码流程的举例说明。
进一步的，在本申请的一些实施例中，三维子带系数扫描次序，包括：
预设的水平位置索引的变化次序；或者，
预设的竖直位置索引的变化次序；或者，
预设的频率索引的变化次序。
其中，三维子带系数阵列中的每个变换系数具有一个水平位置索引、一个竖直位置索引和一个频率索引，例如每个变换系数可以表示为C(bx,by,fs)，其中，bx是水平位置索引，by是竖直位置索引，fs是频率点对应的频率索引。三维子带系数扫描次序是对三维子带系数阵列进行扫描的次序。例如，三维子带系数扫描次序以水平位置索引的变化次序从大到小(或者从小到大)确定三维子带系数阵列中的多个二维子带系数阵列，在每确定出一个二维子带系数阵列之后，再按照前述的系数扫描次序确定该二维子带系数阵列中的一个变换系数，即三维子带系数扫描次序可以用于指示依次对三维子带系数阵列中不同水平位置索引对应的二维子带系数阵列进行编码。
又如，三维子带系数扫描次序以竖直位置索引的变化次序从大到小(或者从小到大)确定三维子带系数阵列中的多个二维子带系数阵列，在每确定出一个二维子带系数阵列之后，再按照前述的系数扫描次序确定该二维子带系数阵列中的一个变换系数，即三维子带系数扫描次序可以用于指示依次对三维子带系数阵列中不同竖直位置索引对应的二维子带系数阵列进行编码。
又如，三维子带系数扫描次序以频率索引的变化次序从大到小(或者从小到大)确定三维子带系数阵列中的多个二维子带系数阵列，在每确定出一个二维子带系数阵列之后，再按照前述的系数扫描次序确定该二维子带系数阵列中的一个变换系数，即三维子带系数扫描次序可以用于指示依次对三维子带系数阵列中不同频率索引对应的二维子带系数阵列进行编码。具体可以根据应用场景确定三维子带系数扫描次序的具体实现方式，此次不做限定。
三维子带系数扫描次序的实现方式有多种，举例说明如下：
三维子带系数阵列表示为C(bx,by,fs)，(bx,by)表示变换系数块水平位置索引和竖直位置索引，bx＝0,…,M-1，by＝0,…,N-1，M＝W/subW，N＝H/subH，其中，M为以单个图像块为单位的水平方向上的最大图像块个数，N为以单个图像块为单位的竖直方向上的最大图像块个数，W表示当前图像的宽度值，H表示当前图像的高度值，subW表示当前图像中图像块的宽度值，subH表示当前图像中图像块的高度值。(fx,fy)表示变换系数块中每个变换系数的频率点，fs表示每个频率点对应的频率索引，其中，fx＝0,..,subW-1，fy＝0,..,subH-1。
方法一：对三维子带系数阵列C(bx,by,fs)中的每个变换系数进行处理时，依次对fs＝0到subW×subH-1层的二维系数子带阵列进行处理，在处理每一层的二维系数子带阵列时，按照前述的系数扫描次序处理二维系数子带阵列C(bx,by)中的每个变换系数。
方法二：对三维子带系数阵列C(bx,by,fs)中的每个变换系数进行处理时，依次对bx＝0到M-1层的二维系数子带阵列进行处理，在处理每一层的二维系数子带阵列时，按照前述的系数扫描次序处理二维系数子带阵列C(by,fs)中的每个变换系数。
方法三：对三维子带系数阵列C(bx,by,fs)中的每个变换系数进行处理时，依次对by＝0 到N-1层的二维系数子带阵列进行处理，在处理每一层的二维系数子带阵列时，按照预置的系数扫描次序处理二维系数子带阵列C(bx,fs)中的每个变换系数。
需要说明的是，预置的系数扫描次序可以是zig-zag扫描次序、逐行扫描次序和逐列扫描次序等。
在本申请的一些实施例中，步骤302按照预设的频率点扫描次序确定一个频率点对应的所述至少两个变换系数块中的变换系数，包括：
F1、按照预设的频率点扫描次序确定至少两个变换系数块中由相同频率点对应的变换系数构成的第一二维子带系数阵列。
步骤303按照预设的系数扫描次序对一个频率点对应的各个变换系数进行编码，包括：
F2、按照预设的系数扫描次序确定第一二维子带系数阵列中的第一变换系数；
F3、根据第一变换系数对应的上下文区域对第一变换系数进行熵编码。
具体的，编码端在获取到至少两个变换系数块之后，编码端还可以按照预设的频率点扫描次序从至少两个变换系数块中获取相同频率点对应的变换系数构成的第一二维子带系数阵列。编码端无需组合出三维子带系数阵列。编码端在确定出第一二维子带系数阵列之后，编码端可以按照预设的系数扫描次序分别对第一二维子带系数阵列中的每个变换系数进行编码。例如，编码端按照预设的系数扫描次序确定第一二维子带系数阵列中的第一变换系数，根据第一变换系数对应的上下文区域对第一变换系数进行熵编码。本申请实施例中通过对二维子带系数阵列中的变换系数进行编码，可以得到图像编码信息，实现对图像的压缩处理。
举例说明如下，编码端首先按照频率点扫描次序分别对至少两个变换系数块中的每个变换系数进行扫描，例如按照频率点扫描次序确定一个频率点(该频率点的频率索引为fs),获取至少两个变换系数块中指定的频率索引fs分别对应的一个变换系数块中的一个变换系数，构成一个频率点的二维子带系数阵列。在确定出一个二维子带系数阵列之后，可以按照预设的系数扫描次序处理该二维子带系数阵列中的每一个变换系数，例如可以采用熵编码对二维子带系数阵列中的每一个变换系数进行编码，以得到图像编码信息。
在本申请的一些实施例中，步骤302按照预设的频率点扫描次序确定一个频率点对应的所述至少两个变换系数块中的变换系数，包括：
G1、按照预设的频率点扫描次序确定一个频率子带包括的至少两个变换系数块中的变换系数，一个频率子带包括的至少两个变换系数块中的变换系数对应于相同的频率点。
步骤303按照预设的系数扫描次序对一个频率点对应的各个变换系数进行编码，包括：
G2、按照预设的系数扫描次序确定一个频率子带包括的第一变换系数；
G3、根据第一变换系数对应的上下文区域对第一变换系数进行熵编码。
其中，前述应用场景中编码端需要生成二维子带系数阵列或者生成三维子带系数阵列，在步骤G1中，编码端还可以采用不生成二维子带系数阵列，也不生成三维子带系数阵列的方式，编码端按照预设的频率点扫描次序确定一个频率子带包括的至少两个变换系数块中的变换系数，一个频率子带包括的至少两个变换系数块中的变换系数对应于相同的频率点，编码端按照预设的系数扫描次序分别对一个频率子带对应的至少两个变换系数块中的变换系数进行编码，例如按照预设的系数扫描次序确定一个频率子带包括的第一变换系数，第 一变换系数可以是一个频率子带包括的多个变换系数中的任意一个变换系数，根据第一变换系数对应的上下文区域对第一变换系数进行熵编码。本申请实施例中通过对频率子带中的变换系数进行编码，可以得到图像编码信息，实现对图像的压缩处理。
在本申请的一些实施例中，以编码端对前述的第一变换系数进行熵编码的具体过程进行说明，对于第一二维子带系数阵列中的其它变换系数的编码过程或者一个频率子带包括的其它变换系数的编码过程，都可以按照第一变换系数的熵编码过程来实现。例如，前述的步骤C4、D5、E6、F3、G3中的根据第一变换系数对应的上下文区域对第一变换系数进行熵编码，包括：
H1、从第一变换系数对应的上下文区域中获取第一变换系数的上下文信息，其中，第一变换系数的上下文信息包括：第一变换系数对应的上下文区域中的已编码变换系数。
具体的，第一变换系数对应的上下文区域可以是预置的，例如根据当前的第一变换系数的水平位置索引、竖直位置索引和频率索引确定第一变换系数对应的上下文区域，该第一变换系数对应的上下文区域中的已编码变换系数，该已编码变换系数是在第一变换系数进行熵编码之前就已经完成熵编码的变换系数，因此从第一变换系数对应的上下文区域中获取第一变换系数的上下文信息。
在本申请的一些实施例中，第一变换系数对应的上下文区域包括如下至少一种：第一变换系数的空间邻域，第一变换系数的子带邻域，第一变换系数的三维邻域。
其中，第一变换系数对应的上下文区域具有多种实现方式，例如可以包括第一变换系数的空间邻域，或第一变换系数的子带邻域，或第一变换系数的三维邻域。其中，第一变换系数的空间邻域是指与第一变换系数处于同一个水平空间或者竖直空间的相邻区域，第一变换系数的子带邻域是指与第一变换系数处于同一个子带的相邻区域，的三维邻域是指与第一变换系数处于同一个三维空间的相邻区域。在第一变换系数的空间邻域，或第一变换系数的子带邻域，或第一变换系数的三维邻域中都可以包括已编码的变换系数，从而编码端可以获取到第一变换系数的上下文信息。
H2、根据第一变换系数的上下文信息对第一变换系数进行估计，以得到第一变换系数的概率分布信息。
H3、根据第一变换系数的概率分布信息对第一变换系数进行算术编码。
在本申请实施例中，编码端在获取到第一变换系数的上下文信息之后，编码端根据第一变换系数的上下文信息对第一变换系数进行熵编码，以得到第一变换系数对应的图像编码信息。编码端执行的熵编码过程可以包括概率估计和算术编码。具体的，编码端根据第一变换系数的上下文信息对第一变换系数进行概率估计，以得到第一变换系数的概率分布，然后编码端根据第一变换系数的概率分布对第一变换系数进行算术编码，以得到第一变换系数对应的图像编码信息。
举例说明如下，在处理每个二维子带系数阵列时，按照系数扫描次序处理每个二维子带系数阵列中的每个变换系数。选择预置上下文区域中已处理的系数作为上下文，并将上下文信息输入概率估计网络，对当前系数的概率分布进行估计，从而获得当前系数的概率分布，并使用该概率分布对当前系数进行算术编码得到当前系数的编码信息。预置上下文区域可以包括该系数的空间邻域和/或子带邻域。
具体的，对当前系数的概率分布进行估计的方法包括：概率估计网络根据该系数的动态取值范围直接对取值范围内的值进行概率估计，得到一个概率向量，概率向量中包括取值范围内每个的值的概率值，其中所有概率值的和为1。又如，对当前系数的概率分布进行估计的方法包括：使用概率分布模型来获得概率向量。例如使用单高斯模型(gaussian single model,GSM)或者混合高斯模型(gaussian mixture model,GMM)建模，使用概率估计网络估计得到模型参数，将模型参数代入概率分布模型中，得到概率向量。
举例说明如下，获得第一变换系数的三维邻域内的已编码系数作为上下文信息后，将上下文信息输入概率估计网络，对该第一变换系数进行概率估计得到该第一变换系数的概率分布，使用该概率分布对当前的第一变换系数进行算术编码，得到该第一变换系数的编码信息。概率估计网络中使用的概率分布估计方法包括基于深度学习网络的概率分布估计方法，例如循环神经网络(recurrent neural network，RNN)和逐像素卷积神经网络(pixel convolutional neural network，PixelCNN)等。
在本申请的一些实施例中，以编码端对前述的第一变换系数进行熵编码的具体过程进行说明，对于第一二维子带系数阵列中的其它变换系数的编码过程或者一个频率子带包括的其它变换系数的编码过程，都可以按照第一变换系数的熵编码过程来实现。例如，前述的步骤C4、D5、E6、F3、G3中的根据第一变换系数对应的上下文区域对第一变换系数进行熵编码，包括：
I1、当第一变换系数为多个变换系数时，从多个变换系数对应的上下文区域中获取多个变换系数的上下文信息，其中，多个变换系数的上下文信息包括：多个变换系数对应的上下文区域中的已编码变换系数；
I2、根据多个变换系数的上下文信息对多个变换系数并行的进行估计，以得到多个变换系数的概率分布信息；
I3、根据多个变换系数的概率分布信息对多个变换系数并行的进行算术编码。
其中，步骤I1至步骤H3的应用场景与前述步骤I1至步骤I3的应用场景的区别在于，步骤I1至步骤I3是针对第一变换系数为多个变换系数，此时可以并行的进行的概率分布的估计以及并行的算术编码。其中，编码端可以获取到多个的变换系数，编码端根据多个的变换系数的上下文信息并行的对多个的变换系数进行熵编码，通过并行的熵编码，可以提高图像编码效率。举例说明如下，编码端在获取系数的概率分布时，也可以对多个变换系数的概率分布并行的进行概率估计，即一次获取多个变换系数的概率分布，然后再按对多个变换系数并行的进行算术编码。
在本申请的一些实施例中，多个变换系数对应的上下文区域包括如下至少一种：多个变换系数的空间邻域，多个变换系数的子带邻域，多个变换系数的三维邻域。
其中，多个变换系数对应的上下文区域具有多种实现方式，例如可以包括多个变换系数的空间邻域，或多个变换系数的子带邻域，或多个变换系数的三维邻域。其中，多个变换系数的空间邻域是指与多个变换系数处于同一个水平空间或者竖直空间的相邻区域，多个变换系数的子带邻域是指与多个变换系数处于同一个子带的相邻区域，的三维邻域是指与第一变换系数处于同一个三维空间的相邻区域。在多个变换系数的空间邻域，或多个变换系数的子带邻域，或多个变换系数的三维邻域中都可以包括已编码的变换系数，从而编 码端可以获取到多个变换系数的上下文信息。
本申请的一些实施例中，步骤302按照预设的频率点扫描次序确定一个频率点对应的所述至少两个变换系数块中的变换系数，包括：
J1、从至少两个变换系数块中获取至少两个二维子带系数阵列，其中，一个二维子带系数阵列包括至少两个变换系数块中相同频率点对应的变换系数，不相同的二维子带系数阵列包括的变换系数对应于不同的频率点；
J2、按照预设的频率点扫描次序确定至少两个二维子带系数阵列对应的至少两个子带图像。
步骤303按照预设的系数扫描次序对一个频率点对应的各个变换系数进行编码，包括：
J3、按照预设的系数扫描次序对至少两个子带图像中的变换系数进行编码。
其中，编码端编码端可以根据频率点的不同对至少两个变换系数块进行排列，从而得到至少两个二维子带系数阵列。同一个二维子带系数阵列中包括相同频率点对应的变换系数块中的变换系数，不同的二维子带系数阵列具有不同的频率点，即编码端可以将相同频率点对应的至少两个变换系数块中的变换系数归属到一个二维子带系数阵列中，因此存在多少个频率点，就可以得到多个频率点对应的二维子带系数阵列。
编码端在获取到至少两个二维子带系数阵列之后，编码端采用对子带图像进行无损编码的方式，以至少两个二维子带系数阵列作为至少两个子带图像，按照预设的频率点扫描次序对至少两个子带图像中的变换系数进行图像无损编码，编码端在T恤无损编码时可以将二维子带系数阵列作为一个子带图像进行编码，简化了无损编码的实现流程。
通过前述对图像处理方法的说明可知，本申请实施例中可以获取至少两个变换系数块，从而针对至少两个变换系数块中相同频率点对应的变换系数，只需要按照频率点扫描次序进行编码就可以得到图像编码信息，因此针对同一个频率点对应的多个变换系数可以同时进行编码，提高图像压缩效率，降低图像编码信息的存储成本，最终节省存储资源。
在本申请实施例中，编码端可以生成图像编码信息之后，编码端可以通过图像传输通道向解码端发送，例如编码端发送压缩码流，该压缩码流中包括图像编码信息。该图像编码信息由编码端对至少两个变换系数块进行编码后得到，解码端需要采用与编码端相应的图像处理方法进行解码。具体的，解码端执行后续步骤311至步骤313。
311、获取图像编码信息。
在本申请实施例中，解码端首先从编码端获取图像编码信息。例如解码端从编码端接收到压缩码流，然后解析该压缩码流，可以得到图像编码信息。
312、按照预设的频率点扫描次序确定图像编码信息中一个频率点对应的待解码变换系数。
313、按照预设的系数扫描次序对一个频率点对应的待解码变换系数进行解码，以得到至少两个变换系数块。
其中，至少两个变换系数块中的每个变换系数块包括：至少两个变换系数，一个变换系数块中的各个变换系数对应于不同的频率点。
在本申请实施例中，解码端获取到图像编码信息之后，图像编码信息中对应于同一个频率点的待解码变换系数进行解码，其中，待解码变换系数又可以称为待解码的变换系数。 例如可使用的解码方法包括熵解码、哈夫曼(Huffman)解码等。在得到某一个频率点对应的所有变换系数之后，按照预设的频率点扫描次序确定下一个频率点，再通过解码得到下一个频率点对应的多个变换系数，按照该频率点扫描次序依次获取到所有的频率点，当通过解码得到所有频率点对应的变换系数之后，解码端可以生成至少两个变换系数块，至少两个变换系数块中的每个变换系数块包括：至少两个变换系数，其中，一个变换系数块中的各个变换系数对应于不同的频率点。对于变换系数块、频率点扫描次序和系数扫描次序的说明，详见前述编码端中的举例。
在本申请的一些实施例中，在编码端执行步骤C1至C4所示的编码的情况下，解码端执行相应的解码方法，具体的，步骤312按照预设的频率点扫描次序确定图像编码信息中一个频率点对应的待解码变换系数，包括：
K1、按照预设的频率点扫描次序确定图像编码信息中的第一待解码二维子带系数阵列，第一待解码二维子带系数阵列包括：图像编码信息中一个频率点对应的待解码变换系数。
步骤就313按照预设的系数扫描次序对一个频率点对应的待解码变换系数进行解码，包括：
K2、按照预设的系数扫描次序确定第一待解码二维子带系数阵列中的第一待解码变换系数；
K3、根据第一待解码变换系数对应的上下文区域对第一待解码变换系数进行熵解码。
具体的，解码端采用与编码端相对应的图像解码方式，若编码端执行步骤C1至C4所示的编码方法，则在解码端收到的图像编码信息中可以包括至少两个待解码二维子带系数阵列。频率点扫描次序是待解码二维子带系数阵列的扫描次序，按照预设的频率点扫描次序从至少两个待解码二维子带系数阵列中获取第一待解码二维子带系数阵列，该第一待解码二维子带系数阵列中包括多个待解码变换系数。
在本申请的一些实施例中，在编码端执行步骤D1至D5所示的编码的情况下，解码端执行相应的解码方法，具体的，步骤312按照预设的频率点扫描次序确定图像编码信息中一个频率点对应的待解码变换系数，包括：
L1、按照预设的频率点扫描次序确定图像编码信息中的待解码三维子带系数阵列，待解码三维子带系数阵列包括：图像编码信息中所有频率点分别对应的待解码变换系数；
L2、按照预设的三维子带系数扫描次序确定待解码三维子带系数阵列中的第一待解码二维子带系数阵列，第一待解码二维子带系数阵列包括：图像编码信息中一个频率点对应的待解码变换系数。
步骤就313按照预设的系数扫描次序对一个频率点对应的待解码变换系数进行解码，包括：
L3、按照预设的系数扫描次序确定第一待解码二维子带系数阵列中的第一待解码变换系数；
L4、根据第一待解码变换系数对应的上下文区域对第一待解码变换系数进行熵解码。
具体的，解码端采用与编码端相对应的图像解码方式，解码端收到的图像编码信息中可以包括待解码三维子带系数阵列。解码端按照预设的三维子带系数扫描次序从待解码三维子带系数阵列中可以获取到至少两个二维子带系数阵列。其中，同一个二维子带系数阵 列中包括相同频率点对应的至少两个变换系数块中的变换系数，不同的二维子带系数阵列具有不同的频率点。
其中，三维子带系数阵列中包括多个二维子带系数阵列，解码端首先根据预设的三维子带系数扫描次序从三维子带系数阵列中确定出一个二维子带系数阵列，例如确定出第一二维子带系数阵列，第一二维子带系数阵列为三维子带系数阵列中的一个二维子带系数阵列。三维子带系数扫描次序是对三维子带系数阵列进行扫描的次序。
解码端采用与编码端相对应的图像解码方式，在编码端采用生成三维子带系数阵列的情况下，解码端预设的系数扫描次序对第二二维子带系数阵列进行无损解码，以得到所述至少两个变换系数块。例如，无损解码可以包括熵解码。
在本申请的一些实施例中，三维子带系数扫描次序，包括：
预设的水平位置索引的变化次序；或者，
预设的竖直位置索引的变化次序；或者，
预设的频率索引的变化次序。
详见前述实施例中对三维子带系数扫描次序的说明。
在本申请的一些实施例中，在编码端执行步骤G1至G3所示的编码的情况下，解码端执行相应的解码方法。具体的，步骤312按照预设的频率点扫描次序确定图像编码信息中一个频率点对应的待解码变换系数，包括：
M1、按照预设的频率点扫描次序确定图像编码信息中一个频率子带包括的待解码变换系数，一个频率子带包括的待解码变换系数对应于相同的频率点。
步骤就313按照预设的系数扫描次序对一个频率点对应的待解码变换系数进行解码，包括：
M2、按照预设的系数扫描次序确定一个频率子带中的第一待解码变换系数；
M3、根据第一待解码变换系数对应的上下文区域对第一待解码变换系数进行熵解码。
其中，解码端采用与编码端相对应的图像解码方式，编码端在编码时可以将二维子带系数阵列作为一个频率子带进行编码，则在解码端收到的图像编码信息中可以包括多个频率子带对应的待解码变换系数。解码端按照频率点扫描次序从图像编码信息中获取一个频率子带对应的待解码变换系数。
解码端可以采用不生成子带系数阵列的方式，例如不生成二维子带系数阵列，也不生成三维子带系数阵列。解码端先确定出第一频率子带，第一频率子带对应的至少两个变换系数块中的变换系数对应于同一个频率点。解码端根据预设的系数扫描次序对第一频率子带对应的图像编码信息进行无损解码，并采样这种方式分别对多个频率子带对应的图像编码信息进行无损解码，以得到多个变换系数。
具体的，解码端获取多个频率子带对应的图像编码信息，解码端采用对频率子带进行无损解码的方式，以至少两个二维子带系数阵列作为至少两个频率子带，按照预设的系数扫描次序对至少两个频率子带对应的图像编码信息进行无损解码，解码端在无损解码时可以将二维子带系数阵列作为一个频率子带进行解码，简化了无损解码的实现流程。
在本申请的一些实施例中，前述步骤K3、L4和M4中的根据第一待解码变换系数对应的上下文区域对第一待解码变换系数进行熵解码，包括：
N1、从第一待解码变换系数对应的上下文区域中获取第一待解码变换系数的上下文信息，其中，第一待解码变换系数对应的上下文区域包括：第一待解码变换系数对应的上下文区域中的已解码变换系数；
N2、根据第一待解码变换系数的上下文信息对第一待解码变换系数进行估计，以得到第一待解码变换系数的概率分布信息；
N3、根据第一待解码变换系数的概率分布信息对第一待解码变换系数进行算术解码，以得到解码后的第一变换系数，解码后的第一变换系数属于至少两个变换系数块。
其中，解码端根据的无损解码可以是熵解码，接下来以第一变换系数的熵解码过程进行说明，对于第一二维子带系数阵列中的其它变换系数的解码过程，都可以按照第一变换系数的熵解码过程来实现。
在本申请实施例中，解码端在获取到第一变换系数的上下文信息之后，解码端根据第一变换系数的上下文信息对第一变换系数进行熵解码，以得到解码后的第一变换系数。解码端执行的熵解码可以包括概率估计和算术解码。具体的，解码端根据第一变换系数的上下文信息对第一变换系数进行概率估计，以得到第一变换系数的概率分布，然后解码端根据第一变换系数的概率分布对第一变换系数进行算术解码，以得到第一变换系数对应的图像解码信息。
举例说明如下，对熵编码后的编码信息进行熵解码时，使用与编码端相同的处理次序对每个码元进行解码。在处理每个码元时，将待解码码元的预置上下文区域内已解码的码元作为上下文输入概率估计网络，估计获得当前码元的概率分布，并使用该概率分布对当前码元进行解码得到当前码元的变换系数。将每个码元解码后，得到二维子带系数阵列。
解码端根据的无损解码可以是熵解码，接下来以第一变换系数的熵解码过程进行说明，对于第一二维子带系数阵列中的其它变换系数的解码过程，都可以按照第一变换系数的熵解码过程来实现。
在本申请实施例中，解码端在获取到第一变换系数的上下文信息之后，解码端根据第一变换系数的上下文信息对第一变换系数进行熵解码，以得到解码后的第一变换系数。解码端执行的熵解码可以包括概率估计和算术解码。具体的，解码端根据第一变换系数的上下文信息对第一变换系数进行概率估计，以得到第一变换系数的概率分布，然后解码端根据第一变换系数的概率分布对第一变换系数进行算术解码，以得到第一变换系数对应的图像解码信息。
举例说明如下，对熵编码后的编码信息进行熵解码时，使用与编码端相同的处理次序对每个码元进行解码。在处理每个码元时，将待解码码元的预置上下文区域内已解码的码元作为上下文输入概率估计网络，估计获得当前码元的概率分布，并使用该概率分布对当前码元进行解码得到当前码元的变换系数。将每个码元解码后，得到二维子带系数阵列。
在本申请的一些实施例中，第一待解码变换系数对应的上下文区域包括如下至少一种：第一待解码变换系数的空间邻域，第一待解码变换系数的子带邻域，第一待解码变换系数的三维邻域。
在本申请的一些实施例中，前述步骤K3、L4和M4中的根据第一待解码变换系数对应的上下文区域对第一待解码变换系数进行熵解码，包括：
P1、当第一待解码变换系数为多个待解码变换系数时，从多个待解码变换系数对应的上下文区域中获取多个待解码变换系数的上下文信息，其中，多个待解码的上下文信息包括：多个待解码变换系数对应的上下文区域中的已解码变换系数；
P2、根据多个待解码变换系数的上下文信息对多个待解码变换系数并行的进行估计，以得到第一待解码变换系数的概率分布信息；
P3、根据多个待解码变换系数的概率分布信息对多个待解码变换系数并行的进行算术解码，以得到解码后的第一变换系数，解码后的第一变换系数属于至少两个变换系数块。
解码端采用与编码端相对应的图像解码方式，编码端在编码时可以并行的进行编码，则在解码端通过并行的解码，可以提高图像解码效率。
在本申请的一些实施例中，多个待解码变换系数对应的上下文区域包括如下至少一种：多个待解码变换系数的空间邻域，多个待解码变换系数的子带邻域，多个待解码变换系数的三维邻域。
在本申请的一些实施例中，在编码端执行步骤G1至G3所示的编码的情况下，解码端执行相应的解码方法，具体的，步骤312按照预设的频率点扫描次序确定图像编码信息中一个频率点对应的待解码变换系数，包括：
Q1、按照预设的频率点扫描次序确定图像编码信息中的第一子带图像，第一子带图像包括：图像编码信息中的一个频率点对应的待解码变换系数。
步骤就313按照预设的系数扫描次序对一个频率点对应的待解码变换系数进行解码，包括：
Q2、按照预设的系数扫描次序对第一子带图像的待解码系数进行解码，以得到至少两个变换系数块。
具体的，解码端采用与编码端相对应的图像解码方式，编码端在编码时可以将二维子带系数阵列作为一个子带图像进行编码，则在解码端收到的图像编码信息中可以包括多个子带图像对应的待解码变换系数。解码端按照频率点扫描次序从图像编码信息中获取第一子带图像对应的待解码变换系数。
解码端采用针对子带图像无损解码，得到多个变换系数之后，解码端采用对子带图像进行无损解码的方式，以至少两个二维子带系数阵列作为至少两个子带图像，按照预设的系数扫描次序对至少两个子带图像对应的图像编码信息进行无损解码，解码端在无损解码时可以将二维子带系数阵列作为一个子带图像进行解码，简化了无损解码的实现流程。
在本申请的一些实施例中，步骤313得到至少两个变换系数块之后，本申请实施例提供的方法还包括：
R1、对至少两个变换系数块进行熵编码，以得到变换系数编码信息；
R2、根据变换系数编码信息生成至少两个图像块；
R3、根据至少两个图像块还原出原始图像区域，原始图像区域包括：原始像素图像区域或者残差图像区域。
其中，解码端采用与编码端相对应的图像解码方式，若编码端通过对变换系数编码信息进行熵解码得到变换系数块，并对变换系数块进行编码，则解码端可以对至少两个变换系数块进行熵编码，以得到变换系数编码信息。本申请实施例中根据变换系数编码信息生 成至少两个图像块，再根据至少两个图像块还原出原始图像区域，从而可以完成图像的解压缩处理。
在本申请的一些实施例中，步骤313得到至少两个变换系数块之后，本申请实施例提供的方法还包括：
S1、对至少两个变换系数块进行反量化和反变换，以得到至少两个图像块；
S2、根据至少两个图像块还原出原始图像区域，原始图像区域包括：原始像素图像区域或者残差图像区域。
在本申请实施例中，解码端在生成至少两个变换系数块之后，解码端根据该至少两个变换系数块生成至少两个图像块。其中，解码端可以通过反变换和反量化的方式对至少两个变换系数块进行处理，从而可以生成至少两个图像块。
对相同频率点对应的至少两个变换系数块中的变换系数进行反变换和反量化，得到至少两个图像块；或，对相同频率点对应的至少两个变换系数块中的变换系数进行熵编码，得到至少两个图像块。
在本申请实施例中，解码端在生成至少两个图像块之后，解码端根据该至少两个图像块还原出原始图像，多个图像块可以通过拼接的方式得到原始图像。
通过前述对图像处理方法的说明可知，本申请实施例中根据预设的频率点扫描次序和频率点扫描次序对图像编码信息中相同频率点对应的待解码变换系数进行解码，就可以得到至少两个变换系数块，由于一个变换系数块中的各个变换系数对应于不同的频率点，因此针对图像编码信息的解码，可以得到同一个频率点对应的不同变换系数，提高图像解压缩效率，降低图像编码信息的存储成本，最终节省存储资源。
为便于更好的理解和实施本申请实施例的上述方案，下面举例相应的应用场景来进行具体说明。
本申请实施例提出一种视频图像压缩中的块变换与系数编码方案，本申请实施例提高图像编码性能，最终节省存储资源，提高视频图像的编码效率。例如，熵编码产生的压缩文件消耗大量服务器侧的存储资源，本申请实施例可应用于熵编码图像的高效无损转码过程中，本申请实施例还可以应用于服务器端，对熵编码产生的压缩文件进行无损转码，提高图像编码性能，最终节省存储资源。
接下来通过五个不同的实施例对本申请实施例提供的图像处理方法进行详细说明，在每个实施例中，先从编码端介绍图像编码过程，在编码端生成图像编码码流之后，编码端向解码端发送，解码端接收到图像编码码流之后，解码端对图像编码码流进行解码，从而还原出原始图像。
实施例一
本实施例涉及一种图像编码方案，首先获取变换系数块(简称为系数块)，再对系数块使用进行无损编码，得到编码后的图像数据。本实施例提供整个编码的流程，例如可以包括对图像区域进行变换和量化，再对量化后的系数进行无损编码，得到编码后的图像数据。如图4所示，图像编码流程主要包括：
步骤401、获取变换系数块。
方法一：将待处理图像区域划分为预置大小的图像块，并对每个图像块进行变换和量 化，得到变换系数块，该变换系数块具体可以是变换系数阵列C(bx,by,fx,fy)。
首先，将待处理图像或者图像区域划分为相同预置大小的子图像块，预置大小的子图像块的尺寸可以是4x4、8x8、16x16、32x32、64x64、128x128和256x256等。
如图5所示，为本申请实施例提供的获取变换系数块的示意图。对预置大小的子图像块进行DCT变换，得到变换系数块。可选的，对变换系数块进行量化，得到量化后的系数阵列C(bx,by,fx,fy)，系数阵列中的每个变换系数可以用C_(bx,by)^(fx,fy)表示。其中，(bx,by)表示系数块水平和竖直的位置索引，bx＝0,…,M-1，by＝0,…,N-1，M＝W/subW，N＝H/subH，其中，M和N分别为以子图像块为单位时的图像宽和高，W和H分别表示当前图像的宽和高，subW和subH分别表示当前图像中子图像块的宽和高。(fx,fy)表示系数块中每个频率点相对于左上顶点的频率点的频率索引，其中，fx＝0,..,subW-1，fy＝0,..,subH-1。本申请实施例中，预置大小的子图像块的尺寸以8x8为例，即子图像块的宽和高均为8，即subW和subH的值均为8，后续不再赘述。
例如图5所示，(bx＝0,by＝0)表示第一个系数块，(bx＝0,by＝1)表示第一行的第二个系数块，依此类推。(fx＝0,fy＝1)表示系数块中相对于左上顶点的频率点的坐标为(0,1)的频率索引。C_(bx＝1,by＝0)^(fx＝3,fy＝2)表示系数块(bx＝1,by＝0)内的位置为(fx＝3,fy＝2)表示的系数值。
方法二：对熵编码后的图像数据进行熵解码，得到图像中8x8块的变换系数(简称为系数)。
使用哈夫曼解码方法对JPEG文件中的所有子图像块的变换系数编码信息进行熵解码，得到一个或者多个子图像块的变换系数块，系数块的每个变换系数记做C_(bx,by)^(fx,fy)。其中，(bx,by)表示系数块的水平和竖直的位置索引，bx＝0,…,M-1，by＝0,…,N-1，M＝W/subW，N＝H/subH，其中，M和N分别为以子图像块为单位的图像宽和高，W和H分别表示当前图像的宽和高，subW和subH分别表示当前图像中子图像块的宽和高。(fx,fy)表示系数块中每个频率点相对于左上顶点的频率点的频率索引，其中，fx＝0,..,subW-1，fy＝0,..,subH-1。在本申请实施例中，JPEG文件中的所有子图像块的宽和高均为8，即subW和subH的值均为8，后续不再赘述。不限定的是，本申请实施例中所描述的技术方案也可应用于subW和subH的取值不为8的情况，例如subW不等于subH。
例如，如图5所示，(bx＝0,by＝0)表示第一个系数块，(bx＝0,by＝1)表示第一行的第二个系数块，依此类推。(fx＝0,fy＝1)表示系数块中相对于左上顶点的频率点的坐标为(0,1)的频率索引。C_(bx＝1,by＝0)^(fx＝3,fy＝2)表示系数块(bx＝0,by＝1)内的位置为(fx＝3,fy＝2)表示的系数值。
步骤402、获取图像中8x8的变换系数块(简称8x8系数块)中指定频率点的变换系数值，构成该频率点的二维子带系数阵列。
具体的，如图6所示，为本申请实施例提供的获取二维子带系数阵列的示意图。获取图像中8x8的变换系数块中指定频率点(fx,fy)的变换系数值，得到该频率点的二维子带系数阵列C^(fx,fy)(bx,by)。
例如，获取图像中所有8x8的变换系数块中频率点(fx＝0,fy＝0)的变换系数值得到的二维子带系数阵列如图6的第一个图所示，同样的，获取剩余频率点的二维子带系数阵列。 一个8x8的变换系数块共有64个频率点，因此可获得当前编码图像的64个二维子带系数阵列。
步骤403、按照预置次序一处理每个二维子带系数阵列，在处理每个二维子带系数阵列中的每个变换系数时，按照预置次序二处理每个变换系数，在处理每个变换系数时，选择预置的空间邻域和子带邻域中已处理的系数作为上下文估计获得当前系数的概率分布，并使用该概率分布对当前系数进行算术编码，得到当前系数的编码信息。
需要说明的是，按照预置次序一处理每个二维子带系数阵列是指按照预置次序一确定每个二维子带系数阵列，照预置次序二处理每个变换系数是指照预置次序二对每个变换系数进行编码。
步骤403.1、按照预置次序一处理每个二维子带系数阵列。
如图7所示，为本申请实施例提供的变换系数块的示意图。设置指定频率点(fx,fy)的二维子带系数阵列的频率索引为fs，每个索引的值对应一个二维子带系数阵列。首先获取图像中的任意一个8x8的变换系数块，按照预置次序一对该系数块内指定频率点(fx,fy)的变换系数进行扫描，则指定频率点(fx,fy)的二维子带系数阵列的索引fs与该频率点在该系数块内的扫描次序相同，按照fs的大小顺序处理fs对应的二维子带系数阵列C^(fx,fy)(bx,by)。以系数块(bx＝0,by＝0)为例，若预置次序一为Z字扫描，则fs＝0对应频率点(fx＝0，fy＝0)的二维子带系数阵列，fs＝1对应频率点(fx＝1，fy＝0)的二维子带系数阵列，fs＝2对应频率点(fx＝0，fy＝1)的二维子带系数阵列，按照fs的大小顺序来处理对应频率点的二维子带系数阵列。其中，fs为整数，取值范围为0,…,subW×subH-1。
步骤403.2、处理每个二维子带系数阵列中的每个变换系数。
在处理每个二维子带系数阵列时，按照预置次序二处理每个二维子带系数阵列中的每个变换系数。选择预置上下文区域中已处理的系数作为上下文，并将上下文信息输入概率估计网络，对当前系数的概率分布进行估计，从而获得当前系数的概率分布，并使用该概率分布对当前系数进行算术编码得到当前系数的编码信息。预置上下文区域(如下文解释)可以包括该系数的空间邻域和/或子带邻域，或者是后续实施例中定义的三维邻域。需要说明的是，在获取系数的概率分布时，也可以对系数的概率分布并行的进行概率估计，即一次获取多个系数的概率分布，然后再按照预置次序二对每个变换系数进行无损编码，或者对多个系数并行的进行无损编码。
具体的，对当前系数的概率分布进行估计的方法如下：方法一：概率估计网络根据该系数的动态取值范围直接对取值范围内的值进行概率估计，得到一个概率向量，概率向量中包括取值范围内每个的值的概率值，其中所有概率值的和为1。方法二：还可以使用概率分布模型来获得概率向量。例如使用单高斯模型或者混合高斯模型建模，使用概率估计网络估计得到模型参数，将模型参数代入概率分布模型中，得到概率向量。
首先，获得该系数预置上下文区域中的系数作为上下文后，将上下文信息输入概率估计网络，对该系数进行概率估计得到该系数的概率分布，使用该概率分布对当前系数进行无损编码，得到该系数的编码信息。概率估计网络中使用的概率分布估计方法可以包括基于深度学习网络的概率分布估计方法，例如循环神经网络和卷积神经网络等。
其中，若系数所在的坐标为(bx0,by0,fs0)，若预置上下文区域内任一坐标位置 (bx1,by1,fs1)满足bx0-M1≤bx1≤bx0+N1，且by0-M2≤by1≤by0+N2，且fs1＝fs0，则(bx1,by1,fs1)位置的系数在该系数的子带邻域内。
若预置上下文区域内任一坐标位置(bx1,by1,fs1)满足bx1＝bx0，且by0-M2≤by1≤by0+N2，且fs0-N3≤fs1≤fs0，或者满足bx0-M1≤bx1≤bx0+N1，且by1＝by0，且fs0-N3≤fs1≤fs0，则(bx1,by1,fs1)位置的系数在该系数的空间邻域内。其中，M1,M2,N1,N2和N3为大于或者等于0的整数，取值可以相等，也可以互不相等，在此不做限定，例如可取值为1，2，3，4或者5。
其中，预置次序一和预置次序二可以是按照之字(zig-zag或者称为Z字)扫描得到的处理次序，还可以使用其他扫描方法得到处理次序，例如逐行扫描、逐列扫描、对角扫描、水平反向扫描和垂直反向扫描等。其中逐行扫描是指从图像的第一行一直连续扫描到最后一行，逐列扫描是指从图像的第一列一直连续扫描到最后一列。
步骤404、获得转码输出文件。
将熵编码后的编码信息输出，得到压缩后的文件。
如图8所示，为本申请实施例提供的一种图像解码流程的示意图。主要包括：
步骤801、获取熵编码后的编码信息，对编码信息进行熵解码得到系数阵列。
对熵编码后的编码信息进行熵解码时，使用与编码端相同的处理次序对每个码元进行解码。即首先按照预置次序一处理已编码的每个二维子带系数阵列，在处理每个已编码的二维子带系数阵列时，按照预置次序二处理每个二维子带系数阵列中的每个已编码的系数。在本实施例中，已编码的系数可以理解为码元。在处理每个码元时，将待解码码元的预置上下文区域内已解码的码元作为上下文输入概率估计网络，估计获得当前码元的概率分布，并使用该概率分布对当前码元进行解码得到当前码元的变换系数。将每个码元解码后，得到多个图像的8x8的变换系数块或系数阵列C(bx,by,fx,fy)。
步骤802、获得解码后的图像数据。
方法一：对应步骤401中的方法一的解码：对系数阵列C(bx,by,fx,fy)进行反变换和反量化，得到解码后的图像数据。
方法二：对应步骤401中的方法二的解码：对8x8的变换系数块进行熵编码，得到熵编码后的图像数据。
本实施例一提供的编码方案中，将输入的图像块划分成预置大小的图像块，然后进行变换和量化，得到变换系数阵列。按照预置次序来处理变换系数阵列中的每个变换系数，并在对其中一个系数做算术编码时，使用该系数邻域已处理的系数作为上下文来估计该系数值的概率分布。本申请实施例可以提高图像的压缩率，减少服务器端的存储资源。
实施例二
本实施例涉及一种图像编码方案，首先获取变换系数块(简称为系数块)，再对系数块使用进行无损编码，得到编码后的图像数据。本实施例提供整个编码的流程，例如可以包括对图像区域进行变换和量化，再对量化后的系数进行无损编码，得到编码后的图像数据。如图9所示，图像编码流程主要包括：
步骤901、获取变换系数块。
本步骤同实施例一的步骤401。
步骤902、获取图像中8x8的变换系数块中指定频率点的变换系数值，构成该频率点的二维子带系数阵列。
本步骤同实施例一的步骤402。
步骤903、按预置次序一排列二维子带系数阵列，得到三维子带系数阵列。
如图10所示，按照预置次序一排列指定频率点(fx,fy)的二维子带系数阵列C^(fx,fy)(bx,by)，并设置排序后的二维子带系数阵列的频率索引为fs。首先，获取图像中的任意一个8x8的变换系数块，按照预置次序一对该系数块内指定频率点(fx,fy)的变换系数进行排序，则指定频率点(fx,fy)的二维子带系数阵列的排序索引与其在该系数块内的排序索引相同，从而确定出指定频率点(fx,fy)的二维子带系数阵列C^(fx,fy)(bx,by)的频率索引fs。以系数块(bx＝0,by＝0)为例，若预置次序一为之字扫描，则fs＝0对应频率点(fx＝0，fy＝0)的二维子带系数阵列，fs＝1对应频率点(fx＝1，fy＝0)的二维子带系数阵列，fs＝2对应频率点(fx＝0，fy＝1)的二维子带系数阵列。并将所述二维子带系数阵列C^(fx,fy)(bx,by)组合在一起，得到如图8所示的三维子带系数阵列C(bx,by,fs)。其中，一维频率索引fs取值为0,…,subW×subH-1中的整数。可以理解的是，频率索引fs是经过预置次序对二维子带系数阵列排序后的索引。
其中，预置次序一可以是按照之字(zig-zag)扫描得到的排列次序，还可以使用其他扫描方法得到预置次序，例如：逐行扫描、逐列扫描和对角扫描等。其中逐行扫描是指从图像的第一行一直连续扫描到最后一行，逐列扫描是指从图像的第一列一直连续扫描到最后一列。
如图11a所示，为本申请实施例提供的三维子带系数阵列的示意图。三维子带系数阵列C(bx,by,fs)可以包括三个维度的数据：bx、by和fs。
需要说明的是，本实施例的步骤901和步骤902同实施例一的步骤401和步骤402相同。不同之处在于，实施例二中增加了将二维子带系数阵列按照预置次序组合成三维子带系数阵列的步骤。
步骤904：对三维子带系数阵列中的每个变换系数进行处理，在处理每一个系数时，使用该系数三维邻域内的已处理系数作为上下文估计获得当前系数的概率分布，并使用该概率分布对当前系数进行算术编码得到当前系数的编码信息。
如图11b所示，为本申请实施例提供的三维的变换系数的示意图。以图中黑色填充的1个方块表示当前处理的变换系数，横线填充的方块表示未编码的变换系数，白色的方块表示已编码的变换系数，则当前处理的变换系数的三维邻域中可以包括已编码的变换系数。
步骤9041、设置三维子带系数阵列C(bx,by,fs)中每个变换系数的处理次序。
具体的，可以使用以下方法中的一种来设置每个变换系数的处理次序，其中方法一是最优方法。
方法一：对三维子带系数阵列C(bx,by,fs)中的每个变换系数进行处理时，依次对fs＝0到subW×subH-1层的二维系数子带阵列进行处理，在处理每一层的二维系数子带阵列时，按照预置次序二处理二维系数子带阵列C(bx,by)中的每个变换系数。
方法二：对三维子带系数阵列C(bx,by,fs)中的每个变换系数进行处理时，依次对bx＝0到M-1层的二维系数子带阵列进行处理，在处理每一层的二维系数子带阵列时，按照预置 次序二处理二维系数子带阵列C(by,fs)中的每个变换系数。
方法三：对三维子带系数阵列C(bx,by,fs)中的每个变换系数进行处理时，依次对by＝0到N-1层的二维系数子带阵列进行处理，在处理每一层的二维系数子带阵列时，按照预置次序二处理二维系数子带阵列C(bx,fs)中的每个变换系数。
需要说明的是，预置次序二可以使用zig-zag扫描、逐行扫描和逐列扫描等处理次序，含义与步骤三中的“预置次序一”相同，在此不做赘述。
步骤9042、获取该系数的概率分布。
在处理一个系数时，使用该系数三维邻域内的已处理系数作为上下文，估计获得该系数的概率分布，并使用该概率分布对当前系数进行算术编码得到当前系数的编码信息。
其中，该系数的三维邻域内的系数可以根据以下方法获取：若系数所在的坐标为(bx0,by0,fs0)，若坐标位置(bx1,by1,fs1)满足bx0-M1≤bx1≤bx0+N1，且by0-M2≤by1≤by0+N2，且fs0-N3≤fs1≤fs0，则(bx1,by1,fs1)位置的系数在该系数的三维邻域内，其中M1,M2,N1,N2和N3为大于或者等于0的整数，取值可以相等，也可以互不相等，在此不做限定，例如可取值为1，2，3，4或者5。因此，该系数三维邻域内的已处理系数还包括了该系数二维邻域内的已处理系数，在此不做限定。特别地，当fs1＝fs0时，当选取的上下文与待处理系数处于同一个二维邻域，此时更利于并行编码，即对多个系数并行的进行概率估计，同时得到多个系数的概率分布，提高了编码效率。
需要说明的是，在获取系数的概率分布时，也可以对系数的概率分布并行的进行概率估计，即一次获取多个系数的概率分布，然后再按照预置次序二对每个变换系数进行无损编码，或者对多个系数并行的进行无损编码。
首先，获得该系数的三维邻域内的系数作为上下文后，将上下文信息输入概率估计网络，对该系数进行概率估计得到该系数的概率分布，使用该概率分布对当前系数进行无损编码，得到该系数的编码信息。概率估计网络中使用的概率分布估计方法包括基于深度学习网络的概率分布估计方法，例如循环神经网络和逐像素卷积神经网络等。
具体的，对系数的概率分布进行估计的方法如下：方法一：概率估计网络根据该系数的动态取值范围直接对取值范围内的值进行概率估计，得到一个概率向量，概率向量中包括取值范围内每个的值的概率值，其中所有概率值的和为1。方法二：还可以使用概率分布模型来获得概率向量。例如使用单高斯模型或者混合高斯模型建模，使用概率估计网络估计得到模型参数，将模型参数代入概率分布模型中，得到概率向量。
步骤905、获得转码输出文件。
将熵编码后的编码信息输出，得到转码输出文件。
如图12所示，为本申请实施例提供的一种图像解码流程的示意图。主要包括：
步骤1201、获取熵编码后的编码信息，对编码信息进行熵解码得到解码后的变换系数。
对熵编码后的编码信息进行熵解码时，使用与编码端相同的处理次序对每个码元进行解码，例如相同的处理次序指的是先按照频率点扫描次序确定图像编码信息中一个频率点对应的待解码系数，再按照系数扫描次序对一个频率点对应的待解码系数进行解码。在处理每个码元时，将待解码码元输入与编码端相同的概率估计网络，使用该码元三维邻域内的已处理码元作为上下文估计获得当前码元的概率分布，并使用该概率分布对当前码元进 行解码得到当前码元的变换系数。将每个码元解码后，得到变换系数块或系数阵列C(bx,by,fx,fy)。
步骤1203、获得解码后的图像数据。
方法一：对应步骤401中的方法一的解码：对系数阵列C(bx,by,fx,fy)进行反变换和反量化，得到解码后的图像数据。
方法二：对应步骤401中的方法二的解码：对8x8的变换系数块进行熵编码，得到熵编码后的图像数据。
本实施例二提供的编码方案中，本实施例将变换系数按频率分量组合得到三维系数阵列C(bx,by,fs)，并在对其中一个系数做算术编码时，使用该系数邻域已处理的系数作为上下文来估计该系数值的概率分布。本申请实施例可以提高图像的压缩率，减少服务器端的存储资源。
实施例三
本实施例涉及一种图像编码方案，首先获取变换系数块(简称为系数块)，再对系数块使用进行无损编码，得到编码后的图像数据。本实施例提供整个编码的流程，例如可以包括对图像区域进行变换和量化，再对量化后的系数进行无损编码，得到编码后的图像数据。
本实施例使用实施例一的步骤401中的方法，获取得到变换系数阵列C(bx,by,fx,fy)后，有以下两种编码方法：
方法一、先按预置次序一分别对系数块中的每个变换系数进行排序，然后再获取相同频率点的系数，进行组合，得到三维子带系数阵列(简称为三维系数阵列)。如图13所示，图像编码流程主要包括：
步骤1301、获取变换系数块。
本步骤同实施例一的步骤401。
步骤1302、按照预置次序一排列8x8的变换系数块中的每个变换系数，得到排序后的8x8的变换系数块。
步骤1303、获取图像中8x8的变换系数块中指定频率点的变换系数值，构成该频率点的二维子带系数阵列，将二维子带系数阵列组合，得到三维子带系数阵列。
步骤1304、对三维子带系数阵列中的每个变换系数进行处理，在处理每一个系数时，使用该系数三维邻域内的已处理系数作为上下文估计获得当前系数的概率分布，并使用该概率分布对当前系数进行算术编码得到当前系数的编码信息。具体的，与实施例二的步骤904相同。
方法二、本方法的实施方案与实施例一类似，无需组合为三维子带系数阵列。
先按预置次序一分别对8x8的变换系数块中的每个变换系数进行扫描，得到频率索引fs，然后再获取8x8的变换系数块中指定频率点的变换系数值，构成该频率点的二维子带系数阵列。
按照fs处理每个二维子带系数阵列，在处理每个二维子带系数阵列中的每个变换系数时，按照预置次序二处理每个二维子带系数阵列中的每个变换系数，在处理每个变换系数时，选择预置的空间邻域和子带邻域中已处理的系数作为上下文估计获得当前系数的概率分布，并使用该概率分布对当前系数进行算术编码得到当前系数的编码信息。
本实施例三提供的编码方案中，本实施例对每个变换系数处理时的处理次序与实施例一和实施例二的处理次序不同。
如图12所示，为本申请实施例提供的一种图像解码流程的示意图。主要包括：
步骤1201、获取熵编码后的编码信息，对编码信息进行熵解码得到解码后的变换系数。
对熵编码后的编码信息进行熵解码时，使用与编码端相同的处理次序对每个码元进行解码，例如相同的处理次序指的是先按照频率点扫描次序确定图像编码信息中一个频率点对应的待解码系数，再按照系数扫描次序对一个频率点对应的待解码系数进行解码。在处理每个码元时，将待解码码元输入与编码端相同的概率估计网络，使用该码元三维邻域内的已处理码元作为上下文估计获得当前码元的概率分布，并使用该概率分布对当前码元进行解码得到当前码元的变换系数。将每个码元解码后，得到变换系数块或系数阵列C(bx,by,fx,fy)。
步骤1202、获得解码后的图像数据。
方法一：对应步骤401中的方法一的解码：对系数阵列C(bx,by,fx,fy)进行反变换和反量化，得到解码后的图像数据。
方法二：对应步骤401中的方法二的解码：对8x8的变换系数块进行熵编码，得到熵编码后的图像数据。
本实施例三提供的编码方案中，本实施例将变换系数按频率分量组合得到三维系数阵列C(bx,by,fs)，并在对其中一个系数做算术编码时，使用该系数邻域已处理的系数作为上下文来估计该系数值的概率分布。本申请实施例可以提高图像的压缩率，减少服务器端的存储资源。
实施例四
本实施例涉及一种图像编码方案，首先获取变换系数块(简称为系数块)，再对系数块使用进行无损编码，得到编码后的图像数据。本实施例提供整个编码的流程，例如可以包括对图像区域进行变换和量化，再对量化后的系数进行无损编码，得到编码后的图像数据。如图14所示，图像编码流程主要包括：
步骤1401、获取变换系数块。
本步骤同实施例一的步骤401。
步骤1402、获取图像中8x8的变换系数块中指定频率点的变换系数值，构成该频率点的二维子带系数阵列。
需要说明的是，本实施例的步骤1401和步骤1402同实施例一的步骤401和步骤402相同。不同之处在于，实施例四中熵编码时，将二维子带系数阵列作为一个子带图像进行编码。
步骤1403、将该频率点对应的二维子带系数阵列作为一个子带图像，在处理一个系数时，使用图像无损编码方法进行编码。
将指定频率点对应的二维子带系数阵列C^(fx,fy)(bx,by)作为一个子带图像，按照预置次序处理子带图像，并设置子带图像处理次序索引为fs。在处理子带图像的一个系数时，使用已有的无损图像处理方法进行编码。例如，H.264与H.265等标准方案中的无损编码模式，如算术编码方法。
其中，处理次序索引fs的获取方法包括：首先获取图像中的任意一个8x8的变换系数块，按照预置次序一对该系数块内指定频率点(fx,fy)的变换系数进行排序，则指定频率点(fx,fy)的二维子带系数阵列的处理次序索引fs与其在该系数块内的排序索引相同。
所述预置次序与实施例一中的含义相同，在此不做赘述。
本实施例四提供的编码方案中，熵编码时，将二维子带系数阵列作为一个子带图像进行编码，方案实现起来更加简单。
实施例五
本实施例涉及一种图像编码方案，首先获取变换系数块(简称为系数块)，再对系数块使用进行无损编码，得到编码后的图像数据。本实施例提供整个编码的流程，例如可以包括对图像区域进行变换和量化，再对量化后的系数进行无损编码，得到编码后的图像数据。如图15所示，图像编码流程主要包括：
步骤1501、获取变换系数块。
本步骤同实施例一的步骤401。
步骤1501、将待处理图像区域划分为预置大小的图像块，并对每个图像块进行变换和量化，得到变换系数阵列。
首先，将待处理图像或者图像区域划分为相同预置大小的子图像块，所述预置大小的子图像块的尺寸可以是4x4、8x8、16x16、32x32、64x64、128x128和256x256等。
如图5所示，对预置大小的子图像块进行DCT变换，得到变换系数块。可选的，对变换系数块进行量化，得到量化后的变换系数阵列C(bx,by,fx,fy)，系数阵列中的每个变换系数可以用C_(bx,by)^(fx,fy)表示。其中，(bx,by)表示系数块水平和竖直的位置索引，bx＝0,…,M-1，by＝0,…,N-1，M＝W/subW，N＝H/subH，其中，M和N分别为以子块计的图像宽和高，W和H分别表示当前图像的宽和高，subW和subH分别表示当前图像中子图像块的宽和高。(fx,fy)表示系数块中每个频率点相对于左上顶点的频率点的频率索引，其中，fx＝0,..,subW-1，fy＝0,..,subH-1。本申请实施例中，预置大小的子图像块的尺寸以8x8为例，即子图像块的宽和高均为8，即subW和subH的值均为8，后续不再赘述。
例如，如图5所示，(bx＝0,by＝0)表示第一个系数块，(bx＝0,by＝1)表示第一行的第二个系数块，依此类推。(fx＝0,fy＝1)表示系数块中相对于左上顶点的频率点的坐标为(0,1)的频率索引。C_(bx＝1,by＝0)^(fx＝3,fy＝2)表示系数块(bx＝1,by＝0)内的位置为(fx＝3,fy＝2)表示的系数值。
本实施例五与实施例一到四的区别在于，在对每个变换系数进行处理时，没有构建二维子带系数阵列这个动作，也没有构建三维系数阵列这个动作，直接将步骤1501获得的系数块按照预置次序一和预置次序二进行处理，实现上较为简单，复杂度更低。
不限定的是，前述步骤1501可以替换为如下步骤：对熵编码后的图像数据进行熵解码，得到图像中8x8块的变换系数。
使用哈夫曼解码方法对JPEG文件中的所有子图像块的变换系数编码信息进行熵解码，得到一个或者多个子图像块的变换系数，简称系数块，系数块的每个变换系数记做C_(bx,by)^(fx,fy)。其中，(bx,by)表示系数块水平和竖直的位置索引，bx＝0,…,M-1，by＝0,…,N-1，M＝W/subW，N＝H/subH，其中M和N分别为以子块计的图像宽和高，W和H 分别表示当前图像的宽和高，subW和subH分别表示当前图像中子图像块的宽和高。(fx,fy)表示系数块中每个频率点相对于左上顶点的频率点的频率索引，其中，fx＝0,..,subW-1，fy＝0,..,subH-1。在本申请实施例中，JPEG文件中的所有子图像块的宽和高均为8，即subW和subH的值均为8，后续不再赘述。不限定的是，本申请实施例中所描述的技术方案也可应用于subW和subH的取值不为8的情况，甚至subW不等于subH。
例如，如图5所示，(bx＝0,by＝0)表示第一个系数块，(bx＝0,by＝1)表示第一行的第二个系数块，依此类推。(fx＝0,fy＝1)表示系数块中相对于左上顶点的频率点的坐标为(0,1)的频率索引。C_(bx＝1,by＝0)^(fx＝3,fy＝2)表示系数块(bx＝0,by＝1)内的位置为(fx＝3,fy＝2)表示的系数值。
步骤1502、按照预置次序一处理当前图像的系数块的每个频率子带，并按照预置次序二处理每个频率子带中的变换系数。在处理每个变换系数时，选择预置上下文区域中已处理的系数作为上下文估计获得当前系数的概率分布，并使用该概率分布对当前系数进行算术编码得到当前系数的编码信息。
不限定的是，前述步骤1502可以替换为如下的步骤：按照预置次序一处理当前图像的系数块中的一个频率点对应的变换系数，并按照预置次序二处理每个频率点对应的各个变换系数。在处理每个变换系数时，选择预置上下文区域中已处理的系数作为上下文估计获得当前系数的概率分布，并使用该概率分布对当前系数进行算术编码得到当前系数的编码信息。
例如，频率子带为8x8的变换系数块中指定频率点(fx,fy)的系数的集合C^(fx,fy)(bx,by)，本实施例中有64个频率子带。首先获取图像中的任意一个8x8的变换系数块，按照预置次序一对该系数块内指定频率点(fx,fy)的系数进行扫描，则指定频率点(fx,fy)的频率子带的索引fs与该频率点在该系数块内的扫描次序相同，按照fs的大小顺序处理fs对应的频率子带。其中，fs为整数，取值范围为0,…,subW×subH-1。
然后，在处理每个频率子带时，按照预置次序二处理频率子带中的每个变换系数。在处理每个变换系数时，选择预置上下文区域中已处理的系数作为上下文，并将上下文信息输入概率估计网络，对系数的概率分布进行估计，从而获得当前系数的概率分布，并使用该概率分布对当前系数进行算术编码得到当前系数的编码信息。预置上下文区域可以包括该系数的空间邻域和/或子带邻域，或者是实施例二中定义的三维邻域。
具体的，对系数的概率分布进行估计的方法如下：方法一：概率估计网络根据该系数的动态取值范围直接对取值范围内的值进行概率估计，得到一个概率向量，概率向量中包括取值范围内每个的值的概率值，其中所有概率值的和为1。方法二：还可以使用概率分布模型来获得概率向量。例如使用单高斯模型或者混合高斯模型建模，使用概率估计网络估计得到模型参数，将模型参数代入概率分布模型中，得到概率向量。
首先，获得该系数的三维邻域内的系数作为上下文后，将上下文信息输入概率估计网络，对该系数进行概率估计得到该系数的概率分布，使用该概率分布对当前系数进行无损编码，得到该系数的编码信息。概率估计网络中使用的概率分布估计方法包括基于深度学习网络的概率分布估计方法，例如循环神经网络和卷积神经网络等。
其中，若系数所在的坐标为(bx0,by0,fs0)，若预置上下文区域内任一坐标位置 (bx1,by1,fs1)满足bx0-M1≤bx1≤bx0+N1，且by0-M2≤by1≤by0+N2，且fs1＝fs0，则(bx1,by1,fs1)位置的系数在该系数的子带邻域内。
若预置上下文区域内任一坐标位置(bx1,by1,fs1)满足bx1＝bx0，且by0-M2≤by1≤by0+N2，且fs0-N3≤fs1≤fs0，或者满足bx0-M1≤bx1≤bx0+N1，且by1＝by0，且fs0-N3≤fs1≤fs0，则(bx1,by1,fs1)位置的系数在该系数的空间邻域内；其中M1,M2,N1,N2和N3为大于或者等于0的整数，取值可以相等，也可以互不相等，在此不做限定，例如，为1，2，3，4或者5。
其中，预置次序一和预置次序二可以是按照之字(zig-zag)扫描得到的处理次序，还可以使用其他扫描方法得到处理次序，例如：逐行扫描、逐列扫描、对角扫描、水平反向扫描和垂直反向扫描等。其中逐行扫描是指从图像的第一行一直连续扫描到最后一行，逐列扫描是指从图像的第一列一直连续扫描到最后一列。
步骤1503、获得转码输出文件。
将熵编码后的编码信息输出，得到转码输出文件。
如图16所示，为本申请实施例提供的一种图像解码流程的示意图。主要包括：
步骤1601、获取熵编码后的编码信息，对编码信息中的每个码元进行熵解码得到解码后的变换系数。
对熵编码后的编码信息进行熵解码，使用与编码端相同的处理次序解码每个码元。首先根据频率子带的索引或者预置次序一获取每个频率子带，再根据预置次序二处理每个频率子带中的码元。在处理每个码元时，将该码元的预置上下文区域内已解码的码元作为上下文输入概率估计网络，获得当前码元的概率分布，并使用该概率分布对当前码元进行解码得到当前码元的变换系数。依次对编码端输出的每个码元进行熵解码得到每个码元的变换系数，从而得到图像的系数块。
步骤1602、获得解码后的图像数据。
方法一：对应步骤401中的方法一的解码：对系数块进行反变换和反量化，得到解码后的图像数据。
方法二：对应步骤401中的方法二的解码：对8x8的变换系数块进行熵编码，得到熵编码后的图像数据。
本实施例五提供的编码方案中，不需要生成二维子带系数阵列，而是直接对频率子带进行编码，方案实现起来更加简单。
本申请实施例涉及视频图像压缩中的块变换与系数编码，变换系数按指定频率点组合得到三维系数阵列C(bx,by,fs)，并对每个频率索引fs对应的系数子图像进行无损编码。变换系数按指定频率点组合得到三维系数阵列C(bx,by,fs)，并在对其中一个系数做算术编码时，使用该系数三维邻域已处理的系数作为上下文来估计该系数值的概率分布。按照预置次序一处理每个二维子带系数阵列C^(fx,fy)(bx,by)，在处理每个二维子带系数阵列中的每个变换系数时，按照预置次序二处理每个二维子带系数阵列中的每个变换系数，使用该系数邻域已处理的系数作为上下文来估计该系数值的概率分布。获取指定频率点的二维子带系数阵列C^(fx,fy)(bx,by)。将该频率点对应的二维子带系数阵列作为一个子带图像，在处理每个子带图像时，按照预置次序一处理每个子带图像。在处理子带图像的 每个变换系数时，使用图像无损编码方法进行编码。按照预置次序一处理当前图像的每个频率子带，并按照预置次序二处理每个频率子带中的变换系数。所述频率子带为8x8的变换系数块中指定频率点(fx,fy)的系数的集合C^(fx,fy)(bx,by)。
通过上述举例说明可知，本申请实施例中将待处理图像区域划分为预置大小的图像块，并对每个图像块进行变换和量化，得到变换系数阵列C(bx,by,fx,fy)。然后按预置次序对其中每一个系数做熵编码，所述预置次序会优先处理每一个频率索引对应的所有系数块的系数。在对每一个系数做熵编码时，按照系数的空间邻近关系与系数的频率分量邻近关系，选择邻近区域已处理的系数作为上下文估计得到当前系数的概率分布，基于该概率分布对当前系数做后续的编码操作。本申请实施例可进一步减小压缩文件的大小，降低服务器侧的视频图像文件存储成本。
需要说明的是，对于前述的各方法实施例，为了简单描述，故将其都表述为一系列的动作组合，但是本邻域技术人员应该知悉，本申请并不受所描述的动作顺序的限制，因为依据本申请，某些步骤可以采用其他顺序或者同时进行。其次，本邻域技术人员也应该知悉，说明书中所描述的实施例均属于优选实施例，所涉及的动作和模块并不一定是本申请所必须的。
为便于更好的实施本申请实施例的上述方案，下面还提供用于实施上述方案的相关装置。
请参阅图17所示，本申请实施例提供的一种图像编码装置1700，可以包括：收发模块1701、处理模块1702，其中，
收发模块，用于获取至少两个变换系数块，所述至少两个变换系数块中的每个变换系数块包括：至少两个变换系数，其中，一个变换系数块中的各个变换系数对应于不同的频率点；
处理模块，用于按照预设的频率点扫描次序确定一个频率点对应的所述至少两个变换系数块中的变换系数；按照预设的系数扫描次序对所述一个频率点对应的各个变换系数进行编码，以得到图像编码信息。在本申请实施例中，可以预先设置或者配置频率点扫描次序，基于该频率点扫描次序确定一个频率点，该频率点可以是任意一个频率点，每一个频率点对应于至少两个变换系数块中每个变换系数块中的一个变换系数，即一个频率点对应于多个变换系数，这多个变换系数来源于不同的变换系数块，可以预先设置或者配置系数扫描次序，基于该系数扫描次序对一个频率点对应的各个变换系数进行编码，最终可以得到图像编码信息。本申请实施例中针对至少两个变换系数块中相同频率点对应的变换系数，只需要按照系数扫描次序进行编码就可以得到图像编码信息，因此针对相同频率点对应的多个变换系数可以实现编码，提高图像压缩效率，且本申请实施例中图像编码信息是按照系数扫描次序对至少两个变换系数块中相同频率点对应的变换系数进行编码得到，编码后的图像编码信息的文件减小，降低图像编码信息的存储成本，最终节省存储资源。
在一种可能的实现方式中，所述处理模块，用于从所述至少两个变换系数块中获取至少两个二维子带系数阵列，其中，一个二维子带系数阵列包括所述至少两个变换系数块中相同频率点对应的变换系数，不相同的二维子带系数阵列包括的变换系数对应于不同的频率点；按照预设的频率点扫描次序确定所述至少两个二维子带系数阵列中的第一二维子带 系数阵列；按照预设的系数扫描次序确定所述第一二维子带系数阵列中的第一变换系数；根据所述第一变换系数对应的上下文区域对所述第一变换系数进行熵编码。
在一种可能的实现方式中，所述处理模块，用于从所述至少两个变换系数块中获取至少两个二维子带系数阵列，其中，一个二维子带系数阵列包括所述至少两个变换系数块中相同频率点对应的变换系数，不相同的二维子带系数阵列包括的变换系数对应于不同的频率点；按照预设的频率点扫描次序对所述至少两个二维子带系数阵列进行组合，以得到三维子带系数阵列；按照预设的三维子带系数扫描次序确定所述三维子带系数阵列中的第一二维子带系数阵列；按照预设的系数扫描次序确定所述第一二维子带系数阵列中第一变换系数；根据所述第一变换系数对应的上下文区域对所述第一变换系数进行熵编码。
在一种可能的实现方式中，所述处理模块，用于按照预设的频率点扫描次序对所述至少两个变换系数块进行排序，得到排序后的至少两个变换系数块；从所述排序后的至少两个变换系数块中获取不同频率点对应的多个二维子带系数阵列；对所述不同频率点对应的多个二维子带系数阵列进行组合，以得到三维子带系数阵列；按照预设的三维子带系数扫描次序从所述三维子带系数阵列中确定第一二维子带系数阵列；按照预设的系数扫描次序确定所述第一二维子带系数阵列中第一变换系数；根据所述第一变换系数对应的上下文区域对所述第一变换系数进行熵编码。
在一种可能的实现方式中，所述三维子带系数扫描次序，包括：预设的水平位置索引的变化次序；或者，预设的竖直位置索引的变化次序；或者，预设的频率索引的变化次序。
在一种可能的实现方式中，所述处理模块，用于按照预设的频率点扫描次序确定所述至少两个变换系数块中由相同频率点对应的变换系数构成的第一二维子带系数阵列；按照预设的系数扫描次序确定所述第一二维子带系数阵列中的第一变换系数；根据所述第一变换系数对应的上下文区域对所述第一变换系数进行熵编码。
在一种可能的实现方式中，所述处理模块，用于按照预设的频率点扫描次序确定一个频率子带包括的所述至少两个变换系数块中的变换系数，所述一个频率子带包括的所述至少两个变换系数块中的变换系数对应于相同的频率点；按照预设的系数扫描次序确定所述一个频率子带包括的第一变换系数；根据所述第一变换系数对应的上下文区域对所述第一变换系数进行熵编码。
在一种可能的实现方式中，所述处理模块，用于从所述第一变换系数对应的上下文区域中获取所述第一变换系数的上下文信息，其中，所述第一变换系数的上下文信息包括：所述第一变换系数对应的上下文区域中的已编码变换系数；根据所述第一变换系数的上下文信息对所述第一变换系数进行估计，以得到所述第一变换系数的概率分布信息；根据所述第一变换系数的概率分布信息对所述第一变换系数进行算术编码。
在一种可能的实现方式中，所述第一变换系数对应的上下文区域包括如下至少一种：所述第一变换系数的空间邻域，所述第一变换系数的子带邻域，所述第一变换系数的三维邻域。
在一种可能的实现方式中，所述处理模块，用于当所述第一变换系数为多个变换系数时，从所述多个变换系数对应的上下文区域中获取所述多个变换系数的上下文信息，其中，所述多个变换系数的上下文信息包括：所述多个变换系数对应的上下文区域中的已编码变 换系数；根据所述多个变换系数的上下文信息对所述多个变换系数并行的进行估计，以得到所述多个变换系数的概率分布信息；根据所述多个变换系数的概率分布信息对所述多个变换系数并行的进行算术编码。
在一种可能的实现方式中，所述多个变换系数对应的上下文区域包括如下至少一种：所述多个变换系数的空间邻域，所述多个变换系数的子带邻域，所述多个变换系数的三维邻域。
在一种可能的实现方式中，所述处理模块，用于从所述至少两个变换系数块中获取至少两个二维子带系数阵列，其中，一个二维子带系数阵列包括所述至少两个变换系数块中相同频率点对应的变换系数，不相同的二维子带系数阵列包括的变换系数对应于不同的频率点；按照预设的频率点扫描次序确定所述至少两个二维子带系数阵列对应的至少两个子带图像；按照预设的系数扫描次序对所述至少两个子带图像中的变换系数进行编码。
在一种可能的实现方式中，所述收发模块，用于获取至少两个图像块的变换系数编码信息；对所述变换系数编码信息进行熵解码，以得到所述至少两个变换系数块。
在一种可能的实现方式中，所述收发模块，用于将原始图像区域划分为至少两个图像块，所述原始图像区域包括：像素图像区域或者残差图像区域；对所述至少两个图像块进行变换和量化，以得到所述至少两个变换系数块。
在本申请的第三方面中，图像处理装置的组成模块还可以执行前述第一方面以及各种可能的实现方式中所描述的步骤，详见前述对第一方面以及各种可能的实现方式中的说明。
请参阅图18所示，本申请实施例提供的一种图像解码装置1800，可以包括：收发模块1801、处理模块1802，其中，
收发模块，用于获取图像编码信息；
处理模块，用于按照预设的频率点扫描次序确定所述图像编码信息中一个频率点对应的待解码变换系数；按照预设的系数扫描次序对所述一个频率点对应的待解码变换系数进行解码，以得到至少两个变换系数块，其中，所述至少两个变换系数块中的每个变换系数块包括：至少两个变换系数，一个变换系数块中的各个变换系数对应于不同的频率点。
在一种可能的实现方式中，所述处理模块，用于按照预设的频率点扫描次序确定所述图像编码信息中的第一待解码二维子带系数阵列，所述第一待解码二维子带系数阵列包括：所述图像编码信息中一个频率点对应的待解码变换系数；按照预设的系数扫描次序确定所述第一待解码二维子带系数阵列中的第一待解码变换系数；根据所述第一待解码变换系数对应的上下文区域对所述第一待解码变换系数进行熵解码。
在一种可能的实现方式中，所述处理模块，用于按照预设的频率点扫描次序确定所述图像编码信息中的待解码三维子带系数阵列，所述待解码三维子带系数阵列包括：所述图像编码信息中所有频率点分别对应的待解码变换系数；按照预设的三维子带系数扫描次序确定所述待解码三维子带系数阵列中的第一待解码二维子带系数阵列，所述第一待解码二维子带系数阵列包括：所述图像编码信息中一个频率点对应的待解码变换系数；按照预设的系数扫描次序确定所述第一待解码二维子带系数阵列中的第一待解码变换系数；根据所述第一待解码变换系数对应的上下文区域对所述第一待解码变换系数进行熵解码。
在一种可能的实现方式中，所述三维子带系数扫描次序，包括：预设的水平位置索引 的变化次序；或者，预设的竖直位置索引的变化次序；或者，预设的频率索引的变化次序。
在一种可能的实现方式中，所述处理模块，用于按照预设的频率点扫描次序确定所述图像编码信息中一个频率子带包括的待解码变换系数，所述一个频率子带包括的待解码变换系数对应于相同的频率点；按照预设的系数扫描次序确定所述一个频率子带中的第一待解码变换系数；根据所述第一待解码变换系数对应的上下文区域对所述第一待解码变换系数进行熵解码。
在一种可能的实现方式中，所述处理模块，用于从所述第一待解码变换系数对应的上下文区域中获取所述第一待解码变换系数的上下文信息，其中，所述第一待解码变换系数对应的上下文区域包括：所述第一待解码变换系数对应的上下文区域中的已解码变换系数；根据所述第一待解码变换系数的上下文信息对所述第一待解码变换系数进行估计，以得到所述第一待解码变换系数的概率分布信息；根据所述第一待解码变换系数的概率分布信息对所述第一待解码变换系数进行算术解码，以得到解码后的第一变换系数，所述解码后的第一变换系数属于所述至少两个变换系数块。
在一种可能的实现方式中，所述第一待解码变换系数对应的上下文区域包括如下至少一种：所述第一待解码变换系数的空间邻域，所述第一待解码变换系数的子带邻域，所述第一待解码变换系数的三维邻域。
在一种可能的实现方式中，所述处理模块，用于当所述第一待解码变换系数为多个待解码变换系数时，从所述多个待解码变换系数对应的上下文区域中获取所述多个待解码变换系数的上下文信息，其中，所述多个待解码的上下文信息包括：所述多个待解码变换系数对应的上下文区域中的已解码变换系数；根据所述多个待解码变换系数的上下文信息对所述多个待解码变换系数并行的进行估计，以得到所述第一待解码变换系数的概率分布信息；根据所述多个待解码变换系数的概率分布信息对所述多个待解码变换系数并行的进行算术解码，以得到解码后的第一变换系数，所述解码后的第一变换系数属于所述至少两个变换系数块。
在一种可能的实现方式中，所述多个待解码变换系数对应的上下文区域包括如下至少一种：所述多个待解码变换系数的空间邻域，所述多个待解码变换系数的子带邻域，所述多个待解码变换系数的三维邻域。
在一种可能的实现方式中，所述处理模块，用于按照预设的频率点扫描次序确定所述图像编码信息中的第一子带图像，所述第一子带图像包括：所述图像编码信息中的一个频率点对应的待解码变换系数；按照预设的系数扫描次序对所述第一子带图像的待解码系数进行解码，以得到所述至少两个变换系数块。
在一种可能的实现方式中，所述处理模块，还用于得到至少两个变换系数块之后，对所述至少两个变换系数块进行熵编码，以得到变换系数编码信息；根据所述变换系数编码信息生成至少两个图像块；根据所述至少两个图像块还原出原始图像区域，所述原始图像区域包括：像素图像区域或者残差图像区域。
在一种可能的实现方式中，所述处理模块，还用于得到至少两个变换系数块之后，对所述至少两个变换系数块进行反量化和反变换，以得到至少两个图像块；根据所述至少两个图像块还原出原始图像区域，所述原始图像区域包括：像素图像区域或者残差图像区域。
需要说明的是，上述装置各模块/单元之间的信息交互、执行过程等内容，由于与本申请方法实施例基于同一构思，其带来的技术效果与本申请方法实施例相同，具体内容可参见本申请前述所示的方法实施例中的叙述，此处不再赘述。
本申请实施例还提供一种计算机存储介质，其中，该计算机存储介质存储有程序，该程序执行包括上述方法实施例中记载的部分或全部步骤。
接下来介绍本申请实施例提供的另一种图像编码装置，请参阅图19所示，图像编码装置1900包括：
接收器1901、发射器1902、处理器1903和存储器1904(其中图像编码装置1900中的处理器1903的数量可以一个或多个，图19中以一个处理器为例)。在本申请的一些实施例中，接收器1901、发射器1902、处理器1903和存储器1904可通过总线或其它方式连接，其中，图19中以通过总线连接为例。
存储器1904可以包括只读存储器和随机存取存储器，并向处理器1903提供指令和数据。存储器1904的一部分还可以包括非易失性随机存取存储器(non-volatile random access memory，NVRAM)。存储器1904存储有操作系统和操作指令、可执行模块或者数据结构，或者它们的子集，或者它们的扩展集，其中，操作指令可包括各种操作指令，用于实现各种操作。操作系统可包括各种系统程序，用于实现各种基础业务以及处理基于硬件的任务。
处理器1903控制图像编码装置的操作，处理器1903还可以称为中央处理单元(central processing unit，CPU)。具体的应用中，图像编码装置的各个组件通过总线系统耦合在一起，其中总线系统除包括数据总线之外，还可以包括电源总线、控制总线和状态信号总线等。但是为了清楚说明起见，在图中将各种总线都称为总线系统。
上述本申请实施例揭示的方法可以应用于处理器1903中，或者由处理器1903实现。处理器1903可以是一种集成电路芯片，具有信号的处理能力。在实现过程中，上述方法的各步骤可以通过处理器1903中的硬件的集成逻辑电路或者软件形式的指令完成。上述的处理器1903可以是通用处理器、数字信号处理器(digital signal processing，DSP)、专用集成电路(application specific integrated circuit，ASIC)、现场可编程门阵列(field-programmable gate array，FPGA)或者其他可编程逻辑器件、分立门或者晶体管逻辑器件、分立硬件组件。可以实现或者执行本申请实施例中的公开的各方法、步骤及逻辑框图。通用处理器可以是微处理器或者该处理器也可以是任何常规的处理器等。结合本申请实施例所公开的方法的步骤可以直接体现为硬件译码处理器执行完成，或者用译码处理器中的硬件及软件模块组合执行完成。软件模块可以位于随机存储器，闪存、只读存储器，可编程只读存储器或者电可擦写可编程存储器、寄存器等本邻域成熟的存储介质中。该存储介质位于存储器1904，处理器1903读取存储器1904中的信息，结合其硬件完成上述方法的步骤。
接收器1901可用于接收输入的数字或字符信息，以及产生与图像编码装置的相关设置以及功能控制有关的信号输入，发射器1902可包括显示屏等显示设备，发射器1902可用于通过外接接口输出数字或字符信息。
本申请实施例中，处理器1903用于执行前述实施例图4所示的由图像编码装置执行的 图像处理方法。
接下来介绍本申请实施例提供的另一种图像解码装置，请参阅图20所示，图像解码装置2000包括：
接收器2001、发射器2002、处理器2003和存储器2004(其中图像解码装置2000中的处理器2003的数量可以一个或多个，图20中以一个处理器为例)。在本申请的一些实施例中，接收器2001、发射器2002、处理器2003和存储器2004可通过总线或其它方式连接，其中，图20中以通过总线连接为例。
存储器2004可以包括只读存储器和随机存取存储器，并向处理器2003提供指令和数据。存储器2004的一部分还可以包括NVRAM。存储器2004存储有操作系统和操作指令、可执行模块或者数据结构，或者它们的子集，或者它们的扩展集，其中，操作指令可包括各种操作指令，用于实现各种操作。操作系统可包括各种系统程序，用于实现各种基础业务以及处理基于硬件的任务。
处理器2003控制图像解码装置的操作，处理器2003还可以称为CPU。具体的应用中，图像解码装置的各个组件通过总线系统耦合在一起，其中总线系统除包括数据总线之外，还可以包括电源总线、控制总线和状态信号总线等。但是为了清楚说明起见，在图中将各种总线都称为总线系统。
上述本申请实施例揭示的方法可以应用于处理器2003中，或者由处理器2003实现。处理器2003可以是一种集成电路芯片，具有信号的处理能力。在实现过程中，上述方法的各步骤可以通过处理器2003中的硬件的集成逻辑电路或者软件形式的指令完成。上述的处理器2003可以是通用处理器、DSP、ASIC、FPGA或者其他可编程逻辑器件、分立门或者晶体管逻辑器件、分立硬件组件。可以实现或者执行本申请实施例中的公开的各方法、步骤及逻辑框图。通用处理器可以是微处理器或者该处理器也可以是任何常规的处理器等。结合本申请实施例所公开的方法的步骤可以直接体现为硬件译码处理器执行完成，或者用译码处理器中的硬件及软件模块组合执行完成。软件模块可以位于随机存储器，闪存、只读存储器，可编程只读存储器或者电可擦写可编程存储器、寄存器等本邻域成熟的存储介质中。该存储介质位于存储器2004，处理器2003读取存储器2004中的信息，结合其硬件完成上述方法的步骤。
本申请实施例中，处理器2003，用于执行前述实施例图4所示的由图像解码装置执行的图像处理方法。
在另一种可能的设计中，当图像处理装置或者图像处理装置为终端内的芯片时，芯片包括：处理单元和通信单元，所述处理单元例如可以是处理器，所述通信单元例如可以是输入/输出接口、管脚或电路等。该处理单元可执行存储单元存储的计算机执行指令，以使该终端内的芯片执行上述第一方面任意一项的无线通信方法。可选地，所述存储单元为所述芯片内的存储单元，如寄存器、缓存等，所述存储单元还可以是所述终端内的位于所述芯片外部的存储单元，如只读存储器(read-only memory，ROM)或可存储静态信息和指令的其他类型的静态存储设备，随机存取存储器(random access memory，RAM)等。
其中，上述任一处提到的处理器，可以是一个通用中央处理器，微处理器，ASIC，或一个或多个用于控制上述第一方面或第二方面方法的程序执行的集成电路。
另外需说明的是，以上所描述的装置实施例仅仅是示意性的，其中所述作为分离部件说明的单元可以是或者也可以不是物理上分开的，作为单元显示的部件可以是或者也可以不是物理单元，即可以位于一个地方，或者也可以分布到多个网络单元上。可以根据实际的需要选择其中的部分或者全部模块来实现本实施例方案的目的。另外，本申请提供的装置实施例附图中，模块之间的连接关系表示它们之间具有通信连接，具体可以实现为一条或多条通信总线或信号线。
通过以上的实施方式的描述，所属邻域的技术人员可以清楚地了解到本申请可借助软件加必需的通用硬件的方式来实现，当然也可以通过专用硬件包括专用集成电路、专用CPU、专用存储器、专用元器件等来实现。一般情况下，凡由计算机程序完成的功能都可以很容易地用相应的硬件来实现，而且，用来实现同一功能的具体硬件结构也可以是多种多样的，例如模拟电路、数字电路或专用电路等。但是，对本申请而言更多情况下软件程序实现是更佳的实施方式。基于这样的理解，本申请的技术方案本质上或者说对现有技术做出贡献的部分可以以软件产品的形式体现出来，该计算机软件产品存储在可读取的存储介质中，如计算机的软盘、U盘、移动硬盘、ROM、RAM、磁碟或者光盘等，包括若干指令用以使得一台计算机设备(可以是个人计算机，服务器，或者网络设备等)执行本申请各个实施例所述的方法。
在上述实施例中，可以全部或部分地通过软件、硬件、固件或者其任意组合来实现。当使用软件实现时，可以全部或部分地以计算机程序产品的形式实现。
所述计算机程序产品包括一个或多个计算机指令。在计算机上加载和执行所述计算机程序指令时，全部或部分地产生按照本申请实施例所述的流程或功能。所述计算机可以是通用计算机、专用计算机、计算机网络、或者其他可编程装置。所述计算机指令可以存储在计算机可读存储介质中，或者从一个计算机可读存储介质向另一计算机可读存储介质传输，例如，所述计算机指令可以从一个网站站点、计算机、服务器或数据中心通过有线(例如同轴电缆、光纤、数字用户线(DSL))或无线(例如红外、无线、微波等)方式向另一个网站站点、计算机、服务器或数据中心进行传输。所述计算机可读存储介质可以是计算机能够存储的任何可用介质或者是包含一个或多个可用介质集成的服务器、数据中心等数据存储设备。所述可用介质可以是磁性介质，(例如，软盘、硬盘、磁带)、光介质(例如，DVD)、或者半导体介质(例如固态硬盘(Solid State Disk，SSD))等。

Claims (28)

  1. 一种图像处理方法，其特征在于，包括：
    获取至少两个变换系数块，所述至少两个变换系数块中的每个变换系数块包括：至少两个变换系数，其中，一个变换系数块中的各个变换系数对应于不同的频率点；
    按照预设的频率点扫描次序确定一个频率点对应的所述至少两个变换系数块中的变换系数；
    按照预设的系数扫描次序对所述一个频率点对应的各个变换系数进行编码，以得到图像编码信息。
  2. 根据权利要求1所述的方法，其特征在于，所述按照预设的频率点扫描次序确定一个频率点对应的所述至少两个变换系数块中的变换系数，包括：
    从所述至少两个变换系数块中获取至少两个二维子带系数阵列，其中，一个二维子带系数阵列包括所述至少两个变换系数块中相同频率点对应的变换系数，不相同的二维子带系数阵列包括的变换系数对应于不同的频率点；
    按照预设的频率点扫描次序确定所述至少两个二维子带系数阵列中的第一二维子带系数阵列；
    所述按照预设的系数扫描次序对所述一个频率点对应的各个变换系数进行编码，包括：
    按照预设的系数扫描次序确定所述第一二维子带系数阵列中的第一变换系数；
    根据所述第一变换系数对应的上下文区域对所述第一变换系数进行熵编码。
  3. 根据权利要求1所述的方法，其特征在于，所述按照预设的频率点扫描次序确定一个频率点对应的所述至少两个变换系数块中的变换系数，包括：
    从所述至少两个变换系数块中获取至少两个二维子带系数阵列，其中，一个二维子带系数阵列包括所述至少两个变换系数块中相同频率点对应的变换系数，不相同的二维子带系数阵列包括的变换系数对应于不同的频率点；
    按照预设的频率点扫描次序对所述至少两个二维子带系数阵列进行组合，以得到三维子带系数阵列；
    按照预设的三维子带系数扫描次序确定所述三维子带系数阵列中的第一二维子带系数阵列；
    所述按照预设的系数扫描次序对所述一个频率点对应的各个变换系数进行编码，包括：
    按照预设的系数扫描次序确定所述第一二维子带系数阵列中第一变换系数；
    根据所述第一变换系数对应的上下文区域对所述第一变换系数进行熵编码。
  4. 根据权利要求1所述的方法，其特征在于，所述按照预设的频率点扫描次序确定一个频率点对应的所述至少两个变换系数块中的变换系数，包括：
    按照预设的频率点扫描次序对所述至少两个变换系数块进行排序，得到排序后的至少两个变换系数块；
    从所述排序后的至少两个变换系数块中获取不同频率点对应的多个二维子带系数阵列；
    对所述不同频率点对应的多个二维子带系数阵列进行组合，以得到三维子带系数阵列；
    按照预设的三维子带系数扫描次序从所述三维子带系数阵列中确定第一二维子带系数 阵列；
    所述按照预设的系数扫描次序对所述一个频率点对应的各个变换系数进行编码，包括：
    按照预设的系数扫描次序确定所述第一二维子带系数阵列中第一变换系数；
    根据所述第一变换系数对应的上下文区域对所述第一变换系数进行熵编码。
  5. 根据权利要求3或4所述的方法，其特征在于，所述三维子带系数扫描次序，包括：
    预设的水平位置索引的变化次序；或者，
    预设的竖直位置索引的变化次序；或者，
    预设的频率索引的变化次序。
  6. 根据权利要求1所述的方法，其特征在于，所述按照预设的频率点扫描次序确定一个频率点对应的所述至少两个变换系数块中的变换系数，包括：
    按照预设的频率点扫描次序确定所述至少两个变换系数块中由相同频率点对应的变换系数构成的第一二维子带系数阵列；
    所述按照预设的系数扫描次序对所述一个频率点对应的各个变换系数进行编码，包括：
    按照预设的系数扫描次序确定所述第一二维子带系数阵列中的第一变换系数；
    根据所述第一变换系数对应的上下文区域对所述第一变换系数进行熵编码。
  7. 根据权利要求1所述的方法，其特征在于，所述按照预设的频率点扫描次序确定一个频率点对应的所述至少两个变换系数块中的变换系数，包括：
    按照预设的频率点扫描次序确定一个频率子带包括的所述至少两个变换系数块中的变换系数，所述一个频率子带包括的所述至少两个变换系数块中的变换系数对应于相同的频率点；
    所述按照预设的系数扫描次序对所述一个频率点对应的各个变换系数进行编码，包括：
    按照预设的系数扫描次序确定所述一个频率子带包括的第一变换系数；
    根据所述第一变换系数对应的上下文区域对所述第一变换系数进行熵编码。
  8. 根据权利要求2至7中任一项所述的方法，其特征在于，所述根据所述第一变换系数对应的上下文区域对所述第一变换系数进行熵编码，包括：
    从所述第一变换系数对应的上下文区域中获取所述第一变换系数的上下文信息，其中，所述第一变换系数的上下文信息包括：所述第一变换系数对应的上下文区域中的已编码变换系数；
    根据所述第一变换系数的上下文信息对所述第一变换系数进行估计，以得到所述第一变换系数的概率分布信息；
    根据所述第一变换系数的概率分布信息对所述第一变换系数进行算术编码。
  9. 根据权利要求8所述的方法，其特征在于，所述第一变换系数对应的上下文区域包括如下至少一种：所述第一变换系数的空间邻域，所述第一变换系数的子带邻域，所述第一变换系数的三维邻域。
  10. 根据权利要求2至7中任一项所述的方法，其特征在于，所述根据所述第一变换系数对应的上下文区域对所述第一变换系数进行熵编码，包括：
    当所述第一变换系数为多个变换系数时，从所述多个变换系数对应的上下文区域中获取所述多个变换系数的上下文信息，其中，所述多个变换系数的上下文信息包括：所述多 个变换系数对应的上下文区域中的已编码变换系数；
    根据所述多个变换系数的上下文信息对所述多个变换系数并行的进行估计，以得到所述多个变换系数的概率分布信息；
    根据所述多个变换系数的概率分布信息对所述多个变换系数并行的进行算术编码。
  11. 根据权利要求10所述的方法，其特征在于，所述多个变换系数对应的上下文区域包括如下至少一种：所述多个变换系数的空间邻域，所述多个变换系数的子带邻域，所述多个变换系数的三维邻域。
  12. 根据权利要求1所述的方法，其特征在于，所述按照预设的频率点扫描次序确定一个频率点对应的所述至少两个变换系数块中的变换系数，包括：
    从所述至少两个变换系数块中获取至少两个二维子带系数阵列，其中，一个二维子带系数阵列包括所述至少两个变换系数块中相同频率点对应的变换系数，不相同的二维子带系数阵列包括的变换系数对应于不同的频率点；
    按照预设的频率点扫描次序确定所述至少两个二维子带系数阵列对应的至少两个子带图像；
    所述按照预设的系数扫描次序对所述一个频率点对应的各个变换系数进行编码，包括：
    按照预设的系数扫描次序对所述至少两个子带图像中的变换系数进行编码。
  13. 根据权利要求1至12中任一项所述的方法，其特征在于，所述获取至少两个变换系数块，包括：
    获取至少两个图像块的变换系数编码信息；
    对所述变换系数编码信息进行熵解码，以得到所述至少两个变换系数块。
  14. 根据权利要求1至12中任一项所述的方法，其特征在于，所述获取至少两个变换系数块，包括：
    将原始图像区域划分为至少两个图像块，所述原始图像区域包括：像素图像区域或者残差图像区域；
    对所述至少两个图像块进行变换和量化，以得到所述至少两个变换系数块。
  15. 一种图像处理方法，其特征在于，包括：
    获取图像编码信息；
    按照预设的频率点扫描次序确定所述图像编码信息中一个频率点对应的待解码变换系数；
    按照预设的系数扫描次序对所述一个频率点对应的待解码变换系数进行解码，以得到至少两个变换系数块，其中，所述至少两个变换系数块中的每个变换系数块包括：至少两个变换系数，一个变换系数块中的各个变换系数对应于不同的频率点。
  16. 根据权利要求15所述的图像处理方法，其特征在于，所述按照预设的频率点扫描次序确定所述图像编码信息中一个频率点对应的待解码变换系数，包括：
    按照预设的频率点扫描次序确定所述图像编码信息中的第一待解码二维子带系数阵列，所述第一待解码二维子带系数阵列包括：所述图像编码信息中一个频率点对应的待解码变换系数；
    所述按照预设的系数扫描次序对所述一个频率点对应的待解码变换系数进行解码，包 括：
    按照预设的系数扫描次序确定所述第一待解码二维子带系数阵列中的第一待解码变换系数；
    根据所述第一待解码变换系数对应的上下文区域对所述第一待解码变换系数进行熵解码。
  17. 根据权利要求15所述的图像处理方法，其特征在于，所述按照预设的频率点扫描次序确定所述图像编码信息中一个频率点对应的待解码变换系数，包括：
    按照预设的频率点扫描次序确定所述图像编码信息中的待解码三维子带系数阵列，所述待解码三维子带系数阵列包括：所述图像编码信息中所有频率点分别对应的待解码变换系数；
    按照预设的三维子带系数扫描次序确定所述待解码三维子带系数阵列中的第一待解码二维子带系数阵列，所述第一待解码二维子带系数阵列包括：所述图像编码信息中一个频率点对应的待解码变换系数；
    所述按照预设的系数扫描次序对所述一个频率点对应的待解码变换系数进行解码，包括：
    按照预设的系数扫描次序确定所述第一待解码二维子带系数阵列中的第一待解码变换系数；
    根据所述第一待解码变换系数对应的上下文区域对所述第一待解码变换系数进行熵解码。
  18. 根据权利要求17所述的方法，其特征在于，所述三维子带系数扫描次序，包括：
    预设的水平位置索引的变化次序；或者，
    预设的竖直位置索引的变化次序；或者，
    预设的频率索引的变化次序。
  19. 根据权利要求15所述的图像处理方法，其特征在于，所述按照预设的频率点扫描次序确定所述图像编码信息中一个频率点对应的待解码变换系数，包括：
    按照预设的频率点扫描次序确定所述图像编码信息中一个频率子带包括的待解码变换系数，所述一个频率子带包括的待解码变换系数对应于相同的频率点；
    所述按照预设的系数扫描次序对所述一个频率点对应的待解码变换系数进行解码，包括：
    按照预设的系数扫描次序确定所述一个频率子带中的第一待解码变换系数；
    根据所述第一待解码变换系数对应的上下文区域对所述第一待解码变换系数进行熵解码。
  20. 根据权利要求16至19中任一项所述的图像处理方法，其特征在于，所述根据所述第一待解码变换系数对应的上下文区域对所述第一待解码变换系数进行熵解码，包括：
    从所述第一待解码变换系数对应的上下文区域中获取所述第一待解码变换系数的上下文信息，其中，所述第一待解码变换系数对应的上下文区域包括：所述第一待解码变换系数对应的上下文区域中的已解码变换系数；
    根据所述第一待解码变换系数的上下文信息对所述第一待解码变换系数进行估计，以 得到所述第一待解码变换系数的概率分布信息；
    根据所述第一待解码变换系数的概率分布信息对所述第一待解码变换系数进行算术解码，以得到解码后的第一变换系数，所述解码后的第一变换系数属于所述至少两个变换系数块。
  21. 根据权利要求20所述的方法，其特征在于，所述第一待解码变换系数对应的上下文区域包括如下至少一种：所述第一待解码变换系数的空间邻域，所述第一待解码变换系数的子带邻域，所述第一待解码变换系数的三维邻域。
  22. 根据权利要求16至19中任一项所述的图像处理方法，其特征在于，所述根据所述第一待解码变换系数对应的上下文区域对所述第一待解码变换系数进行熵解码，包括：
    当所述第一待解码变换系数为多个待解码变换系数时，从所述多个待解码变换系数对应的上下文区域中获取所述多个待解码变换系数的上下文信息，其中，所述多个待解码的上下文信息包括：所述多个待解码变换系数对应的上下文区域中的已解码变换系数；
    根据所述多个待解码变换系数的上下文信息对所述多个待解码变换系数并行的进行估计，以得到所述第一待解码变换系数的概率分布信息；
    根据所述多个待解码变换系数的概率分布信息对所述多个待解码变换系数并行的进行算术解码，以得到解码后的第一变换系数，所述解码后的第一变换系数属于所述至少两个变换系数块。
  23. 根据权利要求22所述的图像处理方法，其特征在于，所述多个待解码变换系数对应的上下文区域包括如下至少一种：所述多个待解码变换系数的空间邻域，所述多个待解码变换系数的子带邻域，所述多个待解码变换系数的三维邻域。
  24. 根据权利要求15所述的图像处理方法，其特征在于，所述按照预设的频率点扫描次序确定所述图像编码信息中一个频率点对应的待解码变换系数，包括：
    按照预设的频率点扫描次序确定所述图像编码信息中的第一子带图像，所述第一子带图像包括：所述图像编码信息中的一个频率点对应的待解码变换系数；
    所述按照预设的系数扫描次序对所述一个频率点对应的待解码变换系数进行解码，包括：
    按照预设的系数扫描次序对所述第一子带图像的待解码系数进行解码，以得到所述至少两个变换系数块。
  25. 根据权利要求15至24中任一项所述的图像处理方法，其特征在于，所述得到至少两个变换系数块之后，所述方法还包括：
    对所述至少两个变换系数块进行熵编码，以得到变换系数编码信息；
    根据所述变换系数编码信息生成至少两个图像块；
    根据所述至少两个图像块还原出原始图像区域，所述原始图像区域包括：像素图像区域或者残差图像区域。
  26. 根据权利要求15至24中任一项所述的图像处理方法，其特征在于，所述得到至少两个变换系数块之后，所述方法还包括：
    对所述至少两个变换系数块进行反量化和反变换，以得到至少两个图像块；
    根据所述至少两个图像块还原出原始图像区域，所述原始图像区域包括：像素图像区 域或者残差图像区域。
  27. 一种图像处理装置，其特征在于，所述图像处理装置包括：处理器和存储器，所述处理器用于与所述存储器耦合，读取并执行所述存储器中的指令，执行如权利要求1至14、或者权利要求15至26中任一项所述的方法。
  28. 一种计算机可读存储介质，包括指令，当其在计算机上运行时，使得计算机执行如权利要求1至14、或者权利要求15至26中任意一项所述的方法。
PCT/CN2021/082657 2020-03-31 2021-03-24 一种图像处理方法和图像处理装置 WO2021197158A1 (zh)

Priority Applications (2)

Application Number Priority Date Filing Date Title
EP21779779.4A EP4117289A4 (en) 2020-03-31 2021-03-24 IMAGE PROCESSING METHOD AND IMAGE PROCESSING DEVICE
US17/954,861 US12022078B2 (en) 2020-03-31 2022-09-28 Picture processing method and apparatus

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202010246679.8 2020-03-31
CN202010246679.8A CN113473139A (zh) 2020-03-31 2020-03-31 一种图像处理方法和图像处理装置

Related Child Applications (1)

Application Number Title Priority Date Filing Date
US17/954,861 Continuation US12022078B2 (en) 2020-03-31 2022-09-28 Picture processing method and apparatus

Publications (1)

Publication Number Publication Date
WO2021197158A1 true WO2021197158A1 (zh) 2021-10-07

Family

ID=77865740

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2021/082657 WO2021197158A1 (zh) 2020-03-31 2021-03-24 一种图像处理方法和图像处理装置

Country Status (4)

Country Link
US (1) US12022078B2 (zh)
EP (1) EP4117289A4 (zh)
CN (1) CN113473139A (zh)
WO (1) WO2021197158A1 (zh)

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20230230283A1 (en) * 2022-01-14 2023-07-20 Fraunhofer-Gesellschaft zur Förderung der angewandten Forschung e.V. Encoder, Decoder and Related Methods
WO2024208149A1 (en) * 2023-04-01 2024-10-10 Douyin Vision Co., Ltd. Method, apparatus, and medium for visual data processing

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5500678A (en) * 1994-03-18 1996-03-19 At&T Corp. Optimized scanning of transform coefficients in video coding
CN1685731A (zh) * 2002-09-27 2005-10-19 皇家飞利浦电子股份有限公司 可分级视频编码
CN101039421A (zh) * 2006-03-16 2007-09-19 华为技术有限公司 在编解码中的实现量化的方法和装置
CN101083767A (zh) * 2006-05-30 2007-12-05 华为技术有限公司 一种视频编解码方法
CN101606391A (zh) * 2007-01-18 2009-12-16 弗劳恩霍夫应用研究促进协会 质量可缩放视频数据流

Family Cites Families (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN100539700C (zh) * 2008-04-30 2009-09-09 华为技术有限公司 图像编码方法和装置
EP2182732A1 (en) * 2008-10-28 2010-05-05 Panasonic Corporation Switching between scans in image coding
US9877048B2 (en) * 2014-06-09 2018-01-23 Qualcomm Incorporated Entropy coding techniques for display stream compression (DSC)
CN104270641B (zh) * 2014-09-30 2018-12-14 杭州华为数字技术有限公司 变换系数的处理方法和装置
US10362310B2 (en) * 2015-10-21 2019-07-23 Qualcomm Incorporated Entropy coding techniques for display stream compression (DSC) of non-4:4:4 chroma sub-sampling

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5500678A (en) * 1994-03-18 1996-03-19 At&T Corp. Optimized scanning of transform coefficients in video coding
CN1685731A (zh) * 2002-09-27 2005-10-19 皇家飞利浦电子股份有限公司 可分级视频编码
CN101039421A (zh) * 2006-03-16 2007-09-19 华为技术有限公司 在编解码中的实现量化的方法和装置
CN101083767A (zh) * 2006-05-30 2007-12-05 华为技术有限公司 一种视频编解码方法
CN101606391A (zh) * 2007-01-18 2009-12-16 弗劳恩霍夫应用研究促进协会 质量可缩放视频数据流

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
See also references of EP4117289A4

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20230230283A1 (en) * 2022-01-14 2023-07-20 Fraunhofer-Gesellschaft zur Förderung der angewandten Forschung e.V. Encoder, Decoder and Related Methods
WO2024208149A1 (en) * 2023-04-01 2024-10-10 Douyin Vision Co., Ltd. Method, apparatus, and medium for visual data processing

Also Published As

Publication number Publication date
US12022078B2 (en) 2024-06-25
EP4117289A4 (en) 2023-08-30
CN113473139A (zh) 2021-10-01
US20230039840A1 (en) 2023-02-09
EP4117289A1 (en) 2023-01-11

Similar Documents

Publication Publication Date Title
US10708584B2 (en) Image decoding method using intra prediction mode
JP4700491B2 (ja) 適応係数スキャン順序付け
CN110800299B (zh) 用于对图像数据的块进行熵代码化的扫描顺序自适应
US9077960B2 (en) Non-zero coefficient block pattern coding
US12022078B2 (en) Picture processing method and apparatus
US11930190B2 (en) Scanning orders for non-transform coding
WO2016172994A1 (zh) 图像编码、解码方法及装置
KR102380579B1 (ko) 비디오 데이터에 관련된 신택스 엘리먼트를 나타내는 이진 심볼들의 시퀀스의 컨텍스트-적응적 이진 산술 코딩을 위한 방법 및 디바이스
US11818397B2 (en) Sparse matrix representation using a boundary of non-zero coefficients
JP2014096638A (ja) 画像符号化装置、画像符号化方法、画像符号化プログラム、画像復号装置、画像復号方法及び画像復号プログラム
Kahu et al. JPEG-based variable block-size image compression using CIE La* b* color space
CN113992635A (zh) 多媒体数据处理方法、装置、设备及存储介质
WO2023044868A1 (zh) 视频编解码方法、设备、系统、及存储介质
CN101310534A (zh) 使用随机域模型改进图片和视频压缩以及帧速率上转换的方法和设备
Li et al. Enlarging context with low cost: Efficient arithmetic coding with trimmed convolution
WO2023065890A1 (zh) 多媒体数据处理方法、装置、计算机设备、计算机可读存储介质及计算机程序产品
US20240127583A1 (en) Systems and methods for end-to-end feature compression in coding of multi-dimensional data
JP2023521678A (ja) 組み合わせられた可逆及び非可逆符号化のための方法及びシステム
CN115398923A (zh) 视频编解码中支持不同颜色格式的系数编解码

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 21779779

Country of ref document: EP

Kind code of ref document: A1

ENP Entry into the national phase

Ref document number: 2021779779

Country of ref document: EP

Effective date: 20221005

NENP Non-entry into the national phase

Ref country code: DE