WO2019001095A1 - 一种视频编码方法、解码方法、装置及电子设备 - Google Patents

一种视频编码方法、解码方法、装置及电子设备 Download PDF

Info

Publication number
WO2019001095A1
WO2019001095A1 PCT/CN2018/083236 CN2018083236W WO2019001095A1 WO 2019001095 A1 WO2019001095 A1 WO 2019001095A1 CN 2018083236 W CN2018083236 W CN 2018083236W WO 2019001095 A1 WO2019001095 A1 WO 2019001095A1
Authority
WO
WIPO (PCT)
Prior art keywords
image
distortion
distorted
distorted image
pixel point
Prior art date
Application number
PCT/CN2018/083236
Other languages
English (en)
French (fr)
Inventor
周璐璐
武晓阳
王莉
浦世亮
Original Assignee
杭州海康威视数字技术股份有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 杭州海康威视数字技术股份有限公司 filed Critical 杭州海康威视数字技术股份有限公司
Publication of WO2019001095A1 publication Critical patent/WO2019001095A1/zh

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/169—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding
    • H04N19/186—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being a colour or a chrominance component
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/102—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or selection affected or controlled by the adaptive coding
    • H04N19/124—Quantisation
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/134—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or criterion affecting or controlling the adaptive coding
    • H04N19/146—Data rate or code amount at the encoder output
    • H04N19/147—Data rate or code amount at the encoder output according to rate distortion criteria
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/169—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding
    • H04N19/17—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being an image region, e.g. an object
    • H04N19/172—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being an image region, e.g. an object the region being a picture, frame or field
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/80—Details of filtering operations specially adapted for video compression, e.g. for pixel interpolation
    • H04N19/82—Details of filtering operations specially adapted for video compression, e.g. for pixel interpolation involving filtering within a prediction loop
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/90—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using coding techniques not provided for in groups H04N19/10-H04N19/85, e.g. fractals
    • H04N19/91—Entropy coding, e.g. variable length coding [VLC] or arithmetic coding

Definitions

  • the present application relates to the field of video codec technology and image processing technology, and in particular, to a video encoding method, and a video decoding method, and related devices and electronic devices.
  • the original video data is subjected to various processing, such as: prediction, transform, quantization, reconstruction, filtering, etc., in which the processed video data may have been pixel-biased relative to the original video data. Shifting, causing visual impairment or artifacts; in addition, in the block-based hybrid coding framework adopted by most video coding systems, because adjacent coding blocks use different coding parameters, such as different transform quantization processes, different prediction methods, Different reference images, etc., the error magnitude and distribution characteristics introduced by each block are independent of each other, and the adjacent block boundaries will produce discontinuities and block effects. These distortions not only affect the subjective and objective quality of the reconstructed image. If the reconstructed image is used as the reference image of the subsequent encoded pixels, it will also affect the prediction accuracy of the subsequent encoded pixels and affect the size of the final video bitstream.
  • various processing such as: prediction, transform, quantization, reconstruction, filtering, etc.
  • a de-distortion filter module is often added after the reconstruction module, as shown in FIG.
  • the traditional de-distortion filter is usually to artificially generalize the characteristics of the distorted image, artificially design the filter structure, and configure the filter coefficients based on experiments and experience, such as the video coding standard H.264 standard, H.265 standard.
  • Deblocking Filter (DF), etc. also has a class of de-distortion filters, which adaptively configure filter structure, filter coefficients, etc. based on some statistical information of local image regions, such as H.265 Sample adaptive offset (SAO) used in the standard, Adaptive Loop Filter (ALF) used in video encoding and decoding, etc., such adaptive filters have high complexity.
  • SAO Sample adaptive offset
  • ALF Adaptive Loop Filter
  • the encoding end needs to write the filter-related parameters that depend on the local statistical information into the code stream to ensure the consistency of the codec, which undoubtedly increases the number of coded bits.
  • deep neural networks can adaptively construct feature descriptions driven by training data, which has higher flexibility and universality.
  • Deep learning has been very successful in many high-level computer vision problems, such as image classification, target detection, and target segmentation.
  • SRCNN super-resolution Convolutional Neural Network
  • CNN Convolutional Neural Network
  • ARCNN Artifacts Reduction Convolutional Neural Network
  • VRCNN variable-filter-size Residue-learning Convolutional Neural Network
  • CNN-based end-to-end image de-distortion algorithm avoids image preprocessing, artificially designing filter coefficients, etc. It automatically learns image distortion features and compensation methods through data driving, which is easier to use, better generalized, and better performance. Obviously, it is especially suitable for the field of image/video compression that combines multiple distortions.
  • An object of the present application is to provide a video encoding method, a decoding method, an apparatus, and an electronic device, so as to implement a de-distortion processing of a distorted image by using a CNN model different from the related technology in a video encoding and decoding process.
  • the specific technical solutions are as follows:
  • An embodiment of the present application provides a video encoding method, including:
  • the edge information component representing a distortion feature of the distortion image relative to the current original frame image
  • the CNN model is training based on a preset training set
  • the preset training set includes an original sample image, a distorted image color component of the plurality of distorted images corresponding to the original sample image, and an edge information component corresponding to each distorted image
  • Entropy encoding the entropy encoded data to obtain an encoded video bitstream the to-entropy encoded data is generated by video encoding the current original frame image, and includes side information for generating the side information component.
  • the distorted image is a reconstructed image generated during video encoding of a current original frame image in a video to be encoded;
  • the distortion image is obtained by filtering the reconstructed image by using other preset filtering methods; the reconstructed image is generated during video encoding of the current original frame image in the video to be encoded.
  • the side information component includes at least one of the following distortion characteristics:
  • generating an edge information component corresponding to the distortion image includes:
  • determining a distortion level value of each pixel of the distorted image includes:
  • generating the side information component corresponding to the distortion image by using the obtained distortion degree value of each pixel point based on the position of each pixel point of the distortion image including:
  • the processed distortion level value of each pixel point is determined as the component value of the same position of the pixel point in the side information component corresponding to the distorted image.
  • the embodiment of the present application further provides a video encoding apparatus, including:
  • a first image obtaining module configured to obtain a distorted image; the distorted image is generated during video encoding of a current original frame image in the video to be encoded;
  • a first generating module configured to generate an edge information component corresponding to the distortion image; and the edge information component represents a distortion feature of the distortion image relative to the current original frame image;
  • a first filtering processing module configured to input a distortion image color component of the distortion image and the edge information component into a pre-established CNN model for convolution filtering processing to obtain a de-distorted image color component; wherein the CNN model
  • the preset training set includes the original sample image, the distorted image color component of the plurality of distorted images corresponding to the original sample image, and the side information component corresponding to each of the distorted images;
  • An entropy coding module configured to perform entropy coding on the entropy coded data to obtain a coded video bitstream; the to-be entropy coded data is generated by video coding the current original frame image, and includes Side information of the information component.
  • the distorted image is a reconstructed image generated during video encoding of a current original frame image in a video to be encoded;
  • the distortion image is obtained by filtering the reconstructed image by using other preset filtering methods, and the reconstructed image is generated during video encoding of the current original frame image in the video to be encoded.
  • the side information component includes at least one of the following distortion characteristics:
  • the first generating module is specifically configured to determine a distortion degree value of each pixel point of the distorted image; and use each acquired pixel based on a position of each pixel point of the distorted image.
  • a distortion level value of the point, the side information component corresponding to the distortion image is generated, wherein each component value included in the side information component corresponds to a pixel point at the same position on the distortion image.
  • the first generating module is configured to acquire a quantization parameter of each coding region of the current original frame image, and obtain a quantization parameter of a coding region where each pixel of the distortion image is located. And determining a distortion degree value for each pixel of the distortion image.
  • the first generating module is specifically configured to:
  • the processed distortion level value of each pixel point is determined as the component value of the same position of the pixel point in the side information component corresponding to the distorted image.
  • the embodiment of the present application further provides an electronic device, including a processor, a communication interface, a memory, and a communication bus; wherein the processor, the communication interface, and the memory complete communication with each other through the communication bus;
  • a memory for storing a computer program
  • the processor when used to execute a program stored on the memory, implements any of the above video encoding method steps.
  • the embodiment of the present application further provides a computer readable storage medium, where the computer readable storage medium stores a computer program, and when the computer program is executed by the processor, implements any of the above video encoding method steps.
  • the embodiment of the present application further provides a computer program, which is implemented by a processor to implement any of the above video encoding method steps.
  • the used CNN model is obtained by training based on a preset training set, where the preset training set includes an original sample image, and a distortion image color of multiple distortion images corresponding to the original sample image.
  • a component, and an edge information component corresponding to each of the distorted images can represent a distortion characteristic of the distorted image relative to the original sample image; in the process of performing video encoding, the distortion generated in the process of video encoding the current original frame image
  • the image first generates an edge information component corresponding to the distorted image, and then inputs the distorted image color component and the side information component of the distorted image into a pre-established CNN model for convolution filtering processing to obtain a de-distorted image color component.
  • the distortion image is de-distorted using a CNN model different from the related technology.
  • An embodiment of the present application provides a video decoding method, including:
  • the distorted image is generated during a process of video decoding the current entropy decoded data
  • the CNN model is training based on a preset training set Obtained, the preset training set includes an original sample image, a distorted image color component of the plurality of distorted images corresponding to the original sample image, and an edge information component corresponding to each distorted image.
  • the distortion image is a reconstructed image generated during video decoding of the current entropy decoded data
  • the distortion image is obtained by filtering the reconstructed image by using another preset filtering method, and the reconstructed image is generated during video decoding of the current entropy decoded data.
  • the obtained de-distorted image color component is used for video decoding of subsequent entropy decoded data in the video bitstream;
  • the side information component represents at least one of the following distortion features:
  • generating, according to the side information included in the current entropy decoded data, the side information component corresponding to the distortion image including:
  • acquiring, from the current entropy decoded data, a distortion level value of each pixel of the distortion image including:
  • generating the side information component corresponding to the distortion image by using the obtained distortion degree value of each pixel point based on the position of each pixel point of the distortion image including:
  • the processed distortion level value of each pixel point is determined as the component value of the same position of the pixel point in the side information component corresponding to the distorted image.
  • the embodiment of the present application further provides a video decoding apparatus, including:
  • An entropy decoding module configured to perform entropy decoding on the received video bitstream to obtain current entropy decoded data
  • a second image obtaining module configured to obtain a distorted image; the distorted image is generated in a process of performing video decoding on the current entropy decoded data;
  • a second generating module configured to generate an edge information component corresponding to the distortion image based on edge information included in the current entropy decoded data; the edge information component indicates that the distortion image corresponds to the current entropy decoded data Distortion characteristics of the original frame image;
  • a second filtering processing module configured to input a distortion image color component of the distortion image and the edge information component into a pre-established CNN model for convolution filtering processing to obtain a de-distorted image color component; wherein the CNN model
  • the preset training set includes an original sample image, a distorted image color component of the plurality of distorted images corresponding to the original sample image, and an edge information component corresponding to each distorted image.
  • the distortion image is a reconstructed image generated during video decoding of the current entropy decoded data
  • the distortion image is obtained by filtering the reconstructed image by using another preset filtering method, and the reconstructed image is generated during video decoding of the current entropy decoded data.
  • the obtained de-distorted image color component is used for video decoding of subsequent entropy decoded data in the video bitstream;
  • the side information component represents at least one of the following distortion features:
  • the second generating module is specifically configured to acquire, from the current entropy decoded data, a distortion degree value of each pixel of the distorted image; and based on each pixel of the distorted image a position of a point, using the obtained distortion degree value of each pixel, to generate an edge information component corresponding to the distortion image, wherein each of the component values included in the side information component and a pixel point at the same position on the distortion image Corresponding.
  • the second generating module is configured to acquire, from the current entropy decoded data, a quantization parameter of each coding region of the original frame image corresponding to the current entropy decoded data, where The quantization parameter of the coding region where each pixel point of the distortion image is determined is determined as the distortion degree value of each pixel point of the distortion image.
  • the second generating module is specifically configured to:
  • the processed distortion level value of each pixel point is determined as the component value of the same position of the pixel point in the side information component corresponding to the distorted image.
  • the embodiment of the present application further provides an electronic device, including a processor, a communication interface, a memory, and a communication bus; wherein the processor, the communication interface, and the memory complete communication with each other through the communication bus;
  • a memory for storing a computer program
  • the processor when used to execute a program stored on the memory, implements any of the above video decoding method steps.
  • the embodiment of the present application further provides a computer readable storage medium, where the computer readable storage medium stores a computer program, and when the computer program is executed by the processor, implements any of the above video decoding method steps.
  • the embodiment of the present application further provides a computer program, which is implemented by a processor to implement any of the above video decoding method steps.
  • the used CNN model is obtained by training based on a preset training set, where the preset training set includes an original sample image, and a distortion image color of multiple distortion images corresponding to the original sample image. a component, and an edge information component corresponding to each of the distorted images, the side information component can represent a distortion feature of the distorted image relative to the original sample image;
  • the current entropy decoded data is obtained from the received video bitstream, and
  • the side information component corresponding to the distorted image is first generated, and then the distorted image color component and the side information component of the distorted image are input into a pre-established convolutional neural network model. Convolution filtering process to obtain the de-distorted image color component.
  • the distortion image is de-distorted using a CNN model different from the related technology.
  • FIG. 1 is a schematic structural diagram of a video coding system in a video coding standard
  • FIG. 2 is a schematic structural diagram of a CNN model provided by an embodiment of the present application.
  • FIG. 3 is a schematic diagram of a data flow of a process of a de-distortion filtering process according to an embodiment of the present disclosure
  • FIG. 4 is a schematic diagram of obtaining a distortion image color component of a distorted image in an embodiment of the present application
  • FIG. 5A is a schematic diagram of side information components in an embodiment of the present application.
  • FIG. 5B is a second schematic diagram of side information components in the embodiment of the present application.
  • FIG. 6 is a flowchart of a video encoding method according to an embodiment of the present application.
  • FIG. 7 is a flowchart of a video encoding method according to an embodiment of the present application.
  • FIG. 8A is a schematic structural diagram of a video encoding system according to an embodiment of the present application.
  • FIG. 8B is a second schematic structural diagram of a video encoding system according to an embodiment of the present application.
  • FIG. 9 is a flowchart of a video decoding method according to an embodiment of the present application.
  • FIG. 10 is a flowchart of a video decoding method according to an embodiment of the present application.
  • 11A is a schematic structural diagram of a video decoding system according to an embodiment of the present application.
  • 11B is a second schematic structural diagram of a video decoding system according to an embodiment of the present application.
  • 11C is a third schematic structural diagram of a video decoding system according to an embodiment of the present application.
  • FIG. 12 is a flowchart of a CNN model training method according to an embodiment of the present application.
  • FIG. 13 is a schematic structural diagram of a video encoding apparatus according to an embodiment of the present disclosure.
  • FIG. 14 is a schematic structural diagram of an electronic device according to an embodiment of the present disclosure.
  • FIG. 15 is a schematic structural diagram of a video decoding apparatus according to an embodiment of the present disclosure.
  • FIG. 16 is a schematic structural diagram of an electronic device according to an embodiment of the present application.
  • the embodiment of the present application provides a video encoding solution, where the video encoding is performed on the current original frame image in the video to be encoded.
  • the de-distorted image may be used as a reference image for the current original frame image Subsequent frame images are video encoded.
  • the embodiment of the present application further provides a video decoding solution, in which the entropy decoding is performed on the received video bitstream to obtain the current entropy decoded data, and the current entropy is a distortion image generated during decoding of the data for video decoding, based on side information included in the current entropy decoded data, generating a corresponding side information component representing a distortion feature of the original frame image corresponding to the current entropy decoded data of the distortion image, and Combining the generated side information component with the distorted image color component of the distorted image as an input of a pre-established CNN model, after performing convolution filtering processing by the CNN model, outputting a de-distorted image color component for generating de-distortion An image, the de-distorted image may be used as a reference image for performing video decoding on the subsequent entropy decoded data of the current entropy decoded data, or as a to-be-played frame
  • the CNN model is obtained by training based on a preset training set, and the preset training set includes an original sample image and a plurality of distortion images corresponding to the original sample image.
  • the distorted image color component and the side information component corresponding to each distorted image are obtained by performing multiple iterations according to the preset network structure based on the initialized network parameters.
  • the training set may include an original sample image, and perform image processing in the video encoding and decoding on the original sample image to obtain a plurality of distorted images with different distortion characteristics, thereby obtaining distortion image color components of the plurality of distorted images, and each distortion.
  • the training set may also include a plurality of original sample images, and perform image processing in the above video codec for each original sample image, respectively, to obtain a plurality of distorted images with different distortion characteristics, thereby obtaining distortion image color components of the plurality of distorted images, And side information components corresponding to each of the distorted images.
  • the side information component representing the distorted feature of the distorted image relative to the original image is used as the input of the model, that is, the image capable of representing the distorted image is introduced.
  • Distortion feature information and through model training and practical application experiments for various side information components, it is determined that when using some specific side information components, it can train a CNN model with stronger generalization ability, that is, on average In the sense, the de-distortion ability of a large number of distorted images will be stronger, and thus the distortion image can be better de-distorted to obtain a de-distorted image color component closer to the original image color component.
  • the related art proposes separately training a set of network parameters for images of different degrees of distortion, and loading a corresponding set of network parameters according to the degree of distortion of the image in the current application in actual use.
  • a large number of parameters waste storage space, and frequent loading of parameters in an application is an unnecessary waste of computing resources.
  • the degree of distortion of the distorted image relative to the original image can be generated.
  • the side information component, the preset training set may include a plurality of distorted images of different degrees of distortion, and the training of the CNN model and the de-distorting filtering of the distorted image using the trained CNN model are introduced to accurately represent the distorted image.
  • the information of the degree of distortion can be used to train the CNN model of the distorted image adapted to various degrees of distortion, so that a distorted effect can be obtained by using only one set of network parameters for the distorted image with different degrees of distortion.
  • FIG. 2 is a schematic structural diagram of a CNN model implementing the above solution, including: an edge information component generating module 21, a CNN 22, and a network training module 23;
  • CNN 22 can include the following three-layer structure:
  • the input layer processing unit 221 is configured to receive an input of the CNN, where the distortion image color component of the distorted image is included in the solution, and the side information component of the distorted image; and the input data is subjected to a convolution filtering process of the first layer;
  • the hidden layer processing unit 222 performs at least one layer of convolution filtering processing on the output data of the input layer processing unit 221.
  • the output layer processing unit 223 performs convolution filtering processing on the output data of the hidden layer processing unit 222, and outputs the result as a de-distorted image color component for generating a de-distorted image.
  • FIG. 3 is a schematic diagram of a data flow for implementing a de-distortion filtering process flow in the above solution, wherein a distorted image color component of the distorted image and an edge information component of the distorted image are input as input data into a pre-trained CNN model,
  • the CNN model can be represented by the CNN of the preset structure and the configured network parameter set, and the input data is subjected to convolution filtering processing of the input layer, the hidden layer and the output layer to obtain de-distorted image data.
  • the input data as the CNN model may include one or more side information components according to actual needs, and may also include one or more distorted image color components, for example, including at least One of the R color component, the G color component, and the B color component, correspondingly, includes one or more de-distorted image color components.
  • the process of performing video codec there may be distortion only for one of the color components, and only the color component of the distorted image may be used as input data during de-distortion processing, such as two types. If there is distortion in the color component, the two color components of the distorted image are taken as input data, and correspondingly, the corresponding de-distorted image color component is output.
  • the stored data of each pixel of an image including the values of all the color components of the pixel, in the embodiment of the present application, when obtaining the distorted image color component of the distorted image, the storage from each pixel may be needed as needed.
  • the values of the desired one or more color components are extracted to obtain a distorted image color component of the distorted image.
  • the value of the R color component of each pixel is extracted therefrom, thereby obtaining the R color component of the distorted image.
  • the side information component which represents the distortion feature of the distorted image relative to the original image
  • the side information component is an expression of the distorted feature determined by the image processing process.
  • the above distortion feature may include at least one of the following distortion features:
  • the side information component can represent the degree of distortion of the distorted image relative to the original image.
  • the side information component can also represent the distorted position of the distorted image relative to the original image.
  • the image is usually divided into a plurality of non-overlapping and non-fixed coding units, and the coding unit performs predictive coding separately.
  • the distortion between coding units is usually not consistent, and pixel mutations usually occur at the boundary of the coding unit. Therefore, the boundary coordinates of the coding unit can be used as a priori edge to represent the distortion position. information.
  • the side information component can also represent the distortion type of the distorted image relative to the original image.
  • different coding units in the image may adopt different prediction modes, and different prediction modes may affect the distribution of residual data, thereby affecting distortion.
  • the characteristics of the image, therefore, the prediction mode of the coding unit can be used as an edge information that characterizes the type of distortion.
  • the side information component may be a combination of one or more of the foregoing, or may be a plurality of side information components of the foregoing one, for example, in the video encoding and decoding process, after image processing.
  • the degree of distortion of the distorted image may be represented by a parameter of physical meaning, or the degree of distortion of the distorted image may be represented by two parameters of different physical meanings. Accordingly, one or more representations may be expressed according to actual needs.
  • the side information component of the degree of distortion is used as input data of the CNN model.
  • the matrix structure of the side information component is the same as the matrix structure of the distorted image color component, wherein the coordinates [0, 0], [0, 1] represent the distortion position, and the element value 1 of the matrix represents the degree of distortion, that is,
  • the side information component can simultaneously indicate the degree of distortion and the position of the distortion.
  • the coordinates [0, 0], [0, 1], [2, 0], [2, 4] represent the distortion position
  • the element values 1 and 2 of the matrix represent the distortion type, that is, the side information component. At the same time, it can indicate the type of distortion and the position of distortion.
  • two side information components respectively illustrated in FIG. 5A and FIG. 5B may be simultaneously included.
  • the side information component may include side information components respectively corresponding to each of the distorted image color components.
  • FIG. 6 a video coding method is provided, as shown in FIG. 6, which specifically includes the following processing steps:
  • Step 61 Obtain a distorted image generated by the video encoding of the current original frame image in the video to be encoded.
  • Step 62 Generate an edge information component corresponding to the distortion image.
  • the side information component represents the distortion feature of the distortion image relative to the current original frame image, and is an expression of the distortion feature determined by the image processing process.
  • Step 63 Input the distortion image color component of the distorted image and the generated side information component into a pre-established CNN model for convolution filtering processing to obtain a de-distorted image color component.
  • the CNN model is obtained by training based on a preset training set, where the preset training set includes an original sample image, and a distortion image color component of the plurality of distorted images corresponding to the original sample image, and an edge information component corresponding to each distorted image. .
  • Step 64 Entropy coding the entropy coded data to obtain a coded video bitstream.
  • the data to be entropy encoded is generated by video encoding the current original frame image, and includes side information for generating edge information components.
  • the distortion image is de-distorted using a CNN model different from the related technology.
  • the video encoding method will be described in detail below by taking an example of the degree of distortion of the distorted image relative to the original image by using the side information component as an example.
  • FIG. 7 is a flowchart of a video encoding method according to an embodiment of the present application, which specifically includes the following processing steps:
  • Step 71 Obtain a distorted image generated during video encoding of a current original frame image in the video to be encoded.
  • the reference image for video encoding the subsequent frame image may also be subjected to inverse quantization, inverse transform, reconstruction, and de-distortion filtering.
  • the distortion image in this step may be a reconstructed image generated after the reconstruction process in the process; or as shown in FIG. 8B, the reconstructed image is filtered by using other preset filtering methods.
  • other preset filtering methods may be DF filtering methods.
  • Step 72 Determine, for the distortion image to be processed, a distortion degree value of each pixel of the distortion image.
  • the physical parameter indicating the degree of distortion may also be different.
  • the corresponding degree of distortion capable of accurately indicating the degree of distortion of the pixel may be determined as follows. value:
  • the quantization parameters of each coding region are known, that is, the quantization parameter of each coding region can be obtained, and the quantization parameter of the coding region where each pixel of the distortion image is located is determined as The degree of distortion of each pixel of the distorted image.
  • Step 73 Generate an edge information component corresponding to the distortion image by using the obtained distortion degree value of each pixel point based on the position of each pixel point of the distortion image.
  • each component value included in the side information component corresponds to a pixel point at the same position on the distorted image.
  • the side information component Since each component value included in the side information component corresponds to a pixel point of the same position on the distorted image, the side information component has the same structure as the distorted image color component of the distorted image, that is, the matrix representing the side information component and the color of the distorted image.
  • the matrix of components is of the same type.
  • the acquired distortion level value of each pixel point may be determined as the component value of the same position of the pixel point in the side information component corresponding to the distortion image, that is, directly
  • the distortion degree value of the pixel is determined as the component value corresponding to the pixel.
  • the acquired distortion degree value of each pixel point may be normalized based on the pixel value range of the distorted image to obtain the degree of distortion after processing.
  • the range of distortion values after processing is the same as the range of pixel values;
  • the processed distortion level value of each pixel point is determined as the component value of the same position of the pixel point in the side information component corresponding to the distorted image.
  • the distortion degree value of the pixel point can be standardized by the following formula:
  • norm(x) is the processed distortion degree value obtained after normalization processing
  • x is the distortion degree value of the pixel point
  • the pixel value range of the distorted image is [PIXEL MIN , PIXEL MAX ]
  • the distortion degree value of the pixel point is taken.
  • the range of values is [QP MIN , QP MAX ].
  • the side information component of the distorted image is generated, and the process of generating the side information component is also understood to be that the side information guide map corresponding to the distorted image is generated, and the side information guide map passes the side information component thereof.
  • the scheme is described by taking the structure of the CNN model including the input layer, the hidden layer, and the output layer as an example.
  • Step 74 The distortion image color component of the distorted image and the generated side information component are used as input data of the pre-established CNN model, and the first layer is subjected to convolution filtering processing by the input layer, which may be specifically as follows:
  • input data can be input to the network through respective channels.
  • the distorted image color component Y of the c y channel and the side information component M of the c m channel can be combined in the channel dimension.
  • the input data I of the c y +c m channel together form the input data I of the c y +c m channel, and perform multidimensional convolution filtering and nonlinear mapping on the input data I by using the following formula to generate n 1 image blocks represented in a sparse form:
  • F 1 (I) is the output of the input layer
  • I is the input of the convolution layer in the input layer
  • * is the convolution operation
  • W 1 is the weight coefficient of the convolution layer filter bank of the input layer
  • B 1 is the input
  • g() is a nonlinear mapping function.
  • W 1 corresponds to n 1 convolution filters, that is, n 1 convolution filters are applied to the input of the convolution layer of the input layer, and n 1 image blocks are output; convolution of each convolution filter
  • the size of the kernel is c 1 ⁇ f 1 ⁇ f 1 , c 1 is the number of input channels, and f 1 is the spatial size of each convolution kernel.
  • Step 75 The hidden layer performs further high-dimensional mapping on the image block F 1 (I) of the sparse representation of the input layer output.
  • the convolution layer number, the convolution layer connection mode, the convolution layer attribute, and the like included in the hidden layer are not limited, and various structures known at present may be adopted, but the hidden layer includes at least 1 convolution layer.
  • the hidden layer contains a N-1 (N ⁇ 2) layer convolutional layer, and the hidden layer processing is represented by:
  • F i (I) g(W i *F i-1 (I)+B i ), i ⁇ 2,3,...,N ⁇ ;
  • F i (I) represents the output of the i-th layer convolutional layer in the CNN
  • * is the convolution operation
  • W i is the weight coefficient of the i-th layer convolutional layer filter bank
  • B i is the convolution layer filter bank
  • the offset coefficient, g() is a nonlinear mapping function.
  • W i corresponds to n i convolution filters, that is, n i convolution filters are applied to the input of the i-th convolution layer, and n i image blocks are output; convolution of each convolution filter
  • the size of the kernel is c i ⁇ f i ⁇ f i , c i is the number of input channels, and f i is the spatial size of each convolution kernel.
  • g() the convolution processing expression of the hidden layer in this embodiment is:
  • F 2 (I) max (0, W 2 * F 1 (I) + B 2 ).
  • Step 76 The output layer aggregates the high-dimensional image block F N (I) output by the hidden layer, and outputs the de-distorted image color component.
  • the structure of the output layer is not limited in the embodiment of the present application, and the output layer may be a Residual Learning structure, a Direct Learning structure, or other structures.
  • the processing using the Residual Learning structure is as follows:
  • the convolution operation is performed on the output of the hidden layer to obtain the compensation residual, and then added to the input distortion image color component to obtain the de-distorted image color component, that is, the de-distorted image is obtained.
  • the output layer processing can be expressed by the following formula:
  • F(I) is the output of the output layer
  • F N (I) is the output of the hidden layer
  • * is the convolution operation
  • W N+1 is the weight coefficient of the convolutional layer filter bank of the output layer
  • B N+ 1 is an offset coefficient of a convolution layer filter bank of the output layer
  • Y is a distorted image color component to be subjected to de-distortion processing without undergoing convolution filtering processing.
  • W N+1 corresponds to n N+1 convolution filters, that is, n N+1 convolution filters are applied to the input of the N+1th convolution layer, and n N+1 image blocks are output.
  • n N+1 is the number of output de-distorted image color components, generally equal to the number of input distortion image color components. If only one de-distorted image color component is output, n N+1 is generally 1
  • the size of the convolution kernel of each convolution filter is c N+1 ⁇ f N+1 ⁇ f N+1
  • c N+1 is the number of input channels
  • f N+1 is the space of each convolution kernel The size on the top.
  • the de-distorted image color component is directly output, that is, the de-distorted image is obtained.
  • the output layer processing can be expressed by the following formula:
  • F(I) is the output of the output layer
  • F N (I) is the output of the hidden layer
  • * is the convolution operation
  • W N+1 is the weight coefficient of the convolutional layer filter bank of the output layer
  • B N+ 1 is the offset coefficient of the convolution layer filter bank of the output layer.
  • W N+1 corresponds to n N+1 convolution filters, that is, n N+1 convolution filters are applied to the input of the N+1th convolution layer, and n N+1 image blocks are output.
  • n N+1 is the number of output de-distorted image color components, generally equal to the number of input distortion image color components. If only one de-distorted image color component is output, n N+1 is generally 1
  • the size of the convolution kernel of each convolution filter is c N+1 ⁇ f N+1 ⁇ f N+1
  • c N+1 is the number of input channels
  • f N+1 is the space of each convolution kernel The size on the top.
  • the output layer adopts a Residual Learning structure, and the output layer includes a convolution layer.
  • the convolution processing expression of the output layer in this embodiment is:
  • step 77 when the data obtained by encoding the video needs to be transmitted to the decoding end, the following step 77 may be further included.
  • Step 77 Entropy coding the entropy coded data to obtain a coded video bitstream.
  • the data to be entropy encoded is generated by video encoding the current original frame image, and includes side information for generating edge information components.
  • the side information used to generate the side information component is the information carried in the video bit stream in the video encoding and decoding standard
  • the video bit stream generated in the related video coding and decoding standard is not in the embodiment of the present application. Add extra bit data.
  • the quantization parameter of the coding region where each pixel point of the distorted image is used is used as the side information
  • the video bit stream generated in the related video codec standard carries the quantization parameter, in this step, according to the related video.
  • the codec standard generates a video bitstream.
  • the side information used to generate the side information component does not exist in the video bit stream generated in the related video codec standard, the side information may be added to the entropy coded data as a type of information required for video decoding.
  • the entropy-encoded data to which the side information is added is entropy encoded to obtain a coded video bitstream, so that the decoding end can obtain the side information from the video bitstream for video decoding processing on the decoding end.
  • the embodiment of the present application further provides a video decoding method, as shown in FIG. 9, which may include the following steps:
  • Step 91 Entropy decoding the received video bitstream to obtain current entropy decoded data.
  • Step 92 Obtain a distortion image, which is generated during video decoding of the current entropy decoded data.
  • Step 93 Generate an edge information component corresponding to the distortion image based on the side information included in the current entropy decoded data, where the edge information component represents a distortion feature of the original frame image corresponding to the current entropy decoded data of the distortion image.
  • Step 94 Input the distortion image color component of the distortion image and the side information component into a pre-established CNN model for convolution filtering processing to obtain a de-distorted image color component.
  • the CNN model is obtained by training based on a preset training set, where the preset training set includes an original sample image, and a distortion image color component of the plurality of distortion images corresponding to the original sample image, and an edge information component corresponding to each distortion image. .
  • the video decoding method will be described in detail below by taking the side information component as the example of the degree of distortion of the original frame image corresponding to the current entropy decoded data.
  • FIG. 10 is a flowchart of a video decoding method according to an embodiment of the present application, which specifically includes the following processing steps:
  • Step 101 The decoding end performs entropy decoding on the received video bitstream to obtain current entropy decoded data.
  • the obtained current entropy decoded data corresponds to one frame image in the original video, and based on the current entropy decoded data, the decoding process is performed, and the decoded image corresponding to the frame image can be obtained, so as to facilitate the description of the technical solution,
  • This frame image is referred to as the original frame image corresponding to the current entropy decoded data.
  • Step 102 Obtain a distortion image that is generated during video decoding of the current entropy decoded data.
  • the current entropy decoded data obtained by entropy decoding the video bitstream includes residual data
  • the process of video decoding the current entropy decoded data includes inverse quantization and inverse transform of the residual data. Reconstruction and de-aliasing filtering processing.
  • the distortion image in this step may be a reconstructed image generated after the reconstruction process in the process; or as shown in FIG. 11B, the reconstructed image is filtered by using other preset filtering methods.
  • other preset filtering methods may be DF filtering methods.
  • Step 103 Acquire side information of the distorted image from the current entropy decoded data, and specifically may obtain a distortion level value of each pixel of the distorted image.
  • the physical parameters indicating the degree of distortion may also be different.
  • the corresponding degree of distortion capable of accurately representing the degree of distortion of the pixel may be determined as follows. value:
  • the quantization parameter of each coding region of the original frame image corresponding to the current entropy decoded data is obtained, and the quantization parameter of the coding region where each pixel point of the distortion image is located is Determine the value of the distortion level for this pixel.
  • Step 104 Generate an edge information component corresponding to the distortion image by using the obtained distortion degree value of each pixel point based on the position of each pixel point of the distortion image.
  • each component value included in the side information component corresponds to a pixel point at the same position on the distorted image.
  • the side information component Since each component value included in the side information component corresponds to a pixel point of the same position on the distorted image, the side information component has the same structure as the distorted image color component of the distorted image, that is, the matrix representing the side information component and the color of the distorted image.
  • the matrix of components is of the same type.
  • the acquired distortion level value of each pixel point may be determined as the component value of the same position of the pixel point in the side information component corresponding to the distortion image, that is, directly
  • the distortion degree value of the pixel is determined as the component value corresponding to the pixel.
  • the acquired distortion degree value of each pixel point may be normalized based on the pixel value range of the distorted image to obtain the degree of distortion after processing.
  • the range of distortion values after processing is the same as the range of pixel values;
  • the processed distortion level value of each pixel point is determined as the component value of the same position of the pixel point in the side information component corresponding to the distorted image.
  • the distortion degree value of the pixel point can be standardized by the following formula:
  • norm(x) is the processed distortion degree value obtained after normalization processing
  • x is the distortion degree value of the pixel point
  • the pixel value range of the distorted image is [PIXEL MIN , PIXEL MAX ]
  • the distortion degree value of the pixel point is taken.
  • the range of values is [QP MIN , QP MAX ].
  • the side information component of the distorted image is generated, and the process of generating the side information component is also understood to be that the side information guide map corresponding to the distorted image is generated, and the side information guide map passes the side information component thereof.
  • the scheme is described by taking the structure of the CNN model including the input layer, the hidden layer, and the output layer as an example.
  • Step 105 The distortion image color component of the distorted image and the generated side information component are used as input data of a pre-established CNN model, and the first layer of convolution filtering processing is performed by the input layer.
  • This step may be the same as step 74 in FIG. 7 above, and will not be described in detail herein.
  • Step 106 The hidden layer performs further high-dimensional mapping on the image block F 1 (I) of the sparse representation of the input layer output.
  • This step may be the same as step 75 in FIG. 7 above, and will not be described in detail herein.
  • Step 107 The output layer aggregates the high-dimensional image block F N (I) output by the hidden layer, and outputs the de-distorted image color component.
  • This step may be the same as step 76 in FIG. 7 above, and will not be described in detail herein.
  • the obtained de-distorted image color component can be used for video decoding of subsequent entropy decoded data in the video bitstream; as shown in FIG. 11C
  • the image may also be used to generate a decoded image corresponding to the original frame image corresponding to the current entropy decoded data as the to-be-played frame image.
  • the pre-established CNN model is used to perform de-distortion filtering processing on the distorted image.
  • the CNN model used for video encoding and video decoding can be the same.
  • the embodiment of the present application further provides a CNN model training method, as shown in FIG. 12, which specifically includes the following processing steps:
  • Step 121 Acquire a preset training set.
  • the preset training set includes an original sample image, a distorted image color component of the plurality of distorted images corresponding to the original sample image, and an edge information component corresponding to each distorted image, and the side information component corresponding to the distorted image represents the distorted image relative to the original sample image. Distortion characteristics. The distortion characteristics of the plurality of distorted images are different.
  • the original sample image (ie, the undistorted natural image) may be subjected to an image processing of different degrees of distortion to obtain respective corresponding distortion images, and according to the steps in the above-described de-distortion method, for each distortion
  • the image generates corresponding side information components, so that each original sample image, the corresponding distortion image, and the corresponding side information component form an image pair, and the image pairs constitute a preset training set ⁇ .
  • the training set may include an original sample image, and the image processing is performed on the original sample image to obtain a plurality of distortion images having different distortion characteristics, and side information components corresponding to each of the distortion images;
  • the training set may also include a plurality of original sample images, and the image processing is performed separately for each of the original sample images to obtain a plurality of distorted images having different distortion characteristics, and side information components corresponding to each of the distorted images.
  • Step 122 Initialize parameters of the network parameter set of the CNN for the CNN of the preset structure.
  • the initialized parameter set may be represented by ⁇ 1 , and the initialized parameters may be set according to actual needs and experience.
  • the high-level parameters related to the training such as the learning rate and the gradient descent algorithm, may be appropriately set, and various manners in the related art may be used, and detailed descriptions are not provided herein.
  • Step 123 Perform forward calculation, as follows:
  • the distortion image color component of each of the distortion images in the preset training set and the corresponding side information component are input to the CNN of the preset structure for convolution filtering processing, and the de-distorted image color component corresponding to the distortion image is obtained.
  • the CNN forward calculation of the parameter set ⁇ i is performed on the preset training set ⁇ , and the output F(Y) of the CNN, that is, the de-distorted image color component corresponding to each distortion image is obtained.
  • the current parameter set is ⁇ 1 .
  • the current parameter set ⁇ i is obtained by adjusting the parameter set ⁇ i-1 used last time. description.
  • Step 124 Determine a loss value of the de-distorted image based on the original image color component of the plurality of original sample images and the obtained de-distorted image color component.
  • MSE Mean Squared Error
  • H represents the number of pairs of images selected from the preset training set in a single training
  • I h represents the input data of the combined edge component and the distorted image color component corresponding to the hth distorted image
  • ⁇ i ) represents the de-distorted image color component calculated by the CNN forwardly in the parameter set ⁇ i for the h-th distorted image
  • X h represents the original image color component corresponding to the h-th distorted image
  • i is the current pre-existing Counts the number of calculations.
  • Step 125 Determine, according to the loss value, whether the CNN of the preset structure adopting the current parameter set converges; if not, go to step 126; if it converges, go to step 127.
  • convergence when the loss value is less than the preset loss value threshold, convergence may be determined; or when the difference between the loss value and the previous calculated loss value is less than the preset change threshold, the convergence is determined, and the application is implemented.
  • the example is not limited here.
  • Step 126 Adjust the parameters in the current parameter set to obtain the adjusted parameter set, and then proceed to step 123 for the next forward calculation.
  • the back propagation algorithm can be used to adjust the parameters in the current parameter set.
  • Step 127 The current parameter set is used as the final parameter set of the output ⁇ final , and the CNN of the preset structure adopting the final parameter set ⁇ final is used as the trained CNN model.
  • another embodiment of the present application further provides a video encoding apparatus, as shown in FIG.
  • a first image obtaining module 131 configured to obtain a distorted image; the distorted image is generated during video encoding of a current original frame image in the video to be encoded;
  • a first generation module 132 configured to generate an edge information component corresponding to the distortion image; and an edge information component represents a distortion feature of the distortion image relative to the current original frame image;
  • the first filtering processing module 133 is configured to input the distortion image color component and the side information component of the distorted image into a pre-established CNN model for convolution filtering processing to obtain a de-distorted image color component; wherein the CNN model is based on preset training
  • the preset training set includes the original sample image, the distorted image color component of the plurality of distorted images corresponding to the original sample image, and the side information component corresponding to each distorted image;
  • the entropy encoding module 134 is configured to entropy encode the entropy encoded data to obtain the encoded video bitstream; the entropy encoded data is generated by video encoding the current original frame image, and includes side information for generating edge information components. .
  • the distorted image may be a reconstructed image generated during video encoding of the current original frame image in the video to be encoded;
  • the distorted image may be obtained by filtering the reconstructed image by using other preset filtering methods, and the reconstructed image is generated during video encoding of the current original frame image in the encoded video.
  • the side information component may include at least one of the following distortion features:
  • the first generation module 132 may be specifically configured to determine a distortion degree value of each pixel point of the distorted image; and use the acquired distortion degree of each pixel point based on the position of each pixel point of the distorted image.
  • the value generates an edge information component corresponding to the distorted image, wherein each component value included in the side information component corresponds to a pixel point at the same position on the distorted image.
  • the first generating module 132 may be configured to obtain a quantization parameter of each coding region of the current original frame image, and determine a quantization parameter of a coding region where each pixel of the distortion image is located as a distortion. The degree of distortion of each pixel of the image.
  • the first generating module 132 may be specifically configured to:
  • the obtained distortion degree value of each pixel point is normalized to obtain the processed distortion degree value, and the processed distortion degree value ranges from the pixel value range; the pixel based on the distorted image point
  • the position of the processed distortion level of each pixel is determined as the component value of the same position of the pixel in the side information component corresponding to the distorted image.
  • another embodiment of the present application further provides an electronic device, as shown in FIG. 14, including a processor 141, a communication interface 142, and a memory. 143 and a communication bus 144, wherein the processor 141, the communication interface 142, and the memory 143 complete communication with each other via the communication bus 144;
  • a memory 143 configured to store a computer program
  • the processor 141 is configured to implement a video encoding method step when executing a program stored on the memory.
  • the video encoding method includes:
  • the distorted image is generated during video encoding of a current original frame image in the video to be encoded;
  • the side information component represents a distortion feature of the distortion image relative to the current original frame image
  • the distortion image color component and the side information component of the distorted image are input into a pre-established convolutional neural network model for convolution filtering processing to obtain a de-distorted image color component; wherein the convolutional neural network model is trained based on the preset training set Obtaining, the preset training set includes an original sample image, a distorted image color component of the plurality of distorted images corresponding to the original sample image, and an edge information component corresponding to each distorted image;
  • the entropy encoded data is entropy encoded to obtain a coded video bitstream; the entropy encoded data is generated by video encoding the current original frame image, and includes side information for generating edge information components.
  • another embodiment of the present application further provides a computer readable storage medium, where the computer readable storage medium stores a computer program,
  • the video encoding method steps are implemented when the computer program is executed by the processor.
  • the video encoding method includes:
  • the distorted image is generated during video encoding of a current original frame image in the video to be encoded;
  • the side information component represents a distortion feature of the distortion image relative to the current original frame image
  • the distortion image color component and the side information component of the distorted image are input into a pre-established convolutional neural network model for convolution filtering processing to obtain a de-distorted image color component; wherein the convolutional neural network model is trained based on the preset training set Obtaining, the preset training set includes an original sample image, a distorted image color component of the plurality of distorted images corresponding to the original sample image, and an edge information component corresponding to each distorted image;
  • the entropy encoded data is entropy encoded to obtain a coded video bitstream; the entropy encoded data is generated by video encoding the current original frame image, and includes side information for generating edge information components.
  • the distortion image is de-distorted using a CNN model different from the related technology.
  • another embodiment of the present application further provides a computer program, which is implemented when the computer program is executed by the processor.
  • the video encoding method includes:
  • the distorted image is generated during video encoding of a current original frame image in the video to be encoded;
  • the side information component represents a distortion feature of the distortion image relative to the current original frame image
  • the distortion image color component and the side information component of the distorted image are input into a pre-established convolutional neural network model for convolution filtering processing to obtain a de-distorted image color component; wherein the convolutional neural network model is trained based on the preset training set Obtaining, the preset training set includes an original sample image, a distorted image color component of the plurality of distorted images corresponding to the original sample image, and an edge information component corresponding to each distorted image;
  • the entropy encoded data is entropy encoded to obtain a coded video bitstream; the entropy encoded data is generated by video encoding the current original frame image, and includes side information for generating edge information components.
  • the distortion image is de-distorted using a CNN model different from the related technology.
  • another embodiment of the present application further provides a video decoding apparatus, as shown in FIG.
  • the entropy decoding module 151 is configured to perform entropy decoding on the received video bitstream to obtain current entropy decoded data.
  • a second image obtaining module 152 configured to obtain a distorted image; the distorted image is generated in a process of video decoding the current entropy decoded data;
  • a second generation module 153 configured to generate an edge information component corresponding to the distortion image based on the side information included in the current entropy decoding data; and the edge information component represents a distortion feature of the original frame image corresponding to the current entropy decoded data of the distortion image;
  • the second filter processing module 154 is configured to input the distortion image color component and the side information component of the distorted image into a pre-established CNN model for convolution filtering processing to obtain a de-distorted image color component; wherein the CNN model is based on preset training
  • the preset training set includes the original sample image, the distorted image color component of the plurality of distorted images corresponding to the original sample image, and the side information component corresponding to each distorted image.
  • the distorted image may be a reconstructed image generated during video decoding of the current entropy decoded data
  • the distorted image may be obtained by filtering the reconstructed image by using other preset filtering methods, and the reconstructed image is generated during the process of video decoding the current entropy decoded data.
  • the resulting de-distorted image color component may be used to video decode subsequent entropy decoded data in the video bitstream;
  • It can be used to generate a decoded image corresponding to the original frame image as the to-be-played frame image.
  • the side information component may represent at least one of the following distortion features:
  • the second generation module 153 may be specifically configured to obtain, from the current entropy decoded data, a distortion degree value of each pixel of the distorted image; and use the acquisition based on the position of each pixel of the distorted image.
  • the distortion degree value of each pixel point generates an edge information component corresponding to the distortion image, wherein each component value included in the side information component corresponds to a pixel point at the same position on the distortion image.
  • the second generating module 153 may be specifically configured to: obtain, from the current entropy decoded data, a quantization parameter of each coding region of the original frame image corresponding to the current entropy decoded data, and each of the distortion images
  • the quantization parameter of the coding region where the pixel is located is determined as the distortion degree value of each pixel of the distortion image.
  • the second generating module 153 may be specifically configured to:
  • the obtained distortion degree value of each pixel point is normalized to obtain the processed distortion degree value, and the processed distortion degree value ranges from the pixel value range; the pixel based on the distorted image point
  • the position of the processed distortion level of each pixel is determined as the component value of the same position of the pixel in the side information component corresponding to the distorted image.
  • another embodiment of the present application further provides an electronic device, as shown in FIG. 16, including a processor 161, a communication interface 162, and a memory. 163 and a communication bus 164, wherein the processor 161, the communication interface 162, and the memory 163 complete communication with each other through the communication bus 164;
  • the processor 161 is configured to implement a video decoding method step when executing a program stored on the memory.
  • the video decoding method includes:
  • the distorted image is generated during video decoding of the current entropy decoded data
  • the edge information component represents a distortion feature of the original frame image corresponding to the current entropy decoded data of the distortion image
  • the distortion image color component and the side information component of the distorted image are input into a pre-established convolutional neural network model for convolution filtering processing to obtain a de-distorted image color component; wherein the convolutional neural network model is trained based on the preset training set
  • the preset training set includes the original sample image, the distorted image color component of the plurality of distorted images corresponding to the original sample image, and the side information component corresponding to each distorted image.
  • another embodiment of the present application further provides a computer readable storage medium, where the computer readable storage medium stores a computer program,
  • the video decoding method steps are implemented when the computer program is executed by the processor.
  • the video decoding method includes:
  • the distorted image is generated during video decoding of the current entropy decoded data
  • the edge information component represents a distortion feature of the original frame image corresponding to the current entropy decoded data of the distortion image
  • the distortion image color component and the side information component of the distorted image are input into a pre-established convolutional neural network model for convolution filtering processing to obtain a de-distorted image color component; wherein the convolutional neural network model is trained based on the preset training set
  • the preset training set includes the original sample image, the distorted image color component of the plurality of distorted images corresponding to the original sample image, and the side information component corresponding to each distorted image.
  • the distortion image is de-distorted using a CNN model different from the related technology.
  • another embodiment of the present application further provides a computer program, which is implemented when the computer program is executed by the processor.
  • the video decoding method includes:
  • the distorted image is generated during video decoding of the current entropy decoded data
  • the edge information component represents a distortion feature of the original frame image corresponding to the current entropy decoded data of the distortion image
  • the distortion image color component and the side information component of the distorted image are input into a pre-established convolutional neural network model for convolution filtering processing to obtain a de-distorted image color component; wherein the convolutional neural network model is trained based on the preset training set
  • the preset training set includes the original sample image, the distorted image color component of the plurality of distorted images corresponding to the original sample image, and the side information component corresponding to each distorted image.
  • the distortion image is de-distorted using a CNN model different from the related technology.

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Compression Or Coding Systems Of Tv Signals (AREA)

Abstract

本申请实施例提供了一种视频编码方法、解码方法、装置及电子设备，包括：获得失真图像，生成失真图像对应的边信息分量，将失真图像的失真图像颜色分量以及边信息分量，输入预先建立的卷积神经网络模型进行卷积滤波处理，得到去失真图像颜色分量，对待熵编码数据进行熵编码，得到编码后的视频比特流，待熵编码数据为对当前原始帧图像进行视频编码生成的，且包括用于生成边信息分量的边信息。采用该方案，实现了在视频编解码过程中，使用一种不同于相关技术的卷积神经网络模型对失真图像进行去失真处理。

Description

一种视频编码方法、解码方法、装置及电子设备
本申请要求于2017年6月26日提交中国专利局、申请号为201710494773.3发明名称为“一种视频编码方法、解码方法、装置及电子设备”的中国专利申请的优先权，其全部内容通过引用结合在本申请中。
技术领域
本申请涉及视频编解码技术领域以及图像处理技术领域，特别是涉及一种视频编码方法，以及一种视频解码方法，及相关装置和电子设备。
背景技术
在视频编码系统中，原始视频数据被进行多种处理，如：预测、变换、量化、重建、滤波等，在这多种处理过程中，已处理的视频数据相对原始视频数据可能已经发生像素偏移，导致视觉障碍或假象；此外，在大多数视频编码系统采用的基于块的混合编码框架下，由于相邻的编码块采用不同的编码参数，如不同的变换量化过程、不同的预测方式、不同参考图像等，各块引入的误差大小及其分布特性相互独立，相邻块边界会产生不连续性，产生块效应。这些失真不但影响重建图像的主客观质量，若重建图像作为后续编码像素的参考图像，还会影响后续编码像素的预测准确性，影响最终视频比特流的大小。
因此，视频编解码系统中，往往会在重建模块之后加入去失真滤波模块，如图1所示。
然而，传统的去失真滤波器通常是人为地去归纳失真图像的特征，人为地设计滤波器结构，以实验和经验为基础配置滤波器系数，例如视频编码标准H.264标准、H.265标准中采用的去块滤波器(Deblocking Filter，DF)等；也有一类去失真滤波器，它们基于局部图像区域的一些统计信息，自适应地配置滤波器结构、滤波器系数等，如H.265标准里使用的样点自适应补偿技术(Sample Adaptive Offset，SAO)、视频编解码里使用的自适应环路滤波技术(Adaptive Loop Filter，ALF)等，这类自适应滤波器复杂度较高，且编码端需要将依赖于局部统计信息的滤波器相关参数写入码流中以保证编解码端的一致性，这无疑增加了编码比特数。
近年来，深度学习理论迅速发展，与传统依赖先验知识的特征提取算法不同，深度神经网络可在训练数据驱动下自适应地构建特征描述，具有更高的灵活性和普适性。
深度学习在许多高层次的计算机视觉问题上已经取得很大的成功，如图像分类、目标检测和目标分割等。2015年业界已经提出了基于卷积神经网络的端到端的图像超分辨率重建算法(Super Resolution Convolutional Neural Network，SRCNN)，该网络相对于当时超分辨率重建领域的最领先技术展现了明显的优势，深度学习开始在低层次计算机视觉问题上显示出巨大的潜力；又如，受卷积神经网络(Convolutional Neural Network，CNN)在超分辨率恢复中的应用启发，业界内提出使用训练得到的多层去失真卷积神经网络(Artifacts Reduction Convolutional Neural Network，ARCNN)对JPEG编码后图像进行后处理滤波以去除图像失真，相对传统的滤波方法获得了明显的主、客观质量提升。
目前，业界内还提出使用训练得到的多层残差学习卷积神经网络(Variable-filter-size Residue-learning Convolutional Neural Network，VRCNN)替代传统的去块滤波技术和自适应样点补偿技术，对帧内编码后的图像进行后处理滤波，实验结果表明相对于上述传统的环内滤波方法，CNN使得重建图像主、客观质量获得了大幅提升。
基于CNN的端到端的图像去失真算法，避免了图像预处理、人工设计滤波器系数等过程，通过数据驱动自动学习图像失真特征及补偿方法，使用更简便、泛化性更佳、性能提升更明显，特别适用于糅合了多种失真的图像/视频压缩领域。
发明内容
本申请实施例的目的在于提供一种视频编码方法、解码方法、装置及电子设备，以实现在视频编解码过程中，使用一种不同于相关技术的CNN模型对失真图像进行去失真处理。具体技术方案如下：
本申请实施例提供一种视频编码方法，包括：
获得失真图像；所述失真图像为对待编码视频中的当前原始帧图像进行 视频编码的过程中生成的；
生成所述失真图像对应的边信息分量；所述边信息分量表示所述失真图像相对所述当前原始帧图像的失真特征；
将所述失真图像的失真图像颜色分量以及所述边信息分量，输入预先建立的CNN模型进行卷积滤波处理，得到去失真图像颜色分量；其中，所述CNN模型为基于预设训练集进行训练得到的，所述预设训练集包括原始样本图像、原始样本图像对应的多个失真图像的失真图像颜色分量、以及每个失真图像对应的边信息分量；
对待熵编码数据进行熵编码，得到编码后的视频比特流；所述待熵编码数据为对所述当前原始帧图像进行视频编码生成的，且包括用于生成所述边信息分量的边信息。
在本申请的一个实施例中，所述失真图像为对待编码视频中的当前原始帧图像进行视频编码的过程中生成的重建图像；或者
所述失真图像为对重建图像采用其他预设滤波方法进行滤波后得到的；所述重建图像为对待编码视频中的当前原始帧图像进行视频编码的过程中生成的。
在本申请的一个实施例中，所述边信息分量至少包括表示如下失真特征之一：
表示所述失真图像相对所述当前原始帧图像的失真程度；
表示所述失真图像相对所述当前原始帧图像的失真位置；
表示所述失真图像相对所述当前原始帧图像的失真类型。
在本申请的一个实施例中，生成所述失真图像对应的边信息分量，包括：
确定所述失真图像每个像素点的失真程度值；
基于所述失真图像各像素点的位置，使用获取的各像素点的失真程度值，生成所述失真图像对应的边信息分量；其中，所述边信息分量包括的每个分量值与所述失真图像上相同位置的像素点相对应。
在本申请的一个实施例中，确定所述失真图像每个像素点的失真程度值，包括：
获取所述当前原始帧图像的每个编码区域的量化参数，将所述失真图像每个像素点所在编码区域的量化参数，确定为所述失真图像每个像素点的失真程度值。
在本申请的一个实施例中，基于所述失真图像各像素点的位置，使用获取的所述各像素点的失真程度值，生成所述失真图像对应的边信息分量，包括：
基于所述失真图像各像素点的位置，将获取的每个像素点的失真程度值，确定为所述失真图像对应的边信息分量中该像素点相同位置的分量值；或者
基于所述失真图像的像素值范围，对获取的所述各像素点的失真程度值进行标准化处理，得到处理后失真程度值，所述处理后失真程度值的取值范围与所述像素值范围相同；基于所述失真图像各像素点的位置，将每个像素点的处理后失真程度值，确定为所述失真图像对应的边信息分量中该像素点相同位置的分量值。
本申请实施例还提供一种视频编码装置，包括：
第一图像获得模块，用于获得失真图像；所述失真图像为对待编码视频中的当前原始帧图像进行视频编码的过程中生成的；
第一生成模块，用于生成所述失真图像对应的边信息分量；所述边信息分量表示所述失真图像相对所述当前原始帧图像的失真特征；
第一滤波处理模块，用于将所述失真图像的失真图像颜色分量以及所述边信息分量，输入预先建立的CNN模型进行卷积滤波处理，得到去失真图像颜色分量；其中，所述CNN模型为基于预设训练集进行训练得到的，所述预设训练集包括原始样本图像、原始样本图像对应的多个失真图像的失真图像颜色分量、以及每个失真图像对应的边信息分量；
熵编码模块，用于对待熵编码数据进行熵编码，得到编码后的视频比特流；所述待熵编码数据为对所述当前原始帧图像进行视频编码生成的，且包 括用于生成所述边信息分量的边信息。
在本申请的一个实施例中，所述失真图像为对待编码视频中的当前原始帧图像进行视频编码的过程中生成的重建图像；或者
所述失真图像为对重建图像采用其他预设滤波方法进行滤波后得到的，所述重建图像为对待编码视频中的当前原始帧图像进行视频编码的过程中生成的。
在本申请的一个实施例中，所述边信息分量至少包括表示如下失真特征之一：
表示所述失真图像相对所述当前原始帧图像的失真程度；
表示所述失真图像相对所述当前原始帧图像的失真位置；
表示所述失真图像相对所述当前原始帧图像的失真类型。
在本申请的一个实施例中，所述第一生成模块，具体用于确定所述失真图像每个像素点的失真程度值；并基于所述失真图像各像素点的位置，使用获取的各像素点的失真程度值，生成所述失真图像对应的边信息分量，其中，所述边信息分量包括的每个分量值与所述失真图像上相同位置的像素点相对应。
在本申请的一个实施例中，所述第一生成模块，具体用于获取所述当前原始帧图像的每个编码区域的量化参数，将所述失真图像每个像素点所在编码区域的量化参数，确定为所述失真图像每个像素点的失真程度值。
在本申请的一个实施例中，所述第一生成模块，具体用于：
基于所述失真图像各像素点的位置，将获取的每个像素点的失真程度值，确定为所述失真图像对应的边信息分量中该像素点相同位置的分量值；或者
基于所述失真图像的像素值范围，对获取的所述各像素点的失真程度值进行标准化处理，得到处理后失真程度值，所述处理后失真程度值的取值范围与所述像素值范围相同；基于所述失真图像各像素点的位置，将每个像素点的处理后失真程度值，确定为所述失真图像对应的边信息分量中该像素点相同位置的分量值。
本申请实施例还提供一种电子设备，包括处理器、通信接口、存储器和通信总线；其中，处理器、通信接口、存储器通过通信总线完成相互间的通信；
存储器，用于存放计算机程序；
处理器，用于执行存储器上所存放的程序时，实现上述任一视频编码方法步骤。
本申请实施例还提供一种计算机可读存储介质，所述计算机可读存储介质内存储有计算机程序，所述计算机程序被处理器执行时实现上述任一视频编码方法步骤。
本申请实施例还提供一种计算机程序，所述计算机程序被处理器执行时实现上述任一视频编码方法步骤。
本申请实施例提供的视频编码方法中，所使用的CNN模型为基于预设训练集进行训练得到的，预设训练集包括原始样本图像，以及原始样本图像对应的多个失真图像的失真图像颜色分量，以及每个失真图像对应的边信息分量，边信息分量能够表示失真图像相对原始样本图像的失真特征；在进行视频编码的过程中，针对当前原始帧图像进行视频编码的过程中生成的失真图像，首先生成失真图像对应的边信息分量，然后将失真图像的失真图像颜色分量以及边信息分量，输入预先建立的CNN模型进行卷积滤波处理，得到去失真图像颜色分量。从而实现了在视频编码过程中，使用一种不同于相关技术的CNN模型对失真图像进行去失真处理。
本申请实施例提供一种视频解码方法，包括：
对接收的视频比特流进行熵解码，得到当前熵解码数据；
获得失真图像；所述失真图像为对所述当前熵解码数据进行视频解码的过程中生成的；
基于所述当前熵解码数据中包括的边信息，生成所述失真图像对应的边信息分量；所述边信息分量表示所述失真图像相对所述当前熵解码数据对应的原始帧图像的失真特征；
将所述失真图像的失真图像颜色分量以及所述边信息分量，输入预先建立的CNN模型进行卷积滤波处理，得到去失真图像颜色分量；其中，所述CNN模型为基于预设训练集进行训练得到的，所述预设训练集包括原始样本图像、原始样本图像对应的多个失真图像的失真图像颜色分量、以及每个失真图像对应的边信息分量。
在本申请的一个实施例中，所述失真图像为对所述当前熵解码数据进行视频解码的过程中生成的重建图像；或者
所述失真图像为对重建图像采用其他预设滤波方法进行滤波后得到的，所述重建图像为对所述当前熵解码数据进行视频解码的过程中生成的。
在本申请的一个实施例中，得到的所述去失真图像颜色分量，用于对所述视频比特流中的后续熵解码数据进行视频解码；和/或
用于生成所述原始帧图像对应的解码后图像，作为待播放帧图像。
在本申请的一个实施例中，所述边信息分量至少表示如下失真特征之一：
表示所述失真图像相对所述原始帧图像的失真程度；
表示所述失真图像相对所述原始帧图像的失真位置；
表示所述失真图像相对所述原始帧图像的失真类型。
在本申请的一个实施例中，基于所述当前熵解码数据中包括的边信息，生成所述失真图像对应的边信息分量，包括：
从所述当前熵解码数据中，获取所述失真图像每个像素点的失真程度值；
基于所述失真图像各像素点的位置，使用获取的各像素点的失真程度值，生成所述失真图像对应的边信息分量；其中，所述边信息分量包括的每个分量值与所述失真图像上相同位置的像素点相对应。
在本申请的一个实施例中，从所述当前熵解码数据中，获取所述失真图像每个像素点的失真程度值，包括：
从所述当前熵解码数据中，获取所述当前熵解码数据对应的原始帧图像的每个编码区域的量化参数，将所述失真图像每个像素点所在编码区域的量 化参数，确定为所述失真图像每个像素点的失真程度值。
在本申请的一个实施例中，基于所述失真图像各像素点的位置，使用获取的所述各像素点的失真程度值，生成所述失真图像对应的边信息分量，包括：
基于所述失真图像各像素点的位置，将获取的每个像素点的失真程度值，确定为所述失真图像对应的边信息分量中该像素点相同位置的分量值；或者
基于所述失真图像的像素值范围，对获取的所述各像素点的失真程度值进行标准化处理，得到处理后失真程度值，所述处理后失真程度值的取值范围与所述像素值范围相同；基于所述失真图像各像素点的位置，将每个像素点的处理后失真程度值，确定为所述失真图像对应的边信息分量中该像素点相同位置的分量值。
本申请实施例还提供一种视频解码装置，包括：
熵解码模块，用于对接收的视频比特流进行熵解码，得到当前熵解码数据；
第二图像获得模块，用于获得失真图像；所述失真图像为对所述当前熵解码数据进行视频解码的过程中生成的；
第二生成模块，用于基于所述当前熵解码数据中包括的边信息，生成所述失真图像对应的边信息分量；所述边信息分量表示所述失真图像相对所述当前熵解码数据对应的原始帧图像的失真特征；
第二滤波处理模块，用于将所述失真图像的失真图像颜色分量以及所述边信息分量，输入预先建立的CNN模型进行卷积滤波处理，得到去失真图像颜色分量；其中，所述CNN模型为基于预设训练集进行训练得到的，所述预设训练集包括原始样本图像、原始样本图像对应的多个失真图像的失真图像颜色分量、以及每个失真图像对应的边信息分量。
在本申请的一个实施例中，所述失真图像为对所述当前熵解码数据进行视频解码的过程中生成的重建图像；或者
所述失真图像为对重建图像采用其他预设滤波方法进行滤波后得到的， 所述重建图像为对所述当前熵解码数据进行视频解码的过程中生成的。
在本申请的一个实施例中，得到的所述去失真图像颜色分量，用于对所述视频比特流中的后续熵解码数据进行视频解码；和/或
用于生成所述原始帧图像对应的解码后图像，作为待播放帧图像。
在本申请的一个实施例中，所述边信息分量至少表示如下失真特征之一：
表示所述失真图像相对所述原始帧图像的失真程度；
表示所述失真图像相对所述原始帧图像的失真位置；
表示所述失真图像相对所述原始帧图像的失真类型。
在本申请的一个实施例中，所述第二生成模块，具体用于从所述当前熵解码数据中，获取所述失真图像每个像素点的失真程度值；并基于所述失真图像各像素点的位置，使用获取的各像素点的失真程度值，生成所述失真图像对应的边信息分量，其中，所述边信息分量包括的每个分量值与所述失真图像上相同位置的像素点相对应。
在本申请的一个实施例中，所述第二生成模块，具体用于从所述当前熵解码数据中，获取所述当前熵解码数据对应的原始帧图像的每个编码区域的量化参数，将所述失真图像每个像素点所在编码区域的量化参数，确定为所述失真图像每个像素点的失真程度值。
在本申请的一个实施例中，所述第二生成模块，具体用于：
基于所述失真图像各像素点的位置，将获取的每个像素点的失真程度值，确定为所述失真图像对应的边信息分量中该像素点相同位置的分量值；或者
基于所述失真图像的像素值范围，对获取的所述各像素点的失真程度值进行标准化处理，得到处理后失真程度值，所述处理后失真程度值的取值范围与所述像素值范围相同；基于所述失真图像各像素点的位置，将每个像素点的处理后失真程度值，确定为所述失真图像对应的边信息分量中该像素点相同位置的分量值。
本申请实施例还提供一种电子设备，包括处理器、通信接口、存储器和 通信总线；其中，处理器、通信接口、存储器通过通信总线完成相互间的通信；
存储器，用于存放计算机程序；
处理器，用于执行存储器上所存放的程序时，实现上述任一视频解码方法步骤。
本申请实施例还提供一种计算机可读存储介质，所述计算机可读存储介质内存储有计算机程序，所述计算机程序被处理器执行时实现上述任一视频解码方法步骤。
本申请实施例还提供一种计算机程序，所述计算机程序被处理器执行时实现上述任一视频解码方法步骤。
本申请实施例提供的视频解码方法中，所使用的CNN模型为基于预设训练集进行训练得到的，预设训练集包括原始样本图像，以及原始样本图像对应的多个失真图像的失真图像颜色分量，以及每个失真图像对应的边信息分量，边信息分量能够表示失真图像相对原始样本图像的失真特征；在进行视频解码的过程中，从接收的视频比特流中得到当前熵解码数据，并针对当前熵解码数据进行视频解码的过程中生成的失真图像，首先生成失真图像对应的边信息分量，然后将失真图像的失真图像颜色分量以及边信息分量，输入预先建立的卷积神经网络模型进行卷积滤波处理，得到去失真图像颜色分量。从而实现了在视频解码过程中，使用一种不同于相关技术的CNN模型对失真图像进行去失真处理。
当然，实施本申请的任一产品或方法并不一定需要同时达到以上所述的所有优点。
附图说明
为了更清楚地说明本申请实施例或相关技术中的技术方案，下面将对实施例或相关技术描述中所需要使用的附图作简单地介绍，显而易见地，下面描述中的附图仅仅是本申请的一些实施例，对于本领域普通技术人员来讲，在不付出创造性劳动的前提下，还可以根据这些附图获得其他的附图。
图1为视频编码标准中视频编码系统的结构示意图；
图2为本申请实施例提供的CNN模型的结构示意图；
图3为本申请实施例提供的去失真滤波处理流程的数据流示意图；
图4为本申请实施例中获得失真图像的失真图像颜色分量的示意图；
图5A为本申请实施例中边信息分量的示意图之一；
图5B为本申请实施例中边信息分量的示意图之二；
图6为本申请实施例提供的视频编码方法的流程图；
图7为本申请实施例提供的视频编码方法的流程图；
图8A为本申请实施例中视频编码系统的结构示意图之一；
图8B为本申请实施例中视频编码系统的结构示意图之二；
图9为本申请实施例提供的视频解码方法的流程图；
图10为本申请实施例提供的视频解码方法的流程图；
图11A为本申请实施例中视频解码系统的结构示意图之一；
图11B为本申请实施例中视频解码系统的结构示意图之二；
图11C为本申请实施例中视频解码系统的结构示意图之三；
图12为本申请实施例提供的CNN模型训练方法的流程图；
图13为本申请实施例提供的视频编码装置的结构示意图；
图14为本申请实施例提供的电子设备的结构示意图；
图15为本申请实施例提供的视频解码装置的结构示意图；
图16为本申请实施例提供的电子设备的结构示意图。
具体实施方式
下面将结合本申请实施例中的附图，对本申请实施例中的技术方案进行清楚、完整地描述，显然，所描述的实施例仅仅是本申请一部分实施例，而不是全部的实施例。基于本申请中的实施例，本领域普通技术人员在没有做出创造性劳动前提下所获得的所有其他实施例，都属于本申请保护的范围。
针对在视频编码过程中，如何使用CNN对失真图像进行去失真处理的问题，本申请实施例提供一种视频编码解决方案，该方案中，针对在对待编码视频中的当前原始帧图像进行视频编码的过程中生成的失真图像，生成对应的能够表示该失真图像相对当前原始帧图像的失真特征的边信息分量，并将生成的边信息分量与该失真图像的失真图像颜色分量，共同作为预先建立的CNN模型的输入，在经过该CNN模型进行卷积滤波处理之后，输出去失真图像颜色分量，用于生成去失真图像，该去失真图像可以作为参考图像，用于对该当前原始帧图像的后续帧图像进行视频编码。
相应的，在视频解码过程中，本申请实施例还提供了一种视频解码解决方案，该方案中，在对接收的视频比特流进行熵解码，得到当前熵解码数据之后，针对在对当前熵解码数据进行视频解码的过程中生成的失真图像，基于当前熵解码数据中包括的边信息，生成对应的表示该失真图像相对当前熵解码数据对应的原始帧图像的失真特征的边信息分量，并将生成的边信息分量与该失真图像的失真图像颜色分量，共同作为预先建立的CNN模型的输入，在经过该CNN模型进行卷积滤波处理之后，输出去失真图像颜色分量，用于生成去失真图像，该去失真图像可以作为参考图像，用于对该当前熵解码数据的后续熵解码数据进行视频解码，也可以作为当前熵解码数据对应的原始帧图像对应的待播放帧图像，用于视频播放。
并且，在上述视频编码解决方案以及视频解码解决方案中，CNN模型为基于预设训练集进行训练得到的，且该预设训练集包括原始样本图像，以及原始样本图像对应的多个失真图像的失真图像颜色分量，以及每个失真图像对应的边信息分量，基于初始化的网络参数，按照预设网络结构进行多次迭代训练获得。
训练集可以包括一个原始样本图像，针对该原始样本图像进行上述视频编解码中的图像处理，得到失真特征不同的多个失真图像，进而得到多个失真图像的失真图像颜色分量、以及每个失真图像对应的边信息分量；
训练集也可以包括多个原始样本图像，分别针对每个原始样本图像进行上述视频编解码中的图像处理，得到失真特征不同的多个失真图像，进而得到多个失真图像的失真图像颜色分量、以及每个失真图像对应的边信息分量。
在进行CNN模型的训练，以及使用训练的CNN模型对失真图像进行去失真滤波时，均将表示失真图像相对原始图像的失真特征的边信息分量作为模型的输入，即引入了能够表示失真图像的失真特征的信息，并通过针对各种边信息分量进行模型训练以及实际应用的试验，确定出在使用一些特定的边信息分量时，能够训练出具备更强泛化能力的CNN模型，即在平均意义上，对大量的具有差异性的失真图像的去失真能力会更强，进而能够对失真图像进行更好的去失真处理，得到更接近原始图像颜色分量的去失真图像颜色分量。
虽然目前已经存在多种基于CNN模型进行图像去失真处理的方法，但当前公开的大多数基于CNN的图像去失真滤波技术中，若使用单一预训练网络对不同失真程度的图像进行后处理滤波，在某些失真程度的图像上会存在质量提升有限或质量损失的问题，说明网络的泛化能力存在一定问题。
为了解决该问题，相关技术提出对不同失真程度的图像分别训练一套网络参数，在实际使用时根据当前应用中图像的失真程度加载相应的一套网络参数。然而，大量的参数浪费了存储空间，应用中参数的频繁加载更是一种不必要的计算资源浪费。
因此，在一种确定的视频编解码过程下，发明一种单一的，能适应不同失真程度图像的后处理滤波的CNN，具有重要的价值。
针对相关技术中仅采用一套网络参数的CNN，无法应对不同失真程度的失真图像的去失真滤波问题，本申请实施例提出的上述解决方案中，可以生成表示失真图像相对原始图像的失真程度的边信息分量，预设训练集中可以包括多个不同失真程度的失真图像，则在进行CNN模型的训练，以及使用训练的CNN模型对失真图像进行去失真滤波时，引入了能够准确表示失真图像的失真程度的信息，从而能够训练出适应于各种不同失真程度的失真图像的CNN模型，进而能够针对不同失真程度的失真图像，仅使用一套网络参数即可以获得较佳的去失真效果。
图2为实现上述解决方案的CNN模型的结构示意图，包括：边信息分量生成模块21，CNN22，网络训练模块23；
其中，CNN22可以包括如下三层结构：
输入层处理单元221，用于接收CNN的输入，本方案中包括失真图像的失真图像颜色分量，以及失真图像的边信息分量；并对输入的数据进行第一层的卷积滤波处理；
隐含层处理单元222，对输入层处理单元221的输出数据，进行至少一层的卷积滤波处理；
输出层处理单元223，对隐含层处理单元222的输出数据，进行最后一层的卷积滤波处理，输出结果作为去失真图像颜色分量，用于生成去失真图像。
图3为实现上述解决方案中的去失真滤波处理流程的数据流的示意图，其中，失真图像的失真图像颜色分量，以及失真图像的边信息分量作为输入数据，输入到预先训练的CNN模型中，CNN模型可以由预设结构的CNN和配置的网络参数集进行表示，输入数据经过输入层、隐含层和输出层的卷积滤波处理之后，得到去失真图像数据。
本申请实施例提供的上述解决方案中，作为CNN模型的输入数据，根据实际需要，可以包括一种或多种边信息分量，也可以包括一种或多种失真图像颜色分量，例如，至少包括R颜色分量、G颜色分量和B颜色分量之一，相应的，包括一种或多种去失真图像颜色分量。
例如，在进行视频编解码的过程中，可能仅针对全部颜色分量中的一种颜色分量存在失真情况，则可以在去失真处理时，仅将失真图像的该颜色分量作为输入数据，如两种颜色分量存在失真情况，则将失真图像的该两种颜色分量均作为输入数据，相应的，均输出对应的去失真图像颜色分量。
一个图像的每个像素点的存储数据，包括该像素点的所有颜色分量的值，本申请实施例中，在获得失真图像的失真图像颜色分量时，可以根据需要，从每个像素点的存储数据中，提取出需要的一种或多种颜色分量的值，从而得到失真图像的失真图像颜色分量。
如图4所示，以RGB颜色空间为例，从中提取出每个像素点的R颜色分量的值，从而得到失真图像的R颜色分量。
对于边信息分量，其表示失真图像相对原始图像的失真特征，是一种由图像处理过程确定的失真特征的表达。
在实际应用中，上述失真特征可以至少包括如下失真特征之一：
失真程度、失真位置，失真类型：
首先，边信息分量可以表示失真图像相对原始图像的失真程度。
其次，边信息分量也可以表示失真图像相对原始图像的失真位置，例如在主流的视频编解码应用中，图像通常被划分为多个不重叠且不固定大小的编码单元，编码单元分别进行预测编码及不同程度的量化处理，编码单元之间的失真通常不具有一致性，在编码单元的边界处通常会产生像素突变，因此，编码单元的边界坐标可以作为一种先验的表征失真位置的边信息。
再次，边信息分量也可以表示失真图像相对原始图像的失真类型，例如在视频编解码应用中，图像中不同编码单元可能采用不同预测模式，不同预测模式会影响残差数据的分布，从而影响失真图像的特征，因此，编码单元的预测模式可以作为一种表征失真类型的边信息。
本申请实施例提供的上述解决方案中，边信息分量可以为上述一种或多种的组合，也可以为上述一种的多个边信息分量，例如，在视频编解码过程中，经过图像处理后，可能通过一种物理含义的参量表示失真图像的失真程度，也可能通过不同物理含义的两种参量表示失真图像的失真程度，相应的，可以根据实际需要，将一种或多种均表示失真程度的边信息分量作为CNN模型的输入数据。
如图5A所示，边信息分量的矩阵结构与失真图像颜色分量的矩阵结构相同，其中，坐标[0,0]、[0,1]表示失真位置，矩阵的元素值1表示失真程度，即边信息分量同时能表示失真程度与失真位置。
又如图5B所示，坐标[0,0]、[0,1]、[2,0]、[2,4]表示失真位置，矩阵的元素值1、2表示失真类型，即边信息分量同时能表示失真类型与失真位置。
并且，本申请实施例提供的上述解决方案中，可以同时包括图5A和图5B分别所示意的两个边信息分量。
根据方案的实际应用情况和需要，当失真图像颜色分量包括多种时，边信息分量可以包括分别与每种失真图像颜色分量对应的边信息分量。
在本申请实施例提供的上述解决方案中，提出了一种视频编码方法，如图6所示，具体包括如下处理步骤：
步骤61、获得失真图像，失真图像为对待编码视频中的当前原始帧图像进行视频编码的过程中生成的。
步骤62、生成失真图像对应的边信息分量。
其中，边信息分量表示失真图像相对当前原始帧图像的失真特征，是一种由图像处理过程确定的失真特征的表达。
步骤63、将失真图像的失真图像颜色分量以及生成的边信息分量，输入预先建立的CNN模型进行卷积滤波处理，得到去失真图像颜色分量。
其中，CNN模型为基于预设训练集进行训练得到的，预设训练集包括原始样本图像，以及原始样本图像对应的多个失真图像的失真图像颜色分量，以及每个失真图像对应的边信息分量。
步骤64、对待熵编码数据进行熵编码，得到编码后的视频比特流。
其中，待熵编码数据为对当前原始帧图像进行视频编码生成的，且包括用于生成边信息分量的边信息。
应用图6所示实施例，实现了在视频编码过程中，使用一种不同于相关技术的CNN模型对失真图像进行去失真处理。
下面结合附图，以边信息分量表示失真图像相对原始图像的失真程度为例，对上述视频编码方法进行详细描述。
图7所示为本申请实施例提供的一种视频编码方法的流程图，具体包括如下处理步骤：
步骤71、获得失真图像，该失真图像为对待编码视频中的当前原始帧图像进行视频编码的过程中生成的。
如图8A所示，在基于视频编码系统，对待编码视频进行视频编码的过程 中，针对当前原始帧图像进行视频编码时，将对其进行预测、变换、量化等处理，并且，为了得到用于对后续帧图像进行视频编码的参考图像，还可以进行反量化、反变换、重建和去失真滤波的处理。
本步骤中的失真图像，如图8A所示，可以是该过程中经过重建处理后生成的重建图像；也可以如图8B所示，是对重建图像采用其他预设滤波方法进行滤波后得到的，例如，其他预设滤波方法可以是DF滤波方法。
步骤72、针对待处理的失真图像，确定失真图像每个像素点的失真程度值。
在实际应用中，对当前原始帧图像进行不同方式的视频编码之后，表示失真程度的物理参量也可能不同，本步骤中，可以采用如下方式确定出对应的能够准确表示像素点失真程度的失真程度值：
针对通过视频编码得到的上述失真图像，每个编码区域的量化参数都是已知的，即可以获取每个编码区域的量化参数，将失真图像每个像素点所在编码区域的量化参数，确定为失真图像每个像素点的失真程度值。
步骤73、基于失真图像各像素点的位置，使用获取的各像素点的失真程度值，生成失真图像对应的边信息分量。
其中，边信息分量包括的每个分量值与失真图像上相同位置的像素点相对应。
由于边信息分量包括的每个分量值与失真图像上相同位置的像素点相对应，则边信息分量与失真图像的失真图像颜色分量的结构相同，即表示边信息分量的矩阵与表示失真图像颜色分量的矩阵是同型的。
本步骤中，可以基于失真图像各像素点的位置，将获取的每个像素点的失真程度值，确定为失真图像对应的边信息分量中该像素点相同位置的分量值，即直接将每个像素点的失真程度值，确定为该像素点对应的分量值。
当失真图像的像素值范围与像素点的失真程度值的取值范围不同时，也可以基于失真图像的像素值范围，对获取的各像素点的失真程度值进行标准化处理，得到处理后失真程度值，处理后失真程度值的取值范围与像素值范 围相同；
然后基于失真图像各像素点的位置，将每个像素点的处理后失真程度值，确定为失真图像对应的边信息分量中该像素点相同位置的分量值。
本步骤中，可以采用如下公式对像素点的失真程度值进行标准化处理：
Figure PCTCN2018083236-appb-000001
其中，norm(x)为标准化处理后得到的处理后失真程度值，x为像素点的失真程度值，失真图像的像素值范围为[PIXEL MIN,PIXEL MAX]，像素点的失真程度值的取值范围为[QP MIN,QP MAX]。
通过上述步骤72和步骤73，即生成了失真图像的边信息分量，生成边信息分量的过程，也可以理解为生成了失真图像对应的边信息引导图，该边信息引导图通过其边信息分量表示失真图像的失真程度，且该边信息引导图与失真图像是等高等宽的。
本申请实施例中，以CNN模型包括输入层、隐含层和输出层的结构为例，对方案进行描述。
步骤74、将失真图像的失真图像颜色分量以及生成的边信息分量，作为预先建立的CNN模型的输入数据，由输入层进行第一层的卷积滤波处理，具体可以如下：
在CNN模型中，输入数据可以是通过各自的通道输入到网络中，本步骤中，可以将c y通道的失真图像颜色分量Y与c m通道的边信息分量M，在通道的维度上进行合并，共同组成c y+c m通道的输入数据I，并采用如下公式对输入数据I进行多维卷积滤波和非线性映射，产生n 1个以稀疏形式表示的图像块：
F 1(I)＝g(W 1*I+B 1)；
其中，F 1(I)为输入层的输出，I为输入层中卷积层的输入，*为卷积操作，W 1为输入层的卷积层滤波器组的权重系数，B 1为输入层的卷积层滤波器组的偏移系数，g()为非线性映射函数。
其中，W 1对应于n 1个卷积滤波器，即有n 1个卷积滤波器作用于输入层的卷积层的输入，输出n 1个图像块；每个卷积滤波器的卷积核的大小为c 1×f 1×f 1，c 1为输入通道数，f 1为每个卷积核在空间上的大小。
在一个具体的实施例中，该输入层的参数可以为：c 1＝2，f 1＝5，n 1＝64，使用修正线性单元(Rectified linear unit，ReLU)函数作为g()，它的函数表达式为：
g(x)＝max(0,x)；
则该实施例中输入层卷积处理表达式为：
F 1(I)＝max(0,W 1*I+B 1)。
步骤75、隐含层对输入层输出的稀疏表示的图像块F 1(I)进行进一步的高维映射。
本申请实施例中，不对隐含层中包含的卷积层层数、卷积层连接方式、卷积层属性等作限定，可以采用目前已知的各种结构，但隐含层中包含至少1个卷积层。
例如，隐含层包含N-1(N≥2)层卷积层，隐含层处理由下式表示：
F i(I)＝g(W i*F i-1(I)+B i),i∈{2,3,…,N}；
其中，F i(I)表示CNN中第i层卷积层的输出，*为卷积操作，W i为第i层卷积层滤波器组的权重系数，B i为卷积层滤波器组的偏移系数，g()为非线性映射函数。
其中，W i对应于n i个卷积滤波器，即有n i个卷积滤波器作用于第i层卷积层的输入，输出n i个图像块；每个卷积滤波器的卷积核的大小为c i×f i×f i，c i为输入通道数，f i为每个卷积核在空间上的大小。
在一个具体的实施例中，该隐含层可以包括1个卷积层，该卷积层的卷积滤波器参数为：c 2＝64，f 2＝1，n 2＝32，使用ReLU函数作为g()，则该实施例中隐含层的卷积处理表达式为：
F 2(I)＝max(0,W 2*F 1(I)+B 2)。
步骤76、输出层对隐含层输出的高维图像块F N(I)进行聚合，输出去失真图像颜色分量。
本申请实施例中不对输出层的结构作限定，输出层可以是Residual Learning结构，也可以是Direct Learning结构，或者其他的结构。
采用Residual Learning结构的处理如下：
对隐含层的输出进行卷积操作获取补偿残差，再与输入的失真图像颜色分量相加，得到去失真图像颜色分量，即得到去失真图像。输出层处理可由下式表示：
F(I)＝W N+1*F N(I)+B N+1+Y；
其中，F(I)为输出层输出，F N(I)为隐含层的输出，*为卷积操作，W N+1为输出层的卷积层滤波器组的权重系数，B N+1为输出层的卷积层滤波器组的偏移系数，Y为未经过卷积滤波处理、欲进行去失真处理的失真图像颜色分量。
其中，W N+1对应于n N+1个卷积滤波器，即有n N+1个卷积滤波器作用于第N+1层卷积层的输入，输出n N+1个图像块，n N+1为输出的去失真图像颜色分量个数，一般与输入的失真图像颜色分量的个数相等，如果只输出一种去失真图像颜色分量，则n N+1一般取值为1；每个卷积滤波器的卷积核的大小为c N+1×f N+1×f N+1，c N+1为输入通道数，f N+1为每个卷积核在空间上的大小。
采用Direct Learning结构的处理如下：
对隐含层的输出进行卷积操作后直接输出去失真图像颜色分量，即得到 去失真图像。输出层处理可由下式表示：
F(I)＝W N+1*F N(I)+B N+1；
其中，F(I)为输出层输出，F N(I)为隐含层的输出，*为卷积操作，W N+1为输出层的卷积层滤波器组的权重系数，B N+1为输出层的卷积层滤波器组的偏移系数。
其中，W N+1对应于n N+1个卷积滤波器，即有n N+1个卷积滤波器作用于第N+1层卷积层的输入，输出n N+1个图像块，n N+1为输出的去失真图像颜色分量个数，一般与输入的失真图像颜色分量的个数相等，如果只输出一种去失真图像颜色分量，则n N+1一般取值为1；每个卷积滤波器的卷积核的大小为c N+1×f N+1×f N+1，c N+1为输入通道数，f N+1为每个卷积核在空间上的大小。
在一个具体的实施例中，该输出层采用Residual Learning结构，输出层包括1个卷积层，该输出层的卷积滤波器参数为：c 3＝32，f 3＝3，n 3＝1，则该实施例中输出层的卷积处理表达式为：
F(I)＝W 3*F 3(I)+B 3+Y。
本申请实施例中，在需要将视频编码后得到的数据传输至解码端时，还可以包括如下步骤77。
步骤77、对待熵编码数据进行熵编码，得到编码后的视频比特流。
其中，待熵编码数据为对当前原始帧图像进行视频编码生成的，且包括用于生成边信息分量的边信息。
如果用于生成边信息分量的边信息，为相关视频编解码标准中，视频比特流中携带的信息，则相比相关视频编解码标准中生成的视频比特流，本申请实施例中，不会增加额外的比特数据。
例如，当使用失真图像每个像素点所在编码区域的量化参数作为边信息 时，由于在相关视频编解码标准中生成的视频比特流即携带有该量化参数，所以，本步骤中，按照相关视频编解码标准生成视频比特流即可。
如果用于生成边信息分量的边信息，在相关视频编解码标准中生成的视频比特流中不存在，则可以将该边信息作为视频解码所需要的一种信息，添加进待熵编码数据，并对添加有该边信息的待熵编码数据进行熵编码，得到编码后的视频比特流，从而使得解码端可以从该视频比特流中获取该边信息，用于解码端的视频解码处理。
相应的，本申请实施例还提供了一种视频解码方法，如图9所示，可以包括如下步骤：
步骤91、对接收的视频比特流进行熵解码，得到当前熵解码数据。
步骤92、获得失真图像，该失真图像为对当前熵解码数据进行视频解码的过程中生成的。
步骤93、基于当前熵解码数据中包括的边信息，生成失真图像对应的边信息分量，边信息分量表示失真图像相对当前熵解码数据对应的原始帧图像的失真特征。
步骤94、将失真图像的失真图像颜色分量以及边信息分量，输入预先建立的CNN模型进行卷积滤波处理，得到去失真图像颜色分量。
其中，CNN模型为基于预设训练集进行训练得到的，预设训练集包括原始样本图像、以及原始样本图像对应的多个失真图像的失真图像颜色分量，以及每个失真图像对应的边信息分量。
应用图9所示实施例，实现了在视频解码过程中，使用一种不同于相关技术的CNN模型对失真图像进行去失真处理。
下面结合附图，以边信息分量表示失真图像相对当前熵解码数据对应的原始帧图像的失真程度为例，对上述视频解码方法进行详细描述。
图10所示为本申请实施例提供的一种视频解码方法的流程图，具体包括 如下处理步骤：
步骤101、解码端对接收的视频比特流进行熵解码，得到当前熵解码数据。
本步骤中，得到的当前熵解码数据与原始视频中的一帧图像相对应，基于当前熵解码数据，进行解码处理，可以得到该帧图像对应的解码后图像，为便于技术方案的描述，将该帧图像称作当前熵解码数据对应的原始帧图像。
步骤102、获得失真图像，该失真图像为对当前熵解码数据进行视频解码的过程中生成的。
如图11A所示，对视频比特流进行熵解码得到的当前熵解码数据中，包括残差数据，对当前熵解码数据进行视频解码的过程，包括对该残差数据的反量化、反变换、重建和去失真滤波的处理。
本步骤中的失真图像，如图11A所示，可以是该过程中经过重建处理后生成的重建图像；也可以如图11B所示，是对重建图像采用其他预设滤波方法进行滤波后得到的，例如，其他预设滤波方法可以是DF滤波方法。
步骤103、从当前熵解码数据中，获取失真图像的边信息，具体可以为获取失真图像的每个像素点的失真程度值。
在实际应用中，对当前熵解码数据进行不同方式的视频解码之后，表示失真程度的物理参量也可能不同，本步骤中，可以采用如下方式确定出对应的能够准确表示像素点失真程度的失真程度值：
针对通过视频解码得到的上述失真图像，从当前熵解码数据中，获取当前熵解码数据对应的原始帧图像的每个编码区域的量化参数，将失真图像每个像素点所在编码区域的量化参数，确定为该像素点的失真程度值。
步骤104、基于失真图像各像素点的位置，使用获取的各像素点的失真程度值，生成失真图像对应的边信息分量。
其中，边信息分量包括的每个分量值与失真图像上相同位置的像素点相对应。
由于边信息分量包括的每个分量值与失真图像上相同位置的像素点相对应，则边信息分量与失真图像的失真图像颜色分量的结构相同，即表示边信息分量的矩阵与表示失真图像颜色分量的矩阵是同型的。
本步骤中，可以基于失真图像各像素点的位置，将获取的每个像素点的失真程度值，确定为失真图像对应的边信息分量中该像素点相同位置的分量值，即直接将每个像素点的失真程度值，确定为该像素点对应的分量值。
当失真图像的像素值范围与像素点的失真程度值的取值范围不同时，也可以基于失真图像的像素值范围，对获取的各像素点的失真程度值进行标准化处理，得到处理后失真程度值，处理后失真程度值的取值范围与像素值范围相同；
然后基于失真图像各像素点的位置，将每个像素点的处理后失真程度值，确定为失真图像对应的边信息分量中该像素点相同位置的分量值。
本步骤中，可以采用如下公式对像素点的失真程度值进行标准化处理：
Figure PCTCN2018083236-appb-000002
其中，norm(x)为标准化处理后得到的处理后失真程度值，x为像素点的失真程度值，失真图像的像素值范围为[PIXEL MIN,PIXEL MAX]，像素点的失真程度值的取值范围为[QP MIN,QP MAX]。
通过上述步骤103和步骤104，即生成了失真图像的边信息分量，生成边信息分量的过程，也可以理解为生成了失真图像对应的边信息引导图，该边信息引导图通过其边信息分量表示失真图像的失真程度，且该边信息引导图与失真图像是等高等宽的。
本申请实施例中，以CNN模型包括输入层、隐含层和输出层的结构为例，对方案进行描述。
步骤105、将失真图像的失真图像颜色分量以及生成的边信息分量，作为预先建立的CNN模型的输入数据，由输入层进行第一层的卷积滤波处理。
本步骤可以与上述图7中的步骤74相同，在此不再进行详细描述。
步骤106、隐含层对输入层输出的稀疏表示的图像块F 1(I)进行进一步的高维映射。
本步骤可以与上述图7中的步骤75相同，在此不再进行详细描述。
步骤107、输出层对隐含层输出的高维图像块F N(I)进行聚合，输出去失真图像颜色分量。
本步骤可以与上述图7中的步骤76相同，在此不再进行详细描述。
本申请实施例提供的上述视频解码方法中，得到的去失真图像颜色分量，如图11A和图11B所示，可以用于对视频比特流中的后续熵解码数据进行视频解码；如图11C所示，也可以用于生成当前熵解码数据对应的该原始帧图像对应的解码后图像，作为待播放帧图像。
在本申请实施例提供的上述解决方案中，在视频编码和视频解码的过程中，均使用了预先建立的CNN模型对失真图像进行去失真滤波处理。视频编码和视频解码所使用的CNN模型可以相同。
下面，对本申请实施例提供的上述CNN模型的训练流程进行详细描述。
本申请实施例还提出了一种CNN模型训练方法，如图12所示，具体包括如下处理步骤：
步骤121、获取预设训练集。预设训练集包括原始样本图像、原始样本图像对应的多个失真图像的失真图像颜色分量、以及每个失真图像对应的边信息分量，失真图像对应的边信息分量表示该失真图像相对原始样本图像的失真特征。该多个失真图像的失真特征不同。
本步骤中，可以预先对原始样本图像(即未失真的自然图像)，进行不同失真程度的一种图像处理，得到各自对应的失真图像，并按照上述去失真方法中的步骤，针对每个失真图像，生成对应的边信息分量，从而将每个原始样本图像、对应的失真图像以及对应的边信息分量组成图像对，由这些图像对组成预设训练集Ω。
训练集可以包括一个原始样本图像，针对该原始样本图像进行上述图像处理，得到失真特征不同的多个失真图像，以及每个失真图像对应的边信息 分量；
训练集也可以包括多个原始样本图像，分别针对每个原始样本图像进行上述图像处理，得到失真特征不同的多个失真图像，以及每个失真图像对应的边信息分量。
步骤122、针对预设结构的CNN，初始化该CNN的网络参数集中的参数，初始化的参数集可以由Θ 1表示，初始化的参数可以根据实际需要和经验进行设置。
本步骤中，还可以对训练相关的高层参数如学习率、梯度下降算法等进行合理的设置，具体可以采用相关技术中的各种方式，在此不再进行详细描述。
步骤123、进行前向计算，具体如下：
将预设训练集中的每个失真图像的失真图像颜色分量以及对应的边信息分量，输入预设结构的CNN进行卷积滤波处理，得到该失真图像对应的去失真图像颜色分量。
本步骤中，具体可以为对预设训练集Ω进行参数集为Θ i的CNN前向计算，获取CNN的输出F(Y)，即每个失真图像对应的去失真图像颜色分量。
第一次进入本步骤处理时，当前参数集为Θ 1，后续再次进入本步骤处理时，当前参数集Θ i为对上一次使用的参数集Θ i-1进行调整后得到的，详见后续描述。
步骤124、基于多个原始样本图像的原始图像颜色分量和得到的去失真图像颜色分量，确定去失真图像的损失值。
具体可以使用均方误差(Mean Squared Error，MSE)公式作为损失函数，得到损失值L(Θ i)，详见如下公式：
Figure PCTCN2018083236-appb-000003
其中，H表示单次训练中从预设训练集中选取的图像对个数，I h表示第h个失真图像对应的由边信息分量和失真图像颜色分量合并后的输入数据， F(I h|Θ i)表示针对第h个失真图像，CNN在参数集Θ i下前向计算得到的去失真图像颜色分量，X h表示第h个失真图像对应的原始图像颜色分量，i为当前已进行前向计算的次数计数。
步骤125、基于损失值确定采用当前参数集的该预设结构的CNN是否收敛；如果不收敛，进入步骤126；如果收敛，进入步骤127。
例如，可以当损失值小于预设损失值阈值时，确定收敛；也可以当本次计算得到损失值与上一次计算得到的损失值之差，小于预设变化阈值时，确定收敛，本申请实施例在此不做限定。
步骤126、对当前参数集中的参数进行调整，得到调整后的参数集，然后进入步骤123，用于下一次前向计算。
具体可以利用反向传播算法对当前参数集中的参数进行调整。
步骤127、将当前参数集作为输出的最终参数集Θ final，并将采用最终参数集Θ final的该预设结构的CNN，作为训练完成的CNN模型。
基于同一发明构思，根据本申请上述实施例提供的视频编码方法，相应地，本申请另一实施例还提供了一种视频编码装置，如图13所示，包括：
第一图像获得模块131，用于获得失真图像；失真图像为对待编码视频中的当前原始帧图像进行视频编码的过程中生成的；
第一生成模块132，用于生成失真图像对应的边信息分量；边信息分量表示失真图像相对当前原始帧图像的失真特征；
第一滤波处理模块133，用于将失真图像的失真图像颜色分量以及边信息分量，输入预先建立的CNN模型进行卷积滤波处理，得到去失真图像颜色分量；其中，CNN模型为基于预设训练集进行训练得到的，预设训练集包括原始样本图像、原始样本图像对应的多个失真图像的失真图像颜色分量、以及每个失真图像对应的边信息分量；
熵编码模块134，用于对待熵编码数据进行熵编码，得到编码后的视频比特流；待熵编码数据为对当前原始帧图像进行视频编码生成的，且包括用于 生成边信息分量的边信息。
在本申请的一个实施例中，失真图像可以为对待编码视频中的当前原始帧图像进行视频编码的过程中生成的重建图像；或者
失真图像可以为对重建图像采用其他预设滤波方法进行滤波后得到的，重建图像为对待编码视频中的当前原始帧图像进行视频编码的过程中生成的。
在本申请的一个实施例中，边信息分量可以至少包括表示如下失真特征之一：
表示失真图像相对当前原始帧图像的失真程度；
表示失真图像相对当前原始帧图像的失真位置；
表示失真图像相对当前原始帧图像的失真类型。
在本申请的一个实施例中，第一生成模块132，具体可以用于确定失真图像每个像素点的失真程度值；并基于失真图像各像素点的位置，使用获取的各像素点的失真程度值，生成失真图像对应的边信息分量，其中，边信息分量包括的每个分量值与失真图像上相同位置的像素点相对应。
在本申请的一个实施例中，第一生成模块132，具体可以用于获取当前原始帧图像的每个编码区域的量化参数，将失真图像每个像素点所在编码区域的量化参数，确定为失真图像每个像素点的失真程度值。
在本申请的一个实施例中，第一生成模块132，具体可以用于：
基于失真图像各像素点的位置，将获取的每个像素点的失真程度值，确定为失真图像对应的边信息分量中该像素点相同位置的分量值；或者
基于失真图像的像素值范围，对获取的各像素点的失真程度值进行标准化处理，得到处理后失真程度值，处理后失真程度值的取值范围与像素值范围相同；基于失真图像各像素点的位置，将每个像素点的处理后失真程度值，确定为失真图像对应的边信息分量中该像素点相同位置的分量值。
应用图13所示实施例，实现了在视频编码过程中，使用一种不同于相关技术的CNN模型对失真图像进行去失真处理。
基于同一发明构思，根据本申请上述实施例提供的视频编码方法，相应地，本申请另一实施例还提供了一种电子设备，如图14所示，包括处理器141、通信接口142、存储器143和通信总线144，其中，处理器141，通信接口142，存储器143通过通信总线144完成相互间的通信；
存储器143，用于存放计算机程序；
处理器141，用于执行存储器上所存放的程序时，实现视频编码方法步骤。其中，视频编码方法，包括：
获得失真图像；失真图像为对待编码视频中的当前原始帧图像进行视频编码的过程中生成的；
生成失真图像对应的边信息分量；边信息分量表示失真图像相对当前原始帧图像的失真特征；
将失真图像的失真图像颜色分量以及边信息分量，输入预先建立的卷积神经网络模型进行卷积滤波处理，得到去失真图像颜色分量；其中，卷积神经网络模型为基于预设训练集进行训练得到的，预设训练集包括原始样本图像、原始样本图像对应的多个失真图像的失真图像颜色分量、以及每个失真图像对应的边信息分量；
对待熵编码数据进行熵编码，得到编码后的视频比特流；待熵编码数据为对当前原始帧图像进行视频编码生成的，且包括用于生成边信息分量的边信息。
应用图14所示实施例，实现了在视频编码过程中，使用一种不同于相关技术的CNN模型对失真图像进行去失真处理。
基于同一发明构思，根据本申请上述实施例提供的视频编码方法，相应地，本申请另一实施例还提供了一种计算机可读存储介质，该计算机可读存储介质内存储有计算机程序，该计算机程序被处理器执行时实现视频编码方法步骤。其中，视频编码方法，包括：
获得失真图像；失真图像为对待编码视频中的当前原始帧图像进行视频编码的过程中生成的；
生成失真图像对应的边信息分量；边信息分量表示失真图像相对当前原始帧图像的失真特征；
将失真图像的失真图像颜色分量以及边信息分量，输入预先建立的卷积神经网络模型进行卷积滤波处理，得到去失真图像颜色分量；其中，卷积神经网络模型为基于预设训练集进行训练得到的，预设训练集包括原始样本图像、原始样本图像对应的多个失真图像的失真图像颜色分量、以及每个失真图像对应的边信息分量；
对待熵编码数据进行熵编码，得到编码后的视频比特流；待熵编码数据为对当前原始帧图像进行视频编码生成的，且包括用于生成边信息分量的边信息。
应用上述实施例，实现了在视频编码过程中，使用一种不同于相关技术的CNN模型对失真图像进行去失真处理。
基于同一发明构思，根据本申请上述实施例提供的视频编码方法，相应地，本申请另一实施例还提供了一种计算机程序，计算机程序被处理器执行时实现视频编码方法步骤。其中，视频编码方法，包括：
获得失真图像；失真图像为对待编码视频中的当前原始帧图像进行视频编码的过程中生成的；
生成失真图像对应的边信息分量；边信息分量表示失真图像相对当前原始帧图像的失真特征；
将失真图像的失真图像颜色分量以及边信息分量，输入预先建立的卷积神经网络模型进行卷积滤波处理，得到去失真图像颜色分量；其中，卷积神经网络模型为基于预设训练集进行训练得到的，预设训练集包括原始样本图像、原始样本图像对应的多个失真图像的失真图像颜色分量、以及每个失真图像对应的边信息分量；
对待熵编码数据进行熵编码，得到编码后的视频比特流；待熵编码数据为对当前原始帧图像进行视频编码生成的，且包括用于生成边信息分量的边信息。
应用上述实施例，实现了在视频编码过程中，使用一种不同于相关技术的CNN模型对失真图像进行去失真处理。
基于同一发明构思，根据本申请上述实施例提供的视频解码方法，相应地，本申请另一实施例还提供了一种视频解码装置，如图15所示，包括：
熵解码模块151，用于对接收的视频比特流进行熵解码，得到当前熵解码数据；
第二图像获得模块152，用于获得失真图像；失真图像为对当前熵解码数据进行视频解码的过程中生成的；
第二生成模块153，用于基于当前熵解码数据中包括的边信息，生成失真图像对应的边信息分量；边信息分量表示失真图像相对当前熵解码数据对应的原始帧图像的失真特征；
第二滤波处理模块154，用于将失真图像的失真图像颜色分量以及边信息分量，输入预先建立的CNN模型进行卷积滤波处理，得到去失真图像颜色分量；其中，CNN模型为基于预设训练集进行训练得到的，预设训练集包括原始样本图像、原始样本图像对应的多个失真图像的失真图像颜色分量、以及每个失真图像对应的边信息分量。
在本申请的一个实施例中，失真图像可以为对当前熵解码数据进行视频解码的过程中生成的重建图像；或者
失真图像可以为对重建图像采用其他预设滤波方法进行滤波后得到的，重建图像为对当前熵解码数据进行视频解码的过程中生成的。
在本申请的一个实施例中，得到的去失真图像颜色分量，可以用于对视频比特流中的后续熵解码数据进行视频解码；和/或
可以用于生成原始帧图像对应的解码后图像，作为待播放帧图像。
在本申请的一个实施例中，边信息分量可以至少表示如下失真特征之一：
表示失真图像相对原始帧图像的失真程度；
表示失真图像相对原始帧图像的失真位置；
表示失真图像相对原始帧图像的失真类型。
在本申请的一个实施例中，第二生成模块153，具体可以用于从当前熵解码数据中，获取失真图像每个像素点的失真程度值；并基于失真图像各像素点的位置，使用获取的各像素点的失真程度值，生成失真图像对应的边信息分量，其中，边信息分量包括的每个分量值与失真图像上相同位置的像素点相对应。
在本申请的一个实施例中，第二生成模块153，具体可以用于从当前熵解码数据中，获取当前熵解码数据对应的原始帧图像的每个编码区域的量化参数，将失真图像每个像素点所在编码区域的量化参数，确定为失真图像每个像素点的失真程度值。
在本申请的一个实施例中，第二生成模块153，具体可以用于：
基于失真图像各像素点的位置，将获取的每个像素点的失真程度值，确定为失真图像对应的边信息分量中该像素点相同位置的分量值；或者
基于失真图像的像素值范围，对获取的各像素点的失真程度值进行标准化处理，得到处理后失真程度值，处理后失真程度值的取值范围与像素值范围相同；基于失真图像各像素点的位置，将每个像素点的处理后失真程度值，确定为失真图像对应的边信息分量中该像素点相同位置的分量值。
应用图15所示实施例，实现了在视频解码过程中，使用一种不同于相关技术的CNN模型对失真图像进行去失真处理。
基于同一发明构思，根据本申请上述实施例提供的视频解码方法，相应地，本申请另一实施例还提供了一种电子设备，如图16所示，包括处理器161、通信接口162、存储器163和通信总线164，其中，处理器161，通信接口162，存储器163通过通信总线164完成相互间的通信；
存储器163，用于存放计算机程序；
处理器161，用于执行存储器上所存放的程序时，实现视频解码方法步骤。其中，视频解码方法，包括：
对接收的视频比特流进行熵解码，得到当前熵解码数据；
获得失真图像；失真图像为对当前熵解码数据进行视频解码的过程中生成的；
基于当前熵解码数据中包括的边信息，生成失真图像对应的边信息分量；边信息分量表示失真图像相对当前熵解码数据对应的原始帧图像的失真特征；
将失真图像的失真图像颜色分量以及边信息分量，输入预先建立的卷积神经网络模型进行卷积滤波处理，得到去失真图像颜色分量；其中，卷积神经网络模型为基于预设训练集进行训练得到的，预设训练集包括原始样本图像、原始样本图像对应的多个失真图像的失真图像颜色分量、以及每个失真图像对应的边信息分量。
应用图16所示实施例，实现了在视频解码过程中，使用一种不同于相关技术的CNN模型对失真图像进行去失真处理。
基于同一发明构思，根据本申请上述实施例提供的视频解码方法，相应地，本申请另一实施例还提供了一种计算机可读存储介质，该计算机可读存储介质内存储有计算机程序，该计算机程序被处理器执行时实现视频解码方法步骤。其中，视频解码方法，包括：
对接收的视频比特流进行熵解码，得到当前熵解码数据；
获得失真图像；失真图像为对当前熵解码数据进行视频解码的过程中生成的；
基于当前熵解码数据中包括的边信息，生成失真图像对应的边信息分量；边信息分量表示失真图像相对当前熵解码数据对应的原始帧图像的失真特征；
将失真图像的失真图像颜色分量以及边信息分量，输入预先建立的卷积神经网络模型进行卷积滤波处理，得到去失真图像颜色分量；其中，卷积神经网络模型为基于预设训练集进行训练得到的，预设训练集包括原始样本图像、原始样本图像对应的多个失真图像的失真图像颜色分量、以及每个失真图像对应的边信息分量。
应用上述实施例，实现了在视频解码过程中，使用一种不同于相关技术的CNN模型对失真图像进行去失真处理。
基于同一发明构思，根据本申请上述实施例提供的视频解码方法，相应地，本申请另一实施例还提供了一种计算机程序，计算机程序被处理器执行时实现视频解码方法步骤。其中，视频解码方法，包括：
对接收的视频比特流进行熵解码，得到当前熵解码数据；
获得失真图像；失真图像为对当前熵解码数据进行视频解码的过程中生成的；
基于当前熵解码数据中包括的边信息，生成失真图像对应的边信息分量；边信息分量表示失真图像相对当前熵解码数据对应的原始帧图像的失真特征；
将失真图像的失真图像颜色分量以及边信息分量，输入预先建立的卷积神经网络模型进行卷积滤波处理，得到去失真图像颜色分量；其中，卷积神经网络模型为基于预设训练集进行训练得到的，预设训练集包括原始样本图像、原始样本图像对应的多个失真图像的失真图像颜色分量、以及每个失真图像对应的边信息分量。
应用上述实施例，实现了在视频解码过程中，使用一种不同于相关技术的CNN模型对失真图像进行去失真处理。
需要说明的是，在本文中，诸如第一和第二等之类的关系术语仅仅用来将一个实体或者操作与另一个实体或操作区分开来，而不一定要求或者暗示这些实体或操作之间存在任何这种实际的关系或者顺序。而且，术语“包括”、“包含”或者其任何其他变体意在涵盖非排他性的包含，从而使得包括一系列要素的过程、方法、物品或者设备不仅包括那些要素，而且还包括没有明确列出的其他要素，或者是还包括为这种过程、方法、物品或者设备所固有的要素。在没有更多限制的情况下，由语句“包括一个……”限定的要素，并不排除在包括所述要素的过程、方法、物品或者设备中还存在另外的相同要素。
本说明书中的各个实施例均采用相关的方式描述，各个实施例之间相同相似的部分互相参见即可，每个实施例重点说明的都是与其他实施例的不同之处。尤其，对于视频编码装置、视频解码装置、电子设备、计算机可读存储介质、计算机程序实施例而言，由于其基本相似于视频编码方法和视频解码方法实施例，所以描述的比较简单，相关之处参见图2-图12所示的视频编 码方法和视频解码方法实施例的部分说明即可。
以上所述仅为本申请的较佳实施例而已，并非用于限定本申请的保护范围。凡在本申请的精神和原则之内所作的任何修改、等同替换、改进等，均包含在本申请的保护范围内。

Claims (32)

  1. 一种视频编码方法，其特征在于，包括：
    获得失真图像；所述失真图像为对待编码视频中的当前原始帧图像进行视频编码的过程中生成的；
    生成所述失真图像对应的边信息分量；所述边信息分量表示所述失真图像相对所述当前原始帧图像的失真特征；
    将所述失真图像的失真图像颜色分量以及所述边信息分量，输入预先建立的卷积神经网络模型进行卷积滤波处理，得到去失真图像颜色分量；其中，所述卷积神经网络模型为基于预设训练集进行训练得到的，所述预设训练集包括原始样本图像、原始样本图像对应的多个失真图像的失真图像颜色分量、以及每个失真图像对应的边信息分量；
    对待熵编码数据进行熵编码，得到编码后的视频比特流；所述待熵编码数据为对所述当前原始帧图像进行视频编码生成的，且包括用于生成所述边信息分量的边信息。
  2. 如权利要求1所述的方法，其特征在于，所述失真图像为对待编码视频中的当前原始帧图像进行视频编码的过程中生成的重建图像；或者
    所述失真图像为对重建图像采用其他预设滤波方法进行滤波后得到的；所述重建图像为对待编码视频中的当前原始帧图像进行视频编码的过程中生成的。
  3. 如权利要求1所述的方法，其特征在于，所述边信息分量至少包括表示如下失真特征之一：
    表示所述失真图像相对所述当前原始帧图像的失真程度；
    表示所述失真图像相对所述当前原始帧图像的失真位置；
    表示所述失真图像相对所述当前原始帧图像的失真类型。
  4. 如权利要求1所述的方法，其特征在于，生成所述失真图像对应的边信息分量，包括：
    确定所述失真图像每个像素点的失真程度值；
    基于所述失真图像各像素点的位置，使用获取的各像素点的失真程度值，生成所述失真图像对应的边信息分量；其中，所述边信息分量包括的每个分量值与所述失真图像上相同位置的像素点相对应。
  5. 如权利要求4所述的方法，其特征在于，确定所述失真图像每个像素点的失真程度值，包括：
    获取所述当前原始帧图像的每个编码区域的量化参数，将所述失真图像每个像素点所在编码区域的量化参数，确定为所述失真图像每个像素点的失真程度值。
  6. 如权利要求4所述的方法，其特征在于，基于所述失真图像各像素点的位置，使用获取的所述各像素点的失真程度值，生成所述失真图像对应的边信息分量，包括：
    基于所述失真图像各像素点的位置，将获取的每个像素点的失真程度值，确定为所述失真图像对应的边信息分量中该像素点相同位置的分量值；或者
    基于所述失真图像的像素值范围，对获取的所述各像素点的失真程度值进行标准化处理，得到处理后失真程度值，所述处理后失真程度值的取值范围与所述像素值范围相同；基于所述失真图像各像素点的位置，将每个像素点的处理后失真程度值，确定为所述失真图像对应的边信息分量中该像素点相同位置的分量值。
  7. 一种视频编码装置，其特征在于，包括：
    第一图像获得模块，用于获得失真图像；所述失真图像为对待编码视频中的当前原始帧图像进行视频编码的过程中生成的；
    第一生成模块，用于生成所述失真图像对应的边信息分量；所述边信息分量表示所述失真图像相对所述当前原始帧图像的失真特征；
    第一滤波处理模块，用于将所述失真图像的失真图像颜色分量以及所述边信息分量，输入预先建立的卷积神经网络模型进行卷积滤波处理，得到去失真图像颜色分量；其中，所述卷积神经网络模型为基于预设训练集进行训 练得到的，所述预设训练集包括原始样本图像、原始样本图像对应的多个失真图像的失真图像颜色分量、以及每个失真图像对应的边信息分量；
    熵编码模块，用于对待熵编码数据进行熵编码，得到编码后的视频比特流；所述待熵编码数据为对所述当前原始帧图像进行视频编码生成的，且包括用于生成所述边信息分量的边信息。
  8. 如权利要求7所述的装置，其特征在于，所述失真图像为对待编码视频中的当前原始帧图像进行视频编码的过程中生成的重建图像；或者
    所述失真图像为对重建图像采用其他预设滤波方法进行滤波后得到的；所述重建图像为对待编码视频中的当前原始帧图像进行视频编码的过程中生成的。
  9. 如权利要求7所述的装置，其特征在于，所述边信息分量至少包括表示如下失真特征之一：
    表示所述失真图像相对所述当前原始帧图像的失真程度；
    表示所述失真图像相对所述当前原始帧图像的失真位置；
    表示所述失真图像相对所述当前原始帧图像的失真类型。
  10. 如权利要求7所述的装置，其特征在于，所述第一生成模块，具体用于确定所述失真图像每个像素点的失真程度值；并基于所述失真图像各像素点的位置，使用获取的各像素点的失真程度值，生成所述失真图像对应的边信息分量，其中，所述边信息分量包括的每个分量值与所述失真图像上相同位置的像素点相对应。
  11. 如权利要求10所述的装置，其特征在于，所述第一生成模块，具体用于获取所述当前原始帧图像的每个编码区域的量化参数，将所述失真图像每个像素点所在编码区域的量化参数，确定为所述失真图像每个像素点的失真程度值。
  12. 如权利要求10所述的装置，其特征在于，所述第一生成模块，具体用于：
    基于所述失真图像各像素点的位置，将获取的每个像素点的失真程度值， 确定为所述失真图像对应的边信息分量中该像素点相同位置的分量值；或者
    基于所述失真图像的像素值范围，对获取的所述各像素点的失真程度值进行标准化处理，得到处理后失真程度值，所述处理后失真程度值的取值范围与所述像素值范围相同；基于所述失真图像各像素点的位置，将每个像素点的处理后失真程度值，确定为所述失真图像对应的边信息分量中该像素点相同位置的分量值。
  13. 一种电子设备，其特征在于，包括处理器、通信接口、存储器和通信总线；其中，处理器、通信接口、存储器通过通信总线完成相互间的通信；
    存储器，用于存放计算机程序；
    处理器，用于执行存储器上所存放的程序时，实现权利要求1-6任一所述的方法步骤。
  14. 一种计算机可读存储介质，其特征在于，所述计算机可读存储介质内存储有计算机程序，所述计算机程序被处理器执行时实现权利要求1-6任一所述的方法步骤。
  15. 一种视频解码方法，其特征在于，包括：
    对接收的视频比特流进行熵解码，得到当前熵解码数据；
    获得失真图像；所述失真图像为对所述当前熵解码数据进行视频解码的过程中生成的；
    基于所述当前熵解码数据中包括的边信息，生成所述失真图像对应的边信息分量；所述边信息分量表示所述失真图像相对所述当前熵解码数据对应的原始帧图像的失真特征；
    将所述失真图像的失真图像颜色分量以及所述边信息分量，输入预先建立的卷积神经网络模型进行卷积滤波处理，得到去失真图像颜色分量；其中，所述卷积神经网络模型为基于预设训练集进行训练得到的，所述预设训练集包括原始样本图像、原始样本图像对应的多个失真图像的失真图像颜色分量、以及每个失真图像对应的边信息分量。
  16. 如权利要求15所述的方法，其特征在于，所述失真图像为对所述当 前熵解码数据进行视频解码的过程中生成的重建图像；或者
    所述失真图像为对重建图像采用其他预设滤波方法进行滤波后得到的；所述重建图像为对所述当前熵解码数据进行视频解码的过程中生成的。
  17. 如权利要求15所述的方法，其特征在于，得到的所述去失真图像颜色分量，用于对所述视频比特流中的后续熵解码数据进行视频解码；和/或
    用于生成所述原始帧图像对应的解码后图像，作为待播放帧图像。
  18. 如权利要求15所述的方法，其特征在于，所述边信息分量至少表示如下失真特征之一：
    表示所述失真图像相对所述原始帧图像的失真程度；
    表示所述失真图像相对所述原始帧图像的失真位置；
    表示所述失真图像相对所述原始帧图像的失真类型。
  19. 如权利要求15所述的方法，其特征在于，基于所述当前熵解码数据中包括的边信息，生成所述失真图像对应的边信息分量，包括：
    从所述当前熵解码数据中，获取所述失真图像每个像素点的失真程度值；
    基于所述失真图像各像素点的位置，使用获取的各像素点的失真程度值，生成所述失真图像对应的边信息分量；其中，所述边信息分量包括的每个分量值与所述失真图像上相同位置的像素点相对应。
  20. 如权利要求19所述的方法，其特征在于，从所述当前熵解码数据中，获取所述失真图像每个像素点的失真程度值，包括：
    从所述当前熵解码数据中，获取所述当前熵解码数据对应的原始帧图像的每个编码区域的量化参数，将所述失真图像每个像素点所在编码区域的量化参数，确定为所述失真图像每个像素点的失真程度值。
  21. 如权利要求19所述的方法，其特征在于，基于所述失真图像各像素点的位置，使用获取的所述各像素点的失真程度值，生成所述失真图像对应的边信息分量，包括：
    基于所述失真图像各像素点的位置，将获取的每个像素点的失真程度值， 确定为所述失真图像对应的边信息分量中该像素点相同位置的分量值；或者
    基于所述失真图像的像素值范围，对获取的所述各像素点的失真程度值进行标准化处理，得到处理后失真程度值，所述处理后失真程度值的取值范围与所述像素值范围相同；基于所述失真图像各像素点的位置，将每个像素点的处理后失真程度值，确定为所述失真图像对应的边信息分量中该像素点相同位置的分量值。
  22. 一种视频解码装置，其特征在于，包括：
    熵解码模块，用于对接收的视频比特流进行熵解码，得到当前熵解码数据；
    第二图像获得模块，用于获得失真图像；所述失真图像为对所述当前熵解码数据进行视频解码的过程中生成的；
    第二生成模块，用于基于所述当前熵解码数据中包括的边信息，生成所述失真图像对应的边信息分量；所述边信息分量表示所述失真图像相对所述当前熵解码数据对应的原始帧图像的失真特征；
    第二滤波处理模块，用于将所述失真图像的失真图像颜色分量以及所述边信息分量，输入预先建立的卷积神经网络模型进行卷积滤波处理，得到去失真图像颜色分量；其中，所述卷积神经网络模型为基于预设训练集进行训练得到的，所述预设训练集包括原始样本图像、原始样本图像对应的多个失真图像的失真图像颜色分量、以及每个失真图像对应的边信息分量。
  23. 如权利要求22所述的装置，其特征在于，所述失真图像为对所述当前熵解码数据进行视频解码的过程中生成的重建图像；或者
    所述失真图像为对重建图像采用其他预设滤波方法进行滤波后得到的；所述重建图像为对所述当前熵解码数据进行视频解码的过程中生成的。
  24. 如权利要求22所述的装置，其特征在于，得到的所述去失真图像颜色分量，用于对所述视频比特流中的后续熵解码数据进行视频解码；和/或
    用于生成所述原始帧图像对应的解码后图像，作为待播放帧图像。
  25. 如权利要求22所述的装置，其特征在于，所述边信息分量至少表示 如下失真特征之一：
    表示所述失真图像相对所述原始帧图像的失真程度；
    表示所述失真图像相对所述原始帧图像的失真位置；
    表示所述失真图像相对所述原始帧图像的失真类型。
  26. 如权利要求22所述的装置，其特征在于，所述第二生成模块，具体用于从所述当前熵解码数据中，获取所述失真图像每个像素点的失真程度值；并基于所述失真图像各像素点的位置，使用获取的各像素点的失真程度值，生成所述失真图像对应的边信息分量，所述边信息分量包括的每个分量值与所述失真图像上相同位置的像素点相对应。
  27. 如权利要求26所述的装置，其特征在于，所述第二生成模块，具体用于从所述当前熵解码数据中，获取所述当前熵解码数据对应的原始帧图像的每个编码区域的量化参数，将所述失真图像每个像素点所在编码区域的量化参数，确定为所述失真图像每个像素点的失真程度值。
  28. 如权利要求26所述的装置，其特征在于，所述第二生成模块，具体用于：
    基于所述失真图像各像素点的位置，将获取的每个像素点的失真程度值，确定为所述失真图像对应的边信息分量中该像素点相同位置的分量值；或者
    基于所述失真图像的像素值范围，对获取的所述各像素点的失真程度值进行标准化处理，得到处理后失真程度值，所述处理后失真程度值的取值范围与所述像素值范围相同；基于所述失真图像各像素点的位置，将每个像素点的处理后失真程度值，确定为所述失真图像对应的边信息分量中该像素点相同位置的分量值。
  29. 一种电子设备，其特征在于，包括处理器、通信接口、存储器和通信总线；其中，处理器、通信接口、存储器通过通信总线完成相互间的通信；
    存储器，用于存放计算机程序；
    处理器，用于执行存储器上所存放的程序时，实现权利要求15-21任一所述的方法步骤。
  30. 一种计算机可读存储介质，其特征在于，所述计算机可读存储介质内存储有计算机程序，所述计算机程序被处理器执行时实现权利要求15-21任一所述的方法步骤。
  31. 一种计算机程序，其特征在于，所述计算机程序被处理器执行时实现权利要求1-6任一所述的方法步骤。
  32. 一种计算机程序，其特征在于，所述计算机程序被处理器执行时实现权利要求15-21任一所述的方法步骤。
PCT/CN2018/083236 2017-06-26 2018-04-16 一种视频编码方法、解码方法、装置及电子设备 WO2019001095A1 (zh)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN201710494773.3 2017-06-26
CN201710494773.3A CN109120937B (zh) 2017-06-26 2017-06-26 一种视频编码方法、解码方法、装置及电子设备

Publications (1)

Publication Number Publication Date
WO2019001095A1 true WO2019001095A1 (zh) 2019-01-03

Family

ID=64740982

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2018/083236 WO2019001095A1 (zh) 2017-06-26 2018-04-16 一种视频编码方法、解码方法、装置及电子设备

Country Status (2)

Country Link
CN (1) CN109120937B (zh)
WO (1) WO2019001095A1 (zh)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112261415A (zh) * 2020-10-23 2021-01-22 青海民族大学 基于过拟合卷积自编码网络的图像压缩编码方法

Families Citing this family (10)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110062226B (zh) * 2018-01-18 2021-06-11 杭州海康威视数字技术股份有限公司 一种视频编码方法、视频解码方法、装置、系统及介质
CN109920049B (zh) * 2019-02-26 2021-05-04 清华大学 边缘信息辅助精细三维人脸重建方法及系统
CN113039792A (zh) * 2019-03-07 2021-06-25 Oppo广东移动通信有限公司 环路滤波实现方法、装置及计算机存储介质
CN113747179B (zh) * 2019-03-07 2023-08-08 Oppo广东移动通信有限公司 环路滤波实现方法、装置及计算机存储介质
KR20210139342A (ko) * 2019-03-24 2021-11-22 광동 오포 모바일 텔레커뮤니케이션즈 코포레이션 리미티드 필터링 방법, 장치, 인코더 및 컴퓨터 저장 매체
CN111741211B (zh) * 2019-03-25 2022-07-29 华为技术有限公司 图像显示方法和设备
CN110971784B (zh) * 2019-11-14 2022-03-25 北京达佳互联信息技术有限公司 一种视频处理方法、装置、电子设备及存储介质
CN112991192B (zh) * 2019-12-18 2023-07-25 杭州海康威视数字技术股份有限公司 图像处理方法、装置、设备及其系统
CN111669587B (zh) * 2020-04-17 2021-07-20 北京大学 一种视频图像的拟态压缩方法、装置、存储介质及终端
CN113542739B (zh) * 2021-07-15 2023-10-20 Oppo广东移动通信有限公司 图像编码方法及装置、图像解码方法及装置、介质、设备

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102972022A (zh) * 2010-04-12 2013-03-13 松下电器产业株式会社 滤波器的定位和选择
CN105611303A (zh) * 2016-03-07 2016-05-25 京东方科技集团股份有限公司 图像压缩系统、解压缩系统、训练方法和装置、显示装置
CN205408063U (zh) * 2016-03-07 2016-07-27 京东方科技集团股份有限公司 图像压缩系统、解压缩系统、显示装置
WO2017066236A1 (en) * 2015-10-15 2017-04-20 Cisco Technology, Inc. Efficient loop filter for video codec

Family Cites Families (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2016127271A1 (en) * 2015-02-13 2016-08-18 Xiaoou Tang An apparatus and a method for reducing compression artifacts of a lossy-compressed image
CA2997193C (en) * 2015-09-03 2021-04-06 Mediatek Inc. Method and apparatus of neural network based processing in video coding

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102972022A (zh) * 2010-04-12 2013-03-13 松下电器产业株式会社 滤波器的定位和选择
WO2017066236A1 (en) * 2015-10-15 2017-04-20 Cisco Technology, Inc. Efficient loop filter for video codec
CN105611303A (zh) * 2016-03-07 2016-05-25 京东方科技集团股份有限公司 图像压缩系统、解压缩系统、训练方法和装置、显示装置
CN205408063U (zh) * 2016-03-07 2016-07-27 京东方科技集团股份有限公司 图像压缩系统、解压缩系统、显示装置

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112261415A (zh) * 2020-10-23 2021-01-22 青海民族大学 基于过拟合卷积自编码网络的图像压缩编码方法
CN112261415B (zh) * 2020-10-23 2022-04-08 青海民族大学 基于过拟合卷积自编码网络的图像压缩编码方法

Also Published As

Publication number Publication date
CN109120937B (zh) 2020-03-27
CN109120937A (zh) 2019-01-01

Similar Documents

Publication Publication Date Title
WO2018214671A1 (zh) 一种失真图像的去失真方法、装置及电子设备
WO2019001095A1 (zh) 一种视频编码方法、解码方法、装置及电子设备
WO2019001094A1 (zh) 一种视频编码方法、解码方法、装置及电子设备
CN110059796B (zh) 卷积神经网络的生成方法及装置
Li et al. An efficient deep convolutional neural networks model for compressed image deblocking
Dong et al. Compression artifacts reduction by a deep convolutional network
Chang et al. Reducing artifacts in JPEG decompression via a learned dictionary
WO2020015330A1 (zh) 基于增强的神经网络的图像复原方法、存储介质及系统
Liu et al. Data-driven soft decoding of compressed images in dual transform-pixel domain
CN110136057B (zh) 一种图像超分辨率重建方法、装置及电子设备
CN111105357B (zh) 一种失真图像的去失真方法、装置及电子设备
WO2021169408A1 (zh) 图像处理方法、装置、电子设备及存储介质
Sun et al. Reduction of JPEG compression artifacts based on DCT coefficients prediction
CN113192147B (zh) 显著性压缩的方法、系统、存储介质、计算机设备及应用
Zhao et al. CREAM: CNN-REgularized ADMM framework for compressive-sensed image reconstruction
Song et al. Compressed image restoration via artifacts-free PCA basis learning and adaptive sparse modeling
Hsu et al. A high-capacity QRD-based blind color image watermarking algorithm incorporated with AI technologies
WO2020062074A1 (en) Reconstructing distorted images using convolutional neural network
Wang et al. JPEG artifacts removal via compression quality ranker-guided networks
Li et al. Rev-ae: A learned frame set for image reconstruction
Parihar et al. UndarkGAN: Low-light Image Enhancement with Cycle-consistent Adversarial Networks
Li et al. A dual-residual network for JPEG compression artifacts reduction
Yang et al. Semantic Preprocessor for Image Compression for Machines
CN112991192A (zh) 图像处理方法、装置、设备及其系统
Hou et al. Quality Enhancement of Compressed Video via CNNs.

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 18825217

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 18825217

Country of ref document: EP

Kind code of ref document: A1