CN115412727A

CN115412727A - Encoding method, decoding method and device thereof

Info

Publication number: CN115412727A
Application number: CN202211053305.XA
Authority: CN
Inventors: 李杨
Original assignee: Vivo Mobile Communication Co Ltd
Current assignee: Vivo Mobile Communication Co Ltd
Priority date: 2022-08-31
Filing date: 2022-08-31
Publication date: 2022-11-29

Abstract

The application provides an encoding method, a decoding method and a device thereof, belonging to the technical field of encoding and decoding, wherein the encoding method comprises the following steps: acquiring a target video, wherein the target video comprises at least one unit to be coded; for each unit to be coded, determining a target coding mode based on N image quality indexes which correspond to the unit to be coded in N coding modes one by one; the image quality index is used for representing the image quality of an image represented by the unit to be coded in a corresponding coding mode, and N is a positive integer greater than 1; and coding the unit to be coded by the target coding mode to obtain a target code stream corresponding to the target video.

Description

Encoding method, decoding method and device thereof

Technical Field

The present application belongs to the technical field of encoding and decoding, and in particular relates to an encoding method, a decoding method and devices thereof.

Background

Currently, video coding technology and super-resolution reconstruction technology are generally used to realize storage and transmission of video between different terminals. The specific encoding and decoding process is as follows: the method comprises the steps that a video to be coded is downsampled at a coding end to obtain a low-resolution video, and the low-resolution video is compressed and coded to save bit numbers; and at a decoding end, carrying out resolution enhancement on the low-resolution video through a super-resolution technology. The video coding technology realizes the high-efficiency compression of video data by removing redundant information between the interior of a video frame and the video frame; the super-resolution reconstruction technique achieves reconstruction from a low-resolution video to a high-resolution video.

However, in the above encoding process, only the minimization of the quality difference between the uncoded video and the encoded video is pursued, the video structure is compressed and color distortion is caused when the video is encoded, and the reconstructed video obtained by using the super-resolution technique at the decoding end is also amplified, so that the video quality of the reconstructed video is reduced.

Disclosure of Invention

An object of the embodiments of the present application is to provide an encoding method, a decoding method, and an apparatus thereof, which can solve the problem of low video quality of a reconstructed video.

In a first aspect, an embodiment of the present application provides an encoding method, where the method includes:

acquiring a target video, wherein the target video comprises at least one unit to be coded;

for each unit to be coded, determining a target coding mode based on N image quality indexes which correspond to the unit to be coded in N coding modes one by one; the image quality index is used for representing the image quality of the image represented by the unit to be coded in the corresponding coding mode, and N is a positive integer greater than 1;

and coding the unit to be coded through the target coding mode to obtain a target code stream corresponding to the target video.

In a second aspect, an embodiment of the present application provides a decoding method, where the method includes:

acquiring a target code stream; the target code stream comprises M units to be decoded, wherein M is a positive integer greater than 1;

and for each unit to be decoded, decoding the unit to be decoded by using a target decoding mode corresponding to the unit to be decoded to obtain a target video.

In a third aspect, an embodiment of the present application provides an encoding apparatus, including:

the device comprises an acquisition module, a coding module and a coding module, wherein the acquisition module is used for acquiring a target video, and the target video comprises at least one unit to be coded;

the device comprises a determining module, a coding module and a coding module, wherein the determining module is used for determining a target coding mode for each unit to be coded based on N image quality indexes which correspond to the unit to be coded in N coding modes one by one; the image quality index is used for representing the image quality of the image represented by the unit to be coded in the corresponding coding mode, and N is a positive integer greater than 1;

and the coding module is used for coding the unit to be coded through the target coding mode to obtain a target code stream corresponding to the target video.

In a fourth aspect, an embodiment of the present application provides a decoding apparatus, including:

the acquisition module is used for acquiring a target code stream; the target code stream comprises M units to be decoded, wherein M is a positive integer greater than 1;

and the decoding module is used for decoding each unit to be decoded through a target decoding mode corresponding to the unit to be decoded to obtain a target video.

In a fifth aspect, the present application provides an electronic device, which includes a processor, a memory, and a program or instructions stored on the memory and executable on the processor, and when executed by the processor, the program or instructions implement the steps of the method according to the first aspect, or implement the steps of the method according to the second aspect.

In a sixth aspect, embodiments of the present application provide a readable storage medium, on which a program or instructions are stored, which when executed by a processor implement the steps of the method according to the first aspect, or implement the steps of the method according to the second aspect.

In a seventh aspect, an embodiment of the present application provides a chip, where the chip includes a processor and a communication interface, where the communication interface is coupled to the processor, and the processor is configured to execute a program or instructions to implement the method according to the first aspect, or implement the steps of the method according to the second aspect.

In an eighth aspect, embodiments of the present application provide a computer program product, stored on a storage medium, which is executed by at least one processor to implement the method according to the first aspect, or to implement the steps of the method according to the second aspect.

In the embodiment of the application, a target video is obtained; for each unit to be coded, determining a target coding mode based on N image quality indexes which correspond to the unit to be coded in N coding modes one by one; and coding the unit to be coded by the target coding mode to obtain a target code stream corresponding to the target video. In the embodiment of the application, in the process of coding the target video, the image quality of compressed images of each unit to be coded, which are represented by the unit to be coded, in the target video in different coding modes after super-resolution reconstruction at the decoding end is fully considered, so that the target coding mode corresponding to each unit to be coded is determined, and then the unit to be coded is coded through the target coding mode, and a target code stream corresponding to the target video is obtained. Therefore, in the process of decoding the target code stream at the subsequent decoding end to obtain the decoded target video, the target video with higher video quality can be obtained, so that the video quality of the reconstructed video is improved.

Drawings

Fig. 1 is a flowchart of an encoding method provided in an embodiment of the present application;

fig. 2 is an application flowchart of an encoding method provided in an embodiment of the present application;

fig. 3 is a second flowchart of an application of the encoding method according to the embodiment of the present application;

fig. 4 is a third flowchart of an application of the encoding method according to the embodiment of the present application;

fig. 5 is a fourth flowchart of an application of the encoding method provided in the embodiment of the present application;

FIG. 6 is a flowchart of a decoding method provided in an embodiment of the present application;

fig. 7 is a block diagram of an encoding apparatus according to an embodiment of the present application;

fig. 8 is a block diagram of a decoding apparatus according to an embodiment of the present application;

FIG. 9 is a block diagram of an electronic device provided by an embodiment of the application;

fig. 10 is a hardware block diagram of an electronic device according to an embodiment of the present application.

Detailed Description

The technical solutions in the embodiments of the present application will be described clearly below with reference to the drawings in the embodiments of the present application, and it is obvious that the described embodiments are some, but not all, embodiments of the present application. All other embodiments that can be derived by one of ordinary skill in the art from the embodiments given herein are intended to be within the scope of the present disclosure.

The terms first, second and the like in the description and in the claims of the present application are used for distinguishing between similar elements and not necessarily for describing a particular sequential or chronological order. It will be appreciated that the data so used may be interchanged under appropriate circumstances such that embodiments of the application may be practiced in sequences other than those illustrated or described herein, and that the terms "first," "second," and the like are generally used herein in a generic sense and do not limit the number of terms, e.g., the first term can be one or more than one. In addition, "and/or" in the specification and claims means at least one of connected objects, a character "/" generally means that a preceding and succeeding related objects are in an "or" relationship.

In the conventional compression encoding method, only the minimization of the quality difference between the non-encoded video and the encoded video is pursued, and specifically, the conventional encoding method can be expressed using the following formula (1):

wherein, I is a video to be compressed,

for compressed video after encoding the video to be compressed using the encoding mode n, Q is used to characterize the distortion metric, R _n The number of bits generated by encoding using the encoding mode n is λ, which is an encoding parameter.

The optimal edition is selected by representationCode pattern, recording the optimal code pattern as

Then

With a minimum value.

The traditional encoding method does not consider that the video structure is compressed and color distortion is caused when the video is encoded, so that the technical problem that the quality of the decoded video obtained by the subsequent decoding end after super-resolution reconstruction is low is solved.

In order to solve the above technical problems, an encoding method is provided in the embodiments of the present application, and the encoding method provided in the embodiments of the present application is described in detail through specific embodiments and application scenarios thereof with reference to the accompanying drawings.

Referring to fig. 1, fig. 1 is a flowchart of an encoding method according to an embodiment of the present application. The encoding method provided by the embodiment of the application comprises the following steps:

and S101, acquiring a target video.

The target video may be uncompressed video data captured by an electronic device using the encoding method provided in the embodiment of the present application through a camera, or may also be video data obtained by decoding a compressed video, which is not specifically limited herein. It should be understood that the target video may be represented as a matrix composed of luminance and chrominance information of pixels of a video frame, and the target video includes various forms such as pictures, two-dimensional videos, three-dimensional videos, and the like.

The target video comprises at least one unit to be coded. The unit to be encoded may be an image block, an image block group, a slice or a frame, and is not particularly limited herein.

S102, for each unit to be coded, determining a target coding mode based on N image quality indexes of the unit to be coded in N coding modes in a one-to-one correspondence mode.

The encoding mode includes, but is not limited to, an inter prediction mode, an intra prediction mode, and an entropy encoding mode, and for example, the encoding mode may be an AMVP mode, a Merge mode, or an affine mode.

In this step, after a target video is obtained, for each unit to be encoded in the target video, a target encoding mode is determined based on N image quality indicators that correspond to the unit to be encoded in N encoding modes one to one, where the image quality indicators are used to represent image quality of an image represented by the unit to be encoded in the corresponding encoding modes, and N is a positive integer greater than 1.

Please refer to the following embodiments for a specific technical solution of how to determine the target encoding mode.

S103, coding the unit to be coded through the target coding mode to obtain a target code stream corresponding to the target video.

In this step, after determining a target coding mode corresponding to a unit to be coded, coding the unit to be coded through the target coding mode, and after coding all the units to be coded, obtaining a target code stream corresponding to a target video.

It should be understood that the encoding method provided by the embodiment of the present application can be expressed by using the following formula (2):

wherein, the first and the second end of the pipe are connected with each other,

for compressed video encoded using coding mode n, Q is used to characterize the distortion metric, R _n The number of bits generated for encoding using the encoding mode n, λ is an encoding parameter, and f characterizes the super-resolution reconstruction.

The optimal coding mode is selected by representation and is recorded as

Then

With a minimum value. The present embodiment is significantly different from the above conventional compression encoding method in that the influence of the super-resolution reconstruction at the decoding end on the compression distortion is considered in the optimization process of the encoding mode at the encoding end.

In the embodiment of the application, a target video is obtained; for each unit to be coded, determining a target coding mode based on N image quality indexes which correspond to the unit to be coded in N coding modes one by one; and coding the unit to be coded by the target coding mode to obtain a target code stream corresponding to the target video. In the embodiment of the application, in the process of encoding the target video, the image quality of a compressed image represented by each unit to be encoded in the target video in different encoding modes after super-resolution reconstruction at a decoding end is fully considered, so that the target encoding mode corresponding to each unit to be encoded is determined, and then the unit to be encoded is encoded through the target encoding mode, and a target code stream corresponding to the target video is obtained. Therefore, in the process of decoding the target code stream at the subsequent decoding end to obtain the reconstructed target video, the target video with higher video quality can be obtained, so that the video quality of the reconstructed video is improved.

Optionally, the determining a target encoding mode based on N image quality indicators that the unit to be encoded has a one-to-one correspondence in N encoding modes includes:

traversing the N encoding modes;

determining N image quality indexes corresponding to the unit to be coded in the N coding modes based on N first images corresponding to the unit to be coded in the N coding modes, wherein the image quality indexes correspond to the first images one by one;

and determining a target coding mode based on the N image quality indexes.

Optionally, the N coding modes are ordered according to a preset order, and for any unit to be coded, each coding mode is traversed according to the preset order of the N coding modes, and an image quality index corresponding to the unit to be coded in each coding mode is determined. And under the condition of traversing the N coding modes, obtaining N image quality indexes, and determining the coding mode corresponding to the image quality index which represents the best image quality in the N image quality indexes as a target coding mode.

In an optional embodiment, the image quality indicator represents image distortion corresponding to each coding mode, in this case, the coding mode corresponding to the image quality indicator representing the minimum image distortion may be determined as the target coding mode.

Another optional implementation manner is that the image quality index represents image quality corresponding to each coding mode, in this case, a coding mode corresponding to an image quality index representing the highest image quality may be determined as the target coding mode.

Another optional embodiment is that the image quality index represents image distortion corresponding to each coding mode, and the coding cost may be determined based on the image quality index and the number of bits corresponding to the unit to be coded in each coding mode, and the coding mode with the minimum coding cost is determined as the target coding mode. Specifically, it can be expressed by the following formula (3):

H＝D+λ*R (3)

wherein, H represents the coding cost, D represents the image distortion, lambda is the coding parameter, and R is the bit number.

One embodiment of determining an image quality indicator is:

optionally, the determining, based on N first images corresponding to the unit to be encoded in the N encoding modes, N image quality indicators corresponding to the unit to be encoded in the N encoding modes includes:

determining a first image corresponding to the unit to be coded in each of the N coding modes;

performing super-resolution reconstruction on the first image to obtain a first reconstructed image of the unit to be encoded in an encoding mode corresponding to the first image;

and performing quality evaluation on the first reconstructed image to obtain an image quality index corresponding to the unit to be coded in the coding mode.

In this embodiment, a first image corresponding to a unit to be encoded in each encoding mode is determined, and the first image is also referred to as a decoded image.

After the first image is determined, super-resolution reconstruction is directly carried out on the first image, and a first reconstructed image corresponding to the unit to be encoded in the encoding mode is obtained. It should be understood that the super-resolution reconstruction is used for super-resolution amplification of the first image, and alternatively, the super-resolution reconstruction may be performed by using a reconstruction method based on image interpolation, a reconstruction method based on deep learning, or other methods, which are not particularly limited herein.

It should be noted that the encoding end and the decoding end may use different super-resolution reconstruction techniques, and in order to improve the quality of the reconstructed video, the encoding end and the decoding end may be configured to use the same super-resolution reconstruction technique.

After the first reconstructed image is obtained, the quality of the first reconstructed image is evaluated, and an image quality index corresponding to the unit to be coded in the coding mode is obtained.

Optionally, the quality evaluation of the first reconstructed image uses a non-reference quality evaluation method, where the non-reference quality evaluation method refers to an algorithm that directly calculates image distortion or image quality under the condition that no original image or no original video is used as a reference. Alternatively, the non-reference quality assessment method includes, but is not limited to, the BRISQUE algorithm or the WaDIQaM-NR algorithm.

To facilitate understanding of the technical solution of the present embodiment, please refer to fig. 2, an application flow of the encoding end and the decoding end applying the technical solution of the present embodiment is shown in fig. 2, and the following description is specifically made:

the method comprises the steps that a coding end obtains a target video, and codes each unit to be coded in the target video by using a coding mode to obtain coded data; decoding the coded data to obtain a first image; and performing super-resolution reconstruction on the first image, and performing quality evaluation on the obtained first reconstructed image to obtain an image quality index of the unit to be encoded in the encoding mode. After traversing all the coding modes, determining a target coding mode; coding the unit to be coded by using a target coding mode until all the units to be coded are coded, and obtaining a target code stream; and transmitting or storing the target code stream to a decoding end.

After receiving the target code stream, the decoding end decodes the target code stream; and performing super-resolution reconstruction on the image to be reconstructed obtained by decoding to obtain a target video.

Another embodiment of determining the image quality index is as follows:

optionally, the determining, based on the first image corresponding to the unit to be encoded in the N encoding modes, N image quality indicators corresponding to the unit to be encoded in the N encoding modes includes:

performing super-resolution reconstruction on the first image and the second image represented by the unit to be encoded respectively to obtain a first reconstructed image of the unit to be encoded in an encoding mode corresponding to the first image and a second reconstructed image corresponding to the unit to be encoded;

and evaluating the quality of the first reconstructed image and the second reconstructed image to obtain the image quality index corresponding to the unit to be coded in the coding mode.

In this embodiment, a first image of a unit to be encoded in each encoding mode is determined, and the first image is also referred to as a decoded image.

After the first image is determined, performing super-resolution reconstruction on the first image to obtain a first reconstructed image of the unit to be encoded in the encoding mode; and performing super-resolution reconstruction on the second image corresponding to the unit to be coded to obtain a second reconstructed image corresponding to the unit to be coded. The second image is an image represented by the unit to be encoded, and the image is an uncompressed image.

After the first reconstructed image and the second reconstructed image are obtained, the quality of the first reconstructed image and the quality of the second reconstructed image are evaluated, and image quality indexes corresponding to the unit to be coded in the coding mode are obtained.

Optionally, the quality evaluation of the first reconstructed image and the second reconstructed image uses a reference quality evaluation method, where the reference quality evaluation method refers to an algorithm that directly calculates image distortion or image quality under a condition that an original image or an original video is used as a reference. Optionally, the above-mentioned reference quality evaluation method includes, but is not limited to, an MSE algorithm, a PSNR algorithm, a structure similarity SSIM algorithm, a multi-scale structure similarity MS-SSIM algorithm, and a VMAF algorithm.

To facilitate understanding of the technical solution of the present embodiment, please refer to fig. 3, and an application flow of the encoding end and the decoding end applying the technical solution of the present embodiment is shown in fig. 3. The application flow shown in fig. 3 differs from the application flow shown in fig. 2 in that: and performing super-resolution reconstruction on a second image corresponding to the unit to be encoded, and performing quality evaluation on the obtained second reconstructed image and the first reconstructed image.

In this embodiment, super-resolution reconstruction is newly added to the uncompressed image to obtain a second reconstructed image, and then quality evaluation is performed on the second reconstructed image in the process of determining the image quality index, so that the accuracy of the image quality represented by the image quality index is improved.

Another embodiment of determining the image quality index is as follows:

when the first image is a target image, performing quality evaluation on a first reconstructed image to obtain an image quality index corresponding to the unit to be encoded in an encoding mode corresponding to the first image;

and when the first image is not the target image, performing quality evaluation on the first image to obtain an image quality index corresponding to the unit to be encoded in the encoding mode.

And under the condition that the first image is the target image, the image content representing the first image is important, performing super-resolution reconstruction on the first image to obtain a first reconstructed image, and performing quality evaluation on the first reconstructed image to obtain an image quality index corresponding to the unit to be encoded in the encoding mode. The target image is an image predetermined based on methods of saliency detection, face detection, key frame detection and the like of a target video.

And if the first image is not the target image, directly evaluating the quality of the first image to obtain the image quality index corresponding to the unit to be coded in the coding mode.

To facilitate understanding of the technical solution of the present embodiment, please refer to fig. 4, and an application flow of the encoding end and the decoding end applying the technical solution of the present embodiment is shown in fig. 4. The application flow shown in fig. 4 differs from the application flow shown in fig. 2 in that: after the first image is obtained, screening the first image, and determining whether the first image is a target image; performing super-resolution reconstruction on the first image under the condition that the first image is the target image to obtain a first reconstructed image; performing quality evaluation on the first reconstructed image; and directly evaluating the quality of the first image when the first image is not the target image.

In this embodiment, the first image is screened, and the super-resolution reconstruction is performed on the first image only when the first image is the target image, so that the super-resolution reconstruction is not performed on all the first images, and the encoding complexity is reduced.

Another embodiment for determining the image quality index is as follows:

performing super-resolution reconstruction on the first image based on auxiliary information of a coding mode corresponding to the first image to obtain a first reconstructed image of the unit to be coded in the coding mode;

After the first image is determined, performing super-resolution reconstruction on the first image based on the auxiliary information to obtain a first reconstructed image of the unit to be encoded in the encoding mode. It should be understood that the auxiliary information includes encoding information of an encoding mode including, but not limited to, an encoding mode and encoding parameters, and image information of a second image corresponding to a unit to be encoded including, but not limited to, an image histogram and image scene information. It should be understood that, in the process of transmitting or storing the target code stream to the decoding end, the encoding end synchronously transmits or stores the auxiliary information to the decoding end.

To facilitate understanding of the technical solution of the present embodiment, please refer to fig. 5, and an application flow of the encoding end and the decoding end applying the technical solution of the present embodiment is shown in fig. 5. The difference between the application flow shown in fig. 5 and the application flow shown in fig. 2 is that: the encoding end carries out super-resolution reconstruction on the first image according to the auxiliary information; and the decoding end carries out super-resolution reconstruction on the image to be reconstructed according to the auxiliary information. Please refer to the following embodiments for the technical solution of how the decoding end performs super-resolution reconstruction on the image to be reconstructed according to the auxiliary information.

In this embodiment, the first image is reconstructed by introducing the auxiliary information, so that the image quality of the first reconstructed image obtained by the super-resolution reconstruction is improved.

Optionally, the determining a first image corresponding to the unit to be encoded in the encoding mode includes:

coding the unit to be coded by using the coding mode to obtain coded data;

and decoding the coded data to obtain a first image.

As described above, the first image is also called a decoded image.

In this embodiment, in a one-pass traversal process, a coding mode may be used to code a unit to be coded to obtain coded data; further, the encoded data is decoded to obtain a first image.

The embodiment of the present application further provides a decoding method, which is described in detail below with reference to the accompanying drawings through specific embodiments and application scenarios thereof.

Fig. 6 shows a flowchart of a decoding method provided in an embodiment of the present application. The decoding method provided by the embodiment of the application comprises the following steps:

s601, acquiring a target code stream.

In this step, the target code stream is a binary code stream, and the target code stream is a code stream transmitted from the encoding end or stored to the decoding end.

The target code stream comprises M units to be decoded, wherein M is a positive integer greater than 1.

S602, for each unit to be decoded, decoding the unit to be decoded through a target decoding mode corresponding to the unit to be decoded to obtain a target video.

As described above, each unit to be encoded corresponds to one target encoding mode. And for each unit to be decoded, decoding the unit to be decoded by using the target decoding mode corresponding to the unit to be decoded to obtain a target video. It should be understood that the target decoding mode corresponds to the target encoding mode of the unit to be encoded.

It should be noted that the decoding method is the inverse process of the encoding method, and the specific decoding manner is not repeated here.

Optionally, the decoding the unit to be decoded by using the target decoding mode corresponding to the unit to be decoded to obtain the target video includes:

decoding the unit to be decoded through a target decoding mode corresponding to the unit to be decoded to obtain an image to be reconstructed;

performing super-resolution reconstruction on the image to be reconstructed to obtain M third reconstructed images corresponding to the M units to be decoded one by one;

and obtaining a target video based on the M third reconstructed images.

In this embodiment, for each unit to be decoded, the target decoding mode corresponding to the unit to be decoded is used for decoding to obtain an image to be reconstructed corresponding to the unit to be decoded, and then the super-resolution reconstruction is performed on the image to be reconstructed. After super-resolution reconstruction is carried out on all the images to be reconstructed, M third reconstructed images are obtained, and the M third reconstructed images correspond to M units to be decoded one by one.

Further, the M third reconstructed images are sequenced according to the preset sequence corresponding to each third reconstructed image, and a target video is obtained.

Optionally, the performing super-resolution reconstruction on the image to be reconstructed includes:

and performing super-resolution reconstruction on the image to be reconstructed based on the auxiliary information corresponding to the target decoding mode.

Wherein the auxiliary information includes encoding information and image information for each encoding mode.

In this embodiment, the super-resolution reconstruction may be performed on the first image based on the auxiliary information, so as to obtain a first reconstructed image corresponding to the unit to be encoded in the current encoding mode.

The following describes the coding apparatus provided in the embodiments of the present application in detail through specific embodiments and application scenarios thereof with reference to the accompanying drawings.

As shown in fig. 7, the encoding apparatus 700 includes:

an obtaining module 701, configured to obtain a target video, where the target video includes at least one unit to be encoded;

a determining module 702, configured to determine, for each unit to be encoded, a target encoding mode based on N image quality indicators that correspond to the unit to be encoded in N encoding modes one to one; the image quality index is used for representing the image quality of the image represented by the unit to be coded in the corresponding coding mode, and N is a positive integer greater than 1;

the encoding module 703 is configured to encode the unit to be encoded through the target encoding mode, so as to obtain a target code stream corresponding to the target video.

Optionally, the determining module 702 is specifically configured to:

traversing the N encoding modes;

and determining a target coding mode based on the N image quality indexes.

Optionally, the determining module 702 is further specifically configured to:

performing super-resolution reconstruction on the first image and a second image corresponding to the unit to be encoded respectively to obtain a first reconstructed image of the unit to be encoded in an encoding mode corresponding to the first image and a second reconstructed image corresponding to the unit to be encoded;

Optionally, the determining module 702 is further specifically configured to:

when the first image is a target image, performing quality evaluation on a first reconstructed image to obtain an image quality index corresponding to the unit to be encoded in an encoding mode corresponding to the first image; the first reconstruction image is obtained by performing super-resolution reconstruction on the first image;

Optionally, the determining module 702 is further specifically configured to:

performing super-resolution reconstruction on the first image based on auxiliary information of a coding mode corresponding to the first image to obtain a first reconstructed image of the unit to be coded in the coding mode; the auxiliary information comprises coding information of the coding mode and image information of a second image corresponding to the unit to be coded;

In the embodiment of the application, in the process of encoding the target video, the image quality of a compressed image represented by each unit to be encoded in the target video in different encoding modes after super-resolution reconstruction at a decoding end is fully considered, so that the target encoding mode corresponding to each unit to be encoded is determined, and then the unit to be encoded is encoded through the target encoding mode, and a target code stream corresponding to the target video is obtained. Therefore, in the process of decoding the target code stream by the subsequent decoding end to obtain the decoded target video, the target video with higher video quality can be obtained, so that the video quality of the reconstructed video is improved.

The decoding apparatus provided in the embodiments of the present application is described in detail below with reference to the accompanying drawings by using specific embodiments and application scenarios thereof.

As shown in fig. 8, the decoding apparatus 800 includes:

an obtaining module 801, configured to obtain a target code stream; the target code stream comprises M units to be decoded, wherein M is a positive integer greater than 1;

a decoding module 802, configured to decode, for each unit to be decoded, the unit to be decoded according to a target decoding mode corresponding to the unit to be decoded, so as to obtain a target video.

Optionally, the decoding module 802 is specifically configured to:

and obtaining a target video based on the M third reconstructed images.

Optionally, the decoding module 802 is further specifically configured to:

performing super-resolution reconstruction on the image to be reconstructed based on auxiliary information corresponding to the target decoding mode;

The encoding apparatus and the decoding apparatus in the embodiments of the present application may be electronic devices, or may be components in electronic devices, such as integrated circuits or chips. The electronic device may be a terminal, or may be a device other than a terminal. The electronic Device may be, for example, a Mobile phone, a tablet computer, a notebook computer, a palm top computer, a vehicle-mounted electronic Device, a Mobile Internet Device (MID), an Augmented Reality (AR)/Virtual Reality (VR) Device, a robot, a wearable Device, an ultra-Mobile personal computer (UMPC), a netbook or a Personal Digital Assistant (PDA), and the like, and may also be a server, a Network Attached Storage (Network Attached Storage, NAS), a personal computer (NAS), a Television (TV), a teller machine, a self-service machine, and the like, and the embodiments of the present application are not limited in particular.

The encoding apparatus and the decoding apparatus in the embodiments of the present application may be apparatuses having an operating system. The operating system may be an Android operating system (Android), an iOS operating system, or other possible operating systems, which is not specifically limited in the embodiments of the present application.

The encoding apparatus provided in the embodiment of the present application can implement each process implemented in the method embodiment of fig. 1, and is not described here again to avoid repetition.

The decoding apparatus provided in the embodiment of the present application can implement each process implemented in the method embodiment of fig. 6, and is not described here again to avoid repetition.

Optionally, as shown in fig. 9, an electronic device 900 is further provided in an embodiment of the present application, and includes a processor 901, a memory 902, and a program or an instruction that is stored in the memory 902 and is executable on the processor 901, where the program or the instruction is executed by the processor 901 to implement each process of the foregoing encoding method embodiment, or to implement each process of the foregoing decoding method embodiment, and can achieve the same technical effect, and is not described herein again to avoid repetition.

It should be noted that the electronic device in the embodiment of the present application includes the mobile electronic device and the non-mobile electronic device described above.

Fig. 10 is a schematic diagram of a hardware structure of an electronic device implementing an embodiment of the present application.

The electronic device 1000 includes, but is not limited to: a radio frequency unit 1001, a network module 1002, an audio output unit 1003, an input unit 1004, a sensor 1005, a display unit 1006, a user input unit 707, an interface unit 1008, a memory 1009, and a processor 1010.

Those skilled in the art will appreciate that the electronic device 1000 may further comprise a power source (e.g., a battery) for supplying power to various components, and the power source may be logically connected to the processor 1010 through a power management system, so as to implement functions of managing charging, discharging, and power consumption through the power management system. The electronic device structure shown in fig. 10 does not constitute a limitation of the electronic device, and the electronic device may include more or less components than those shown, or combine some components, or arrange different components, and thus, the description is not repeated here.

The input unit 1004 is further configured to obtain a target video;

the processor 1010 is further configured to, for each unit to be encoded, determine a target encoding mode based on N image quality indicators that correspond to the unit to be encoded in N encoding modes one to one;

Or, the input unit 1004 is further configured to obtain a target code stream;

the processor 1010 is further configured to, for each unit to be decoded, decode the unit to be decoded through a target decoding mode corresponding to the unit to be decoded, so as to obtain a target video.

It should be understood that in the embodiment of the present application, the input Unit 1004 may include a Graphics Processing Unit (GPU) 10041 and a microphone 10042, and the Graphics Processing Unit 10041 processes image data of still pictures or videos obtained by an image capturing device (such as a camera) in a video capturing mode or an image capturing mode. The display unit 1006 may include a display panel 10061, and the display panel 10061 may be configured in the form of a liquid crystal display, an organic light emitting diode, or the like. The user input unit 1006 includes at least one of a touch panel 10061 and other input devices 10062. The touch panel 10061 is also referred to as a touch screen. The touch panel 10061 may include two parts of a touch detection device and a touch controller. Other input devices 10062 may include, but are not limited to, a physical keyboard, function keys (e.g., volume control keys, switch keys, etc.), a trackball, a mouse, and a joystick, which are not described in detail herein.

The memory 1009 may be used to store software programs as well as various data. The memory 1009 may mainly include a first storage area storing a program or an instruction and a second storage area storing data, wherein the first storage area may store an operating system, an application program or an instruction (such as a sound playing function, an image playing function, and the like) required for at least one function, and the like. Further, the memory 1009 may include volatile memory or nonvolatile memory, or the memory 1009 may include both volatile and nonvolatile memory. The non-volatile Memory may be a Read-Only Memory (ROM), a Programmable ROM (PROM), an Erasable PROM (EPROM), an Electrically Erasable PROM (EEPROM), or a flash Memory. The volatile Memory may be a Random Access Memory (RAM), a Static Random Access Memory (Static RAM, SRAM), a Dynamic Random Access Memory (Dynamic RAM, DRAM), a Synchronous Dynamic Random Access Memory (Synchronous DRAM, SDRAM), a Double Data Rate Synchronous Dynamic Random Access Memory (Double Data Rate SDRAM, ddr SDRAM), an Enhanced Synchronous SDRAM (ESDRAM), a Synchronous Link DRAM (SLDRAM), and a Direct bus RAM (DRRAM). The memory 1009 in the embodiments of the subject application includes, but is not limited to, these and any other suitable types of memory.

Processor 1010 may include one or more processing units; optionally, the processor 1010 integrates an application processor, which primarily handles operations related to the operating system, user interface, and applications, and a modem processor, which primarily handles wireless communication signals, such as a baseband processor. It will be appreciated that the modem processor described above may not be integrated into processor 1010.

The embodiments of the present application further provide a readable storage medium, where a program or an instruction is stored, and when the program or the instruction is executed by a processor, the program or the instruction implements each process of the foregoing encoding method embodiment, or implements each process of the foregoing decoding method embodiment, and can achieve the same technical effect, and in order to avoid repetition, details are not repeated here.

The processor is the processor in the electronic device described in the above embodiment. The readable storage medium includes a computer readable storage medium, such as a computer Read Only Memory (ROM), a Random Access Memory (RAM), a magnetic or optical disk, and the like.

The embodiment of the present application further provides a chip, where the chip includes a processor and a communication interface, the communication interface is coupled to the processor, and the processor is configured to execute a program or an instruction to implement each process of the encoding method embodiment or each process of the decoding method embodiment, and can achieve the same technical effect, and in order to avoid repetition, the details are not repeated here.

It should be understood that the chips mentioned in the embodiments of the present application may also be referred to as system-on-chip, system-on-chip or system-on-chip, etc.

Embodiments of the present application provide a computer program product, where the program product is stored in a storage medium, and the program product is executed by at least one processor to implement each process of the foregoing encoding method embodiment or each process of the foregoing decoding method embodiment, and can achieve the same technical effect, and in order to avoid repetition, details are not repeated here.

It should be noted that, in this document, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising a … …" does not exclude the presence of another identical element in a process, method, article, or apparatus that comprises the element. Further, it should be noted that the scope of the methods and apparatus of the embodiments of the present application is not limited to performing the functions in the order illustrated or discussed, but may include performing the functions in a substantially simultaneous manner or in a reverse order based on the functions involved, e.g., the methods described may be performed in an order different than that described, and various steps may be added, omitted, or combined. In addition, features described with reference to certain examples may be combined in other examples.

Through the description of the foregoing embodiments, it is clear to those skilled in the art that the method of the foregoing embodiments may be implemented by software plus a necessary general hardware platform, and certainly may also be implemented by hardware, but in many cases, the former is a better implementation. Based on such understanding, the technical solutions of the present application may be embodied in the form of a computer software product, which is stored in a storage medium (such as ROM/RAM, magnetic disk, optical disk) and includes instructions for enabling a terminal (such as a mobile phone, a computer, a server, or a network device) to execute the method according to the embodiments of the present application.

While the present embodiments have been described with reference to the accompanying drawings, it is to be understood that the invention is not limited to the precise embodiments described above, which are meant to be illustrative and not restrictive, and that various changes may be made therein by those skilled in the art without departing from the spirit and scope of the invention as defined by the appended claims.

Claims

1. A method of encoding, comprising:

2. The method according to claim 1, wherein the determining a target coding mode based on N image quality indicators corresponding to the unit to be coded in N coding modes one to one comprises:

traversing the N encoding modes;

and determining a target coding mode based on the N image quality indexes.

3. The method according to claim 2, wherein the determining, based on the N first images corresponding to the unit to be encoded in the N encoding modes, N image quality indicators corresponding to the unit to be encoded in the N encoding modes comprises:

4. The method according to claim 2, wherein the determining, based on the N first images corresponding to the unit to be encoded in the N encoding modes, N image quality indicators corresponding to the unit to be encoded in the N encoding modes comprises:

performing super-resolution reconstruction on the first image and a second image represented by the unit to be encoded respectively to obtain a first reconstructed image of the unit to be encoded in an encoding mode corresponding to the first image and a second reconstructed image corresponding to the unit to be encoded;

and performing quality evaluation on the first reconstructed image and the second reconstructed image to obtain an image quality index corresponding to the unit to be coded in the coding mode.

5. The method according to claim 2, wherein the determining, based on the N first images corresponding to the unit to be encoded in the N encoding modes, N image quality indicators corresponding to the unit to be encoded in the N encoding modes comprises:

6. The method according to claim 2, wherein the determining, based on the N first images corresponding to the unit to be encoded in the N encoding modes, N image quality indicators corresponding to the unit to be encoded in the N encoding modes comprises:

7. A method of decoding, comprising:

and for each unit to be decoded, decoding the unit to be decoded through a target decoding mode corresponding to the unit to be decoded to obtain a target video.

8. The method of claim 7, wherein the decoding the unit to be decoded in the target decoding mode corresponding to the unit to be decoded to obtain the target video comprises:

and obtaining a target video based on the M third reconstructed images.

9. The method according to claim 8, wherein the performing super-resolution reconstruction on the image to be reconstructed comprises:

performing super-resolution reconstruction on the image to be reconstructed based on the auxiliary information corresponding to the target decoding mode;

10. An encoding apparatus, comprising:

the device comprises an acquisition module, a coding module and a decoding module, wherein the acquisition module is used for acquiring a target video, and the target video comprises at least one unit to be coded;

11. The apparatus of claim 10, wherein the determining module is specifically configured to:

traversing the N encoding modes;

and determining a target coding mode based on the N image quality indexes.

12. The apparatus of claim 11, wherein the determining module is further specifically configured to:

13. The apparatus of claim 11, wherein the determining module is further specifically configured to:

14. The apparatus of claim 11, wherein the determining module is further specifically configured to:

15. The apparatus of claim 11, wherein the determining module is further specifically configured to:

16. A decoding apparatus, comprising:

17. The apparatus of claim 16, wherein the decoding module is specifically configured to:

decoding the unit to be decoded by using a target decoding mode corresponding to the unit to be decoded to obtain an image to be reconstructed;

and obtaining a target video based on the M third reconstructed images.

18. The apparatus of claim 17, wherein the decoding module is further specifically configured to:

19. An electronic device comprising a processor, a memory and a program or instructions stored on the memory and executable on the processor, which program or instructions, when executed by the processor, implement the steps of the encoding method of any one of claims 1 to 6 or the steps of the decoding method of any one of claims 7 to 9.

20. A readable storage medium, characterized in that it stores thereon a program or instructions which, when executed by a processor, implement the steps of the encoding method according to any one of claims 1 to 6, or the steps of the decoding method according to any one of claims 7 to 9.