CN112351276B

CN112351276B - Video encoding method and device and video decoding method and device

Info

Publication number: CN112351276B
Application number: CN202011216902.0A
Authority: CN
Inventors: 徐琴琴; 豆修鑫; 樊鸿飞; 张文杰; 蔡媛
Original assignee: Beijing Kingsoft Cloud Network Technology Co Ltd
Current assignee: Beijing Kingsoft Cloud Network Technology Co Ltd
Priority date: 2020-11-04
Filing date: 2020-11-04
Publication date: 2024-05-31
Anticipated expiration: 2040-11-04
Also published as: CN112351276A

Abstract

The application relates to a video coding method and device, and a video decoding method and device, wherein the video coding method comprises the following steps: determining coding information of a video frame to be coded, wherein the video frame to be coded is a non-key frame in a target video, and the coding information is used for indicating coding quality and decoding delay of the video frame; determining a target coding mode matched with the video frame from a target coding mode set corresponding to the video frame according to the coding information, wherein the target coding mode set comprises a coding mode allowing the video frame to be used, and only a forward reference frame is configured for a non-key frame in the coding mode allowing the video frame to be used; the video frames are encoded according to a target encoding mode. The application solves the technical problem of poor flexibility of encoding and decoding of video in the related technology.

Description

Video encoding method and device and video decoding method and device

Technical Field

The present application relates to the field of computers, and in particular, to a method and apparatus for encoding video, and a method and apparatus for decoding video.

Background

In the existing video coding technology, the influence of the delay of a coding mode on user experience is only considered generally, and better experience can be ensured when a user plays videos by reducing the delay as much as possible, but the method cannot adjust the coding and decoding processes of the videos according to different playing scenes, so that the flexibility of the video coding and decoding processes is poor, and videos suitable for the current playing environment of the user cannot be provided for the user.

In view of the above problems, no effective solution has been proposed at present.

Disclosure of Invention

The application provides a video coding method and device and a video decoding method and device, which at least solve the technical problem of poor coding and decoding flexibility of videos in the related technology.

According to an aspect of an embodiment of the present application, there is provided a video encoding method including:

determining coding information of a video frame to be coded, wherein the video frame to be coded is a non-key frame in a target video, and the coding information is used for indicating coding quality and decoding delay of the video frame;

determining a target coding mode matched with the video frame from a target coding mode set corresponding to the video frame according to the coding information, wherein the target coding mode set comprises a coding mode allowing the video frame to be used, and only configuring a forward reference frame by a non-key frame in the coding mode allowing the video frame to be used;

And encoding the video frame according to the target encoding mode.

According to an aspect of an embodiment of the present application, there is provided a video decoding method including:

determining decoding information of a video frame to be decoded, wherein the video frame to be decoded is a non-key frame in a target video, and the decoding information is used for indicating the decoding quality and decoding delay of the video frame;

Determining a target decoding mode matched with the video frame from a target decoding mode set corresponding to the video frame according to the decoding information, wherein the target decoding mode set comprises a decoding mode allowing the video frame to be used, and only configuring a forward reference frame by a non-key frame in the decoding mode allowing the video frame to be used;

and decoding the video frame according to the target decoding mode.

According to another aspect of the embodiment of the present application, there is also provided an apparatus for encoding video, including:

A first determining module, configured to determine encoding information of a video frame to be encoded, where the video frame to be encoded is a non-key frame in a target video, and the encoding information is used to indicate encoding quality and decoding delay of the video frame;

a second determining module, configured to determine, according to the coding information, a target coding mode that matches the video frame from a target coding mode set corresponding to the video frame, where the target coding mode set includes a coding mode that allows the video frame to be used, and a non-key frame only configures a forward reference frame in the coding mode that allows the video frame to be used;

And the encoding module is used for encoding the video frames according to the target encoding mode.

According to another aspect of the embodiment of the present application, there is also provided a decoding apparatus for video, including:

A first determining module, configured to determine decoding information of a video frame to be decoded, where the video frame to be decoded is a non-key frame in a target video, and the decoding information is used to indicate decoding quality and decoding delay of the video frame;

A second determining module, configured to determine, according to the decoding information, a target decoding mode that matches the video frame from a target decoding mode set corresponding to the video frame, where the target decoding mode set includes a decoding mode that allows the video frame to be used, and a non-key frame in the decoding mode that allows the video frame to be used configures only a forward reference frame;

and the decoding unit is used for decoding the video frames according to the target decoding mode.

According to another aspect of the embodiments of the present application, there is also provided a storage medium including a stored program that executes the above-described method when running.

According to another aspect of the embodiments of the present application, there is also provided an electronic device including a memory, a processor, and a computer program stored on the memory and executable on the processor, the processor executing the method described above by the computer program.

In the embodiment of the application, the coding information of the video frame to be coded is determined, wherein the video frame to be coded is a non-key frame in the target video, and the coding information is used for indicating the coding quality and decoding delay of the video frame; determining a target coding mode matched with the video frame from a target coding mode set corresponding to the video frame according to the coding information, wherein the target coding mode set comprises a coding mode allowing the video frame to be used, and only a forward reference frame is configured for a non-key frame in the coding mode allowing the video frame to be used; according to the method for encoding the video frames in the target encoding mode, the encoding information of the current video frames to be encoded is acquired, the target encoding mode is determined for the video frames to be encoded from a plurality of encoding modes according to the encoding information, and the video frames to be encoded are encoded according to the target encoding mode, so that the relation between the encoding quality and the decoding delay can be balanced in the encoding process, and the non-key frames in the image group in each of the plurality of encoding modes are only configured with the forward reference frames, and each encoding mode belongs to the encoding mode with low delay, so that the aim of considering the decoding delay and the encoding quality is fulfilled, the technical effect of improving the encoding and decoding flexibility of the video is achieved, and the technical problem of poor encoding and decoding flexibility of the video in the related art is solved.

Drawings

The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the invention and together with the description, serve to explain the principles of the invention.

In order to more clearly illustrate the embodiments of the invention or the technical solutions of the prior art, the drawings which are used in the description of the embodiments or the prior art will be briefly described, and it will be obvious to a person skilled in the art that other drawings can be obtained from these drawings without inventive effort.

Fig. 1 is a schematic view of a hardware environment of a video encoding and decoding method according to an embodiment of the present application;

FIG. 2 is a flow chart of an alternative video encoding method according to an embodiment of the application;

FIG. 3 is a schematic diagram of an alternative encoding mode 1 according to an embodiment of the present application;

FIG. 4 is a schematic diagram of an alternative encoding mode 2 according to an embodiment of the present application;

FIG. 5 is a schematic diagram of an alternative encoding mode 3 through encoding mode 8 according to an embodiment of the present application;

FIG. 6 is a schematic diagram of an alternative LDP coding mode according to an embodiment of the present application;

FIG. 7 is a flow chart of an alternative method of decoding video according to an embodiment of the application;

FIG. 8 is a schematic diagram of an alternative video encoding apparatus according to an embodiment of the present application;

fig. 9 is a schematic diagram of an alternative video decoding apparatus according to an embodiment of the present application;

fig. 10 is a block diagram of an electronic device according to an embodiment of the present application.

Detailed Description

In order that those skilled in the art will better understand the present application, a technical solution in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings in which it is apparent that the described embodiments are only some embodiments of the present application, not all embodiments. All other embodiments, which can be made by those skilled in the art based on the embodiments of the present application without making any inventive effort, shall fall within the scope of the present application.

It should be noted that the terms "first," "second," and the like in the description and the claims of the present application and the above figures are used for distinguishing between similar objects and not necessarily for describing a particular sequential or chronological order. It is to be understood that the data so used may be interchanged where appropriate such that the embodiments of the application described herein may be implemented in sequences other than those illustrated or otherwise described herein. Furthermore, the terms "comprises," "comprising," and "having," and any variations thereof, are intended to cover a non-exclusive inclusion, such that a process, method, system, article, or apparatus that comprises a list of steps or elements is not necessarily limited to those steps or elements expressly listed but may include other steps or elements not expressly listed or inherent to such process, method, article, or apparatus.

Alternatively, in the present embodiment, the video encoding and decoding method may be applied to a hardware environment constituted by the terminal 101 and the server 103 as shown in fig. 1. As shown in fig. 1, the server 103 is connected to the terminal 101 through a network, which may be used to provide services (such as game services, application services, multimedia services, etc.) to the terminal or clients installed on the terminal, and a database may be provided on the server or independent of the server, for providing data storage services to the server 103, where the network includes, but is not limited to: the terminal 101 is not limited to a PC, a mobile phone, a tablet computer, or the like. The video encoding method of the embodiment of the present application may be performed by the server 103, and the video decoding method may be performed by the terminal 101. The method for decoding video performed by the terminal 101 according to the embodiment of the present application may be performed by a client installed thereon.

According to an aspect of an embodiment of the present application, a method embodiment of encoding video is provided. Fig. 2 is a flowchart of an alternative video encoding method according to an embodiment of the present application, as shown in fig. 2, the method may include the steps of:

Step S202, determining coding information of a video frame to be coded, wherein the video frame to be coded is a non-key frame in a target video, and the coding information is used for indicating coding quality and decoding delay of the video frame;

Step S204, determining a target coding mode matched with the video frame from a target coding mode set corresponding to the video frame according to the coding information, wherein the target coding mode set comprises a coding mode allowing the video frame to be used, and only configuring a forward reference frame by a non-key frame in the coding mode allowing the video frame to be used;

and step S206, encoding the video frame according to the target encoding mode.

Through the steps S202 to S206, for the current video frame to be encoded, the encoding information is obtained, the target encoding mode is determined for the video frame to be encoded from multiple encoding modes according to the encoding information, and the video frame to be encoded is encoded according to the target encoding mode, so that the encoding process can balance the relation between the encoding quality and the decoding delay, and the non-key frames in the image group in each of the multiple encoding modes are only configured with the forward reference frame, and each encoding mode belongs to the encoding mode with low delay, so that the purpose of considering the decoding delay and the encoding quality is achieved, the technical effect of improving the encoding and decoding flexibility of the video is achieved, and the technical problem of poor encoding and decoding flexibility of the video in the related art is solved.

Alternatively, in this embodiment, the video encoding method may be applied to, but not limited to, live video, real-time communication, panoramic video, video on demand, and other scenes. The target video may include, but is not limited to, live, real-time communication (e.g., video call) or video streaming in panoramic video, video files in panoramic video or video on demand, and so forth.

As an alternative embodiment, during the playing of the panoramic video, the encoding mode may be adaptively adjusted by, but not limited to, the following processes:

Under the condition that the main view angle area of panoramic video display is detected to be switched to a first area, acquiring a video code stream to be switched corresponding to the first area, wherein the definition of the video code stream to be switched is higher than that of an original video code stream displayed by the first area before the area is switched;

Determining coding information of a video frame to be coded in the video code stream to be switched, wherein the video frame to be coded is a non-key frame in the video code stream to be switched, and the coding information is used for indicating coding quality and decoding delay of the video frame;

encoding the video frame according to the target encoding mode to obtain an encoded video code stream;

And sending the coded video code stream to a display device of the panoramic video.

Optionally, in this embodiment, when encoding a non-key frame in a video bitstream of a panoramic video, a target encoding mode used by each non-key frame is determined according to encoding information of the non-key frame for indicating encoding quality and decoding delay, so as to adaptively adjust an encoding process, so that the encoding process considers both a requirement of delay and a requirement of image quality, thereby ensuring low-delay transmission while ensuring image quality as much as possible, and improving experience of a user in watching the panoramic video.

As an alternative embodiment, during the playing of the video asset transmitted in real time, the encoding mode may be adaptively adjusted by, but not limited to, the following processes:

Acquiring video resources transmitted in real time from a video acquisition end;

determining coding information of a video frame to be coded in the video resource transmitted in real time, wherein the video frame to be coded is a non-key frame in a target video, and the coding information is used for indicating coding quality and decoding delay of the video frame;

Encoding the video frame according to the target encoding mode to obtain an encoded video resource;

And sending the coded video resources to a video playing end.

Optionally, in this embodiment, in the process of performing real-time transmission of video, for each non-key frame in the video resource, the target coding mode used by each non-key frame is determined according to the coding information of the non-key frame for indicating the coding quality and the decoding delay, so as to adaptively adjust the coding process, so that the coding process can consider both the requirement of delay and the requirement of image quality, thereby ensuring the low-delay transmission while ensuring the image quality as much as possible, and improving the experience of the user in the process of performing real-time audio/video communication.

In the solution provided in step S202, the group of pictures (Group of Pictures, GOP) refers to a set of consecutive pictures in the video. Typically, the first picture in the GOP is an I-frame, which is intra-coded, without reference to other frames. Other frames in the GOP may be B frames or P frames, where P frame forward references are encoded and B frame bi-directional references are encoded.

Optionally, in this embodiment, the non-key frames in the image group only configure the forward reference frames, that is, the image group only includes the I-frames and the P-frames, and configuring the reference frames in different forms for the P-frames obtains different coding modes.

Alternatively, in the present embodiment, the division manner of the image group may be, but not limited to, a manner including 8P frames, a manner including 16P frames, or the like. In this embodiment, a manner of 8P frames is taken as an example, and other manners are similar to the manner of 8P frames, and are not described herein.

Alternatively, in this embodiment, the smaller the coding distortion of the video, the lower the coding rate, the better the coding quality. The above coding quality may be measured, but is not limited to, using peak signal to Noise Ratio (PSNR), or a function of code rate and coding loss.

Alternatively, in the present embodiment, the decoding delay may be measured using, but not limited to, the encoding speed, or the delay time at the decoding end, or the like.

As an alternative embodiment, determining the coding information of the video frame to be coded comprises:

S11, determining target parameters corresponding to the target video, wherein the target parameters are used for balancing the coding quality and decoding delay of the video frame;

s12, constructing an objective function as the coding information according to the code rate, the coding loss, the decoding delay and the objective parameter corresponding to the video frame.

Alternatively, in the present embodiment, the encoded information may be represented by a function of a code rate, a coding loss, a decoding delay, and a target parameter configuration. Such as: and constructing an expression of a weighted sum of the code rate, the coding loss and the decoding delay by taking the target parameter as a weight.

As an alternative embodiment, the constructing an objective function as the encoding information according to the code rate, the encoding loss, the decoding delay, and the target parameter corresponding to the video frame may include, but is not limited to, the following:

s21, constructing a first function related to the code rate and the coding loss;

s22, constructing a second function related to the decoding delay;

s23, determining the weighted sum of the first function and the second function as the target function, wherein the weight of the first function is a difference value between 1 and the target parameter, and the weight of the second function is the target parameter.

Alternatively, in the present embodiment, the objective function may be expressed by the following formula:

J＝(1-λ)Q(R,D)+λL(Decode)

Where Q (R, D) represents the coding quality, which is a function (i.e. a first function) of the code rate R and the coding loss D. L (Decode) represents the delay time (i.e., the second function) at the decoding end, and is dynamic in relation to the frame it references, since it is possible to transform its coding mode when each frame is coded. Lambda is the target parameter.

Alternatively, in the present embodiment, the target parameter is used to balance the encoding quality and decoding delay of the video frame, indicating a tendency toward low delay when λ is large, and indicating a tendency toward encoding quality when λ is small. The value of lambda can be empirically set, and lambda >0.5 when the decoding delay requirement is high; when the coding quality requirement is higher, λ <0.5.

As an alternative embodiment, the target parameters corresponding to the target video may be determined, but are not limited to, by:

S31, determining the coding requirement of the coding scene of the target video;

s32, determining the target parameter from a first threshold range in the condition that the coding requirement of the coding scene indicates the video quality of the target video;

S33, determining the target parameter from a second threshold range in the condition that the coding requirement of the coding scene indicates the video delay of the target video, wherein the first threshold range is lower than the second threshold range.

Optionally, in this embodiment, different encoding scenarios have different requirements on the encoding process, such as: scenes such as live video and real-time communication have lower decoding delay, and scenes such as panoramic video and video playing have higher coding quality. According to different requirements of different coding scenes on the coding process, different target parameters are used for balancing coding quality and decoding delay.

Alternatively, in the present embodiment, if the encoding scene requires video quality, a smaller target parameter is selected, and if the encoding scene requires video delay, a larger target parameter is selected.

Alternatively, in the present embodiment, table 1 shows that the range of values of different target parameters λ tends to select the coding modes, as shown in table 1, λ is between 0.1 and 0.5, λ is between 6 and LDP, λ is 0.5, λ is between 5 and 1, and λ is between 0.5 and 1, and λ is between 1 and 1.

TABLE 1

Lambda range of values	Coding mode (GOP 8 coding mode)
		0.1<λ<0.5	Coding mode 6 to LDP coding
λ＝0.5	Coding mode 5
		0.5<λ<1	Coding modes 1 to 4

In the technical solution provided in step S204, the target coding mode set corresponding to the video frame includes a coding mode that allows the video frame to be used. Such as: for a P frame with POC 3, the target set of coding modes may include two or three of coding mode 1, coding mode 2, and coding mode 3.

As an optional embodiment, according to the encoding information, determining, from a set of target encoding modes corresponding to the video frame, a target encoding mode matching the video frame includes:

S41, determining the value of the objective function corresponding to each coding mode included in the target coding mode set;

s42, determining the coding mode with the minimum value of the corresponding objective function as the target coding mode.

Alternatively, in the present embodiment, a coding mode that minimizes the value of the objective function is used as the target coding mode to code the video frame to be coded. Such as: the target coding mode set includes: mode a, mode B and mode C, ja= (1- λ) q1+λl1 corresponding to mode a, jb= (1- λ) q2+λl2 corresponding to mode B, jc= (1- λ) q3+λl3 corresponding to mode C, wherein JB is the minimum value, mode B is determined as the target encoding mode.

Alternatively, in the present embodiment, the target encoding mode set may be obtained by, but not limited to, determining an encoding mode that allows the video frame to be used from the following encoding modes:

First coding mode: non-key frames in the group of pictures refer only to key frames in the group of pictures (i.e., mode 1).

Fig. 3 is a schematic diagram of an alternative coding mode 1 according to an alternative embodiment of the present application, as shown in fig. 3, all P frames refer to I frames only. POC (pictur order count) denotes the display order of the source video frames at the time of video encoding.

Second coding mode: the first non-key frame in the group of pictures refers only to the key frame in the group of pictures, the second non-key frame refers to the first non-key frame and key frame, and the third following non-key frame refers to the second non-key frame and key frame (i.e., mode 2).

Fig. 4 is a schematic diagram of an alternative coding mode 2 according to an alternative embodiment of the present application, in which, as shown in fig. 4, P frames with POC 1 refer to only I frames, P frames with POC 2 to 8 refer to I frames and P frames with POC 1.

The i-th coding mode: the first non-key frame in the group of pictures refers only to the key frame in the group of pictures, the i-th preceding and i-th non-key frame refers to the previous non-key frame and key frame, and the i-th following non-key frame refers to the i-th non-key frame and key frame, where i is an integer greater than 3 and less than N, and N is the number of non-key frames included in the group of pictures (i.e., modes 3 through 7).

Nth coding mode: each non-key frame in the group of images references a previous non-key frame and key frame (i.e., mode 8).

Fig. 5 is a schematic diagram of an alternative coding modes 3 to 8 according to an embodiment of the present application, and as shown in fig. 5, for coding modes 3 to 7, the first P frame refers to only the I frame, the I-th previous and I-th P frame refers to the previous P frame and I frame, and the I-th subsequent P frame refers to the I-th P frame and I frame. For coding mode 8, each P frame references a previous P frame and I frame.

N+1st coding mode: the mth and mth non-key frames in the image group refer to the previous non-key frame and key frame, and the mth and following non-key frames refer to the mth non-key frame and the previous non-key frame, wherein M is an integer greater than 2 and less than N (i.e., LDP mode).

Fig. 6 is a schematic diagram of an alternative LDP coding mode according to an embodiment of the application, as shown in fig. 6, the 4 th previous and 4 th P frame references the previous P frame and I frame, and the 4 th subsequent P frame references the 4 th P frame and the previous P frame.

It should be noted that, an optional LDP coding mode is given here, and the LDP coding mode in this embodiment may also be another kind of LDP coding mode, which is not limited in this embodiment.

In the technical scheme provided in step S206, the video frame to be encoded is encoded according to the target encoding mode, and the next non-key frame continues to determine its corresponding encoding mode through the above process.

According to an aspect of an embodiment of the present application, a method embodiment of decoding video is provided. Fig. 7 is a flowchart of an alternative video decoding method according to an embodiment of the present application, as shown in fig. 7, the method may include the steps of:

Step S702, determining decoding information of a video frame to be decoded, wherein the video frame to be decoded is a non-key frame in a target video, and the decoding information is used for indicating decoding quality and decoding delay of the video frame;

Step S704, according to the decoding information, determining a target decoding mode matched with the video frame from a target decoding mode set corresponding to the video frame, where the target decoding mode set includes a decoding mode allowing the video frame to be used, and only configuring a forward reference frame by a non-key frame in the decoding mode allowing the video frame to be used;

step S706, decoding the video frame according to the target decoding mode.

As an alternative embodiment, during the playing of the panoramic video, the decoding mode may be adaptively adjusted by, but not limited to, the following processes:

Determining decoding information of a video frame to be decoded in the video code stream to be switched, wherein the video frame to be decoded is a non-key frame in the video code stream to be switched, and the decoding information is used for indicating decoding quality and decoding delay of the video frame;

Decoding the video frame according to the target decoding mode to obtain a decoded video code stream;

displaying the decoded video code stream on a display device of the panoramic video.

Optionally, in this embodiment, when decoding a non-key frame in a video bitstream of a panoramic video, a target decoding mode used by each non-key frame is determined according to decoding information of the non-key frame for indicating decoding quality and decoding delay, so as to adaptively adjust a decoding process, so that the decoding process considers both a requirement of delay and a requirement of image quality, thereby ensuring low-delay transmission while ensuring image quality as much as possible, and improving experience of a user in watching the panoramic video.

As an alternative embodiment, during the playing of the video asset transmitted in real time, the decoding mode may be adaptively adjusted by, but not limited to, the following processes:

acquiring video resources transmitted in real time from a video transmission end;

Determining decoding information of a video frame to be decoded in the video resource transmitted in real time, wherein the video frame to be decoded is a non-key frame in a target video, and the decoding information is used for indicating the decoding quality and decoding delay of the video frame;

decoding the video frame according to the target decoding mode to obtain a decoded video resource;

and playing the decoded video resources at a video playing end.

Optionally, in this embodiment, in the process of performing real-time transmission of video, for each non-key frame in the video resource, the target decoding mode used by each non-key frame is determined according to the decoding information of the non-key frame for indicating the decoding quality and the decoding delay, so as to adaptively adjust the decoding process, so that the decoding process can consider both the requirement of delay and the requirement of image quality, thereby ensuring the image quality as much as possible while ensuring low-delay transmission, and improving the experience of the user in the process of performing real-time audio/video communication.

As an alternative embodiment, in the step S702, the decoding information of the video frame to be decoded may be determined, but is not limited to, in the following manner:

determining a target parameter corresponding to the target video, wherein the target parameter is used for balancing the decoding quality and the decoding delay of the video frame;

and constructing an objective function as the decoding information according to the code rate, the decoding loss, the decoding delay and the objective parameter corresponding to the video frame.

As an alternative embodiment, in the above-described process of determining the decoding information, the objective function may be configured as the decoding information in the following manner, but is not limited to:

Constructing a first function with respect to the code rate and the decoding loss;

constructing a second function related to the decoding delay;

and determining a weighted sum of the first function and the second function as the target function, wherein the weight of the first function is a difference value between 1 and the target parameter, and the weight of the second function is the target parameter.

As an alternative embodiment, in the above process of determining the decoding information, the following manner may be used to determine the target parameters corresponding to the target video, but is not limited to:

determining a decoding requirement of a decoding scene of the target video;

Determining the target parameter from a first threshold range in the event that a decoding requirement of the decoding scene indicates a video quality of the target video;

the target parameter is determined from a second threshold range, where the decoding requirement of the decoding scene indicates a video delay of the target video, wherein the first threshold range is lower than the second threshold range.

As an alternative embodiment, the target decoding mode that matches the video frame may be determined, but is not limited to, by:

Determining a value of the objective function corresponding to each decoding mode included in the set of target decoding modes;

And determining a decoding mode with the minimum value of the corresponding objective function as the target decoding mode.

As an alternative embodiment, before determining a target decoding mode matching the video frame from a target decoding mode set corresponding to the video frame according to the decoding information, the target decoding mode set may be obtained by, but is not limited to, the following ways:

determining that non-key frames in the group of pictures in the first decoding mode refer only to key frames in the group of pictures;

Determining that a first non-key frame in the image group in the second decoding mode only references key frames in the image group, the second non-key frame references the first non-key frame and the key frames, and the third non-key frame references the second non-key frame and the key frames;

Determining that a first non-key frame in an image group in an ith decoding mode only references key frames in the image group, the i previous and i non-key frames referring to previous non-key frames and key frames, and the i subsequent non-key frames referring to the i non-key frames and key frames, wherein i is an integer greater than 3 and less than N, and N is the number of non-key frames included in the image group;

determining that each non-key frame in the group of pictures in the nth decoding mode references a previous non-key frame and a key frame;

Determining that an Mth preceding non-key frame and an Mth non-key frame in an image group in an (n+1) th decoding mode refer to a previous non-key frame and a key frame, and an Mth following non-key frame refers to an Mth non-key frame and a previous non-key frame, wherein M is an integer greater than 2 and less than N;

And determining a decoding mode which is allowed to be used by the video frame from the first decoding mode, the second decoding mode, the ith decoding mode, the Nth decoding mode and the (n+1) th decoding mode to obtain the target decoding mode set.

It should be noted that, in this embodiment, the decoding process of the video is similar to the encoding process described above, and will not be described here again.

It should be noted that, for simplicity of description, the foregoing method embodiments are all described as a series of acts, but it should be understood by those skilled in the art that the present application is not limited by the order of acts described, as some steps may be performed in other orders or concurrently in accordance with the present application. Further, those skilled in the art will also appreciate that the embodiments described in the specification are all preferred embodiments, and that the acts and modules referred to are not necessarily required for the present application.

From the description of the above embodiments, it will be clear to a person skilled in the art that the method according to the above embodiments may be implemented by means of software plus the necessary general hardware platform, but of course also by means of hardware, but in many cases the former is a preferred embodiment. Based on such understanding, the technical solution of the present application may be embodied essentially or in a part contributing to the prior art in the form of a software product stored in a storage medium (such as ROM/RAM, magnetic disk, optical disk) comprising several instructions for causing an electronic device (which may be a mobile phone, a computer, a server, or a network device, etc.) to perform the method according to the embodiments of the present application.

According to another aspect of the embodiment of the present application, there is also provided a video encoding apparatus for implementing the video encoding method. Fig. 8 is a schematic diagram of an alternative video encoding apparatus according to an embodiment of the present application, as shown in fig. 8, the apparatus may include:

a first determining module 82, configured to determine encoding information of a video frame to be encoded, where the video frame to be encoded is a non-key frame in a target video, and the encoding information is used to indicate encoding quality and decoding delay of the video frame;

A second determining module 84, configured to determine, according to the encoding information, a target encoding mode that matches the video frame from a target encoding mode set corresponding to the video frame, where the target encoding mode set includes an encoding mode that allows the video frame to be used, and only forward reference frames are configured by non-key frames in the encoding mode that allows the video frame to be used;

an encoding module 86 is configured to encode the video frame according to the target encoding mode.

It should be noted that, the first determining module 82 in this embodiment may be used to perform step S202 in the embodiment of the present application, the second determining module 84 in this embodiment may be used to perform step S204 in the embodiment of the present application, and the encoding module 86 in this embodiment may be used to perform step S206 in the embodiment of the present application.

It should be noted that the above modules are the same as examples and application scenarios implemented by the corresponding steps, but are not limited to what is disclosed in the above embodiments. It should be noted that the above modules may be implemented in software or hardware as a part of the apparatus in the hardware environment shown in fig. 1.

By the module, the coding information of the current video frame to be coded is acquired, the target coding mode is determined for the video frame to be coded from a plurality of coding modes according to the coding information, and the video frame to be coded is coded according to the target coding mode, so that the relation between coding quality and decoding delay can be balanced in the coding process, and the non-key frames in the image group in each coding mode of the plurality of coding modes are only configured with forward reference frames, and each coding mode belongs to a coding mode with low delay, so that the aim of considering the decoding delay and the coding quality is fulfilled, the technical effect of improving the coding and decoding flexibility of the video is realized, and the technical problem of poor coding and decoding flexibility of the video in the related technology is solved.

As an alternative embodiment, the first determining module includes:

A first determining unit, configured to determine a target parameter corresponding to the target video, where the target parameter is used to balance coding quality and decoding delay of the video frame;

and the construction unit is used for constructing an objective function as the coding information according to the code rate, the coding loss, the decoding delay and the objective parameter corresponding to the video frame.

As an alternative embodiment, the construction unit is configured to:

Constructing a first function with respect to said code rate and said coding loss;

constructing a second function related to the decoding delay;

As an alternative embodiment, the first determining unit is configured to:

determining the coding requirements of the coding scene of the target video;

determining the target parameter from a first threshold range in the event that the encoding requirement of the encoding scene indicates the video quality of the target video;

The target parameter is determined from a second threshold range in the event that the encoding requirement of the encoding scene indicates a video delay of the target video, wherein the first threshold range is lower than the second threshold range.

As an alternative embodiment, the second determining module includes:

a second determining unit, configured to determine a value of the objective function corresponding to each coding mode included in the target coding mode set;

And a third determining unit, configured to determine, as the target coding mode, a coding mode in which the value of the corresponding objective function is minimum.

As an alternative embodiment, the device is further configured to:

Before determining a target coding mode matched with the video frame from a target coding mode set corresponding to the video frame according to the coding information, determining that a non-key frame in an image group in a first coding mode only references a key frame in the image group;

determining that a first non-key frame in the image group in the second coding mode only references key frames in the image group, the second non-key frame references the first non-key frame and the key frames, and the third non-key frame references the second non-key frame and the key frames;

Determining that a first non-key frame in an image group in an ith coding mode only references key frames in the image group, the i previous and i non-key frames referring to previous non-key frames and key frames, and the i subsequent non-key frames referring to the i non-key frames and key frames, wherein i is an integer greater than 3 and less than N, and N is the number of non-key frames included in the image group;

Determining that each non-key frame in the group of pictures in the nth coding mode references a previous non-key frame and a key frame;

Determining that an Mth preceding non-key frame and an Mth non-key frame in an image group under an N+1th coding mode refer to a previous non-key frame and a key frame, and an Mth following non-key frame refers to an Mth non-key frame and a previous non-key frame, wherein M is an integer greater than 2 and less than N;

and determining a coding mode which is allowed to be used by the video frame from the first coding mode, the second coding mode, the ith coding mode, the Nth coding mode and the (n+1) th coding mode to obtain the target coding mode set.

According to another aspect of the embodiment of the present application, there is also provided a video encoding apparatus for implementing the video decoding method. Fig. 9 is a schematic diagram of an alternative video decoding apparatus according to an embodiment of the present application, as shown in fig. 9, the apparatus may include:

A first determining module 92, configured to determine decoding information of a video frame to be decoded, where the video frame to be decoded is a non-key frame in a target video, and the decoding information is used to indicate decoding quality and decoding delay of the video frame;

A second determining module 94, configured to determine, according to the decoding information, a target decoding mode that matches the video frame from a target decoding mode set corresponding to the video frame, where the target decoding mode set includes a decoding mode that allows the video frame to be used, and a non-key frame in the decoding mode that allows the video frame to be used configures only a forward reference frame;

a decoding unit 96 for decoding the video frames according to the target decoding mode.

It should be noted that, the first determining module 92 in this embodiment may be used to perform step S702 in the embodiment of the present application, the second determining module 94 in this embodiment may be used to perform step S704 in the embodiment of the present application, and the decoding module 96 in this embodiment may be used to perform step S706 in the embodiment of the present application.

As an alternative embodiment, the first determining module is configured to:

Determining a target parameter corresponding to the target video, wherein the target parameter is used for balancing the encoding quality and decoding delay of the video frame;

and constructing an objective function as the coding information according to the code rate, the coding loss, the decoding delay and the objective parameter corresponding to the video frame.

As an alternative embodiment, the first determining module is configured to:

constructing a second function related to the decoding delay;

As an alternative embodiment, the first determining module is configured to:

determining the coding requirements of the coding scene of the target video;

As an alternative embodiment, the second determining module is configured to:

Determining a value of the objective function corresponding to each coding mode included in the target coding mode set;

And determining the coding mode with the minimum value of the corresponding objective function as the target coding mode.

As an alternative embodiment, the device is further configured to:

It should be noted that the above modules are the same as examples and application scenarios implemented by the corresponding steps, but are not limited to what is disclosed in the above embodiments. It should be noted that the above modules may be implemented in software or in hardware as part of the apparatus shown in fig. 1, where the hardware environment includes a network environment.

According to a further aspect of the embodiments of the present invention, there is also provided an electronic device for implementing the above-described video encoding method or video decoding method, as shown in fig. 10, the electronic device comprising a memory 1002 and a processor 1004, the memory 1002 having stored therein a computer program, the processor 1004 being arranged to perform the steps of any of the method embodiments described above by means of the computer program.

Alternatively, in this embodiment, the electronic apparatus may be located in at least one network device of a plurality of network devices of the computer network.

Alternatively, in the present embodiment, the above-described processor may be configured to execute the following steps by a computer program:

Encoding the video frame according to the target encoding mode

Optionally, in this embodiment, the above processor may be further configured to execute the following steps by a computer program:

Decoding the video frame according to the target decoding mode

Alternatively, it will be understood by those skilled in the art that the structure shown in fig. 10 is only schematic, and the electronic device may also be a smart phone (such as an Android mobile phone, an iOS mobile phone, etc.), a tablet computer, a palm computer, a mobile internet device (MobileInternetDevices, MID), a PAD, etc. Fig. 10 is not limited to the structure of the electronic device. For example, the electronic device may also include more or fewer components (e.g., network interfaces, etc.) than shown in FIG. 10, or have a different configuration than shown in FIG. 10.

The memory 1002 may be configured to store software programs and modules, such as program instructions/modules corresponding to the video encoding or decoding method and apparatus in the embodiments of the present invention, and the processor 1004 executes the software programs and modules stored in the memory 1002 to perform various functional applications and data processing, that is, implement the video encoding or decoding method described above. The memory 1002 may include high-speed random access memory, and may also include non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid state memory. In some examples, the memory 1002 may further include memory located remotely from the processor 1004, which may be connected to the terminal via a network. Examples of such networks include, but are not limited to, the internet, intranets, local area networks, mobile communication networks, and combinations thereof. The memory 1002 may be used for storing information or data in a video codec process, among other things. As an example, as shown in fig. 10, the memory 1002 may include, but is not limited to, a first determining module 10022, a second determining module 10024, and an encoding module 10026 in the video encoding apparatus. In addition, other module units in the video encoding apparatus may be included, but are not limited to, and are not described in detail in this example.

As another example, the memory 1002 may further include, but is not limited to, a first determining module, a second determining module, and a decoding module in the decoding apparatus of the video. In addition, other module units in the video decoding apparatus may be included, but are not limited to, and are not described in detail in this example.

Optionally, the transmission device 1006 is configured to receive or transmit data via a network. Specific examples of the network described above may include wired networks and wireless networks. In one example, the transmission means 1006 includes a network adapter (NetworkInterfaceController, NIC) that can be connected to other network devices and routers via a network cable to communicate with the internet or a local area network. In one example, the transmission device 1006 is a radio frequency (RadioFrequency, RF) module that is configured to communicate wirelessly with the internet.

In addition, the electronic device further includes: a display 1008 for displaying information or data in a video encoding and decoding process; and a connection bus 1010 for connecting the respective module parts in the above-described electronic device.

The embodiment of the application also provides a storage medium. Alternatively, in the present embodiment, the above-described storage medium may be used to execute the program code of the encoding method of video.

Alternatively, in this embodiment, the storage medium may be located on at least one network device of the plurality of network devices in the network shown in the above embodiment.

Alternatively, in the present embodiment, the storage medium is configured to store program code for performing the steps of:

And encoding the video frame according to the target encoding mode.

Optionally, in the present embodiment, the storage medium is further configured to store program code for performing the steps of:

and decoding the video frame according to the target decoding mode.

Alternatively, specific examples in this embodiment may refer to examples described in the foregoing embodiments, and this embodiment is not described herein.

Alternatively, in the present embodiment, the storage medium may include, but is not limited to: a usb disk, a Read-Only Memory (ROM), a random access Memory (RAM, random Access Memory), a removable hard disk, a magnetic disk, or an optical disk, or other various media capable of storing program codes.

The foregoing embodiment numbers of the present application are merely for the purpose of description, and do not represent the advantages or disadvantages of the embodiments.

The integrated units in the above embodiments may be stored in the above-described computer-readable storage medium if implemented in the form of software functional units and sold or used as separate products. Based on such understanding, the technical solution of the present application may be embodied in essence or a part contributing to the prior art or all or part of the technical solution in the form of a software product stored in a storage medium, comprising several instructions for causing one or more computer devices (which may be personal computers, servers or network devices, etc.) to perform all or part of the steps of the method described in the embodiments of the present application.

In the foregoing embodiments of the present application, the descriptions of the embodiments are emphasized, and for a portion of this disclosure that is not described in detail in this embodiment, reference is made to the related descriptions of other embodiments.

In several embodiments provided by the present application, it should be understood that the disclosed client may be implemented in other manners. The above-described embodiments of the apparatus are merely exemplary, and the division of the units, such as the division of the units, is merely a logical function division, and may be implemented in another manner, for example, multiple units or components may be combined or may be integrated into another system, or some features may be omitted, or not performed. Alternatively, the coupling or direct coupling or communication connection shown or discussed with each other may be through some interfaces, units or modules, or may be in electrical or other forms.

The units described as separate units may or may not be physically separate, and units shown as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

In addition, each functional unit in the embodiments of the present application may be integrated in one processing unit, or each unit may exist alone physically, or two or more units may be integrated in one unit. The integrated units may be implemented in hardware or in software functional units.

The foregoing is merely a preferred embodiment of the present application and it should be noted that modifications and adaptations to those skilled in the art may be made without departing from the principles of the present application, which are intended to be comprehended within the scope of the present application.

Claims

1. A method of encoding video, comprising:

encoding the video frame according to the target encoding mode;

Before determining a target coding mode matched with the video frame from a target coding mode set corresponding to the video frame according to the coding information, determining that a non-key frame in an image group in a first coding mode only references a key frame in the image group; determining that a first non-key frame in the image group in the second coding mode only references key frames in the image group, the second non-key frame references the first non-key frame and the key frames, and the third non-key frame references the second non-key frame and the key frames; determining that a first non-key frame in an image group in an ith coding mode only references key frames in the image group, the i previous and i non-key frames referring to previous non-key frames and key frames, and the i subsequent non-key frames referring to the i non-key frames and key frames, wherein i is an integer greater than 3 and less than N, and N is the number of non-key frames included in the image group; determining that each non-key frame in the group of pictures in the nth coding mode references a previous non-key frame and a key frame; determining that an Mth preceding non-key frame and an Mth non-key frame in an image group under an N+1th coding mode refer to a previous non-key frame and a key frame, and an Mth following non-key frame refers to an Mth non-key frame and a previous non-key frame, wherein M is an integer greater than 2 and less than N; and determining a coding mode which is allowed to be used by the video frame from the first coding mode, the second coding mode, the ith coding mode, the Nth coding mode and the (n+1) th coding mode to obtain the target coding mode set.

2. The method of claim 1, wherein determining the encoding information for the video frame to be encoded comprises:

3. The method of claim 2, wherein constructing an objective function as the encoding information based on the code rate, the coding loss, the decoding delay, and the target parameter corresponding to the video frame comprises:

constructing a second function related to the decoding delay;

4. The method of claim 2, wherein determining the target parameters corresponding to the target video comprises:

determining the coding requirements of the coding scene of the target video;

5. The method of claim 2, wherein determining a target coding mode that matches the video frame from a set of target coding modes corresponding to the video frame based on the coding information comprises:

6. A method of decoding video, comprising:

Decoding the video frame according to the target decoding mode;

Before determining a target decoding mode matched with the video frame from a target decoding mode set corresponding to the video frame according to the decoding information, determining that non-key frames in an image group in a first decoding mode only refer to key frames in the image group; determining that a first non-key frame in the image group in the second decoding mode only references key frames in the image group, the second non-key frame references the first non-key frame and the key frames, and the third non-key frame references the second non-key frame and the key frames; determining that a first non-key frame in an image group in an ith decoding mode only references key frames in the image group, the i previous and i non-key frames referring to previous non-key frames and key frames, and the i subsequent non-key frames referring to the i non-key frames and key frames, wherein i is an integer greater than 3 and less than N, and N is the number of non-key frames included in the image group; determining that each non-key frame in the group of pictures in the nth decoding mode references a previous non-key frame and a key frame; determining that an Mth preceding non-key frame and an Mth non-key frame in an image group in an (n+1) th decoding mode refer to a previous non-key frame and a key frame, and an Mth following non-key frame refers to an Mth non-key frame and a previous non-key frame, wherein M is an integer greater than 2 and less than N; and determining a decoding mode which is allowed to be used by the video frame from the first decoding mode, the second decoding mode, the ith decoding mode, the Nth decoding mode and the (n+1) th decoding mode to obtain the target decoding mode set.

7. The method of claim 6, wherein determining decoding information for a video frame to be decoded comprises:

8. The method of claim 7, wherein constructing an objective function as the decoding information based on the code rate, decoding loss, decoding delay, and the objective parameter corresponding to the video frame comprises:

constructing a second function related to the decoding delay;

9. The method of claim 7, wherein determining the target parameters corresponding to the target video comprises:

determining a decoding requirement of a decoding scene of the target video;

10. The method of claim 7, wherein determining a target decoding mode that matches the video frame from a set of target decoding modes corresponding to the video frame based on the decoding information comprises:

11. A video encoding apparatus, comprising:

the encoding module is used for encoding the video frames according to the target encoding mode;

The second determining module is further configured to: before determining a target coding mode matched with the video frame from a target coding mode set corresponding to the video frame according to the coding information, determining that a non-key frame in an image group in a first coding mode only references a key frame in the image group; determining that a first non-key frame in the image group in the second coding mode only references key frames in the image group, the second non-key frame references the first non-key frame and the key frames, and the third non-key frame references the second non-key frame and the key frames; determining that a first non-key frame in an image group in an ith coding mode only references key frames in the image group, the i previous and i non-key frames referring to previous non-key frames and key frames, and the i subsequent non-key frames referring to the i non-key frames and key frames, wherein i is an integer greater than 3 and less than N, and N is the number of non-key frames included in the image group; determining that each non-key frame in the group of pictures in the nth coding mode references a previous non-key frame and a key frame; determining that an Mth preceding non-key frame and an Mth non-key frame in an image group under an N+1th coding mode refer to a previous non-key frame and a key frame, and an Mth following non-key frame refers to an Mth non-key frame and a previous non-key frame, wherein M is an integer greater than 2 and less than N; and determining a coding mode which is allowed to be used by the video frame from the first coding mode, the second coding mode, the ith coding mode, the Nth coding mode and the (n+1) th coding mode to obtain the target coding mode set.

12. A video decoding apparatus, comprising:

A decoding unit configured to decode the video frame according to the target decoding mode;

The second determining module is further configured to: before determining a target decoding mode matched with the video frame from a target decoding mode set corresponding to the video frame according to the decoding information, determining that non-key frames in an image group in a first decoding mode only refer to key frames in the image group; determining that a first non-key frame in the image group in the second decoding mode only references key frames in the image group, the second non-key frame references the first non-key frame and the key frames, and the third non-key frame references the second non-key frame and the key frames; determining that a first non-key frame in an image group in an ith decoding mode only references key frames in the image group, the i previous and i non-key frames referring to previous non-key frames and key frames, and the i subsequent non-key frames referring to the i non-key frames and key frames, wherein i is an integer greater than 3 and less than N, and N is the number of non-key frames included in the image group; determining that each non-key frame in the group of pictures in the nth decoding mode references a previous non-key frame and a key frame; determining that an Mth preceding non-key frame and an Mth non-key frame in an image group in an (n+1) th decoding mode refer to a previous non-key frame and a key frame, and an Mth following non-key frame refers to an Mth non-key frame and a previous non-key frame, wherein M is an integer greater than 2 and less than N; and determining a decoding mode which is allowed to be used by the video frame from the first decoding mode, the second decoding mode, the ith decoding mode, the Nth decoding mode and the (n+1) th decoding mode to obtain the target decoding mode set.

13. A storage medium comprising a stored program, wherein the program when run performs the method of any one of the preceding claims 1 to 10.

14. An electronic device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, characterized in that the processor performs the method of any of the preceding claims 1 to 10 by means of the computer program.