CN110166771B

CN110166771B - Video encoding method, video encoding device, computer equipment and storage medium

Info

Publication number: CN110166771B
Application number: CN201810863443.1A
Authority: CN
Inventors: 黄晓政
Original assignee: Tencent Technology Shenzhen Co Ltd
Current assignee: Tencent Technology Shenzhen Co Ltd
Priority date: 2018-08-01
Filing date: 2018-08-01
Publication date: 2022-09-23
Anticipated expiration: 2038-08-01
Also published as: CN110166771A

Abstract

The present application relates to a video encoding method, apparatus, computer-readable storage medium and computer device, the method comprising: acquiring a video frame to be encoded, and acquiring a first encoding complexity corresponding to the video frame to be encoded; acquiring a second coding complexity corresponding to a forward video frame before a video frame to be coded, wherein the first coding complexity and the second coding complexity are obtained by pre-coding the corresponding video frame and calculating according to pre-coded data; determining a target coding mode corresponding to a video frame to be coded according to the first coding complexity and the second coding complexity; and coding the video frame to be coded according to the target coding mode to obtain corresponding coded data. The scheme provided by the application can reduce the video coding time and improve the video coding efficiency.

Description

Video encoding method, video encoding device, computer equipment and storage medium

Technical Field

The present application relates to the field of computer technologies, and in particular, to a video encoding method and apparatus, a computer device, and a storage medium.

Background

With the development of the technical field of computers, the application scenes of videos are wider and wider. The traditional encoder encodes each video frame in a video without distinction, and the encoding mode encodes scenes of video frames with low attention in an original video without distinction for the video frames with low attention, so that the encoding time is long easily, and the encoding efficiency is low.

Disclosure of Invention

In view of the foregoing, it is desirable to provide a video encoding method, apparatus, computer device and storage medium, which can reduce encoding time and improve encoding efficiency by encoding in different encoding modes according to the attention of video frames during video encoding.

A method of video encoding, the method comprising:

acquiring a video frame to be encoded, and acquiring a first encoding complexity corresponding to the video frame to be encoded;

acquiring a second coding complexity corresponding to a forward video frame before a video frame to be coded, wherein the first coding complexity and the second coding complexity are obtained by pre-coding the corresponding video frame and calculating according to pre-coded data;

determining a target coding mode corresponding to the video frame to be coded according to the first coding complexity and the second coding complexity;

and coding the video frame to be coded according to the target coding mode to obtain corresponding coded data.

A video encoding apparatus, the apparatus comprising:

the first coding complexity acquisition module is used for acquiring a video frame to be coded and acquiring first coding complexity corresponding to the video frame to be coded;

the second coding complexity acquisition module is used for acquiring a second coding complexity corresponding to a forward video frame before a video frame to be coded, wherein the first coding complexity and the second coding complexity are obtained by pre-coding the corresponding video frame and calculating according to pre-coded data;

the encoding mode determining module is used for determining a target encoding mode corresponding to the video frame to be encoded according to the first encoding complexity and the second encoding complexity;

and the video frame to be coded encoding module is used for encoding the video frame to be coded according to the target coding mode to obtain corresponding encoded data.

A computer device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, the processor implementing the steps when executing the program of:

A computer-readable storage medium having stored thereon a computer program which, when executed by a processor, causes the processor to perform the steps of:

determining a target coding mode corresponding to a video frame to be coded according to the first coding complexity and the second coding complexity;

The video coding method, the video coding device, the computer readable storage medium and the computer equipment obtain a video frame to be coded, obtain a first coding complexity corresponding to the video frame to be coded, obtain a second coding complexity corresponding to a forward video frame before the video frame to be coded, wherein the first coding complexity and the second coding complexity are obtained by pre-coding the corresponding video frame, determining a target coding mode corresponding to the video frame to be coded according to the first coding complexity and the second coding complexity, and coding the video frame to be coded according to the target coding mode to obtain corresponding coded data. The coding method of the video frame to be coded is determined according to the first coding complexity of the video frame to be coded and the second coding complexity of the forward video frame before the video frame to be coded, the attention of the video frame to be coded is determined according to the first coding complexity of the video frame to be coded and the second coding complexity of the forward video frame before the video frame to be coded, and the corresponding coding method is determined in a self-adaptive mode according to the attention of the video frame to be coded, so that the coding time can be reduced, and the coding efficiency is improved.

Drawings

FIG. 1 is a diagram of an exemplary video coding method;

FIG. 2 is a flowchart illustrating a video encoding method according to an embodiment;

FIG. 3 is a flow diagram illustrating steps for generating a first encoding complexity and a second encoding complexity in one embodiment;

FIG. 4 is a flowchart illustrating the target interframe complexity generation step in one embodiment;

FIG. 5 is a flowchart illustrating the inter-frame complexity generation step for each pre-encoded video block in one embodiment;

FIG. 6 is a flowchart illustrating the encoding complexity determination step for a current video frame according to one embodiment;

FIG. 7 is a flowchart illustrating an embodiment of determining an encoding mode corresponding to a video frame to be encoded according to a first encoding complexity and a second encoding complexity;

FIG. 8 is a flowchart illustrating a video encoding method according to another embodiment;

FIG. 8A is a diagram illustrating a video encoding method according to one embodiment;

FIG. 9 is a diagram illustrating an exemplary embodiment of a video encoding method;

FIG. 10 is a block diagram showing the structure of a video encoding apparatus according to one embodiment;

FIG. 11 is a block diagram showing the structure of a video encoding apparatus according to another embodiment;

FIG. 12 is a block diagram that illustrates the structure of a target inter-frame complexity generation module in one embodiment;

FIG. 13 is a block diagram of the structure of an encoding complexity determination module in one embodiment;

FIG. 14 is a block diagram showing a configuration of a computer device according to an embodiment.

Detailed Description

In order to make the objects, technical solutions and advantages of the present application more apparent, the present application is described in further detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of and not restrictive on the broad application.

Fig. 1 is a diagram of an application environment of a video encoding method according to an embodiment. Referring to fig. 1, the video encoding method is applied to a video encoding system. The video coding system includes a terminal 110 and a server 120. The terminal 110 and the server 120 are connected through a network. The terminal 110 may specifically be a desktop terminal or a mobile terminal, and the mobile terminal may specifically be at least one of a mobile phone, a tablet computer, a notebook computer, and the like. The server 120 may be implemented as a stand-alone server or a server cluster composed of a plurality of servers.

Specifically, the terminal 110 obtains a video frame to be encoded, obtains a first encoding complexity corresponding to the video frame to be encoded, and obtains a second encoding complexity corresponding to a forward video frame before the video frame to be encoded, where the first encoding complexity and the second encoding complexity are obtained by pre-encoding the corresponding video frame, and then obtaining the encoding mode corresponding to the video frame to be encoded according to pre-encoding data, and finally obtaining the corresponding encoded data by encoding the video frame to be encoded according to a target encoding mode. Or, the terminal 110 sends the video frame to be encoded to the server 120, after the server 120 obtains the video frame to be encoded, the first encoding complexity corresponding to the video frame to be encoded is obtained, the second encoding complexity corresponding to the forward video frame before the video frame to be encoded is obtained, the first encoding complexity and the second encoding complexity are obtained by pre-encoding the corresponding video frame, the encoding mode corresponding to the video frame to be encoded is determined according to the pre-encoding data, the video frame to be encoded is encoded according to the target encoding mode to obtain corresponding encoded data, and finally the encoded data are sent to the terminal 110.

In one embodiment, as shown in fig. 2, a video encoding method is provided. The embodiment is mainly illustrated by applying the method to the terminal 110 or the server 120 in fig. 1. Referring to fig. 2, the video encoding method specifically includes the following steps:

step 202, obtaining a video frame to be encoded, and obtaining a first encoding complexity corresponding to the video frame to be encoded.

The video frame to be encoded is a basic unit constituting a video to be encoded, and the video to be encoded is a frame sequence constituted by a plurality of video frames. The video to be encoded refers to video without video encoding, and the video encoding refers to a mode of converting a file in a certain video format into a file in another video format by a specific compression technology. The video frame to be encoded may be a video frame acquired in real time for instant messaging, or a video frame corresponding to a stored video.

The encoding complexity reflects the complexity of the video frame during encoding, and the higher the encoding complexity is, the longer the consumed encoding time is. The first encoding complexity corresponding to the video frame to be encoded may be calculated through different algorithms, such as calculating the first encoding complexity corresponding to the video frame to be encoded through texture detection of image content, or calculating the first encoding complexity corresponding to the video frame to be encoded through a pixel difference between the video frame to be encoded and a pre-encoding reference frame, where the pixel difference may be counted by Sum of Absolute Differences (SAD) or may be represented by a rate distortion cost, and so on. The first coding complexity can be obtained by pre-coding the video frame to be coded to obtain corresponding pre-coded data and then calculating according to the pre-coded data.

The pre-coding refers to a process of pre-processing the video frame to be coded before the video frame to be coded is subjected to video coding. The pre-coding process includes but is not limited to a process of dividing a video frame to be coded, and then performing pre-coding inter-frame prediction and pre-coding intra-frame prediction on each pre-coding video block generated in the process of dividing the video frame to be coded, and finally calculating to obtain pre-coding data. The pre-coding data refers to the coding data generated in the process of pre-coding the video frame to be coded.

Specifically, after the terminal acquires the video frame to be encoded, in order to reduce the encoding time of the last video encoder and thereby improve the encoding efficiency, the video frame to be encoded may be pre-encoded before the video frame to be encoded is subjected to video encoding, for example, the video frame to be encoded is pre-encoded first to obtain pre-encoded data corresponding to the video frame to be encoded, and finally, a first encoding complexity is obtained by calculation according to the pre-encoded data, and a higher first encoding complexity indicates a higher complexity of the encoding process of the video frame to be encoded. The process of pre-coding the video frame to be coded may be, but is not limited to, dividing the video frame to be coded to obtain a plurality of pre-coded video blocks, respectively performing pre-coded inter-frame prediction and pre-coded intra-frame prediction on each pre-coded video block to obtain corresponding target inter-frame complexity and target intra-frame complexity, and finally calculating according to the target inter-frame complexity and the target intra-frame complexity corresponding to each pre-coded video block to obtain the first coding complexity.

And 204, acquiring a second coding complexity corresponding to a forward video frame before the video frame to be coded, wherein the first coding complexity and the second coding complexity are obtained by pre-coding the corresponding video frame and calculating according to pre-coded data.

The forward video frame refers to a video frame before a video frame to be encoded, and may be an encoded forward video frame or a discarded non-encoded forward video frame. Specifically, the forward video frame of the video frame to be encoded can be obtained according to a preset rule. The predetermined rule may be customized, and the customization may be, but is not limited to, obtaining a predetermined number of video frames from all video frames before the video frame to be encoded as a forward video frame, and the like. The second coding complexity is coding complexity obtained by pre-coding the acquired forward video frame and calculating according to pre-coded data generated by pre-coding. In one embodiment, the first coding complexity and the second coding complexity are calculated by using the same type of complex algorithm.

Specifically, after the terminal acquires the video frame to be encoded, the terminal acquires a forward video frame before the video frame to be encoded according to a preset rule, and in order to calculate a second encoding complexity corresponding to the forward video frame, the forward video frame needs to be pre-encoded. The pre-coding may be, but not limited to, firstly dividing the forward video frame to obtain a plurality of pre-coded video blocks, then performing pre-coded inter-frame prediction and pre-coded intra-frame prediction on each pre-coded video block to obtain target inter-frame complexity and target intra-frame complexity corresponding to each pre-coded video block, and finally calculating to obtain second coding complexity corresponding to the forward video frame according to the target inter-frame complexity and the target intra-frame complexity corresponding to each pre-coded video block. The pre-coding is a coding mode which consumes less time than the actual coding process, and in order to ensure that the pre-coding consumes less time than the actual coding process, a fast prediction algorithm is adopted for pre-coding inter-frame prediction and pre-coding intra-frame prediction in the pre-coding process.

And step 206, determining a target coding mode corresponding to the video frame to be coded according to the first coding complexity and the second coding complexity.

The encoding mode is a mode of encoding a video frame to be encoded by a video encoder, and the encoding mode includes, but is not limited to, a first encoding mode, a second encoding mode, or the like. Wherein, different coding modes adopt different coding algorithms to code the video frame to be coded. Different coding algorithms mean that different coding parameters are set during coding, such as different motion search ranges, different quantization parameters, different rate distortion cost calculation modes, different code rate information, and the like. The first encoding mode may be to discard the entire video frame, that is, to not perform video encoding on the discarded video frame, and the second encoding mode may be to perform a fast encoding algorithm on the video frame, to reduce a motion search range, to perform a simple encoding mode, and the like. The encoding time corresponding to the encoding time of the first encoding mode is different from the encoding time corresponding to the second encoding mode, and specifically, the corresponding encoding time is less in time consumption through a fast encoding algorithm, wherein the video frame with high attention is encoded by using the encoding mode with less time consumption.

Specifically, after a first coding complexity corresponding to a video frame to be coded and a second coding complexity corresponding to a forward video frame before the video frame to be coded are obtained, a coding mode corresponding to the video frame to be coded is determined according to the first coding complexity and the second coding complexity. The method for determining the coding mode corresponding to the video frame to be coded according to the first coding complexity and the second coding complexity is self-defined, wherein the self-definition can be that a corresponding complexity difference is obtained according to the first coding complexity and the second coding complexity, a target coding mode corresponding to the video frame to be coded is determined according to the relation between the complexity difference and each candidate coding mode, or whether the complexity difference between the first coding complexity and the second coding complexity exceeds a preset range is judged, so that the target coding mode corresponding to the video frame to be coded is determined, or when the complexity difference between the first coding complexity and the second coding complexity reaches a preset threshold, the target coding mode corresponding to the video frame to be coded is determined, and the like. The complexity difference may be a difference between the first encoding complexity and the second encoding complexity, or a ratio of the first encoding complexity and the second encoding complexity, and so on. The relationship between the preset range and the complexity gap and each candidate encoding mode can be preset according to actual requirements.

And step 208, coding the video frame to be coded according to the target coding mode to obtain corresponding coded data.

The coded data is related data generated by a video coder when the video coder codes a video frame to be coded. Specifically, after the coding mode corresponding to the video frame to be coded is determined according to the first coding complexity and the second coding complexity, the video frame to be coded is coded according to the video coder corresponding to the target coding mode, so that the corresponding coded data is obtained. Where the video encoder is used to encode a video frame to be encoded, the video encoder may be, but is not limited to, a physical video encoder, a software video encoder, and the like.

In one embodiment, if the target coding mode is to discard the video frame to be coded, the video frame to be coded is discarded according to the target coding mode, which indicates that the video frame to be coded is a video frame with low attention, that is, a video frame with low user interest, and for the video frame to be coded with low attention or low interest, the video encoder is not required to be used for video coding, and the video frame to be coded is directly discarded, so that the coding time is reduced, and the coding efficiency is improved.

In one embodiment, if the encoding complexity of the video frame to be encoded is high, the target encoding mode is an encoding mode for performing simple video encoding by using a fast encoding algorithm, and the video frame to be encoded is subjected to simple video encoding by using the fast encoding algorithm according to the target encoding mode, so that the time consumed by video encoding can be reduced, and the encoding efficiency can be improved.

The video coding method comprises the steps of obtaining a video frame to be coded, obtaining a first coding complexity corresponding to the video frame to be coded, obtaining a second coding complexity corresponding to a forward video frame before the video frame to be coded, wherein the first coding complexity and the second coding complexity are obtained by pre-coding the corresponding video frame, calculating according to pre-coded data, determining a target coding mode corresponding to the video frame to be coded according to the first coding complexity and the second coding complexity, and coding the video frame to be coded according to the target coding mode to obtain corresponding coded data. The coding method of the video frame to be coded is determined according to the first coding complexity of the video frame to be coded and the second coding complexity of the forward video frame before the video frame to be coded, the attention of the video frame to be coded is determined according to the first coding complexity of the video frame to be coded and the second coding complexity of the forward video frame before the video frame to be coded, and the corresponding coding method is determined in a self-adaptive mode according to the attention of the video frame to be coded, so that the coding time can be reduced, and the coding efficiency is improved.

In one embodiment, as shown in fig. 3, the generating of the first encoding complexity and the second encoding complexity, when the first encoding complexity is calculated, the video frame to be encoded is taken as the current video frame, and when the second encoding complexity is calculated, the forward video frame is taken as the current video frame, includes:

step 302, a current video frame is segmented to obtain a plurality of pre-encoded video blocks.

The first coding complexity is obtained by performing pre-coding calculation on the video frame to be coded, so that when the first coding complexity is calculated, the video frame to be coded is taken as the current video frame, and similarly, when the second coding complexity is calculated, the forward video frame before the video frame to be coded is taken as the current video frame because the second coding complexity is obtained by performing pre-coding calculation on the forward video frame before the video frame to be coded.

Specifically, the current video frame may be segmented according to a preset segmentation rule to obtain a plurality of pre-encoded video blocks. The preset segmentation rule may be customized, and the customization may be, but is not limited to, uniformly segmenting the current video frame according to the preset same proportion, such as dividing the current video frame into 8 × 8 pre-encoded video blocks. Or the current video frame is divided according to the encoding rule of the encoder, for example, a partial region of the current video frame is divided into 8 × 8 pre-encoded video blocks, and the other partial region is divided into 16 × 16 pre-encoded video blocks.

And step 304, performing pre-coding inter-frame prediction on each pre-coding video block to obtain target inter-frame complexity corresponding to each pre-coding video block.

The inter-frame prediction refers to inter-frame prediction in a pre-coding process, and the inter-frame prediction is to use the correlation of a video time domain and use pixels of adjacent video frames to predict pixels of a current video frame so as to achieve the purpose of effectively removing video time domain redundancy. The pre-coding inter prediction includes, but is not limited to, motion estimation, motion compensation, etc., and the motion estimation includes, but is not limited to, motion search methods, motion estimation criteria, sub-pixel interpolation, and motion vector estimation.

Specifically, after a current video frame is divided to obtain a plurality of pre-coding video blocks, pre-coding inter-frame prediction is performed on each pre-coding video block to obtain target inter-frame complexity corresponding to each pre-coding video block. The pre-coding inter-frame prediction of each pre-coding video block can be performed by performing motion search on each pre-coding video block through a simple and rapid motion search algorithm, and the target inter-frame complexity corresponding to each pre-coding video block is obtained according to the motion search result. The motion search is a process of finding the best pre-coded video block in an allowed search range, and the target inter-frame complexity may be the target inter-frame complexity corresponding to each pre-coded video block by using a relative offset between each pre-coded video block and the corresponding best pre-coded video block.

And step 306, performing pre-coding intra-frame prediction on each pre-coding video block to obtain target intra-frame complexity corresponding to each pre-coding video block.

The pre-coding intra-frame prediction refers to intra-frame prediction in the pre-coding process, and the intra-frame prediction achieves the purpose of effectively removing video space redundancy according to a predicted pixel value and an actual pixel value by utilizing the spatial correlation of adjacent pre-coding video blocks. The pre-coding intra-frame prediction includes, but is not limited to, traversing all prediction modes to obtain residual signals under each prediction mode, and then transforming the residual signals to calculate absolute errors, SAD values and the like, wherein the prediction modes include a horizontal direction mode, a vertical direction mode and a DC mode.

Specifically, after a current video frame is divided to obtain a plurality of pre-coding video blocks, pre-coding intra-frame prediction is performed on each pre-coding video block to obtain target intra-frame complexity corresponding to each pre-coding video block. The pre-coding intra-frame prediction of each pre-coding video block can be simply that intra-frame prediction of a horizontal direction mode, a vertical direction mode and a DC mode is carried out on each pre-coding video block, so that residual signals corresponding to each pre-coding video block in each prediction mode are obtained, then transformation calculation is carried out on the residual signals of each pre-coding video block, so that absolute errors and SAD values of each pre-coding video block in each prediction mode are obtained, and finally, target SAD values are selected from the absolute errors and the SAD values in each prediction mode according to preset rules to serve as target intra-frame complexity corresponding to each pre-coding video block.

And 308, determining the coding complexity corresponding to the current video frame according to the target inter-frame complexity and the target intra-frame complexity corresponding to each pre-coded video block.

Specifically, after the target inter-frame complexity corresponding to each pre-coding video block is obtained by performing pre-coding inter-frame prediction on each pre-coding video block and the target intra-frame complexity corresponding to each pre-coding video block is obtained by performing pre-coding intra-frame prediction on each pre-coding video block, the coding complexity corresponding to the current video frame is determined according to the target inter-frame complexity and the target intra-frame complexity corresponding to each pre-coding video block. The method for determining the coding complexity corresponding to the current video frame can be customized according to the target inter-frame complexity and the target intra-frame complexity corresponding to each pre-coding video block, and the customization can be that the target complexity is selected from the target inter-frame complexity and the target intra-frame complexity corresponding to each pre-coding video block according to a first preset condition, then the target complexity of each pre-coding video block is counted to obtain the comprehensive complexity, and the comprehensive complexity is used as the coding complexity corresponding to the current video frame. Or firstly, the target inter-frame complexity and the target intra-frame complexity corresponding to each pre-coded video block are counted to obtain a comprehensive standard inter-frame complexity and a comprehensive target intra-frame complexity respectively, then the target comprehensive complexity is selected from the comprehensive standard inter-frame complexity and the comprehensive target intra-frame complexity according to a second preset condition, and the target comprehensive complexity is used as the coding complexity corresponding to the current video frame. The statistical process may be a summation calculation, a weighting calculation, or the like. The first preset condition and the second preset condition may be set in advance according to actual requirements.

In one embodiment, as shown in fig. 4, performing pre-coding inter prediction on each pre-coded video block to obtain a target inter complexity corresponding to each pre-coded video block includes:

step 402, obtaining a forward video frame of the current video frame, and obtaining pre-coding reference video blocks corresponding to the pre-coding video blocks from the forward video frame.

And when the first encoding complexity is calculated, taking the video frame to be encoded as the current video frame. And when calculating the second encoding complexity, the forward video frame is taken as the current video frame. The forward video frame of the current video frame refers to a video frame before the current video frame, and the forward video frame of the current video frame can be obtained according to a preset rule. The preset rule may be customized, and the customization may be, but is not limited to, obtaining a preset number of video frames from video frames before the current video frame as forward video frames, and the like. And further, acquiring pre-coded reference video blocks corresponding to the preset coded video blocks. The pre-coding reference video block is a video block used for reference in a pre-coding process, wherein the pre-coding reference block corresponding to each pre-coding video block is obtained, and the pre-coding reference video block corresponding to each pre-coding video block can be obtained according to a pre-setting reference rule. The preset reference rule includes, but is not limited to, a reference relationship between the current video frame and the reference frame, and the reference relationship may specifically be, but is not limited to, a resolution size relationship.

Specifically, a reference frame corresponding to the current video frame is obtained according to a reference relationship between the current video frame and the reference frame in a preset reference rule, the obtained reference frame is divided according to a preset division rule which is the same as the current video frame to obtain a plurality of reference video blocks, and a pre-coding reference video block corresponding to each pre-coding video block is obtained from the plurality of reference video blocks.

And 404, calculating to obtain first inter-frame complexity corresponding to each pre-coding video block according to the pixel values of each pre-coding video block and the corresponding pre-coding reference video block, and taking the first inter-frame complexity as target inter-frame complexity.

The inter-frame complexity is used to represent the complexity of the pre-coding inter-frame prediction of each pre-coded video block. The inter-frame complexity may be calculated from pixel values of each pre-encoded video block and the corresponding pre-encoded reference video block, or calculated from locations of each pre-encoded video block and the corresponding pre-encoded reference video block, and so on. The first inter-frame complexity is calculated based on the pixel values of each pre-encoded video block and the corresponding pre-encoded reference video block.

Specifically, after pre-coding reference video blocks corresponding to the pre-coding video blocks are acquired from a forward video frame, the first inter-frame complexity corresponding to each pre-coding video block is calculated according to the pre-coding video block and the pixel value of the corresponding pre-coding reference video block. Wherein, the pixel value can be calculated according to the RGB value of the image. The first inter-frame complexity corresponding to each pre-coding video block may be obtained by calculating according to the pixel values of each pre-coding video block and the corresponding pre-coding reference video block, but is not limited to, using a pixel value difference value of the pixel values of each pre-coding video block and the corresponding pre-coding reference video block as the first inter-frame complexity corresponding to each pre-coding video block, or calculating a pixel value difference value of the pixel values of each pre-coding video block and the corresponding pre-coding reference video block according to a preset proportion to obtain the first inter-frame complexity corresponding to each pre-coding video block, and the like. And finally, taking the first inter-frame complexity corresponding to each pre-coding video block as the target inter-frame complexity corresponding to each pre-coding video block.

In one embodiment, as shown in fig. 5, performing pre-coding inter prediction on each pre-coded video block to obtain the inter complexity corresponding to each pre-coded video block includes:

step 502, calculating the second inter-frame complexity corresponding to each pre-coding video block according to the position of each pre-coding video block and the corresponding pre-coding reference video block.

Wherein the second inter-frame complexity is calculated according to the positions of each pre-coded video block and the corresponding pre-coded reference video block, and the like. The second inter-frame complexity is calculated based on the locations of each pre-coded video block and the corresponding pre-coded reference video block. Wherein the second inter-frame complexity represents the motion complexity, which can be represented by a motion vector, and so on.

The second inter-frame complexity obtained by calculating according to the positions of each pre-coding video block and the corresponding pre-coding reference video block may be, but is not limited to, using a difference value between the positions of each pre-coding video block and the corresponding pre-coding reference video block as the second inter-frame complexity corresponding to each pre-coding video block, or calculating a difference value between the positions of each pre-coding video block and the corresponding pre-coding reference video block according to a preset proportion to obtain the second inter-frame complexity corresponding to each pre-coding video block, or using a ratio of the positions of each pre-coding video block and the corresponding pre-coding reference video block as the second inter-frame complexity corresponding to each pre-coding video block, and the like.

In one embodiment, if each pre-encoded video block and the corresponding pre-encoded reference video block have undergone image motion in a first dimension and a second dimension, the second inter-frame complexity corresponding to each pre-coding video block calculated according to the positions of each pre-coding video block and the corresponding pre-coding reference video block may be, but not limited to, first calculating a first-dimension inter-frame complexity according to the positions of each pre-coding video block and the corresponding pre-coding reference video block in the first dimension, and calculating to obtain a second dimension inter-frame complexity according to the position of each pre-coding video block and the corresponding pre-coding reference video block in the second dimension, and then, respectively calculating according to the first dimension interframe complexity of each pre-coding video block and the corresponding pre-coding reference video block in the first dimension and the second dimension interframe complexity in the second dimension to obtain the second interframe complexity corresponding to each pre-coding video block. The second inter-frame complexity corresponding to each pre-coding video block is obtained through calculation according to the first dimension inter-frame complexity of each pre-coding video block and the corresponding pre-coding reference video block in the first dimension and the second dimension inter-frame complexity of the corresponding pre-coding video block in the second dimension, but not limited to, the first dimension inter-frame complexity of each pre-coding video block and the corresponding pre-coding reference video block in the first dimension and the second dimension inter-frame complexity of the corresponding pre-coding reference video block in the second dimension are counted to obtain the comprehensive inter-frame complexity, and then the comprehensive inter-frame complexity corresponding to each pre-coding video block is used as the second inter-frame complexity corresponding to each pre-coding video block.

And step 504, calculating to obtain the target interframe complexity corresponding to each pre-coding video block according to the first interframe complexity and the second interframe complexity.

Specifically, after the first inter-frame complexity and the second inter-frame complexity corresponding to each pre-coding video block are obtained, the target inter-frame complexity corresponding to each pre-coding video block is obtained through calculation according to the first inter-frame complexity and the second inter-frame complexity. The target inter-frame complexity corresponding to each pre-coding video block obtained by calculation according to the first inter-frame complexity and the second inter-frame complexity may be, but is not limited to, taking an average value of the first inter-frame complexity and the second inter-frame complexity corresponding to each pre-coding video block as the target inter-frame complexity corresponding to each pre-coding video block. Or selecting the target inter-frame complexity corresponding to each pre-coding video block from the first inter-frame complexity and the second inter-frame complexity corresponding to each pre-coding video block according to a preset rule, wherein the preset rule can be customized, and the customization can be to use the inter-frame complexity with the minimum inter-frame complexity as the target inter-frame complexity and the like. Or performing weighted calculation on the first inter-frame complexity and the second inter-frame complexity corresponding to each pre-coding video block to obtain the comprehensive inter-frame complexity corresponding to each pre-coding video block, and taking the comprehensive inter-frame complexity as the target inter-frame complexity corresponding to each pre-coding video block.

In one embodiment, as shown in fig. 6, determining the encoding complexity corresponding to the current video frame according to the target inter-frame complexity and the target intra-frame complexity corresponding to each pre-encoded video block includes:

step 602, selecting a target complexity from the target inter-frame complexity and the target intra-frame complexity according to a preset condition.

And step 604, counting the target complexity corresponding to each pre-coding video block, and calculating to obtain the coding complexity corresponding to the current video frame.

Specifically, after each pre-coding video block is subjected to pre-coding inter-frame prediction and pre-coding intra-frame prediction respectively to obtain target inter-frame complexity and target intra-frame complexity corresponding to each pre-coding video block, the target complexity is selected from the target inter-frame complexity and the target intra-frame complexity according to preset conditions. The preset condition may be customized, and the customization may be to use a complexity with a low complexity of the target inter-frame complexity and the target intra-frame complexity as the target complexity, or use a complexity with a complexity lower than a preset complexity threshold of the target inter-frame complexity and the target intra-frame complexity as the target complexity, or use an average complexity of the target inter-frame complexity and the target intra-frame complexity as the target complexity, and the like.

Further, after the target complexity corresponding to each pre-coding video block is selected and obtained from the target inter-frame complexity and the target intra-frame complexity according to the preset conditions, the target complexity corresponding to each pre-coding video block is counted, and therefore the coding complexity corresponding to the current video frame is obtained through calculation. The statistical calculation mode may be self-defined, and the self-definition may be summation calculation, for example, the sum of the target complexities corresponding to the pre-coding video blocks is used as the coding complexity corresponding to the current video frame. Or the self-defining may be weighting calculation, for example, weighting calculation is performed on the target complexity corresponding to each pre-coded video block according to a preset weight to obtain a comprehensive target complexity, and the comprehensive target complexity is used as the coding complexity corresponding to the current video frame. Or the customization may be an average calculation, such as taking an average of target complexities corresponding to the respective pre-encoded video blocks as the encoding complexity corresponding to the current video frame. Or the customization may be a variance calculation, such as taking the variance of the target complexity corresponding to each pre-encoded video block as the encoding complexity corresponding to the current video frame.

In one embodiment, as shown in fig. 7, determining a coding mode corresponding to a video frame to be coded according to a first coding complexity and a second coding complexity includes:

step 702, when the first encoding complexity is greater than the second encoding complexity and the complexity difference between the first encoding complexity and the second encoding complexity exceeds a first preset range, determining that the encoding mode corresponding to the video frame to be encoded is the first encoding mode.

Specifically, determining the encoding mode corresponding to the video frame to be encoded according to the first encoding complexity and the second encoding complexity may be, when the first encoding complexity is greater than the second encoding complexity, indicating that the video frame to be encoded corresponding to the first encoding complexity is more complex than the forward video frame corresponding to the second encoding complexity, calculating a complexity difference between the first encoding complexity and the second encoding complexity, and then determining whether the complexity difference exceeds a first preset range, if the complexity difference between the first encoding complexity and the second encoding complexity exceeds the first preset range, indicating that the first encoding mode is more suitable for the video frame to be encoded, thereby determining that the encoding mode corresponding to the video frame to be encoded is the first encoding mode, and when the complexity difference between the first encoding complexity and the second encoding complexity does not exceed the first preset range, indicating that the first encoding mode is not suitable for the video frame to be encoded, therefore, the target encoding manner corresponding to the video frame to be encoded may be determined in step 704, or the target encoding manner corresponding to the video frame to be encoded may be directly determined to be the second encoding manner. The encoding time of the first encoding mode is less than that of the second encoding mode, and specifically, the encoding time of the first encoding mode is less than that of the second encoding mode through a fast encoding algorithm in the first encoding mode. If the complexity difference between the first encoding complexity and the second encoding complexity exceeds the first preset range, the complexity of the video frame to be encoded corresponding to the first encoding complexity is high, and the video frame needs to be encoded by using the first encoding mode, so that the encoding time is reduced, and the encoding efficiency is improved.

The complexity difference may be a complexity difference between the first encoding complexity and the second encoding complexity, or a ratio of the first encoding complexity and the second encoding complexity may be a complexity difference. The first preset range can be preset according to actual needs, and the first coding mode can be a coding mode that the whole video frame is directly discarded and no video coding is performed, so that the coding time can be reduced and the coding efficiency can be improved. Or the first encoding mode may be a simple video encoding mode for video frames, such as video encoding using a fast encoding algorithm.

In an embodiment, if it is determined that the target encoding mode corresponding to the video frame to be encoded is the first encoding mode, it indicates that the video frame to be encoded is the video frame content with low interest level and low interest level, and therefore, the video frame to be encoded can be discarded, which can improve the encoding efficiency, reduce the encoding time, and enable people to quickly view the content of interest.

Step 704, when the first encoding complexity is smaller than the second encoding complexity and the complexity difference between the first encoding complexity and the second encoding complexity exceeds a second predetermined range, determining the encoding mode corresponding to the video frame to be encoded as the second encoding mode.

Specifically, determining the encoding mode corresponding to the video frame to be encoded according to the first encoding complexity and the second encoding complexity may also be, when the first encoding complexity is smaller than the second encoding complexity, it indicates that the forward video frame corresponding to the second encoding complexity is more complex than the video frame to be encoded corresponding to the first encoding complexity, calculate a complexity difference between the first encoding complexity and the second encoding complexity, then determine whether the complexity difference exceeds a second preset range, if the complexity difference between the first encoding complexity and the second encoding complexity exceeds the second preset range, it indicates that the second encoding mode is more suitable for the video frame to be encoded, so it determines that the encoding mode corresponding to the video frame to be encoded is the second encoding mode, and when the complexity difference between the first encoding complexity and the second encoding complexity does not exceed the second preset range, it indicates that the second encoding mode is not suitable for the video frame to be encoded, therefore, the target encoding method corresponding to the video frame to be encoded may be determined in step 702, or the target encoding method corresponding to the video frame to be encoded may be directly determined as the first encoding method. When the first coding complexity is smaller than the second coding complexity, and the complexity difference between the first coding complexity and the second coding complexity exceeds a second preset range, the complexity of the video frame to be coded corresponding to the first coding complexity is low, the video frame does not need to be coded by using the first coding mode, and the video can be coded directly by using the second coding mode.

The complexity difference may be a complexity difference between the first coding complexity and the second coding complexity, or may be a ratio of the first coding complexity and the second coding complexity. The second preset range may be preset according to actual needs, the second encoding mode may be to perform video encoding on the video frame, and the first encoding mode may be to directly discard the entire video frame without performing any video encoding. The encoding time of the first encoding mode is lower than that of the second encoding mode.

In one embodiment, if it is determined that the target encoding mode corresponding to the video frame to be encoded is the second encoding mode, it is determined that the encoded video frame is the video frame content with high interest level and high interest level, and therefore, the video frame needs to be video-encoded, so that the video frame content is presented to people for viewing.

It should be noted that step 702 and step 704 are not executed sequentially, and may also be executed simultaneously, or step 704 is executed first and then step 702 is executed, or only step 704 is executed.

In one embodiment, as shown in fig. 8, the video encoding method further includes:

step 802, calculating a ratio of the first encoding complexity to the second encoding complexity, and when the ratio is greater than a first threshold, determining that a complexity difference between the first encoding complexity and the second encoding complexity exceeds a first preset range.

Step 804, when the ratio is smaller than the second threshold, it is determined that the complexity difference between the first encoding complexity and the second encoding complexity exceeds a second predetermined range.

The first threshold may be set in advance according to actual needs, and may be set to be a first threshold greater than 1. The second threshold may also be set in advance according to actual needs, and may be set to be a second threshold smaller than 1. Similarly, the first preset range and the second preset range may be set according to actual needs, such as specific needs of an application scenario. If the first predetermined range is set to be larger, the first threshold is adjusted accordingly, so that the first threshold can be adjusted to be larger. Similarly, if the second predetermined range is set to be larger, the second threshold is adjusted accordingly, so that the second threshold can be adjusted to be larger. The specific setting of the first preset range may be to combine the complexity differences smaller than the first threshold into a first preset range, and the specific setting of the second preset range may also be to combine the complexity differences smaller than the second threshold into a second preset range, where the first preset range and the second preset range may represent the complexity differences between the first coding complexity and the second coding complexity. The complexity difference may be a ratio of the first encoding complexity to the second encoding complexity, or a difference between the first encoding complexity and the second encoding complexity.

Specifically, a ratio of the first coding complexity to the second coding complexity is calculated, and then whether the ratio of the first coding complexity to the second coding complexity is greater than a first threshold is determined, and if the ratio of the first coding complexity to the second coding complexity is greater than the first threshold, it is determined that the complexity difference between the first coding complexity and the second coding complexity is greater than a first preset range. Similarly, if the ratio of the first coding complexity to the second coding complexity is smaller than the second threshold, which indicates that the complexity difference between the first coding complexity and the second coding complexity is larger, it is determined that the complexity difference between the first coding complexity and the second coding complexity exceeds the second predetermined range.

In one embodiment, the first threshold is set to a ratio threshold of 4 between the first coding complexity and the second coding complexity, the second threshold is set to a ratio threshold of 1/4 between the first coding complexity and the second coding complexity, and the first predetermined range is 0-4. Assuming that the first encoding complexity is 25 and the second encoding complexity is 5, the ratio of the first encoding complexity to the second encoding complexity is 5. And judging whether the ratio is greater than a first threshold value or not, and judging that the complexity difference between the first coding complexity and the second coding complexity exceeds a first preset range because the ratio 5 of the first coding complexity and the second coding complexity is greater than a first threshold value 4.

In one embodiment, the first threshold is set to a ratio threshold of 4 between the first encoding complexity and the second encoding complexity, the second threshold is set to a ratio threshold of 1/4 between the first encoding complexity and the second encoding complexity, and the second predetermined range is 0-1/4. Assuming that the first encoding complexity is 1 and the second encoding complexity is 8, the ratio of the first encoding complexity to the second encoding complexity is 1/8. And judging whether the ratio is smaller than a second threshold value or not, and judging that the complexity difference between the first coding complexity and the second coding complexity exceeds a second preset range because the ratio 1/8 of the first coding complexity and the second coding complexity is smaller than a second threshold value 1/4.

In one embodiment, determining a coding mode corresponding to a video frame to be coded according to a first coding complexity and a second coding complexity includes: and when the complexity difference between the first coding complexity and the second coding complexity is smaller than a third preset range, determining that the target coding mode corresponding to the video frame to be coded is the coding mode corresponding to the forward video frame.

The third preset range may be set according to actual needs, and specifically may be set according to a specific application scene of the video frame to be encoded. The third predetermined range may be set to 1, and the complexity difference may be a ratio of the first encoding complexity to the second encoding complexity, or a difference between the first encoding complexity and the second encoding complexity.

Specifically, when the complexity difference between the first encoding complexity and the second encoding complexity is smaller than a third preset range, it is determined that the target encoding mode corresponding to the video frame to be encoded is the encoding mode corresponding to the forward video frame. The third preset range may be set as an interval range, or may be set as a specific threshold. When the complexity difference between the first encoding complexity and the second encoding complexity is smaller than a third preset range, it is indicated that the complexity of the video frame to be encoded corresponding to the first encoding complexity is the same as or similar to the complexity of the forward video frame corresponding to the second encoding complexity, and therefore, the target encoding mode corresponding to the video frame to be encoded is the encoding mode corresponding to the forward video frame, so that the target encoding mode corresponding to the video frame to be encoded and the encoding mode corresponding to the forward video frame are kept unchanged.

In one embodiment, as shown in FIG. 8A, FIG. 8A shows a schematic diagram of a video encoding method in one embodiment. In fig. 8A, the 5 th frame is used as a video frame to be encoded, the 4 th frame is used as a forward video frame, and the 4 th frame meets the condition of the first encoding mode, so that the encoding mode of the 4 th frame video frame is the first encoding mode. As can be seen from the line graph in fig. 8A, if the encoding complexity corresponding to the 5 th frame video frame is the same as the encoding complexity corresponding to the 4 th frame video frame, the encoding mode corresponding to the 5 th frame video frame is directly determined to be the same as the encoding mode corresponding to the 4 th frame video frame, and both are the first encoding mode. Similarly, the 4 th frame, the 5 th frame, the 6 th frame, the 7 th frame and the 8 th frame in fig. 8A have the same encoding complexity, and thus the encoding methods of these frames are the same. If the first coding mode of the 4 th frame of video frame is to discard the whole video frame and not to perform video coding, the 4 th frame of video frame, the 5 th frame of video frame, the 6 th frame of video frame, the 7 th frame of video frame and the 8 th frame of video frame are all directly discarded and not to perform video coding, thereby improving the coding efficiency and reducing the coding time.

In one embodiment, the video frame to be encoded is a screen recording video frame to be encoded.

The video frame to be coded can be a video frame to be coded recorded on a screen in a scene recorded on the screen, the video frame to be coded recorded on the screen is directly acquired from screen pixels, and the screen recording means that all contents appearing on the screen are recorded. Specifically, the terminal collects pixels displayed on a screen into RGB data, converts the collected RGB data into a YUV color space to obtain converted data, and can record a video frame to be coded by using the converted data as the screen. The conversion of the acquired RGB data into the YUV color space can be obtained by a color conversion algorithm in the prior art, and will not be described herein.

In a scenario of a slideshow, as shown in fig. 9, fig. 9 shows an application scenario diagram of the video coding method in one embodiment, a terminal is performing a presentation of the slideshow, and for an effect of the slideshow, a rendering is often performed between two slides, for example, some animation effects are added. Where fig. 9a and 9d correspond to normal slide contents, i.e. slide contents presented to the user for viewing, while fig. 9b and 9c are animation effects in a slide application scenario, such as blinds, fade-in, etc., and fig. 9b and 9c are schematic diagrams of animation effects between the slides of fig. 9a cut into the slides of fig. 9 d. For the viewer, the viewer may not pay attention to the switching animation effect among the slides, but only pay attention to the content of the slides, so that the switching animation effect among the slides which are not paid attention to by people is discarded and is not coded, thereby reducing the coding time and improving the coding efficiency. Specifically, fig. 9a, 9b, 9c, and 9d are used as video frames to be encoded, a first encoding complexity corresponding to the video frames to be encoded is obtained, a second encoding complexity corresponding to a forward video frame before the video frames to be encoded is obtained, the first encoding complexity and the second encoding complexity are obtained by pre-encoding the corresponding video frames, a target encoding mode corresponding to the video frames to be encoded is determined according to the pre-encoded data, and the video frames to be encoded are encoded according to the target encoding mode to obtain corresponding encoded data. The encoding modes corresponding to fig. 9b and 9c may be the first encoding mode, for example, the video frame to be encoded is directly discarded without video encoding, so that the content corresponding to fig. 9b and 9c is discarded during encoding, and the content corresponding to fig. 9b and 9c does not exist after decoding and is not presented to the viewer for viewing, thereby reducing the encoding time and improving the encoding efficiency, while the content corresponding to fig. 9a and 9d is encoded during encoding, and thus the content corresponding to fig. 9a and 9d can be decoded and presented to the viewer for viewing.

In a specific embodiment, a video encoding method is provided, which specifically includes the following steps:

step 902, obtaining a video frame to be encoded, and obtaining a first encoding complexity corresponding to the video frame to be encoded. The video frame to be encoded may be a video frame to be encoded recorded on a screen.

Step 904, a second coding complexity corresponding to a forward video frame before the video frame to be coded is obtained, where the first coding complexity and the second coding complexity are obtained by performing pre-coding on the corresponding video frame and calculating according to pre-coded data.

Step 904a, when calculating the first encoding complexity, regard the video frame to be encoded as the current video frame; and when the second coding complexity is calculated, taking the forward video frame as the current video frame, and dividing the current video frame to obtain a plurality of pre-coding video blocks.

And 904b, performing pre-coding inter-frame prediction on each pre-coding video block to obtain target inter-frame complexity corresponding to each pre-coding video block.

Step 904ba, obtaining a forward video frame of the current video frame, and obtaining pre-encoded reference video blocks corresponding to the pre-encoded video blocks from the forward video frame.

Step 904bb, calculating to obtain a first inter-frame complexity corresponding to each pre-coding video block according to the pixel values of each pre-coding video block and the corresponding pre-coding reference video block, and taking the first inter-frame complexity as a target inter-frame complexity.

And 904bc, calculating to obtain second inter-frame complexity corresponding to each pre-coding video block according to the position of each pre-coding video block and the corresponding pre-coding reference video block.

And step 904bd, calculating to obtain target inter-frame complexity corresponding to each pre-coding video block according to the first inter-frame complexity and the second inter-frame complexity.

And 904c, performing pre-coding intra-frame prediction on each pre-coding video block to obtain target intra-frame complexity corresponding to each pre-coding video block.

And step 904d, determining the coding complexity corresponding to the current video frame according to the target inter-frame complexity and the target intra-frame complexity corresponding to each pre-coded video block.

And 904da, selecting the target complexity from the target inter-frame complexity and the target intra-frame complexity according to a preset condition.

And step 904db, counting the target complexity corresponding to each pre-coded video block, and calculating to obtain the coding complexity corresponding to the current video frame.

And 904dba, selecting the target complexity from the target inter-frame complexity and the target intra-frame complexity according to preset conditions.

And step 904dbb, counting the target complexity corresponding to each pre-coded video block, and calculating to obtain the coding complexity corresponding to the current video frame.

Step 906, determining a target coding mode corresponding to the video frame to be coded according to the first coding complexity and the second coding complexity.

Step 906a, when the first encoding complexity is greater than the second encoding complexity and the complexity difference between the first encoding complexity and the second encoding complexity exceeds a first preset range, determining that the encoding mode corresponding to the video frame to be encoded is the first encoding mode. And/or

Step 906b, when the first coding complexity is smaller than the second coding complexity and the complexity difference between the first coding complexity and the second coding complexity exceeds a second preset range, determining that the coding mode corresponding to the video frame to be coded is the second coding mode.

Step 906c, calculating a ratio of the first encoding complexity to the second encoding complexity, and when the ratio is greater than a first threshold, determining that the complexity difference between the first encoding complexity and the second encoding complexity exceeds a first preset range.

In step 906d, when the ratio is smaller than the second threshold, it is determined that the complexity difference between the first encoding complexity and the second encoding complexity exceeds a second predetermined range.

And 908, coding the video frame to be coded according to the target coding mode to obtain corresponding coded data.

Step 908a, when the complexity difference between the first encoding complexity and the second encoding complexity is smaller than a third predetermined range, determining that the target encoding mode corresponding to the video frame to be encoded is the encoding mode corresponding to the forward video frame.

It should be understood that, although the steps in the above-described flowcharts are shown in sequence as indicated by the arrows, the steps are not necessarily performed in sequence as indicated by the arrows. The steps are not performed in the exact order unless explicitly stated herein, and may be performed in other orders. Moreover, at least a portion of the steps in the above-described flowcharts may include multiple sub-steps or multiple stages, which are not necessarily performed at the same time, but may be performed at different times, and the order of performing the sub-steps or the stages is not necessarily sequential, but may be performed alternately or alternately with other steps or at least a portion of the sub-steps or stages of other steps.

As shown in fig. 10, in one embodiment, there is provided a video encoding apparatus 1000, including:

the first encoding complexity obtaining module 1002 is configured to obtain a video frame to be encoded, and obtain a first encoding complexity corresponding to the video frame to be encoded.

The second coding complexity obtaining module 1004 is configured to obtain a second coding complexity corresponding to a forward video frame before the video frame to be coded, where the first coding complexity and the second coding complexity are obtained by performing pre-coding on the corresponding video frame and calculating according to pre-coding data.

And the coding mode determining module 1006 is configured to determine a target coding mode corresponding to the video frame to be coded according to the first coding complexity and the second coding complexity.

And the to-be-coded video frame coding module 1008 is configured to code the to-be-coded video frame according to the target coding mode to obtain corresponding coded data.

In one embodiment, as shown in fig. 11, the video encoding apparatus 1000 further comprises:

when the first coding complexity is calculated, taking a video frame to be coded as a current video frame;

when calculating the second encoding complexity, taking the forward video frame as the current video frame;

a current video frame dividing module 1102, configured to divide a current video frame to obtain a plurality of pre-encoded video blocks.

And the target inter-frame complexity generating module 1104 is configured to perform pre-coding inter-frame prediction on each pre-coding video block to obtain target inter-frame complexity corresponding to each pre-coding video block.

And a target intra-frame complexity generating module 1106, configured to perform pre-coding intra-frame prediction on each pre-coded video block, so as to obtain a target intra-frame complexity corresponding to each pre-coded video block.

The encoding complexity determining module 1108 is configured to determine an encoding complexity corresponding to the current video frame according to the target inter-frame complexity and the target intra-frame complexity corresponding to each pre-encoded video block.

In one embodiment, as shown in FIG. 12, the target inter-frame complexity generation module 1104 includes:

a pre-coding reference video block obtaining unit 1104a, configured to obtain a forward video frame of the current video frame, and obtain pre-coding reference video blocks corresponding to the pre-coding video blocks from the forward video frame.

And the target inter-frame complexity calculation unit 1104b is configured to calculate a first inter-frame complexity corresponding to each pre-coding video block according to the pixel values of each pre-coding video block and the corresponding pre-coding reference video block, and use the first inter-frame complexity as the target inter-frame complexity.

In an embodiment, the target inter-frame complexity generating module 1104 is further configured to calculate a second inter-frame complexity corresponding to each pre-coded video block according to the position of each pre-coded video block and the corresponding pre-coded reference video block, and calculate a target inter-frame complexity corresponding to each pre-coded video block according to the first inter-frame complexity and the second inter-frame complexity.

In one embodiment, as shown in fig. 13, the encoding complexity determination module 1108 includes:

the target complexity selecting unit 1108a is configured to select a target complexity from the target inter-frame complexity and the target intra-frame complexity according to a preset condition.

And the coding complexity calculation unit 1108b is configured to count target complexities corresponding to the pre-coded video blocks, and calculate to obtain a coding complexity corresponding to the current video frame.

In one embodiment, the encoding mode determining module 1006 is further configured to determine, when the first encoding complexity is greater than the second encoding complexity and a complexity difference between the first encoding complexity and the second encoding complexity exceeds a first preset range, that the encoding mode corresponding to the video frame to be encoded is the first encoding mode; and/or determining the coding mode corresponding to the video frame to be coded as the second coding mode when the first coding complexity is smaller than the second coding complexity and the complexity difference between the first coding complexity and the second coding complexity exceeds a second preset range.

In one embodiment, the encoding manner determining module 1006 is further configured to calculate a ratio of the first encoding complexity to the second encoding complexity, and when the ratio is greater than a first threshold, determine that a complexity difference between the first encoding complexity and the second encoding complexity exceeds a first preset range; and when the ratio is smaller than a second threshold value, judging that the complexity difference between the first coding complexity and the second coding complexity exceeds a second preset range.

In an embodiment, the encoding mode determining module 1006 is further configured to determine that the target encoding mode corresponding to the video frame to be encoded is the encoding mode corresponding to the forward video frame when the complexity difference between the first encoding complexity and the second encoding complexity is smaller than a third preset range.

In one embodiment, the video frame to be encoded is recorded for the screen.

FIG. 14 is a diagram illustrating an internal structure of a computer device in one embodiment. The computer device may specifically be the terminal 110 in fig. 1. As shown in fig. 14, the computer apparatus includes a processor, a memory, a network interface, an input device, and a display screen, which are connected through a system bus. The memory comprises a nonvolatile storage medium and an internal memory. The non-volatile storage medium of the computer device stores an operating system and may also store a computer program that, when executed by the processor, causes the processor to implement the video encoding method. The internal memory may also have a computer program stored therein, which, when executed by the processor, causes the processor to perform the video encoding method. The display screen of the computer equipment can be a liquid crystal display screen or an electronic ink display screen, and the input device of the computer equipment can be a touch layer covered on the display screen, a key, a track ball or a touch pad arranged on the shell of the computer equipment, an external keyboard, a touch pad or a mouse and the like.

It will be appreciated by those skilled in the art that the configuration shown in fig. 14 is a block diagram of only a portion of the configuration associated with the present application, and is not intended to limit the computing device to which the present application may be applied, and that a particular computing device may include more or less components than those shown, or may combine certain components, or have a different arrangement of components.

In one embodiment, the video encoding apparatus provided herein may be implemented in the form of a computer program that is executable on a computer device such as that shown in fig. 14. The memory of the computer device may store various program modules constituting the video encoding apparatus, such as a first encoding complexity obtaining module, a second encoding complexity obtaining module, an encoding mode determining module, and a to-be-encoded video frame encoding module shown in fig. 10. The respective program modules constitute computer programs that cause the processors to execute the steps in the video encoding methods of the respective embodiments of the present application described in the present specification.

For example, the computer device shown in fig. 14 may perform, by using the first encoding complexity obtaining module in the video encoding apparatus shown in fig. 10, obtaining the video frame to be encoded, and obtaining the first encoding complexity corresponding to the video frame to be encoded. The computer equipment can execute the second coding complexity corresponding to the forward video frame before the video frame to be coded through a second coding complexity obtaining module, and the first coding complexity and the second coding complexity are obtained by pre-coding the corresponding video frame and calculating according to pre-coded data. The computer equipment can execute the target coding mode corresponding to the video frame to be coded according to the first coding complexity and the second coding complexity through the coding mode determining module. The computer equipment can execute the coding of the video frame to be coded according to the target coding mode through the video frame to be coded coding module to obtain the corresponding coded data.

In one embodiment, a computer device is proposed, the computer device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, the processor implementing the following steps when executing the computer program: acquiring a video frame to be encoded, and acquiring a first encoding complexity corresponding to the video frame to be encoded; acquiring a second coding complexity corresponding to a forward video frame before a video frame to be coded, wherein the first coding complexity and the second coding complexity are obtained by pre-coding the corresponding video frame and calculating according to pre-coded data; determining a target coding mode corresponding to the video frame to be coded according to the first coding complexity and the second coding complexity; and coding the video frame to be coded according to the target coding mode to obtain corresponding coded data.

In one embodiment, the generating of the first encoding complexity and the second encoding complexity comprises: when the first coding complexity is calculated, taking a video frame to be coded as a current video frame; when calculating the second encoding complexity, taking the forward video frame as the current video frame; segmenting a current video frame to obtain a plurality of pre-coding video blocks; performing pre-coding inter-frame prediction on each pre-coding video block to obtain target inter-frame complexity corresponding to each pre-coding video block; performing pre-coding intra-frame prediction on each pre-coding video block to obtain target intra-frame complexity corresponding to each pre-coding video block; and determining the coding complexity corresponding to the current video frame according to the target inter-frame complexity and the target intra-frame complexity corresponding to each pre-coding video block.

In one embodiment, the pre-coding inter prediction is performed on each pre-coded video block, and obtaining the target inter complexity corresponding to each pre-coded video block includes: acquiring a forward video frame of a current video frame, and acquiring pre-coding reference video blocks corresponding to the pre-coding video blocks from the forward video frame; and calculating to obtain first inter-frame complexity corresponding to each pre-coding video block according to the pixel values of each pre-coding video block and the corresponding pre-coding reference video block, and taking the first inter-frame complexity as target inter-frame complexity.

In one embodiment, the pre-coding inter prediction of each pre-coded video block to obtain the inter complexity corresponding to each pre-coded video block includes: calculating according to the positions of each pre-coding video block and the corresponding pre-coding reference video block to obtain second inter-frame complexity corresponding to each pre-coding video block; and calculating to obtain the target inter-frame complexity corresponding to each pre-coding video block according to the first inter-frame complexity and the second inter-frame complexity.

In one embodiment, determining the encoding complexity corresponding to the current video frame according to the target inter-frame complexity and the target intra-frame complexity corresponding to each pre-encoded video block includes: selecting a target complexity from the target inter-frame complexity and the target intra-frame complexity according to a preset condition; and counting the target complexity corresponding to each pre-coding video block, and calculating to obtain the coding complexity corresponding to the current video frame.

In one embodiment, determining a target encoding mode corresponding to a video frame to be encoded according to a first encoding complexity and a second encoding complexity includes: when the first coding complexity is larger than the second coding complexity and the complexity difference between the first coding complexity and the second coding complexity exceeds a first preset range, determining the coding mode corresponding to the video frame to be coded as a first coding mode; and/or determining the coding mode corresponding to the video frame to be coded as the second coding mode when the first coding complexity is smaller than the second coding complexity and the complexity difference between the first coding complexity and the second coding complexity exceeds a second preset range.

In one embodiment, the computer program further causes the processor to perform the steps of: calculating the ratio of the first coding complexity to the second coding complexity, and judging that the complexity difference between the first coding complexity and the second coding complexity exceeds a first preset range when the ratio is greater than a first threshold; and when the ratio is smaller than a second threshold value, judging that the complexity difference between the first coding complexity and the second coding complexity exceeds a second preset range.

In one embodiment, determining a target encoding mode corresponding to a video frame to be encoded according to a first encoding complexity and a second encoding complexity includes: and when the complexity difference between the first coding complexity and the second coding complexity is smaller than a third preset range, determining that the target coding mode corresponding to the video frame to be coded is the coding mode corresponding to the forward video frame.

In one embodiment, the video frame to be encoded is recorded for the screen.

In one embodiment, a computer readable storage medium is provided, having a computer program stored thereon, which, when executed by a processor, causes the processor to perform the steps of: acquiring a video frame to be encoded, and acquiring a first encoding complexity corresponding to the video frame to be encoded; acquiring a second coding complexity corresponding to a forward video frame before a video frame to be coded, wherein the first coding complexity and the second coding complexity are obtained by pre-coding the corresponding video frame and calculating according to pre-coded data; determining a target coding mode corresponding to the video frame to be coded according to the first coding complexity and the second coding complexity; and coding the video frame to be coded according to the target coding mode to obtain corresponding coded data.

In one embodiment, the generating of the first encoding complexity and the second encoding complexity comprises: when the first encoding complexity is calculated, taking a video frame to be encoded as a current video frame; when calculating the second encoding complexity, taking the forward video frame as the current video frame; dividing a current video frame to obtain a plurality of pre-coded video blocks; performing pre-coding inter-frame prediction on each pre-coding video block to obtain target inter-frame complexity corresponding to each pre-coding video block; performing pre-coding intra-frame prediction on each pre-coding video block to obtain target intra-frame complexity corresponding to each pre-coding video block; and determining the coding complexity corresponding to the current video frame according to the target inter-frame complexity and the target intra-frame complexity corresponding to each pre-coding video block.

It will be understood by those skilled in the art that all or part of the processes of the methods of the embodiments described above can be implemented by a computer program, which can be stored in a non-volatile computer-readable storage medium, and can include the processes of the embodiments of the methods described above when the program is executed. Any reference to memory, storage, database, or other medium used in the embodiments provided herein may include non-volatile and/or volatile memory, among others. Non-volatile memory can include read-only memory (ROM), Programmable ROM (PROM), Electrically Programmable ROM (EPROM), Electrically Erasable Programmable ROM (EEPROM), or flash memory. Volatile memory can include Random Access Memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in a variety of forms, such as Static RAM (SRAM), Dynamic RAM (DRAM), Synchronous DRAM (SDRAM), Double Data Rate SDRAM (DDRSDRAM), Enhanced SDRAM (ESDRAM), Synchronous Link DRAM (SLDRAM), Rambus Direct RAM (RDRAM), direct bus dynamic RAM (DRDRAM), and bus dynamic RAM (RDRAM).

All possible combinations of the technical features in the above embodiments may not be described for the sake of brevity, but should be considered as being within the scope of the present disclosure as long as there is no contradiction between the combinations of the technical features.

The above-mentioned embodiments only express several embodiments of the present application, and the description thereof is more specific and detailed, but not construed as limiting the scope of the present application. It should be noted that various changes and modifications can be made by those skilled in the art without departing from the spirit and scope of the invention. Therefore, the protection scope of the present patent should be subject to the appended claims.

Claims

1. A video encoding method, comprising:

acquiring a second coding complexity corresponding to a forward video frame before the video frame to be coded, wherein the first coding complexity and the second coding complexity are obtained by pre-coding the corresponding video frame and calculating according to pre-coded data; the first coding complexity and the second coding complexity are determined according to target complexity corresponding to each pre-coding video block in a current video frame, each pre-coding video block is obtained by dividing the current video frame, the target complexity is obtained by calculating according to target inter-frame complexity and target intra-frame complexity corresponding to the same pre-coding video block, the current video frame corresponding to the first coding complexity is the video frame to be coded, and the current video frame corresponding to the second coding complexity is the forward video frame;

determining a target coding mode corresponding to the video frame to be coded according to the first coding complexity and the second coding complexity, wherein the target coding mode comprises the following steps: when the first coding complexity is greater than the second coding complexity and the complexity difference between the first coding complexity and the second coding complexity exceeds a first preset range, determining that a target coding mode corresponding to the video frame to be coded is a first coding mode, wherein the first coding mode is to discard the video frame to be coded;

2. The method of claim 1, wherein the generating of the first coding complexity and the second coding complexity comprises:

when the first coding complexity is calculated, taking the video frame to be coded as a current video frame;

when calculating a second encoding complexity, taking the forward video frame as a current video frame;

dividing the current video frame to obtain a plurality of pre-coding video blocks;

performing pre-coding inter-frame prediction on each pre-coding video block to obtain target inter-frame complexity corresponding to each pre-coding video block;

performing pre-coding intra-frame prediction on each pre-coding video block to obtain target intra-frame complexity corresponding to each pre-coding video block;

and determining the coding complexity corresponding to the current video frame according to the target inter-frame complexity and the target intra-frame complexity corresponding to each pre-coding video block.

3. The method of claim 2, wherein the pre-coding inter-prediction for each pre-coded video block to obtain the target inter-frame complexity for each pre-coded video block comprises:

acquiring a forward video frame of the current video frame, and acquiring pre-coding reference video blocks corresponding to the pre-coding video blocks from the forward video frame;

and calculating to obtain a first inter-frame complexity corresponding to each pre-coding video block according to the pixel values of each pre-coding video block and the corresponding pre-coding reference video block, and taking the first inter-frame complexity as the target inter-frame complexity.

4. The method of claim 2, wherein the pre-coding inter-prediction for each pre-coded video block to obtain the inter-frame complexity for each pre-coded video block comprises:

calculating to obtain first inter-frame complexity corresponding to each pre-coding video block according to the pixel values of each pre-coding video block and the corresponding pre-coding reference video block;

calculating according to the positions of the pre-coding video blocks and the corresponding pre-coding reference video blocks to obtain second inter-frame complexity corresponding to the pre-coding video blocks;

and calculating to obtain the target inter-frame complexity corresponding to each pre-coding video block according to the first inter-frame complexity and the second inter-frame complexity.

5. The method of claim 2, wherein determining the encoding complexity corresponding to the current video frame according to the target inter-frame complexity and the target intra-frame complexity corresponding to the pre-encoded video blocks comprises:

taking the complexity between target frames and the average complexity of the complexity in the target frames as the target complexity;

and counting the target complexity corresponding to each pre-coding video block, and calculating to obtain the coding complexity corresponding to the current video frame.

6. The method of claim 1, wherein determining the target coding mode corresponding to the video frame to be coded according to the first coding complexity and the second coding complexity comprises:

and when the first coding complexity is smaller than the second coding complexity and the complexity difference between the first coding complexity and the second coding complexity exceeds a second preset range, determining that the target coding mode corresponding to the video frame to be coded is a second coding mode, wherein the coding time of the second coding mode is longer than that of the first coding mode.

7. The method of claim 6, further comprising:

calculating a ratio of the first coding complexity to the second coding complexity, and when the ratio is greater than a first threshold value, judging that the complexity difference between the first coding complexity and the second coding complexity exceeds a first preset range;

and when the ratio is smaller than a second threshold value, judging that the complexity difference between the first coding complexity and the second coding complexity exceeds a second preset range.

8. The method according to claim 1, wherein the determining the target encoding mode corresponding to the video frame to be encoded according to the first encoding complexity and the second encoding complexity comprises:

and when the complexity difference between the first coding complexity and the second coding complexity is smaller than a third preset range, determining that the target coding mode corresponding to the video frame to be coded is the coding mode corresponding to the forward video frame.

9. The method of claim 1, wherein the video frame to be encoded is a screen recorded video frame to be encoded.

10. A video encoding apparatus, characterized in that the apparatus comprises:

a second coding complexity obtaining module, configured to obtain a second coding complexity corresponding to a forward video frame before the video frame to be coded, where the first coding complexity and the second coding complexity are obtained by performing pre-coding on the corresponding video frame and calculating according to pre-coding data; the first coding complexity and the second coding complexity are determined according to target complexity corresponding to each pre-coding video block in a current video frame, each pre-coding video block is obtained by dividing the current video frame, the target complexity is obtained by calculating according to target inter-frame complexity and target intra-frame complexity corresponding to the same pre-coding video block, the current video frame corresponding to the first coding complexity is the video frame to be coded, and the current video frame corresponding to the second coding complexity is the forward video frame;

a coding mode determining module, configured to determine a target coding mode corresponding to the video frame to be coded according to the first coding complexity and the second coding complexity, where the coding mode determining module includes: when the first coding complexity is greater than the second coding complexity and the complexity difference between the first coding complexity and the second coding complexity exceeds a first preset range, determining that a target coding mode corresponding to the video frame to be coded is a first coding mode, and discarding the video frame to be coded;

and the video frame to be coded coding module is used for coding the video frame to be coded according to the target coding mode to obtain corresponding coded data.

11. The apparatus of claim 10, wherein the apparatus comprises:

the current video frame segmentation module is used for segmenting the current video frame to obtain a plurality of pre-coding video blocks;

the target inter-frame complexity generation module is used for performing pre-coding inter-frame prediction on each pre-coding video block to obtain target inter-frame complexity corresponding to each pre-coding video block;

the target intra-frame complexity generating module is used for carrying out pre-coding intra-frame prediction on each pre-coding video block to obtain target intra-frame complexity corresponding to each pre-coding video block;

and the encoding complexity determining module is used for determining the encoding complexity corresponding to the current video frame according to the target inter-frame complexity and the target intra-frame complexity corresponding to each pre-encoding video block.

12. The apparatus of claim 11, wherein the target inter-frame complexity generation module comprises:

a pre-coding reference video block obtaining unit, configured to obtain a forward video frame of the current video frame, and obtain pre-coding reference video blocks corresponding to the pre-coding video blocks from the forward video frame;

and the target inter-frame complexity calculating unit is used for calculating and obtaining the first inter-frame complexity corresponding to each pre-coding video block according to the pixel values of each pre-coding video block and the corresponding pre-coding reference video block, and taking the first inter-frame complexity as the target inter-frame complexity.

13. The apparatus of claim 11, wherein the target inter-frame complexity generation module is further configured to obtain a forward video frame of the current video frame, and obtain pre-encoded reference video blocks corresponding to the pre-encoded video blocks from the forward video frame; calculating to obtain first inter-frame complexity corresponding to each pre-coding video block according to the pixel values of each pre-coding video block and the corresponding pre-coding reference video block; calculating to obtain second inter-frame complexity corresponding to each pre-coding video block according to the position of each pre-coding video block and the corresponding pre-coding reference video block; and calculating to obtain the target inter-frame complexity corresponding to each pre-coding video block according to the first inter-frame complexity and the second inter-frame complexity.

14. The apparatus of claim 11, wherein the encoding complexity determination module comprises:

the target complexity selecting unit is used for taking the complexity between target frames and the average complexity of the complexity in the target frames as the target complexity;

and the coding complexity calculating unit is used for counting the target complexity corresponding to each pre-coding video block and calculating to obtain the coding complexity corresponding to the current video frame.

15. The apparatus of claim 10, wherein the encoding method determining module is further configured to determine that the target encoding method corresponding to the video frame to be encoded is the second encoding method when the first encoding complexity is smaller than the second encoding complexity and a complexity difference between the first encoding complexity and the second encoding complexity exceeds a second predetermined range, and an encoding time of the second encoding method is longer than an encoding time of the first encoding method.

16. The apparatus of claim 15, wherein the coding scheme determining module is further configured to calculate a ratio of the first coding complexity to the second coding complexity, and determine that a complexity difference between the first coding complexity and the second coding complexity exceeds a first preset range when the ratio is greater than a first threshold; and when the ratio is smaller than a second threshold value, judging that the complexity difference between the first coding complexity and the second coding complexity exceeds a second preset range.

17. The apparatus of claim 10, wherein the encoding scheme determining module is further configured to determine that the target encoding scheme corresponding to the video frame to be encoded is the encoding scheme corresponding to the forward video frame when the complexity difference between the first encoding complexity and the second encoding complexity is smaller than a third predetermined range.

18. The apparatus of claim 10, wherein the video frame to be encoded is a screen recorded video frame to be encoded.

19. A computer-readable storage medium, storing a computer program which, when executed by a processor, causes the processor to carry out the steps of the method according to any one of claims 1 to 9.

20. A computer device comprising a memory and a processor, the memory storing a computer program that, when executed by the processor, causes the processor to perform the steps of the method according to any one of claims 1 to 9.