CN110166770B

CN110166770B - Video encoding method, video encoding device, computer equipment and storage medium

Info

Publication number: CN110166770B
Application number: CN201810803554.3A
Authority: CN
Inventors: 黄晓政
Original assignee: Tencent Technology Shenzhen Co Ltd
Current assignee: Tencent Technology Shenzhen Co Ltd
Priority date: 2018-07-18
Filing date: 2018-07-18
Publication date: 2022-09-23
Anticipated expiration: 2038-07-18
Also published as: CN110166770A

Abstract

The present application relates to a video encoding method, apparatus, computer-readable storage medium and computer device, the method comprising: acquiring a video frame to be coded, and acquiring image motion information corresponding to the video frame to be coded, wherein the image motion information comprises at least one of first image motion information and second image motion information; the first image motion information is obtained by pre-coding a video frame to be coded; the second image motion information is obtained according to the coded data of the historical coded video frame before the video frame to be coded; determining a target coding mode corresponding to a video frame to be coded according to the image motion information; and coding the video frame to be coded according to the target coding mode to obtain corresponding coded data. The scheme provided by the application can adaptively select a proper target coding mode for coding aiming at different video contents, thereby improving the coding efficiency.

Description

Video encoding method, video encoding device, computer equipment and storage medium

Technical Field

The present application relates to the field of computer technologies, and in particular, to a video encoding method, apparatus, computer device, and storage medium.

Background

With the development of the technical field of computers, the application scenes of videos are wider and wider. The conventional encoder encodes each video frame in a video without distinction, and for a scene in which a target video is suddenly inserted into an original video, the encoding mode corresponding to the original video is still adopted for encoding the target video which is suddenly inserted, so that the encoding efficiency is low.

Disclosure of Invention

In view of the foregoing, it is desirable to provide a video encoding method, an apparatus, a computer device and a storage medium, which can adaptively select an appropriate target encoding method for different video contents to encode in a video encoding process, so as to improve encoding efficiency.

A method of video encoding, the method comprising:

acquiring a video frame to be coded, and acquiring image motion information corresponding to the video frame to be coded, wherein the image motion information comprises at least one of first image motion information and second image motion information;

the first image motion information is obtained by pre-coding a video frame to be coded;

the second image motion information is obtained according to the coded data of the historical coded video frame before the video frame to be coded;

determining a target coding mode corresponding to a video frame to be coded according to the image motion information;

and coding the video frame to be coded according to the target coding mode to obtain corresponding coded data.

A video encoding apparatus, the apparatus comprising:

the device comprises a to-be-coded video frame acquisition module, a coding module and a coding module, wherein the to-be-coded video frame acquisition module is used for acquiring a to-be-coded video frame and acquiring image motion information corresponding to the to-be-coded video frame, and the image motion information comprises at least one of first image motion information and second image motion information;

the target coding mode acquisition module is used for determining a target coding mode corresponding to the video frame to be coded according to the image motion information;

and the video frame to be coded encoding module is used for encoding the video frame to be coded according to the target coding mode to obtain corresponding encoded data.

A computer device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, the processor implementing the steps when executing the program of:

A computer-readable storage medium having stored thereon a computer program which, when executed by a processor, causes the processor to perform the steps of:

The video coding method, the video coding device, the computer readable storage medium and the computer device are used for obtaining a video frame to be coded, obtaining image motion information corresponding to the video frame to be coded, wherein the image motion information comprises at least one of first image motion information and second image motion information, the first image motion information is obtained by pre-coding the video frame to be coded, the second image motion information is obtained according to coded data of historical coded video frames before the video frame to be coded, a target coding mode corresponding to the video frame to be coded is determined according to the image motion information, and the corresponding coded data is obtained by coding the video frame to be coded according to the target coding mode. Because the image motion information corresponding to different video contents is different and the target coding modes corresponding to different video contents are also different, a suitable target coding mode can be adaptively selected for coding different video contents, so that the coding efficiency is improved.

Drawings

FIG. 1 is a diagram of an exemplary video coding method;

FIG. 2 is a flow diagram illustrating a video encoding method according to one embodiment;

FIG. 3 is a flow chart illustrating the step of generating first image motion information according to one embodiment;

FIG. 4 is a flow chart illustrating the step of generating statistical motion fluctuation information according to an embodiment;

FIG. 5 is a flowchart illustrating the second image motion information generation step in one embodiment;

FIG. 6 is a flowchart illustrating the step of determining a target encoding scheme according to an embodiment;

FIG. 7 is a flowchart illustrating a video encoding method according to another embodiment;

FIG. 7A is a diagram illustrating an exemplary video encoding method;

FIG. 8 is a block diagram showing the structure of a video encoding apparatus according to one embodiment;

FIG. 9 is a block diagram illustrating an exemplary video encoding apparatus according to another embodiment;

FIG. 10 is a block diagram of a statistical motion fluctuation information calculation module in one embodiment;

FIG. 11 is a block diagram showing a structure of a video encoding apparatus according to still another embodiment;

FIG. 12 is a block diagram showing a structure of a video encoding apparatus according to still another embodiment;

FIG. 13 is a block diagram that illustrates the architecture of a computing device in one embodiment.

Detailed Description

In order to make the objects, technical solutions and advantages of the present application more apparent, the present application is described in further detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of and not restrictive on the broad application.

FIG. 1 is a diagram of an embodiment of a video coding method. Referring to fig. 1, the video encoding method is applied to a video encoding system. The video coding system includes a terminal 110 and a server 120. The terminal 110 and the server 120 are connected through a network. The terminal 110 may specifically be a desktop terminal or a mobile terminal, and the mobile terminal may specifically be at least one of a mobile phone, a tablet computer, a notebook computer, and the like. The server 120 may be implemented as a stand-alone server or a server cluster composed of a plurality of servers.

Specifically, the terminal 110 obtains a video frame to be encoded, obtains image motion information corresponding to the video frame to be encoded, and the image motion information includes at least one of first image motion information and second image motion information, where the first image motion information is obtained by pre-encoding the video frame to be encoded, the second image motion information is obtained according to encoded data of a historical encoded video frame before the video frame to be encoded, the terminal determines a target encoding mode corresponding to the video frame to be encoded according to the image motion information, then encodes the video frame to be encoded according to the target encoding mode to obtain corresponding encoded data, and can send the encoded data obtained by encoding to the server 120. Or, the terminal 110 sends the video frame to be encoded to the server 120, after the server 120 obtains the video frame to be encoded, the server 120 obtains image motion information corresponding to the video frame to be encoded, determines a target encoding mode corresponding to the video frame to be encoded according to the image motion information, encodes the video frame to be encoded according to the target encoding mode to obtain corresponding encoded data, and finally sends the encoded data to the terminal 110.

In one embodiment, as shown in fig. 2, a video encoding method is provided. The embodiment is mainly illustrated by applying the method to the terminal 110 in fig. 1. Referring to fig. 2, the video encoding method specifically includes the following steps:

step 202, acquiring a video frame to be encoded, and acquiring image motion information corresponding to the video frame to be encoded, wherein the image motion information comprises at least one of first image motion information and second image motion information; the first image motion information is obtained by pre-coding a video frame to be coded; the second image motion information is derived from encoded data of historically encoded video frames prior to the video frame to be encoded.

The video frame to be encoded is a basic unit constituting a video to be encoded, and the video to be encoded is a frame sequence constituted by a plurality of video frames. The video to be coded refers to video which is not subjected to video coding, and the video coding refers to a mode of converting a file in a certain video format into a file in another video format through a specific compression technology. The video frame to be encoded may be a video frame acquired in real time for instant messaging, or a video frame corresponding to a stored video. The image motion information is information related to image motion when the video frame to be encoded performs image motion. Wherein the image motion includes, but is not limited to, horizontal motion in a first dimension, vertical motion in a second dimension, and the like. The image motion information includes, but is not limited to, at least one of first image motion information obtained by pre-encoding a video frame to be encoded, or second image motion information obtained from encoded data of a historical encoded video frame before the video to be encoded, and the like.

In order to reduce the encoding time of the video encoder and improve the encoding efficiency, the video frame to be encoded may be pre-encoded before the video frame to be encoded is subjected to video encoding, where pre-encoding refers to a process of pre-processing the video frame to be encoded before the video frame to be encoded is subjected to video encoding. The pre-coding includes, but is not limited to, dividing a video frame to be coded to obtain a plurality of pre-coded video blocks, and then further calculating to obtain first image motion information according to the pre-coded video blocks. The first image motion information may be, but is not limited to, statistical motion fluctuation information obtained by pre-encoding a video frame to be encoded, where the statistical motion fluctuation information is obtained by counting image motion information of each pre-encoded video block. The image motion information of each pre-coding video block may be a first-dimension motion displacement corresponding to each pre-coding video block in a first dimension, or a second-dimension motion displacement corresponding to each pre-coding video block in a second dimension, or obtained by comprehensive calculation according to the first-dimension motion displacement corresponding to each pre-coding video block in the first dimension and the second-dimension motion displacement corresponding to each pre-coding video block in the second dimension, and so on.

The historical encoded video frame is a frame sequence which is subjected to video encoding before the video frame to be encoded, and because the historical encoded video frame is obtained by encoding through the video encoder, encoded data related to the video encoding can be generated in the process of encoding the historical encoded video frame through the video encoder. Further, the second image motion information may be derived from the encoded data of the historical encoded video frames. The method for obtaining the motion information of the second image according to the encoded data of the historical encoded video frame may be customized, and the customization may be, but is not limited to, directly using the motion vector in the encoded data of the historical encoded video frame as the motion information of the second image, or obtaining the motion vector corresponding to each encoded video block of the historical encoded video frame first, then obtaining the motion information of the second image according to the motion vector corresponding to each encoded video block, and so on. The motion vector synthesis according to each coded video block includes, but is not limited to, synthesizing the motion vector of each video coding block in the first dimension and the motion vector of each video coding block in the second dimension.

And 204, determining a target coding mode corresponding to the video frame to be coded according to the image motion information.

The encoding mode is a mode of encoding a video frame to be encoded by a video encoder, and the encoding mode includes, but is not limited to, a first encoding mode, a second encoding mode, and the like. Wherein, different coding modes adopt different coding algorithms to code the video frame to be coded. Different coding algorithms mean that different coding parameters are set during coding, such as different motion search ranges, different quantization parameters, different rate distortion cost calculation modes, different code rate information, and the like.

Specifically, the target encoding mode corresponding to the video frame to be encoded may be determined according to the image motion information, and if the image motion information of the video frame to be encoded only includes corresponding image motion information in a single dimension, for example, the image motion information of the video frame to be encoded in the first dimension is used as the image motion information of the video frame to be encoded, or the image motion information of the video frame to be encoded in the second dimension is used as the image motion information of the video frame to be encoded. When the image motion information of the video frame to be encoded conforms to a first preset rule, the target encoding mode corresponding to the video frame to be encoded can be determined to be the first encoding mode, or when the image motion information of the video frame to be encoded conforms to a second preset rule, the target encoding mode corresponding to the video frame to be encoded can be determined to be the second encoding mode, and so on. The first preset rule and the second preset rule can be customized according to actual requirements.

In one embodiment, if the image motion information includes a plurality of image motion information, where each image motion information has corresponding image motion information in multiple dimensions, such as first image motion information and second image motion information, the first image motion information has corresponding image motion information in the first dimension and corresponding image motion information in the second dimension, and the second image motion information has corresponding image motion information in the first dimension and corresponding image motion information in the second dimension. When the image motion information corresponding to the first image motion information in the first dimension conforms to a first preset rule and the image motion information corresponding to the first image motion information in the second dimension conforms to a second preset rule, it may be determined that the target coding mode corresponding to the video frame to be coded is the first coding mode, and/or when the image motion information corresponding to the second image motion information in the first dimension conforms to a third preset rule and the image motion information corresponding to the second image motion information in the second dimension conforms to a fourth preset rule, it may be determined that the target coding mode corresponding to the video frame to be coded is the second coding mode, and so on. The first preset rule, the second preset rule, the third preset rule and the fourth preset rule can be defined according to actual requirements.

In an embodiment, if the image motion information includes multi-dimensional image motion information, such as first image motion information and second image motion information, when the image motion information corresponding to the first image motion information in the first dimension is smaller than a first preset threshold and the image motion information corresponding to the first image motion information in the second dimension is smaller than a second preset threshold, it is determined that the target encoding mode corresponding to the video frame to be encoded is the first encoding mode. Or when the image motion information corresponding to the second image motion information in the first dimension is greater than a third preset threshold and the image motion information corresponding to the second image motion information in the second dimension is greater than a fourth preset threshold, determining that the target coding mode corresponding to the video frame to be coded is the second coding mode, and the like.

And step 206, coding the video frame to be coded according to the target coding mode to obtain corresponding coded data.

The coded data is related data generated when a video coder codes a video frame to be coded. Specifically, after the target encoding mode corresponding to the video frame to be encoded is determined according to the image motion information, the corresponding video encoder may be switched to encode the video frame to be encoded according to the target encoding mode, so as to obtain the corresponding encoded data. Where the video encoder is used to encode a video frame to be encoded, the video encoder may be, but is not limited to, a physical video encoder, a software video encoder, and the like. Before determining the target encoding mode corresponding to the video frame to be encoded, the first encoder is already used to encode the historical encoded video frame of the video frame to be encoded, and the target encoding mode corresponding to the video frame to be encoded is the encoding mode of the second encoder, so that specifically, the first encoder is switched to the second encoder to encode the video frame to be encoded, and the specific switching from the first encoder to the second encoder may be, but is not limited to, the same video encoder changing encoding parameters to switch, and the like. Or before the target coding mode corresponding to the video frame to be coded is determined, the historical video frame to be coded corresponding to the video frame to be coded is not subjected to video coding, and after the target coding mode corresponding to the video frame to be coded is determined, the historical video frame to be coded and the video frame to be coded are coded together by using the target coding mode corresponding to the video frame to be coded, so that the corresponding coded data are obtained.

The video coding method comprises the steps of obtaining a video frame to be coded, obtaining image motion information corresponding to the video frame to be coded, wherein the image motion information comprises at least one of first image motion information and second image motion information, the first image motion information is obtained by pre-coding the video frame to be coded, the second image motion information is obtained according to coded data of historical coded video frames before the video frame to be coded, determining a target coding mode corresponding to the video frame to be coded according to the image motion information, and coding the video frame to be coded according to the target coding mode to obtain corresponding coded data. Because the image motion information corresponding to different video contents is different and the target coding modes corresponding to different video contents are also different, the suitable target coding mode can be adaptively selected for coding aiming at different video contents, thereby improving the coding efficiency.

In one embodiment, as shown in fig. 3, the generating of the first image motion information includes:

step 302, a video frame to be encoded is divided to obtain a plurality of pre-encoded video blocks.

The first image motion information is obtained by pre-coding a video frame to be coded, and the pre-coding is a process of pre-processing the video frame to be coded before the video frame to be coded is subjected to video coding. Specifically, the video frame to be encoded may be segmented according to a preset segmentation rule to obtain a plurality of pre-encoded video blocks. The preset segmentation rule may be customized, and the customization may be, but is not limited to, uniformly segmenting the video frame to be encoded according to the preset same proportion, for example, dividing the video frame to be encoded into 8 × 8 pre-encoded video blocks. Or the video frame to be coded is divided according to the coding rule of the coder, for example, a partial region of the video frame to be coded is divided into 8 × 8 pre-coded video blocks, and the other partial region is divided into 16 × 16 pre-coded video blocks.

Step 304, acquiring a forward video frame of the video frame to be encoded, and acquiring pre-encoded reference video blocks corresponding to the pre-encoded video blocks.

The forward video frame refers to a frame before a video frame to be encoded, and specifically, the forward video frame of the video frame to be encoded can be obtained according to a preset rule. The preset rule may be customized, and the customization may be, but is not limited to, obtaining a preset number of video frames from video frames before the video frame to be encoded as a forward video frame, and the like. And further, acquiring pre-coded reference video blocks corresponding to the preset coded video blocks. The pre-coding reference video block is a video block used for reference in a pre-coding process, wherein the pre-coding reference block corresponding to each pre-coding video block is obtained, and the pre-coding reference video block corresponding to each pre-coding video block can be obtained according to a pre-setting reference rule. The preset reference rule includes, but is not limited to, a reference relationship between a video frame to be encoded and a reference frame, and the reference relationship may specifically be, but is not limited to, a resolution size relationship. Specifically, a reference frame corresponding to the video frame to be coded is obtained according to a reference relation between the video frame to be coded and the reference frame in a preset reference rule, the obtained reference frame is divided according to a preset division rule which is the same as the video frame to be coded to obtain a plurality of reference video blocks, and a pre-coding reference video block corresponding to each pre-coding video block is obtained from the plurality of reference video blocks.

And step 306, calculating to obtain statistical motion fluctuation information corresponding to the video frame to be coded according to the positions of each pre-coding video block and the corresponding pre-coding reference video block, and taking the statistical motion fluctuation information as the first image motion information.

The motion fluctuation information is used for representing relevant data generated by image motion change, and the statistical motion fluctuation information refers to the sum of motion fluctuation information corresponding to each pre-coding video block. The statistical motion fluctuation information can be obtained by calculation according to the position relation between each pre-coding video block and the corresponding pre-coding reference video block. Specifically, but not limited to, the sum of the position differences between each pre-coding video block and the corresponding pre-coding reference video block is used as statistical motion fluctuation information, or the position differences between each pre-coding video block and the corresponding pre-coding reference video block are comprehensively calculated according to a preset proportion to obtain statistical motion fluctuation information, or the variance of the position differences between each pre-coding video block and the corresponding pre-coding reference video block is used as statistical motion fluctuation information, or the average of the position differences between each pre-coding video block and the corresponding pre-coding reference video block is used as statistical motion fluctuation information, and the like.

In an embodiment, if each pre-coding video block and the corresponding pre-coding reference video block perform image motion in the first dimension and the second dimension, the statistical motion fluctuation information corresponding to the video frame to be coded is calculated according to the positions of each pre-coding video block and the corresponding pre-coding reference video block, which may be, but not limited to, firstly calculating the statistical motion fluctuation information in the first dimension and the statistical motion fluctuation information in the second dimension of each pre-coding video block and the corresponding pre-coding reference video block, and then calculating the statistical motion fluctuation information corresponding to the video frame to be coded according to the statistical motion fluctuation information in the first dimension and the statistical motion fluctuation information in the second dimension of each pre-coding video block and the corresponding pre-coding reference video block. Further, after the statistical motion fluctuation information corresponding to the video frame to be encoded is obtained, the statistical motion fluctuation information is used as the first image motion information.

In one embodiment, as shown in fig. 4, the calculating the statistical motion fluctuation information corresponding to the video frame to be encoded according to the positions of each pre-encoded video block and the corresponding pre-encoded reference video block includes:

step 402, calculating according to the positions of each pre-coding video block and the corresponding pre-coding reference video block to obtain the first-dimension motion displacement corresponding to each pre-coding video block in the first dimension.

And 404, calculating to obtain first-dimension statistical motion fluctuation information corresponding to the video frame to be coded according to the first-dimension motion displacement corresponding to each pre-coding video block.

If each pre-coding video block and the corresponding pre-coding reference video block respectively perform image motion in a first dimension and a second dimension, calculating to obtain statistical motion fluctuation information corresponding to the video frame to be coded according to first-dimension motion fluctuation information of each preset coding video block and the corresponding pre-coding reference video block in the first dimension and second-dimension motion fluctuation information of the preset coding video block and the corresponding pre-coding reference video block in the second dimension. For the first-dimension statistical motion fluctuation information of each preset coding video block and the corresponding pre-coding reference video block in the first dimension, specifically, the first-dimension motion displacement corresponding to each pre-coding video block in the first dimension is obtained by calculating according to the position relationship between each pre-coding video block and the corresponding pre-coding reference video block, and the calculation of the first-dimension motion displacement may specifically be, but is not limited to, taking the relative offset of the positions of each pre-coding video block and the corresponding pre-coding reference video block as the first-dimension motion displacement. The first dimension motion displacement is used for representing the position change of each pre-coding video block and the corresponding pre-coding reference video block in the first dimension.

Further, after the first-dimension motion displacement corresponding to the first dimension of each pre-coding video block is obtained, the first-dimension statistical motion fluctuation information corresponding to the video frame to be coded is further calculated according to the first-dimension motion displacement corresponding to each pre-coding video block. The calculation of the first-dimension motion displacement may be, but is not limited to, calculating a variance of the first-dimension motion displacement corresponding to each pre-encoded video block to obtain first-dimension statistical motion fluctuation information corresponding to the video frame to be encoded. Or calculating an average value of the first-dimension motion displacement corresponding to each pre-coding video block to obtain first-dimension statistical motion fluctuation information corresponding to the video frame to be coded. Or performing weighted calculation on the first-dimension motion displacement corresponding to each pre-coding video block to obtain first-dimension statistical motion fluctuation information corresponding to the video frame to be coded, and the like.

And 406, calculating a second dimension motion displacement corresponding to each pre-coding video block in a second dimension according to the positions of each pre-coding video block and the corresponding pre-coding reference video block.

And step 408, calculating to obtain second dimension statistical motion fluctuation information corresponding to the video frame to be coded according to the second dimension motion displacement corresponding to each pre-coded video block.

And calculating to obtain second-dimension statistical motion fluctuation information corresponding to the video frame to be coded according to the positions of each preset coding video block and the corresponding pre-coding reference video block in the second dimension. Specifically, a second-dimension motion displacement corresponding to each pre-coding video block in a second dimension is calculated according to a position relationship between each pre-coding video block and the corresponding pre-coding reference video block, and the second-dimension motion displacement calculation may specifically be, but is not limited to, taking a relative offset between the positions of each pre-coding video block and the corresponding pre-coding reference video block as the second-dimension motion displacement. The second dimension motion displacement is used for representing the position change of each pre-coding video block and the corresponding pre-coding reference video block in the second dimension.

Further, after obtaining the second-dimension motion displacement corresponding to the second dimension of each pre-coding video block, further calculating according to the second-dimension motion displacement corresponding to each pre-coding video block to obtain second-dimension statistical motion fluctuation information corresponding to the video frame to be coded. The calculation of the second-dimension motion displacement may be, but is not limited to, calculating a variance of the second-dimension motion displacement corresponding to each pre-encoded video block to obtain second-dimension statistical motion fluctuation information corresponding to the video frame to be encoded. Or calculating the average value of the second dimension motion displacement corresponding to each pre-coding video block to obtain the second dimension statistical motion fluctuation information corresponding to the video frame to be coded. Or performing weighted calculation on the second dimension motion displacement corresponding to each pre-coding video block to obtain second dimension statistical motion fluctuation information corresponding to the video frame to be coded, and the like.

And step 410, obtaining statistical motion fluctuation information corresponding to the video frame to be coded according to the first dimension statistical motion fluctuation information and the second dimension statistical motion fluctuation information.

Specifically, after the first dimension statistical motion fluctuation information and the second dimension statistical motion fluctuation information are obtained, the statistical motion fluctuation information corresponding to the video frame to be encoded is obtained through calculation according to the first dimension statistical motion fluctuation information and the second dimension statistical motion fluctuation information. The statistical motion fluctuation information corresponding to the video frame to be encoded obtained by calculation according to the first-dimension statistical motion fluctuation information and the second-dimension statistical motion fluctuation information may be, but is not limited to, calculating an average value of the first-dimension statistical motion fluctuation information and the second-dimension statistical motion fluctuation information to obtain statistical motion fluctuation information corresponding to the video frame to be encoded. Or calculating the variance of the first dimension statistical motion fluctuation information and the second dimension statistical motion fluctuation information to obtain the statistical motion fluctuation information corresponding to the video frame to be coded. Or performing weighted calculation on the first dimension statistical motion fluctuation information and the second dimension statistical motion fluctuation information to obtain statistical motion fluctuation information corresponding to the video frame to be coded, and the like.

In one embodiment, as shown in fig. 5, the generating of the second image motion information includes:

step 502, obtaining historical encoded video frames before a video frame to be encoded.

Step 504, obtaining motion vectors corresponding to each coded video block in the historical coded video frame.

Step 506, calculating to obtain statistical motion fluctuation information corresponding to the historical encoded video frame according to the motion vector corresponding to each encoded video block, and using the statistical motion fluctuation information as second image motion information, wherein the statistical motion fluctuation information includes corresponding first-dimension statistical motion fluctuation information and second-dimension statistical motion fluctuation information.

The second image motion information is obtained according to the coded data of the historical coded video frame before the video frame to be coded, and the historical coded video frame refers to the frame sequence which is subjected to video coding before the video frame to be coded. Since the historical encoded video frames are encoded by the video encoder, encoded data related to video encoding is generated during the encoding of the historical encoded video frames by the video encoder, wherein the motion vector is one of the encoded data. The motion vector refers to dividing a historical encoded video frame into a plurality of blocks or macro blocks, seeking to search the position of each block or macro block in an adjacent frame image, and finding out the relative offset of the spatial position between the two blocks or macro blocks as the motion vector.

Specifically, historical encoded video frames before the video frame to be encoded are obtained, wherein the specific number of the historical encoded video frames before the video frame to be encoded is obtained can be customized, and then the historical encoded video frames are segmented according to preset segmentation rules to obtain each encoded video block. The preset segmentation rule may be customized, and the customization may be, but is not limited to, segmenting the historical encoded video frame according to the encoding rule of the video encoder. And after obtaining each coded video block corresponding to the historical coded video frame, obtaining a motion vector generated by each coded video block in the video coding process. The motion vector is specifically a motion vector corresponding to each encoded video block, which is obtained by searching the position of each encoded video block in the adjacent video frame image according to the reference rule of each encoded video block, and calculating a relative offset according to the position of each encoded video block in the historical encoded video frame and the position of each encoded video block in the adjacent video frame image.

Further, the statistical motion fluctuation information corresponding to the historical encoded video frame is obtained through calculation according to the motion vector corresponding to each encoded video block. The specific calculation may be, but is not limited to, taking the sum of the motion vectors corresponding to the encoded video blocks as statistical motion fluctuation information, or comprehensively calculating the motion vectors corresponding to the encoded video blocks according to a specific gravity to obtain statistical motion fluctuation information, or taking the variance of the motion vectors corresponding to the encoded video blocks as statistical motion fluctuation information, or taking the average of the motion vectors corresponding to the encoded video blocks as statistical motion fluctuation information, and so on.

In one embodiment, if each encoded video block performs image motion in a first dimension and a second dimension, the first-dimension statistical motion fluctuation information is calculated according to a first-dimension motion vector of each encoded video block in the first dimension, and the second-dimension statistical motion fluctuation information is calculated according to a second-dimension motion vector of each encoded video block in the second dimension. The specific calculation of the first-dimension statistical motion fluctuation information and the second-dimension statistical motion fluctuation information may be, but is not limited to, performing variance calculation on the first-dimension motion vector or the second-dimension motion vector to obtain corresponding first-dimension statistical motion fluctuation information or second-dimension statistical motion fluctuation information. Or averaging the first-dimension motion vector or the second-dimension motion vector to obtain corresponding first-dimension statistical motion fluctuation information or second-dimension statistical motion fluctuation information, and the like.

Further, after obtaining the first dimension statistical motion fluctuation information and the second dimension statistical motion fluctuation information of each coded video block, calculating to obtain the statistical motion fluctuation information corresponding to the historical coded video frame according to the first dimension statistical motion fluctuation information and the second dimension statistical motion fluctuation information. The statistical motion fluctuation information corresponding to the historical encoded video frames obtained by specifically calculating according to the first-dimension statistical motion fluctuation information and the second-dimension statistical motion fluctuation information may be, but is not limited to, the statistical motion fluctuation information corresponding to the historical encoded video frames obtained by calculating according to a preset proportion the first-dimension statistical motion fluctuation information and the second-dimension statistical motion fluctuation information, and the like.

In one embodiment, as shown in fig. 6, determining a target encoding mode corresponding to a video frame to be encoded according to image motion information includes:

step 602, when the motion state corresponding to the image motion information is consistent motion, determining that the target coding mode corresponding to the video frame to be coded is the first coding mode.

Step 604, otherwise, determining that the target coding mode corresponding to the video frame to be coded is the second coding mode.

The image motion information is information related to image motion generated when the video frame to be coded moves, and when the motion state corresponding to the image motion information is consistent motion, the motion directions of each frame of image of the video frame to be coded are relatively consistent. If each pixel point of the video frame to be coded is taken as an individual, when each pixel point moves towards a motion direction, the motion state corresponding to the image motion information of the video frame to be coded is considered to be consistent motion.

Specifically, when the motion state corresponding to the image motion information is consistent motion, it may be determined that the target encoding mode corresponding to the video frame to be encoded is a first encoding mode, where the first encoding mode may be an encoding mode corresponding to a first encoder, and the first encoder is more suitable for the video frame to be encoded with consistent motion. On the contrary, when the motion state corresponding to the image operation information is not consistent motion, it indicates that the image motion directions corresponding to the video frames to be encoded are relatively dispersed, and the first encoder is not suitable for the video frames to be encoded with the relatively dispersed motion directions, and the target encoding mode corresponding to the video frames to be encoded with the relatively dispersed image motion directions is the second encoding mode. Similarly, the second encoding mode may be an encoding mode corresponding to the second encoder, and the second encoder is more suitable for video frames to be encoded with more dispersed motion directions.

In one embodiment, determining a target encoding mode corresponding to a video frame to be encoded according to image motion information includes: and when the first dimension fluctuation value corresponding to the first dimension statistic motion fluctuation information is smaller than a first preset threshold value and the second dimension fluctuation value corresponding to the second dimension statistic motion fluctuation information is smaller than a second preset threshold value, switching the target coding mode corresponding to the video frame to be coded into the first coding mode.

The first dimension fluctuation value is used for representing the fluctuation of the first dimension statistical motion fluctuation information, and similarly, the second dimension fluctuation value is used for representing the fluctuation of the second dimension statistical motion fluctuation information. Specifically, when the first dimension fluctuation value corresponding to the first dimension statistical motion fluctuation information is smaller than a first preset threshold value and the second dimension fluctuation value corresponding to the second dimension statistical motion fluctuation information is smaller than a second preset threshold value, the target coding mode corresponding to the video frame to be coded can be switched to the first coding mode, which indicates that the first coding mode corresponding to the first encoder is more suitable for the video frame to be coded, and the coding efficiency of the video frame to be coded can be improved. If the first dimension fluctuation value corresponding to the first dimension statistical motion fluctuation information is a first dimension variance value, the first preset threshold is T1, the second dimension fluctuation value corresponding to the second dimension statistical motion fluctuation information is a second dimension variance value, and the second preset threshold is T2, that is, when the first dimension variance value corresponding to the first dimension is smaller than the first preset threshold T1 and the second dimension variance value corresponding to the second dimension is smaller than the second preset threshold T2, in order to improve the encoding efficiency of the video frame to be encoded, the video frame to be encoded is encoded by using the encoding method corresponding to the first encoder.

Otherwise, determining that the coding mode of the video frame to be coded is the second coding mode, and coding the video frame to be coded by using the coding mode corresponding to the second coder in order to ensure the coding efficiency of the video frame to be coded. Specifically, before the coding mode corresponding to the video frame to be coded is not determined, the historical coded video frame before the video frame to be coded is coded through the coding mode corresponding to the second coder, when the target coding mode of the video frame to be coded is determined to be the first coding mode, the second coder is switched to the first coder when the video frame to be coded is coded, and the video frame to be coded is coded through the first coding mode corresponding to the first coder, so that the coding efficiency of the video frame to be coded is improved. The first preset threshold and the second preset threshold can be set according to actual needs.

In one embodiment, determining a target encoding mode corresponding to a video frame to be encoded according to image motion information includes: and when the first dimension fluctuation value corresponding to the first dimension statistic motion fluctuation information is larger than a third preset threshold value and/or the second dimension fluctuation value corresponding to the second dimension statistic motion fluctuation information is larger than a fourth preset threshold value, switching the target coding mode corresponding to the video frame to be coded into a second coding mode.

The second coding method is more suitable for video frames to be coded which are scattered with the image motion direction, so that when the first dimension fluctuation value corresponding to the first dimension statistical motion fluctuation information is greater than the third preset threshold value, the target coding method corresponding to the video frames to be coded can be determined as the second coding method. Or when the second dimension fluctuation value corresponding to the second dimension statistical motion fluctuation information is greater than a fourth preset threshold, determining the target coding mode corresponding to the video frame to be coded as the second coding mode. Or when the first dimension fluctuation value corresponding to the first dimension statistical motion fluctuation information is greater than a third preset threshold and the second dimension fluctuation value corresponding to the second dimension statistical motion fluctuation information is greater than a fourth preset threshold, determining the target coding mode corresponding to the video frame to be coded as the second coding mode. The third preset threshold and the fourth preset threshold can be set according to actual needs.

In one embodiment, historical encoded video frames before a video frame to be encoded are encoded through an encoding mode corresponding to a first encoder, when it is determined that a target encoding mode corresponding to the video frame to be encoded is a second encoding mode according to image motion information, when the video frame to be encoded is encoded, in order to improve a video encoding rate of the video frame to be encoded, the first encoder is switched to a second encoder, and the video frame to be encoded is encoded through the second encoding mode corresponding to the second encoder, so that the encoding efficiency of the video frame to be encoded is improved.

In one embodiment, as shown in fig. 7, the video encoding method further includes:

step 702, obtaining a continuous historical forward video frame before a video frame to be encoded.

Step 704, obtaining image motion information corresponding to each continuous historical forward video frame.

Step 706, determining a target coding mode corresponding to the video frame to be coded according to the image motion information corresponding to each continuous historical forward video frame and the image motion information corresponding to the video frame to be coded.

The historical forward video frames are historical encoded video frames before the video frames to be encoded, and the specific number of the continuous historical forward video frames before the video frames to be encoded are acquired can be set according to actual requirements. Because the obtained historical forward video frames are all coded by the coder, the image motion information corresponding to each continuous historical forward video frame is obtained from the coded data corresponding to each continuous historical forward video frame according to the coded data corresponding to each continuous historical forward video frame. Image motion information includes, but is not limited to, motion vectors for respective consecutive historical forward video frames, and the like.

Further, the target coding mode corresponding to the video frame to be coded can be determined according to the fluctuation value between the image motion information corresponding to each continuous historical forward video frame and the image motion information corresponding to the video frame to be coded. The image motion information corresponding to each successive historical forward video frame may be, but is not limited to, a motion vector and the like corresponding to each successive historical forward video frame, and the image motion information corresponding to the video frame to be encoded may be, but is not limited to, obtained by searching through an image search algorithm.

In one embodiment, the motion search range of the first encoding mode is larger than the motion search range of the second encoding mode; and/or the sub-pixel interpolation algorithm in the first coding mode is in a closed state; and/or the complexity of the intra-coding mode in the first coding mode is higher than the complexity of the intra-coding mode in the second coding mode.

The first coding mode is more suitable for video frames to be coded with single image motion direction, and for application scenes with a large range, the reference blocks which are more similar can be quickly found by adopting the first coding mode for coding, and the prediction residual is small, so that the motion search range of the first coding mode is larger than that of the second coding mode.

In one embodiment, since the screen content is usually scrolled rapidly up and down or left and right in the application scenario of recording the screen content, the motion search range of the first encoding mode is larger than that of the second encoding mode. The first encoding mode may be an encoding mode corresponding to a screen video encoder, and the second encoding mode may be an encoding mode corresponding to a general encoder.

The sub-pixel interpolation is a process of obtaining reference data at a sub-pixel level by interpolating reference data of an integer pixel in a video frame to be coded, and the sub-pixel interpolation can also be called sub-pixel interpolation. Since the video frame to be encoded using the first encoding method moves according to integer pixels, the first encoder corresponding to the first encoding method usually turns off the sub-pixel interpolation algorithm. In the application scenario of the screen content, the screen coding content is directly acquired from the screen pixels, so that the screen content moves according to the whole pixels, the first encoder is a screen encoder, and the screen encoder usually turns off the sub-pixel interpolation algorithm.

The complexity of the intra-frame coding mode in the first coding mode is higher than that of the intra-frame coding mode in the second coding mode, and the complexity of the intra-frame coding mode means that the first encoder corresponding to the first coding mode has additional coding tools, such as a palette mode, an intra-frame block copy mode, and the like. The second encoder corresponding to the second encoding method does not have these additional encoding tools. Where the palette mode represents the color of each pixel in terms of color tables and indices, and the intra block copy mode is a block that copies the same frame of video texture or closely resembles it.

In this embodiment, by the difference between the first encoder corresponding to the first encoding mode and the second encoder corresponding to the second encoding mode, the encoding mode corresponding to the appropriate encoder is adopted for encoding according to the characteristics of the video frame to be encoded, so that the encoding efficiency of the video frame to be encoded can be improved, and the encoding time can be reduced.

In one embodiment, the video frame to be encoded is a screen recording video frame to be encoded.

The video frame to be encoded can be a video frame to be encoded recorded on a screen in a scene recorded on the screen, the video frame to be encoded recorded on the screen is directly acquired from screen pixels, and the screen recording means that all contents appearing on the screen are recorded. Specifically, the terminal collects pixels displayed on a screen into RGB data, converts the collected RGB data into a YUV color space, obtains converted data, and records the converted data as a screen into a video frame to be coded. The conversion of the acquired RGB data into the YUV color space can be obtained by a color conversion algorithm in the prior art, and will not be described herein.

In a specific embodiment, a video encoding method is provided, which specifically includes the following steps:

step 802, acquiring a video frame to be encoded, and acquiring image motion information corresponding to the video frame to be encoded, where the image motion information includes at least one of first image motion information and second image motion information, the first image motion information is obtained by pre-encoding the video frame to be encoded, and the second image motion information is obtained according to encoded data of a historical encoded video frame before the video frame to be encoded.

Step 802a, recording a video frame to be encoded for a screen.

Step 802b, the generating step of the first image motion information comprises:

and step 802ba, dividing the video frame to be coded to obtain a plurality of pre-coded video blocks.

Step 802bb, obtaining a forward video frame of the video frame to be encoded, and obtaining pre-encoded reference video blocks corresponding to the pre-encoded video blocks.

And step 802bc, calculating to obtain statistical motion fluctuation information corresponding to the video frame to be coded according to the positions of each pre-coding video block and the corresponding pre-coding reference video block, and taking the statistical motion fluctuation information as the first image motion information.

Step 802bca, calculating a first-dimension motion displacement corresponding to each pre-coding video block in the first dimension according to the position of each pre-coding video block and the corresponding pre-coding reference video block.

Step 802bcb, calculating to obtain first dimension statistical motion fluctuation information corresponding to the video frame to be encoded according to the first dimension motion displacement corresponding to each pre-encoded video block.

And step 802bcc, calculating the second dimension motion displacement of each pre-coding video block in the second dimension according to the position of each pre-coding video block and the corresponding pre-coding reference video block.

And step 802bcd, calculating to obtain second dimension statistical motion fluctuation information corresponding to the video frame to be coded according to the second dimension motion displacement corresponding to each pre-coding video block.

And step 802bce, obtaining statistical motion fluctuation information corresponding to the video frame to be encoded according to the first dimension statistical motion fluctuation information and the second dimension statistical motion fluctuation information.

Step 802c, the generating step of the second image motion information comprises:

step 802ca, obtaining historical encoded video frames before a video frame to be encoded.

Step 802cb obtains motion vectors corresponding to each encoded video block in the historical encoded video frame.

And step 802cc, calculating to obtain statistical motion fluctuation information corresponding to historical coded video frames according to motion vectors corresponding to the coded video blocks, and using the statistical motion fluctuation information as second image motion information, wherein the statistical motion fluctuation information comprises corresponding first-dimension statistical motion fluctuation information and second-dimension statistical motion fluctuation information.

And step 804, determining a target coding mode corresponding to the video frame to be coded according to the image motion information.

Step 804a, when the motion state corresponding to the image motion information is consistent motion, determining that a target coding mode corresponding to a video frame to be coded is a first coding mode;

step 804b, otherwise, determining that the target coding mode corresponding to the video frame to be coded is the second coding mode.

Step 804c, when the first dimension fluctuation value corresponding to the first dimension statistical motion fluctuation information is smaller than a first preset threshold value, and the second dimension fluctuation value corresponding to the second dimension statistical motion fluctuation information is smaller than a second preset threshold value, switching the target coding mode corresponding to the video frame to be coded into the first coding mode.

Step 804d, when the first dimension fluctuation value corresponding to the first dimension statistical motion fluctuation information is greater than a third preset threshold value, and/or the second dimension fluctuation value corresponding to the second dimension statistical motion fluctuation information is greater than a fourth preset threshold value, switching the target coding mode corresponding to the video frame to be coded into the second coding mode.

Step 804f, the motion search range of the first coding mode is larger than the motion search range of the second coding mode; and/or the sub-pixel interpolation algorithm in the first coding mode is in a closed state; and/or the complexity of the intra-coding mode in the first coding mode is higher than the complexity of the intra-coding mode in the second coding mode.

And 806, coding the video frame to be coded according to the target coding mode to obtain corresponding coded data.

Step 808, acquiring a continuous historical forward video frame before the video frame to be encoded.

Step 810, obtaining image motion information corresponding to each continuous historical forward video frame.

Step 812, determining a target coding mode corresponding to the video frame to be coded according to the image motion information corresponding to each continuous historical forward video frame and the image motion information corresponding to the video frame to be coded.

It should be understood that, although the steps in the above-described flowcharts are shown in order as indicated by the arrows, the steps are not necessarily performed in order as indicated by the arrows. The steps are not performed in a strict order unless explicitly stated herein, and may be performed in other orders. Moreover, at least a portion of the steps in the above-described flowcharts may include multiple sub-steps or multiple stages, which are not necessarily performed at the same time, but may be performed at different times, and the order of performing the sub-steps or the stages is not necessarily sequential, but may be performed alternately or alternately with other steps or at least a portion of the sub-steps or stages of other steps.

In a text content scene, as shown in fig. 7A, fig. 7A shows an application scene diagram of a video coding method in an embodiment, a terminal is displaying word text content, when a segment of video is suddenly inserted into the word text content, and for the suddenly inserted video, if a first encoder that still uses the word text content for coding encodes the suddenly inserted video, the coding efficiency is easily low. Therefore, the terminal takes the suddenly inserted video as the video frame to be coded, acquires the image motion information corresponding to the video frame to be coded, determines the target coding mode corresponding to the video frame to be coded according to the image motion information, and codes the video frame to be coded according to the target coding mode to obtain the corresponding coded data. Specifically, a first coding mode corresponding to a first coder corresponding to word text content is switched to a target coder corresponding to a target coding mode, and a video frame to be coded is coded through the target coder. By applying the video coding method to the application scene of the text content, different video contents are suddenly inserted into the text content, a proper target coding mode can be adaptively selected for coding, and the coder is switched, so that the coding efficiency is improved.

As shown in fig. 8, in one embodiment, there is provided a video encoding apparatus 900, the apparatus comprising:

the to-be-coded video frame acquiring module 902 is configured to acquire a to-be-coded video frame and acquire image motion information corresponding to the to-be-coded video frame, where the image motion information includes at least one of first image motion information and second image motion information, the first image motion information is obtained by pre-coding the to-be-coded video frame, and the second image motion information is obtained according to coded data of a historical coded video frame before the to-be-coded video frame.

And a target coding mode obtaining module 904, configured to determine a target coding mode corresponding to the video frame to be coded according to the image motion information.

And a to-be-encoded video frame encoding module 906, configured to encode the to-be-encoded video frame according to the target encoding mode to obtain corresponding encoded data.

In one embodiment, as shown in fig. 9, the video encoding apparatus 900 further includes:

a to-be-encoded video frame dividing module 1002, configured to divide a to-be-encoded video frame to obtain a plurality of pre-encoded video blocks.

A forward video frame obtaining module 1004, configured to obtain a forward video frame of a video frame to be encoded, and obtain pre-encoded reference video blocks corresponding to each pre-encoded video block.

And the statistical motion fluctuation information calculation module 1006 is configured to calculate statistical motion fluctuation information corresponding to the video frame to be encoded according to the positions of each pre-encoded video block and the corresponding pre-encoded reference video block, and use the statistical motion fluctuation information as the first image motion information.

In one embodiment, as shown in fig. 10, the statistical motion fluctuation information calculation module 1006 includes:

and a first-dimension motion displacement calculation unit 1006a, configured to calculate, according to the positions of each pre-coding video block and the corresponding pre-coding reference video block, a first-dimension motion displacement corresponding to the first dimension of each pre-coding video block.

And the first dimension statistical motion fluctuation information calculation unit 1006b is configured to calculate, according to the first dimension motion displacement corresponding to each pre-encoded video block, to obtain first dimension statistical motion fluctuation information corresponding to the video frame to be encoded.

And a second dimension motion displacement calculation unit 1006c, configured to calculate, according to the positions of each pre-coding video block and the corresponding pre-coding reference video block, a second dimension motion displacement corresponding to the second dimension of each pre-coding video block.

And a second dimension statistical motion fluctuation information calculation unit 1006d, configured to calculate, according to the second dimension motion displacement corresponding to each pre-encoded video block, second dimension statistical motion fluctuation information corresponding to the video frame to be encoded.

And a statistical motion fluctuation information generation unit 1006e, configured to obtain statistical motion fluctuation information corresponding to the video frame to be encoded according to the first-dimension statistical motion fluctuation information and the second-dimension statistical motion fluctuation information.

In one embodiment, as shown in fig. 11, the video encoding apparatus 900 further includes:

a historical encoded video frame obtaining module 1102, configured to obtain a historical encoded video frame before a video frame to be encoded.

And a motion vector obtaining module 1104, configured to obtain a motion vector corresponding to each coded video block in the historical coded video frame.

A second image motion information generating module 1106, configured to calculate, according to the motion vector corresponding to each encoded video block, statistical motion fluctuation information corresponding to a historical encoded video frame, and use the statistical motion fluctuation information as second image motion information, where the statistical motion fluctuation information includes corresponding first-dimension statistical motion fluctuation information and second-dimension statistical motion fluctuation information.

In one embodiment, the target encoding mode obtaining module 904 is further configured to determine that a target encoding mode corresponding to a video frame to be encoded is a first encoding mode when a motion state corresponding to the image motion information is consistent motion; otherwise, determining the target coding mode corresponding to the video frame to be coded as the second coding mode.

In an embodiment, the target encoding mode obtaining module 904 is further configured to switch the target encoding mode corresponding to the video frame to be encoded to the first encoding mode when the first dimension fluctuation value corresponding to the first dimension statistical motion fluctuation information is smaller than a first preset threshold and the second dimension fluctuation value corresponding to the second dimension statistical motion fluctuation information is smaller than a second preset threshold.

In an embodiment, the target encoding mode obtaining module 904 is further configured to switch the target encoding mode corresponding to the video frame to be encoded to the second encoding mode when the first dimension fluctuation value corresponding to the first dimension statistical motion fluctuation information is greater than a third preset threshold and/or the second dimension fluctuation value corresponding to the second dimension statistical motion fluctuation information is greater than a fourth preset threshold.

In one embodiment, as shown in fig. 12, the video encoding apparatus 900 further includes:

a historical forward video frame obtaining module 1202, configured to obtain continuous historical forward video frames before a video frame to be encoded.

An image motion information obtaining module 1204, configured to obtain image motion information corresponding to each consecutive historical forward video frame.

The target coding mode determining module 1206 is configured to determine a target coding mode corresponding to the video frame to be coded according to the image motion information corresponding to each consecutive historical forward video frame and the image motion information corresponding to the video frame to be coded.

In one embodiment, the video frame to be encoded is recorded for the screen.

FIG. 13 is a diagram illustrating an internal structure of a computer device in one embodiment. The computer device may specifically be the terminal 110 in fig. 1. As shown in fig. 13, the computer apparatus includes a processor, a memory, a network interface, an input device, and a display screen, which are connected through a system bus. Wherein the memory includes a non-volatile storage medium and an internal memory. The non-volatile storage medium of the computer device stores an operating system and may also store a computer program that, when executed by the processor, causes the processor to implement the video encoding method. The internal memory may also have stored therein a computer program that, when executed by the processor, causes the processor to perform a video encoding method. The display screen of the computer equipment can be a liquid crystal display screen or an electronic ink display screen, and the input device of the computer equipment can be a touch layer covered on the display screen, a key, a track ball or a touch pad arranged on the shell of the computer equipment, an external keyboard, a touch pad or a mouse and the like.

Those skilled in the art will appreciate that the architecture shown in fig. 13 is merely a block diagram of some of the structures associated with the disclosed aspects and is not intended to limit the computing devices to which the disclosed aspects apply, as particular computing devices may include more or less components than those shown, or may combine certain components, or have a different arrangement of components.

In one embodiment, the video encoding apparatus provided in the present application may be implemented in the form of a computer program that is executable on a computer device such as the one shown in fig. 13. The memory of the computer device may store various program modules constituting the video encoding apparatus, such as a video frame to be encoded acquiring module, a target encoding mode acquiring module, and a video frame to be encoded encoding module shown in fig. 8. The computer program constituted by the respective program modules causes the processor to execute the steps in the video encoding method of the respective embodiments of the present application described in the present specification.

For example, the computer device shown in fig. 13 may perform, by using a to-be-encoded video frame acquisition module in the video encoding apparatus shown in fig. 8, acquiring a to-be-encoded video frame, and acquiring image motion information corresponding to the to-be-encoded video frame, where the image motion information includes at least one of first image motion information and second image motion information, the first image motion information is obtained by pre-encoding the to-be-encoded video frame, and the second image motion information is obtained according to encoded data of a history encoded video frame before the to-be-encoded video frame. The computer equipment can execute the target coding mode corresponding to the video frame to be coded according to the image motion information through the target coding mode acquisition module. The computer equipment can execute the coding of the video frame to be coded according to the target coding mode through the video frame to be coded coding module to obtain the corresponding coded data.

In one embodiment, a computer device is proposed, the computer device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, the processor implementing the following steps when executing the computer program: acquiring a video frame to be coded, and acquiring image motion information corresponding to the video frame to be coded, wherein the image motion information comprises at least one of first image motion information and second image motion information; the first image motion information is obtained by pre-coding a video frame to be coded; the second image motion information is obtained according to the coded data of the historical coded video frame before the video frame to be coded; determining a target coding mode corresponding to a video frame to be coded according to the image motion information; and coding the video frame to be coded according to the target coding mode to obtain corresponding coded data.

In one embodiment, the generating of the first image motion information comprises: dividing a video frame to be coded to obtain a plurality of pre-coded video blocks; acquiring a forward video frame of a video frame to be coded, and acquiring pre-coding reference video blocks corresponding to the pre-coding video blocks; and calculating to obtain statistical motion fluctuation information corresponding to the video frame to be coded according to the positions of each pre-coding video block and the corresponding pre-coding reference video block, and taking the statistical motion fluctuation information as the first image motion information.

In one embodiment, the calculating the statistical motion fluctuation information corresponding to the video frame to be encoded according to the positions of each pre-encoded video block and the corresponding pre-encoded reference video block includes: calculating according to the positions of each pre-coding video block and the corresponding pre-coding reference video block to obtain the first-dimension motion displacement corresponding to each pre-coding video block in the first dimension; calculating to obtain first dimension statistical motion fluctuation information corresponding to a video frame to be coded according to the first dimension motion displacement corresponding to each pre-coded video block; calculating according to the positions of each pre-coding video block and the corresponding pre-coding reference video block to obtain second dimension motion displacement corresponding to each pre-coding video block in a second dimension; calculating to obtain second dimension statistical motion fluctuation information corresponding to the video frame to be coded according to the second dimension motion displacement corresponding to each pre-coded video block; and according to the first dimension statistical motion fluctuation information and the second dimension statistical motion fluctuation information, obtaining statistical motion fluctuation information corresponding to the video frame to be coded.

In one embodiment, the generating of the second image motion information comprises: acquiring historical coded video frames before a video frame to be coded; obtaining a motion vector corresponding to each coded video block in a historical coded video frame; and calculating to obtain statistical motion fluctuation information corresponding to historical coded video frames according to the motion vectors corresponding to the coded video blocks, and taking the statistical motion fluctuation information as second image motion information, wherein the statistical motion fluctuation information comprises corresponding first-dimension statistical motion fluctuation information and second-dimension statistical motion fluctuation information.

In one embodiment, determining a target encoding mode corresponding to a video frame to be encoded according to image motion information includes: when the motion state corresponding to the image motion information is consistent motion, determining that a target coding mode corresponding to a video frame to be coded is a first coding mode; otherwise, determining the target coding mode corresponding to the video frame to be coded as the second coding mode.

In one embodiment, the computer program further causes the processor to perform the steps of: acquiring continuous historical forward video frames before a video frame to be coded; acquiring image motion information corresponding to each continuous historical forward video frame; and determining a target coding mode corresponding to the video frame to be coded according to the image motion information corresponding to each continuous historical forward video frame and the image motion information corresponding to the video frame to be coded.

In one embodiment, a computer-readable storage medium is provided, having stored thereon a computer program which, when executed by a processor, causes the processor to perform the steps of: acquiring a video frame to be coded, and acquiring image motion information corresponding to the video frame to be coded, wherein the image motion information comprises at least one of first image motion information and second image motion information; the first image motion information is obtained by pre-coding a video frame to be coded; the second image motion information is obtained according to the coded data of the historical coded video frame before the video frame to be coded; determining a target coding mode corresponding to a video frame to be coded according to the image motion information; and coding the video frame to be coded according to the target coding mode to obtain corresponding coded data.

In one embodiment, the calculating the statistical motion fluctuation information corresponding to the video frame to be encoded according to the positions of each pre-encoded video block and the corresponding pre-encoded reference video block includes: calculating to obtain first-dimension motion displacement corresponding to each pre-coding video block in the first dimension according to the positions of each pre-coding video block and the corresponding pre-coding reference video block; calculating to obtain first dimension statistical motion fluctuation information corresponding to a video frame to be coded according to the first dimension motion displacement corresponding to each pre-coded video block; calculating to obtain second dimension motion displacement corresponding to each pre-coding video block in a second dimension according to the positions of each pre-coding video block and the corresponding pre-coding reference video block; calculating to obtain second dimension statistical motion fluctuation information corresponding to the video frame to be coded according to the second dimension motion displacement corresponding to each pre-coded video block; and according to the first dimension statistical motion fluctuation information and the second dimension statistical motion fluctuation information, obtaining statistical motion fluctuation information corresponding to the video frame to be coded.

In one embodiment, the generating of the second image motion information comprises: acquiring historical encoded video frames before a video frame to be encoded; obtaining motion vectors corresponding to all coded video blocks in historical coded video frames; and calculating to obtain statistical motion fluctuation information corresponding to historical coded video frames according to the motion vectors corresponding to the coded video blocks, and taking the statistical motion fluctuation information as second image motion information, wherein the statistical motion fluctuation information comprises corresponding first-dimension statistical motion fluctuation information and second-dimension statistical motion fluctuation information.

In one embodiment, determining a target encoding mode corresponding to a video frame to be encoded according to image motion information includes: when the motion state corresponding to the image motion information is consistent motion, determining that a target coding mode corresponding to a video frame to be coded is a first coding mode; otherwise, determining that the target coding mode corresponding to the video frame to be coded is the second coding mode.

It will be understood by those skilled in the art that all or part of the processes of the methods of the embodiments described above can be implemented by a computer program, which can be stored in a non-volatile computer-readable storage medium, and can include the processes of the embodiments of the methods described above when the program is executed. Any reference to memory, storage, database, or other medium used in the embodiments provided herein may include non-volatile and/or volatile memory, among others. Non-volatile memory can include read-only memory (ROM), Programmable ROM (PROM), Electrically Programmable ROM (EPROM), Electrically Erasable Programmable ROM (EEPROM), or flash memory. Volatile memory can include Random Access Memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in a variety of forms, such as Static RAM (SRAM), Dynamic RAM (DRAM), Synchronous DRAM (SDRAM), Double Data Rate SDRAM (DDRSDRAM), Enhanced SDRAM (ESDRAM), Synchronous Link DRAM (SLDRAM), Rambus Direct RAM (RDRAM), direct bus dynamic RAM (DRDRAM), and bus dynamic RAM (RDRAM).

All possible combinations of the technical features in the above embodiments may not be described for the sake of brevity, but should be considered as being within the scope of the present disclosure as long as there is no contradiction between the combinations of the technical features.

The above-mentioned embodiments only express several embodiments of the present application, and the description thereof is more specific and detailed, but not construed as limiting the scope of the present application. It should be noted that, for a person skilled in the art, several variations and modifications can be made without departing from the concept of the present application, which falls within the scope of protection of the present application. Therefore, the protection scope of the present patent should be subject to the appended claims.

Claims

1. A video encoding method, comprising:

the first image motion information is obtained by pre-coding the video frame to be coded;

determining a target coding mode corresponding to the video frame to be coded according to the image motion information, wherein the target coding mode comprises the following steps: when the motion state corresponding to the image motion information is consistent motion, determining that a target coding mode corresponding to the video frame to be coded is a first coding mode; otherwise, determining that the target coding mode corresponding to the video frame to be coded is a second coding mode; the complexity of an intra-frame coding mode in the first coding mode is higher than that of the intra-frame coding mode in the second coding mode; the complexity of the intra-frame coding mode in the first coding mode is higher than that of the intra-frame coding mode in the second coding mode, which means that the first encoder corresponding to the first coding mode has an additional coding tool, and the additional coding tool includes at least one of a palette mode or an intra-frame block copy mode; the motion state corresponding to the image motion information is consistent motion, which means that all pixel points in the video frame to be coded move towards the same motion direction;

2. The method of claim 1, wherein the generating of the first image motion information comprises:

dividing the video frame to be coded to obtain a plurality of pre-coded video blocks;

acquiring a forward video frame of the video frame to be coded, and acquiring pre-coding reference video blocks corresponding to the pre-coding video blocks;

and calculating to obtain statistical motion fluctuation information corresponding to the video frame to be coded according to the positions of the pre-coding video blocks and the corresponding pre-coding reference video blocks, and taking the statistical motion fluctuation information as the first image motion information.

3. The method of claim 2, wherein the calculating the statistical motion fluctuation information corresponding to the video frame to be encoded according to the locations of each of the pre-encoded video blocks and the corresponding pre-encoded reference video block comprises:

calculating to obtain a first-dimension motion displacement corresponding to each pre-coding video block in a first dimension according to the positions of each pre-coding video block and the corresponding pre-coding reference video block;

calculating to obtain first dimension statistical motion fluctuation information corresponding to the video frame to be coded according to the first dimension motion displacement corresponding to each pre-coded video block;

calculating to obtain second dimension motion displacement corresponding to each pre-coding video block in a second dimension according to the positions of each pre-coding video block and the corresponding pre-coding reference video block;

calculating to obtain second dimension statistical motion fluctuation information corresponding to the video frame to be coded according to the second dimension motion displacement corresponding to each pre-coded video block;

and obtaining the statistical motion fluctuation information corresponding to the video frame to be coded according to the first dimension statistical motion fluctuation information and the second dimension statistical motion fluctuation information.

4. The method of claim 1, wherein the generating of the second image motion information comprises:

acquiring historical coded video frames before the video frames to be coded;

obtaining a motion vector corresponding to each coded video block in the historical coded video frame;

and calculating to obtain statistical motion fluctuation information corresponding to historical coded video frames according to the motion vectors corresponding to the coded video blocks, and using the statistical motion fluctuation information as the second image motion information, wherein the statistical motion fluctuation information comprises corresponding first-dimension statistical motion fluctuation information and second-dimension statistical motion fluctuation information.

5. The method according to claim 4, wherein the statistical motion fluctuation information is calculated by any one of the following methods:

taking the sum of the motion vectors corresponding to the coded video blocks as the statistical motion fluctuation information;

comprehensively calculating the motion vector corresponding to each coded video block according to specific gravity to obtain the statistical motion fluctuation information;

taking the variance of the motion vector corresponding to each coded video block as the statistical motion fluctuation information;

and taking the average value of the motion vectors corresponding to the coded video blocks as the statistical motion fluctuation information.

6. The method according to claim 3 or 4, wherein the determining the target encoding mode corresponding to the video frame to be encoded according to the image motion information comprises:

when a first dimension fluctuation value corresponding to the first dimension statistic motion fluctuation information is smaller than a first preset threshold value and a second dimension fluctuation value corresponding to the second dimension statistic motion fluctuation information is smaller than a second preset threshold value, switching a target coding mode corresponding to the video frame to be coded into a first coding mode;

the complexity of an intra-frame coding mode in the first coding mode is higher than that of an intra-frame coding mode in the second coding mode; the complexity of the intra-frame coding mode in the first coding mode is higher than that of the intra-frame coding mode in the second coding mode, which means that the first encoder corresponding to the first coding mode has an additional coding tool, and the additional coding tool includes at least one of a palette mode or an intra-frame block copy mode.

7. The method according to claim 3 or 4, wherein the determining a target encoding mode corresponding to the video frame to be encoded according to the image motion information comprises:

when the first dimension fluctuation value corresponding to the first dimension statistical motion fluctuation information is larger than a third preset threshold value, and/or

When a second dimension fluctuation value corresponding to the second dimension statistical motion fluctuation information is larger than a fourth preset threshold value, switching a target coding mode corresponding to the video frame to be coded into a second coding mode;

the complexity of the intra-frame coding mode in the second coding mode is lower than that of the intra-frame coding mode in the first coding mode; the complexity of the intra-frame coding mode in the second coding mode is lower than that of the intra-frame coding mode in the first coding mode, which means that the first encoder corresponding to the first coding mode has an additional coding tool, and the additional coding tool includes at least one of a palette mode or an intra-frame block copy mode.

8. The method of claim 1, further comprising:

acquiring continuous historical forward video frames before the video frame to be coded;

acquiring image motion information corresponding to each continuous historical forward video frame;

and determining a target coding mode corresponding to the video frame to be coded according to the image motion information corresponding to each continuous historical forward video frame and the image motion information corresponding to the video frame to be coded.

9. The method according to claim 1, wherein the motion search range of the first encoding mode is larger than the motion search range of the second encoding mode; and/or

And the sub-pixel interpolation algorithm in the first coding mode is in an off state.

10. The method of claim 1, wherein the video frame to be encoded is a screen recorded video frame to be encoded.

11. A video encoding apparatus, characterized in that the apparatus comprises:

a target coding mode obtaining module, configured to determine a target coding mode corresponding to the video frame to be coded according to the image motion information, where the target coding mode obtaining module includes: when the motion state corresponding to the image motion information is consistent motion, determining that a target coding mode corresponding to the video frame to be coded is a first coding mode; otherwise, determining that the target coding mode corresponding to the video frame to be coded is a second coding mode; the complexity of an intra-frame coding mode in the first coding mode is higher than that of the intra-frame coding mode in the second coding mode; the complexity of the intra-frame coding mode in the first coding mode is higher than that of the intra-frame coding mode in the second coding mode, which means that the first encoder corresponding to the first coding mode has additional coding tools, and the additional coding tools include at least one of a palette mode or an intra-frame block copy mode; the motion state corresponding to the image motion information is consistent motion, which means that all pixel points in the video frame to be coded move towards the same motion direction;

12. The apparatus of claim 11, further comprising:

the video frame to be coded is divided by the dividing module and is used for dividing the video frame to be coded to obtain a plurality of pre-coded video blocks;

a forward video frame obtaining module, configured to obtain a forward video frame of the video frame to be encoded, and obtain pre-encoded reference video blocks corresponding to the pre-encoded video blocks;

and the statistical motion fluctuation information calculation module is used for calculating statistical motion fluctuation information corresponding to the video frame to be coded according to the positions of the pre-coding video blocks and the corresponding pre-coding reference video blocks, and using the statistical motion fluctuation information as the first image motion information.

13. The apparatus of claim 12, wherein the statistical motion fluctuation information calculation module comprises:

the first dimension motion displacement calculation unit is used for calculating and obtaining the first dimension motion displacement corresponding to each pre-coding video block in the first dimension according to the positions of each pre-coding video block and the corresponding pre-coding reference video block;

a first dimension statistical motion fluctuation information calculation unit, configured to calculate, according to the first dimension motion displacement corresponding to each pre-encoded video block, first dimension statistical motion fluctuation information corresponding to the video frame to be encoded;

the second dimension motion displacement calculation unit is used for calculating and obtaining second dimension motion displacement corresponding to each pre-coding video block in a second dimension according to the positions of each pre-coding video block and the corresponding pre-coding reference video block;

a second dimension statistical motion fluctuation information calculation unit, configured to calculate second dimension statistical motion fluctuation information corresponding to the video frame to be encoded according to the second dimension motion displacement corresponding to each pre-encoded video block;

and the statistical motion fluctuation information generation unit is used for obtaining the statistical motion fluctuation information corresponding to the video frame to be coded according to the first-dimension statistical motion fluctuation information and the second-dimension statistical motion fluctuation information.

14. The apparatus of claim 11, further comprising:

a historical encoded video frame acquisition module, configured to acquire a historical encoded video frame before the video frame to be encoded;

a motion vector obtaining module, configured to obtain a motion vector corresponding to each encoded video block in the historical encoded video frame;

and the second image motion information generation module is used for calculating and obtaining statistical motion fluctuation information corresponding to historical coded video frames according to the motion vectors corresponding to the coded video blocks, and using the statistical motion fluctuation information as the second image motion information, wherein the statistical motion fluctuation information comprises corresponding first-dimension statistical motion fluctuation information and second-dimension statistical motion fluctuation information.

15. The apparatus according to claim 14, wherein the statistical motion fluctuation information is calculated by any one of the following methods:

16. The apparatus according to claim 13 or 14, wherein the target encoding manner obtaining module is further configured to switch the target encoding manner corresponding to the video frame to be encoded to the first encoding manner when a first dimension fluctuation value corresponding to the first dimension statistical motion fluctuation information is smaller than a first preset threshold and a second dimension fluctuation value corresponding to the second dimension statistical motion fluctuation information is smaller than a second preset threshold; the complexity of an intra-frame coding mode in the first coding mode is higher than that of an intra-frame coding mode in the second coding mode; the complexity of the intra-frame coding mode in the first coding mode is higher than that of the intra-frame coding mode in the second coding mode, which means that the first encoder corresponding to the first coding mode has an additional coding tool, and the additional coding tool includes at least one of a palette mode or an intra-frame block copy mode.

17. The apparatus according to claim 13 or 14, wherein the target encoding mode obtaining module is further configured to switch the target encoding mode corresponding to the video frame to be encoded to the second encoding mode when a first dimension fluctuation value corresponding to the first dimension statistical motion fluctuation information is greater than a third preset threshold and/or a second dimension fluctuation value corresponding to the second dimension statistical motion fluctuation information is greater than a fourth preset threshold; the complexity of the intra-frame coding mode in the second coding mode is lower than that of the intra-frame coding mode in the first coding mode; the complexity of the intra-frame coding mode in the second coding mode is lower than that of the intra-frame coding mode in the first coding mode, which means that the first encoder corresponding to the first coding mode has additional coding tools, and the additional coding tools include at least one of a palette mode or an intra-frame block copy mode.

18. The apparatus of claim 11, further comprising:

a history forward video frame obtaining module, configured to obtain continuous history forward video frames before the video frame to be encoded;

the image motion information acquisition module is used for acquiring image motion information corresponding to each continuous historical forward video frame;

and the target coding mode determining module is used for determining the target coding mode corresponding to the video frame to be coded according to the image motion information corresponding to each continuous historical forward video frame and the image motion information corresponding to the video frame to be coded.

19. The apparatus according to claim 11, wherein the motion search range of the first encoding scheme is larger than the motion search range of the second encoding scheme; and/or

20. The apparatus of claim 11, wherein the video frame to be encoded is a screen recorded video frame to be encoded.

21. A computer-readable storage medium, storing a computer program which, when executed by a processor, causes the processor to carry out the steps of the method according to any one of claims 1 to 10.

22. A computer device comprising a memory and a processor, the memory storing a computer program that, when executed by the processor, causes the processor to perform the steps of the method according to any one of claims 1 to 10.