CN117459732A

CN117459732A - Video encoding method, apparatus, device, readable storage medium, and program product

Info

Publication number: CN117459732A
Application number: CN202311390711.XA
Authority: CN
Inventors: 宁沛荣; 曲建峰
Original assignee: Shuhang Technology Beijing Co ltd
Current assignee: Shuhang Technology Beijing Co ltd
Priority date: 2023-10-25
Filing date: 2023-10-25
Publication date: 2024-01-26
Anticipated expiration: 2043-10-25
Also published as: CN117459732B

Abstract

Provided are a video encoding method, apparatus, device, readable storage medium, and program product, the method comprising: acquiring video characteristic data of a video to be processed and a video coding parameter sequence comprising a plurality of video coding parameters; determining a first video coding parameter and a second video coding parameter from a plurality of video coding parameters, wherein the prediction quality relation data corresponding to the first video coding parameter and the second video coding parameter indicates a numerical relation between a prediction coding quality value and a reference coding quality value of a target coding quality index, and the prediction coding quality value is used for indicating the coding quality of a video to be processed based on the video coding parameters by a video coder; and carrying out coding processing on the video to be processed according to the target video coding parameters by utilizing a video coder to obtain a video coding result. The method provided by the application can enable the video encoder to match the coding quality value of the video with the reference coding quality value, so that the video coding quality reaches the preset requirement.

Description

Video encoding method, apparatus, device, readable storage medium, and program product

Technical Field

The present application relates to the field of computer technology, and in particular, to a video encoding method, a video encoding apparatus, a computer device, a computer readable storage medium, and a computer program product.

Background

With the continuous development of video coding technology, the technical standards of video coding are increasing, for example: h.264 standard, h.265 standard, AV1, etc. The quality evaluation index of the video coding result is also various.

In video encoding processes, it is often desirable that the encoding quality of the video encoder meet target requirements (e.g., the encoding quality of the video matches the encoding quality of the reference). However, video encoders based on different coding standards have different video coding capabilities, and thus, how to match the coding quality of the video encoder with the target quality is a problem to be solved at present.

Disclosure of Invention

The embodiment of the application provides a video coding method, a device, equipment, a readable storage medium and a program product, which can enable a video coder to match a coding quality value of a video with a reference coding quality value, thereby enabling the video coding quality to reach a preset requirement and saving code rate.

In one aspect, an embodiment of the present application provides a video encoding method, including:

Acquiring video characteristic data of a video to be processed and acquiring a video coding parameter sequence, wherein the video coding parameter sequence comprises a plurality of video coding parameters;

determining a first video coding parameter and a second video coding parameter from the plurality of video coding parameters, wherein the arrangement positions of the first video coding parameter and the second video coding parameter in the video coding parameter sequence are adjacent, and the prediction quality relation data corresponding to the first video coding parameter and the second video coding parameter are different; the predicted quality relation data is obtained by processing the video characteristic data and the video coding parameters; the prediction quality relation data is used for indicating a numerical relation between a prediction coding quality value and a reference coding quality value, the prediction coding quality value is used for indicating the coding quality of the video to be processed by a video encoder based on the video coding parameter, and the reference coding quality value is a coding quality value corresponding to a target coding quality index;

determining a target video coding parameter from the first video coding parameter and the second video coding parameter;

and carrying out coding processing on the video to be processed by utilizing the video coder according to the target video coding parameters to obtain a video coding result, wherein the coding quality value of the video coding result is matched with the reference coding quality value corresponding to the target coding quality index.

In one aspect, an embodiment of the present application provides a video encoding apparatus, including:

the device comprises an acquisition unit, a processing unit and a processing unit, wherein the acquisition unit is used for acquiring video characteristic data of a video to be processed and acquiring a video coding parameter sequence, and the video coding parameter sequence comprises a plurality of video coding parameters;

a determining unit, configured to determine a first video coding parameter and a second video coding parameter from the plurality of video coding parameters, where the arrangement positions of the first video coding parameter and the second video coding parameter in the video coding parameter sequence are adjacent, and prediction quality relationship data corresponding to the first video coding parameter and the second video coding parameter are different; the predicted quality relation data is obtained by processing the video characteristic data and the video coding parameters; the prediction quality relation data is used for indicating a numerical relation between a prediction coding quality value and a reference coding quality value, the prediction coding quality value is used for indicating the coding quality of the video to be processed by a video encoder based on the video coding parameter, and the reference coding quality value is a coding quality value corresponding to a target coding quality index;

The determining unit is further configured to determine a target video coding parameter from the first video coding parameter and the second video coding parameter;

and the processing unit is used for carrying out coding processing on the video to be processed according to the target video coding parameters by utilizing the video coder to obtain a video coding result, and the coding quality value of the video coding result is matched with the reference coding quality value corresponding to the target coding quality index.

In one aspect, embodiments of the present application provide a computer device, including: the video coding method comprises a processor, a communication interface and a memory, wherein the processor, the communication interface and the memory are connected with each other, executable program codes are stored in the memory, and the processor is used for calling the executable program codes to realize the video coding method provided by the embodiment of the application.

Accordingly, the embodiment of the application also provides a computer readable storage medium, wherein instructions are stored in the computer readable storage medium, when the computer readable storage medium runs on a computer, the computer is enabled to realize the video coding method provided by the embodiment of the application.

Accordingly, embodiments of the present application also provide a computer program product comprising a computer program or computer instructions stored in a computer readable storage medium. The processor of the computer device reads the computer program or the computer instructions from the computer readable storage medium, and the processor executes the computer program or the computer instructions, so that the computer device implements the video encoding method provided by the embodiment of the application.

According to the video coding method provided by the embodiment of the application, the target video coding parameters can be determined from a plurality of video coding parameters by utilizing the prediction quality relation data, so that the processing efficiency of video coding is effectively improved; the prediction quality relation data can be accurately determined, and the accuracy and the effectiveness of the data are effectively ensured; the video can be encoded by utilizing the target video encoding parameters, so that the encoding quality value of the encoding result is matched with the reference encoding quality value corresponding to the target encoding quality index, the matching of the encoding quality and the preset quality is realized, the mapping between the video encoding parameters and the encoding quality index is established, and the code rate is effectively saved.

Drawings

In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings that are required in the embodiments or the description of the prior art will be briefly described below, it being obvious that the drawings in the following description are only some embodiments of the present application, and that other drawings may be obtained according to these drawings without inventive effort for a person skilled in the art.

Fig. 1 is a schematic system architecture of a video coding system according to an embodiment of the present application;

Fig. 2 is a schematic flow chart of a video encoding method according to an embodiment of the present application;

fig. 3 is a schematic diagram of a method for determining video coding parameters according to an embodiment of the present application;

fig. 4 is a schematic diagram of a video encoding method according to an embodiment of the present application;

FIG. 5 is a schematic flow chart of a model training method according to an embodiment of the present application;

FIG. 6 is a schematic diagram of a model training method provided in an embodiment of the present application;

fig. 7 is a flowchart of another video encoding method according to an embodiment of the present application;

fig. 8 is a block diagram of a video encoding apparatus according to an embodiment of the present application;

fig. 9 is a block diagram of a computer device according to an embodiment of the present application.

Detailed Description

The following description of the embodiments of the present application will be made clearly and fully with reference to the accompanying drawings, in which it is evident that the embodiments described are only some, but not all, of the embodiments of the present application. All other embodiments, which can be made by one of ordinary skill in the art based on the embodiments herein without making any inventive effort, are intended to be within the scope of the present application.

It should be noted that the descriptions of "first," "second," and the like in the embodiments of the present application are for descriptive purposes only and are not to be construed as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Thus, a technical feature defining "first", "second" may include at least one such feature, either explicitly or implicitly.

With the continuous development of computer technology, video coding standards are increasing. Video encoders based on different coding standards have different coding capabilities. In some cases, it is desirable that the coding quality of the video encoder be able to meet target requirements (e.g., the coding quality of the video matches the coding quality of the reference). Therefore, how to match the coding quality of the video encoder with the target quality is a problem to be solved.

Based on this, the embodiment of the application provides a video coding method, which can acquire video characteristic data of a video to be processed and acquire a video coding parameter sequence, wherein the video coding parameter sequence comprises a plurality of video coding parameters; determining a first video coding parameter and a second video coding parameter from a plurality of video coding parameters, wherein the arrangement positions of the first video coding parameter and the second video coding parameter in a video coding parameter sequence are adjacent, and the prediction quality relation data corresponding to the first video coding parameter and the second video coding parameter are different; the predicted quality relation data is obtained by processing video characteristic data and video coding parameters; the prediction quality relation data is used for indicating a numerical relation between a prediction coding quality value and a reference coding quality value, the prediction coding quality value is used for indicating the coding quality of the video to be processed based on the video coding parameter, and the reference coding quality value is a coding quality value corresponding to a target coding quality index; determining a target video coding parameter from the first video coding parameter and the second video coding parameter; and carrying out coding treatment on the video to be treated according to the target video coding parameters by using a video coder to obtain a video coding result, wherein the coding quality value of the video coding result is matched with the reference coding quality value corresponding to the target coding quality index. By the method provided by the embodiment of the application, the video encoder can be matched with the reference encoding quality value aiming at the encoding quality value of the video, so that the encoding quality of the video can reach the preset requirement, and the code rate is saved.

The video coding method provided by the embodiment of the application can be applied to the field of cloud computing. Cloud computing (clouding) is a computing model that distributes computing tasks across a large pool of computers, enabling various application systems to acquire computing power, storage space, and information services as needed. The cloud computing resource pool mainly comprises: computing devices (which are virtualized machines, including operating systems), storage devices, network devices. In the method, after the video characteristic data and the video coding parameter sequence of the video to be processed are obtained, the prediction quality relation data of the video coding parameters can be determined by using a prediction model according to a cloud computing technology, the first video coding parameters and the second video coding parameters are determined from a plurality of video coding parameters, further the target video coding parameters are determined, and the video to be processed is coded by using a video coder, so that the coding quality value of a video coding result is matched with the reference coding quality value, the video coding efficiency is improved, and the code rate is saved.

The architecture of the video coding system provided in the embodiments of the present application will be described below with reference to the accompanying drawings.

Referring to fig. 1, the system architecture of a video coding system provided in an embodiment of the present application includes a terminal device 101, a video coding device 102, and a database 103, where the video coding device 102 may interact with the terminal device 101 and the database 103. Wherein:

The terminal device 101 may receive the video encoding result transmitted by the video encoding device 102, and decode the received video encoding result to obtain an original video. The terminal device 101 may be, but is not limited to, a handheld device (e.g., a smart phone, a tablet computer), a computing device (e.g., a personal computer (Personal Computer, PC), an in-vehicle terminal, a smart voice interaction device, a wearable device, or other smart appliance, etc. having video decoding and communication functions.

The video encoding device 102 may receive the video to be processed sent by other devices, process the video to be processed, and also may generate the video to be processed. The video encoding device 102 includes a video encoder therein. The video encoding device 102 may be a video encoding server, where the video encoding device 102 may be an independent physical server, or may be a server cluster or a distributed system formed by multiple physical servers, or may be a cloud server that provides cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, content delivery networks (Content Delivery Network, CDN), and basic cloud computing services such as big data and artificial intelligence platforms.

The database 103 is used to store relevant data of the video encoding device 102, such as: video to be processed, video coding parameter sequences, video coding results and the like. The database 103 may be a local database in the video encoding device 102, or may be a cloud database (i.e. a database deployed in the cloud) associated with the video encoding device 102, specifically may be deployed based on any one of a private cloud, a public cloud, a hybrid cloud, an edge cloud, and the like, so that the functions of the cloud databases that are focused are different.

The operation of the video coding system shown in fig. 1 will be described in detail as follows:

the terminal device 101 may transmit reference coding quality data corresponding to the target coding quality index and a video acquisition request for the video to be processed to the video coding device 102. After receiving the video acquisition request, the video encoding device 102 may acquire video feature data of the video to be processed and a video encoding parameter sequence from the database 103, where the video encoding parameter sequence includes a plurality of video encoding parameters; the video encoding device 102 may determine a first video encoding parameter and a second video encoding parameter from a plurality of video encoding parameters, where the first video encoding parameter and the second video encoding parameter are adjacent to each other in an arrangement position in the video encoding parameter sequence, and prediction quality relationship data corresponding to the first video encoding parameter and the second video encoding parameter are different; the predicted quality relation data is obtained by processing video characteristic data and video coding parameters; the prediction quality relation data is used for indicating a numerical relation between a prediction coding quality value and a reference coding quality value, the prediction coding quality value is used for indicating the coding quality of the video to be processed based on the video coding parameter, and the reference coding quality value is a coding quality value corresponding to a target coding quality index; determining a target video coding parameter from the first video coding parameter and the second video coding parameter; the video encoding device 102 may encode the video to be processed according to the target video encoding parameter by using a video encoder to obtain a video encoding result, where the encoding quality value of the video encoding result matches the reference encoding quality value corresponding to the target encoding quality index. The video encoding device 102 sends the video encoding result to the terminal device 101, and the terminal device 101 decodes according to the video encoding result to obtain the video to be processed. By the video coding method provided by the embodiment of the application, the coding quality value of the video coder can be matched with the reference coding quality value, so that the video coding result reaches the preset condition, the code rate can be effectively saved, and the consumed data resources can be reduced.

It will be appreciated that the architecture diagram of the video encoding system described in the embodiments of the present application is for more clearly describing the video encoding method of the embodiments of the present application, and does not constitute a limitation on the video encoding method provided in the embodiments of the present application. It will be appreciated by those of ordinary skill in the art that the number of terminal devices 101, video encoding devices 102, and databases 103 in fig. 1 are merely illustrative. Any number of devices may be configured as desired for a service implementation. Moreover, with the evolution of the system architecture and the appearance of new service scenarios, the video coding method provided by the embodiment of the application is also applicable to similar technical problems.

Referring to fig. 2, fig. 2 is a flowchart of a video encoding method according to an embodiment of the present application. The video encoding method may be implemented by the video encoding device 102 described above, or may be implemented by other devices. The flow of the video coding method provided in the embodiment of the application includes, but is not limited to:

s201, obtaining video characteristic data of a video to be processed and obtaining a video coding parameter sequence, wherein the video coding parameter sequence comprises a plurality of video coding parameters.

In this embodiment of the present application, the video to be processed may be a live video in a live video scene, may be an on-demand video in a video on demand scene, or may be a video stored in a video encoding device. There are a number of different video coding standards, such as: advanced video coding (Advanced Video Coding, AVC, also known as the h.264 standard), high efficiency video coding (High Efficiency Video Coding, HEVC, also known as the h.265 standard), AV1 standard, and so forth. Video encoders based on different video coding standards have different video coding capabilities, such as: for the same video, the video is encoded by using a video encoder based on the H.264 standard to obtain a first encoding result, the video is encoded by using a video encoder based on the AV1 standard to obtain a second encoding result, and if the encoding quality of the first encoding result is matched with that of the second encoding result, the code rate of the second encoding result is less than that of the first encoding result when the video is decoded. The code Rate (Data Rate) refers to the Data traffic used by the video file per unit time. Therefore, for video encoders based on different video coding indexes, in order to make the coding quality reach the preset requirement and save the code rate as much as possible, the coding quality of the video encoder can be controlled by using the video coding parameters, so that the coding quality value of the video encoder for the video is matched with the reference coding quality value. The method provided by the embodiment of the application can be utilized to determine the target video coding parameters, so that the coding quality value of the video coder is matched with the reference coding quality value. A sequence of video coding parameters may be obtained, the sequence of video coding parameters comprising a plurality of video coding parameters, the video coding parameters being used to control the coding quality of the video encoder. By the method provided by the embodiment of the application, the video characteristic data of the video to be processed and a plurality of video coding parameters can be obtained, so that the target video coding parameters can be conveniently determined from the plurality of video coding parameters.

In an embodiment, the implementation manner of obtaining the video feature data of the video to be processed may be: performing downsampling processing on the video to be processed to obtain downsampled video; and (3) encoding the downsampled video by using a feature encoder to obtain video feature data of the video to be processed, wherein the video feature data comprises one or more of video block division information, video frame peak signal-to-noise ratio, block division coefficients and video macro block information. In this embodiment of the present application, downsampling may be performed on a video to be processed to obtain a downsampled video, and the downsampled video is encoded by using a feature encoder (the feature encoder may be a video encoder based on the h.264 standard) to obtain video feature data of the video to be processed, where the video feature data may include one or more of video block division information, video frame peak signal-to-noise ratio, block division coefficients, and video macroblock information. The video block division information is related information of coding block division of a video frame when a video encoder codes the video; the peak signal-to-noise ratio of the video frame can comprise peak signal-to-noise ratios (Peak Signal to Noise Ratio, PSNR) of different video frame types (such as intra-frame (I-frame), forward predictive-frame (P-frame), bi-predictive interpolation-frame (bi-directional interpolated prediction frame, B-frame) and the like) in the video, and can reflect the quality of the different video frame types; the block division coefficients are the transformation coefficients of different division blocks in the video coding process; the video macroblock information may include skip (skip) macroblock information and direct (direct) macroblock information in a video encoding process. The video characteristic data of the video to be processed are used for predicting the predicted quality relation data corresponding to the video coding parameters, so that the predicted quality relation data obtained by prediction can be more accurate.

In an embodiment, the video coding parameters included in the video coding parameter sequence may be fixed rate coefficients (Constant Rate Factor, CRF) of the video encoder, where the fixed rate coefficients may enable the coding quality of each video frame in the video to be constant when the video encoder encodes the video, and the code rates of each video frame are different, so that the coding quality of the video coding results is consistent. The plurality of video coding parameters included in the video coding parameter sequence may be determined according to a video encoder, for example: the video encoder is a video encoder based on the AV1 standard, and the CRF value range of the video encoder is [45,60], so the video encoding parameter sequence may include 16 video encoding parameters, where the video encoding parameters are integers in the [45,60] interval. The video coding parameters in the sequence of video coding parameters may be arranged in order from small to large, for example: the first video coding parameter in the sequence of video coding parameters is 45, the second video coding parameter is 46, the third video coding parameter is 47, and so on. By the method provided by the embodiment of the application, a plurality of video coding parameters can be determined, so that the target video coding parameters can be conveniently and quickly determined from the plurality of video coding parameters.

In an embodiment, the target coding quality index is any one of a peak signal-to-noise ratio quality index, a structural similarity quality index, and a mixing quality index. A Peak signal-to-noise ratio (PSNR) quality index is a quality index for determining video coding quality by performing difference calculation on pixel points in an original video image and a processed video image; the structural similarity quality index is also called Multi-scale structural similarity (Multi-Scale structural similarity index, MS-SSIM) quality index, which is a quality index for determining video coding quality by comparing the structural similarity of an original video image and a processed video image; the hybrid quality index, also called Video multi-method Accesment Fusion (VMAF), is a Video coding quality assessment index combining three indexes of visual information fidelity (Visual Quality Fidelity, VIF), detail loss index (Detail Loss Measure, DLM) and temporal motion index/average correlation position pixel difference (temporal information, TI).

S202, determining a first video coding parameter and a second video coding parameter from the plurality of video coding parameters, wherein the arrangement positions of the first video coding parameter and the second video coding parameter in the video coding parameter sequence are adjacent, and the prediction quality relation data corresponding to the first video coding parameter and the second video coding parameter are different; the predicted quality relation data is obtained by processing the video characteristic data and the video coding parameters; the prediction quality relation data is used for indicating a numerical relation between a prediction coding quality value and a reference coding quality value, the prediction coding quality value is used for indicating the coding quality of the video to be processed by a video encoder based on the video coding parameter, and the reference coding quality value is a coding quality value corresponding to a target coding quality index.

In this embodiment of the present application, the first video coding parameter and the second video coding parameter may be determined from a plurality of video coding parameters, where the arrangement positions of the first video coding parameter and the second video coding parameter in the video coding parameter sequence are adjacent, for example: the video coding parameters included in the video coding parameter sequence are 45, 46, 47 to 60 in order, and the first video coding parameter may be 46 and the second video coding parameter may be 47. The prediction quality relation data corresponding to the first video coding parameters are obtained by processing the video characteristic data of the equipment to be processed and the first video coding parameters, and the prediction quality relation data corresponding to the second video coding parameters are obtained by performing prediction processing on the video characteristic data of the equipment to be processed and the second video coding parameters by using a prediction model. The prediction quality relation data is used for predicting a numerical relation between a coding quality value and a reference coding quality value, wherein the prediction coding quality value is used for indicating the coding quality of the video to be processed based on the video coding parameter, and the reference coding quality value is a coding quality value corresponding to the target coding quality index. The video encoder may be a video encoder based on the h.264 standard, a video encoder based on the h.265 standard, or a video encoder based on the AV1 standard. The prediction quality relationship data corresponding to the video coding parameter may indicate that the video encoder determines the target video coding parameter according to the prediction quality relationship data corresponding to the video coding parameter based on the relationship between the prediction coding quality value of the video to be processed and the reference coding quality value corresponding to the target coding quality index, so that the coding quality value of the video encoder for the video to be processed is matched with the reference coding quality value. The prediction quality relationship data corresponding to the first video coding parameter and the second video coding parameter are different, for example: the prediction quality relationship data corresponding to the first video coding parameter may indicate that the prediction coding quality value is less than the reference coding quality value, and the prediction quality relationship data corresponding to the second video coding parameter may indicate that the prediction coding quality value is greater than the reference coding quality value. By the method provided by the embodiment of the application, the first video coding parameter and the second video coding parameter can be determined from a plurality of video coding parameters, which is beneficial to determining the target video coding parameter according to the first video coding parameter and the second video coding parameter in the follow-up process, so that the coding quality value of the video coder aiming at the video to be processed is matched with the reference coding quality value.

In one embodiment, the prediction quality relationship data may be obtained by performing prediction processing on the video feature data and the video coding parameters using a prediction model, and the prediction model may be obtained by training an initial model using a sample data set, where the sample data set includes: the video encoder comprises video characteristic data of a sample video, a plurality of video encoding parameters and sample quality relation data corresponding to each video encoding parameter, wherein the sample quality relation data is used for indicating a numerical relation between a sample encoding quality value and a reference encoding quality value, and the sample encoding quality value is used for indicating the encoding quality of the video encoder for the sample video based on the video encoding parameters. The predictive model is trained using a plurality of sample data sets from an initial model, which may be a deep model (Deep Cross Network, DCN) or a deep neural network model (Deep Neural Networks, DNN). The sample data set may include: video characteristic data of a sample video, a plurality of video coding parameters and sample quality relation data corresponding to each video coding parameter; wherein the sample quality relationship data may be used to indicate a numerical relationship between a sample encoding quality value and a reference encoding quality value, the sample encoding quality value may be used to indicate an encoding quality of the video encoder for the sample video based on the video encoding parameters. By the method provided by the embodiment of the application, the corresponding relation among the video characteristic data, the video coding parameters and the coding quality can be established by the prediction model, so that the prediction quality relation data corresponding to the video coding parameters can be accurately determined by using the prediction model, and the target video coding parameters can be rapidly and accurately determined.

In an embodiment, the predicted quality relationship data is first quality relationship data or second quality relationship data, where the first quality relationship data indicates that the predicted encoding quality value is greater than the reference encoding quality value, and the second quality relationship data indicates that the predicted encoding quality value is less than or equal to the reference encoding quality value, and the implementation manner of determining the first video encoding parameter and the second video encoding parameter from the plurality of video encoding parameters may be: inputting the video coding parameters and video characteristic data of the video to be processed into a prediction model in sequence according to the ordering sequence of a plurality of video coding parameters indicated by the video coding parameter sequence to perform prediction processing, so as to obtain prediction quality relation data corresponding to the 1 st video coding parameter to the K th video coding parameter respectively; if the predicted quality relation data corresponding to the 1 st video coding parameter to the K-1 st video coding parameter are all the first quality relation data and the predicted quality relation data corresponding to the K video coding parameter are the second quality relation data, determining the K-1 st video coding parameter as the first video coding parameter and the K video coding parameter as the second video coding parameter; the number of the video coding parameters is M, M is an integer greater than 1, and K is an integer greater than 1 and less than or equal to M. The video coding parameters in the video coding parameter sequence are arranged according to a certain ordering rule (for example, the order from small to large or from large to small), the number of the video coding parameters contained in the video coding parameter sequence is M, and M is an integer greater than 1; inputting video coding parameters and video characteristic data of a video to be processed into a prediction model value in turn according to the ordering sequence of a plurality of video coding parameters indicated by a video coding parameter sequence to perform prediction processing, so as to obtain prediction quality relation data respectively corresponding to the 1 st video coding parameter to the K (K is an integer greater than 1 and less than or equal to M) video coding parameters; if the predicted quality relationship data corresponding to the 1 st video coding parameter to the K-1 st video coding parameter are all the first quality relationship data and the predicted quality relationship data corresponding to the K-1 st video coding parameter are the second quality relationship data, the K-1 st video coding parameter can be determined as the first video coding parameter and the K-1 st video coding parameter can be determined as the second video coding parameter. The method utilizes the principle of dichotomy and can accurately and quickly determine the demarcation parameters. For example: the video coding parameter sequence comprises 5 video coding parameters, wherein the 1 st video coding parameter is 45, the 2 nd video coding parameter is 46, the 3 rd video coding parameter is 47, the 4 th video coding parameter is 48, and the 5 th video coding parameter is 49; the method can be based on the principle of dichotomy, according to the ordering sequence of 5 video coding parameters indicated by a video coding parameter sequence, inputting the 1 st video coding parameter and video characteristic data of the video to be processed into a prediction model for prediction processing, so as to obtain prediction quality relation data corresponding to the 1 st video coding parameter, wherein the prediction quality relation data is first quality relation data; inputting the 2 nd video coding parameter and video characteristic data of the video to be processed into a prediction model for prediction processing to obtain prediction quality relation data corresponding to the 2 nd video coding parameter, wherein the prediction quality relation data is first quality relation data; inputting the 3 rd video coding parameter and video characteristic data of the video to be processed into a prediction model for prediction processing to obtain prediction quality relation data corresponding to the 3 rd video coding parameter, wherein the prediction quality relation data is first quality relation data; inputting the 4 th video coding parameter and video characteristic data of the video to be processed into a prediction model for prediction processing to obtain prediction quality relation data corresponding to the 4 th video coding parameter, wherein the prediction quality relation data is second quality relation data; at this time, K is 4, the prediction quality relationship data corresponding to the 1 st video coding parameter to the (K-1 is 3) th video coding parameter are all the first quality relationship data, and the prediction quality relationship data corresponding to the K (K is 4) th video coding parameter is the second quality relationship data, which indicates that the coding quality of the video to be processed based on the 3 rd video coding parameter is greater than the reference coding quality value, the coding quality of the video to be processed based on the 4 th video coding parameter is less than or equal to the reference coding quality value, that is, the 3 rd video coding parameter and the 4 th video coding parameter are boundary parameters, and the coding quality of the video to be processed based on the 3 rd video coding parameter or the 4 th video coding parameter is matched with the reference coding quality value.

It is assumed that the video coding sequence includes 8 video coding parameters, which are respectively: 1. 2, 3, 4, 5, 6, 7, 8; and inputting the video characteristic data of the video to be processed and the video coding parameters into a prediction model in sequence according to the arrangement sequence of the video coding parameters in the video coding parameter sequence to process, so as to obtain the prediction quality relation data of the video coding parameters, as shown in the following table one.

List one

The first table contains the first 6 video coding parameters in the video coding parameter sequence, which are 1, 2, 3, 4, 5 and 6 respectively; the value of the first quality relation data in the first table is 0, and the value of the second quality relation data is 1; as shown in table one, the predicted quality relationship data corresponding to the 6 video coding parameters are 0, 1, respectively. As can be seen from the first table, when the video coding parameter is less than or equal to 5, the predictive coding quality value is greater than the reference coding quality value, that is, the coding quality value of the video encoder for the video to be processed is greater than the reference coding quality value; when the video coding parameter is equal to 56, the predictive coding quality value is less than or equal to the reference coding quality value, i.e. when the predictive coding quality value of the video encoder for the video to be processed is less than or equal to the reference coding quality value, it is indicated that when the video coding parameter is 55 or 56, the coding quality value of the video encoder for the video to be processed is closest to the reference coding quality value, i.e. the coding quality value of the video encoder for the video to be processed matches the reference coding quality value, therefore, the video coding parameters 55 and 56 are boundary parameters, the video coding parameter 55 can be determined as a first video coding parameter, and the video coding parameter 56 can be determined as a second video coding parameter. By the method provided by the embodiment of the application, the first video coding parameter and the second video coding parameter can be accurately determined from the plurality of video coding parameters by using the prediction model, so that the processing efficiency is improved, the target video coding parameter is determined, and the coding quality of the video is ensured.

Please refer to fig. 3, which is a schematic diagram of a video coding parameter determining method according to an embodiment of the present application. In fig. 3, the video coding parameter sequence includes a plurality of video coding parameters (the number of the video coding parameters is M, and M is an integer greater than 1), and the video coding parameters in the video coding parameter sequence are arranged in order from small to large; the value of the predicted quality relation data is first quality relation data or second quality relation data, the first quality relation data indicates that the predicted coding quality value is larger than the reference coding quality value, and the second quality relation data indicates that the predicted coding quality value is smaller than or equal to the reference coding quality value. Determining video coding parameters from the video coding parameter sequence according to the arrangement sequence, inputting the video coding parameters and video characteristic data of the video to be processed into a prediction model for processing, and obtaining prediction quality relation data of the video coding parameters; judging whether the predicted quality relation data of the video coding parameters are first quality relation data, if not, ending the processing flow, if so, inputting the next video coding parameter of the video coding parameters in the video coding parameter sequence and the video characteristic data of the video to be processed into a prediction model for processing to obtain the predicted quality relation data of the next video coding parameters, judging whether the predicted quality relation data of the next video coding parameters are second quality relation data, if so, determining the video coding parameters and the next video coding parameters of the video coding parameters as boundary parameters, determining the video coding parameters as first video coding parameters, determining the next video coding parameters of the video coding parameters as second video coding parameters, and ending the processing flow; if the predicted quality relationship data of the next video coding parameter is not the second quality relationship data, the video coding parameter is re-determined from the sequence of video coding parameters and the predicted quality relationship data of the video coding parameter is determined. By the method provided by the embodiment of the application, the prediction quality relation data of the video coding parameters in the video coding parameter sequence can be calculated one by one, so that the demarcation parameters can be determined, and further the first video coding parameters and the second video coding parameters can be accurately determined.

S203, determining target video coding parameters from the first video coding parameters and the second video coding parameters.

In the embodiment of the application, the arrangement positions of the first video coding parameter and the second video coding parameter in the video coding parameter sequence are adjacent, and the prediction quality relation data corresponding to the first video coding parameter and the second video coding parameter are different, which indicates that the first video coding parameter and the second video coding parameter are boundary parameters, and the video encoder is matched with the reference coding quality value based on the boundary parameters aiming at the coding quality value of the video to be processed. The target video encoding parameter may be determined from the first video encoding parameter and the second video encoding parameter. For example: the target coding quality index is VMAF, the reference coding quality value is 91, and the video encoder is a video encoder based on the AV1 standard, the target video coding parameters can be determined, so that the target coding quality value of the video encoder based on the AV1 standard for the video to be processed is 91, and the coding capability of the video encoder based on the AV1 standard is stronger, so that the cost of the code rate is lower. By the method provided by the embodiment of the application, the target video coding parameters can be accurately determined, so that the target coding quality value of the video to be processed is matched with the reference coding quality value based on the target video coding parameters, the code rate is effectively saved, and the consumed resources are reduced under the condition that the coding quality of the video to be processed meets the preset requirement.

In an embodiment, the value of the first video coding parameter is smaller than the value of the second video coding parameter, the smaller the value of the video coding parameter, the larger the corresponding code rate of the video encoder, the higher the coding quality of the video by the video encoder, the implementation manner of determining the target video coding parameter from the first video coding parameter and the second video coding parameter may be: if the quality requirement is higher than the code rate requirement for the coding of the video coder, determining the first video coding parameter as a target video coding parameter; if the code rate requirement is higher than the quality requirement for the coding setting of the video coder, the video coding parameter is determined as the target video coding parameter. The value of the first video coding parameter is smaller than the value of the second video coding parameter, when the video coding parameter is a fixed rate Coefficient (CRF), the smaller the value of the video coding parameter, the larger the code rate spent by the video coder for the video to be processed, the higher the coding quality of the video to be processed by the video coder, namely the coding quality of the video to be processed based on the first video coding parameter is higher than the coding quality of the video to be processed based on the second video coding parameter by the video coder; the code rate spent by the video encoder for the video to be processed based on the first video encoding parameter is greater than the code rate spent by the video encoder for the video to be processed based on the second video encoding parameter. The first video encoding parameter may be determined to be a target video encoding parameter if the encoding for the video encoder is set to be higher than the code rate requirement, and the second video encoding parameter may be determined to be a target video encoding parameter if the encoding device for the video encoder is set to be higher than the code rate requirement. By the method provided by the embodiment of the application, the target video coding parameter can be determined from the first video coding parameter and the second video coding parameter according to specific quality requirements and code rate requirements, so that the target video coding parameter can meet application requirements.

S204, the video encoder is utilized to encode the video to be processed according to the target video encoding parameters, so as to obtain a video encoding result, and the encoding quality value of the video encoding result is matched with the reference encoding quality value corresponding to the target encoding quality index.

In this embodiment of the present application, after determining the target video encoding parameter, the video encoder may be used to encode the video to be processed according to the target video encoding parameter to obtain a video encoding result, where the encoding quality value of the video encoding result matches the reference encoding quality value. Aiming at the video to be processed, the video encoder is matched with the reference coding quality value corresponding to the target coding quality index based on the coding quality value of the target video coding parameter, so that the video coding result costs as little code rate as possible on the premise of meeting the coding quality requirement, thereby effectively saving the cost and improving the processing efficiency; meanwhile, the video encoder is matched with the reference quality value aiming at the coding quality value of the video to be processed, so that the coding quality of the video to be processed meets the requirements of users, and the experience of the users is better ensured. By the method provided by the embodiment of the application, the video encoder can be matched with the reference encoding quality value aiming at the encoding quality value of the video, so that the effects of reducing cost and enhancing efficiency are realized, the code rate is greatly saved, and the experience of a user is ensured.

In an embodiment, the implementation manner of determining the encoding quality value of the video encoding result may be: and performing quality evaluation processing on the video coding result by using the target coding quality index to obtain a coding quality value of the video coding result. The target coding quality index may be any one of a peak signal-to-noise ratio quality index, a structural similarity quality index and a mixing quality index. By the method provided by the embodiment of the application, the quality of the video coding result can be quantized into a specific coding quality value, so that the size relation between the coding quality value and the reference coding quality value can be intuitively reflected.

In an embodiment, the method provided by the embodiment of the application can only predict a plurality of video coding parameters to obtain the predicted quality relation data, so that the processing steps are effectively simplified, and the consumed resources are reduced; the prediction model establishes a mapping relation between video coding parameters and reference coding quality values, so that the method provided by the application can realize accurate prediction, for example: in addition to determining the target video coding parameter corresponding to the video encoder when the reference coding quality value corresponding to the target coding quality indicator VMAF is 91 by using the method of the present application, the reference coding quality value corresponding to the target coding quality indicator may also be determined when the video coding parameter is a critical value (for example, 45 or 60). In some cases, the prediction accuracy of the method provided by the present application exceeds 90% when the absolute value of the prediction error of the target coding quality indicator VMAF is less than 1; the prediction accuracy of the method provided by the present application exceeds 90% when the absolute value of the prediction error of the target coding quality index PSNR is less than 0.2. By the method provided by the embodiment of the application, the target video coding parameters can be accurately predicted.

Please refer to fig. 4, which is a schematic diagram of a video encoding method according to an embodiment of the present application. For the video to be processed, the video to be processed can be processed to obtain video characteristic data of the video to be processed; inputting video characteristic data of the video to be processed and video coding parameters in the video coding parameter sequence into a prediction model for prediction processing to obtain prediction quality relation data corresponding to the video coding parameters; the first video coding parameter and the second video coding parameter can be determined according to the prediction quality relation data corresponding to the video coding parameters; determining a target video coding parameter from the first video coding parameter and the second video coding parameter according to the coding setting of the video encoder; the video encoder can be utilized to encode the video to be processed based on the target video encoding parameters to obtain a video encoding result, and the encoding quality value of the video encoding result is matched with the reference encoding quality value. By the method provided by the embodiment of the application, the target video coding parameters can be rapidly determined, the processing efficiency of video coding is effectively improved, the video coding result meets the requirements, and the code rate is saved.

According to the video coding method provided by the embodiment of the application, the target video coding parameters can be determined from a plurality of video coding parameters, so that the coding quality value of the video to be processed by the video coder is matched with the reference coding quality value, namely, the coding quality of the video by the video processor can be matched with the preset requirement, the accuracy of video coding control is effectively improved, and the experience of a user is ensured; the prediction model can be utilized to accurately determine the first video coding parameter and the second video coding parameter, so that the time spent by video coding processing can be reduced, the efficiency of video coding processing is effectively improved, and the repeated redundant processing process is avoided; the target video coding parameters can be determined according to the quality requirements and the code rate requirements in the coding setting, the method can be suitable for various scenes, the code rate can be effectively saved under the condition that the video coding result meets the requirements, and the consumed resources are reduced.

Referring to fig. 5, fig. 5 is a flow chart of a model training method according to an embodiment of the present application. The model training method may be implemented by the video encoding device 102 described above, or may be implemented by other devices. The flow of the model training method provided in the embodiment of the application includes, but is not limited to:

S501, acquiring the sample video and the video coding parameters, and coding the sample video by using a feature coder to obtain video feature data of the sample video.

In this embodiment of the present application, a plurality of sample videos may be acquired for model training, where values of a plurality of video coding parameters are determined according to a video encoder, for example: if the video encoder is a video encoder based on the H.265 standard, the value range of the video encoding parameters can be [1,51], and the video encoding parameters are integers; if the video encoder is a video encoder based on the AV1 standard, the range of values of the video encoding parameters may be [45, 60], and the video encoding parameters may be integers. The feature encoder may be used to encode the sample video to obtain video feature data of the sample video, where the feature encoder is the same as the feature encoder that processes the video to be processed to obtain video feature data of the video to be processed in the above embodiment. The method provided by the embodiment of the application can acquire the sample video, and train the initial model by utilizing the sample video to obtain the prediction model, so that the overall efficiency of video coding processing is improved.

It should be noted that the plurality of video coding parameters involved in model training are the same as the video coding parameters included in the video coding parameter sequence at the time of actual use.

In an embodiment, the implementation manner of encoding the sample video by using the feature encoder to obtain the video feature data of the sample video may be: carrying out downsampling treatment on the sample video to obtain a sample downsampled video; and (3) encoding the sample downsampled video by using a feature encoder to obtain video feature data of the sample video, wherein the video feature data comprises one or more of video block division information, video frame peak signal-to-noise value, block division coefficient and video macro block information. The sample video may be processed using a feature encoder to obtain video feature data for the sample video. By the method provided by the embodiment of the application, the video characteristic data of the sample video can be obtained, and the model can be conveniently built in a corresponding relation by using the video characteristic data, so that the prediction accuracy of the model is improved.

S502, the video encoder is utilized to encode the sample video according to a reference video encoding parameter to obtain a sample video encoding result, a sample encoding quality value of the sample video encoding result is determined, sample quality relation data corresponding to the reference video encoding parameter is determined according to the reference encoding quality value and the sample encoding quality value, and the reference video encoding parameter is any one video encoding parameter of the video encoding parameters.

In this embodiment, in the model training process, a plurality of sample videos may be used to perform model training processing, and here, a sample video is taken as an example to describe: the sample video may be encoded by a video encoder according to a reference video encoding parameter, to obtain a sample video encoding result, and a sample encoding quality value of the sample video encoding result may be determined, where the reference video encoding parameter is any one of a plurality of video encoding parameters. For example: there are 5 video coding parameters, and the 5 video coding parameters are 45, 46, 47, 48, 49, respectively, then for any one of the 5 video coding parameters, a video encoder may be used to encode the sample video according to the video coding parameter to obtain a sample video coding result corresponding to the video coding parameter, and may determine a sample coding quality value corresponding to the video coding parameter, and then may determine sample coding quality values corresponding to the 5 video coding parameters are 96, 95, 94, 93, 92, respectively. After the sample coding quality value of the sample video coding result is determined, sample quality relation data corresponding to the reference video coding parameter can be determined according to the sample coding quality value and the reference coding quality value, and the sample quality relation data can indicate the magnitude relation between the sample coding quality value and the reference coding quality value. The steps shown in the embodiments of the present application may be performed according to each video coding parameter in the plurality of video coding parameters, to determine sample quality relationship data corresponding to each video coding parameter. By the method provided by the embodiment of the application, the size relation between the sample coding quality value and the reference coding quality value can be converted into specific sample quality relation data, so that the model training of the initial model by using the sample quality relation data is facilitated, and a prediction model capable of determining the prediction quality relation data is obtained.

In an embodiment, the implementation manner of determining the sample coding quality value of the sample video coding result may be: and carrying out quality evaluation processing on the sample video coding result by utilizing the target coding quality index to obtain a sample coding quality value of the sample video coding result. The target coding quality index may be any one of a peak signal-to-noise ratio quality index, a structural similarity quality index and a mixing quality index. By the method provided by the embodiment of the application, the quality of the sample video coding result can be quantized into specific data, and subsequent comparison processing is facilitated.

In an embodiment, the implementation manner of determining the sample quality relation data corresponding to the reference video coding parameter according to the reference coding quality value and the sample coding quality value may be: comparing the reference code quality value with the sample code quality value; if the comparison result indicates that the sample coding quality value is larger than the reference coding quality value, determining sample quality relation data corresponding to the reference video coding parameter as first quality relation data; and if the comparison result indicates that the sample coding quality value is smaller than or equal to the reference coding quality value, determining that the sample quality relation data corresponding to the reference video coding parameter is the second quality relation data. The reference coding quality value and the sample coding quality value are both data indicating coding quality, the reference coding quality value and the sample coding quality value can be compared to obtain a comparison result, and if the comparison result indicates that the sample coding quality value is greater than the reference coding quality value, the sample quality relation data corresponding to the reference video coding parameter can be determined to be first quality relation data; if the comparison result indicates that the sample coding quality value is less than or equal to the reference coding quality value, it may be determined that the sample quality relationship data corresponding to the reference video coding parameter is second quality relationship data, for example: the reference coding quality value is 23, the sample coding quality value is 24, the sample coding quality value is greater than the reference coding quality value, and the sample quality relation data corresponding to the reference video coding parameter is the first quality relation data (0). For a plurality of video coding parameters, sample quality relationship data corresponding to each video coding parameter may be determined. For example: there are 6 video coding parameters, 46, 47, 48, 49, 50, 51 respectively; assuming that the value of the first quality relation data is 0 and the value of the second quality relation data is 1, calculating to know that the sample relation quality data corresponding to the 6 video coding parameters are 0, 1 and 1 respectively; it can be determined that the sample coding quality value is greater than the reference coding quality value when the video coding parameter is less than or equal to 48, i.e., the coding quality value for the sample video is greater than the reference coding quality value at that time; when the video coding parameter is greater than or equal to 49, the sample coding quality value is less than or equal to the reference coding quality value, that is, when the coding quality value of the video coder for the sample video is less than or equal to the reference coding quality value, it is indicated that when the video coding parameter is 48 or 49, the coding quality value of the video coder for the sample video is closest to the reference coding quality value, that is, the coding quality value of the video coder for the sample video is matched with the reference coding quality value. Video coding parameters 48 and 49 may be used as demarcation parameters. By the method provided by the embodiment of the application, the sample quality relation data of each video coding parameter can be determined, the subsequent model training by using the sample quality relation data is facilitated, and the model learns the relation between the quality relation data and the predicted quality of the video encoder, so that the predicted quality relation data can be accurately predicted.

S503, after obtaining sample quality relation data corresponding to each video coding parameter in the plurality of video coding parameters, performing model training on an initial model by using video characteristic data of the sample video, the plurality of video coding parameters and the sample quality relation data corresponding to each video coding parameter to obtain the prediction model.

In this embodiment of the present application, after obtaining sample quality relationship data corresponding to each video coding parameter in the plurality of video coding parameters, a plurality of sample data sets may be formed by using video feature data of a sample video, the plurality of video coding parameters, and the sample quality relationship data corresponding to each video coding parameter. For example: and 5 video coding parameters exist, 5 sample data sets can be formed according to the video characteristic data of the sample video, a plurality of video coding parameters and sample quality relation data corresponding to each video coding parameter, each sample data set comprises the video characteristic data of the sample video, the video coding parameters and the sample quality relation data corresponding to the video coding parameters, and the video coding parameters included in each sample data set are different from each other. The initial model may be model trained using a plurality of sample data sets to obtain a predictive model. By the method provided by the embodiment of the application, the prediction accuracy of the prediction model can be effectively provided, so that the coding quality of the video coder is ensured.

In an embodiment, the model training is performed on the initial model by using a plurality of sample data sets, so as to obtain an implementation manner of the prediction model, which may be: inputting video characteristic data and video coding parameters of sample videos in any sample data group into an initial prediction model for prediction processing aiming at any sample data group in a plurality of sample data groups to obtain initial quality relation data; carrying out loss calculation on the initial quality relation data and the sample quality relation data in any sample data set to obtain loss data corresponding to any sample data set; and carrying out model parameter adjustment on the initial model according to the loss data corresponding to each sample data group in the plurality of sample data groups to obtain a prediction model. The model parameter adjustment can be carried out on the initial model for multiple times by utilizing a plurality of sample data sets corresponding to a plurality of sample videos, when the model parameter adjustment times of the initial model reach the preset times or the prediction accuracy of the initial model reaches the preset accuracy, the model training process of the initial model can be determined to be completed, and the initial model at the moment can be determined to be the prediction model. By the method provided by the embodiment of the application, the training process of the initial model can be realized rapidly and limitedly, so that the prediction model obtained by training according to the initial model can have good prediction accuracy.

In an embodiment, the prediction model may be a fully connected neural network model, which may include at least two fully connected layers and corresponding weight parameters. For example: the prediction model comprises two full-connection layers, wherein the first full-connection layer can be used for carrying out feature extraction processing for extracting relevant features of input video feature data and video coding parameters, and the second full-connection layer can be used for carrying out feature classification processing for carrying out classification processing according to the extracted relevant features, so that prediction quality relation data is obtained. In the model training process, the initial model can comprise at least two full-connection layers and initial model weight parameters, video characteristic data and video coding parameters of a sample video can be input into the initial model for prediction processing to obtain initial quality relation data, loss data can be determined according to the initial quality relation data and the sample quality relation data corresponding to the video coding parameters, and the loss data is utilized to adjust the initial model weight parameters, so that the trained prediction model can construct a mapping relation among the video characteristic data, the video coding parameters and reference coding quality values of the video. In the use process of the model, video characteristic data and video coding parameters of the video to be processed are input into a prediction model, so that prediction quality relation data can be obtained, and the target video coding parameters can be determined. By the method provided by the embodiment of the application, the prediction model can construct the mapping relation between the video coding parameters of the encoder and the reference coding quality values, so that the target video coding parameters can be accurately and rapidly determined, and the coding quality of the encoder can meet the requirements.

Referring to fig. 6, a schematic diagram of a model training method according to an embodiment of the present application is shown. In fig. 6, taking a training process of an initial model as an example: the video encoder can be utilized to encode the sample video based on the video encoding parameters to obtain a sample encoding quality value; and sample quality relation data corresponding to the video coding parameters can be determined according to the sample coding quality value and the reference coding quality value corresponding to the target coding quality index; the sample data set comprises video characteristic data, video coding parameters and sample quality relation data corresponding to the video coding parameters of the sample video, and the video characteristic data and the video coding parameters of the sample video in the sample data set can be input into an initial model for prediction processing to obtain initial quality relation data; carrying out loss calculation on the initial quality relation data and sample quality relation data corresponding to the video coding parameters to obtain loss data corresponding to the video coding parameters; model parameter adjustment can be performed on the initial model by using the loss data, so that model training is realized. By the method provided by the embodiment of the application, the initial model can be trained by utilizing the plurality of sample data sets corresponding to the plurality of sample videos, so that the model training efficiency is effectively provided, and the accuracy of model prediction is ensured.

According to the model training method provided by the embodiment of the application, the initial model can be trained by utilizing the video characteristic data, a plurality of video coding parameters and the sample data group containing the sample quality relation data corresponding to each video coding parameter of the sample video, so that the initial model can accurately determine the corresponding prediction quality relation data according to the video characteristic data and the video coding parameters of the video to be processed, the efficiency of video coding processing is effectively improved, and the matching of the coding quality of video encoders based on different coding standards for the same video is ensured; the quality of the coding result can be converted into a specific numerical value by utilizing the target coding quality index, so that corresponding sample quality relation data can be conveniently determined, and the model training efficiency is effectively improved.

Referring to fig. 7, fig. 7 is a flowchart of another video encoding method according to an embodiment of the present application. The video encoding method may be implemented by the video encoding device 102 described above, or may be implemented by other devices. The video encoding method provided by the embodiment of the application is taken as an example for describing application to a live video scene, and the flow of the video encoding method includes but is not limited to:

S701, the live broadcast end sends a first live broadcast video to the server.

In the embodiment of the application, the live broadcast terminal can record or generate the live broadcast video, and the first direct broadcast video is a part of the live broadcast video. The live side may send the first live video to the server.

S702, the server processes the first direct-play video to obtain video feature data of the first direct-play video.

In the embodiment of the application, the server can perform downsampling processing on the first direct broadcast video to obtain a downsampled direct broadcast video; and the downsampled live video is encoded by utilizing a feature encoder to obtain video feature data of the first live video, wherein the video feature data can comprise one or more of video block division information, video frame peak signal-to-noise value, block division coefficient and video macro block information.

S703, the server acquires a video coding parameter sequence including a plurality of video coding parameters, and determines a first video coding parameter and a second video coding parameter from the plurality of video coding parameters.

In this embodiment of the present application, the video coding parameter sequence may include a plurality of video coding parameters, and the video coding parameters in the video coding parameter sequence are arranged according to a certain order. In a live scene, the quality requirement for video coding may be changed according to the actual requirement of a user, in order to save a code rate when the coding quality of a live video is matched with the coding quality required by the user, the method provided by the embodiment of the application may be adopted to determine a first video coding parameter and a second video coding parameter from a plurality of video coding parameters, wherein the arrangement positions of the first video coding parameter and the second video coding parameter in a video coding parameter sequence are adjacent, and the prediction quality relation data corresponding to the first video coding parameter and the second video coding parameter are different; the predicted quality relation data is obtained by processing video characteristic data and video coding parameters; the prediction quality relationship data is used for indicating a numerical relationship between a prediction coding quality value and a reference coding quality value, the prediction coding quality value is used for indicating the coding quality of the video encoder for the first direct broadcast video based on the video coding parameter, and the reference coding quality value is a coding quality value corresponding to the target coding quality index. A specific implementation of determining the first video coding parameter and the second video coding parameter from the plurality of video coding parameters may be as shown in step S202 described above.

In an embodiment, the target coding quality indicator may be any one of a peak signal-to-noise ratio quality indicator, a structural similarity quality indicator, and a hybrid quality indicator (VMAF). The prediction quality relationship data may be obtained by performing prediction processing on the video feature data and the video coding parameters using a prediction model, and the prediction model may be obtained by training an initial model using a sample data set including: the video encoder comprises video characteristic data of a sample video, a plurality of video encoding parameters and sample quality relation data corresponding to each video encoding parameter, wherein the sample quality relation data is used for indicating a numerical relation between a sample encoding quality value and a reference encoding quality value, and the sample encoding quality value is used for indicating the encoding quality of the video encoder for the sample video based on the video encoding parameters. For example: the reference coding quality value is a coding quality value 93 corresponding to the VMAF, and then the prediction model may construct a mapping relationship between video coding feature data, video coding parameters and the reference coding quality value of the video, input the video coding feature data and the video coding parameters of the video, and may predict to obtain prediction quality relationship data indicating the numerical relationship. The specific training procedure of the prediction model may be as shown in the above steps S501-S503.

S704, the server determines target video coding parameters from the first video coding parameters and the second video coding parameters, and performs coding processing on the first direct-play video by utilizing the video coder according to the target video coding parameters to obtain a first direct-play video coding result.

In this embodiment of the present application, after determining the first video encoding parameter and the second video encoding parameter, the server may determine the target video encoding parameter according to the application requirement. For example: the target coding quality index is VMAF, the reference coding quality value is 93, the video encoder is a video encoder based on AV1 standard, the larger the video coding parameter is, the lower the video coding quality is, the less the spent code rate is, the first video coding parameter is smaller than the second video coding parameter, in the live scene, the second video coding parameter can be determined as the target video coding parameter if the application requirement is spent code rate is less; assuming that the application requirement is that the coding quality is high, the first video coding parameter may be determined as the target video coding parameter. After the target video coding parameters are determined, a video encoder can be utilized to encode the first direct-play video according to the target video coding parameters to obtain a first direct-play video coding result, and the coding quality value of the first direct-play video coding result is matched with the reference coding quality value. By the method provided by the embodiment of the application, the code rate can be effectively saved and the consumed resources can be reduced under the condition that the coding quality of the first direct-play video meets the preset requirement.

It should be noted that the predictive model needs to be retrained when the reference video coding values change. For example: the current prediction model is obtained by training when the reference video coding value is 93 corresponding to VMAF (which may be abbreviated as "vmaf=93"), the prediction model can predict the numerical relationship between the coding quality of the video encoder and "vmaf=93", and if the reference video coding value is changed to 94 corresponding to VMAF or 23 corresponding to PSNR, the prediction model needs to be trained again to obtain a new prediction model, and the prediction process can be performed by using the new prediction model.

S705, the server sends the first direct broadcast video coding result to the playing end.

In this embodiment of the present application, after determining the first direct-broadcast video encoding result, the server may send the first direct-broadcast video encoding result to the playing end, so as to implement transmission of the direct-broadcast video.

S706, the playing end carries out video decoding processing on the first direct-play video coding result to obtain a first direct-play video.

In this embodiment of the present application, the playing end may receive the first direct-play video encoding result and perform video decoding, thereby obtaining a first direct-play video that meets an application requirement, and completing playing of the first direct-play video. By the method provided by the embodiment of the application, the video encoder can be matched with the reference encoding quality value aiming at the encoding quality value of the video, so that the effects of reducing cost and enhancing efficiency are realized, the code rate is greatly saved, the variable application requirements are met, and the experience of a user is improved.

It should be noted that, when the server determines the target video encoding parameter and encodes the first direct-broadcast video by using the target video encoding parameter, the direct-broadcast terminal sends the second direct-broadcast video to the server, and the server may directly encode the second direct-broadcast video by using the target video encoding parameter to obtain a second direct-broadcast video encoding result and send the second direct-broadcast video encoding result to the playing terminal.

According to the video coding method provided by the embodiment of the application, the server can determine the target video coding parameters from a plurality of video coding parameters, so that the coding quality value of the video coder on the live video is matched with the reference coding quality value, namely, the coding quality of the video processor on the video can reach the preset requirement to be matched, the accuracy of video coding control is effectively improved, and the experience of a user is improved; the time spent on video coding processing can be reduced, the efficiency of video coding processing is effectively improved, and the repeated redundant processing process is avoided; the target video coding parameters can be determined according to the quality requirements and the code rate requirements in the coding setting, the method can be suitable for various scenes, the code rate can be effectively saved under the condition that the video coding result meets the requirements, and the consumed resources are reduced.

Referring to fig. 8, fig. 8 is a block diagram of a video encoding apparatus according to an embodiment of the present application. The device comprises:

an obtaining unit 801, configured to obtain video feature data of a video to be processed and obtain a video coding parameter sequence, where the video coding parameter sequence includes a plurality of video coding parameters;

a determining unit 802, configured to determine a first video coding parameter and a second video coding parameter from the plurality of video coding parameters, where the arrangement positions of the first video coding parameter and the second video coding parameter in the video coding parameter sequence are adjacent, and prediction quality relationship data corresponding to the first video coding parameter and the second video coding parameter are different; the predicted quality relation data is obtained by processing the video characteristic data and the video coding parameters; the prediction quality relation data is used for indicating a numerical relation between a prediction coding quality value and a reference coding quality value, the prediction coding quality value is used for indicating the coding quality of the video to be processed by a video encoder based on the video coding parameter, and the reference coding quality value is a coding quality value corresponding to a target coding quality index;

The determining unit 802 is further configured to determine a target video coding parameter from the first video coding parameter and the second video coding parameter;

and a processing unit 803, configured to perform coding processing on the video to be processed according to the target video coding parameter by using the video encoder, so as to obtain a video coding result, where a coding quality value of the video coding result matches with a reference coding quality value corresponding to the target coding quality index.

In an embodiment, the prediction quality relation data is obtained by performing prediction processing on the video feature data and the video coding parameters by using a prediction model, the prediction model is obtained by training an initial model by using a sample data set, and the sample data set includes: the video encoder comprises video characteristic data of a sample video, the plurality of video encoding parameters and sample quality relation data corresponding to each video encoding parameter, wherein the sample quality relation data is used for indicating a numerical relation between a sample encoding quality value and the reference encoding quality value, and the sample encoding quality value is used for indicating the encoding quality of the video encoder for the sample video based on the video encoding parameters.

In an embodiment, the obtaining unit 801 is further configured to: acquiring the sample video and the plurality of video coding parameters, and coding the sample video by using a feature coder to obtain video feature data of the sample video; the video encoder is utilized to encode the sample video according to a reference video encoding parameter to obtain a sample video encoding result, a sample encoding quality value of the sample video encoding result is determined, sample quality relation data corresponding to the reference video encoding parameter is determined according to the reference encoding quality value and the sample encoding quality value, and the reference video encoding parameter is any one video encoding parameter of the plurality of video encoding parameters; after obtaining sample quality relation data corresponding to each video coding parameter in the plurality of video coding parameters, performing model training on an initial model by utilizing video characteristic data of the sample video, the plurality of video coding parameters and the sample quality relation data corresponding to each video coding parameter to obtain the prediction model.

In an embodiment, the obtaining unit 801 is specifically configured to, when determining, according to the reference encoding quality value and the sample encoding quality value, sample quality relation data corresponding to the reference video encoding parameter: comparing the reference encoding quality value with the sample encoding quality value; if the comparison result indicates that the sample coding quality value is larger than the reference coding quality value, determining sample quality relation data corresponding to the reference video coding parameter as first quality relation data; and if the comparison result indicates that the sample coding quality value is smaller than or equal to the reference coding quality value, determining that the sample quality relation data corresponding to the reference video coding parameter is second quality relation data.

In an embodiment, the predicted quality relation data is first quality relation data or second quality relation data, the first quality relation data indicating that the predicted coding quality data is greater than the reference quality data, the second quality relation data indicating that the predicted coding quality data is less than or equal to the reference quality data; the determining unit 802 is specifically configured to, when determining a target video coding parameter from the first video coding parameter and the second video coding parameter: inputting the video coding parameters and the video characteristic data of the video to be processed into the prediction model in sequence according to the ordering sequence of the video coding parameters indicated by the video coding parameter sequence to perform prediction processing, so as to obtain prediction quality relation data corresponding to the 1 st video coding parameter to the K th video coding parameter respectively; if the predicted quality relation data corresponding to the 1 st video coding parameter to the K-1 st video coding parameter are the first quality relation data and the predicted quality relation data corresponding to the K-1 th video coding parameter are the second quality relation data, determining the K-1 st video coding parameter as the first video coding parameter and the K-th video coding parameter as the second video coding parameter; the number of the video coding parameters is M, M is an integer greater than 1, and K is an integer greater than 1 and less than or equal to M.

In an embodiment, the value of the first video coding parameter is smaller than the value of the second video coding parameter, and the smaller the value of the video coding parameter is, the larger the code rate of the video encoder is, and the higher the coding quality of the video by the video encoder is; the determining unit 802 is specifically configured to, when determining a target video coding parameter from the first video coding parameter and the second video coding parameter: if the coding set for the video coder is that the quality requirement is higher than the code rate requirement, determining the first video coding parameter as a target video coding parameter; and if the code rate requirement is higher than the quality requirement for the coding setting of the video coder, determining the second video coding parameter as a target video coding parameter.

In an embodiment, the target coding quality index is any one quality index of a peak signal-to-noise ratio quality index, a structural similarity quality index and a mixing quality index.

In an embodiment, the obtaining unit 801 is specifically configured to, when obtaining video feature data of a video to be processed: performing downsampling processing on the video to be processed to obtain downsampled video; and encoding the downsampled video by using a feature encoder to obtain video feature data of the video to be processed, wherein the video feature data comprises one or more of video block division information, video frame peak signal-to-noise value, block division coefficient and video macro block information.

It may be understood that the functions of each functional unit of the video encoding apparatus in the embodiments of the present application may be specifically implemented according to the video encoding method in the embodiments of the method, and the specific implementation process may refer to the relevant description in the embodiments of the video encoding method, which is not repeated herein.

By the computer equipment provided by the embodiment of the application, the coding quality value of the video to be processed by the video coder is matched with the reference coding quality value, so that the accuracy of video coding control is effectively improved, and the experience of a user is ensured; the time spent on video coding processing can be reduced, the efficiency of video coding processing is effectively improved, and the repeated redundant processing process is avoided; the method is applicable to various scenes, and can effectively save the code rate and reduce the consumed resources under the condition that the video coding result meets the requirement.

Referring to fig. 9, fig. 9 is a block diagram of a computer device according to an embodiment of the present application. The computer device described in the embodiment of the present application includes: processor 901, communication interface 902, and memory 903. The processor 901, the communication interface 902, and the memory 903 may be connected by a bus or other means, which is exemplified in the embodiment of the present application.

Among them, the processor 901 (or CPU (Central Processing Unit, central processing unit)) is a computing core and a control core of a computer device, which can parse various instructions in the computer device and process various data of the computer device, for example: the CPU can be used for analyzing a startup and shutdown instruction sent by a user to the computer equipment and controlling the computer equipment to perform startup and shutdown operation; and the following steps: the CPU may transmit various types of interaction data between internal structures of the computer device, and so on. The communication interface 902 may optionally include a standard wired interface, a wireless interface (e.g., wi-Fi, mobile communication interface, etc.), controlled by the processor 901 for transceiving data. The Memory 903 (Memory) is a Memory device in a computer device for storing programs and data. It will be appreciated that the memory 903 here may include both built-in memory of the computer device and extended memory supported by the computer device. The memory 903 provides storage space that stores the operating system of the computer device, which may include, but is not limited to: android systems, iOS systems, windows Phone systems, etc., which are not limiting in this application.

In the present embodiment, the processor 901 performs the following operations by executing executable program code in the memory 903:

In an embodiment, the processor 901 is further configured to: acquiring the sample video and the plurality of video coding parameters, and coding the sample video by using a feature coder to obtain video feature data of the sample video; the video encoder is utilized to encode the sample video according to a reference video encoding parameter to obtain a sample video encoding result, a sample encoding quality value of the sample video encoding result is determined, sample quality relation data corresponding to the reference video encoding parameter is determined according to the reference encoding quality value and the sample encoding quality value, and the reference video encoding parameter is any one video encoding parameter of the plurality of video encoding parameters; after obtaining sample quality relation data corresponding to each video coding parameter in the plurality of video coding parameters, performing model training on an initial model by utilizing video characteristic data of the sample video, the plurality of video coding parameters and the sample quality relation data corresponding to each video coding parameter to obtain the prediction model.

In an embodiment, the processor 901 is specifically configured to, when determining the sample quality relation data corresponding to the reference video coding parameter according to the reference coding quality value and the sample coding quality value: comparing the reference encoding quality value with the sample encoding quality value; if the comparison result indicates that the sample coding quality value is larger than the reference coding quality value, determining sample quality relation data corresponding to the reference video coding parameter as first quality relation data; and if the comparison result indicates that the sample coding quality value is smaller than or equal to the reference coding quality value, determining that the sample quality relation data corresponding to the reference video coding parameter is second quality relation data.

In an embodiment, the predicted quality relation data is first quality relation data or second quality relation data, the first quality relation data indicating that the predicted coding quality data is greater than the reference quality data, the second quality relation data indicating that the predicted coding quality data is less than or equal to the reference quality data; the processor 901, when determining a target video coding parameter from the first video coding parameter and the second video coding parameter, is specifically configured to: inputting the video coding parameters and the video characteristic data of the video to be processed into the prediction model in sequence according to the ordering sequence of the video coding parameters indicated by the video coding parameter sequence to perform prediction processing, so as to obtain prediction quality relation data corresponding to the 1 st video coding parameter to the K th video coding parameter respectively; if the predicted quality relation data corresponding to the 1 st video coding parameter to the K-1 st video coding parameter are the first quality relation data and the predicted quality relation data corresponding to the K-1 th video coding parameter are the second quality relation data, determining the K-1 st video coding parameter as the first video coding parameter and the K-th video coding parameter as the second video coding parameter; the number of the video coding parameters is M, M is an integer greater than 1, and K is an integer greater than 1 and less than or equal to M.

In an embodiment, the value of the first video coding parameter is smaller than the value of the second video coding parameter, and the smaller the value of the video coding parameter is, the larger the code rate of the video encoder is, and the higher the coding quality of the video by the video encoder is; the processor 901, when determining a target video coding parameter from the first video coding parameter and the second video coding parameter, is specifically configured to: if the coding set for the video coder is that the quality requirement is higher than the code rate requirement, determining the first video coding parameter as a target video coding parameter; and if the code rate requirement is higher than the quality requirement for the coding setting of the video coder, determining the second video coding parameter as a target video coding parameter.

In one embodiment, the processor 901, when acquiring video feature data of a video to be processed, is specifically configured to: performing downsampling processing on the video to be processed to obtain downsampled video; and encoding the downsampled video by using a feature encoder to obtain video feature data of the video to be processed, wherein the video feature data comprises one or more of video block division information, video frame peak signal-to-noise value, block division coefficient and video macro block information.

In a specific implementation, the processor 901, the communication interface 902, and the memory 903 described in the embodiments of the present application may execute an implementation manner of a video encoding device described in a video encoding method provided in the embodiments of the present application, or may execute an implementation manner described in a video encoding apparatus provided in the embodiments of the present application, which is not described herein again.

Embodiments of the present application also provide a computer-readable storage medium having a computer program stored therein, which when run on a computer, causes the computer to perform the video encoding method according to embodiments of the present application. The specific implementation manner may refer to the foregoing description, and will not be repeated here.

Embodiments of the present application also provide a computer program product comprising a computer program or computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer program or computer instructions from the computer readable storage medium, the processor executing the computer program or computer instructions, causing the computer device to perform a video encoding method as described in embodiments of the present application. The specific implementation manner may refer to the foregoing description, and will not be repeated here.

It should be noted that, for simplicity of description, the foregoing method embodiments are all expressed as a series of action combinations, but it should be understood by those skilled in the art that the present application is not limited by the described order of action, as some steps may take other order or be performed simultaneously according to the present application. Further, those skilled in the art will also appreciate that the embodiments described in the specification are all preferred embodiments, and that the acts and modules referred to are not necessarily required in the present application.

Those of ordinary skill in the art will appreciate that all or part of the steps in the various methods of the above embodiments may be implemented by a program to instruct related hardware, the program may be stored in a computer readable storage medium, and the storage medium may include: flash disk, read-Only Memory (ROM), random-access Memory (Random Access Memory, RAM), magnetic or optical disk, and the like.

The foregoing disclosure is only illustrative of some of the embodiments of the present application and is not, of course, to be construed as limiting the scope of the appended claims, and therefore, all changes that come within the meaning and range of equivalency of the claims are therefore intended to be embraced therein.

Claims

1. A method of video encoding, the method comprising:

2. The method of claim 1, wherein the predictive quality relationship data is obtained by predictive processing the video feature data and the video coding parameters using a predictive model obtained by training an initial model using a sample data set comprising: the video encoder comprises video characteristic data of a sample video, the plurality of video encoding parameters and sample quality relation data corresponding to each video encoding parameter, wherein the sample quality relation data is used for indicating a numerical relation between a sample encoding quality value and the reference encoding quality value, and the sample encoding quality value is used for indicating the encoding quality of the video encoder for the sample video based on the video encoding parameters.

3. The method according to claim 2, wherein the method further comprises:

acquiring the sample video and the plurality of video coding parameters, and coding the sample video by using a feature coder to obtain video feature data of the sample video;

the video encoder is utilized to encode the sample video according to a reference video encoding parameter to obtain a sample video encoding result, a sample encoding quality value of the sample video encoding result is determined, sample quality relation data corresponding to the reference video encoding parameter is determined according to the reference encoding quality value and the sample encoding quality value, and the reference video encoding parameter is any one video encoding parameter of the plurality of video encoding parameters;

after obtaining sample quality relation data corresponding to each video coding parameter in the plurality of video coding parameters, performing model training on an initial model by utilizing video characteristic data of the sample video, the plurality of video coding parameters and the sample quality relation data corresponding to each video coding parameter to obtain the prediction model.

4. The method of claim 3, wherein said determining sample quality relationship data corresponding to the reference video coding parameter from the reference coding quality value and the sample coding quality value comprises:

Comparing the reference encoding quality value with the sample encoding quality value;

if the comparison result indicates that the sample coding quality value is larger than the reference coding quality value, determining sample quality relation data corresponding to the reference video coding parameter as first quality relation data;

and if the comparison result indicates that the sample coding quality value is smaller than or equal to the reference coding quality value, determining that the sample quality relation data corresponding to the reference video coding parameter is second quality relation data.

5. The method of any of claims 1-4, wherein the predicted quality relationship data is first quality relationship data indicating that the predicted encoded quality data is greater than the reference quality data or second quality relationship data indicating that the predicted encoded quality data is less than or equal to the reference quality data;

the determining a target video coding parameter from the first video coding parameter and the second video coding parameter comprises:

inputting the video coding parameters and the video characteristic data of the video to be processed into the prediction model in sequence according to the ordering sequence of the video coding parameters indicated by the video coding parameter sequence to perform prediction processing, so as to obtain prediction quality relation data corresponding to the 1 st video coding parameter to the K th video coding parameter respectively;

If the predicted quality relation data corresponding to the 1 st video coding parameter to the K-1 st video coding parameter are the first quality relation data and the predicted quality relation data corresponding to the K-1 th video coding parameter are the second quality relation data, determining the K-1 st video coding parameter as the first video coding parameter and the K-th video coding parameter as the second video coding parameter;

the number of the video coding parameters is M, M is an integer greater than 1, and K is an integer greater than 1 and less than or equal to M.

6. The method of claim 5, wherein the value of the first video encoding parameter is smaller than the value of the second video encoding parameter, and the smaller the value of the video encoding parameter, the greater the code rate of the video encoder and the higher the encoding quality of the video by the video encoder;

if the coding set for the video coder is that the quality requirement is higher than the code rate requirement, determining the first video coding parameter as a target video coding parameter;

And if the code rate requirement is higher than the quality requirement for the coding setting of the video coder, determining the second video coding parameter as a target video coding parameter.

7. The method according to any one of claims 1-4, wherein the target coding quality indicator is any one of a peak signal-to-noise ratio quality indicator, a structural similarity quality indicator, and a mixing quality indicator.

8. The method according to any one of claims 1-4, wherein the acquiring video feature data of the video to be processed comprises:

performing downsampling processing on the video to be processed to obtain downsampled video;

and encoding the downsampled video by using a feature encoder to obtain video feature data of the video to be processed, wherein the video feature data comprises one or more of video block division information, video frame peak signal-to-noise value, block division coefficient and video macro block information.

9. A video encoding device, the device comprising:

10. A computer device, comprising: the video coding method according to any one of claims 1-8, comprising a processor, a communication interface and a memory, said processor, said communication interface and said memory being interconnected, wherein said memory stores executable program code, said processor being adapted to invoke said executable program code.

11. A computer readable storage medium having stored therein computer instructions which, when run on a computer, cause the computer to implement the video encoding method of any of claims 1-8.

12. A computer program product, characterized in that the computer program product comprises a computer program or computer instructions which, when executed by a processor, implements the video encoding method according to any of claims 1-8.