WO2022111631A1

WO2022111631A1 - Video transmission method, server, terminal, and video transmission system

Info

Publication number: WO2022111631A1
Application number: PCT/CN2021/133497
Authority: WO
Inventors: 鲁威; 王祺; 孙龙; 林焕; 胡康康
Original assignee: 华为技术有限公司
Priority date: 2020-11-30
Filing date: 2021-11-26
Publication date: 2022-06-02
Also published as: CN114584805A

Abstract

Disclosed by embodiments of the present application is a video transmission method, applicable to the field of video transmission, used for reducing transmission bandwidth while ensuring video quality at a receiving terminal. The method of the embodiments of the present application comprises: a server obtaining a super-resolution model obtained by training a target video at a first resolution and said target video at a second resolution; sending said super-resolution model and the target video at a third resolution to a terminal; the terminal performing a super-resolution reconstruction of the target video at the third resolution on the basis of the super-resolution model; in comparison with super-resolution reconstruction by means of a generic super-resolution model, the video image quality is improved.

Description

Video transmission method, server, terminal and video transmission system

This application claims the priority of the Chinese patent application with the application number 202011373386.2 and the invention title "Video Transmission Method, Server, Terminal and Video Transmission System", which was submitted to the State Intellectual Property Office of China on November 30, 2020, the entire contents of which are obtained through Reference is incorporated in this application.

technical field

The present application relates to the technical field of video transmission, and in particular, to a video transmission method, server, terminal and video transmission system.

Background technique

With the development of electronic imaging technology and the continuous improvement of screen resolution, the traffic consumption of image resources such as images and videos is increasing, which brings very high bandwidth costs. Reducing the bandwidth cost of video has become a top priority for current video platforms. Compressed transmission using a smaller resolution is a commonly used bandwidth reduction technology, which can reduce the image transmission bandwidth from the video platform to the user terminal.

Since a video with a smaller resolution cannot meet the user's high-quality viewing requirements, generally, the user terminal side (hereinafter referred to as the terminal side) will restore the resolution of the received video before displaying it. Super-resolution (hereinafter referred to as: super-resolution) technology is an important computer vision and image processing method to restore high-resolution pictures from low-resolution pictures. Super-resolution technology based on deep learning is developing rapidly. In the existing method, a super-resolution model is preset on the device side to perform super-resolution reconstruction on low-quality video to obtain high-resolution and high-quality video.

Since different videos have different content, style and image quality, the preset model cannot cover all video scenes, resulting in a large gap between the image quality of the restored super-resolution video and the original high-resolution video. Even in some scenes the quality of the video is degraded.

SUMMARY OF THE INVENTION

The embodiment of the present application provides a video transmission method, which is applied to the field of video transmission, and is used for reducing the transmission bandwidth while ensuring the video quality of the receiving end.

A first aspect of the embodiments of the present application provides a video transmission method, including: a server acquiring a super-score model of a target video, where the super-score model is based on the target video at a first resolution and the target video at a second resolution The target video is acquired through training, the second resolution is smaller than the first resolution, and the magnification of the super-resolution model is the difference between the pixels of the first resolution and the second resolution in the length direction or the width direction. ratio; the server sends the target video of the third resolution and the super-resolution model to the terminal, and the super-resolution model is used to super-resolution the target video of the third resolution at the magnification ratio. Resolution reconstruction to obtain the target video at a fourth resolution.

In the video transmission method provided by the embodiment of the present application, the server sends the target video of the third resolution and the super-resolution model to the terminal, and the super-resolution model is based on the target video of different resolutions (the first resolution and the second resolution). For training and acquisition, the terminal may super-score the target video of the third resolution according to the magnification of the super-resolution model to obtain the target video of the fourth resolution. Since the super-score model sent by the server to the terminal in the video transmission method in the embodiment of the present application is obtained based on target video training, compared with the prior art, the terminal performs over-score for various target videos according to a single preset super-score model. By using the super-score model of the method, a better super-score effect can be obtained when the target video is restored by the over-score, that is, the image quality of the restored target video of the fourth resolution is higher.

In a possible implementation manner of the first aspect, the target video is a single video or a set of multiple videos of the same type. Specifically, the target video can be an episode of a series or an independent video such as a news video, a movie, etc., that is, the super-score model is independently trained for a single video; or, the target video can be multiple videos of the same type, For example, a series in a series or a season broadcast, or a season, or the same series of videos of a personal blogger, etc., that is, multiple videos related in content, characters or performances as target videos, these videos usually have similar The style and image quality of these videos are independently trained for the super-scoring model, and the video quality obtained by the super-scoring recovery is better.

In a possible implementation manner of the first aspect, the third resolution is equal to the second resolution; the fourth resolution is equal to the first resolution.

In the video transmission method provided by the embodiment of the present application, in a possible implementation manner, the target video sent by the server to the terminal is the target video of lower resolution (that is, the second resolution) used for training the super-score model. The super-resolution model is used to restore the target video of the second resolution to the target video of the first resolution. In this implementation scenario, the effect of super-resolution restoration is better, and the image quality of the restored target video is better.

In a possible implementation manner of the first aspect, the obtaining, by the server, the super-score model of the target video includes: the server inputting the target video of the first resolution and the target video of the second resolution into a volume The neural network model is accumulated to obtain the over-fitted super-score model.

In the video transmission method provided by the embodiment of the present application, the server can input target videos of different resolutions into the convolutional neural network model for training, and obtain an over-fitted super-score model. Although the over-fitted super-score model is not applicable to other non- Overscore of the target video, but works extremely well in the overscore of the training dataset i.e. the target video.

In a possible implementation manner of the first aspect, the obtaining, by the server, the super-score model of the target video includes: the server determining the video frames in the video of the first resolution and the video of the second resolution by the server Perform data cleaning on the video frames in the video frame to obtain the target video of the first resolution and the target video of the second resolution; the server performs data cleaning on the target video of the first resolution and the target video of the second resolution The target video of the second resolution is trained to obtain the super-score model.

In the video transmission method provided by the embodiment of the present application, the server may also perform data cleaning on the video frames of the target video before training the super-score model. The data cleaning includes removing video frames such as blurred frames and low-information frames. Using videos for model training can improve model performance.

In a possible implementation manner of the first aspect, the sending, by the server, the target video of the third resolution and the super-resolution model to the terminal specifically includes: the server sending the super-resolution model to the terminal The server sends a data packet to the terminal, and the data packet includes the weight parameter of the super-score model and the target video of the third resolution.

In the video transmission method provided by the embodiment of the present application, there are multiple ways for the server to send the target video and the super-resolution model to the terminal, which can be sent separately or simultaneously. Specifically, when the server sends the super-score model to the terminal, the super-score model can be sent in two parts: the structure of the super-score model and the weight parameters of the super-score model. In a possible implementation, the server sends the super-score model to the terminal in advance. The structure of the model, and then send the weight parameters of the super-score model and the target video to the terminal. Further, when the server sends to the terminal, the target video will be compressed into different data packets for transmission. Perform super-score on the target video. In this implementation, the super-score models of different target videos can have the same model structure. When the super-score model of each target video is sent to the terminal, only the corresponding weight parameters need to be sent. That is, it is possible to reduce the amount of data transmission by this.

A second aspect of the embodiments of the present application provides a video transmission method, including: a terminal receiving a super-resolution model of a target video sent by a server, where the super-resolution model is based on the target video of a first resolution and a second resolution of the target video The target video is obtained through training, the second resolution is smaller than the first resolution, and the magnification of the super-resolution model is the first resolution and the second resolution in the length direction or in the width direction. and the target video of the third resolution; the terminal performs super-resolution reconstruction on the target video of the third resolution at the magnification according to the super-resolution model, and obtains the target video at the fourth resolution.

In the video transmission method provided by the embodiment of the present application, the terminal receives the target video and the super-resolution model of the third resolution sent by the server, because the super-resolution model is based on the target video of different resolutions (the first resolution and the second resolution). For video training acquisition, compared with the prior art in which the terminal performs over-score for various target videos according to a single over-score model, the super-score model of this method can obtain a better over-score effect when performing over-score recovery on the target video. , that is, the image quality of the restored target video of the fourth resolution is higher.

In a possible implementation manner of the second aspect, the target video is a single video or a set of multiple videos of the same type.

In the video transmission method provided by the embodiments of the present application, the target video has multiple possible forms, which increases the flexibility of solution implementation.

In a possible implementation manner of the second aspect, the third resolution is equal to the second resolution; the fourth resolution is equal to the first resolution.

In a typical implementation of the video transmission method provided by the embodiment of the present application, the target video of the third resolution obtained by the terminal is the target of the lower resolution (ie the second resolution) used for training the super-score model. Video, the super-resolution model is used to restore the target video of the second resolution to the target video of the first resolution. In this implementation scenario, the effect of super-resolution restoration is better, and the image quality of the restored target video is better. it is good.

In a possible implementation manner of the second aspect, the super-score model includes: an overfitting obtained by inputting the target video of the first resolution and the target video of the second resolution into a convolutional neural network model The combined superscore model.

In the video transmission method provided by the embodiment of the present application, although the over-fitted super-score model is not suitable for the over-score of other non-target videos, it has an excellent effect in the over-score of the training data set, that is, the target video.

In a possible implementation manner of the second aspect, the super-score model includes: a super-score model obtained by training the target video of the first resolution and the target video of the second resolution , the target video of the first resolution and the target video of the second resolution are composed of video frames in the video of the first resolution and video frames of the video of the second resolution Data cleaning is performed.

In the video transmission method provided by the embodiment of the present application, the data set used for model training is obtained by data cleaning of the video frames of the target video, and the data cleaning includes removing video frames such as blurred frames and low-information frames, thereby improving the super-resolution model. model effect.

In a possible implementation manner of the second aspect, the terminal receiving the super-resolution model of the target video sent by the server, and the target video of the third resolution includes: receiving, by the terminal, the target video sent by the server. The structure of the super-resolution model; the terminal receives a data packet sent by the server, where the data packet includes the weight parameter of the super-resolution model and the target video of the third resolution.

In the video transmission method provided by the embodiment of the present application, there are multiple ways for the terminal to obtain the target video and the super-score model, which may be obtained separately or at the same time. Specifically, the terminal can receive the structure of the super-resolution model and the weight parameters of the super-resolution model respectively. In a possible implementation manner, the terminal first receives the structure of the super-resolution model sent by the server, and then obtains the weight parameters of the super-resolution model and target video. Further, when the server sends to the terminal, the target video will be compressed into different data packets for transmission. Perform super-score on the target video. In this implementation, the super-score models of different target videos can have the same model structure, and the super-score models corresponding to different target videos can be obtained only by receiving the corresponding weight parameters. Thus, the amount of data transfer can be reduced.

A third aspect of the embodiments of the present application provides a server, including: an acquisition module, configured to acquire a super-score model of a target video, where the super-score model is based on the target video at a first resolution and all images at a second resolution. The target video is acquired by training, the second resolution is smaller than the first resolution, and the magnification of the super-resolution model is the ratio of the pixels of the first resolution and the second resolution in the length direction or the width of the pixels. The ratio of pixels in the direction; the sending module is used to send the target video of the third resolution and the super-division model to the terminal, and the super-division model is used to measure the third resolution at the magnification ratio. Perform super-resolution reconstruction on the target video to obtain the target video of the fourth resolution.

In a possible implementation manner of the third aspect, the target video is a single video or a set of multiple videos of the same type.

In a possible implementation manner of the third aspect, the third resolution is equal to the second resolution; the fourth resolution is equal to the first resolution.

In a possible implementation manner of the third aspect, the obtaining module is specifically configured to: input the target video of the first resolution and the target video of the second resolution into a convolutional neural network model, and obtain the The fitted super-score model.

In a possible implementation manner of the third aspect, the acquiring module is specifically configured to: perform data cleaning on the video frames in the video of the first resolution and the video frames in the video of the second resolution , obtain the target video of the first resolution and the target video of the second resolution; compare the target video of the first resolution and the target video of the second resolution Perform training to obtain the super-score model.

In a possible implementation manner of the third aspect, the sending module is specifically configured to: send the structure of the superdivision model to the terminal; send a data packet to the terminal, where the data packet includes the superdivision model The weight parameter of the sub-model and the target video of the third resolution.

A fourth aspect of an embodiment of the present application provides a terminal, including: a receiving module configured to receive a super-resolution model of a target video sent by a server, where the super-resolution model is based on the target video of a first resolution and a second resolution The target video of the first resolution is obtained by training, the second resolution is smaller than the first resolution, and the magnification of the super-resolution model is the first resolution and the second resolution in the length direction or in the width direction. and the target video of the third resolution; a processing module, configured to perform super-resolution on the target video of the third resolution with the magnification according to the super-resolution model Reconstruct, and obtain the target video of the fourth resolution.

In a possible implementation manner of the fourth aspect, the target video is a single video or a set of multiple videos of the same type.

In a possible implementation manner of the fourth aspect, the third resolution is equal to the second resolution; the fourth resolution is equal to the first resolution.

In a possible implementation manner of the fourth aspect, the super-score model includes: an overfitting obtained by inputting the target video of the first resolution and the target video of the second resolution into a convolutional neural network model The combined superscore model.

In a possible implementation manner of the fourth aspect, the super-score model includes: a super-score model obtained by training the target video of the first resolution and the target video of the second resolution , the target video of the first resolution and the target video of the second resolution are composed of video frames in the video of the first resolution and video frames of the video of the second resolution Data cleaning is performed.

In a possible implementation manner of the fourth aspect, the receiving module is specifically configured to: receive the structure of the super-resolution model sent by the server; receive a data packet sent by the server, where the data packet includes the weight parameters of the super-resolution model and the target video of the third resolution.

A fifth aspect of an embodiment of the present application provides a server, including: one or more processors and a memory; wherein, computer-readable instructions are stored in the memory; the one or more processors read the computer The instructions are readable to cause the terminal to implement the method according to any one of the above-mentioned first aspect and various possible implementation manners of the first aspect.

A sixth aspect of an embodiment of the present application provides a terminal, including: one or more processors and a memory; wherein, the memory stores computer-readable instructions; the one or more processors read the computer The instructions are readable to cause the terminal to implement the method according to any one of the above-mentioned second aspect and various possible implementation manners of the second aspect.

A seventh aspect of an embodiment of the present application provides a video transmission system, including: the server described in any one of the foregoing first aspect and various possible implementation manners of the first aspect, and the foregoing second aspect and the second The terminal described in any one of various possible implementation manners of the aspect.

An eighth aspect of the embodiments of the present application provides a computer program product containing instructions, characterized in that, when it runs on a computer, the computer is caused to execute the first aspect, the second aspect and various possible implementations described above. The method of any one of the methods.

A ninth aspect of an embodiment of the present application provides a computer-readable storage medium, including instructions, characterized in that, when the instructions are executed on a computer, the computer is made to execute the first aspect, the second aspect, and various possibilities described above. The method described in any one of the implementation manners.

A tenth aspect of the embodiments of the present application provides a chip, including a processor. The processor is configured to read and execute the computer program stored in the memory to perform the method in any possible implementation manner of any of the above aspects. Optionally, the chip includes a memory, and the memory and the processor are connected to the memory through a circuit or a wire. Further optionally, the chip further includes a communication interface, and the processor is connected to the communication interface. The communication interface is used for receiving data and/or information to be processed, the processor obtains the data and/or information from the communication interface, processes the data and/or information, and outputs the processing result through the communication interface. The communication interface may be an input-output interface.

Wherein, the technical effect brought by any one of the third aspect, the fourth aspect, the fifth aspect, the sixth aspect, the seventh aspect, the eighth aspect, the ninth aspect or the tenth aspect can refer to the first aspect The technical effects brought about by the corresponding implementations in the above will not be repeated here.

As can be seen from the above technical solutions, the embodiments of the present application have the following advantages:

The server sends the target video of the third resolution and the super-resolution model of the target video to the terminal. The super-resolution model consists of the target video of the first resolution (higher resolution) and the target video of the second resolution (lower resolution). The target video is trained, so when it is used for the super-score of the target video, the recovered video is closer to the original high-resolution video than the general super-score model, and the image quality is improved.

Description of drawings

1 is a system architecture diagram of a video transmission method;

2 is a system architecture diagram of a video transmission method in an embodiment of the application;

3a is a schematic diagram of an embodiment of a video transmission method implemented on a server side in an embodiment of the present application;

3b is a schematic diagram of an embodiment of a video transmission method implemented by a terminal side in an embodiment of the present application;

3c is a schematic flowchart of implementing a video transmission method in an embodiment of the present application;

4a is an interaction diagram of a video transmission method in an embodiment of the present application;

FIG. 4b is a schematic diagram of another embodiment of the video transmission method in the embodiment of the present application;

5 is a schematic diagram of a data cleaning method in an embodiment of the present application;

FIG. 6 is a schematic diagram of super-resolution model training in an embodiment of the present application;

7 is a schematic structural diagram of a data block of a superdivision model in a data packet in an embodiment of the present application;

8 is a schematic structural diagram of a header of a data packet in an embodiment of the present application;

9 is a schematic structural diagram of a data packet in an embodiment of the present application;

10 is a schematic diagram of a terminal decoding a data packet in an embodiment of the present application;

11 is a schematic diagram of a terminal acquiring a super-resolution image in an embodiment of the application;

FIG. 12 is a schematic diagram of an embodiment of a server in an embodiment of the present application;

FIG. 13 is a schematic diagram of an embodiment of a terminal in an embodiment of the present application;

FIG. 14 is a schematic diagram of another embodiment of the server in the embodiment of the present application;

FIG. 15 is a schematic diagram of another embodiment of the terminal in the embodiment of the present application.

Detailed ways

For ease of understanding, some technical terms involved in the embodiments of the present application are briefly introduced below:

overfitting

Overfitting means that in the process of model parameter fitting, since the training data contains sampling error, the complex model takes the sampling error into account during training and performs a good fit. The specific performance is that the model is in the training set. The effect is good on the test set, but the effect on the test set is poor, and the model generalization ability is weak. The two main reasons for overfitting are a small amount of training data, or an overly complex model.

Usually, the model is generally used to predict unknown data (data not in the training set), and the effect of the overfitting model is poor. Therefore, during the training process, the data volume of the training set is enlarged and the appropriate model is used to avoid overfitting. fit so that the model fits the true rule enough without having to fit too much sampling error. Generally speaking, a model with strong generalization ability is a good model in the usual sense. In model training, it is necessary to avoid overfitting and pursue generalization to make the model applicable to a wider range.

However, the video transmission method provided by the embodiment of the present application cleverly utilizes the characteristics of the overfitting model, and trains the super-score model with a small amount of data (for example, a single video or multiple videos of the same type), and obtains the over-fitting super-score model. The over-fitting super-score model is specially used for the over-score of the training data. Although it is not suitable for the over-score of the non-training set, it can obtain a better over-score effect for the over-score of the training data, that is, recovery high-resolution video with higher quality.

The resolution of the image: refers to the amount of information stored in the image, which is how many pixels there are in each inch of the image, usually expressed as "the number of horizontal pixels × the number of vertical pixels". The image resolution is 640*480, which means the number of horizontal pixels is 640 and the number of vertical pixels is 480. P stands for progressive scanning. Commonly used image resolutions of 360P refer to 480×360; 720P refers to 1280×720; 1080p refers to 1920×1080; 4K refers to 3840×2160.

Understandably, the higher the resolution of the image, the more data it contains, and the richer the details can be, but at the same time it requires more computer storage resources.

The magnification of the super-resolution model refers to the ratio of the number of pixels in the length direction of the image after super-resolution and the image before super-resolution, or the ratio of the number of pixels in the width direction of the two images.

Super resolution (super resolution, SR)

Super-resolution for short, is to improve the resolution of the original image by means of hardware or software, and obtain a high resolution (high resolution, HR) image or image sequence through a low resolution (low resolution, LR) image or image sequence. The process is super-resolution reconstruction.

Low-resolution images (low resolution, LR) and high-resolution images (high resolution, HR) correspond to each other and are used to input deep learning models, such as convolutional network models for training, HR images are the original images corresponding to the LR images , the LR image is super-reconstructed by the super-resolution model to obtain a super-resolution (SR) image, and the process of obtaining an SR image from the LR image can be called super-resolution reconstruction or super-resolution restoration.

Deep Learning: Deep learning is a method in machine learning based on representational learning of data. An observation (eg, an image) can be represented in a variety of ways, such as a vector of intensity values for each pixel, or more abstractly as a series of edges, regions of a particular shape, etc. Instead, it is easier to learn tasks from examples (e.g., face recognition or facial expression recognition) using some specific representation. The benefit of deep learning is to replace handcrafted features with efficient algorithms for unsupervised or semi-supervised feature learning and hierarchical feature extraction.

Convolutional Neural Network: Do the inner product of the image (different data window data) and the filter matrix (a set of fixed weights: because the multiple weights of each neuron are fixed, it can be regarded as a constant filter) ( The operation of multiplying and summing element by element is the so-called convolution operation, which is also the source of the name of the convolutional neural network.

Data cleaning: Data cleaning is the process of re-examining and verifying data to remove duplicate information, correct existing errors, and provide data consistency. That is, to remove or repair the "dirty samples" that affect subsequent model training. In this application, data cleaning includes removing blurred and less informative video frames.

The embodiments of the present application will be described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. Those of ordinary skill in the art know that, with the development of technology and the emergence of new scenarios, the technical solutions provided in the embodiments of the present application are also applicable to similar technical problems.

The terms "first", "second" and the like in the description and claims of the present application and the above drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It is to be understood that data so used may be interchanged under appropriate circumstances so that the embodiments described herein can be practiced in sequences other than those illustrated or described herein. Furthermore, the terms "comprising" and "having", and any variations thereof, are intended to cover non-exclusive inclusion, for example, a process, method, system, product or device comprising a series of steps or modules is not necessarily limited to those expressly listed Rather, those steps or modules may include other steps or modules not expressly listed or inherent to the process, method, product or apparatus. The naming or numbering of the steps in this application does not mean that the steps in the method flow must be executed in the time/logical sequence indicated by the naming or numbering, and the named or numbered process steps can be implemented according to the The technical purpose is to change the execution order, as long as the same or similar technical effects can be achieved.

Please refer to Figure 1, which is an example of a classic terminal video application scenario, including five parts: input video (1001), video encoding (1002), network transmission (1003), video decoding (1004), and output video (1005) at the transmitter . The input video (1001) of the sender is the input part of the scene, and is the video saved in the cloud server. The video coding (1002) is to compress the input video images and reduce the redundant information of the video to facilitate network transmission. The network transmission (1003) stage is to transmit the video from the sender to the receiver. The video decoding (1004) is used to decode the encoded video transmitted by the network and restore the video to the state before encoding. The output video (1005) is the video output synthesized by the decoder.

The cloud sends high-resolution video (such as 4k resolution video) to the client through the network. The bandwidth cost of the video platform is relatively high. At the same time, when the user's network status is poor, the video playback on the client side is prone to freeze problems.

Video platforms usually use smaller resolutions for compressed transmission to reduce the image transmission bandwidth to the user terminal. The image received by the user terminal side needs to be restored to the resolution before being displayed. Due to the loss of high-frequency information during compression and transmission, the traditional upsampling algorithm is used to interpolate the low-resolution video to obtain the high-resolution video, which is prone to blurring or compression noise, and the image quality is poor.

Super-resolution technology is an important computer vision and image processing method to restore high-resolution images from low-resolution images. Super-resolution technology based on deep learning is developing rapidly. In the existing methods, a super-resolution model is preset on the terminal side to perform super-resolution reconstruction on low-resolution videos to obtain high-resolution and high-quality videos.

However, different videos have different contents, styles and image quality. Using a single preset model cannot cover all video scenes, resulting in a large gap between the restored image quality and the original high-resolution image quality, and even in some scenes The quality of the video will be degraded.

In this embodiment of the present application, the target video is a single video or a collection of multiple videos of the same type. Specifically, the target video may be an episode of a series, or an independent video such as a news video or a movie. In this embodiment of the present application, the super-score model can be independently trained for a single video. Alternatively, the target video may also be multiple videos of the same type, such as a series or a season of a series, a season broadcast, or a video of the same series of a personal blogger, that is, related in content, characters or performances. multiple videos. Since these videos usually have similar style and quality, training the super-score model on these videos independently, the quality of the videos recovered by super-score is better.

With the video transmission method provided by the embodiment of the present application, the server and the terminal can reduce the transmission bandwidth and improve the image quality of the video restored on the terminal side by means of device-cloud collaboration.

(1) Cloud side: that is, the server side, which uses the high-resolution video of a set of videos and the video frames corresponding to the low-resolution video to generate a training data set. Overfitting the overscore model. The low-resolution video of the set of videos and the over-fitted super-resolution neural network model corresponding to the set are sent to the end-side.

(2) End-to-end: Using the over-fitting super-resolution neural network model delivered by the cloud side, the set of low-resolution videos is reconstructed and restored to high-resolution and high-quality videos through super-resolution technology.

As shown in FIG. 2 , the embodiment of the present application uses a deep learning-based video transmission solution through device-cloud collaboration.

On the server side of the cloud, the video of the first resolution video (2001) and its corresponding second resolution video (2002) will be used as the data set of the super-score model training module (2003) of the video, wherein the second resolution video rate is less than the first resolution. The super-score model training module (2003) uses the dataset to train the video-specific super-score model. Different videos will have their corresponding over-score models. Optionally, the second resolution video (2002) and its corresponding over-score model. The sub-models are encoded together in the video and model encoding module (2004) as data packets for transmission over the network (2005).

On the end side, the video and model decoding module (2006) decodes the received data packets to obtain the second resolution video (2007) of the set and its corresponding super-resolution model (2008). (2009) Output the first resolution video (2010).

Referring to FIG. 3a and FIG. 3b below, the video transmission method in the embodiment of the present application is introduced from the server side and the terminal side, respectively.

Referring to FIG. 3 a , the video transmission method in this embodiment of the present application may include steps 3101 to 3103 .

3101. The server obtains the target video of the first resolution and the target video of the second resolution;

The server obtains two different resolution versions of the target video, including the target video of the first resolution and the target video of the second resolution, where the second resolution (or low resolution in this embodiment of the present application) is smaller than the first resolution rate (or referred to as high resolution in the embodiments of this application). In the embodiments of the present application, the target video is taken as an example of a single episode for introduction. Usually video platforms store multiple resolution versions of the same video. Common resolutions include: 360P, 720P, 1080P or 4K. As shown in Fig. 3c, the server obtains the video of the target video (3001) of the first resolution of a single episode (for example: the video of the first episode of a certain series with the resolution of 1080P) and the corresponding target video of the second resolution (3001). 3002) (for example: the video of the first episode of the series with a resolution of 4K) as the data set of the super-score model (3003). The video frame of the target video of the second resolution is used as the data (data) of the dataset, and the corresponding video frame of the target video of the first resolution is used as the label of the dataset. The quality of the data set is very important for the training effect of the super-resolution model. Optionally, the data set used for training the super-resolution model can be a data set containing all video frames of the target video without data cleaning, or a data set that has been Data set with some video frames deleted after data cleaning. Optionally, data cleaning (3004) is performed on the dataset (3003), the data cleaning includes removing blurred frames, filtering low-frequency information, etc., to obtain a cleaned data dataset (3005) and a label dataset (3006).

3102. The server performs training according to the target video of the first resolution and the target video of the second resolution to obtain a super-score model;

The server performs overfitting training on the super-score model according to the target video of the first resolution and the target video of the second resolution, and obtains the over-fitted super-score model. The magnification of the super-resolution model is the ratio of pixels in the width direction or the ratio of pixels in the length direction between the first resolution and the second resolution. Optionally, a single video corresponds to one super-score model. In other possible implementations, a set of multiple videos of the same type corresponds to a super-score model. Optionally, the super-score model corresponding to each video in the set of multiple videos of the same type has the same model structure and different model parameters.

As shown in Figure 3c, the server builds a super-resolution neural network model, and the super-resolution model is trained (3007) based on the data dataset (3005) and the label dataset (3006) generated in step 3101. Optionally, a corresponding super-score model is trained for each episode of videos. This step will result in a superscoring model (3008) for that single episode video. Optionally, the server inputs the data set into the convolutional neural network model for training, and obtains an overfitted super-score model.

3103. The server sends the super-resolution model and the target video of the third resolution to the terminal;

The server sends the super-resolution model obtained in step 3102 and the target video of the third resolution to the terminal. It should be noted that the third resolution may be the same as or different from the first resolution and the second resolution, and the specific value is not limited. For example, the third resolution may be 360P, 720P or 1080P, and the third It can be flexibly selected according to the network status. It can be understood that the target video of the third resolution contains all the video frames of the original video, and no data cleaning is required.

Specifically, as shown in Figure 3c, the encoding module (3010) encodes the super-resolution model (3008) and the target video (3009) of the third resolution into data packets and sends them to the network (3011) for transmission.

Optionally, the superresolution model will be transmitted in binary file format. Optionally, the model structure of the super-resolution model is pre-updated to the terminal side. When there is a video transmission requirement, the server only needs to send the weights parameters of the super-resolution model corresponding to the video to the terminal, without sending the model structure again. Optionally, in order to ensure that the model accepted by the terminal has data consistency, the tail of the binary file contains a hash value generated by the model.

The super-resolution model sent to the terminal is used to super-score the target video of the third resolution, for example, to restore a video with a resolution of 1080P to a video with a resolution of 4K.

In the data transmission method provided by the embodiment of the present application, since the target video of the third resolution and the super-resolution model are sent to the terminal, the network bandwidth occupied by directly sending the video of the fourth resolution is lower. The model is obtained by training according to the target video, and the effect of super-score recovery is better for the target video, and the video quality obtained by over-score recovery is higher.

Referring to FIG. 3b below, the steps performed on the terminal side by the video transmission method provided by the embodiment of the present application are described.

3201. The terminal receives the super-resolution model and the target video of the third resolution sent by the server;

The terminal receives the super-resolution model and the target video of the third resolution sent by the cloud-side server. As shown in Figure 3c, specifically, the terminal receives the data packets transmitted by the server through the network transmission, and decodes the data packets through the decoding module (3012) to obtain the super-division model (3014) and the target video of the third resolution (3013).

The super-score model is obtained by the server performing overfitting training according to the target video of the first resolution and the target video of the second resolution. The magnification of the super-resolution model is the ratio of pixels in the width direction or the ratio of pixels in the length direction between the first resolution and the second resolution. The super-resolution model is used to perform resolution magnification for the target video of the third resolution according to the magnification ratio. Exemplarily, if the first resolution is 4K and the second resolution is 1080P, the magnification is twice, that is, the pixels of the image with the resolution of 1080P are enlarged twice in the length direction and the width direction.

Optionally, the decoding module (3012) uses the same hash method as that of the cloud server to perform hash processing on the super-score model (3014), obtains a hash value, and uses the hash value to perform consistency check, if the same, it is considered that The model is reliable; if it is different, an error message is sent to the cloud, requiring the server to resend the correct packet.

3202. The terminal performs super-resolution processing on the target video of the third resolution according to the super-resolution model, and obtains the target video of the fourth resolution;

As shown in Figure 3c, the terminal uses the over-fit super-score model (3014) to perform video super-score processing (3015) on the target video (3013) of the third resolution, and generates the target video of the fourth resolution (3016) and sent to the terminal display device. The target video of the fourth resolution is obtained by enlarging the target video of the third resolution by the magnification of the super-resolution model. Exemplarily, if the third resolution is 1080P and the magnification of the super-resolution model is 2 times, the resolution of the target video obtained by the super-resolution processing is 4K.

The network bandwidth occupied by the terminal receiving the target video of the third resolution is relatively low. In addition, since the terminal performs resolution amplification according to the overfitting super-resolution model corresponding to the target video, the obtained image quality of the target video of the fourth resolution is higher.

The low-resolution video received by the terminal and the network bandwidth occupied by the super-score model are low. In addition, since the over-score model is obtained by overfitting and training according to the target video, the image quality obtained by the over-score restoration of the target video is higher.

The following describes the video transmission method provided by the embodiment of the present application from the perspective of device-cloud collaboration, please refer to FIG. 4a.

4101. The server obtains the data set;

The server needs to fetch the dataset before training the superscore model. Optionally, if the target video is any set of videos in the series of videos, then the video frame of the target video of the second resolution corresponding to the set video and the video frame of the target video of the corresponding first resolution are extracted as the set. A dataset of super-score models for video. The video frame of the low-resolution video is used as the data of the dataset, and the video frame of the corresponding high-resolution video is used as the label of the dataset.

Optionally, as shown in Figure 4b, in this step, the cloud server invokes a third-party video encoding and decoding service (4002) to decode the stored video resources (4001), such as Huawei Video, iQiyi Video or Youku Video, etc. to obtain a video. frame. In this embodiment, only a single video is used as the target video for introduction. Exemplarily, for the 4K resource of the first episode of the TV series "The Secret XX" as the target video of the first resolution (4003), the 1080P resource of the first episode of the TV series is used as the target video of the second resolution (4004), the same Each of the video frames of the first resolution and the video frames of the second resolution of the set video corresponds to each other.

Use the video decoding service (4002) to decode and obtain the video frame (4004) of the second resolution of the target video, and the corresponding video frame (4003) of the first resolution, and each frame of the decoded high- and low-definition video corresponds to each other, exemplary For example, the f_1 frame in the video frame (4004) of the second resolution corresponds to the f_1 frame of the video frame (4003) of the first resolution.

Since the quality of the dataset is very important for the effect of super-resolution model training, optionally, before training the super-resolution model 4005 in this step, the video frames (4003) of the first resolution and the video frames of the second resolution may be (4004) Data cleaning is performed by removing blurred frames, filtering low-frequency information, etc. The specific process of performing data cleaning on the data set is shown in Figure 5: The target video of the first resolution (5001) is decoded to obtain the first resolution. A video frame, the target video of the second resolution (5002) is decoded to obtain a video frame of the second resolution. For each decoded frame, the resolution detection algorithm is used to detect that each frame in the classified data set is a high-definition frame or a fuzzy frame or a low-information frame, and the data set obtained by eliminating the fuzzy frame and the low-information frame is the data after data cleaning. A data set, in which the video frames of the first resolution of the target video will be used as labels for training the super-score model, and the video frames corresponding to the second resolution will be used as the data (data) of this set of models.

4102. The server trains the super-scoring model;

As shown in FIG. 6 , the video frames (6001) of the first resolution of the set of videos and the corresponding video frames (6002) of the second resolution videos generated in step 4101 are used as the data sets of the super-score model. A video frame of one resolution has a corresponding video frame of a second resolution. Using the data set to perform overfitting training on the super-resolution neural network model, an over-fitting super-resolution model bound to the set of videos is obtained (6003).

4103. The server encodes the data packet;

As shown in Fig. 4b, the encoding module (4006) encodes the super-segmented model (4005) of the set of videos generated in step 4102 and the video frame (4004) of the second resolution to obtain a data packet.

Optionally, the processing of the super-score model by the encoding module, as shown in Figure 7, generates a binary model file model (6002-2) that only contains weights parameters and does not contain the model structure; After processing, get the hash (6002-3) value and put it at the end of the binary model file; the header (6002-1) at the front of the model 6002 is to set some parameters of the module. Three small modules (6002-1, 6002-2, 6002-3) together make up the model (6002) in Figure 7. As shown in FIG. 8 , before the first data packet is transmitted, the extension flag bit X of the data packet header is set to 1, so that the data packet can be extended with custom data. As shown in Figure 9, when the first data packet is sent, the extended flag bit of the header (6001) is turned on, and the model data (6002) is placed in the middle of the header (6001) and the payload (6003), and the payload ( 6003) is the video data. Since the model file is included in the first data packet sent from the cloud to the terminal, the second and subsequent data packets sent by the server to the terminal will set the extended flag position of the packet header to 0 (as shown in Figure 9) , there is no extended data after the header in the data packet, and there is no model file between the header (6001) and the payload (6003), which effectively ensures the efficiency and security of data transmission.

4104. The server sends the low-resolution video and the super-resolution model to the terminal;

As shown in Figure 4b, the server sends a data packet including the super-resolution model (4005) and the video frame (4004) of the second resolution to the terminal through network transmission, and the terminal receives the data packet. It can be understood that, according to the real-time network transmission situation or user requirements, the server can select target videos of different resolutions to send to the terminal. In the example, the video of the second resolution is sent as an example for introduction.

4105. The terminal decodes the data packet;

The terminal receives the data packets transmitted and delivered by the cloud side through the network, and decodes the data packets (4010) through the video frame and super-division model decoding module (4009) to obtain the super-division model (4011) and the target video of the second resolution (4012). In order to avoid the model data being tampered with or lost in the network, the decoding model needs to perform consistency check on the model data. As shown in FIG. 10 , the decoding module obtains the binary model data block (7001) from the data packet, and decodes the data block (7004) to obtain the binary model (7002) and the hash value (7003). Hash (7005) the binary model (7002) using the same hashing method as in the cloud to obtain a Hash value (7006), and perform consistency check on the Hash value (7003) and the Hash value (7006), if they are the same, The model is considered reliable. If it is different, it will send an error message to the cloud and ask the cloud to resend the data packet.

4106. The terminal super-divides low-resolution video and sends it for display;

Referring to Figure 4b, the terminal uses the end-side inference engine (4013) to perform video super-resolution processing on the target video (4012) of the second resolution through the super-resolution model (4011) to obtain the target video of the first resolution (4014), and sends it to the target video (4014). Displayed on the end-side display module (4015). As shown in Figure 11, the super-resolution model can perform super-resolution reconstruction of lower-resolution video frames to obtain higher-resolution video frames.

The video transmission method provided by the embodiments of the present application can use a smaller bandwidth (for example, reduce the bandwidth by half) for transmission on the basis of keeping the video quality of the terminal side unchanged. On the one hand, the method reduces the bandwidth cost of the video platform and increases the market competitiveness of the video platform. For example, a video with a resolution of 4K is about 2 GB (Gigabyte) in size, while a video with a resolution of 1080p transmitted is about 450 MB (Megabyte), and the size of the super-score model is about 10 MB. It can be seen that the video provided by the embodiment of the present application is about 450 MB (Megabyte). The transmission method can reduce the transmission bandwidth. On the other hand, since the super-score model sent by the server to the terminal is an over-fitting model obtained by training based on the target video, the effect of super-score on the target video is better than that of the general-purpose super-score model, and the over-score acquisition The video quality is higher.

The video transmission method provided by the present application is described above, and the server that implements the video transmission method is introduced below. Please refer to FIG. 12 , which is a schematic diagram of an embodiment of the server in the embodiment of the present application.

Only one or more of the various modules in FIG. 12 may be implemented in software, hardware, firmware, or a combination thereof. The software or firmware includes, but is not limited to, computer program instructions or code, and can be executed by a hardware processor. The hardware includes, but is not limited to, various types of integrated circuits, such as a central processing unit (CPU), a digital signal processor (DSP), a field programmable gate array (FPGA), or an application specific integrated circuit (ASIC).

The server includes:

The acquisition module 1201 is used to acquire a super-score model of a target video, and the super-score model is obtained by training according to the target video of the first resolution and the target video of the second resolution, and the second resolution is less than For the first resolution, the magnification of the super-resolution model is the ratio of pixels in the length direction or the ratio of pixels in the width direction of the first resolution and the second resolution;

Sending module 1202, configured to send the target video of the third resolution and the super-resolution model to the terminal, where the super-resolution model is used to perform the target video of the third resolution at the magnification ratio. Super-resolution reconstruction to obtain the target video at a fourth resolution.

Optionally, the target video includes a single video or a set of multiple videos of the same type.

Optionally, the third resolution is equal to the second resolution; the fourth resolution is equal to the first resolution.

Optionally, the obtaining module 1201 is specifically configured to: input the target video of the first resolution and the target video of the second resolution into a convolutional neural network model, and obtain the over-fitted super-score model. .

Optionally, the obtaining module 1201 is specifically configured to: perform data cleaning on the video frames in the video of the first resolution and the video frames in the video of the second resolution, and obtain the first resolution. the target video of the first resolution and the target video of the second resolution; perform training on the target video of the first resolution and the target video of the second resolution, and obtain the super-score Model.

Optionally, the sending module 1202 is specifically configured to: send the structure of the super-score model to the terminal; send a data packet to the terminal, the data packet including the weight parameter of the super-score model and the the target video at a third resolution.

The server provided by the embodiment of the present application acquires the super-division model of the target video and the target video of the third resolution through the acquisition module, and sends them to the terminal by the sending module. Compared with the prior art, the over-score recovery based on the general over-score model can also improve the video quality.

A terminal that implements the video transmission method is introduced below. Please refer to FIG. 13 , which is a schematic diagram of an embodiment of the terminal in the embodiment of the present application.

Only one or more of the various modules in FIG. 13 may be implemented in software, hardware, firmware, or a combination thereof. The software or firmware includes, but is not limited to, computer program instructions or code, and can be executed by a hardware processor. The hardware includes, but is not limited to, various types of integrated circuits, such as a central processing unit (CPU), a digital signal processor (DSP), a field programmable gate array (FPGA), or an application specific integrated circuit (ASIC).

The terminal includes: a receiving module 1301, configured to receive a super-score model of a target video sent by a server, where the super-score model is acquired by training according to the target video of the first resolution and the target video of the second resolution, The second resolution is smaller than the first resolution, the magnification of the super-resolution model is the ratio of the pixels of the first resolution and the second resolution in the length direction or in the width direction, and the third resolution rate of the target video;

The processing module 1302 is configured to perform super-resolution reconstruction on the target video of the third resolution at the magnification according to the super-resolution model, and obtain the target video of the fourth resolution.

Optionally, the super-score model includes: an over-fitted super-score model obtained by inputting the target video of the first resolution and the target video of the second resolution into a convolutional neural network model.

Optionally, the super-score model includes: a super-score model obtained by training the target video of the first resolution and the target video of the second resolution, and all the images of the first resolution. The target video and the target video of the second resolution are obtained by performing data cleaning on the video frames in the video of the first resolution and the video frames of the video of the second resolution.

Optionally, the receiving module 1301 is specifically configured to: receive the structure of the super-score model sent by the server; receive a data packet sent by the server, where the data packet includes the weight parameters of the super-score model and the target video at the third resolution.

In the terminal provided by the embodiment of the present application, the receiving module receives the superdivision model of the target video and the target video of the third resolution sent by the server, and the bandwidth of video transmission is relatively low, and performing superdivision recovery based on the superdivision model is more efficient than the prior art. The super-score recovery based on the general super-score model can improve the video quality.

Please refer to FIG. 14 , which is a schematic diagram of another embodiment of the server in the embodiment of the present application;

The server in this embodiment of the present application may be a physical machine or a virtual machine running on abstract hardware resources. In an actual application scenario, it may be a server that provides various cloud services. The device form is not limited.

The server 1400 provided in this embodiment may vary greatly due to different configurations or performance, and may include one or more processors 1401 and a memory 1402, where programs or data are stored in the memory 1402.

Among them, the memory 1402 may be volatile storage or non-volatile storage. Optionally, the processor 1401 is one or more central processing units (CPU, Central Processing Unit, which can be a single-core CPU or a multi-core CPU. The processor 1401 can communicate with the memory 1402 to execute on the server 1400 . A sequence of instructions in memory 1402.

The server 1400 also includes one or more wired or wireless network interfaces 1403, such as Ethernet interfaces.

Optionally, although not shown in FIG. 14 , the server 1400 may also include one or more power supplies; one or more input/output interfaces, which may be used to connect a monitor, mouse, keyboard, touch screen device or sensing device etc., the input and output interfaces are optional components, which may or may not exist, and are not limited here.

For the process performed by the processor 1401 in the server 1400 in this embodiment, reference may be made to the method process described in the foregoing method embodiments, and details are not repeated here.

Please refer to FIG. 15 , which is a schematic diagram of another embodiment of the terminal in the embodiment of the present application;

The terminal 1500 provided in this embodiment may be various types of terminals with display functions, such as a mobile phone, a tablet computer, a desktop computer, a smart screen, or a wearable device, and the specific device form is not limited in this embodiment of the present application.

The terminal 1500 may vary greatly due to different configurations or performances, and may include one or more processors 1501 and a memory 1502 in which programs or data are stored.

Among them, the memory 1502 may be volatile storage or non-volatile storage. Optionally, the processor 1501 is one or more central processing units (CPU, Central Processing Unit, which can be a single-core CPU or a multi-core CPU. The processor 1501 can communicate with the memory 1502 and execute on the terminal 1500 A series of instructions in memory 1502.

The terminal 1500 also includes one or more wired or wireless network interfaces 1503, such as Ethernet interfaces.

Optionally, although not shown in FIG. 15 , the terminal 1500 may also include one or more power supplies; one or more input and output interfaces, which may be used to connect a display, a mouse, a keyboard, a touch screen device or a sensing device etc., the input and output interfaces are optional components, which may or may not exist, and are not limited here.

For the process performed by the processor 1501 in the terminal 1500 in this embodiment, reference may be made to the method process described in the foregoing method embodiments, and details are not repeated here.

Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working process of the system, device and unit described above may refer to the corresponding process in the foregoing method embodiments, which will not be repeated here.

In the several embodiments provided in this application, it should be understood that the disclosed system, apparatus and method may be implemented in other manners. For example, the apparatus embodiments described above are only illustrative. For example, the division of the units is only a logical function division. In actual implementation, there may be other division methods. For example, multiple units or components may be combined or Can be integrated into another system, or some features can be ignored, or not implemented. On the other hand, the shown or discussed mutual coupling or direct coupling or communication connection may be through some interfaces, indirect coupling or communication connection of devices or units, and may be in electrical, mechanical or other forms.

The units described as separate components may or may not be physically separated, and components displayed as units may or may not be physical units, that is, may be located in one place, or may be distributed to multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution in this embodiment.

In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically alone, or two or more units may be integrated into one unit. The above-mentioned integrated units may be implemented in the form of hardware, or may be implemented in the form of software functional units.

The integrated unit, if implemented in the form of a software functional unit and sold or used as an independent product, may be stored in a computer-readable storage medium. Based on this understanding, the technical solutions of the present application can be embodied in the form of software products in essence, or the parts that contribute to the prior art, or all or part of the technical solutions, and the computer software products are stored in a storage medium , including several instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM), random access memory (RAM), magnetic disk or optical disk and other media that can store program codes .

As mentioned above, the above embodiments are only used to illustrate the technical solutions of the present application, but not to limit them.

Claims

A video transmission method, comprising:

The server obtains a super-resolution model of the target video, and the super-resolution model is obtained by training according to the target video of a first resolution and the target video of a second resolution, and the second resolution is smaller than the first resolution The magnification of the super-resolution model is the ratio of the pixels of the first resolution and the second resolution in the length direction or in the width direction;

The server sends the target video of the third resolution and the super-resolution model to the terminal, and the super-resolution model is used to perform super-resolution on the target video of the third resolution at the magnification ratio Reconstruct to obtain the target video at a fourth resolution.
The method according to claim 1, wherein the target video is a single video or a set of multiple videos of the same type.
The method according to claim 1 or 2, characterized in that,

the third resolution is equal to the second resolution;

The fourth resolution is equal to the first resolution.
The method according to any one of claims 1 to 3, wherein the obtaining, by the server, the super-score model of the target video comprises:

The server inputs the target video of the first resolution and the target video of the second resolution into a convolutional neural network model, and obtains the over-fitted super-score model.
The method according to any one of claims 1 to 3, wherein the obtaining, by the server, the super-score model of the target video comprises:

The server performs data cleaning on the video frames in the video of the first resolution and the video frames in the video of the second resolution, and obtains the target video of the first resolution and the second resolution. the target video of the resolution;

The server performs training on the target video of the first resolution and the target video of the second resolution to obtain the super-score model.
The method according to any one of claims 1 to 5, wherein the sending, by the server to the terminal, the target video of the third resolution and the super-resolution model specifically includes:

The server sends the structure of the super-resolution model to the terminal;

The server sends a data packet to the terminal, where the data packet includes the weight parameter of the super-score model and the target video of the third resolution.
A video transmission method, comprising:

The terminal receives the super-score model of the target video sent by the server, and the super-score model is obtained by training according to the target video of the first resolution and the target video of the second resolution, and the second resolution is smaller than the The first resolution, the magnification of the super-resolution model is the ratio of the pixels of the first resolution and the second resolution in the length direction or in the width direction, and the target video of the third resolution;

The terminal performs super-resolution reconstruction on the target video of the third resolution at the magnification according to the super-resolution model, and obtains the target video of the fourth resolution.
The method according to claim 7, wherein the target video is a single video or a set of multiple videos of the same type.
The method according to claim 7 or 8, wherein,

the third resolution is equal to the second resolution;

The fourth resolution is equal to the first resolution.
The method according to any one of claims 7 to 9, wherein the super-score model comprises:

An overfitting super-score model obtained by a convolutional neural network model is input from the target video of the first resolution and the target video of the second resolution.
The method according to any one of claims 7 to 10, wherein the super-score model comprises:

The super-score model obtained by training the target video of the first resolution and the target video of the second resolution, the target video of the first resolution and the target video of the second resolution The target video is obtained by performing data cleaning on video frames in the video of the first resolution and video frames in the video of the second resolution.
The method according to any one of claims 7 to 11, wherein the terminal receives the super-score model of the target video sent by the server, and the target video of the third resolution comprises:

receiving, by the terminal, the structure of the super-resolution model sent by the server;

The terminal receives a data packet sent by the server, where the data packet includes the weight parameter of the super-score model and the target video of the third resolution.
A server, characterized in that it includes:

The acquisition module is used to acquire the super-resolution model of the target video, and the super-resolution model is obtained by training according to the target video of the first resolution and the target video of the second resolution, and the second resolution is smaller than the target video. the first resolution, the magnification of the super-resolution model is the ratio of the pixels of the first resolution and the second resolution in the length direction or the ratio of the pixels in the width direction;

A sending module is configured to send the target video of the third resolution and the super-resolution model to the terminal, and the super-resolution model is used to perform super-resolution on the target video of the third resolution at the magnification ratio. Resolution reconstruction to obtain the target video at a fourth resolution.
The server according to claim 13, wherein the target video is a single video or a collection of multiple videos of the same type.
The server according to claim 13 or 14, characterized in that:

the third resolution is equal to the second resolution;

The fourth resolution is equal to the first resolution.
The server according to any one of claims 13 to 15, wherein the obtaining module is specifically configured to:

Inputting the target video of the first resolution and the target video of the second resolution into a convolutional neural network model to obtain the over-fitted super-score model.
The server according to any one of claims 13 to 16, wherein the obtaining module is specifically configured to:

Data cleaning is performed on the video frames in the video of the first resolution and the video frames in the video of the second resolution, and the target video of the first resolution and the target video of the second resolution are obtained. the target video;

The target video of the first resolution and the target video of the second resolution are trained to obtain the super-score model.
The server according to any one of claims 13 to 17, wherein the sending module is specifically configured to:

sending the structure of the super-resolution model to the terminal;

Send a data packet to the terminal, where the data packet includes the weight parameter of the super-score model and the target video of the third resolution.
A terminal, characterized in that it includes:

The receiving module is used to receive the super-score model of the target video sent by the server. ratio is less than the first resolution, the magnification of the super-resolution model is the ratio of the pixels of the first resolution and the second resolution in the length direction or in the width direction, and the target of the third resolution video;

A processing module, configured to perform super-resolution reconstruction on the target video of the third resolution at the magnification according to the super-resolution model, to obtain the target video of the fourth resolution.
The terminal according to claim 19, wherein the target video is a single video or a set of multiple videos of the same type.
The terminal according to claim 19 or 20, wherein,

the third resolution is equal to the second resolution;

The fourth resolution is equal to the first resolution.
The terminal according to any one of claims 19 to 21, wherein the super-score model comprises:

An overfitting super-score model obtained by a convolutional neural network model is input from the target video of the first resolution and the target video of the second resolution.
The terminal according to any one of claims 19 to 22, wherein the super-score model comprises:

The super-score model obtained by training the target video of the first resolution and the target video of the second resolution, the target video of the first resolution and the target video of the second resolution The target video is obtained by performing data cleaning on video frames in the video of the first resolution and video frames in the video of the second resolution.
The terminal according to any one of claims 19 to 23, wherein the receiving module is specifically configured to:

receiving the structure of the super-resolution model sent by the server;

Receive a data packet sent by the server, where the data packet includes the weight parameter of the super-score model and the target video of the third resolution.
A server, characterized by comprising: one or more processors and a memory; wherein,

computer-readable instructions are stored in the memory;

The computer readable instructions are read by the one or more processors to cause the terminal to implement the method of any one of claims 1 to 6.
A terminal, comprising: one or more processors and memories; wherein,

computer-readable instructions are stored in the memory;

The computer readable instructions are read by the one or more processors to cause the terminal to implement the method of any one of claims 7 to 12.
A video transmission system, comprising:

The server according to any one of claims 1 to 6, and the terminal according to any one of claims 7 to 12.
A computer program product, characterized by comprising computer-readable instructions which, when executed on a computer, cause the computer to perform the method of any one of claims 1 to 12.
A computer-readable storage medium, characterized by comprising computer-readable instructions, which, when the computer-readable instructions are executed on a computer, cause the computer to execute the method according to any one of claims 1 to 12 .