CN115567720A - Video transmission method, device, storage medium and equipment - Google Patents
Video transmission method, device, storage medium and equipment Download PDFInfo
- Publication number
- CN115567720A CN115567720A CN202211101542.9A CN202211101542A CN115567720A CN 115567720 A CN115567720 A CN 115567720A CN 202211101542 A CN202211101542 A CN 202211101542A CN 115567720 A CN115567720 A CN 115567720A
- Authority
- CN
- China
- Prior art keywords
- network
- parameters
- video
- real
- generating
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 230000005540 biological transmission Effects 0.000 title claims abstract description 53
- 238000000034 method Methods 0.000 title claims abstract description 41
- 238000004364 calculation method Methods 0.000 claims abstract description 47
- 238000012549 training Methods 0.000 claims abstract description 46
- 230000006870 function Effects 0.000 claims description 16
- 238000000605 extraction Methods 0.000 claims description 5
- 230000008569 process Effects 0.000 claims description 4
- 238000010586 diagram Methods 0.000 description 3
- 230000000007 visual effect Effects 0.000 description 3
- 238000013528 artificial neural network Methods 0.000 description 2
- 230000009467 reduction Effects 0.000 description 2
- 238000013473 artificial intelligence Methods 0.000 description 1
- 230000003190 augmentative effect Effects 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000004891 communication Methods 0.000 description 1
- 230000006835 compression Effects 0.000 description 1
- 238000007906 compression Methods 0.000 description 1
- 238000001514 detection method Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 238000001093 holography Methods 0.000 description 1
- 230000010354 integration Effects 0.000 description 1
- 238000005259 measurement Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 238000012545 processing Methods 0.000 description 1
- 230000000750 progressive effect Effects 0.000 description 1
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/42—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals characterised by implementation details or hardware specially adapted for video compression or decompression, e.g. dedicated software implementation
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
- H04N21/21—Server components or server architectures
- H04N21/218—Source of audio or video content, e.g. local disk arrays
- H04N21/2187—Live feed
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
- H04N21/23—Processing of content or additional data; Elementary server operations; Server middleware
- H04N21/234—Processing of video elementary streams, e.g. splicing of video streams or manipulating encoded video stream scene graphs
- H04N21/2343—Processing of video elementary streams, e.g. splicing of video streams or manipulating encoded video stream scene graphs involving reformatting operations of video signals for distribution or compliance with end-user requests or end-user device requirements
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
- H04N21/23—Processing of content or additional data; Elementary server operations; Server middleware
- H04N21/234—Processing of video elementary streams, e.g. splicing of video streams or manipulating encoded video stream scene graphs
- H04N21/2343—Processing of video elementary streams, e.g. splicing of video streams or manipulating encoded video stream scene graphs involving reformatting operations of video signals for distribution or compliance with end-user requests or end-user device requirements
- H04N21/234363—Processing of video elementary streams, e.g. splicing of video streams or manipulating encoded video stream scene graphs involving reformatting operations of video signals for distribution or compliance with end-user requests or end-user device requirements by altering the spatial resolution, e.g. for clients with a lower screen resolution
Landscapes
- Engineering & Computer Science (AREA)
- Multimedia (AREA)
- Signal Processing (AREA)
- Databases & Information Systems (AREA)
- Two-Way Televisions, Distribution Of Moving Picture Or The Like (AREA)
Abstract
The application discloses a video transmission method, a device, a storage medium and equipment, which can be applied to the field of big data, and the method comprises the following steps: generating a first flow type calculation operation based on parameters for generating a network in a countermeasure network and a data source for generating a real-time video, wherein the parameters are obtained by training in advance; generating a second streaming calculation job based on the parameters for generating the network in the confrontation network and the receiver of the real-time video; identifying the first flow calculation operation as a message producer of a pre-constructed message queue, and identifying the second flow calculation operation as a message consumer of the message queue; and controlling the message producer to send each coded data to the message queue, and controlling the message consumer to obtain each coded data from the message queue. Compared with the prior art, the method does not need to consume time to encode and decode the real-time video, and the encoded data of the video frames are used as the transmission unit, so that the data transmission efficiency can be improved, and the video transmission speed is effectively improved.
Description
Technical Field
The present application relates to the field of big data, and in particular, to a video transmission method, apparatus, storage medium, and device.
Background
At present, the transmission of real-time video usually adopts the format of h.246 coding for transmission, i.e. firstly, the file is coded according to the h.246 specification, and after the data is transmitted, the file is decoded and restored into the original image.
However, in the existing transmission method, encoding and decoding are required before and after data transmission, which consumes a lot of computing resources and time cost, and the compression capability of the h.246 format file is limited, and the memory of the compressed packet occupies a large amount of transmission data, which increases the bandwidth consumption and reduces the video transmission speed.
Therefore, how to increase the video transmission speed becomes a problem to be solved urgently in the field.
Disclosure of Invention
The application provides a video transmission method, a video transmission device, a storage medium and video transmission equipment, and aims to improve the video transmission speed.
In order to achieve the above object, the present application provides the following technical solutions:
a video transmission method, comprising:
generating a first flow type calculation operation based on parameters for generating a network in a countermeasure network and a data source for generating a real-time video, wherein the parameters are obtained by training in advance; the real-time video comprises a sequence of video frames; the video frame sequence comprises a plurality of video frames, and the video frames are sequenced according to the sequence of playing time from front to back; the first streaming computing job is to: coding each video frame generated by the data source to obtain coded data of each video frame;
generating a second streaming computing job based on the parameters of the generation network in the generation countermeasure network and the receiver of the real-time video; the second streaming computing job is to: decoding each coded data to obtain the real-time video, and sending the real-time video to the receiver;
identifying the first streaming computing job as a message producer of a pre-constructed message queue, and identifying the second streaming computing job as a message consumer of the message queue;
and controlling the message producer to send each coded data to the message queue, and controlling the message consumer to obtain each coded data from the message queue.
Optionally, the training process for generating the countermeasure network includes:
marking a plurality of pre-obtained high-resolution images as sample images;
acquiring a generation network and a judgment network of a preset generation countermeasure network; the generation network comprises an encoder and a decoder;
taking the sample image as the input of the encoder, and performing resolution characteristic extraction on the sample image by using the encoder to obtain a resolution characteristic vector;
taking the resolution characteristic vector as the input of the decoder, and fusing the resolution characteristic vector by using the decoder to obtain a target image output by the generation network;
inputting the target image and the sample image into a discrimination network, continuously adjusting a loss function of the discrimination network and various parameters of the generation network by taking the difference between the resolution of the target image and the resolution of the sample image as a training target until the value of the loss function of the discrimination network is less than a preset threshold value, and determining that the training of the generation countermeasure network is completed.
Optionally, the parameters of the generation network include parameters of a convolution kernel of the generation network and parameters of an encoder included in the generation network;
generating a first flow computing job based on parameters for generating a network in a countermeasure network and a data source for generating a real-time video, which are obtained by pre-training, and comprising:
extracting a convolution kernel parameter of a generated network from the generated countermeasure network obtained by pre-training, and generating a stream algorithm logic based on the convolution kernel parameter;
and extracting parameters of the encoder from the generation countermeasure network, and substituting the parameters of the encoder and a data source for generating a real-time video into the streaming algorithm logic to obtain a first streaming calculation operation by taking the parameters of the encoder and the data source as video encoding operators.
Optionally, the parameters of the generation network include parameters of a convolution kernel of the generation network and parameters of a decoder included in the generation network;
generating a second streaming computing job based on the parameters of the generation countermeasure network and the receiver of the real-time video, comprising:
extracting a convolution kernel parameter of a generated network from the generated countermeasure network obtained by pre-training, and generating a stream algorithm logic based on the convolution kernel parameter;
and extracting the parameters of the decoder from the generation countermeasure network, and substituting the parameters of the decoder and the receiver of the real-time video into the streaming algorithm logic to obtain a second streaming calculation operation by taking the parameters of the decoder and the receiver of the real-time video as video decoding operators.
Optionally, the controlling the message producer to send each encoded data to the message queue, and controlling the message consumer to obtain each encoded data from the message queue includes:
after the message producer obtains the video frame sequence generated by the data source, controlling the message producer to encode each video frame in the video frame sequence to obtain the encoded data of each video frame, and sequentially sending the encoded data of each video frame in the video frame sequence to the message queue according to the sequence of the sequence from front to back;
and controlling the message consumer to acquire each coded data from the message queue according to a first-in first-out sequence, decoding each coded data to obtain the real-time video, and sending the real-time video to the receiving party.
A video transmission apparatus comprising:
the first job generation unit is used for generating a first flow calculation job based on parameters for generating a network in a countermeasure network and a data source for generating a real-time video, wherein the parameters are obtained by training in advance; the real-time video comprises a sequence of video frames; the video frame sequence comprises a plurality of video frames, and the video frames are sequenced according to the sequence of playing time from front to back; the first streaming computing job is to: coding each video frame generated by the data source to obtain coded data of each video frame;
a second job generation unit configured to generate a second streaming calculation job based on the parameters of the generation countermeasure network and the receiver of the real-time video; the second streaming computing job is to: decoding each coded data to obtain the real-time video, and sending the real-time video to the receiver;
an identification unit, configured to identify the first streaming calculation job as a message producer of a pre-constructed message queue, and identify the second streaming calculation job as a message consumer of the message queue;
and the control unit is used for controlling the message producer to send each coded data to the message queue and controlling the message consumer to obtain each coded data from the message queue.
Optionally, the first job generating unit is specifically configured to:
marking a plurality of pre-obtained high-resolution images as sample images;
acquiring a preset generation network and a judgment network for generating a countermeasure network; the generation network comprises an encoder and a decoder;
taking the sample image as the input of the encoder, and performing resolution characteristic extraction on the sample image by using the encoder to obtain a resolution characteristic vector;
taking the resolution characteristic vector as the input of the decoder, and fusing the resolution characteristic vector by using the decoder to obtain a target image output by the generation network;
inputting the target image and the sample image into a discrimination network, continuously adjusting a loss function of the discrimination network and various parameters of the generation network by taking the difference between the resolution of the target image and the resolution of the sample image as a training target until the value of the loss function of the discrimination network is less than a preset threshold value, and determining that the training of the generation countermeasure network is completed.
Optionally, the parameters of the generation network include parameters of a convolution kernel of the generation network and parameters of an encoder included in the generation network;
the first job generation unit is specifically configured to:
extracting a convolution kernel parameter of a generated network from the generated countermeasure network obtained by pre-training, and generating a stream algorithm logic based on the convolution kernel parameter;
and extracting parameters of the encoder from the generation countermeasure network, and substituting the parameters of the encoder and a data source for generating a real-time video into the streaming algorithm logic to obtain a first streaming calculation operation by taking the parameters of the encoder and the data source as video encoding operators.
A computer-readable storage medium including a stored program, wherein the program executes the video transmission method.
A video transmission device comprising: a processor, a memory, and a bus; the processor and the memory are connected through the bus;
the memory is used for storing programs, and the processor is used for running programs, wherein the programs execute the video transmission method during running.
According to the technical scheme, the first flow type calculation operation is generated based on parameters for generating the network in the countermeasure network and a data source for generating the real-time video, wherein the parameters are obtained through pre-training. Generating a second streaming computing job based on the parameters for generating the generation network in the confrontation network and the receiver of the real-time video. The first streaming computing job is identified as a message producer of the pre-built message queue and the second streaming computing job is identified as a message consumer of the message queue. And controlling the message producer to send each coded data to the message queue, and controlling the message consumer to obtain each coded data from the message queue. According to the method and the device, the parameters of the generation network in the generation countermeasure network obtained by training are utilized to generate the first flow type calculation operation and the second flow type calculation operation, and effective transmission of the real-time video is achieved based on the message queue, the first flow type calculation operation and the second flow type calculation operation.
Drawings
In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings used in the embodiments or the prior art descriptions will be briefly described below, it is obvious that the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative efforts.
Fig. 1a is a schematic flowchart of a video transmission method according to an embodiment of the present application;
fig. 1b is a schematic flowchart of a video transmission method according to an embodiment of the present application;
fig. 1c is a schematic diagram of a generation network according to an embodiment of the present application;
fig. 2 is a schematic flowchart of another video transmission method according to an embodiment of the present application;
fig. 3 is a schematic diagram of an architecture of a video transmission device according to an embodiment of the present disclosure.
Detailed Description
The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application, and it is obvious that the described embodiments are only a part of the embodiments of the present application, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present application.
As shown in fig. 1a and fig. 1b, a schematic flow chart of a video transmission method provided in the embodiment of the present application includes the following steps:
s101: and marking a plurality of high-resolution images obtained in advance as sample images.
The web crawler tool can be used for capturing a plurality of high-resolution images from the internet, and specifically, the formats of the images can be unified into the same image format, such as a jpg format.
S102: and acquiring a preset generation network and a judgment network for generating the countermeasure network.
Wherein the generation network adopts an encoder-decoder structure, i.e. the generation network comprises an encoder and a decoder.
It should be noted that, the encoder is used for a residual error module for performing downsampling, the residual error module includes a plurality of convolutional layers, and specifically, the number of convolutional layers in the residual error module may be set by a skilled person according to an actual situation.
In addition, the decoder comprises a residual module for upsampling, the residual module comprises a plurality of convolutional layers, and specifically, the number of convolutional layers in the residual module can be set by a skilled person according to actual conditions.
In this embodiment of the present application, the preset generation countermeasure network may specifically be a super-resolution generation countermeasure network, and specifically, an encoder and a decoder shown in the generation network in the super-resolution generation countermeasure network may be shown in fig. 1 c.
Generally speaking, a generating network is a neural network similar to a coder and a decoder, an original image is rolled and pooled to finally obtain a highly abstract feature sequence, the feature sequence is reduced to the size of the original image through operations such as convolution/deconvolution, and parameters are continuously optimized through training, so that the reduced image has the effects of noise reduction, deblurring, super-resolution reduction and the like.
The discrimination network is a neural network for detection, and is particularly used for detecting the visual resolution of an image, and when the image looks high in definition, the discrimination network result score is high, otherwise, the discrimination network result score is low. The generation network and the discrimination network are trained simultaneously, and mutual promotion is realized to finally obtain the generation network capable of improving the visual resolution of the image.
S103: and taking the sample image as the input of an encoder, and extracting the resolution characteristic of the sample image by using the encoder to obtain a resolution characteristic vector.
S104: and taking the resolution characteristic vector as the input of a decoder, and fusing the resolution characteristic vector by using the decoder to obtain a target image which is output to the outside by the generation network.
S105: inputting the target image and the sample image into a discrimination network, continuously adjusting a loss function of the discrimination network and generating various parameters of the network by taking the difference between the resolution of the target image and the resolution of the sample image as a training target until the value of the loss function of the discrimination network is less than a preset threshold value, and determining that the training of the generated confrontation network is completed.
The specific type of the loss function can be set by a technician according to actual conditions.
The parameters include a convolution kernel parameter of the generation network, a parameter of the encoder, and a parameter of the decoder.
S106: and extracting the convolution kernel parameters of the generation network from the generation countermeasure network obtained by training, and generating the streaming algorithm logic based on the convolution kernel parameters of the generation network.
The streaming algorithm is used for performing data processing on streaming data, and the real-time video shown in the embodiment of the present application may be regarded as streaming data.
S107: and extracting parameters of an encoder from the generated countermeasure network obtained by training, and substituting the parameters of the encoder and a data source for generating the real-time video into the streaming algorithm logic to obtain a first streaming calculation operation by taking the parameters of the encoder and the data source as video encoding operators.
Wherein the first streaming computing job is operable to: and coding the real-time video generated by the data source to obtain the coded data of the real-time video.
It should be noted that the real-time video includes a video frame sequence, the video frame sequence includes a plurality of video frames, and each video frame is ordered according to the sequence of the playing time from front to back. So-called video frames are also images in nature.
S108: and extracting parameters of a decoder from the generated countermeasure network obtained by training, and substituting the parameters of the decoder and a receiver of the real-time video into the streaming algorithm logic to obtain second streaming calculation operation by taking the parameters of the decoder and the receiver of the real-time video as video decoding operators.
Wherein the second streaming computing job is operable to: and decoding the coded data to obtain a real-time video, and sending the real-time video to a receiving party.
S109: the first streaming computing job is identified as a message producer of the pre-built message queue, and the second streaming computing job is identified as a message consumer of the message queue.
The message producer and the message consumer subscribe to the same theme (topic) of the message queue together, namely, the message producer sends the video frame generated by the message producer to the same theme, and the message consumer acquires the video frame from the same theme.
It should be noted that the message queue may be a kafka queue.
S110: after the message producer obtains the video frame sequence generated by the data source, the message producer is controlled to encode each video frame in the video frame sequence to obtain the encoded data of each video frame, and the encoded data of each video frame in the video frame sequence is sequentially sent to the message queue according to the sequence of the sequence from front to back.
S111: and controlling the message consumer to acquire each coded data from the message queue according to the first-in first-out sequence, decoding each coded data to obtain a real-time video, and sending the real-time video to a receiving party.
Based on the above flow shown in S101-S111, the present embodiment can achieve the following beneficial effects:
1. the advantages of the streaming algorithm logic and the message queue are utilized, and the real-time video is guaranteed not to lose data in the data transmission process.
2. The image visual quality of the real-time video is improved by utilizing the generated countermeasure network, and the watching experience of the user is improved.
3. By combining the generation of the countermeasure network and the streaming algorithm logic, the effective integration of the computing resources of the big data platform can be realized.
4. In the process of transmitting data of the real-time video, only highly abstract characteristic data (namely coded data of video frames) is transmitted, so that the data volume is small, the bandwidth is effectively saved, the data transmission efficiency is improved, and the video transmission speed is effectively improved.
In summary, in the embodiment, the first streaming calculation job and the second streaming calculation job are generated by using the parameters of the generated network in the generation countermeasure network obtained by training, and effective transmission of the real-time video is realized based on the message queue, the first streaming calculation job and the second streaming calculation job.
It should be noted that, in the above embodiment, reference is made to S101, which is an optional implementation manner of the video transmission method shown in the embodiment of the present application. In addition, S112 mentioned in the foregoing embodiment is also an optional implementation manner of the video transmission method shown in the embodiment of the present application. For this reason, the flow mentioned in the above embodiment can be summarized as the method shown in fig. 2.
As shown in fig. 2, a schematic flow chart of another video transmission method provided in the embodiment of the present application includes the following steps:
s201: and generating a first flow type calculation operation based on parameters for generating the generation network in the countermeasure network and a data source for generating the real-time video, wherein the parameters are obtained by training in advance.
Wherein the real-time video comprises a sequence of video frames; the video frame sequence comprises a plurality of video frames, and the video frames are sequenced according to the sequence of the playing time from front to back; the first streaming computing job is for: and coding each video frame generated by the data source to obtain the coded data of each video frame.
S202: generating a second streaming computing job based on the parameters for generating the generation network in the confrontation network and the receiver of the real-time video.
Wherein the second streaming computing job is to: and decoding each coded data to obtain a real-time video, and sending the real-time video to a receiving party.
S203: the first streaming computing job is identified as a message producer of the pre-built message queue and the second streaming computing job is identified as a message consumer of the message queue.
S204: and controlling the message producer to send each coded data to the message queue, and controlling the message consumer to obtain each coded data from the message queue.
In summary, in the embodiment, the first streaming calculation job and the second streaming calculation job are generated by using the parameters of the generated network in the generation countermeasure network obtained by training, and effective transmission of the real-time video is realized based on the message queue, the first streaming calculation job and the second streaming calculation job.
It should be noted that the video transmission method provided by the invention can be used in the fields of artificial intelligence, block chaining, distribution, cloud computing, big data, internet of things, mobile internet, network security, chip, virtual reality, augmented reality, holography, quantum computing, quantum communication, quantum measurement, digital twinning, and finance. The foregoing is merely an example, and does not limit the application field of the video transmission method provided by the present invention.
The video transmission method provided by the invention can be used in the financial field or other fields, for example, can be used in transaction application scenes in the financial field. The other fields are arbitrary fields other than the financial field, for example, the electric power field. The foregoing is merely an example, and does not limit the application field of the video transmission method provided by the present invention.
Corresponding to the video transmission method provided by the embodiment of the application, the embodiment of the application also provides a video transmission device.
As shown in fig. 3, a schematic architecture diagram of a video transmission apparatus according to an embodiment of the present application includes:
a first job generation unit 100, configured to generate a first streaming calculation job based on a pre-trained parameter for generating a network in a countermeasure network and a data source for generating a real-time video; the real-time video comprises a sequence of video frames; the video frame sequence comprises a plurality of video frames, and the video frames are sequenced from front to back according to the playing time; the first streaming computing job is for: and coding each video frame generated by the data source to obtain the coded data of each video frame.
Optionally, the first job generating unit 100 is specifically configured to: marking a plurality of pre-obtained high-resolution images as sample images; acquiring a generation network and a judgment network of a preset generation countermeasure network; the generating network comprises an encoder and a decoder; taking the sample image as the input of an encoder, and extracting the resolution characteristic of the sample image by using the encoder to obtain a resolution characteristic vector; taking the resolution characteristic vector as the input of a decoder, and fusing the resolution characteristic vector by using the decoder to obtain a target image which is output by a generation network; inputting the target image and the sample image into a discrimination network, continuously adjusting a loss function of the discrimination network and generating various parameters of the network by taking the difference between the resolution of the target image and the resolution of the sample image as a training target until the value of the loss function of the discrimination network is less than a preset threshold value, and determining that the training of the generated confrontation network is completed.
The parameters for generating the network comprise parameters of a convolution kernel of the network and parameters of an encoder contained in the network.
The first job generation unit 100 is specifically configured to: extracting convolution kernel parameters of a generation network from the generation countermeasure network obtained by pre-training, and generating a stream algorithm logic based on the convolution kernel parameters; and extracting parameters of an encoder from the generation countermeasure network, and substituting the parameters of the encoder and a data source for generating the real-time video into the streaming algorithm logic to obtain a first streaming calculation operation by taking the parameters of the encoder and the data source as video encoding operators.
The parameters for generating the network comprise the parameters of the convolution kernel of the network and the parameters of a decoder contained in the network.
The first job generation unit 100 is specifically configured to: extracting convolution kernel parameters of a generation network from the generation countermeasure network obtained by pre-training, and generating a stream algorithm logic based on the convolution kernel parameters; and extracting parameters of a decoder from the generated countermeasure network, and substituting the parameters of the decoder and a receiver of the real-time video into the streaming algorithm logic to obtain a second streaming calculation operation by taking the parameters of the decoder and the receiver of the real-time video as video decoding operators.
A second job generation unit 200 for generating a second streaming calculation job based on the parameters for generating the generation network in the countermeasure network and the receiver of the real-time video; the second streaming computing job is for: and decoding each coded data to obtain a real-time video, and sending the real-time video to a receiving party.
An identifying unit 300 for identifying the first streaming computing job as a message producer of the pre-built message queue and the second streaming computing job as a message consumer of the message queue.
And the control unit 400 is used for controlling the message producer to send each coded data to the message queue and controlling the message consumer to obtain each coded data from the message queue.
Optionally, the control unit 400 is specifically configured to: after a message producer obtains a video frame sequence generated by a data source, the message producer is controlled to encode each video frame in the video frame sequence to obtain the encoded data of each video frame, and the encoded data of each video frame in the video frame sequence is sequentially sent to a message queue according to the sequence of the sequence from front to back; and controlling the message consumer to acquire each coded data from the message queue according to the first-in first-out sequence, decoding each coded data to obtain a real-time video, and sending the real-time video to a receiving party.
In summary, in the embodiment, the first streaming calculation job and the second streaming calculation job are generated by using the parameters of the generated network in the generation countermeasure network obtained by training, and effective transmission of the real-time video is realized based on the message queue, the first streaming calculation job and the second streaming calculation job.
The present application also provides a computer-readable storage medium including a stored program, wherein the program performs the video transmission method provided by the present application.
The present application also provides a video transmission device, including: a processor, memory, and a bus. The processor is connected with the memory through a bus, the memory is used for storing programs, and the processor is used for running the programs, wherein when the programs are run, the video transmission method provided by the application is executed, and the method comprises the following steps:
generating a first flow type calculation operation based on parameters for generating a network in a countermeasure network and a data source for generating a real-time video, wherein the parameters are obtained by training in advance; the real-time video comprises a sequence of video frames; the video frame sequence comprises a plurality of video frames, and the video frames are sequenced according to the sequence of playing time from front to back; the first streaming computing job is to: coding each video frame generated by the data source to obtain coded data of each video frame;
generating a second streaming computing job based on the parameters of the generation network in the generation countermeasure network and the receiver of the real-time video; the second streaming computing job is to: decoding each coded data to obtain the real-time video, and sending the real-time video to the receiver;
identifying the first streaming computing job as a message producer of a pre-constructed message queue, and identifying the second streaming computing job as a message consumer of the message queue;
and controlling the message producer to send each coded data to the message queue, and controlling the message consumer to obtain each coded data from the message queue.
Specifically, on the basis of the above embodiment, the training process for generating the countermeasure network includes:
marking a plurality of pre-obtained high-resolution images as sample images;
acquiring a generation network and a judgment network of a preset generation countermeasure network; the generation network comprises an encoder and a decoder;
taking the sample image as the input of the encoder, and performing resolution characteristic extraction on the sample image by using the encoder to obtain a resolution characteristic vector;
taking the resolution characteristic vector as the input of the decoder, and fusing the resolution characteristic vector by using the decoder to obtain a target image output by the generation network;
inputting the target image and the sample image into a discrimination network, continuously adjusting a loss function of the discrimination network and various parameters of the generation network by taking the difference between the resolution of the target image and the resolution of the sample image as a training target until the value of the loss function of the discrimination network is less than a preset threshold value, and determining that the training of the generation countermeasure network is completed.
Specifically, on the basis of the above embodiment, the parameters of the generation network include parameters of a convolution kernel of the generation network and parameters of an encoder included in the generation network;
generating a first flow computing operation based on parameters of a generation network in a generation countermeasure network obtained by pre-training and a data source for generating a real-time video, comprising:
extracting a convolution kernel parameter of a generated network from the generated countermeasure network obtained by pre-training, and generating a stream algorithm logic based on the convolution kernel parameter;
and extracting parameters of the encoder from the generation countermeasure network, and substituting the parameters of the encoder and a data source for generating a real-time video into the streaming algorithm logic to obtain a first streaming calculation operation by taking the parameters of the encoder and the data source as video encoding operators.
Specifically, on the basis of the above embodiment, the parameters of the generation network include parameters of a convolution kernel of the generation network and parameters of a decoder included in the generation network;
generating a second streaming computing job based on the parameters of the generation countermeasure network and the receiver of the real-time video, comprising:
extracting a convolution kernel parameter of a generated network from the generated countermeasure network obtained by pre-training, and generating a stream algorithm logic based on the convolution kernel parameter;
and extracting the parameters of the decoder from the generation countermeasure network, and substituting the parameters of the decoder and the receiver of the real-time video into the streaming algorithm logic to obtain a second streaming calculation operation by taking the parameters of the decoder and the receiver of the real-time video as video decoding operators.
Specifically, on the basis of the above embodiment, the controlling the message producer to send each encoded data to the message queue, and controlling the message consumer to obtain each encoded data from the message queue includes:
after the message producer obtains the video frame sequence generated by the data source, controlling the message producer to encode each video frame in the video frame sequence to obtain the encoded data of each video frame, and sequentially sending the encoded data of each video frame in the video frame sequence to the message queue according to the sequence of the sequence from front to back;
and controlling the message consumer to acquire each coded data from the message queue according to a first-in first-out sequence, decoding each coded data to obtain the real-time video, and sending the real-time video to the receiving party.
The functions described in the method of the embodiment of the present application, if implemented in the form of software functional units and sold or used as independent products, may be stored in a storage medium readable by a computing device. Based on such understanding, part of the contribution to the prior art of the embodiments of the present application or part of the technical solution may be embodied in the form of a software product stored in a storage medium and including several instructions for causing a computing device (which may be a personal computer, a server, a mobile computing device or a network device) to execute all or part of the steps of the method described in the embodiments of the present application. And the aforementioned storage medium includes: u disk, removable hard disk, read only memory, random access memory, magnetic or optical disk, etc. for storing program codes.
The embodiments are described in a progressive manner, each embodiment focuses on differences from other embodiments, and the same or similar parts among the embodiments are referred to each other.
The previous description of the disclosed embodiments is provided to enable any person skilled in the art to make or use the present application. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the generic principles defined herein may be applied to other embodiments without departing from the spirit or scope of the application. Thus, the present application is not intended to be limited to the embodiments shown herein but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.
Claims (10)
1. A video transmission method, comprising:
generating a first flow type calculation operation based on parameters for generating a network in a countermeasure network and a data source for generating a real-time video, wherein the parameters are obtained by training in advance; the real-time video comprises a sequence of video frames; the video frame sequence comprises a plurality of video frames, and the video frames are sequenced according to the sequence of playing time from front to back; the first streaming computing job is to: coding each video frame generated by the data source to obtain coded data of each video frame;
generating a second streaming computing job based on the parameters of the generation network in the generation countermeasure network and the receiver of the real-time video; the second streaming computing job is to: decoding each coded data to obtain the real-time video, and sending the real-time video to the receiver;
identifying the first streaming computing job as a message producer of a pre-constructed message queue, and identifying the second streaming computing job as a message consumer of the message queue;
and controlling the message producer to send each coded data to the message queue, and controlling the message consumer to obtain each coded data from the message queue.
2. The method of claim 1, wherein the training process for generating the countermeasure network comprises:
marking a plurality of pre-obtained high-resolution images as sample images;
acquiring a preset generation network and a judgment network for generating a countermeasure network; the generation network comprises an encoder and a decoder;
taking the sample image as the input of the encoder, and performing resolution characteristic extraction on the sample image by using the encoder to obtain a resolution characteristic vector;
taking the resolution characteristic vector as the input of the decoder, and fusing the resolution characteristic vector by using the decoder to obtain a target image output by the generation network;
inputting the target image and the sample image into a discrimination network, continuously adjusting a loss function of the discrimination network and various parameters of the generation network by taking the difference between the resolution of the target image and the resolution of the sample image as a training target until the value of the loss function of the discrimination network is less than a preset threshold value, and determining that the training of the generation countermeasure network is completed.
3. The method according to claim 1, wherein the parameters of the generation network comprise parameters of a convolution kernel of the generation network, parameters of an encoder included in the generation network;
generating a first flow computing operation based on parameters of a generation network in a generation countermeasure network obtained by pre-training and a data source for generating a real-time video, comprising:
extracting a convolution kernel parameter of a generated network from the generated countermeasure network obtained by pre-training, and generating a stream algorithm logic based on the convolution kernel parameter;
and extracting parameters of the encoder from the generation countermeasure network, and substituting the parameters of the encoder and a data source for generating a real-time video into the streaming algorithm logic to obtain a first streaming calculation operation by taking the parameters of the encoder and the data source as video encoding operators.
4. The method according to claim 1, wherein the parameters of the generation network comprise parameters of a convolution kernel of the generation network, parameters of a decoder included in the generation network;
generating a second streaming computing job based on the parameters of the generation countermeasure network and the receiver of the real-time video, comprising:
extracting a convolution kernel parameter of a generated network from the generated countermeasure network obtained by pre-training, and generating a stream algorithm logic based on the convolution kernel parameter;
and extracting parameters of the decoder from the generation countermeasure network, and substituting the parameters of the decoder and a receiver of the real-time video into the streaming algorithm logic to obtain second streaming calculation operation by taking the parameters of the decoder and the receiver of the real-time video as video decoding operators.
5. The method of claim 1, wherein said controlling the message producer to send each of the encoded data to the message queue and controlling the message consumer to retrieve each of the encoded data from the message queue comprises:
after the message producer obtains the video frame sequence generated by the data source, controlling the message producer to encode each video frame in the video frame sequence to obtain the encoded data of each video frame, and sequentially sending the encoded data of each video frame in the video frame sequence to the message queue according to the sequence of the sequence from front to back;
and controlling the message consumer to acquire each coded data from the message queue according to a first-in first-out sequence, decoding each coded data to obtain the real-time video, and sending the real-time video to the receiving party.
6. A video transmission apparatus, comprising:
the first job generation unit is used for generating a first flow calculation job based on parameters for generating a network in a countermeasure network and a data source for generating a real-time video, wherein the parameters are obtained by training in advance; the real-time video comprises a sequence of video frames; the video frame sequence comprises a plurality of video frames, and the video frames are sequenced according to the sequence of playing time from front to back; the first streaming computing job is to: coding each video frame generated by the data source to obtain coded data of each video frame;
a second job generation unit configured to generate a second streaming calculation job based on the parameters of the generation countermeasure network and the receiver of the real-time video; the second streaming computing job is to: decoding each coded data to obtain the real-time video, and sending the real-time video to the receiver;
an identification unit, configured to identify the first streaming calculation job as a message producer of a pre-constructed message queue, and identify the second streaming calculation job as a message consumer of the message queue;
and the control unit is used for controlling the message producer to send each coded data to the message queue and controlling the message consumer to obtain each coded data from the message queue.
7. The apparatus according to claim 6, wherein the first job generation unit is specifically configured to:
marking a plurality of pre-obtained high-resolution images as sample images;
acquiring a preset generation network and a judgment network for generating a countermeasure network; the generation network comprises an encoder and a decoder;
taking the sample image as the input of the encoder, and performing resolution characteristic extraction on the sample image by using the encoder to obtain a resolution characteristic vector;
taking the resolution characteristic vector as the input of the decoder, and fusing the resolution characteristic vector by using the decoder to obtain a target image output by the generation network;
inputting the target image and the sample image into a discrimination network, continuously adjusting a loss function of the discrimination network and various parameters of the generation network by taking the difference between the resolution of the target image and the resolution of the sample image as a training target until the value of the loss function of the discrimination network is less than a preset threshold value, and determining that the training of the generation countermeasure network is completed.
8. The apparatus of claim 6, wherein the parameters of the generator network comprise parameters of a convolution kernel of the generator network, parameters of an encoder included in the generator network;
the first job generation unit is specifically configured to:
extracting a convolution kernel parameter of a generated network from the generated countermeasure network obtained by pre-training, and generating a stream algorithm logic based on the convolution kernel parameter;
and extracting parameters of the encoder from the generation countermeasure network, and substituting the parameters of the encoder and a data source for generating a real-time video into the streaming algorithm logic to obtain a first streaming calculation operation by taking the parameters of the encoder and the data source as video encoding operators.
9. A computer-readable storage medium, characterized in that the computer-readable storage medium comprises a stored program, wherein the program executes the video transmission method according to any one of claims 1 to 5.
10. A video transmission device, comprising: a processor, a memory, and a bus; the processor and the memory are connected through the bus;
the memory is used for storing a program, and the processor is used for executing the program, wherein the program executes the video transmission method according to any one of claims 1 to 5.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211101542.9A CN115567720A (en) | 2022-09-09 | 2022-09-09 | Video transmission method, device, storage medium and equipment |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211101542.9A CN115567720A (en) | 2022-09-09 | 2022-09-09 | Video transmission method, device, storage medium and equipment |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN115567720A true CN115567720A (en) | 2023-01-03 |
Family
ID=84741048
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202211101542.9A Pending CN115567720A (en) | 2022-09-09 | 2022-09-09 | Video transmission method, device, storage medium and equipment |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN115567720A (en) |
Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109803112A (en) * | 2017-11-16 | 2019-05-24 | 中兴通讯股份有限公司 | Video analysis management method based on big data, apparatus and system, storage medium |
| CN111400326A (en) * | 2020-02-28 | 2020-07-10 | 深圳市赛为智能股份有限公司 | Smart city data management system and method thereof |
| CN111462161A (en) * | 2020-03-31 | 2020-07-28 | 厦门亿联网络技术股份有限公司 | System, method, storage medium and equipment for extracting real-time video portrait |
| CN112070665A (en) * | 2020-08-03 | 2020-12-11 | 武汉大学 | Method for generating confrontation type video super-resolution reconstruction and identifying authenticity of reconstructed image |
| CN112527525A (en) * | 2020-12-11 | 2021-03-19 | 广州伊智信息科技有限公司 | Distributed event bus processing method based on message queue, terminal and medium |
| US11106919B1 (en) * | 2020-06-02 | 2021-08-31 | ULTINOUS Zrt. | Processing of video streams |
| CN114723760A (en) * | 2022-05-19 | 2022-07-08 | 北京世纪好未来教育科技有限公司 | Portrait segmentation model training method and device and portrait segmentation method and device |
| CN114915783A (en) * | 2021-02-07 | 2022-08-16 | 华为技术有限公司 | Encoding method and apparatus |
-
2022
- 2022-09-09 CN CN202211101542.9A patent/CN115567720A/en active Pending
Patent Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109803112A (en) * | 2017-11-16 | 2019-05-24 | 中兴通讯股份有限公司 | Video analysis management method based on big data, apparatus and system, storage medium |
| CN111400326A (en) * | 2020-02-28 | 2020-07-10 | 深圳市赛为智能股份有限公司 | Smart city data management system and method thereof |
| CN111462161A (en) * | 2020-03-31 | 2020-07-28 | 厦门亿联网络技术股份有限公司 | System, method, storage medium and equipment for extracting real-time video portrait |
| US11106919B1 (en) * | 2020-06-02 | 2021-08-31 | ULTINOUS Zrt. | Processing of video streams |
| CN112070665A (en) * | 2020-08-03 | 2020-12-11 | 武汉大学 | Method for generating confrontation type video super-resolution reconstruction and identifying authenticity of reconstructed image |
| CN112527525A (en) * | 2020-12-11 | 2021-03-19 | 广州伊智信息科技有限公司 | Distributed event bus processing method based on message queue, terminal and medium |
| CN114915783A (en) * | 2021-02-07 | 2022-08-16 | 华为技术有限公司 | Encoding method and apparatus |
| CN114723760A (en) * | 2022-05-19 | 2022-07-08 | 北京世纪好未来教育科技有限公司 | Portrait segmentation model training method and device and portrait segmentation method and device |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| EP3777207B1 (en) | Content-specific neural network distribution | |
| WO2018150083A1 (en) | A method and technical equipment for video processing | |
| US11410275B2 (en) | Video coding for machine (VCM) based system and method for video super resolution (SR) | |
| US20210150769A1 (en) | High efficiency image and video compression and decompression | |
| EP4373086A1 (en) | Image processing method and apparatus, medium, and electronic device | |
| US20200380261A1 (en) | Resource optimization based on video frame analysis | |
| CN110099280B (en) | Video service quality enhancement method under limitation of wireless self-organizing network bandwidth | |
| CN113473126B (en) | Video stream processing method and device, electronic equipment and computer readable medium | |
| CN112565777A (en) | Deep learning model-based video data transmission method, system, medium and device | |
| WO2023174144A1 (en) | Super-resolution reconstruction method and apparatus for cloud desktop image, device, and storage medium | |
| CN115733991A (en) | Video coding method and system with reinforcement learning rendering perception bit rate control | |
| Zhu et al. | A semantic-aware transmission with adaptive control scheme for volumetric video service | |
| WO2023124461A1 (en) | Video coding/decoding method and apparatus for machine vision task, device, and medium | |
| CN116567246A (en) | AVC coding method and device | |
| CN115567720A (en) | Video transmission method, device, storage medium and equipment | |
| WO2022100173A1 (en) | Video frame compression method and apparatus, and video frame decompression method and apparatus | |
| CN113709517A (en) | Online video stream processing system based on weak terminal | |
| CN111491166A (en) | Dynamic compression system and method based on content analysis | |
| CN116668702B (en) | Video coding method, device, terminal equipment and storage medium | |
| US11647153B1 (en) | Computer-implemented method, device, and computer program product | |
| CN110868615B (en) | Video processing method and device, electronic equipment and storage medium | |
| US20230254592A1 (en) | System and method for reducing transmission bandwidth in edge cloud systems | |
| CN115426519A (en) | Method and system for playing H265 video in browser | |
| CN118138703A (en) | Video data stream soft coding method, device, equipment and medium | |
| CN116419032A (en) | Video playing method, device, equipment and computer readable storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |