CN114866806A - Streaming improvement method and device applying video and audio preprocessing and storage medium - Google Patents
Streaming improvement method and device applying video and audio preprocessing and storage medium Download PDFInfo
- Publication number
- CN114866806A CN114866806A CN202210461886.4A CN202210461886A CN114866806A CN 114866806 A CN114866806 A CN 114866806A CN 202210461886 A CN202210461886 A CN 202210461886A CN 114866806 A CN114866806 A CN 114866806A
- Authority
- CN
- China
- Prior art keywords
- streaming
- preprocessing
- data source
- video
- video data
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000007781 pre-processing Methods 0.000 title claims abstract description 143
- 238000000034 method Methods 0.000 title claims abstract description 52
- 230000006872 improvement Effects 0.000 title claims abstract description 19
- 230000008569 process Effects 0.000 claims abstract description 19
- 239000012634 fragment Substances 0.000 claims abstract description 16
- 238000004806 packaging method and process Methods 0.000 claims abstract description 8
- 238000012545 processing Methods 0.000 claims description 12
- 230000009466 transformation Effects 0.000 claims description 11
- 238000001914 filtration Methods 0.000 claims description 9
- 230000008802 morphological function Effects 0.000 claims description 3
- 238000010276 construction Methods 0.000 claims description 2
- 238000012544 monitoring process Methods 0.000 description 11
- 238000013473 artificial intelligence Methods 0.000 description 9
- 230000005540 biological transmission Effects 0.000 description 5
- 238000003062 neural network model Methods 0.000 description 4
- 230000008878 coupling Effects 0.000 description 3
- 238000010168 coupling process Methods 0.000 description 3
- 238000005859 coupling reaction Methods 0.000 description 3
- 238000010586 diagram Methods 0.000 description 3
- 238000013528 artificial neural network Methods 0.000 description 2
- 238000004891 communication Methods 0.000 description 2
- 238000005516 engineering process Methods 0.000 description 2
- PXFBZOLANLWPMH-UHFFFAOYSA-N 16-Epiaffinine Natural products C1C(C2=CC=CC=C2N2)=C2C(=O)CC2C(=CC)CN(C)C1C2CO PXFBZOLANLWPMH-UHFFFAOYSA-N 0.000 description 1
- 238000005260 corrosion Methods 0.000 description 1
- 230000007797 corrosion Effects 0.000 description 1
- 238000000605 extraction Methods 0.000 description 1
- 238000003780 insertion Methods 0.000 description 1
- 230000037431 insertion Effects 0.000 description 1
- 238000007689 inspection Methods 0.000 description 1
- 239000011159 matrix material Substances 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000013139 quantization Methods 0.000 description 1
- 229910052704 radon Inorganic materials 0.000 description 1
- SYUHGPGVQRZVTB-UHFFFAOYSA-N radon atom Chemical compound [Rn] SYUHGPGVQRZVTB-UHFFFAOYSA-N 0.000 description 1
- 238000013519 translation Methods 0.000 description 1
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
- H04N21/23—Processing of content or additional data; Elementary server operations; Server middleware
- H04N21/234—Processing of video elementary streams, e.g. splicing of video streams, manipulating MPEG-4 scene graphs
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
- H04N21/23—Processing of content or additional data; Elementary server operations; Server middleware
- H04N21/234—Processing of video elementary streams, e.g. splicing of video streams, manipulating MPEG-4 scene graphs
- H04N21/23418—Processing of video elementary streams, e.g. splicing of video streams, manipulating MPEG-4 scene graphs involving operations for analysing video streams, e.g. detecting features or characteristics
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
- H04N21/23—Processing of content or additional data; Elementary server operations; Server middleware
- H04N21/234—Processing of video elementary streams, e.g. splicing of video streams, manipulating MPEG-4 scene graphs
- H04N21/2343—Processing of video elementary streams, e.g. splicing of video streams, manipulating MPEG-4 scene graphs involving reformatting operations of video signals for distribution or compliance with end-user requests or end-user device requirements
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
- H04N21/435—Processing of additional data, e.g. decrypting of additional data, reconstructing software from modules extracted from the transport stream
- H04N21/4351—Processing of additional data, e.g. decrypting of additional data, reconstructing software from modules extracted from the transport stream involving reassembling additional data, e.g. rebuilding an executable program from recovered modules
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
- Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management
Abstract
The invention relates to a streaming improvement method, a streaming improvement device and a storage medium applying audio and video preprocessing. The invention receives and caches the original video data of the streaming data source; forming a corresponding preprocessing strategy based on preprocessing configuration aiming at any streaming data source, and preprocessing the original video data of the streaming data source through the preprocessing strategy to form preprocessed video data; the method comprises the following steps that all preprocessing configuration items, preprocessing configuration item parameters and the matching relation among the configuration items in preprocessing configuration are set according to a video preprocessing process required by a video receiver of a streaming data source; packaging fragments of original video data and/or preprocessed video data fragments of related streaming data sources into a plurality of streaming packets according to streaming mode requests of a video receiver aiming at least one streaming data source to form corresponding streaming; the video receiver receives the original video data and/or the preprocessed video data required by the parsing streaming.
Description
Technical Field
The present invention relates to the field of streaming technologies, and in particular, to a streaming improvement method and apparatus applying audio and video preprocessing, and a storage medium.
Background
The current audio and video streaming technology is widely applied to aspects of monitoring scenes, video software, live broadcast software, artificial intelligence and the like, and is used for supporting a streaming data source to transmit video data to a video receiver in real time.
The video receiver has different requirements on the quality of streaming video data and the format of video frames under different scenes. For example, in a monitoring scene, when video data of multiple streaming data sources are integrally displayed, multiple channels of streaming data are required, which is limited by network bandwidth, and the quality of video transmitted by each streaming data is often low, and when video data of a certain streaming data source is required to be displayed in the monitoring scene, the streaming data source needs to receive streaming of high-quality video data transmitted by the streaming data source. For example, in video software or live broadcast software, a plurality of widgets are often required to dynamically display previews of each video or live broadcast, at this time, video data required by the widgets are often required to be compressed and have relatively low quality, and the video data required by the video windows after video click-in or live broadcast has relatively high quality. For example, in the field of artificial intelligence, after an original video is acquired, operation equipment of an artificial intelligence neural network is operated, and before various models are input, certain preprocessing is carried out on a video frame of original video data, so that the processed video data meet the input requirements of the models. At present, in order to meet the needs of different video receivers, the pixels of original video data of a streaming data source are often higher, and the pixels of the original video data are larger. If the video receiver needs relatively low-quality video data and transmits high-quality video data to the video receiver, network transmission pressure is brought to the video receiver. In the field of artificial intelligence, besides network transmission pressure, the computing equipment also needs to preprocess video data while operating an artificial intelligence neural network, and computing resources of the computing equipment are divided by preprocessing, so that the artificial intelligence neural network model is low in resource occupation ratio, and low in analysis efficiency and poor in real-time performance.
Disclosure of Invention
To solve the above technical problems or at least partially solve the above technical problems, the present invention provides a streaming improvement method, device and storage medium applying video and audio preprocessing.
In a first aspect, the present invention provides a streaming improvement method for applying audio-video preprocessing, including: receiving and caching original video data of a streaming data source;
forming a corresponding preprocessing strategy based on preprocessing configuration aiming at any streaming data source, and preprocessing the original video data of the streaming data source through the preprocessing strategy to form preprocessed video data; the method comprises the following steps that all preprocessing configuration items, preprocessing configuration item parameters and the matching relation among the configuration items in preprocessing configuration are set according to a video preprocessing process required by a video receiver of a streaming data source;
packaging fragments of original video data and/or preprocessed video data fragments of related streaming data sources into a plurality of streaming packets according to streaming mode requests of a video receiver aiming at least one streaming data source to form corresponding streaming;
the video receiver receives the original video data and/or the preprocessed video data required by the parsing streaming.
Further, a preprocessing configuration item function library for recording the preprocessing configuration item function is created in advance;
according to any preprocessing configuration item in any preprocessing configuration aiming at the streaming data source, calling a corresponding preprocessing configuration item function from a preprocessing configuration item function library, and transmitting a preprocessing configuration item parameter of the preprocessing configuration item to the called preprocessing configuration item function; and constructing the relation between the called pretreatment configuration item functions based on the matching relation between the pretreatment configuration items in the pretreatment configuration to form a pretreatment strategy.
Further, the preprocessing configuration functions for implementing the preprocessing configuration items include spatial domain transformation functions of the video frame images, morphological functions of the video frame images, noise filtering processing functions, image transformation functions, frame processing functions, and video frame image gray scale processing functions.
Furthermore, the streaming mode request is configured with at least one stream data source, a stream identifier associated with a single stream data source, and video receiver information, wherein the stream identifier is at least one of a first stream identifier, a second stream identifier, and a third stream identifier indicating a streaming mode;
and controlling a stream data source to stream according to the stream mode indicated by the stream identifier of the stream data source.
Furthermore, if the stream identifier associated with a stream data source in a stream mode request is a first stream identifier, packaging the original video data of the stream data source into a stream and sending the stream to the video receiver in the stream mode request;
if the streaming identifier associated with a streaming data source in the streaming mode request is a second streaming identifier, packaging the pre-processed video data of the streaming data source into streaming and sending the streaming to a video receiver in the streaming mode request;
if the streaming identifier associated with a streaming data source in the streaming mode request is the third streaming identifier, the original video data and the preprocessed video data of the streaming data source are packaged into a streaming and sent to the video receiver in the streaming mode request.
Furthermore, when the original video data and the preprocessed video data of a stream data source are packaged in the same stream, an original video data segment of the original video data is packaged in a main packet, the preprocessed video data segment corresponding to the original data segment is packaged in a custom packet which is immediately packaged in the main packet of the original data segment, and the main packet and the custom packet are sequentially connected in series to form a whole which comprises the original video data and the preprocessed video data stream.
In a second aspect, the present invention provides an apparatus for improving video streaming, comprising:
at least one streaming data source, said streaming data source producing raw video data;
the preprocessing module is coupled with the streaming data source, receives original video data generated by the cache streaming data source, and preprocesses the original video data by utilizing a target preprocessing strategy aiming at the target streaming data source to form preprocessed video data;
and the streaming module encapsulates the fragments of the original video data and/or the preprocessed video data fragments of the related streaming data source into a plurality of streaming packets according to the streaming mode request of at least one video receiver aiming at least one streaming data source, forms a corresponding streaming and sends the corresponding streaming to the corresponding video receiver coupled with the streaming module.
Furthermore, the preprocessing strategy building module is coupled with the preprocessing module, and the preprocessing strategy building module calls a corresponding preprocessing configuration item function from the preprocessing configuration item function library according to any preprocessing configuration item in any preprocessing configuration item and transfers the preprocessing configuration item parameter of the preprocessing configuration item to the called preprocessing configuration item function; and constructing the relation between the called pretreatment configuration item functions based on the matching relation between the pretreatment configuration items in the pretreatment configuration to form a pretreatment strategy.
In a third aspect, the present invention provides a storage medium for implementing a streaming improvement method using audio-video preprocessing, where the storage medium for implementing the streaming improvement method using audio-video preprocessing stores at least one instruction, and reads and executes the instruction to implement the streaming improvement method using audio-video preprocessing.
Compared with the prior art, the technical scheme provided by the embodiment of the invention has the following advantages:
in the invention, a corresponding preprocessing strategy is formed based on preprocessing configuration aiming at any streaming data source, and the preprocessing strategy is utilized to preprocess the original video data of the streaming data source to form preprocessed video data. The pre-processing configuration is based on the video pre-processing procedure settings required by the video recipient. And during streaming, according to a streaming mode request sent by a video receiver, encapsulating the fragments of the original video data and/or the preprocessed video data fragments of the related streaming data source in a plurality of streaming packets to form corresponding streaming, and receiving and analyzing the streaming by the video receiver to obtain the required original video data and/or the preprocessed video data. The video receiver controls the streaming mode based on the requirement, and only the preprocessed video data processed by the corresponding preprocessing strategy is streamed when the video data with relatively low quality is required, so that the data transmission quantity between the streaming data source and the video receiver can be effectively reduced, and the network resource is saved. For the video receiver, the preprocessing process is decoupled from the video receiver, so that the operation resource of the video receiver can be saved.
Drawings
The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the invention and together with the description, serve to explain the principles of the invention.
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below, and it is obvious for those skilled in the art that other drawings can be obtained according to the drawings without inventive exercise.
Fig. 1 is a flowchart of a streaming improvement method using audio/video preprocessing according to an embodiment of the present invention;
FIG. 2 is a flow diagram providing a process for constructing a pre-processing policy based on a pre-processing configuration, according to an embodiment of the present invention;
fig. 3 is a schematic diagram of streaming packets including original video data and pre-processed video data according to an embodiment of the present invention;
fig. 4 is a schematic diagram of an apparatus for improving audio/video streaming based on a streaming improvement method using audio/video preprocessing according to an embodiment of the present invention.
Detailed Description
In order to make the objects, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are some, but not all, embodiments of the present invention. All other embodiments, which can be obtained by a person skilled in the art without any inventive step based on the embodiments of the present invention, are within the scope of the present invention.
It should be noted that, in this document, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising an … …" does not exclude the presence of other identical elements in a process, method, article, or apparatus that comprises the element.
Example 1
Referring to fig. 1, an embodiment of the invention provides a streaming improvement method using audio/video preprocessing, including:
s100, receiving and caching original video data of a streaming data source. The image is a two-dimensional continuous function, when the electronic device digitally processes the image, the image is digitized in space and brightness, the spatial digitization discretizes the spatial coordinates of the image to form pixel squares, the brightness is digitized by taking the average value of the image brightness on the pixel squares as the value of the squares, and a layered quantization mode is often adopted in the brightness digitization process, namely, the brightness is mapped to a finite value from a darkest value to a brightest value. Thus, a digital image is essentially an M × N matrix. For an image of the same size, the smaller the pixel grid, the higher the resolution of the digital image and the sharper the digital image, and the more data is contained. For an image of the same size and resolution, the more luminance gradation values, the smoother the image, and the larger the amount of data contained therein. The original video data resolution of streaming data sources tends to be high and contain a relatively large amount of data.
S200, forming a corresponding preprocessing strategy based on the preprocessing configuration aiming at any streaming data source. And the matching relation among the various preprocessing configuration items, the parameters of the preprocessing configuration items and the various configuration items in the preprocessing configuration is set according to the video preprocessing process required by the video receiver of the streaming data source.
In a specific implementation process, referring to fig. 2, the process of forming a corresponding preprocessing policy based on the preprocessing configuration for any streaming data source includes:
s201, a preprocessing configuration item function library for recording preprocessing configuration item functions is created in advance.
S202, according to any preprocessing configuration item in any preprocessing configuration aiming at the streaming data source, calling a corresponding preprocessing configuration item function from a preprocessing configuration item function library. In the specific implementation process, a preprocessing configuration is created based on the requirement of a video receiver, and the preprocessing configuration comprises at least one preprocessing configuration item, parameters of the preprocessing configuration item and the matching relationship among the configuration items.
Specifically, the preprocessing configuration function for implementing the preprocessing configuration item includes:
spatial domain transformation function of video frame image, such as scaling function, rotation function, clipping function, turning function, translation function, affine transformation function, projection transformation function, and box-type transformation function of video frame image.
Morphological functions of the video frame image, such as corrosion functions, expansion functions, distance transformation functions and image neighborhood operation block operation functions.
Noise filtering processing functions such as spatial filtering template (laplacian template, Sobel template, Prewitt template, and Roboert template) functions, frequency domain filtering (low pass filtering, high pass filtering, band stop filtering, and homomorphic filtering) functions.
Image transformation functions such as fourier transform functions, discrete cosine transform functions, Radon transform functions, Hough transform functions, Fan-Bean projection transform functions, and wavelet transform functions.
And frame processing functions, such as frame extraction processing function and frame insertion processing function.
The video frame image gray processing function, such as interpolation function for adjusting gray level, and graying function.
The preprocessing configuration function library is designed into an expandable library, and functions provided by a video receiver developer can be added into the preprocessing configuration function library after safety inspection, so that the preprocessing configuration function library can support the preprocessing configuration of a more video receiver.
S203, transmitting the parameters of the preprocessing configuration items to the called preprocessing configuration item function;
s204, establishing the relation between the called pretreatment configuration item functions based on the matching relation between the pretreatment configuration items in the pretreatment configuration, and forming a pretreatment strategy. The coordination relationship between the preprocessing configuration items includes but is not limited to sequential execution, loop nesting, and invocation.
S300, preprocessing the original video data of the streaming data source by a preprocessing strategy aiming at the streaming data source to form preprocessed video data.
S400, the video receiver initiates a streaming mode request aiming at the streaming data source according to the requirement. The streaming mode request is configured with at least one streaming data source, a stream identifier associated with a single streaming data source and video receiver information, wherein the stream identifier is at least one of a first stream identifier, a second stream identifier and a third stream identifier indicating a streaming mode; according to the streaming mode request, a streaming data source is made to stream in the streaming mode indicated by the streaming identification of the streaming data source.
S500, according to the streaming mode request of the video receiver aiming at least one streaming data source, the fragments of the original video data and/or the preprocessed video data fragments of the related streaming data source are packaged in a plurality of streaming packets to form the corresponding streaming.
Specifically, if the streaming identifier associated with a streaming data source is the first streaming identifier in the streaming mode request, the original video data of the streaming data source is encapsulated into streaming and sent to the video receiver in the streaming mode request;
if the streaming identifier associated with a streaming data source in the streaming mode request is a second streaming identifier, packaging the pre-processed video data of the streaming data source into streaming and sending the streaming to a video receiver in the streaming mode request;
if the stream identifier associated with the stream data source is the third stream identifier in the stream mode request, packaging the original video data and the preprocessed video data of the stream data source into a stream and sending the stream to the video receiver in the stream mode request. Referring to fig. 3, when the original video data and the pre-processed video data of a stream data source are encapsulated in the same stream, an original video data segment of the original video data is encapsulated in a main packet, a pre-processed video data segment corresponding to the original data segment is encapsulated in a custom packet immediately encapsulating the original data segment main packet, and the main packet and the custom packet are sequentially connected in series to form a stream including the original video data and the pre-processed video data.
S600, the video receiver receives and analyzes the streaming to obtain the required original video data and/or the required preprocessed video data.
Example 2
In this embodiment, the embodiment 1 is applied to a monitoring scene, a video scaling configuration item is included in the preprocessing configuration of the monitoring scene, and a scaling function of a video frame image in a spatial transform function of the video frame image is called to process original video data of a streaming data source based on the video scaling configuration item. When a video receiver of the monitoring scene displays videos of multiple streaming data sources, the monitoring scene sends a streaming mode request containing a second streaming data identifier, and receives a preprocessed video image for reducing original video data of the streaming data sources. When a video receiver of a monitoring scene independently shows videos of a streaming data source, the monitoring scene sends a streaming mode request containing a first streaming flag, and receives original video data of the independently shown streaming data source.
Example 3
In this embodiment, the embodiment 1 is applied to video or live broadcast software, the preprocessing configuration of the monitoring scene includes a video scaling configuration item, and the scaling function of the video frame image in the spatial domain transformation function of the video frame image is called to process the original video data of the streaming data source based on the video scaling configuration item. When the video or live broadcast software displays the video preview of the streaming data source through the small window, a video receiver of the video or live broadcast software sends a streaming mode request containing a second streaming flag and receives a preprocessed video image for reducing original video data of the streaming data source. When a video receiver of video or live broadcast software enters a video or live broadcast page, a monitoring scene sends a streaming mode request containing a first streaming data mark and receives original video data of a streaming data source.
Example 4
In this embodiment, the embodiment 1 is applied to a scene of an artificial intelligence analysis video, and the preprocessing configuration of the scene of the artificial intelligence analysis video is set according to the preprocessing process of the video frame image required by the artificial intelligence neural network model. And the arithmetic device of the artificial intelligent neural network model sends a streaming mode request containing a second streaming identifier, receives the preprocessed video data and inputs the preprocessed video data into the artificial intelligent neural network model for operation.
Example 5
Referring to fig. 4, an embodiment of the present application provides an audio/video streaming improvement apparatus based on a streaming improvement method using audio/video preprocessing, including:
at least one streaming data source, the streaming data source generating raw video data.
And the preprocessing module is coupled with the streaming data source, receives original video data generated by the cache streaming data source, and preprocesses the original video data by utilizing a target preprocessing strategy aiming at the target streaming data source to form preprocessed video data.
And the streaming module is coupled with the streaming data source and the preprocessing module, and the streaming module is used for requesting to package fragments of original video data and/or preprocessed video data fragments of the related streaming data source into a plurality of streaming packets according to the streaming mode of at least one video receiver aiming at the streaming data source so as to form corresponding streaming and send the corresponding streaming to the corresponding video receiver coupled with the streaming module.
The preprocessing strategy construction module is coupled with the preprocessing module, calls a corresponding preprocessing configuration item function from a preprocessing configuration item function library according to any preprocessing configuration item in any preprocessing configuration item, and transmits the preprocessing configuration item parameter of the preprocessing configuration item to the called preprocessing configuration item function; and constructing the relation between the called pretreatment configuration item functions based on the matching relation between the pretreatment configuration items in the pretreatment configuration to form a pretreatment strategy.
Example 6
The embodiment of the application provides a storage medium for realizing a streaming improving method for applying audio-video preprocessing, the storage medium for realizing the streaming improving method for applying audio-video preprocessing is provided with at least one transmission interface, stores at least one instruction, and reads and executes the instruction from the transmission interface to realize the streaming improving method for applying audio-video preprocessing.
In the embodiments provided in the present invention, it should be understood that the disclosed structures may be implemented in other manners. For example, the above-described structural embodiments are merely illustrative, and for example, the division of the units is only one type of logical functional division, and there may be other divisions when the actual implementation is performed, for example, a plurality of units or components may be combined or integrated into another system, or some features may be omitted, or not executed. In addition, the shown or discussed mutual coupling or direct coupling or communication connection may be an indirect coupling or communication connection through some interfaces, structures or units, and may be in an electrical, mechanical or other form.
The units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of the embodiment.
In addition, functional units in the embodiments of the present invention may be integrated into one processing unit, or each unit may exist alone physically, or two or more units are integrated into one unit. The integrated unit can be realized in a form of hardware, and can also be realized in a form of a software functional unit.
The foregoing are merely exemplary embodiments of the present invention, which enable those skilled in the art to understand or practice the present invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the generic principles defined herein may be applied to other embodiments without departing from the spirit or scope of the invention. Thus, the present invention is not intended to be limited to the embodiments shown herein but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.
Claims (9)
1. A streaming improvement method using audio-video preprocessing, comprising: receiving and caching original video data of a streaming data source;
forming a corresponding preprocessing strategy based on preprocessing configuration aiming at any streaming data source, and preprocessing original video data of the streaming data source through the preprocessing strategy to form preprocessed video data; the method comprises the following steps that all preprocessing configuration items, preprocessing configuration item parameters and the matching relation among the configuration items in preprocessing configuration are set according to a video preprocessing process required by a video receiver of a streaming data source;
packaging fragments of original video data and/or preprocessed video data fragments of related streaming data sources into a plurality of streaming packets according to streaming mode requests of a video receiver aiming at least one streaming data source to form corresponding streaming;
the video receiver receives the original video data and/or the preprocessed video data required by the parsing streaming.
2. The streaming enhancement method of claim 1, wherein a pre-process profile function library is created in advance, where the pre-process profile function is recorded;
according to any preprocessing configuration item in any preprocessing configuration aiming at the streaming data source, calling a corresponding preprocessing configuration item function from a preprocessing configuration item function library, and transmitting a preprocessing configuration item parameter of the preprocessing configuration item to the called preprocessing configuration item function; and constructing the relation between the called pretreatment configuration item functions based on the matching relation between the pretreatment configuration items in the pretreatment configuration to form a pretreatment strategy.
3. The method of claim 2, wherein the pre-processing configuration function for implementing pre-processing configuration items comprises spatial transformation function of video frame images, morphological function of video frame images, noise filtering processing function, image transformation function, frame processing function, and video frame image gray scale processing function.
4. The method of claim 1, wherein the streaming request includes at least one streaming data source, a stream identifier associated with a single streaming data source, and video receiver information, wherein the stream identifier is at least one of a first stream identifier, a second stream identifier, and a third stream identifier indicating a streaming mode;
according to the streaming mode request, a streaming data source is made to stream in the streaming mode indicated by the streaming identification of the streaming data source.
5. The method of claim 4, wherein if the stream id associated with a stream data source in a streaming request is the first stream id, the original video data of the stream data source is encapsulated as a stream and sent to the video receiver in the streaming request;
if the streaming identifier associated with a streaming data source in the streaming mode request is a second streaming identifier, packaging the pre-processed video data of the streaming data source into streaming and sending the streaming to a video receiver in the streaming mode request;
if the streaming identifier associated with a streaming data source in the streaming mode request is the third streaming identifier, the original video data and the preprocessed video data of the streaming data source are packaged into a streaming and sent to the video receiver in the streaming mode request.
6. The method of claim 5, wherein when the original video data and the pre-processed video data of a stream data source are encapsulated in a same stream, an original video data segment of the original video data is encapsulated in a host packet, a pre-processed video data segment corresponding to the original video data segment is encapsulated in a custom packet next to the host packet encapsulating the original video data segment, and the host packet and the custom packet are sequentially concatenated to form a stream comprising the original video data and the pre-processed video data.
7. An audio/video stream enhancement apparatus based on the stream enhancement method of any one of claims 1 to 6, comprising:
at least one streaming data source, said streaming data source producing raw video data;
the preprocessing module is coupled with the streaming data source, receives original video data generated by the cache streaming data source, and preprocesses the original video data by utilizing a target preprocessing strategy aiming at the target streaming data source to form preprocessed video data;
and the streaming module encapsulates the fragments of the original video data and/or the preprocessed video data fragments of the related streaming data source into a plurality of streaming packets according to the streaming mode request of at least one video receiver aiming at least one streaming data source, forms a corresponding streaming and sends the corresponding streaming to the corresponding video receiver coupled with the streaming module.
8. The video streaming enhancement device of claim 7, comprising: the preprocessing strategy construction module is coupled with the preprocessing module, calls a corresponding preprocessing configuration item function from a preprocessing configuration item function library according to any preprocessing configuration item in any preprocessing configuration item, and transmits the preprocessing configuration item parameter of the preprocessing configuration item to the called preprocessing configuration item function; and constructing the relation between the called pretreatment configuration item functions based on the matching relation between the pretreatment configuration items in the pretreatment configuration to form a pretreatment strategy.
9. A storage medium for implementing a streaming improvement method using audio-visual preprocessing, wherein the storage medium for implementing the streaming improvement method using audio-visual preprocessing stores at least one instruction, reads and executes the instruction to implement the streaming improvement method using audio-visual preprocessing according to any one of claims 1 to 6.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210461886.4A CN114866806B (en) | 2022-04-28 | 2022-04-28 | Streaming improvement method, device and storage medium applying video and audio preprocessing |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210461886.4A CN114866806B (en) | 2022-04-28 | 2022-04-28 | Streaming improvement method, device and storage medium applying video and audio preprocessing |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN114866806A true CN114866806A (en) | 2022-08-05 |
| CN114866806B CN114866806B (en) | 2023-07-18 |
Family
ID=82633409
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210461886.4A Active CN114866806B (en) | 2022-04-28 | 2022-04-28 | Streaming improvement method, device and storage medium applying video and audio preprocessing |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114866806B (en) |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102710963A (en) * | 2011-02-22 | 2012-10-03 | 仁宝电脑工业股份有限公司 | Method and system for adjusting video and audio quality of video stream |
| US20150208103A1 (en) * | 2012-08-08 | 2015-07-23 | National University Of Singapore | System and Method for Enabling User Control of Live Video Stream(s) |
| CN108124183A (en) * | 2016-11-29 | 2018-06-05 | 达升企业股份有限公司 | With it is synchronous obtain it is audio-visual to carry out the method for one-to-many video stream |
-
2022
- 2022-04-28 CN CN202210461886.4A patent/CN114866806B/en active Active
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102710963A (en) * | 2011-02-22 | 2012-10-03 | 仁宝电脑工业股份有限公司 | Method and system for adjusting video and audio quality of video stream |
| US20150208103A1 (en) * | 2012-08-08 | 2015-07-23 | National University Of Singapore | System and Method for Enabling User Control of Live Video Stream(s) |
| CN108124183A (en) * | 2016-11-29 | 2018-06-05 | 达升企业股份有限公司 | With it is synchronous obtain it is audio-visual to carry out the method for one-to-many video stream |
Also Published As
| Publication number | Publication date |
|---|---|
| CN114866806B (en) | 2023-07-18 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US8988502B2 (en) | System and method for transmission, processing, and rendering of stereoscopic and multi-view images | |
| US8606037B2 (en) | Techniques to improve contrast enhancement | |
| Chen et al. | A new journey from sdrtv to hdrtv | |
| US11445201B2 (en) | Video encoding and decoding method, device, and system, and storage medium | |
| US6597736B1 (en) | Throughput enhanced video communication | |
| US5724450A (en) | Method and system for color image compression in conjunction with color transformation techniques | |
| WO2023010754A1 (en) | Image processing method and apparatus, terminal device, and storage medium | |
| JP7359521B2 (en) | Image processing method and device | |
| JP2002044422A (en) | Image processor and processing method for generating low-resolution low bit depth image | |
| US10356404B1 (en) | Image processing using just-noticeable-difference thresholds | |
| JP2000504911A (en) | Facsimile compliant image compression method and system | |
| CN110740350B (en) | Image processing method, image processing device, terminal equipment and computer readable storage medium | |
| JP6401777B2 (en) | Send and receive composite images | |
| JP2022502955A (en) | Video encoding and decoding methods, and equipment | |
| CN111083496A (en) | Data processing method and related product | |
| WO2010113086A1 (en) | System and format for encoding data and three-dimensional rendering | |
| US9979985B2 (en) | Transmission device, transmitting method, reception device, and receiving method | |
| CN111083450A (en) | Vehicle-mounted-end image remote output method, device and system | |
| EP4018410A1 (en) | Watermark-based image reconstruction | |
| CN114866806A (en) | Streaming improvement method and device applying video and audio preprocessing and storage medium | |
| CN112399069B (en) | Image encoding method and apparatus, storage medium, and electronic device | |
| WO2023093240A1 (en) | Video data processing method, apparatus and device, and computer storage medium | |
| CN115567711A (en) | Desktop image dynamic acquisition method and device and computer readable storage medium | |
| JP2002512470A (en) | Method and apparatus for supporting a video protocol in a network environment | |
| CN108933945B (en) | GIF picture compression method, device and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |