CN114866806B - Streaming improvement method, device and storage medium applying video and audio preprocessing - Google Patents
Streaming improvement method, device and storage medium applying video and audio preprocessing Download PDFInfo
- Publication number
- CN114866806B CN114866806B CN202210461886.4A CN202210461886A CN114866806B CN 114866806 B CN114866806 B CN 114866806B CN 202210461886 A CN202210461886 A CN 202210461886A CN 114866806 B CN114866806 B CN 114866806B
- Authority
- CN
- China
- Prior art keywords
- streaming
- preprocessing
- video
- video data
- data source
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000007781 pre-processing Methods 0.000 title claims abstract description 153
- 238000000034 method Methods 0.000 title claims abstract description 47
- 230000006872 improvement Effects 0.000 title claims abstract description 23
- 230000008569 process Effects 0.000 claims abstract description 20
- 239000012634 fragment Substances 0.000 claims abstract description 14
- 238000012545 processing Methods 0.000 claims description 13
- 238000004806 packaging method and process Methods 0.000 claims description 9
- 238000010276 construction Methods 0.000 claims description 6
- 230000008802 morphological function Effects 0.000 claims description 3
- 238000012544 monitoring process Methods 0.000 description 11
- 230000009466 transformation Effects 0.000 description 10
- 238000013473 artificial intelligence Methods 0.000 description 6
- 230000005540 biological transmission Effects 0.000 description 5
- 238000013528 artificial neural network Methods 0.000 description 3
- 230000008878 coupling Effects 0.000 description 3
- 238000010168 coupling process Methods 0.000 description 3
- 238000005859 coupling reaction Methods 0.000 description 3
- 238000003062 neural network model Methods 0.000 description 3
- 238000004891 communication Methods 0.000 description 2
- 238000010586 diagram Methods 0.000 description 2
- 238000005516 engineering process Methods 0.000 description 2
- PXFBZOLANLWPMH-UHFFFAOYSA-N 16-Epiaffinine Natural products C1C(C2=CC=CC=C2N2)=C2C(=O)CC2C(=CC)CN(C)C1C2CO PXFBZOLANLWPMH-UHFFFAOYSA-N 0.000 description 1
- 230000010339 dilation Effects 0.000 description 1
- 230000003628 erosive effect Effects 0.000 description 1
- 238000000605 extraction Methods 0.000 description 1
- 238000001914 filtration Methods 0.000 description 1
- 238000003780 insertion Methods 0.000 description 1
- 230000037431 insertion Effects 0.000 description 1
- 238000007689 inspection Methods 0.000 description 1
- 239000011159 matrix material Substances 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 238000013138 pruning Methods 0.000 description 1
- 229910052704 radon Inorganic materials 0.000 description 1
- SYUHGPGVQRZVTB-UHFFFAOYSA-N radon atom Chemical compound [Rn] SYUHGPGVQRZVTB-UHFFFAOYSA-N 0.000 description 1
- 238000013519 translation Methods 0.000 description 1
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
- H04N21/23—Processing of content or additional data; Elementary server operations; Server middleware
- H04N21/234—Processing of video elementary streams, e.g. splicing of video streams or manipulating encoded video stream scene graphs
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
- H04N21/23—Processing of content or additional data; Elementary server operations; Server middleware
- H04N21/234—Processing of video elementary streams, e.g. splicing of video streams or manipulating encoded video stream scene graphs
- H04N21/23418—Processing of video elementary streams, e.g. splicing of video streams or manipulating encoded video stream scene graphs involving operations for analysing video streams, e.g. detecting features or characteristics
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
- H04N21/23—Processing of content or additional data; Elementary server operations; Server middleware
- H04N21/234—Processing of video elementary streams, e.g. splicing of video streams or manipulating encoded video stream scene graphs
- H04N21/2343—Processing of video elementary streams, e.g. splicing of video streams or manipulating encoded video stream scene graphs involving reformatting operations of video signals for distribution or compliance with end-user requests or end-user device requirements
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
- H04N21/435—Processing of additional data, e.g. decrypting of additional data, reconstructing software from modules extracted from the transport stream
- H04N21/4351—Processing of additional data, e.g. decrypting of additional data, reconstructing software from modules extracted from the transport stream involving reassembling additional data, e.g. rebuilding an executable program from recovered modules
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
- Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management
Landscapes
- Engineering & Computer Science (AREA)
- Multimedia (AREA)
- Signal Processing (AREA)
- Information Transfer Between Computers (AREA)
- Two-Way Televisions, Distribution Of Moving Picture Or The Like (AREA)
Abstract
The invention relates to a streaming improvement method, a streaming improvement device and a storage medium applying video preprocessing. The invention receives and caches the original video data of the streaming data source; forming a corresponding preprocessing strategy based on preprocessing configuration aiming at any streaming data source, and preprocessing the streaming data source original video data through the preprocessing strategy to form preprocessed video data; wherein, each preprocessing configuration item, preprocessing configuration item parameters and coordination relation among each configuration item in preprocessing configuration are set according to the video preprocessing process required by the video receiver of the streaming data source; according to a streaming mode request of a video receiver for at least one streaming data source, encapsulating fragments of original video data and/or preprocessed video data fragments of related streaming data sources in a plurality of streaming packets to form corresponding streaming; the video receiver receives the original video data and/or the preprocessed video data required by analyzing the streaming acquisition.
Description
Technical Field
The present invention relates to the field of streaming technologies, and in particular, to a streaming improvement method, apparatus and storage medium applying audio/video preprocessing.
Background
The current streaming technology of audio and video is widely applied to the aspects of monitoring scenes, video software, live broadcast software, artificial intelligence and the like, and is used for supporting a streaming data source to transmit video data to a video receiver in real time.
The video receiver has different requirements on the quality of the video data and the video frame format of the streaming under different scenes. If video data of a multi-stream data source is displayed in a monitoring scene, multi-channel streaming is required, which is limited by network bandwidth, the video quality transmitted by each streaming is often lower, and when video data of a certain stream data source is required to be displayed in the monitoring scene, the streaming data source is required to be accepted to transmit high-quality video data. For example, in video software or live broadcast software, a plurality of small windows are often required to dynamically display previews of each video or live broadcast, at this time, video data required by the small windows often needs to be compressed and has relatively low quality, and after video clicking or live broadcast, video data required by the video windows needs to have relatively high quality. In the field of artificial intelligence, operation equipment of an artificial intelligence neural network is operated, and after an original video is acquired, a certain preprocessing is required to be performed on video frames of the original video data before various models are input, so that the processed video data meets the input requirements of the models. At present, in order to meet the needs of different video receivers, the pixels of the original video data are often higher, and the original video data are larger. If the video receiver needs relatively low-quality video data, but transmits high-quality video data to the video receiver, network transmission pressure is brought to the video receiver. In addition to network transmission pressure, the computing equipment needs to preprocess video data while running the artificial intelligent neural network, and computing resources of the computing equipment are divided by preprocessing, so that the computing resources of the computing equipment are divided into a small proportion of model resources of the artificial intelligent neural network, and analysis efficiency is low and real-time performance is poor.
Disclosure of Invention
In order to solve the above technical problems or at least partially solve the above technical problems, the present invention provides a streaming improvement method, apparatus and storage medium applying audio-visual preprocessing.
In a first aspect, the present invention provides a streaming improvement method applying audio-visual preprocessing, including: receiving and caching original video data of a streaming data source;
forming a corresponding preprocessing strategy based on preprocessing configuration aiming at any streaming data source, and preprocessing the streaming data source original video data through the preprocessing strategy to form preprocessed video data; wherein, each preprocessing configuration item, preprocessing configuration item parameters and coordination relation among each configuration item in preprocessing configuration are set according to the video preprocessing process required by the video receiver of the streaming data source;
according to a streaming mode request of a video receiver for at least one streaming data source, encapsulating fragments of original video data and/or preprocessed video data fragments of related streaming data sources in a plurality of streaming packets to form corresponding streaming;
the video receiver receives the original video data and/or the preprocessed video data required by analyzing the streaming acquisition.
Further, a preprocessing configuration item function library for recording preprocessing configuration item functions is created in advance;
according to any preprocessing configuration item in any preprocessing configuration aiming at the streaming data source, calling a corresponding preprocessing configuration item function from a preprocessing configuration item function library, and transmitting preprocessing configuration item parameters of the preprocessing configuration item to the called preprocessing configuration item function; and constructing the relation among the called preprocessing configuration item functions based on the coordination relation among the preprocessing configuration items in the preprocessing configuration to form a preprocessing strategy.
Still further, the preprocessing configuration function implementing the preprocessing configuration item includes a spatial domain transformation function of a video frame image, a morphological function of a video frame image, a noise filtering processing function, an image transformation function, a frame processing function, and a video frame image gray scale processing function.
Further, the streaming mode request is configured to include at least one streaming data source, a streaming identifier associated with a single streaming data source, and video receiver information, wherein the streaming identifier is at least one of a first streaming identifier, a second streaming identifier, and a third streaming identifier indicating a streaming mode;
controlling a streaming data source to stream according to the streaming mode indicated by the streaming identification.
Further, if in a streaming request, the streaming identifier associated with a streaming data source is the first streaming identifier, the original video data of the streaming data source is encapsulated into a streaming and sent to the video receiver in the streaming request;
if the streaming identifier associated with a streaming data source is a second streaming identifier in a streaming request, packaging the preprocessed video data of the streaming data source into a streaming to be sent to a video receiver in the streaming request;
if the stream identifier associated with a stream data source is the third stream identifier in a streaming request, the original video data and the preprocessed video data of the stream data source are packaged into a serial stream and sent to the video receiver in the streaming request.
Furthermore, when the original video data and the preprocessed video data of a streaming data source are packaged in the same streaming, an original video data segment of the original video data is packaged in a main packet, the preprocessed video data segment corresponding to the original data segment is packaged in a client packet immediately packaging the main packet of the original data segment, and the whole formed by the main packet and the client packet is sequentially connected in series to form a streaming comprising the original video data and the preprocessed video data.
In a second aspect, the present invention provides an apparatus for improving video streaming, comprising:
at least one streaming data source, the streaming data source generating raw video data;
the preprocessing module is coupled with the streaming data source, receives the original video data generated by the cache streaming data source, and preprocesses the original video data by utilizing a target preprocessing strategy aiming at the target streaming data source to form preprocessed video data;
and the streaming module is used for requesting the fragments of the original video data and/or the preprocessed video data of the related streaming data sources to be packaged in a plurality of streaming packets according to the streaming mode of at least one video receiver aiming at least one streaming data source, forming corresponding streaming and sending the corresponding streaming to the corresponding video receiver coupled with the streaming module.
Further, the preprocessing strategy construction module is coupled with the preprocessing module, and the preprocessing strategy construction module invokes a corresponding preprocessing configuration item function from the preprocessing configuration item function library according to any preprocessing configuration item in any preprocessing configuration, and transmits the preprocessing configuration item parameters of the preprocessing configuration item to the invoked preprocessing configuration item function; and constructing the relation among the called preprocessing configuration item functions based on the coordination relation among the preprocessing configuration items in the preprocessing configuration to form a preprocessing strategy.
In a third aspect, the present invention provides a storage medium for implementing a stream improvement method for applying audio/video preprocessing, where the storage medium implementing the stream improvement method for applying audio/video preprocessing stores at least one instruction, and reads and executes the instruction to implement the stream improvement method for applying audio/video preprocessing.
Compared with the prior art, the technical scheme provided by the embodiment of the invention has the following advantages:
in the invention, a corresponding preprocessing strategy is formed based on preprocessing configuration aiming at any streaming data source, and the preprocessing strategy is utilized to preprocess the original video data of the streaming data source to form preprocessed video data. The preprocessing configuration is based on the video preprocessing procedure settings required by the video receiver. And when in streaming, according to a streaming mode request sent by a video receiver, the fragments of the original video data and/or the preprocessed video data fragments of the related streaming data sources are packaged in a plurality of streaming packets to form corresponding streaming, and the video receiver receives the original video data and/or the preprocessed video data required by the streaming analysis. When relatively low-quality video data is needed, the video receiver only streams the preprocessed video data processed by the corresponding preprocessing strategy, so that the data transmission quantity between the streaming data source and the video receiver can be effectively reduced, and network resources are saved. For the video receiver, the preprocessing process is decoupled from the video receiver, so that the operation resources of the video receiver can be saved.
Drawings
The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the invention and together with the description, serve to explain the principles of the invention.
In order to more clearly illustrate the embodiments of the invention or the technical solutions of the prior art, the drawings which are used in the description of the embodiments or the prior art will be briefly described, and it will be obvious to a person skilled in the art that other drawings can be obtained from these drawings without inventive effort.
Fig. 1 is a flowchart of a stream improvement method using audio/video preprocessing according to an embodiment of the present invention;
FIG. 2 is a flow chart providing a method for constructing a preprocessing strategy based on a preprocessing configuration in accordance with an embodiment of the present invention;
FIG. 3 is a schematic diagram of a streaming packet including original video data and pre-processed video data according to an embodiment of the present invention;
fig. 4 is a schematic diagram of an audio/video stream improving apparatus based on a stream improving method applying audio/video preprocessing according to an embodiment of the present invention.
Detailed Description
For the purpose of making the objects, technical solutions and advantages of the embodiments of the present invention more apparent, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention, and it is apparent that the described embodiments are some embodiments of the present invention, but not all embodiments of the present invention. All other embodiments, which can be made by those skilled in the art based on the embodiments of the invention without making any inventive effort, are intended to be within the scope of the invention.
It should be noted that, in this document, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one … …" does not exclude the presence of other like elements in a process, method, article, or apparatus that comprises the element.
Example 1
Referring to fig. 1, an embodiment of the present invention provides a streaming improvement method applying audio/video preprocessing, including:
s100, receiving and caching the original video data of the streaming data source. The image is a two-dimensional continuous function, when the electronic equipment carries out digital processing on the image, the image is digitized in space and brightness, the space coordinate of the image is discretized by the digitization to form a pixel grid, the brightness is digitized by taking the average value of the image brightness on the pixel grid as the value of the grid, and the brightness is often quantized in a layering way in the digitizing process, namely, the brightness is mapped into a limited value from the darkest value to the brightest value. Thus, the digital image is essentially an M x N matrix. For an image of the same size, the smaller the pixel grid, the higher the resolution of the digital image, the more clear the same amount of data is contained. For images of the same size and resolution, the more the brightness levels, the smoother the image, and the more data is contained. The original video data resolution of the streaming data source is often high, including a relatively large amount of data.
S200, forming a corresponding preprocessing strategy based on preprocessing configuration aiming at any streaming data source. The preprocessing configuration items, the preprocessing configuration item parameters and the coordination relation among the configuration items in the preprocessing configuration are set according to the video preprocessing process required by the video receiver of the streaming data source.
In the implementation process, referring to fig. 2, the process of forming a corresponding preprocessing strategy based on the preprocessing configuration for any streaming data source includes:
s201, a preprocessing configuration item function library recording preprocessing configuration item functions is created in advance.
S202, according to any preprocessing configuration item in any preprocessing configuration for the streaming data source, a corresponding preprocessing configuration item function is called from a preprocessing configuration item function library. In the implementation process, preprocessing configuration is created based on the requirement of a video receiver, and the preprocessing configuration comprises at least one preprocessing configuration item, preprocessing configuration item parameters and coordination relations among all configuration items.
Specifically, the preprocessing configuration function for implementing the preprocessing configuration item includes:
spatial transform functions of video frame images, such as scaling functions, rotation functions, cropping functions, pruning functions, flipping functions, translation functions, affine transform functions, projective transform functions, box transform functions, of video frame images.
Morphological functions of video frame images such as erosion functions, dilation functions, distance transformation functions, and image neighborhood operation block operation functions.
Noise filter processing functions such as spatial filter template (laplace template, sobel template, prewitt template, and robert template) functions, frequency domain filter (low-pass filter, high-pass filter, band-stop filter, and homomorphic filter) functions.
Image transformation functions such as fourier transformation functions, discrete cosine transformation functions, radon transformation functions, hough transformation functions, fan-Bean projection transformation functions, and wavelet transformation functions.
Frame processing functions, such as a frame extraction processing function, a frame insertion processing function.
Video frame image gray processing functions, such as interpolation functions for adjusting gray levels, and graying functions.
The preprocessing configuration function library is designed into an expandable library, and functions provided by a video receiver developer can be added into the preprocessing configuration function library after safety inspection, so that the preprocessing configuration function library can support preprocessing configuration of a more advanced video receiver.
S203, the preprocessing configuration item parameters of the preprocessing configuration item are transmitted to the called preprocessing configuration item function;
s204, constructing the relation among the called preprocessing configuration item functions based on the coordination relation among the preprocessing configuration items in the preprocessing configuration to form a preprocessing strategy. The coordination relationships between the preprocessing configuration items include, but are not limited to, sequential execution, loop nesting, call.
S300, preprocessing original video data of the streaming data source by a preprocessing strategy aiming at the streaming data source to form preprocessed video data.
S400, the video receiver initiates a streaming mode request for a streaming data source according to the requirement. Configuring a streaming mode request comprising at least one streaming data source, a streaming identifier and video receiver information, wherein the streaming identifier is at least one of a first streaming identifier, a second streaming identifier and a third streaming identifier which indicate streaming modes; and streaming a streaming data source according to the streaming mode request according to the streaming mode indicated by the streaming identification.
S500, according to the streaming mode request of the video receiver for at least one streaming data source, the fragments of the original video data and/or the preprocessed video data fragments of the related streaming data source are packaged in a plurality of streaming packets to form corresponding streaming.
Specifically, if in a streaming request, a streaming identifier associated with a streaming data source is a first streaming identifier, then the original video data of the streaming data source is encapsulated into a streaming and sent to a video receiver in the streaming request;
if the streaming identifier associated with a streaming data source is a second streaming identifier in a streaming request, packaging the preprocessed video data of the streaming data source into a streaming to be sent to a video receiver in the streaming request;
if the stream identifier associated with a stream data source is the third stream identifier in a streaming request, the original video data and the preprocessed video data of the stream data source are packaged into a serial stream and sent to the video receiver in the streaming request. Referring to fig. 3, when the original video data and the pre-processed video data of a streaming data source are packaged in the same stream, an original video data segment of the original video data is packaged in a main packet, the pre-processed video data segment corresponding to the original data segment is packaged in a client packet immediately after the main packet of the original data segment is packaged, and the whole formed by the main packet and the client packet is serially connected in turn to form a stream comprising the original video data and the pre-processed video data.
S600, the video receiver receives the original video data and/or the preprocessed video data required by the analysis of the streaming acquisition.
Example 2
In this embodiment, embodiment 1 is applied to a monitoring scene, where the preprocessing configuration of the monitoring scene includes a video scaling configuration item, and the scaling function of the video frame image in the spatial transform function of the video frame image is called based on the video scaling configuration item to process the original video data of the streaming data source. When a video receiver of the monitoring scene displays the video of the multi-stream data source, the monitoring scene sends a streaming mode request containing a second streaming identifier, and receives a preprocessed video image which reduces the original video data of the streaming data source. When a video receiver of the monitoring scene independently displays a video of a streaming data source, the monitoring scene sends a streaming mode request containing a first streaming mark and receives original video data independently displaying the streaming data source.
Example 3
In this embodiment, embodiment 1 is applied to video or live broadcast software, where the preprocessing configuration of the monitoring scene includes a video scaling configuration item, and the scaling function of the video frame image in the spatial transform function of the video frame image is called based on the video scaling configuration item to process the original video data of the streaming data source. When the video or live broadcast software displays the video preview of the streaming data source through the small window, a video receiver of the video or live broadcast software sends a streaming mode request containing a second streaming mark and receives a preprocessed video image for shrinking the original video data of the streaming data source. When a video receiver of video or live broadcast software enters a video or live broadcast page, a monitoring scene sends a streaming mode request containing a first streaming mark and receives original video data of a streaming data source.
Example 4
In this embodiment, embodiment 1 is applied to the scene of the artificial intelligence analysis video, and the preprocessing configuration of the artificial intelligence analysis video scene is set according to the preprocessing process of the video frame image required by the artificial intelligence neural network model. And the operation equipment where the artificial intelligent neural network model is located sends a streaming mode request containing the second streaming identifier, receives the preprocessed video data, and inputs the preprocessed video data into the artificial intelligent neural network model for operation.
Example 5
Referring to fig. 4, an embodiment of the present application provides an apparatus for improving video streaming based on a streaming improvement method applying video preprocessing, including:
at least one streaming data source that generates raw video data.
The preprocessing module is coupled to the streaming data source, receives the original video data generated by the buffered streaming data source, and preprocesses the original video data to form preprocessed video data by using a target preprocessing strategy for the target streaming data source.
And the streaming module is coupled with the streaming data source and the preprocessing module, and encapsulates the fragments of the original video data and/or the preprocessed video data of the related streaming data source into a plurality of streaming packets according to the streaming mode request of the at least one video receiver for the at least one streaming data source, so as to form a corresponding streaming and send the corresponding streaming to the corresponding video receiver coupled with the streaming module.
The pretreatment strategy construction module is coupled with the pretreatment module, and the pretreatment strategy construction module calls a corresponding pretreatment configuration item function from a pretreatment configuration item function library according to any pretreatment configuration item in any pretreatment configuration, and transmits the pretreatment configuration item parameters of the pretreatment configuration item to the called pretreatment configuration item function; and constructing the relation among the called preprocessing configuration item functions based on the coordination relation among the preprocessing configuration items in the preprocessing configuration to form a preprocessing strategy.
Example 6
The embodiment of the application provides a storage medium for realizing a streaming improvement method of application video preprocessing, wherein the storage medium for realizing the streaming improvement method of application video preprocessing is provided with at least one transmission interface, at least one instruction is stored, and the instruction is read from the transmission interface and executed to realize the streaming improvement method of application video preprocessing.
In the embodiments provided in the present invention, it should be understood that the disclosed structure may be implemented in other manners. For example, the structural embodiments described above are merely illustrative, and for example, the division of the units is merely a logical function division, and there may be other manners of division in actual implementation, for example, multiple units or components may be combined or integrated into another system, or some features may be omitted, or not performed. Alternatively, the coupling or direct coupling or communication connection shown or discussed with each other may be an indirect coupling or communication connection via interfaces, structures or units, which may be in electrical, mechanical or other forms.
The units described as separate units may or may not be physically separate, and units shown as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.
In addition, each functional unit in the embodiments of the present invention may be integrated in one processing unit, or each unit may exist alone physically, or two or more units may be integrated in one unit. The integrated units may be implemented in hardware or in software functional units.
The foregoing is only a specific embodiment of the invention to enable those skilled in the art to understand or practice the invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the generic principles defined herein may be applied to other embodiments without departing from the spirit or scope of the invention. Thus, the present invention is not intended to be limited to the embodiments shown herein but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.
Claims (4)
1. A streaming improvement method using audio-visual preprocessing, comprising: receiving and caching original video data of a streaming data source;
forming a corresponding preprocessing strategy based on preprocessing configuration aiming at any streaming data source, and preprocessing the original video data of the streaming data source through the preprocessing strategy to form preprocessed video data;
the process of forming the pretreatment strategy comprises the following steps:
pre-creating a preprocessing configuration item function library for recording preprocessing configuration item functions;
according to any preprocessing configuration item in any preprocessing configuration aiming at the streaming data source, calling a corresponding preprocessing configuration item function from a preprocessing configuration item function library, and transmitting preprocessing configuration item parameters of the preprocessing configuration item to the called preprocessing configuration item function; constructing the relation among the called preprocessing configuration item functions based on the coordination relation among all the preprocessing configuration items in the preprocessing configuration to form a preprocessing strategy; wherein, each preprocessing configuration item, preprocessing configuration item parameters and coordination relation among each configuration item in preprocessing configuration are set according to the video preprocessing process required by the video receiver of the streaming data source;
according to a streaming mode request of a video receiver for at least one streaming data source, encapsulating fragments of original video data and pre-processed video data fragments of related streaming data sources in a plurality of streaming packets to form corresponding streaming, including:
in a streaming mode request initiated by a video receiver, if a streaming identifier associated with a streaming data source is a third streaming identifier, packaging an original video data segment of original video data into a main packet, packaging a preprocessed video data segment corresponding to the original video data segment into a client packet immediately packaging the main packet of the original video data segment, and sequentially concatenating an entirety formed by the main packet and the client packet to form a streaming comprising the original video data and the preprocessed video data; the streaming mode request comprises at least one streaming data source, a streaming identifier and video receiver information, wherein the streaming identifier is associated with a single streaming data source;
the video receiver receives the original video data and the preprocessed video data required by analyzing the streaming acquisition.
2. The streaming improvement method according to claim 1, wherein said preprocessing configuration function implementing preprocessing configuration item includes spatial transform function of video frame image, morphological function of video frame image, noise filter processing function, image transform function, frame processing function, video frame image gray processing function.
3. A video stream improvement apparatus based on the stream improvement method of claim 1 or 2, comprising:
at least one streaming data source, the streaming data source generating raw video data;
the preprocessing module is coupled with the streaming data source, receives the original video data generated by the cache streaming data source, and preprocesses the original video data by utilizing a target preprocessing strategy aiming at the target streaming data source to form preprocessed video data;
the pretreatment strategy construction module is coupled with the pretreatment module, and the pretreatment strategy construction module calls a corresponding pretreatment configuration item function from a pretreatment configuration item function library according to any pretreatment configuration item in any pretreatment configuration, and transmits the pretreatment configuration item parameters of the pretreatment configuration item to the called pretreatment configuration item function; constructing the relation among the called preprocessing configuration item functions based on the coordination relation among all the preprocessing configuration items in the preprocessing configuration to form a preprocessing strategy; wherein, each preprocessing configuration item, preprocessing configuration item parameters and coordination relation among each configuration item in preprocessing configuration are set according to the video preprocessing process required by the video receiver of the streaming data source;
the streaming module encapsulates a fragment of original video data and a fragment of preprocessed video data of a related streaming data source in a plurality of streaming packets according to a streaming mode request of at least one video receiver for at least one streaming data source, forms a corresponding streaming and sends the corresponding streaming to a corresponding video receiver coupled to the streaming module, and comprises:
in a streaming mode request initiated by a video receiver, if a streaming identifier associated with a streaming data source is a third streaming identifier, packaging an original video data segment of original video data into a main packet, packaging a preprocessed video data segment corresponding to the original video data segment into a client packet immediately packaging the main packet of the original video data segment, and sequentially concatenating an entirety formed by the main packet and the client packet to form a streaming comprising the original video data and the preprocessed video data; the streaming request includes at least one streaming data source, a streaming identifier associated with the single streaming data source, and video receiver information.
4. A storage medium for implementing a stream improvement method for applying a video pre-process, wherein the storage medium implementing the stream improvement method for applying a video pre-process stores at least one instruction, and a processor reads and executes the instruction to implement the stream improvement method for applying a video pre-process according to claim 1 or 2.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210461886.4A CN114866806B (en) | 2022-04-28 | 2022-04-28 | Streaming improvement method, device and storage medium applying video and audio preprocessing |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210461886.4A CN114866806B (en) | 2022-04-28 | 2022-04-28 | Streaming improvement method, device and storage medium applying video and audio preprocessing |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN114866806A CN114866806A (en) | 2022-08-05 |
| CN114866806B true CN114866806B (en) | 2023-07-18 |
Family
ID=82633409
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210461886.4A Active CN114866806B (en) | 2022-04-28 | 2022-04-28 | Streaming improvement method, device and storage medium applying video and audio preprocessing |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114866806B (en) |
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102710963A (en) * | 2011-02-22 | 2012-10-03 | 仁宝电脑工业股份有限公司 | Method and system for adjusting video and audio quality of video stream |
| CN108124183A (en) * | 2016-11-29 | 2018-06-05 | 达升企业股份有限公司 | With it is synchronous obtain it is audio-visual to carry out the method for one-to-many video stream |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2014025319A1 (en) * | 2012-08-08 | 2014-02-13 | National University Of Singapore | System and method for enabling user control of live video stream(s) |
-
2022
- 2022-04-28 CN CN202210461886.4A patent/CN114866806B/en active Active
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102710963A (en) * | 2011-02-22 | 2012-10-03 | 仁宝电脑工业股份有限公司 | Method and system for adjusting video and audio quality of video stream |
| CN108124183A (en) * | 2016-11-29 | 2018-06-05 | 达升企业股份有限公司 | With it is synchronous obtain it is audio-visual to carry out the method for one-to-many video stream |
Also Published As
| Publication number | Publication date |
|---|---|
| CN114866806A (en) | 2022-08-05 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| EP3751857A1 (en) | A method, an apparatus and a computer program product for volumetric video encoding and decoding | |
| US6141693A (en) | Method and apparatus for extracting digital data from a video stream and using the digital data to configure the video stream for display on a television set | |
| US8988502B2 (en) | System and method for transmission, processing, and rendering of stereoscopic and multi-view images | |
| US7027616B2 (en) | Monitoring system | |
| DE602004009591T2 (en) | DIFFERENTIAL CODING BY USING A 3D GRAPHIC PROCESSOR | |
| EP3456058A1 (en) | Bit depth remapping based on viewing parameters | |
| CN105681720A (en) | Video playing processing method and device | |
| CN111277829A (en) | Encoding and decoding method and device | |
| EP4018410A1 (en) | Watermark-based image reconstruction | |
| CN114866806B (en) | Streaming improvement method, device and storage medium applying video and audio preprocessing | |
| CN111432142B (en) | Video synthesis method, device, equipment and storage medium | |
| CN112261417B (en) | Video pushing method and system, equipment and readable storage medium | |
| WO2023226504A1 (en) | Media data processing methods and apparatuses, device, and readable storage medium | |
| US20220012849A1 (en) | Image processing device and method | |
| CN115567711A (en) | Desktop image dynamic acquisition method and device and computer readable storage medium | |
| CN116489457A (en) | Video display control method, device, equipment, system and storage medium | |
| CN106027991B (en) | Medical video image live broadcast all-in-one | |
| CN110418197B (en) | Video transcoding method and device and computer readable storage medium | |
| CN106157213A (en) | A kind of medical video image live broadcasting method | |
| CN205987159U (en) | Live all -in -one of medical treatment video | |
| JP7471731B2 (en) | METHOD FOR ENCAPSULATING MEDIA FILES, METHOD FOR DECAPSULATING MEDIA FILES AND RELATED DEVICES | |
| JP2001144968A (en) | Multimedia information converter | |
| CN115131730B (en) | Intelligent video analysis method and system based on edge terminal | |
| CN114666477B (en) | Video data processing method, device, equipment and storage medium | |
| US20230088144A1 (en) | Data processing method and apparatus for immersive media, related device, and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |