CN113784209A - Multimedia data stream processing method and device - Google Patents
Multimedia data stream processing method and device Download PDFInfo
- Publication number
- CN113784209A CN113784209A CN202111031283.2A CN202111031283A CN113784209A CN 113784209 A CN113784209 A CN 113784209A CN 202111031283 A CN202111031283 A CN 202111031283A CN 113784209 A CN113784209 A CN 113784209A
- Authority
- CN
- China
- Prior art keywords
- video frame
- frame picture
- target
- multimedia data
- filter
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000003672 processing method Methods 0.000 title claims abstract description 16
- 238000000034 method Methods 0.000 claims abstract description 40
- 238000012545 processing Methods 0.000 claims description 42
- 238000004590 computer program Methods 0.000 claims description 13
- 230000001960 triggered effect Effects 0.000 claims description 6
- 230000006835 compression Effects 0.000 description 11
- 238000007906 compression Methods 0.000 description 11
- 230000004048 modification Effects 0.000 description 7
- 238000012986 modification Methods 0.000 description 7
- 230000008569 process Effects 0.000 description 6
- 238000010586 diagram Methods 0.000 description 5
- 230000005540 biological transmission Effects 0.000 description 4
- 238000005516 engineering process Methods 0.000 description 4
- 238000004422 calculation algorithm Methods 0.000 description 3
- 238000004891 communication Methods 0.000 description 3
- 230000003287 optical effect Effects 0.000 description 3
- 238000005070 sampling Methods 0.000 description 3
- 238000005538 encapsulation Methods 0.000 description 2
- 238000001914 filtration Methods 0.000 description 2
- 230000006870 function Effects 0.000 description 2
- 230000003068 static effect Effects 0.000 description 2
- 230000002457 bidirectional effect Effects 0.000 description 1
- 238000004364 calculation method Methods 0.000 description 1
- 230000015556 catabolic process Effects 0.000 description 1
- 230000006837 decompression Effects 0.000 description 1
- 238000006731 degradation reaction Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 230000010365 information processing Effects 0.000 description 1
- 230000003993 interaction Effects 0.000 description 1
- 238000010295 mobile communication Methods 0.000 description 1
- 238000004806 packaging method and process Methods 0.000 description 1
- 238000009877 rendering Methods 0.000 description 1
- 230000004044 response Effects 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
- H04N21/44—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
- H04N21/21—Server components or server architectures
- H04N21/218—Source of audio or video content, e.g. local disk arrays
- H04N21/2187—Live feed
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
- H04N21/439—Processing of audio elementary streams
- H04N21/4398—Processing of audio elementary streams involving reformatting operations of audio signals
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
- H04N21/44—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs
- H04N21/4402—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs involving reformatting operations of video signals for household redistribution, storage or real-time display
- H04N21/440218—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs involving reformatting operations of video signals for household redistribution, storage or real-time display by transcoding between formats or standards, e.g. from MPEG-2 to MPEG-4
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
- H04N21/44—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs
- H04N21/4402—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs involving reformatting operations of video signals for household redistribution, storage or real-time display
- H04N21/440263—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs involving reformatting operations of video signals for household redistribution, storage or real-time display by altering the spatial resolution, e.g. for displaying on a connected PDA
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N5/00—Details of television systems
- H04N5/222—Studio circuitry; Studio devices; Studio equipment
- H04N5/262—Studio circuits, e.g. for mixing, switching-over, change of character of image, other special effects ; Cameras specially adapted for the electronic generation of special effects
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N5/00—Details of television systems
- H04N5/222—Studio circuitry; Studio devices; Studio equipment
- H04N5/262—Studio circuits, e.g. for mixing, switching-over, change of character of image, other special effects ; Cameras specially adapted for the electronic generation of special effects
- H04N5/268—Signal distribution or switching
Landscapes
- Engineering & Computer Science (AREA)
- Multimedia (AREA)
- Signal Processing (AREA)
- Databases & Information Systems (AREA)
- Compression Or Coding Systems Of Tv Signals (AREA)
Abstract
The application discloses a multimedia data stream processing method. The method comprises the following steps: decoding the input multi-path multimedia data stream through a decoder to obtain video frame pictures of the multi-path multimedia data stream; selecting a target video frame picture input into the filter from video frame pictures of the multi-path multimedia data stream through a data selector; judging whether the frame of the target video is modified by a filter through a frame judgment module; when the target video frame picture is judged not to be modified by the filter, judging whether the parameter information in the target video frame picture is the same as the parameter information of the target multimedia data stream or not by the frame judging module; and outputting the undecoded video binary data corresponding to the target video frame picture to a streaming media server when the parameter information in the target video frame picture is judged to be the same as the parameter information of the target multimedia data stream. The method and the device can avoid the video stream with high-quality input from being recoded, and improve the quality of the output stream.
Description
Technical Field
The present application relates to the field of video technologies, and in particular, to a method and an apparatus for processing a multimedia data stream.
Background
The live broadcast service uses the cloud director service when editing, switching and mixing multi-path on-demand files or live broadcast streaming media. Specifically, after a plurality of input video streams (including audio and video signals) are decoded, the director operator switches the input through a data selector in an editing module, then filters the audio and video signals which are specified to be input through a filter in the editing module, inputs the filtered signals into an encoder for compression, and then transmits the compressed video streams into a streaming media server.
However, the inventor found that, since the encoding parameters of different input video streams may be different, the audio/video signal data processed by the editing module may be encoded uniformly according to the same set of encoding parameters, and since the output of the cloud director is generally live streaming, the encoding complexity is low in order to ensure low delay and maintain the frame rate. This results in a significant degradation of video quality when some of the input is a high quality video stream (e.g., high complexity compressed 4K, 8K video) during the second encoding pass.
Disclosure of Invention
In view of this, a method, an apparatus, a computer device, and a computer readable storage medium for processing a multimedia data stream are provided to solve the problem that the video quality is greatly reduced when the existing cloud broadcasting station performs a second encoding on a high-quality video stream.
The application provides a multimedia data stream processing method, which is applied to a cloud broadcasting guide platform, wherein the cloud broadcasting guide platform comprises a decoder, an editing module, a frame judging module and an encoder, the editing module comprises a data selector and a filter, and the method comprises the following steps:
the cloud director station decodes the input multi-path multimedia data stream through the decoder to obtain video frame pictures of the multi-path multimedia data stream;
the cloud director selects a target video frame picture input into the filter from video frame pictures of the multi-path multimedia data stream through the data selector;
the cloud director judges whether the target video frame picture is modified by the filter through the frame judgment module;
when the target video frame picture is judged not to be modified by the filter, the cloud program director judges whether parameter information in the target video frame picture is the same as parameter information of a target multimedia data stream or not through the frame judgment module;
and when the parameter information in the target video frame picture is judged to be the same as the parameter information of the target multimedia data stream, the cloud director platform outputs the video binary data which is not decoded and corresponds to the target video frame picture to a streaming media server.
Optionally, after the step of determining, by the frame determination module, whether the target video frame picture is modified by the filter, the cloud director further includes:
when the target video frame picture is judged to be modified by the filter, the cloud director platform encodes the video frame picture obtained by modifying the filter by the encoder;
and the cloud director platform outputs the video binary data obtained by encoding to the streaming media service through the encoder.
Optionally, after the step of determining, by the frame determination module, whether the parameter information in the target video frame picture is the same as the parameter information of the target multimedia data stream, the cloud director further includes:
when the parameter information in the target video frame picture is judged to be different from the parameter information of the target multimedia data stream, the cloud director platform carries out coding processing on the video frame picture which is obtained without being modified by the filter through the coder;
and the cloud director platform outputs the video binary data obtained by encoding to the streaming media service through the encoder.
Optionally, the method further comprises:
when the data selector receives a stream switching operation triggered by a user, the cloud director judges whether a target video frame picture input into the filter at the current moment is a video frame picture meeting a preset condition or not through the frame judgment module;
and when the target video frame picture input into the filter at the current moment is judged not to be the video frame picture meeting the preset condition, the cloud director selects a frame picture in the multimedia data stream corresponding to the stream switching operation from the video frame pictures of the multi-path multimedia data stream through the data selector to serve as the target video frame picture input into the filter at the first moment, wherein the first moment is the time after the current moment.
Optionally, after the step of determining, by the frame determination module, whether the target video frame picture input into the filter at the current moment is a video frame picture meeting a preset condition, the cloud director further includes:
when the target video frame picture input into the filter at the current moment is judged to be the video frame picture meeting the preset condition, the cloud program director judges whether the target video frame picture input into the filter at the current moment is the key frame or not through the frame judgment module;
and when the target video frame picture input into the filter at the current moment is judged to be a key frame, the cloud director selects a frame picture in the multimedia data stream corresponding to the stream switching operation from video frame pictures of the multi-path multimedia data stream through the data selector as the target video frame picture input into the filter at the first moment, wherein the first moment is the time after the current moment.
Optionally, after the step of determining, by the frame determination module, whether a target video frame picture input into the filter at the current time is a key frame, the cloud director further includes:
when the target video frame picture input into the filter at the current moment is judged not to be a key frame, the cloud director selects a first target frame picture from video frame pictures of the multi-path multimedia data stream through the data selector as the target video frame picture input into the filter at the first moment, wherein the first target frame picture is a frame picture in the multimedia data stream selected before the stream switching operation;
when the frame judgment module judges that the target video frame picture input into the filter is a key frame at the second moment, the cloud director selects a second target frame picture from video frame pictures of multiple paths of multimedia data streams through the data selector as a target video frame picture input into the filter at a third moment, wherein the second target frame picture is a frame picture in the multimedia data streams corresponding to the stream switching operation, the second moment is a time later than the first moment and earlier than the third moment, and the third moment is a time after the second moment.
Optionally, the method further comprises:
when the second target frame picture input into the filter is judged to be the video frame picture meeting the preset condition through the frame judgment module at the third moment and the second target frame picture is not the key frame, the cloud broadcasting guide platform carries out coding processing on the second target frame picture through the coder;
the cloud director platform outputs the video binary data obtained by encoding to the streaming media service through the encoder;
and when the second target frame picture input into the filter is judged to be the video frame picture meeting the preset condition through the frame judgment module at the fourth moment and the second target frame picture is the key frame, the cloud director platform outputs the non-decoded video binary data corresponding to the second target frame picture to a streaming media server.
The application also provides a multimedia data stream processing apparatus, which is applied to a cloud broadcasting guide station, the cloud broadcasting guide station comprises a decoder, an editing module, a frame judgment module and an encoder, the editing module comprises a data selector and a filter, and the multimedia data stream processing apparatus comprises:
the decoding module is used for decoding the input multi-path multimedia data stream through the decoder to obtain video frame pictures of the multi-path multimedia data stream;
a selection module for selecting a target video frame picture inputted into the filter from video frame pictures of the multi-path multimedia data stream through the data selector;
the first judgment module is used for judging whether the target video frame picture is modified by the filter through the frame judgment module;
the second judging module is used for judging whether the parameter information in the target video frame picture is the same as the parameter information of the target multimedia data stream or not through the frame judging module when the target video frame picture is judged not to be modified by the filter;
and the output module is used for outputting the video binary data which is not decoded and corresponds to the target video frame picture to a streaming media server when the parameter information in the target video frame picture is judged to be the same as the parameter information of the target multimedia data stream.
The present application further provides a computer device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, the processor implementing the steps of the above method when executing the computer program.
The present application also provides a computer-readable storage medium having stored thereon a computer program which, when being executed by a processor, carries out the steps of the above-mentioned method.
According to the multimedia data stream processing method, the frame judgment module is additionally arranged on the cloud program director to judge the video frame picture, when the video frame picture is judged to be the input frame meeting the requirement, the operation of skipping the encoder processing of the video frame is realized, and the non-decoded video binary data is directly output to the streaming media server; otherwise, the decoded signal data are transmitted to an encoder for encoding, so that the video stream with high-quality input can be prevented from being recoded, the quality of an output stream is improved, and the characteristic that the cloud director platform smoothly switches all input streams is kept.
Drawings
Fig. 1 is an architecture diagram of a cloud director service system in the prior art;
FIG. 2 is a flow chart of an embodiment of a method for processing a multimedia data stream according to the present application;
FIG. 3 is a flow chart of another embodiment of a method for processing a multimedia data stream according to the present application;
FIG. 4 is a flow chart of another embodiment of a method for processing a multimedia data stream according to the present application;
FIG. 5 is a flow chart of another embodiment of a method for processing a multimedia data stream according to the present application;
FIG. 6 is a flow chart of another embodiment of a method for processing a multimedia data stream according to the present application;
FIG. 7 is a flow chart of another embodiment of a method for processing a multimedia data stream according to the present application;
FIG. 8 is a flow chart of another embodiment of a method for processing a multimedia data stream according to the present application;
FIG. 9 is an architecture diagram of one embodiment of a multimedia data stream processing system according to the present application;
fig. 10 is a schematic hardware configuration diagram of a computer device for executing a multimedia data stream processing method according to an embodiment of the present application.
Detailed Description
The advantages of the present application are further illustrated below with reference to the accompanying drawings and specific embodiments.
Reference will now be made in detail to the exemplary embodiments, examples of which are illustrated in the accompanying drawings. When the following description refers to the accompanying drawings, like numbers in different drawings represent the same or similar elements unless otherwise indicated. The implementations described in the exemplary embodiments below are not intended to represent all implementations consistent with the present disclosure. Rather, they are merely examples of apparatus and methods consistent with certain aspects of the present disclosure, as detailed in the appended claims.
The terminology used in the present disclosure is for the purpose of describing particular embodiments only and is not intended to be limiting of the disclosure. As used in this disclosure and the appended claims, the singular forms "a," "an," and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise. It should also be understood that the term "and/or" as used herein refers to and encompasses any and all possible combinations of one or more of the associated listed items.
It is to be understood that although the terms first, second, third, etc. may be used herein to describe various information, such information should not be limited to these terms. These terms are only used to distinguish one type of information from another. For example, first information may also be referred to as second information, and similarly, second information may also be referred to as first information, without departing from the scope of the present disclosure. The word "if" as used herein may be interpreted as "at … …" or "when … …" or "in response to a determination", depending on the context.
In the description of the present application, it should be understood that the numerical references before the steps do not identify the order of performing the steps, but merely serve to facilitate the description of the present application and to distinguish each step, and therefore should not be construed as limiting the present application.
For convenience of understanding the inventive principle of the present application, an existing cloud director service system is introduced first.
For convenience of understanding the inventive principle of the present application, an existing cloud director service system is introduced first.
Referring to fig. 1, the cloud director service system includes a cloud director, a streaming server, and a video player.
The cloud director platform comprises a decoder, an editing module, an encoder and a packaging transmission module. The decoder is used for decoding input multi-channel video stream data (such as on-demand audio and video files, live broadcast streaming media and the like). The editing module is configured to edit the decoded multiple paths of video stream data, for example, an operator of the director console selects input video stream data through a data selector in the editing module, edits the selected video stream data through a filter, and inputs the edited video stream data into an encoder. The encoder is used for encoding the received video stream data and inputting the encoded video stream data to the encapsulation transmission module. The encapsulation transmission module is used for encapsulating the received video stream data and transmitting the encapsulated video stream data to the streaming media server. The video player is used for acquiring video stream data from the streaming media server and rendering the acquired video stream data so as to play a video.
The cloud director is a multimedia service deployed on the cloud, and can convert a plurality of multimedia data streams (such as on-demand audio/video files, live streaming media and the like) into new multimedia output after operations such as mixing, switching, editing and the like are performed.
Streaming media refers to a form of media that streams audio, video, and multimedia files over a network. Compared with the network playing mode of watching after downloading, the streaming media is typically characterized in that continuous audio and video information is compressed and then put on a network server, and a user can watch while downloading without waiting for the whole file to be downloaded. Due to the superiority of the streaming media technology, the technology is widely applied to video-on-demand, video conference, remote education, remote medical treatment and online live broadcast systems. Streaming media technology has gained rapid development in recent years as a sign of new generation internet applications.
The streaming media server is a core system of the streaming media application, and is a key platform for an operator to provide video services to users. The main functions of the streaming media server are to collect, cache, schedule, transmit and play streaming media contents. The main performance aspects of a streaming media application system depend on the performance and quality of service of the media server. Therefore, the streaming server is the foundation of the streaming application system and is the most important component.
Fig. 2 is a flowchart illustrating a multimedia data stream processing method according to an embodiment of the present application. The method of the present application is applied to the cloud director, where the cloud director includes a decoder, an editing module, a frame determination module, and an encoder, and the editing module includes a data selector and a filter. As can be seen from the figure, the multimedia data stream processing method provided in this embodiment includes:
and step S20, the cloud director platform decodes the input multi-channel multimedia data stream through the decoder to obtain multi-channel video frame pictures.
Specifically, the multi-path multimedia data stream may be multi-path live streaming data, or multi-path disk files, or may include N-path live streaming data and M-path disk files at the same time, where N and M are natural numbers greater than or equal to 1.
It should be noted that the disk file in the present embodiment refers to a multimedia file (such as an on-demand video file) stored in a storage device.
The decoding refers to a process of obtaining the brightness, color signals or audio sampling signals of the video by the encoded binary data through a corresponding decompression algorithm.
As an example, assuming that the input multi-channel multimedia data streams are main and standby live broadcast streams of the electronic competition, the cloud director can decode the main and standby live broadcast streams to obtain a live broadcast video frame picture corresponding to the main live broadcast stream and obtain a live broadcast video frame picture corresponding to the standby live broadcast stream.
Step S21, the cloud director selects a target video frame picture input into the filter from video frame pictures of the multi-path multimedia data stream through the data selector.
Specifically, the user may select which multimedia data stream to output to the streaming media server from the multiple multimedia data streams through the data selector. When the user selects, the data selector selects the video frame picture belonging to the multimedia data stream selected by the user from the video frame pictures of the multimedia data streams as the target video frame picture input into the filter.
Step S22, the cloud director determines, through the frame determination module, whether the frame of the target video frame is modified by the filter.
Specifically, the modification processing includes coding, smearing, changing audio, adjusting volume, and the like, on the target video frame picture.
Step S23, when it is determined that the target video frame picture is not modified by the filter, the cloud director determines, by the frame determination module, whether parameter information in the target video frame picture is the same as parameter information of the target multimedia data stream.
Specifically, when the frame determination module determines that the target video frame picture is not modified by the filter, the frame determination module further determines whether parameter information in the target video frame picture is the same as parameter information of the target multimedia data stream.
The parameter information refers to key parameters of a video frame picture, and may include resolution, coding format, pixel format, channel, signal format, and sampling rate.
The target multimedia data stream may be a multimedia data stream that is pre-specified by the user for reference, or may be set as a multimedia data stream with the highest quality among the multiple multimedia data streams by default, so that the user does not need to specify the target multimedia data stream. As an example, the multi-path multimedia data stream includes 3 paths of multimedia data streams with resolutions of 720P, 1080P and 4K, respectively, and the user can designate the 4K multimedia data stream as the target multimedia data stream.
It should be noted that the video frame picture mentioned in the present application refers to a frame picture including image data and corresponding audio data.
Step S24, when it is determined that the parameter information in the target video frame picture is the same as the parameter information of the target multimedia data stream, the cloud director outputs the un-decoded video binary data corresponding to the target video frame picture to the streaming media server.
Specifically, when the frame determination module determines that the parameter information in the target video frame picture is the same as the parameter information of the target multimedia data stream, the cloud director can directly output the undecoded video binary data corresponding to the target video frame picture to the streaming media server without first inputting the two video frame pictures into the filter, then inputting the video frame picture output by the filter into the encoder for encoding, and finally inputting the encoded video binary data into the streaming media server through the encoder. That is, when the frame decision module determines that the parameter information in the target video frame picture is the same as the parameter information of the target multimedia data stream, the step of encoder encoding may be skipped.
According to the multimedia data stream processing method, the frame judgment module is additionally arranged on the cloud program director to judge the video frame picture, when the video frame picture is judged to be the input frame meeting the requirement, the operation of skipping the encoder processing of the video frame is realized, and the non-decoded video binary data is directly output to the streaming media server; otherwise, the decoded signal data are transmitted to an encoder for encoding, so that the video stream with high-quality input can be prevented from being recoded, the quality of an output stream is improved, and the characteristic that the cloud director platform smoothly switches all input streams is kept.
In an exemplary embodiment, referring to fig. 3, after the step of determining, by the frame determination module, whether the target video frame is modified by the filter, the cloud director further includes: step S30, when the target video frame picture is judged to be modified by the filter, the cloud director uses the encoder to encode the video frame picture obtained by the modification of the filter; step S31, the cloud director outputs the encoded video binary data to the streaming service through the encoder.
Specifically, when the frame determination module determines that the target video frame picture is modified by the filter, the encoder encodes the video frame picture output after the modification by the filter to encode the video frame picture into corresponding video binary data, and after the encoding is completed, the encoder outputs the video binary data to the streaming media server.
The encoding refers to a process of performing lossy compression on brightness and color signals of a video or audio sampling signals through a certain algorithm to obtain binary data.
In this embodiment, when the frame determination module determines that the target video frame is modified by the filter, the video frame is encoded, so that it is ensured that the video output to the user for viewing is stable.
In an exemplary embodiment, referring to fig. 4, after the step of determining, by the frame determination module, whether the parameter information in the target video frame picture is the same as the parameter information of the target multimedia data stream, the cloud director further includes: step S40, when the parameter information in the target video frame picture is judged to be different from the parameter information of the target multimedia data stream, the cloud director uses the encoder to encode the video frame picture which is obtained without the modification processing of the filter; step S41, the cloud director outputs the encoded video binary data to the streaming service through the encoder.
Specifically, when the frame determination module determines that the parameter information in the target video frame picture is different from the parameter information of the target multimedia data stream, indicating that the target video frame picture is not a high-quality video frame picture, in order to ensure that the video output to the user for viewing is stable, the encoder encodes the video frame picture obtained without modification processing (only performing common filtering processing, such as noise signal filtering) by the filter, so as to encode the video frame picture into corresponding video binary data, and after encoding is completed, the encoder outputs the video binary data to the streaming media server.
In the embodiment, when the frame judgment module judges that the parameter information in the target video frame picture is different from the parameter information of the target multimedia data stream, the video frame picture is coded, so that the video output to a user for watching is stable.
In an exemplary embodiment, referring to fig. 5, the method further comprises:
step S50, when the data selector receives a stream switching operation triggered by a user, the cloud director determines, through the frame determination module, whether a target video frame input into the filter at the current time is a video frame that meets a preset condition.
Specifically, the stream switching operation is used to switch the currently input nth multimedia data stream to the mth multimedia data stream. For example, the currently input first path multimedia data stream is switched to the 3 rd path multimedia data stream as the output stream.
The preset condition is that the target video frame picture is not modified by a filter, and the parameter information in the target video frame picture is the same as the parameter information of the target multimedia data stream.
And the current moment is the time when the user triggers the flow switching operation.
Step S51, when it is determined that the target video frame picture input into the filter at the current time is not the video frame picture satisfying the preset condition, the cloud director selects, through the data selector, a frame picture in the multimedia data stream corresponding to the stream switching operation from among the video frame pictures of the multiple multimedia data streams as the target video frame picture input into the filter at the first time, where the first time is a time after the current time.
Specifically, when the frame determination module determines that the target video frame picture input into the filter at the current moment is not the video frame picture satisfying the preset condition, the data selector selects the frame picture in the multimedia data stream corresponding to the stream switching operation from the video frame pictures of the multiple multimedia data streams as the target video frame picture input into the filter at the first moment.
As an example, assuming that the current time is 12: 5: 10 seconds, and the stream switching operation is to switch the 1 st multimedia data stream to the 2 nd multimedia data stream, after 12: 5: 10 seconds, the data selector may select a frame picture belonging to the 2 nd multimedia data stream from video frame pictures of the multiple multimedia data streams as a target video frame picture input into the filter, thereby implementing the switching of the video frame pictures.
In this embodiment, when it is determined that the target video frame picture input to the filter at the current time is not the video frame picture satisfying the preset condition, switching of the video frame pictures is directly achieved, so that the accuracy of switching can be improved.
In an exemplary embodiment, referring to fig. 6, after the step of determining, by the frame determination module, whether the target video frame inputted to the filter at the current time is a video frame meeting a preset condition, the cloud director further includes:
step S60, when it is determined that the target video frame picture input into the filter at the current time is a video frame picture satisfying a preset condition, the cloud director determines, through the frame determination module, whether the target video frame picture input into the filter at the current time is a key frame.
In particular, the key frame (I frame), also called intra picture (I picture), is usually the first frame of each GOP (a video compression technique used by MPEG), and is moderately compressed, and serves as a reference point for random access, and can be regarded as a picture, and belongs to intra-frame compression.
When the frame judgment module judges that the target video frame picture input into the filter at the current moment is the video frame picture meeting the preset condition, the frame judgment module further judges whether the target video frame picture input into the filter at the current moment is the key frame.
Step S61, when it is determined that the target video frame picture input into the filter at the current time is a key frame, the cloud director selects, through the data selector, a frame picture in the multimedia data stream corresponding to the stream switching operation from among the video frame pictures of the multiple multimedia data streams as the target video frame picture input into the filter at the first time, where the first time is a time after the current time.
Specifically, when the frame determination module determines that the target video frame picture input into the filter at the current moment is a key frame, the data selector selects a frame picture in the multimedia data stream corresponding to the stream switching operation from video frame pictures of the multiple multimedia data streams as the target video frame picture input into the filter at the first moment.
As an example, assuming that the current time is 12: 5: 10 seconds, the stream switching operation is to switch the 1 st multimedia data stream to the 2 nd multimedia data stream, and the target video frame picture input into the filter at the current time is a key frame, after 12: 5: 10 seconds, the data selector selects a frame picture belonging to the 2 nd multimedia data stream from the video frame pictures of the multiple multimedia data streams as the target video frame picture input into the filter, thereby implementing the switching of the video frame pictures.
In the embodiment, when the target video frame picture input into the filter at the current moment is judged to be the key frame, the switching of the video frame pictures is directly realized, so that the switching accuracy can be improved.
In an exemplary embodiment, referring to fig. 7, after the step of determining, by the frame determination module, whether the target video frame picture input into the filter at the current time is a key frame, the cloud director further includes:
step S70, when it is determined that the target video frame picture input into the filter at the current time is not a key frame, the cloud director selects a first target frame picture from video frame pictures of multiple multimedia data streams through the data selector as the target video frame picture input into the filter at the first time, where the first target frame picture is a frame picture in the multimedia data stream selected before the stream switching operation.
Specifically, that the target video frame picture is not a key frame means that the target video frame picture is a P frame or a B frame.
The P frame is a frame obtained by compressing data of the frame based on a difference between the frame and an adjacent previous frame (key frame or P frame). The method for jointly compressing the P frame and the key frame can achieve higher compression without obvious compression traces, and belongs to interframe compression.
The B frame is a frame obtained by compressing the present frame data using an inter-frame compression algorithm of bidirectional prediction. When compressing a frame into a B frame, it compresses the frame according to the difference of the adjacent previous frame, the current frame and the next frame data, that is, only the difference between the current frame and the previous and next frames is recorded. Only with B-frame compression can 200 be achieved: 1, high compression. In general, key frame compression is least efficient, P frames are higher, and B frames are highest.
The first target frame picture is a frame picture in the multimedia data stream selected before the stream switching operation, for example, before the stream switching operation, the 1 st path of multimedia data stream is selected as output, and the first target frame picture is a frame picture belonging to the 1 path of multimedia data stream in the video frame pictures of the multiple paths of multimedia data streams.
Step S71, when the frame determination module determines that the target video frame picture input into the filter is a key frame at a second time, the cloud director selects, through the data selector, a second target frame picture from video frame pictures of the multi-path multimedia data stream as the target video frame picture input into the filter at a third time, where the second target frame picture is a frame picture in the multimedia data stream corresponding to the stream cut operation, the second time is a time later than the first time and earlier than the third time, and the third time is a time after the second time.
Specifically, the frame determination module may determine whether a target video frame currently input into the filter is a key frame in real time, and when the frame determination module determines that the target video frame currently input into the filter is the key frame at the second time, the data selector may select the second target frame from the video frame of the multiple multimedia data streams as the target video frame input into the filter at the third time.
The first target frame picture is a frame picture in the multimedia data stream corresponding to the stream switching operation, for example, if the stream switching operation is to switch the 1 st path multimedia data stream to the 2 nd path multimedia data stream, the second target frame picture is a frame picture belonging to the 2 nd path multimedia data stream in the video frame pictures of the multiple paths of multimedia data streams.
As an example, assuming that the first time is 12: 5: 10 seconds, the second time is 12: 5: 15 seconds, and the third time is 12: 5: 15 seconds later, and the switching operation is to switch the 1 st multimedia data stream to the 2 nd multimedia data stream, when the 12: 5: 15 seconds time determines that the target video frame picture input into the filter is an I frame, the data selector will start to use the frame picture belonging to the 2 nd multimedia data stream from the video frame pictures of the multi-path multimedia data stream after 12: 5: 15 seconds as the target video frame picture input into the filter. The data selector still takes the frame pictures belonging to 1 path of multimedia data stream in the video frame pictures of the multi-path multimedia data stream as the target video frame pictures input into the filter between 12 points, 5 minutes, 10 seconds and 12 points, 5 minutes and 15 seconds, and the real cut flow logic is shown as the following table:
in this embodiment, when the B frame in input 1 receives a user-triggered stream switching operation to switch input 1 to input 2, the video frames actually input to the streaming service are the B frame, P frame, B frame, P frame, I frame of input 1 and the B frame, P frame, I frame, and P frame of input 2 in sequence.
In the embodiment, the I frame is always terminated in the process of switching to other input multimedia data streams, so that the situation that the reference frame cannot be found in the previous frame after the stream switching is triggered is avoided.
In an exemplary embodiment, referring to fig. 8, the method further comprises: step S80, when the frame determination module determines that the second target frame picture input into the filter at the third time is the video frame picture meeting the preset condition and the second target frame picture is not the key frame, the cloud director uses the encoder to encode the second target frame picture; step S81, the cloud director outputs the video binary data obtained by encoding to the streaming media service through the encoder; step S82, when the fourth time is determined by the frame determination module that the second target frame picture input into the filter is the video frame picture meeting the preset condition, and the second target frame picture is the key frame, the cloud director outputs the undecoded video binary data corresponding to the second target frame picture to the streaming media server, where the fourth time is a time after the third time.
Specifically, when the video frame picture input to the filter at the third time is a video frame picture that requires the skip of the encoding step, at this time, in order to ensure the stability of the output picture, the encoding step may not be skipped, but the encoder performs encoding processing on the second target frame picture, and then the encoder outputs the encoded video binary data to the streaming media service until the frame determination module determines that the second target frame picture input to the filter is the video frame picture that satisfies the preset condition at the fourth time and the second target frame picture is the key frame, and then the undecoded video binary data corresponding to the second target frame picture is output to the streaming media server.
For example, if the third time is 12 points, 5 minutes and 15 seconds, and the fourth time is 12 points, 5 minutes and 20 seconds, the second target frame picture is input to the encoder for encoding between 12 points, 5 minutes and 15 seconds and 12 points, 5 minutes and 20 seconds, and the encoded binary video data is input to the streaming server. The non-decoded video binary data corresponding to the second target frame picture is output to the streaming media server after 12: 5: 20 seconds.
In this embodiment, the stability of the output picture is ensured by performing re-encoding in the key frame empty period of the two input streams.
Fig. 9 is a block diagram of a multimedia data stream processing apparatus 90 according to an embodiment of the present invention.
In this embodiment, the multimedia data stream processing apparatus 90 is applied to a cloud broadcasting guide, the cloud broadcasting guide includes a decoder, an editing module, a frame determination module and an encoder, the editing module includes a data selector and a filter, the multimedia data stream processing apparatus 90 includes a series of computer program instructions stored in a memory, and when the computer program instructions are executed by a processor, the multimedia data stream processing function of each embodiment of the present application can be implemented. In some embodiments, the multimedia data stream processing apparatus 90 may be divided into one or more modules based on the specific operations implemented by the computer program instructions, and the specific divided modules are as follows:
a decoding module 91, configured to decode the input multiple multimedia data streams through the decoder to obtain video frame pictures of the multiple multimedia data streams;
a selecting module 92, configured to select, through the data selector, a target video frame picture input into the filter from video frame pictures of the multiple multimedia data streams;
a first judging module 93, configured to judge, by the frame judging module, whether the target video frame is modified by the filter;
a second judging module 94, configured to, when it is judged that the target video frame picture is not modified by the filter, judge, by the frame judging module, whether parameter information in the target video frame picture is the same as parameter information of a target multimedia data stream;
an output module 95, configured to output, to the streaming server, the undecoded video binary data corresponding to the target video frame picture when it is determined that the parameter information in the target video frame picture is the same as the parameter information of the target multimedia data stream.
In an exemplary embodiment, the multimedia data stream processing apparatus 90 further includes an encoding module.
And the coding module is used for coding the video frame picture obtained by modifying the filter through the coder when the target video frame picture is judged to be modified through the filter.
The output module 95 is further configured to output the encoded video binary data to the streaming media service through the encoder.
In an exemplary embodiment, the encoding module is further configured to, when it is determined that the parameter information in the target video frame picture is not the same as the parameter information of the target multimedia data stream, perform, by the encoder, encoding processing on the video frame picture that is obtained without performing the modification processing on the filter.
The output module 95 is further configured to output the encoded video binary data to the streaming media service through the encoder.
In an exemplary embodiment, the multimedia data stream processing apparatus 90 further includes a third determining module.
And the third judging module is used for judging whether a target video frame picture input into the filter at the current moment is a video frame picture meeting a preset condition or not through the frame judging module when the data selector receives a stream switching operation triggered by a user.
The selecting module 92 is further configured to select, by the data selector, a frame picture in the multimedia data stream corresponding to the stream switching operation from video frame pictures in the multiple multimedia data streams as a target video frame picture input into the filter at a first time, when it is determined that the target video frame picture input into the filter at the current time is not a video frame picture satisfying a preset condition, where the first time is a time after the current time.
In an exemplary embodiment, the multimedia data stream processing apparatus 90 further includes a fourth determining module.
The fourth judging module is configured to judge, by the frame judging module, whether the target video frame picture input into the filter at the current time is a key frame when it is judged that the target video frame picture input into the filter at the current time is a video frame picture satisfying a preset condition.
The selecting module 92 is further configured to select, by the data selector, a frame picture in the multimedia data stream corresponding to the stream switching operation from video frame pictures in the multiple multimedia data streams as a target video frame picture input into the filter at a first time when it is determined that a target video frame picture input into the filter at the current time is a key frame, where the first time is a time after the current time.
In an exemplary embodiment, the selecting module 92 is further configured to select, by the data selector, a first target frame picture from video frame pictures of the multiple multimedia data streams as the target video frame picture input into the filter at the first time when it is determined that the target video frame picture input into the filter at the current time is not a key frame, where the first target frame picture is a frame picture in the multimedia data stream selected before the stream switching operation.
The selecting module 92 is further configured to select, by the data selector, a second target frame picture from video frame pictures of multiple multimedia data streams as a target video frame picture input into the filter at a third time when the target video frame picture input into the filter is determined to be a key frame by the frame determining module at the second time, where the second target frame picture is a frame picture in a multimedia data stream corresponding to the stream cut operation, the second time is a time later than the first time and earlier than the third time, and the third time is a time after the second time.
In an exemplary embodiment, the encoding module is further configured to perform, when the second target frame picture input to the filter at the third time is determined by the frame determination module to be a video frame picture satisfying the preset condition and the second target frame picture is not a key frame, encoding processing on the second target frame picture by the encoder.
The output module 95 is further configured to output the encoded video binary data to the streaming media service through the encoder.
The output module 95 is further configured to, when the fourth time is determined by the frame determination module that the second target frame picture input into the filter is the video frame picture meeting the preset condition, and the second target frame picture is the key frame, output the un-decoded video binary data corresponding to the second target frame picture to the streaming media server, where the fourth time is a time after the third time.
According to the multimedia data stream processing method, the frame judgment module is additionally arranged on the cloud program director to judge the video frame picture, when the video frame picture is judged to be the input frame meeting the requirement, the operation of skipping the encoder processing of the video frame is realized, and the non-decoded video binary data is directly output to the streaming media server; otherwise, the decoded signal data are transmitted to an encoder for encoding, so that the video stream with high-quality input can be prevented from being recoded, the quality of an output stream is improved, and the characteristic that the cloud director platform smoothly switches all input streams is kept.
Fig. 10 schematically shows a hardware architecture diagram of a computer device 10 adapted to implement the multimedia data stream processing method according to an embodiment of the present application. In the present embodiment, the computer device 10 is a device capable of automatically performing numerical calculation and/or information processing in accordance with a command set or stored in advance. As shown in fig. 10, computer device 10 includes at least, but is not limited to: the memory 120, processor 121, and network interface 122 may be communicatively linked to each other by a system bus. Wherein:
the memory 120 includes at least one type of computer-readable storage medium, which may be volatile or non-volatile, and particularly, includes a flash memory, a hard disk, a multimedia card, a card type memory (e.g., SD or DX memory, etc.), a Random Access Memory (RAM), a Static Random Access Memory (SRAM), a Read Only Memory (ROM), an Electrically Erasable Programmable Read Only Memory (EEPROM), a Programmable Read Only Memory (PROM), a magnetic memory, a magnetic disk, an optical disk, and the like. In some embodiments, the memory 120 may be an internal storage module of the computer device 10, such as a hard disk or a memory of the computer device 10. In other embodiments, the memory 120 may also be an external storage device of the computer device 10, such as a plug-in hard disk, a Smart Media Card (SMC), a Secure Digital (SD) Card, a Flash memory Card (Flash Card), and the like, provided on the computer device 10. Of course, memory 120 may also include both internal and external memory modules of computer device 10. In this embodiment, the memory 120 is generally used for storing an operating system installed in the computer device 10 and various application software, such as program codes of a multimedia data stream processing method. In addition, the memory 120 may also be used to temporarily store various types of data that have been output or are to be output.
Processor 121 may be a Central Processing Unit (CPU), controller, microcontroller, microprocessor, or other multimedia data stream Processing chip in some embodiments. The processor 121 is generally configured to control the overall operation of the computer device 10, such as performing control and processing related to data interaction or communication with the computer device 10. In this embodiment, the processor 121 is configured to execute the program code stored in the memory 120 or process data.
Network interface 122 may comprise a wireless network interface or a wired network interface, with network interface 122 typically being used to establish communication links between computer device 10 and other computer devices. For example, the network interface 122 is used to connect the computer device 10 to an external terminal via a network, establish a data transmission channel and a communication link between the computer device 10 and the external terminal, and the like. The network may be a wireless or wired network such as an Intranet (Intranet), the Internet (Internet), a Global System of Mobile communication (GSM), Wideband Code Division Multiple Access (WCDMA), a 4G network, a 5G network, Bluetooth (Bluetooth), or Wi-Fi.
It is noted that FIG. 10 only shows a computer device having components 120-122, but it is understood that not all of the shown components are required to be implemented, and that more or fewer components may be implemented instead.
In this embodiment, the multimedia data stream processing method stored in the memory 120 may be divided into one or more program modules and executed by one or more processors (in this embodiment, the processor 121) to complete the present application.
The present application provides a computer readable storage medium, on which a computer program is stored, and the computer program, when executed by a processor, implements the steps of the multimedia data stream processing method in the embodiments.
In this embodiment, the computer-readable storage medium includes a flash memory, a hard disk, a multimedia card, a card type memory (e.g., SD or DX memory, etc.), a Random Access Memory (RAM), a Static Random Access Memory (SRAM), a Read Only Memory (ROM), an Electrically Erasable Programmable Read Only Memory (EEPROM), a Programmable Read Only Memory (PROM), a magnetic memory, a magnetic disk, an optical disk, and the like. In some embodiments, the computer readable storage medium may be an internal storage unit of the computer device, such as a hard disk or a memory of the computer device. In other embodiments, the computer readable storage medium may be an external storage device of the computer device, such as a plug-in hard disk, a Smart Media Card (SMC), a Secure Digital (SD) Card, a Flash memory Card (Flash Card), and the like provided on the computer device. Of course, the computer-readable storage medium may also include both internal and external storage devices of the computer device. In this embodiment, the computer-readable storage medium is generally used for storing an operating system and various types of application software installed in the computer device, for example, the program codes of the multimedia data stream processing method in the embodiment, and the like. Further, the computer-readable storage medium may also be used to temporarily store various types of data that have been output or are to be output.
The above-described embodiments of the apparatus are merely illustrative, and the units described as separate parts may or may not be physically separate, and the parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on at least two network units. Some or all of the modules can be screened out according to actual needs to achieve the purpose of the scheme of the embodiment of the application. One of ordinary skill in the art can understand and implement it without inventive effort.
Through the above description of the embodiments, those skilled in the art will clearly understand that each embodiment can be implemented by software plus a general hardware platform, and certainly can also be implemented by hardware. It will be understood by those skilled in the art that all or part of the processes of the methods of the embodiments described above can be implemented by hardware related to instructions of a computer program, which can be stored in a computer readable storage medium, and when executed, can include the processes of the embodiments of the methods described above. The storage medium may be a magnetic disk, an optical disk, a Read-only memory (ROM), a Random Access Memory (RAM), or the like.
Finally, it should be noted that: the above embodiments are only used for illustrating the technical solutions of the present application, and not for limiting the same; although the present application has been described in detail with reference to the foregoing embodiments, it should be understood by those of ordinary skill in the art that: the technical solutions described in the foregoing embodiments may still be modified, or some or all of the technical features may be equivalently replaced; and the modifications or the substitutions do not make the essence of the corresponding technical solutions depart from the scope of the technical solutions of the embodiments of the present application.
Claims (10)
1. A multimedia data stream processing method is applied to a cloud broadcasting guide station, and is characterized in that the cloud broadcasting guide station comprises a decoder, an editing module, a frame judging module and an encoder, the editing module comprises a data selector and a filter, and the method comprises the following steps:
the cloud director station decodes the input multi-path multimedia data stream through the decoder to obtain video frame pictures of the multi-path multimedia data stream;
the cloud director selects a target video frame picture input into the filter from video frame pictures of the multi-path multimedia data stream through the data selector;
the cloud director judges whether the target video frame picture is modified by the filter through the frame judgment module;
when the target video frame picture is judged not to be modified by the filter, the cloud program director judges whether parameter information in the target video frame picture is the same as parameter information of a target multimedia data stream or not through the frame judgment module;
and when the parameter information in the target video frame picture is judged to be the same as the parameter information of the target multimedia data stream, the cloud director platform outputs the video binary data which is not decoded and corresponds to the target video frame picture to a streaming media server.
2. The method of claim 1, wherein after the step of determining, by the frame determination module, whether the target video frame picture is modified by the filter, the cloud director further comprises:
when the target video frame picture is judged to be modified by the filter, the cloud director platform encodes the video frame picture obtained by modifying the filter by the encoder;
and the cloud director platform outputs the video binary data obtained by encoding to the streaming media service through the encoder.
3. The method of claim 1, wherein after the step of determining, by the frame determination module, whether the parameter information in the target video frame picture is the same as the parameter information of the target multimedia data stream, the cloud director further comprises:
when the parameter information in the target video frame picture is judged to be different from the parameter information of the target multimedia data stream, the cloud director platform carries out coding processing on the video frame picture which is obtained without being modified by the filter through the coder;
and the cloud director platform outputs the video binary data obtained by encoding to the streaming media service through the encoder.
4. A method for processing a multimedia data stream according to any of claims 1 to 3, characterized in that the method further comprises:
when the data selector receives a stream switching operation triggered by a user, the cloud director judges whether a target video frame picture input into the filter at the current moment is a video frame picture meeting a preset condition or not through the frame judgment module;
and when the target video frame picture input into the filter at the current moment is judged not to be the video frame picture meeting the preset condition, the cloud director selects a frame picture in the multimedia data stream corresponding to the stream switching operation from the video frame pictures of the multi-path multimedia data stream through the data selector to serve as the target video frame picture input into the filter at the first moment, wherein the first moment is the time after the current moment.
5. The method of claim 4, wherein after the step of determining, by the frame determination module, whether the target video frame inputted into the filter at the current time is a video frame that satisfies a preset condition, the cloud director further comprises:
when the target video frame picture input into the filter at the current moment is judged to be the video frame picture meeting the preset condition, the cloud program director judges whether the target video frame picture input into the filter at the current moment is the key frame or not through the frame judgment module;
and when the target video frame picture input into the filter at the current moment is judged to be a key frame, the cloud director selects a frame picture in the multimedia data stream corresponding to the stream switching operation from video frame pictures of the multi-path multimedia data stream through the data selector as the target video frame picture input into the filter at the first moment, wherein the first moment is the time after the current moment.
6. The method of claim 5, wherein after the step of determining, by the frame determination module, whether the target video frame picture input into the filter at the current time is a key frame, the cloud director further comprises:
when the target video frame picture input into the filter at the current moment is judged not to be a key frame, the cloud director selects a first target frame picture from video frame pictures of multiple paths of multimedia data streams through the data selector as the target video frame picture input into the filter at the first moment, wherein the first target frame picture is a frame picture in the multimedia data streams selected before the stream switching operation;
when the frame judgment module judges that the target video frame picture input into the filter is a key frame at the second moment, the cloud director selects a second target frame picture from video frame pictures of multiple paths of multimedia data streams through the data selector as a target video frame picture input into the filter at a third moment, wherein the second target frame picture is a frame picture in the multimedia data streams corresponding to the stream switching operation, the second moment is a time later than the first moment and earlier than the third moment, and the third moment is a time after the second moment.
7. The method of processing multimedia data streams of claim 6, further comprising:
when the second target frame picture input into the filter is judged to be the video frame picture meeting the preset condition through the frame judgment module at the third moment and the second target frame picture is not the key frame, the cloud broadcasting guide platform carries out coding processing on the second target frame picture through the coder;
the cloud director platform outputs the video binary data obtained by encoding to the streaming media service through the encoder;
when the fourth time is judged that the second target frame picture input into the filter is the video frame picture meeting the preset condition through the frame judgment module, and the second target frame picture is the key frame, the cloud director platform outputs un-decoded video binary data corresponding to the second target frame picture to the streaming media server, wherein the fourth time is the time after the third time.
8. A multimedia data stream processing device is applied to a cloud broadcasting station, and is characterized in that the cloud broadcasting station comprises a decoder, an editing module, a frame judging module and an encoder, the editing module comprises a data selector and a filter, and the multimedia data stream processing device comprises:
the decoding module is used for decoding the input multi-path multimedia data stream through the decoder to obtain video frame pictures of the multi-path multimedia data stream;
a selection module for selecting a target video frame picture inputted into the filter from video frame pictures of the multi-path multimedia data stream through the data selector;
the first judgment module is used for judging whether the target video frame picture is modified by the filter through the frame judgment module;
the second judging module is used for judging whether the parameter information in the target video frame picture is the same as the parameter information of the target multimedia data stream or not through the frame judging module when the target video frame picture is judged not to be modified by the filter;
and the output module is used for outputting the video binary data which is not decoded and corresponds to the target video frame picture to a streaming media server when the parameter information in the target video frame picture is judged to be the same as the parameter information of the target multimedia data stream.
9. A computer device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, the processor implementing the steps of the method of any one of claims 1 to 7 when executing the computer program.
10. A computer-readable storage medium having stored thereon a computer program, characterized in that: the computer program, when executed by a processor, implementing the steps of the method of any one of claims 1 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111031283.2A CN113784209B (en) | 2021-09-03 | 2021-09-03 | Multimedia data stream processing method and device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111031283.2A CN113784209B (en) | 2021-09-03 | 2021-09-03 | Multimedia data stream processing method and device |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN113784209A true CN113784209A (en) | 2021-12-10 |
| CN113784209B CN113784209B (en) | 2023-11-21 |
Family
ID=78840973
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202111031283.2A Active CN113784209B (en) | 2021-09-03 | 2021-09-03 | Multimedia data stream processing method and device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113784209B (en) |
Citations (13)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101808233A (en) * | 2009-02-18 | 2010-08-18 | 杭州海康威视数字技术股份有限公司 | Universal system for realizing video monitoring of mobile phone and method thereof |
| CN102318356A (en) * | 2010-05-07 | 2012-01-11 | 西门子企业通讯有限责任两合公司 | Method and device for modifying a coded data stream |
| CN107027045A (en) * | 2017-04-11 | 2017-08-08 | 广州华多网络科技有限公司 | Pushing video streaming control method, device and video flowing instructor in broadcasting end |
| CN107197320A (en) * | 2017-06-21 | 2017-09-22 | 北京小米移动软件有限公司 | Net cast methods, devices and systems |
| CN108259923A (en) * | 2017-09-27 | 2018-07-06 | 广州华多网络科技有限公司 | A kind of net cast method, system and equipment |
| WO2019007027A1 (en) * | 2017-07-04 | 2019-01-10 | 武汉斗鱼网络科技有限公司 | Video playing method and system, electronic device and readable storage medium |
| CN109391822A (en) * | 2017-08-10 | 2019-02-26 | 徐文波 | Video guide's method, apparatus and terminal device on line |
| WO2020019180A1 (en) * | 2018-07-25 | 2020-01-30 | 深圳市大疆创新科技有限公司 | Image encoding control method and device, storage medium and unmanned aerial vehicle |
| CN111182316A (en) * | 2019-10-24 | 2020-05-19 | 腾讯科技（深圳）有限公司 | Stream switching method and device of media resource, storage medium and electronic device |
| CN111327896A (en) * | 2018-12-13 | 2020-06-23 | 浙江宇视科技有限公司 | Video transmission method and device, electronic equipment and readable storage medium |
| CN111669619A (en) * | 2020-06-04 | 2020-09-15 | 北京奇艺世纪科技有限公司 | Video stream data switching method, device, terminal and readable storage medium |
| CN111918096A (en) * | 2020-07-21 | 2020-11-10 | 上海网达软件股份有限公司 | Method, device and equipment for fast switching input source of cloud director and storage medium |
| CN113079406A (en) * | 2021-03-19 | 2021-07-06 | 上海哔哩哔哩科技有限公司 | Video processing method and device |
-
2021
- 2021-09-03 CN CN202111031283.2A patent/CN113784209B/en active Active
Patent Citations (14)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101808233A (en) * | 2009-02-18 | 2010-08-18 | 杭州海康威视数字技术股份有限公司 | Universal system for realizing video monitoring of mobile phone and method thereof |
| CN102318356A (en) * | 2010-05-07 | 2012-01-11 | 西门子企业通讯有限责任两合公司 | Method and device for modifying a coded data stream |
| CN107027045A (en) * | 2017-04-11 | 2017-08-08 | 广州华多网络科技有限公司 | Pushing video streaming control method, device and video flowing instructor in broadcasting end |
| CN107197320A (en) * | 2017-06-21 | 2017-09-22 | 北京小米移动软件有限公司 | Net cast methods, devices and systems |
| WO2019007027A1 (en) * | 2017-07-04 | 2019-01-10 | 武汉斗鱼网络科技有限公司 | Video playing method and system, electronic device and readable storage medium |
| CN109391822A (en) * | 2017-08-10 | 2019-02-26 | 徐文波 | Video guide's method, apparatus and terminal device on line |
| CN108259923A (en) * | 2017-09-27 | 2018-07-06 | 广州华多网络科技有限公司 | A kind of net cast method, system and equipment |
| WO2020019180A1 (en) * | 2018-07-25 | 2020-01-30 | 深圳市大疆创新科技有限公司 | Image encoding control method and device, storage medium and unmanned aerial vehicle |
| CN112204970A (en) * | 2018-07-25 | 2021-01-08 | 深圳市大疆创新科技有限公司 | Image coding control method and device, storage medium and unmanned aerial vehicle |
| CN111327896A (en) * | 2018-12-13 | 2020-06-23 | 浙江宇视科技有限公司 | Video transmission method and device, electronic equipment and readable storage medium |
| CN111182316A (en) * | 2019-10-24 | 2020-05-19 | 腾讯科技（深圳）有限公司 | Stream switching method and device of media resource, storage medium and electronic device |
| CN111669619A (en) * | 2020-06-04 | 2020-09-15 | 北京奇艺世纪科技有限公司 | Video stream data switching method, device, terminal and readable storage medium |
| CN111918096A (en) * | 2020-07-21 | 2020-11-10 | 上海网达软件股份有限公司 | Method, device and equipment for fast switching input source of cloud director and storage medium |
| CN113079406A (en) * | 2021-03-19 | 2021-07-06 | 上海哔哩哔哩科技有限公司 | Video processing method and device |
Also Published As
| Publication number | Publication date |
|---|---|
| CN113784209B (en) | 2023-11-21 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN102150427B (en) | System and method for video encoding using adaptive loop filter | |
| KR101859155B1 (en) | Tuning video compression for high frame rate and variable frame rate capture | |
| US20080130736A1 (en) | Methods and devices for coding and decoding images, telecommunications system comprising such devices and computer program implementing such methods | |
| JP3840895B2 (en) | Video encoded data conversion apparatus and method | |
| KR20060109251A (en) | Method and apparatus for using reference picture group in video signal encoding/decoding process | |
| CN117061758B (en) | Video encoder, video decoder, encoding and decoding methods for implementing advanced video coding concepts, and video data stream | |
| CN111800653B (en) | Video decoding method, system, device and computer readable storage medium | |
| JP2006203662A (en) | Moving picture coder, moving picture decoder, and coded stream generating method | |
| KR101032243B1 (en) | Method and system for scalable bitstream extraction | |
| KR20090125150A (en) | Systems and methods for adaptively determining i frames for acquisition and base and enhancement layer balancing | |
| CN115134629B (en) | Video transmission method, system, equipment and storage medium | |
| CN101523918B (en) | Video encoding | |
| CN117692650A (en) | Image encoding/decoding method, electronic device, and computer-readable storage medium | |
| CN108353183B (en) | Method for coding a video data stream based on group of pictures (GOP) | |
| CN113784209B (en) | Multimedia data stream processing method and device | |
| CN102783148A (en) | Method and arrangement for video coding | |
| CN110636332A (en) | Video processing method and device and computer readable storage medium | |
| CN109120939B (en) | Video App decoding method and device | |
| US20070009024A1 (en) | Apparatus and method for producing a picture data stream and for processing a picture data stream | |
| JP5173613B2 (en) | Video encoding apparatus and control method thereof | |
| CN110300305B (en) | Video encoding apparatus and method | |
| US20160360236A1 (en) | Method and Apparatus for Entropy Transcoding | |
| JP7273504B2 (en) | VIDEO ENCODED STREAM EDITING DEVICE AND PROGRAM | |
| WO2013114826A1 (en) | Image decoding device | |
| CN113784210B (en) | Pre-monitoring frequency playing method and cloud broadcasting guide table service system |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |