CN114095781A - Multimedia data processing method and device, electronic equipment and storage medium - Google Patents
Multimedia data processing method and device, electronic equipment and storage medium Download PDFInfo
- Publication number
- CN114095781A CN114095781A CN202111290534.9A CN202111290534A CN114095781A CN 114095781 A CN114095781 A CN 114095781A CN 202111290534 A CN202111290534 A CN 202111290534A CN 114095781 A CN114095781 A CN 114095781A
- Authority
- CN
- China
- Prior art keywords
- plug
- multimedia data
- target
- playing
- multimedia
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000003672 processing method Methods 0.000 title claims abstract description 34
- 238000012545 processing Methods 0.000 claims abstract description 51
- 238000000034 method Methods 0.000 claims abstract description 24
- 230000004044 response Effects 0.000 claims abstract description 13
- 230000005540 biological transmission Effects 0.000 claims description 37
- 230000000007 visual effect Effects 0.000 claims description 11
- 238000004590 computer program Methods 0.000 claims description 8
- 238000012163 sequencing technique Methods 0.000 claims description 4
- 238000012856 packing Methods 0.000 claims description 3
- 230000000694 effects Effects 0.000 abstract description 7
- 238000007726 management method Methods 0.000 description 56
- 230000006870 function Effects 0.000 description 12
- 238000004891 communication Methods 0.000 description 6
- 238000010586 diagram Methods 0.000 description 6
- 210000004709 eyebrow Anatomy 0.000 description 6
- 230000008569 process Effects 0.000 description 5
- 230000003796 beauty Effects 0.000 description 4
- 238000005516 engineering process Methods 0.000 description 4
- 238000009877 rendering Methods 0.000 description 4
- 238000004422 calculation algorithm Methods 0.000 description 3
- 230000008901 benefit Effects 0.000 description 2
- 230000018109 developmental process Effects 0.000 description 2
- 230000003287 optical effect Effects 0.000 description 2
- 230000002159 abnormal effect Effects 0.000 description 1
- 230000004931 aggregating effect Effects 0.000 description 1
- 238000006243 chemical reaction Methods 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 230000009977 dual effect Effects 0.000 description 1
- 230000003631 expected effect Effects 0.000 description 1
- 238000001914 filtration Methods 0.000 description 1
- 238000003780 insertion Methods 0.000 description 1
- 230000037431 insertion Effects 0.000 description 1
- 238000012423 maintenance Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000004806 packaging method and process Methods 0.000 description 1
- 238000013468 resource allocation Methods 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- 230000033772 system development Effects 0.000 description 1
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
- H04N21/44—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/44—Arrangements for executing specific programs
- G06F9/445—Program loading or initiating
- G06F9/44521—Dynamic linking or loading; Link editing at or after load time, e.g. Java class loading
- G06F9/44526—Plug-ins; Add-ons
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
- H04N21/439—Processing of audio elementary streams
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
- H04N21/44—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs
- H04N21/44012—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs involving rendering scenes according to scene graphs, e.g. MPEG-4 scene graphs
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/80—Generation or processing of content or additional data by content creator independently of the distribution process; Content per se
- H04N21/81—Monomedia components thereof
- H04N21/8166—Monomedia components thereof involving executable data, e.g. software
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/80—Generation or processing of content or additional data by content creator independently of the distribution process; Content per se
- H04N21/83—Generation or processing of protective or descriptive data associated with content; Content structuring
- H04N21/84—Generation or processing of descriptive data, e.g. content descriptors
Landscapes
- Engineering & Computer Science (AREA)
- Software Systems (AREA)
- Multimedia (AREA)
- Signal Processing (AREA)
- General Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Stored Programmes (AREA)
Abstract
The invention provides a multimedia data processing method, a multimedia data processing device, electronic equipment and a storage medium, and relates to the technical field of embedded software. The multimedia data processing method comprises the following steps: receiving a multimedia playing request sent by a first system, wherein the multimedia playing request carries a playing plug-in identifier; generating a plug-in management unit in the second system in response to the multimedia playing request; generating a target plug-in on the plug-in management unit according to the playing plug-in identification; processing the multimedia data by using the target plug-in to obtain target multimedia data; and transmitting the target multimedia data to the first system so that the first system plays the target multimedia data in a visualized form. The method can improve the processing effect of the multimedia data.
Description
Technical Field
The present invention relates to the field of embedded technologies, and in particular, to a multimedia data processing method and apparatus, an electronic device, and a storage medium.
Background
With the development of embedded technology, the application of embedded technology is spread in almost all fields, for example, smart home, industrial control enterprises, consumer electronics, and the like. The embedded system is a special computer system which is centered on application, is based on modern computer technology and can flexibly cut software and hardware modules according to user requirements, such as functions, reliability, cost, volume, power consumption or environment. The resource allocation, task scheduling or coordination of the software and hardware are all independent of an operating system, and the operating system generally used on the terminal comprises an Android operating system, and the operating system can provide upper-layer applications with certain functions and realize the playing of multimedia data by processing the multimedia data.
The multimedia data method in the prior art has the problem of poor processing effect.
Disclosure of Invention
The invention provides a multimedia data processing method, a multimedia data processing device, electronic equipment and a storage medium, which are used for overcoming the defect of poor multimedia data processing effect in the prior art and achieving the purpose of improving the multimedia data processing effect.
The invention provides a multimedia data processing method, which comprises the following steps: receiving a multimedia playing request sent by a first system, wherein the multimedia playing request carries a playing plug-in identifier; generating a plug-in management unit in the second system in response to the multimedia playing request; generating a target plug-in on the plug-in management unit according to the playing plug-in identification; processing the multimedia data by using the target plug-in to obtain target multimedia data; and transmitting the target multimedia data to the first system so that the first system plays the target multimedia data in a visualized form.
According to a multimedia data processing method provided by the present invention, the transmitting the target multimedia data to the first system comprises: acquiring the target multimedia data type; obtaining a target transmission channel corresponding to the target multimedia data type according to the corresponding relation between the target multimedia data type and the transmission channel; and transmitting the target multimedia data to the first system through the target transmission channel.
According to a multimedia data processing method provided by the present invention, said generating a plug-in management unit in the second system in response to the multimedia play request comprises: acquiring a playing parameter in the multimedia playing request; determining a unit generation tool corresponding to the plug-in management unit as a target unit generation tool according to the corresponding relation between the playing parameters and the unit generation tool; generating a plug-in management unit in the second system using the target unit generation tool.
According to a multimedia data processing method provided by the present invention, the method further comprises: acquiring a first description language of the playing parameters; the first description language is a description language of the playing parameters in the first system; converting the first description language into an intermediate description language; converting the intermediate description language into a second description language according to the corresponding relation between the intermediate description language and the second description language; the second description language is a description language of the playing parameter in the second system.
According to the multimedia data processing method provided by the present invention, the generating a target plug-in on the plug-in management unit according to the playing plug-in identifier comprises: acquiring one or more intermediate playing plug-ins corresponding to the playing plug-in identification from a candidate playing plug-in set; sequencing each intermediate playing plug-in according to the priority of the playing plug-ins to obtain an intermediate playing plug-in sequence; packing each intermediate playing plug-in the intermediate playing plug-in sequence into an integral plug-in; and determining the whole plug-in as a target plug-in generated on the plug-in management unit.
According to a multimedia data processing method provided by the present invention, the method further comprises:
canceling the generated result between the current plug-in and the target plug-in, and switching the state of the multimedia playing request to the state of the event before the initial request is not responded.
The present invention also provides a multimedia data processing apparatus, comprising: the multimedia playing request receiving module is used for receiving a multimedia playing request sent by a first system, wherein the multimedia playing request carries a playing plug-in identifier; the plug-in management unit generating module is used for responding to the multimedia playing request and generating a plug-in management unit in the second system; the target plug-in generating module is used for generating a target plug-in on the plug-in management unit according to the playing plug-in identification; the target multimedia data obtaining module is used for processing the multimedia data by using the target plug-in to obtain target multimedia data; and the target multimedia data transmission module is used for transmitting the target multimedia data to the first system so that the first system can play the target multimedia data in a visual form.
The invention also provides an electronic device comprising a memory, a processor and a computer program stored on the memory and operable on the processor, wherein the processor implements the steps of the multimedia data processing method as described in any one of the above when executing the program.
The invention also provides a non-transitory computer-readable storage medium having stored thereon a computer program which, when executed by a processor, performs the steps of the multimedia data processing method as described in any of the above.
The multimedia data processing method, the multimedia data processing device, the electronic equipment and the storage medium provided by the invention receive a multimedia playing request sent by a first system, wherein the multimedia playing request carries a playing plug-in identifier; generating a plug-in management unit in the second system in response to the multimedia playing request; generating a target plug-in on a plug-in management unit according to the playing plug-in identification; processing the multimedia data by using the target plug-in to obtain target multimedia data; and transmitting the target multimedia data to the first system so that the first system plays the target multimedia data in a visual form. Through the combination of the first system and the second system and the related processing of the plug-in corresponding to the multimedia playing request by the second system, the multimedia data can be played on the first system in a visual mode, and the processing effect of the multimedia data is improved.
Drawings
In order to more clearly illustrate the technical solutions of the present invention or the prior art, the drawings needed for the description of the embodiments or the prior art will be briefly described below, and it is obvious that the drawings in the following description are some embodiments of the present invention, and those skilled in the art can also obtain other drawings according to the drawings without creative efforts.
FIG. 1 is a schematic diagram of an application scenario of a multimedia data processing method provided by the present invention;
FIG. 2 is a flow chart of a multimedia data processing method according to the present invention;
FIG. 3 is a second flowchart illustrating a multimedia data processing method according to the present invention;
FIG. 4 is a third flowchart illustrating a multimedia data processing method according to the present invention;
FIG. 5 is a fourth flowchart illustrating a multimedia data processing method according to the present invention;
FIG. 6 is a fifth flowchart illustrating a multimedia data processing method according to the present invention;
FIG. 7 is a schematic structural diagram of a multimedia data processing method according to the present invention;
FIG. 8 is a second schematic structural diagram of a multimedia data processing method according to the present invention;
FIG. 9 is a schematic diagram of a multimedia data processing apparatus according to the present invention;
fig. 10 is a schematic structural diagram of an electronic device provided by the present invention.
Detailed Description
In order to make the objects, technical solutions and advantages of the present invention clearer, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings, and it is obvious that the described embodiments are some, but not all embodiments of the present invention. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
The multimedia data processing method provided by the application can be applied to the application environment shown in fig. 1, and is particularly applied to a multimedia data processing system. The multimedia data processing system includes a terminal 102 and a terminal 104, wherein the terminal 102 communicates with the terminal 104 through a network. Specifically, the terminal 102 receives a multimedia playing request sent by a first system, where the multimedia playing request carries a playing plugin identifier; generating a plug-in management unit in a second system in the terminal 104 in response to the multimedia play request; generating a target plug-in on a plug-in management unit according to the playing plug-in identification; processing the multimedia data by using the target plug-in to obtain target multimedia data; the target multimedia data is transmitted to the first system in the terminal 102 so that the first system plays the target multimedia data in a visualized form. The terminal 102 or the terminal 104 may be, but is not limited to, various personal computers, notebook computers, smart phones, tablet computers, and portable wearable devices. It will be appreciated that the above-described terminals 102 and 104 may be incorporated in the form of terminals having both a first system and a second system.
The multimedia data processing method of the present invention is described below with reference to fig. 2 to 6.
In one embodiment, as shown in fig. 2, a multimedia data processing method is provided, which is described by taking the method as an example for being applied to a terminal, and includes the following steps:
step 202, receiving a multimedia playing request sent by a first system, where the multimedia playing request carries a playing plugin identifier.
The multimedia playing request refers to a request capable of playing multimedia data. The playing plug-in identifier refers to a mark capable of uniquely identifying the playing plug-in.
Specifically, the terminal receives a multimedia playing request sent by the first system by receiving a triggering operation on the multimedia application, wherein the multimedia playing request carries a playing plug-in identifier.
And step 204, responding to the multimedia playing request, and generating a plug-in management unit in the second system.
The plug-in management unit refers to a unit capable of uniformly managing plug-ins, and the unit may include plug-ins in each multimedia data processing stage or plug-ins of different types. For example, the plug-in management unit comprises a stream taking plug-in, a decoding plug-in or an algorithm plug-in of the multimedia video stream.
Specifically, after receiving a multimedia playing request sent by a first system, the terminal responds to the multimedia playing request and generates a plug-in management unit in a parallel system, namely a second system, which has a compatible relationship with the first system.
And step 206, generating a target plug-in on the plug-in management unit according to the playing plug-in identification.
Specifically, after the terminal generates the plug-in management unit in the second system, the terminal may generate the target plug-in on the plug-in management unit by playing the plug-in identifier. For example, a Picture in MP4(Moving Picture Experts Group 4) format needs to be played on the terminal, the playing format corresponding to the Picture is mpeg2 format, the format can be used as a playing plug-in identifier, and a plug-in with mpeg2enc type target plug-in is generated on the plug-in management unit.
And 208, processing the multimedia data by using the target plug-in to obtain target multimedia data.
The multimedia data refers to data capable of supporting multimedia playing or data containing multimedia playing content.
Specifically, after the terminal generates the target plug-in the second system, the target plug-in is used to process the multimedia data, and the processing includes acquiring, encoding, transmitting, decoding and the like of the multimedia data.
Step 210, transmitting the target multimedia data to the first system, so that the first system plays the target multimedia data in a visualized form.
Specifically, after the terminal obtains the target multimedia data, the terminal transmits the target multimedia data to a first system which is compatible with a second system and operates in parallel, so that the first system plays the target multimedia data in a visual mode, and the whole process from sending a request to playing of the multimedia data by the terminal is completed.
In the multimedia data processing method, a multimedia playing request sent by a first system is received, wherein the multimedia playing request carries a playing plug-in identifier; generating a plug-in management unit in the second system in response to the multimedia playing request; generating a target plug-in on a plug-in management unit according to the playing plug-in identification; processing the multimedia data by using the target plug-in to obtain target multimedia data; and transmitting the target multimedia data to the first system so that the first system plays the target multimedia data in a visual form. Through the combination of the first system and the second system and the related processing of the plug-in corresponding to the multimedia playing request by the second system, the multimedia data can be played on the first system in a visual mode, and the processing effect of the multimedia data is improved.
In one embodiment, as shown in fig. 3, transmitting the target multimedia data to the first system includes:
step 302, obtain the target multimedia data type.
The multimedia data type refers to a type when the multimedia data is played, and the type includes pictures, videos, audios and the like.
Specifically, the terminal judges the obtained target multimedia data through the multimedia data type judgment plug-in, and obtains the target multimedia data type.
And 304, obtaining a target transmission channel corresponding to the target multimedia data type according to the corresponding relation between the target multimedia data type and the transmission channel.
The transmission channel refers to a communication channel capable of transmitting multimedia data.
Specifically, after the terminal acquires the target multimedia data types, each target multimedia data type is provided with a corresponding transmission channel, and a target transmission channel corresponding to the target multimedia data type is acquired.
And step 306, transmitting the target multimedia data to the first system through the target transmission channel.
Specifically, after obtaining the target transmission channel, the terminal may transmit the corresponding target multimedia data to the first system through the target transmission channel. It can be understood that the multimedia data is processed in the second system of the terminal, and the processed target multimedia data is transmitted to the first system through the corresponding transmission channel. For example, the audio is passed back through an arc (audio return channel) channel.
In this embodiment, the target transmission channel corresponding to the target multimedia data type is obtained according to the corresponding relationship between the target multimedia data type and the transmission channel, and the target multimedia data is transmitted to the first system through the target transmission channel, so that different types of multimedia data can be transmitted through different transmission channels, and the purpose of improving the processing efficiency of the multimedia data is achieved.
In one embodiment, as shown in fig. 4, the generating of the plug-in management unit in the second system in response to the multimedia play request includes:
step 402, obtaining the playing parameters in the multimedia playing request.
The playing parameter is a parameter that enables the multimedia data to realize the playing function thereof. For example, the playing parameter is a parameter for an eyebrow selection in a video.
Specifically, the terminal may obtain the playing parameter in the multimedia playing request through the parameter identifier of the playing parameter itself. The parameter identification can be a text description, a unique label and the like. For example, the text of the parameter describes "make-up/eyebrow/woman", and the playing parameter obtained in the multimedia playing request is a playing parameter for processing eyebrow.
Step 404, determining a unit generation tool corresponding to the plug-in management unit as a target unit generation tool according to the corresponding relationship between the playing parameter and the unit generation tool.
Specifically, after the terminal acquires the playing parameters, the terminal determines to generate a corresponding target unit generation tool according to the corresponding relationship between the playing parameters and the unit generation tools. For example, if the acquired playback parameter is a playback parameter for processing an eyebrow, a function or a script tool of the relevant plug-in generation management unit is called as a target unit generation tool.
At step 406, a plug-in management unit is generated in the second system using the target unit generation tool.
Specifically, after determining the target unit generation tool, the terminal generates the plug-in management unit in the second system by using the target unit generation tool. For example, the target unit generation tool is a creation function that performs a rendering function on the eyebrows, and the plug-in management unit is generated in the second system using the creation function. For example, a pipeline having an eyebrow rendering function is generated in the second system by using the creating function, and the pipeline refers to a plug-in management unit capable of aggregating various plug-ins and performing a certain function by using the various plug-ins in a pipeline form.
In this embodiment, the purpose of accurately determining the plug-in management unit can be achieved by obtaining the playing parameter in the multimedia playing request, determining the unit generation tool corresponding to the plug-in management unit according to the corresponding relationship between the playing parameter and the unit generation tool, using the unit generation tool as the target unit generation tool, and generating the plug-in management unit in the second system.
In one embodiment, as shown in fig. 5, the multimedia data processing method further includes:
step 502, acquiring a first description language of a playing parameter; the first description language is a description language of the playing parameters in the first system.
The description language is a language which describes the playing parameters into a language which can be recognized by the terminal and has certain grammar rules.
Specifically, after the terminal acquires the playing parameter, the description language of the playing parameter is determined according to the syntactic structure of the playing parameter, so that the first description language of the playing parameter is acquired. For example, if the language for describing the playback parameters is a syntax structure of JAVA (computer programming language), the first description language obtained is JAVA. The language is a language in a first system of the terminal.
Step 504, convert the first description language into an intermediate description language.
Specifically, after acquiring the first description language, the terminal converts the first description language into a common description language, and the common description language can be used as a tool for performing language conversion on a first system and a second system of the terminal.
Step 506, converting the intermediate description language into a second description language according to the corresponding relation between the intermediate description language and the second description language; the second description language is a description language of the playing parameters in the second system.
Specifically, after the terminal obtains the intermediate description language, the terminal converts the intermediate description language into the second description language according to the corresponding relationship between the intermediate description language and the second description language. For example, the first description language in the first system of the terminal is JAVA language, the second description language in the second system is C + + (computer description language), and the intermediate description language may be different from the first description language and the second description language, and the first description language is first converted into the intermediate description language, and then the intermediate description language is converted into the second description language.
In the embodiment, a first description language of the playing parameter is acquired; the first description language is a description language of the playing parameters in the first system, the first description language is converted into an intermediate description language, the intermediate description language is converted into the second description language according to the corresponding relation between the intermediate description language and the second description language, and the purpose of realizing the compatibility of the first system and the second system in the terminal can be achieved.
In one embodiment, as shown in fig. 6, generating the target plug-in on the plug-in management unit according to the playing plug-in identifier includes:
step 602, obtaining one or more intermediate playing plug-ins corresponding to the playing plug-in identification from the candidate playing plug-in set.
Specifically, various types of playing plug-ins are stored in the second system of the terminal, and corresponding plug-ins can be acquired at the playing plug-in identification according to the playing plug-in identification in the multimedia playing request. It can be understood that each multimedia playing request includes one or more playing plug-in identifiers, and accordingly, one or more intermediate playing plug-ins corresponding to the playing plug-in identifiers are obtained.
And step 604, sequencing the intermediate playing plugins according to the priorities of the playing plugins to obtain an intermediate playing plugin sequence.
The priority is determined according to a priority identifier on the multimedia playing plug-in, or according to the sequence of the plug-ins on the plug-in management unit, and can determine the execution sequence of the plug-ins on the plug-in management unit, wherein the higher the priority is, the earlier the execution time is; the lower the priority, the later the execution time.
Specifically, after acquiring one or more intermediate playing plug-ins corresponding to the playing plug-in identification, the terminal sorts each intermediate playing plug-in according to the priority of the playing plug-ins to obtain an intermediate playing plug-in sequence. For example, the priority of playing a plug-in is represented by a number, the larger the number, the higher the priority; the priority of the middle playing plug-in 1 is 50, the priority of the middle playing plug-in 2 is 20, the middle playing plug-in 1 is ordered before the middle playing plug-in 2, when the plug-in is executed, the middle playing plug-in 1 is executed first, and then the middle playing plug-in 2 is executed.
And 606, packaging each intermediate playing plug-in the intermediate playing plug-in sequence into an integral plug-in.
Specifically, after the terminal obtains the sequence of the intermediate playing plug-ins, each intermediate playing plug-in the sequence can be packaged into an integral plug-in, so that the integral plug-ins can be integrally called after the same multimedia playing request is received, and the plug-in calling efficiency is improved.
Step 608, determining the whole plug-in as the target plug-in generated on the plug-in management unit.
Specifically, after the terminal obtains the whole plug-in, the whole plug-in is determined as the target plug-in, so that the target plug-in is directly called when a corresponding plug-in management unit is established, and the processing efficiency of the multimedia data is further improved. For example, a coding plug-in, a decoding plug-in, a rendering plug-in, a blurring algorithm plug-in, a filter plug-in, and the like are required for performing beauty processing on a portrait in a video, the plug-ins can be synthesized into a plug-in management unit of the portrait beauty plug-in, and when a play identifier of the portrait beauty processing exists in a multimedia play request, the plug-in management unit of the portrait beauty plug-in is directly called.
In this embodiment, one or more middle playing plugins corresponding to the playing plugin identifiers are obtained from the candidate playing plugin set, each middle playing plugin is sequenced according to the priority of the playing plugin to obtain a middle playing plugin sequence, each middle playing plugin in the middle playing plugin sequence is packaged into an integral plugin, and the integral plugin is determined to be a target plugin generated on the plugin management unit, so that the purpose of improving the multimedia data processing efficiency can be achieved.
In one embodiment, the multimedia data processing method further comprises: and canceling the generated result between the current plug-in and the target plug-in, and switching the state of the multimedia playing request to the state of the event before the initial request is not responded.
Specifically, when the terminal receives a multimedia playing request, corresponding plug-ins are searched in the second system according to playing plug-in identifiers in the request, when a certain plug-in is searched for to be abnormal, the operation of the plug-ins which are searched before and the searched plug-ins are cancelled, the plug-ins in the plug-in set are traversed in the mode of the initial multimedia playing request again, and the target plug-ins are regenerated.
In this embodiment, the exception handling capability in the multimedia data processing process can be improved by canceling the generation result between the current plug-in and the target plug-in and switching the state of the multimedia play request to the event state before the initial request is not responded.
In an embodiment, for example, the first system of the terminal is an Android operating system (Android), and the second system is a Linux operating system, where the Linux operating system is another operating system different from the Android operating system. The Android operating system can provide a video playing frame, the playing frame provides upper-layer application for video playing, Android coding and decoding in the prior art are mainly performed by Medialayer and Mediaodec, functional processing on images, videos and the like is not abundant, and the videos can be processed more abundantly in a plug-in mode through a pipeline multimedia architecture in a Gstreamer class in the Linux operating system. Specifically, the functions of acquisition, encoding, decoding, rendering, filtering and the like are realized in a plug-in mode. Finally, the modules are connected in series to achieve the expected effect. One advantage that is apparent is that insertion of specified effects can be made to the video. For example, the processing of beautifying the portrait in the video may be implemented by inserting the plug-in into the pipeline, or the processing of blurring the image may be implemented by adding a corresponding blurring algorithm to the pipeline, or the video may be processed in a hot-plug manner. As shown in fig. 7, pipelines can be combined with supporting pipelines, and can be conveniently cut down in the form of switches. The bottommost is all the plug-in sets in the framework, including the plug-ins owned by the Gstreamer framework and the customized plug-ins. For example, a filescr plug-in for reading a file or a gst-droid plug-in for transmitting video data to an Android operating system, etc. The Android operating system can communicate with the Linux operating system through a compatible interface. Therefore, the problems of complicated development program and high maintenance difficulty caused by Android operating system development are solved, and the compatibility of the multimedia part of the dual system is enhanced.
In an embodiment, as shown in fig. 8, when starting to play a video in the Android operating system, a related control interface or interface control function is called through a system compatible interface, a pipeline is established in the Linux operating system, a functional plug-in required for video playing is inserted into the pipeline to process the video, the processed video returns video data and audio data to the Android operating system through different transmission channels, and the processed video and audio are played in a system application in a visual form on an application interface supported by the Android operating system.
The following describes the multimedia data processing apparatus provided by the present invention, and the multimedia data processing apparatus described below and the multimedia data processing method described above may be referred to correspondingly.
In one embodiment, as shown in fig. 9, there is provided a multimedia data processing apparatus 900 including: a multimedia playing request receiving module 902, a plug-in management unit generating module 904, a target plug-in generating module 906, a target multimedia data obtaining module 908 and a target multimedia data transmitting module 910, wherein: a multimedia playing request receiving module 902, configured to receive a multimedia playing request sent by a first system, where the multimedia playing request carries a playing plugin identifier; a plug-in management unit generating module 904, configured to generate a plug-in management unit in the second system in response to the multimedia play request; a target plug-in generating module 906, configured to generate a target plug-in on the plug-in management unit according to the playing plug-in identifier; a target multimedia data obtaining module 908, configured to process the multimedia data by using a target plug-in to obtain target multimedia data; and a target multimedia data transmission module 910, configured to transmit the target multimedia data to the first system, so that the first system plays the target multimedia data in a visualized form.
In one embodiment, the target multimedia data transmission module 910 includes: the device comprises a target multimedia data type acquisition unit, a target transmission channel acquisition unit and a target multimedia data transmission unit, wherein: the target multimedia data type acquisition unit is used for acquiring a target multimedia data type; a target transmission channel obtaining unit, configured to obtain a target transmission channel corresponding to a target multimedia data type according to a correspondence between the target multimedia data type and the transmission channel; and the target multimedia data transmission unit is used for transmitting the target multimedia data to the first system through the target transmission channel.
In one embodiment, the plug-in management unit generating module 904 is configured to obtain a playing parameter in the multimedia playing request; determining a unit generation tool corresponding to the plug-in management unit as a target unit generation tool according to the corresponding relation between the playing parameters and the unit generation tools; and generating the plug-in management unit in the second system by utilizing the target unit generation tool.
In one embodiment, the multimedia data processing device is used for acquiring a first description language of the playing parameters; the first description language is a description language of the playing parameters in the first system; converting the first description language into an intermediate description language; converting the intermediate description language into a second description language according to the corresponding relation between the intermediate description language and the second description language; the second description language is a description language of the playing parameters in the second system.
In one embodiment, the target plug-in generation module 906 is configured to obtain one or more intermediate playing plug-ins corresponding to the playing plug-in identifier from the candidate playing plug-in set; sequencing each intermediate playing plug-in according to the priority of the playing plug-ins to obtain an intermediate playing plug-in sequence; packing each intermediate playing plug-in the intermediate playing plug-in sequence into an integral plug-in; and determining the whole plug-in as a target plug-in generated on the plug-in management unit.
In one embodiment, the multimedia data processing device is used for performing revocation processing on a generation result between the current plug-in and the target plug-in and switching the state of the multimedia playing request to an event state before the initial request is not responded.
Fig. 10 illustrates a physical structure diagram of an electronic device, and as shown in fig. 10, the electronic device may include: a processor (processor)1010, a communication Interface (Communications Interface)1020, a memory (memory)1030, and a communication bus 1040, wherein the processor 1010, the communication Interface 1020, and the memory 1030 communicate with each other via the communication bus 1040. Processor 1010 may invoke logic instructions in memory 1030 to perform a method of multimedia data processing, the method comprising: receiving a multimedia playing request sent by a first system, wherein the multimedia playing request carries a playing plug-in identifier; generating a plug-in management unit in the second system in response to the multimedia playing request; generating a target plug-in on a plug-in management unit according to the playing plug-in identification; processing the multimedia data by using the target plug-in to obtain target multimedia data; and transmitting the target multimedia data to the first system so that the first system plays the target multimedia data in a visual form.
Furthermore, the logic instructions in the memory 1030 can be implemented in software functional units and stored in a computer readable storage medium when the logic instructions are sold or used as independent products. Based on such understanding, the technical solution of the present invention may be embodied in the form of a software product, which is stored in a storage medium and includes instructions for causing a computer device (which may be a personal computer, a server, or a network device) to execute all or part of the steps of the method according to the embodiments of the present invention. And the aforementioned storage medium includes: a U-disk, a removable hard disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a magnetic disk or an optical disk, and other various media capable of storing program codes.
In another aspect, the present invention also provides a computer program product comprising a computer program stored on a non-transitory computer-readable storage medium, the computer program comprising program instructions which, when executed by a computer, enable the computer to perform a multimedia data processing method provided by the above methods, the method comprising: receiving a multimedia playing request sent by a first system, wherein the multimedia playing request carries a playing plug-in identifier; generating a plug-in management unit in the second system in response to the multimedia playing request; generating a target plug-in on a plug-in management unit according to the playing plug-in identification; processing the multimedia data by using the target plug-in to obtain target multimedia data; and transmitting the target multimedia data to the first system so that the first system plays the target multimedia data in a visual form.
In yet another aspect, the present invention also provides a non-transitory computer-readable storage medium having stored thereon a computer program, which when executed by a processor is implemented to perform the multimedia data processing method provided above, the method comprising: receiving a multimedia playing request sent by a first system, wherein the multimedia playing request carries a playing plug-in identifier; generating a plug-in management unit in the second system in response to the multimedia playing request; generating a target plug-in on a plug-in management unit according to the playing plug-in identification; processing the multimedia data by using the target plug-in to obtain target multimedia data; and transmitting the target multimedia data to the first system so that the first system plays the target multimedia data in a visual form.
The above-described embodiments of the apparatus are merely illustrative, and the units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of the present embodiment. One of ordinary skill in the art can understand and implement it without inventive effort.
Through the above description of the embodiments, those skilled in the art will clearly understand that each embodiment can be implemented by software plus a necessary general hardware platform, and certainly can also be implemented by hardware. With this understanding in mind, the above-described technical solutions may be embodied in the form of a software product, which can be stored in a computer-readable storage medium such as ROM/RAM, magnetic disk, optical disk, etc., and includes instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to execute the methods described in the embodiments or some parts of the embodiments.
Finally, it should be noted that: the above examples are only intended to illustrate the technical solution of the present invention, but not to limit it; although the present invention has been described in detail with reference to the foregoing embodiments, it will be understood by those of ordinary skill in the art that: the technical solutions described in the foregoing embodiments may still be modified, or some technical features may be equivalently replaced; and such modifications or substitutions do not depart from the spirit and scope of the corresponding technical solutions of the embodiments of the present invention.
Claims (10)
1. A method for processing multimedia data, comprising:
receiving a multimedia playing request sent by a first system, wherein the multimedia playing request carries a playing plug-in identifier;
generating a plug-in management unit in the second system in response to the multimedia playing request;
generating a target plug-in on the plug-in management unit according to the playing plug-in identification;
processing the multimedia data by using the target plug-in to obtain target multimedia data;
and transmitting the target multimedia data to the first system so that the first system plays the target multimedia data in a visualized form.
2. The method of claim 1, wherein the transmitting the target multimedia data to the first system comprises:
acquiring the target multimedia data type;
obtaining a target transmission channel corresponding to the target multimedia data type according to the corresponding relation between the target multimedia data type and the transmission channel;
and transmitting the target multimedia data to the first system through the target transmission channel.
3. The method of claim 1, wherein the generating a plug-in management unit in the second system in response to the multimedia play request comprises:
acquiring a playing parameter in the multimedia playing request;
determining a unit generation tool corresponding to the plug-in management unit as a target unit generation tool according to the corresponding relation between the playing parameters and the unit generation tool;
generating a plug-in management unit in the second system using the target unit generation tool.
4. The method of claim 3, further comprising:
acquiring a first description language of the playing parameters; the first description language is a description language of the playing parameters in the first system;
converting the first description language into an intermediate description language;
converting the intermediate description language into a second description language according to the corresponding relation between the intermediate description language and the second description language; the second description language is a description language of the playing parameter in the second system.
5. The method of claim 1, wherein the generating a target plug-in on the plug-in management unit according to the playing plug-in identifier comprises:
acquiring one or more intermediate playing plug-ins corresponding to the playing plug-in identification from a candidate playing plug-in set;
sequencing each intermediate playing plug-in according to the priority of the playing plug-ins to obtain an intermediate playing plug-in sequence;
packing each intermediate playing plug-in the intermediate playing plug-in sequence into an integral plug-in;
and determining the whole plug-in as a target plug-in generated on the plug-in management unit.
6. The method of any of claims 1-5, wherein the method further comprises:
canceling the generated result between the current plug-in and the target plug-in, and switching the state of the multimedia playing request to the state of the event before the initial request is not responded.
7. A multimedia data processing apparatus, comprising:
the multimedia playing request receiving module is used for receiving a multimedia playing request sent by a first system, wherein the multimedia playing request carries a playing plug-in identifier;
the plug-in management unit generating module is used for responding to the multimedia playing request and generating a plug-in management unit in the second system;
the target plug-in generating module is used for generating a target plug-in on the plug-in management unit according to the playing plug-in identification;
the target multimedia data obtaining module is used for processing the multimedia data by using the target plug-in to obtain target multimedia data;
and the target multimedia data transmission module is used for transmitting the target multimedia data to the first system so that the first system can play the target multimedia data in a visual form.
8. The multimedia data processing apparatus of claim 7, wherein the target multimedia data transmission module comprises:
the target multimedia data type acquisition unit is used for acquiring the target multimedia data type;
a target transmission channel obtaining unit, configured to obtain a target transmission channel corresponding to the target multimedia data type according to a correspondence between the target multimedia data type and the transmission channel;
and the target multimedia data transmission unit is used for transmitting the target multimedia data to the first system through the target transmission channel.
9. An electronic device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, characterized in that the steps of the multimedia data processing method according to any of claims 1 to 6 are implemented when the processor executes the program.
10. A non-transitory computer-readable storage medium, on which a computer program is stored, which, when being executed by a processor, carries out the steps of the multimedia data processing method according to any one of claims 1 to 6.
Priority Applications (2)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111290534.9A CN114095781B (en) | 2021-11-02 | Multimedia data processing method and device, electronic equipment and storage medium | |
| PCT/CN2022/106373 WO2023077866A1 (en) | 2021-11-02 | 2022-07-19 | Multimedia data processing method and apparatus, electronic device, and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111290534.9A CN114095781B (en) | 2021-11-02 | Multimedia data processing method and device, electronic equipment and storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN114095781A true CN114095781A (en) | 2022-02-25 |
| CN114095781B CN114095781B (en) | 2024-09-24 |
Family
ID=
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2023077866A1 (en) * | 2021-11-02 | 2023-05-11 | 北京字节跳动网络技术有限公司 | Multimedia data processing method and apparatus, electronic device, and storage medium |
Citations (11)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101827251A (en) * | 2010-05-07 | 2010-09-08 | 深圳创维－Rgb电子有限公司 | Method and device for playing network streaming media |
| CN107690084A (en) * | 2017-04-11 | 2018-02-13 | 腾讯科技（北京）有限公司 | The playing method and device of media file |
| CN108170503A (en) * | 2018-01-24 | 2018-06-15 | 腾讯科技（深圳）有限公司 | A kind of method, terminal and the storage medium of cross-system operation Android application |
| CN207742930U (en) * | 2018-03-09 | 2018-08-17 | 广州馨睿科电子科技有限公司 | A kind of multisystem audio playing apparatus |
| CN111416989A (en) * | 2020-04-28 | 2020-07-14 | 北京金山云网络技术有限公司 | Video live broadcast method and system and electronic equipment |
| CN111723219A (en) * | 2020-06-30 | 2020-09-29 | 百度在线网络技术（北京）有限公司 | Method, apparatus, device and storage medium for playing multimedia |
| CN111930340A (en) * | 2020-09-21 | 2020-11-13 | 北京技德系统技术有限公司 | Virtual audio device and method based on Linux and Android compatible technology |
| CN112269604A (en) * | 2020-11-11 | 2021-01-26 | 北京技德系统技术有限公司 | Window interaction method and device for compatibly running Android application on Linux |
| CN112269603A (en) * | 2020-10-16 | 2021-01-26 | 北京技德系统技术有限公司 | Graphic display method and device for compatibly running Android application on Linux |
| CN113220474A (en) * | 2021-04-21 | 2021-08-06 | 麒麟软件有限公司 | Android application sharing Linux system screen data method based on Android-EMUGL |
| WO2021159672A1 (en) * | 2020-02-11 | 2021-08-19 | 深圳壹账通智能科技有限公司 | Face image recognition method and apparatus |
Patent Citations (11)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101827251A (en) * | 2010-05-07 | 2010-09-08 | 深圳创维－Rgb电子有限公司 | Method and device for playing network streaming media |
| CN107690084A (en) * | 2017-04-11 | 2018-02-13 | 腾讯科技（北京）有限公司 | The playing method and device of media file |
| CN108170503A (en) * | 2018-01-24 | 2018-06-15 | 腾讯科技（深圳）有限公司 | A kind of method, terminal and the storage medium of cross-system operation Android application |
| CN207742930U (en) * | 2018-03-09 | 2018-08-17 | 广州馨睿科电子科技有限公司 | A kind of multisystem audio playing apparatus |
| WO2021159672A1 (en) * | 2020-02-11 | 2021-08-19 | 深圳壹账通智能科技有限公司 | Face image recognition method and apparatus |
| CN111416989A (en) * | 2020-04-28 | 2020-07-14 | 北京金山云网络技术有限公司 | Video live broadcast method and system and electronic equipment |
| CN111723219A (en) * | 2020-06-30 | 2020-09-29 | 百度在线网络技术（北京）有限公司 | Method, apparatus, device and storage medium for playing multimedia |
| CN111930340A (en) * | 2020-09-21 | 2020-11-13 | 北京技德系统技术有限公司 | Virtual audio device and method based on Linux and Android compatible technology |
| CN112269603A (en) * | 2020-10-16 | 2021-01-26 | 北京技德系统技术有限公司 | Graphic display method and device for compatibly running Android application on Linux |
| CN112269604A (en) * | 2020-11-11 | 2021-01-26 | 北京技德系统技术有限公司 | Window interaction method and device for compatibly running Android application on Linux |
| CN113220474A (en) * | 2021-04-21 | 2021-08-06 | 麒麟软件有限公司 | Android application sharing Linux system screen data method based on Android-EMUGL |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2023077866A1 (en) * | 2021-11-02 | 2023-05-11 | 北京字节跳动网络技术有限公司 | Multimedia data processing method and apparatus, electronic device, and storage medium |
Also Published As
| Publication number | Publication date |
|---|---|
| WO2023077866A1 (en) | 2023-05-11 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN106302103B (en) | Information processing method and system and application server | |
| CN104837052A (en) | Playing method of multimedia data and device | |
| US11699073B2 (en) | Network off-line model processing method, artificial intelligence processing device and related products | |
| CN110442327A (en) | A kind of application builds method, apparatus, server | |
| CN110418181B (en) | Service processing method and device for smart television, smart device and storage medium | |
| CN117389647A (en) | Plug-in generation method, application development method, device, equipment and medium | |
| CN114095781A (en) | Multimedia data processing method and device, electronic equipment and storage medium | |
| CN114095781B (en) | Multimedia data processing method and device, electronic equipment and storage medium | |
| CN116932712A (en) | Multi-mode input interactive information generation method, device, equipment and medium | |
| CN109525550B (en) | Data message processing method, device and system | |
| CN107743116B (en) | Information transmission method and device, computer equipment and computer readable storage medium | |
| KR20030085518A (en) | Method and equipment for managing interactions in the mpeg-4 standard | |
| CN114566173A (en) | Audio mixing method, device, equipment and storage medium | |
| CN110147518A (en) | Local resource access method, local resource access mechanism and electronic equipment | |
| CN113453075A (en) | Method and device for decoding multi-channel video, electronic equipment and storage medium | |
| CN107426611B (en) | multi-path output method and system based on video transcoding | |
| CN114007077B (en) | Method and device for processing multimedia resources, electronic equipment and storage medium | |
| CN107070910B (en) | Communication method and electronic equipment | |
| CN116204201B (en) | Service processing method and device | |
| CN114666514B (en) | Data processing method, device, electronic equipment and storage medium | |
| CN108848398A (en) | A kind of distribution method, device, terminal and the storage medium of local barrage message | |
| CN116132214B (en) | Event transmission method, device, equipment and medium based on event bus model | |
| CN114499759B (en) | Message generation method and device, electronic equipment and storage medium | |
| CN111866491B (en) | Master-slave interactive 3D video display method and system under multi-person scene | |
| CN114817006B (en) | Stack information writing method, device, equipment and medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| TA01 | Transfer of patent application right | ||
| TA01 | Transfer of patent application right |
Effective date of registration: 20220407 Address after: 100041 B-0035, 2 floor, 3 building, 30 Shixing street, Shijingshan District, Beijing. Applicant after: BEIJING BYTEDANCE NETWORK TECHNOLOGY Co.,Ltd. Address before: 2005, floor 2, No. 39, West Street, Haidian District, Beijing 100082 Applicant before: Beijing jingling Information System Technology Co.,Ltd. |
|
| GR01 | Patent grant |