CN116886673A

CN116886673A - Method and device for determining multimedia processing flow, electronic equipment and storage medium

Info

Publication number: CN116886673A
Application number: CN202211515801.2A
Authority: CN
Inventors: 黄浩成
Original assignee: Guangzhou Kaidelian Software Technology Co ltd
Current assignee: Guangzhou Kaidelian Software Technology Co ltd
Priority date: 2022-11-28
Filing date: 2022-11-28
Publication date: 2023-10-13

Abstract

The application discloses a method and a device for determining a multimedia processing flow, electronic equipment and a storage medium, and belongs to the technical field of computers. The method comprises the following steps: receiving multimedia data sent by source equipment, and acquiring source parameters and target parameters of the multimedia data; determining a flow node for processing the multimedia data according to the source parameter and the target parameter; and determining a processing link based on the flow node to process the multimedia data. According to the technical scheme, the flow node can be determined according to the operation required by multimedia data processing, so that the problems of long interaction delay time and poor user experience caused by adding unnecessary flow nodes when the multimedia data is processed in the past are avoided. Therefore, processing resources can be saved, interaction delay time is shortened, and high-quality interactive teaching is improved.

Description

Method and device for determining multimedia processing flow, electronic equipment and storage medium

Technical Field

The application belongs to the technical field of computers, and particularly relates to a method and a device for determining a multimedia processing flow, electronic equipment and a storage medium.

Background

With the continuous development of internet technology, network teaching also becomes a mainstream teaching mode. The network teaching can integrate high-quality teaching resources, promote interactive teaching based on courses and course content construction, and the network teaching can lighten the course construction intensity of teachers. The network teaching generally uses a teacher camera, a student camera, teacher computer courseware and the like as source equipment, and the source equipment is commonly connected to a recording and broadcasting host, and the recording and broadcasting host can synchronize the obtained display pictures to a local teaching end and a remote teaching section according to the display modes of all pictures of the set rules.

In general, conventional streaming media processing uses a Pipeline parallel Pipeline processing concept, and the Pipeline mode is a data processing mode, which divides the whole data processing process into a plurality of nodes and processes data in a Pipeline manner. However, in the Pipeline mode, for example, the processing of multimedia data between the source device and the recording and playing host is fixed, and in some cases, because some attributes of the multimedia data in the display picture constructed by the recording and playing host and the multimedia data acquired by the source device are communicated, an original part of processing nodes are not needed, so that during the processing of the multimedia data, the processing efficiency of the multimedia data is greatly affected by the creation node and the execution node, resource waste is caused, and interaction speed is affected.

Disclosure of Invention

The embodiment of the application provides a method, a device, electronic equipment and a storage medium for determining a multimedia processing flow, which can solve the problems that the efficiency of a streaming media processing mode is low, and the same flow nodes are repeatedly created so that the interaction delay time is long and the user experience is poor. By the method for determining the multimedia processing flow, the flow nodes can be determined according to the operation required by multimedia data processing, so that processing resources are saved, interaction delay time is shortened, and high-quality interactive teaching is improved.

In a first aspect, an embodiment of the present application provides a method for determining a multimedia processing flow, where the method includes:

receiving multimedia data sent by source equipment, and acquiring source parameters and target parameters of the multimedia data;

determining a flow node for processing the multimedia data according to the source parameter and the target parameter;

and determining a processing link based on the flow node to process the multimedia data.

Further, determining a processing link based on the flow node to process the multimedia data, including:

if the flow nodes are at least two, determining at least one candidate link formed between the starting node and the ending node based on the at least two flow nodes;

Determining one of the candidate links as a processing link based on the performance parameter of the at least one candidate link, and processing the multimedia data by the processing link; wherein the performance parameters include a processing efficiency parameter and a processing resource occupancy parameter.

Further, determining a flow node for processing the multimedia data according to the source parameter and the target parameter, including:

and determining the flow node of the multimedia data according to the same part and different parts of the target parameter and the source parameter.

Further, determining a flow node of the multimedia data according to the same part and different parts of the target parameter and the source parameter includes:

if the different parts of the target parameter and the source parameter comprise resolution ratios, determining that a flow node of the multimedia data comprises a scaling node;

if the different parts of the target parameter and the source parameter comprise formats, determining that a flow node of the multimedia data comprises a format conversion node;

and if the different parts of the target parameter and the source parameter comprise definition indexes, determining that the flow node of the multimedia data comprises a sharpening node and/or a denoising node.

Further, after determining the flow node of the multimedia data according to the same part and different parts of the target parameter and the source parameter, the method further comprises:

reading whether a flow node of the multimedia data exists in a node list processed for the current source equipment;

if so, multiplexing the created nodes in the node list;

if not, generating a new node according to the source parameter and the target parameter, and using the new node as a flow node of the multimedia data.

Further, the source parameter is read by a source device based on the multimedia data;

the target parameter is determined based on the display result of the multimedia data in the picture of the recording and playing host.

In a second aspect, an embodiment of the present application provides a device for determining a multimedia processing flow, where the device includes:

the acquisition module is used for receiving the multimedia data sent by the source equipment and acquiring the source parameters and the target parameters of the multimedia data;

the determining module is used for determining a flow node for processing the multimedia data according to the source parameter and the target parameter;

and the processing module is used for determining a processing link based on the flow node so as to process the multimedia data.

Further, the processing module is specifically configured to:

In a third aspect, an embodiment of the present application provides an electronic device, including a processor, a memory, and a program or instruction stored on the memory and executable on the processor, the program or instruction implementing the steps of the method according to the first aspect when executed by the processor.

In a fourth aspect, embodiments of the present application provide a readable storage medium having stored thereon a program or instructions which when executed by a processor perform the steps of the method according to the first aspect.

In a fifth aspect, an embodiment of the present application provides a chip, where the chip includes a processor and a communication interface, where the communication interface is coupled to the processor, and where the processor is configured to execute a program or instructions to implement a method according to the first aspect.

In the embodiment of the application, multimedia data sent by source equipment is received, and source parameters and target parameters of the multimedia data are obtained; determining a flow node for processing the multimedia data according to the source parameter and the target parameter; and determining a processing link based on the flow node to process the multimedia data. By the method for determining the multimedia processing flow, the flow nodes can be determined according to the operation required by multimedia data processing, so that processing resources are saved, interaction delay time is shortened, and high-quality interactive teaching is improved. The problems of long interaction delay time and poor user experience caused by adding unnecessary flow nodes when processing multimedia data in the past are avoided.

Drawings

Fig. 1 is a flow chart of a method for determining a multimedia processing flow according to a first embodiment of the present application;

fig. 2 is a flow chart of a method for determining a multimedia processing flow according to a second embodiment of the present application;

fig. 3 is a schematic structural diagram of a determining device for a multimedia processing flow according to a third embodiment of the present application;

fig. 4 is a schematic structural diagram of an electronic device according to a fourth embodiment of the present application.

Detailed Description

In order to make the objects, technical solutions and advantages of the present application more apparent, the following detailed description of specific embodiments of the present application is given with reference to the accompanying drawings. It is to be understood that the specific embodiments described herein are merely illustrative of the application and are not limiting thereof. It should be further noted that, for convenience of description, only some, but not all of the matters related to the present application are shown in the accompanying drawings. Before discussing exemplary embodiments in more detail, it should be mentioned that some exemplary embodiments are described as processes or methods depicted as flowcharts. Although a flowchart depicts operations (or steps) as a sequential process, many of the operations can be performed in parallel, concurrently, or at the same time. Furthermore, the order of the operations may be rearranged. The process may be terminated when its operations are completed, but may have additional steps not included in the figures. The processes may correspond to methods, functions, procedures, subroutines, and the like.

The technical solutions of the embodiments of the present application will be clearly described below with reference to the drawings in the embodiments of the present application, and it is apparent that the described embodiments are some embodiments of the present application, but not all embodiments. All other embodiments, which are obtained by a person skilled in the art based on the embodiments of the present application, fall within the scope of protection of the present application.

The terms first, second and the like in the description and in the claims, are used for distinguishing between similar elements and not necessarily for describing a particular sequential or chronological order. It is to be understood that the data so used may be interchanged, as appropriate, such that embodiments of the present application may be implemented in sequences other than those illustrated or described herein, and that the objects identified by "first," "second," etc. are generally of a type, and are not limited to the number of objects, such as the first object may be one or more. Furthermore, in the description and claims, "and/or" means at least one of the connected objects, and the character "/", generally means that the associated object is an "or" relationship.

The method, the device, the electronic equipment and the storage medium for determining the multimedia processing flow provided by the embodiment of the application are described in detail below through specific embodiments and application scenes thereof with reference to the accompanying drawings.

Example 1

Fig. 1 is a flow chart of a method for determining a multimedia processing flow according to an embodiment of the application.

As shown in fig. 1, the method specifically comprises the following steps:

s101, receiving multimedia data sent by source equipment, and acquiring source parameters and target parameters of the multimedia data.

Firstly, the execution scene of the scheme can be a scene of processing the picture shot by the teacher camera, the picture shot by the student camera, the picture shot by the computer and the like by using the recording and broadcasting host, and carrying out local display or network synchronous display.

Based on the above usage scenario, it can be understood that the execution subject of the present application may be a recording and playing host, which is not limited herein.

In this scheme, the method is executed by the recording and broadcasting host.

The recording and broadcasting host can be a device which is finished by a computer and a network and takes live broadcasting guide broadcasting as a main part and has a partial video recording function and has the property of a server. The recording and broadcasting host comprises a Gao Qinglu broadcasting host, an interactive recording and broadcasting host and a studio recording and broadcasting host. In this scheme, can use Gao Qinglu to broadcast the host computer, gao Qinglu broadcast the host computer and be the integrated record of a section collection courseware record, live broadcast, order broadcast, intelligent guide broadcast and storage etc. multifunctional in an organic whole broadcast equipment, based on embedded hardware architecture design, the product is highly stable. The equipment supports audio and video acquisition and recording of 5 scenes (teacher panorama, teacher close-up, student panorama, student close-up and computer signals) and realizes full-automatic tracking and broadcasting guide by matching with a tracking system.

The source device may be a device used for shooting a teacher or a student, or may be a device used for playing PPT (Microsoft Office Power Point, presentation) used for a teacher to explain a course. The photographing device may be a camera, for example, a camera for photographing a teacher panorama, a camera for a teacher close-up, a camera for a student panorama, and a camera for a student close-up.

The multimedia data may be video pictures taken using a source device or courseware played. For example, when a student or a teacher is photographed using a photographing apparatus, the multimedia data may be a photographed video picture of the student or the teacher; when the teacher uses the device to give lessons, the multimedia data can be a video file of the PPT used by the teacher for giving lessons.

The source parameters may be parameters of video pictures shot by the source device, including parameters such as resolution, format, definition, bit rate, frame rate, etc., and may also be parameters of video files recorded by courseware when the courseware is played by the source device, including parameters such as resolution, color, sound, etc.

The target parameter may be a parameter of a video picture or courseware to be played, which is determined by the recording and playing host according to the switching mechanism. The switching mechanism may be a preset mechanism, and may be a mechanism for switching video content in a main window or an auxiliary window according to a determination of a person sound or an action in a picture. For example, when a student is identified to raise his hand in the picture transmitted by the student camera, the main window may be switched to a student close-up picture taken by the student close-up camera that is speaking the student. At this time, the teacher close-up image photographed by the teacher close-up camera at the position of the main window is switched to the auxiliary window, the window coordinates are changed at this time, the corresponding resolution is also changed, and the changed resolution can be the target parameter.

The resolution is determined according to the window size and window coordinates, and generally, the upper left corner of the screen of the computer is defined as the origin of coordinates, namely, (0, 0), the horizontal direction is the right direction of the X-axis, the vertical direction is the downward direction of the Y-axis, the coordinates are in px (pixel point), the range of the X-axis and the Y-axis coordinate system is determined by the screen resolution, if the range of the X-axis is [0, 1920], the range of the Y-axis is [0, 1080], the resolution is 1920X 1080. A picture also has its coordinate system, the "picture origin" being located in the upper left corner of the picture window. Starting from the origin of coordinates, the horizontal right direction is the forward direction of the X axis and the vertical direction is the downward direction is the forward direction of the Y axis. When the size of the picture setting is equal to the resolution of the screen and the picture is maximized, the origin of the picture is coincident with the origin of coordinates; when the picture is smaller than the screen resolution, the picture locates its display position by the "picture origin". If the resolution of the main window is 1024×768, the origin of the picture coincides with the origin of the coordinates, the origin of the picture is (0, 0), and the window length is 1024px and the width is 768px; when switching to the auxiliary window of the student close-up screen, the resolution may be 240 x 180, and correspondingly, the screen origin is (1025, 769), the window length is 240px, and the width is 180px.

The recording and playing host can receive the multimedia data transmitted by the source device through Wi-Fi (wireless network). Wireless networks are a wireless networking technology, in the past, computers were connected by network cables, while Wi-Fi is networked by radio waves; it is common to use a wireless router, and the effective coverage area of the wireless router can be networked by Wi-Fi connection, and if the wireless router is connected to an ADSL (Asymmetric Digital Subscriber Line ) line or another internet surfing line, the wireless router is also called a hotspot. The ADSL line, i.e. the asymmetric digital subscriber line, is a new high-speed broadband technology running on the original common telephone line and has the characteristics of high downlink speed, wide bandwidth, excellent performance and the like.

In the scheme, when the student video is acquired by using the student video camera, the video data is encoded by the camera after the video data is shot, the multimedia data which can be transmitted is obtained after encoding, and the multimedia data is transmitted to the recording and broadcasting host through a wireless communication technology. After the recording and broadcasting host obtains the multimedia data transmitted by the source equipment, the video decoder is used for decoding the video, and the source parameters and the target parameters of the multimedia data can be obtained after decoding. Since the original video data is too huge, both physical storage and network transmission are burdensome, encoding is required to scale, and in particular, h.264 encoding can be performed on the video. h.264 is a highly scaled digital video codec standard, has a strong error-resistant characteristic, and can be suitable for video transmission in wireless channels with high packet loss rate and serious interference. And meanwhile, hierarchical coding transmission under different network resources is supported, so that stable image quality is obtained. The method is also suitable for video transmission in different networks, and has good network affinity. Wireless communication is a long-distance transmission communication between a plurality of nodes without transmission via conductors or cables, and can be performed by radio, or the like.

Taking resolution as an example, the recording and playing host can obtain the resolution parameters of the student camera in advance, and obtain the window position where the student picture recorded by the current student camera is located, for example, a main window or an auxiliary window, specifically, an auxiliary window at what position, and the like. The source parameters may then be determined from the resolution parameters at the time of the shot, and the target parameters may be determined from the window position and the window size.

S102, determining a flow node for processing the multimedia data according to the source parameter and the target parameter.

A process node may be a series of link steps in a process, with a process task consisting of several nodes. The node types comprise task nodes, active nodes, branch nodes, aggregation nodes, decision nodes and edge nodes, and the flow logic is realized through the combination of the nodes of different types. In this scheme, when processing the multimedia data, the corresponding flow node can be determined according to the operation to be used in the processing, where the processing of the multimedia data is determined according to the source parameter and the target parameter. For example, the resolution of the source parameter is 1920×1080, and the target resolution is 1024×768, when the multimedia data is processed, the original video is scaled, and correspondingly, the flow node is a scaling node.

The recording and broadcasting host can determine specific operation to be performed during conversion according to the source parameter and the target parameter of the multimedia data, and determine corresponding flow nodes according to the specific operation.

On the basis of the above technical solution, optionally, determining a flow node for processing the multimedia data according to the source parameter and the target parameter includes:

Generating the flow node using different portions of the target parameter and the source parameter, e.g., the source parameter and the target parameter have the same resolution, may not require generating a scaling node to scale the resolution of the multimedia data.

In the scheme, the same part and different parts of the target parameter and the source parameter are compared, so that the flow node is determined, the flow node can be automatically determined, subsequent processing is carried out based on the flow node, the problem that the node has no practical processing meaning after the node is established in a fixed processing mode is avoided, for example, a scaling node is established, the source parameter and the target parameter of the multimedia are 1080P, that is, scaling processing is not needed, and therefore, resource waste and longer delay caused by processing are solved.

On the basis of the above technical solution, optionally, determining the flow node of the multimedia data according to the same part and different parts of the target parameter and the source parameter includes:

In this scheme, the resolution may be the precision of the screen image, which refers to how many pixels the display can display. The higher the resolution, the greater the number of pixels and the more accurate the sensed image. In the case of the same screen size, the higher the resolution, the finer and finer the display effect, which can be expressed as horizontal pixels by vertical pixels.

When the different parts of the target parameter and the source parameter are resolution ratios during processing of the multimedia data, the original multimedia data is determined to be scaled, and the flow node is further determined to be a scaling node according to the required scaling operation.

The format may be a video format, i.e. a format when video is saved. Video file formats are classified differently, e.g., microsoft video includes: wmv, asf, asx format, real Player includes: the rm, rmvb formats, MPEG video includes: mpg, mpeg, mpe format, cell phone video includes: 3gp format, apple video includes: mov format, sony video includes: mp4, m4v format, other common videos include: avi, dat, mkv, flv, vob format.

Because the video shot by the camera and the video played by the recording and playing host computer may have different formats, after the recording and playing host computer receives the video transmitted by the camera through the wireless communication technology, if the different parts are different in format, the process node is further determined to be a format conversion node according to the required format conversion operation. The format conversion may be performed by a video converter, which is a tool for converting video formats such as mp4, avi, and mov.

The sharpness may be the sharpness of video quality and sound quality, and if the video quality is not sharp, the sharpness of the video quality may be adjusted by adjusting the sharpening parameters. Specifically, the method can utilize video editing software to adjust, put video into video software, correspondingly adjust sharpening parameters according to source parameters and target parameters, and correspondingly, after determining to perform sharpening operation when processing multimedia data, further determine that the flow nodes comprise sharpening nodes. If the tone quality of the video is unclear, the video needs to be subjected to denoising processing, namely processing for eliminating video noise. Specifically, the processing can be performed by using an audio filter, the threshold value of the audio filter is correspondingly adjusted according to the source parameter and the target parameter, and accordingly, after the denoising operation is determined to be performed when the multimedia data is processed, it can be further determined that the flow node comprises a denoising node. The audio filter is actually an audio preset, and some parameters used for audio conversion are saved.

In the scheme, different parts of the target parameter and the source parameter are thinned into the resolution, format and definition indexes, the target parameter and the source parameter are respectively compared, and then the flow node is determined, so that the determination of the flow node is more accurate, the processing resources are further saved, the processing speed is further improved, the bidirectional interaction delay time is shortened, and high-quality interactive teaching is provided.

S103, determining a processing link based on the flow node to process the multimedia data.

The processing link may be a necessary flow to connect all the required flow nodes in the multimedia data processing to determine the processing of the multimedia data. For example, when the student camera transmits the shot video to the recording and broadcasting host for processing, the video is firstly subjected to video coding, then transmitted to the recording and broadcasting host through a wireless communication technology, the recording and broadcasting host decodes the video, and then enters the next stage for displaying or encoding live broadcast through distribution processing and the like. If the source parameter is converted into the target parameter and the scaled flow node is needed, and the video is to be directly played as a live picture, the processing link is decoding-distributing-scaling-displaying.

The processing of the multimedia data may be a process in which the recording and playing host performs different processing on the multimedia data according to different process nodes of the processing link, so as to obtain the multimedia data with the target parameters. The processing according to the flow nodes is executed according to the sequence of the flow nodes.

On the basis of the above technical solution, optionally, determining a processing link based on the flow node to process the multimedia data includes:

In this scheme, since the flow node is determined according to the source parameter and the target parameter of the multimedia data, for example, if the source parameter and the target parameter are different only in resolution, the flow node is scaled only, and accordingly, the processing link is only one; if the source and destination parameters differ in resolution and format, the flow node may be a scaling node and a format conversion node, while in determining the processing links, there may be three processing links, one decoding-distributing-scaling-format conversion-display, the second decoding-distributing-format conversion-scaling-display, and the third decoding-distributing-format conversion and scaling-display. Wherein the third format conversion and scaling is parallel processing.

The performance parameter may be a basis for selecting a candidate link, including a processing efficiency parameter and a processing resource occupancy parameter. The processing efficiency parameter may be determined based on the processing speed. The faster the processing speed, the higher the processing efficiency; the slower the processing speed, the lower the processing efficiency. The processing resource occupancy parameter may be an amount of processing occupied GPUs (graphics processing unit graphics processors). The more GPUs the process occupies, the greater the process resource occupancy parameters; the fewer GPUs that are occupied by processing, the smaller the processing resource occupancy parameters.

The GPU is a microprocessor which is specially used for carrying out image and graphic related operation on personal computers, workstations, game machines and some mobile devices (such as tablet computers, smart phones and the like), so that the dependence of a display card on a CPU (central processing unit central processing unit) is reduced, and partial original CPU work is carried out.

In this scheme, the candidate link may be selected as the processing link according to the processing requirement and the performance parameter. For example, if the flow node is a scaling node and a format conversion node, and there are three candidate links, if the processing time required at this time is the shortest, the processing efficiency parameters of the three candidate links are compared. The first candidate link decoding-distribution-scaling-format conversion-display processing efficiency parameter is 70%, the second candidate link decoding-distribution-format conversion-scaling-display processing efficiency parameter is 50%, and the third candidate link decoding-distribution-format conversion and scaling-display processing efficiency parameter is 90%, then the third candidate link is selected because the greater the processing efficiency parameter, the faster the processing speed and the shorter the processing time. If the processing resource occupation parameter of the decoding-distributing-scaling-format conversion-display of the first candidate link is 5, the processing resource occupation parameter of the decoding-distributing-format conversion-scaling-display of the second candidate link is 7, and the processing resource occupation parameter of the decoding-distributing-format conversion-scaling-display of the third candidate link is 9, the first candidate link is selected because the smaller the processing resource occupation parameter is, the smaller the occupied GPU number is.

In the scheme, by setting the mode of determining the processing link based on the performance parameters of the candidate links, the corresponding performance parameters can be automatically searched according to the current requirements, and the optimal candidate links are selected, so that the effects of saving processing resources and improving processing speed are achieved. Further shortens the two-way interaction delay time and provides high-quality interaction teaching.

On the basis of the technical scheme, optionally, the source parameters are read by source equipment based on the multimedia data;

In the scheme, the relevant parameters including resolution, format, definition and the like are recorded when the source equipment leaves the factory, and the relevant parameters of the source equipment are parameters of the video shot by using the equipment. For example, the format parameter of the source device is mp4, and correspondingly, the format parameter in the source parameter is mp4.

The reading can be a process that the recording and broadcasting host sends out a command for reading the source parameters, the command is transmitted to the source equipment through the wireless communication technology, the source equipment receives the command and inquires the source equipment parameters recorded in the chip, and the parameter is transmitted to the recording and broadcasting host through the wireless communication technology after the inquiry is successful.

The determining the target parameter may be a process that the recording host determines the relevant parameter according to the display frame, for example, the display frame in the recording host is a close-up frame of the student, the resolution of the frame is 800×600, and the resolution parameter in the target parameter is further determined to be 800×600.

In the scheme, the source parameters are determined by reading the source equipment parameters and the target parameters are determined by recording and broadcasting the picture display results in the host, so that the speed of determining the source parameters and the target parameters can be improved, and the speed of processing the multimedia data is improved to a certain extent.

In this embodiment, multimedia data sent by a source device is received, and a source parameter and a target parameter of the multimedia data are obtained; determining a flow node for processing the multimedia data according to the source parameter and the target parameter; and determining a processing link based on the flow node to process the multimedia data. By the method for determining the multimedia processing flow, the flow nodes can be determined according to the operation required by multimedia data processing, so that processing resources are saved, interaction delay time is shortened, and high-quality interactive teaching is improved. The problems of long interaction delay time and poor user experience caused by adding unnecessary flow nodes when processing multimedia data in the past are avoided.

Example two

Fig. 2 is a flow chart of a method for determining a multimedia processing flow according to a second embodiment of the present application.

As shown in fig. 2, the method specifically comprises the following steps:

s201, receiving multimedia data sent by source equipment, and acquiring source parameters and target parameters of the multimedia data.

S202, determining a flow node for processing the multimedia data according to the source parameter and the target parameter.

S203, reading whether a flow node of the multimedia data exists in a node list processed for the current source equipment; if so, executing S204; if not, S205 is performed.

The node list may be a database table stored in a database containing all the process nodes that have been created, and this database table contains the process node names and specific operation modes, and may be expressed as: "flow node name-specific operation mode". For example, if the resolution of the source parameter is 1920×1080 and the target parameter is 1280×720, the flow node is determined to be the scaling node, the specific operation mode may be represented as 1920×1080-1280×720, and the mode stored in the node list may be represented as: "scaling node 1920 x 1080-1280 x 720".

The record playing host can inquire whether a flow node of the multimedia data exists or not by calling a node list stored in a database.

S204, multiplexing the created nodes in the node list.

The created node may be a used flow node, for example, when the first multimedia data is processed, the used flow node is a scaling node, and the specific operation mode is 1920×1080-1280×720, and if the second multimedia data is still processed by using the scaling node, the specific operation mode is 1920×1080-1280×720, the existence of the multimedia data flow node is considered. At the moment, a real physical node is not required to be created, only one node is required to be virtualized, and then the created node is used for processing the multimedia data. Multiplexing the created nodes not only ensures that the names of the flow nodes are the same, but also ensures that the specific operation modes are completely the same, otherwise, the created nodes cannot be multiplexed.

Multiplexing may be a process of processing multimedia data using an existing pipe, since after determining a flow node, corresponding processing is performed in the pipe, if it is found after querying the database table that the flow node already exists, it may be determined that a corresponding processing pipe exists, and then the pipe may be directly used when performing the next processing.

It will be appreciated that multiplexing supported by this scheme is that only multimedia data sent out by the sink device is available. Specifically, it may be considered that this solution maintains a node list for each channel, and in the case that the current channel is a channel between the source device and the recording and playing host, and the source parameter and the target parameter are different, the created node list may be preferentially read, if the created node list is already created, the created node list may be directly multiplexed, and if the created node list is not created, the node needs to be created and used.

And S205, generating a new node according to the source parameter and the target parameter, and using the new node as a flow node of the multimedia data.

The new node may be a new flow node created in the database table when there is no flow node needed in the node list, and accordingly, a new pipe for processing multimedia data is created. For example, when it is determined that the multimedia data processing is to be scaled, the specific operation mode is 1920×1080-600×360 (i.e. the source parameter is 1920×1080 and the target parameter is 600×360), and only the data of scaled nodes to 1920×1080-1280×720 exist in the node list, it is determined that there is no created node, and a new node is needed, a piece of data is first created in the database table, where the data is: scaling nodes 1920 x 1080-600 x 360, and then creating a corresponding pipeline for processing multimedia data to perform multimedia data processing.

S206, determining a processing link based on the flow node to process the multimedia data.

In this embodiment, by querying the node list to determine whether there is a flow node of the multimedia data, the actually required node may be dynamically allocated, so as to save processing resources more. The parallel processing flow formed by the nodes is automatically shortened, so that the response time of the whole multimedia is further improved, the two-way interaction delay time is shortened, and high-quality interaction teaching is provided. And complex node relation construction and channel management by a designer are avoided.

Example III

Fig. 3 is a schematic structural diagram of a determining device for a multimedia processing flow according to a third embodiment of the present application.

As shown in fig. 3, the method specifically includes the following steps:

an obtaining module 301, configured to receive multimedia data sent by a source device, and obtain a source parameter and a target parameter of the multimedia data;

a determining module 302, configured to determine a flow node for processing the multimedia data according to the source parameter and the target parameter;

a processing module 303, configured to determine a processing link based on the flow node, so as to process the multimedia data.

Further, the processing module 303 is specifically configured to:

In this embodiment, an obtaining module is configured to receive multimedia data sent by a source device, and obtain a source parameter and a target parameter of the multimedia data; the determining module is used for determining a flow node for processing the multimedia data according to the source parameter and the target parameter; and the processing module is used for determining a processing link based on the flow node so as to process the multimedia data. By the determination device of the multimedia processing flow, the flow nodes can be determined according to the operation required by multimedia data processing, so that processing resources are saved, interaction delay time is shortened, and high-quality interactive teaching is improved. The problems of long interaction delay time and poor user experience caused by adding unnecessary flow nodes when processing multimedia data in the past are avoided.

The determining device of the multimedia processing flow in the embodiment of the application can be a device, and can also be a component, an integrated circuit or a chip in the terminal. The device may be a mobile electronic device or a non-mobile electronic device. By way of example, the mobile electronic device may be a cell phone, tablet computer, notebook computer, palm computer, vehicle mounted electronic device, wearable device, ultra-mobile personal computer (ultra-mobile personal computer, UMPC), netbook or personal digital assistant (personal digital assistant, PDA), etc., and the non-mobile electronic device may be a server, network attached storage (Network Attached Storage, NAS), personal computer (personal computer, PC), television (TV), teller machine or self-service machine, etc., and embodiments of the present application are not limited in particular.

The device for determining the multimedia processing flow in the embodiment of the application can be a device with an operating system. The operating system may be an Android operating system, an ios operating system, or other possible operating systems, and the embodiment of the present application is not limited specifically.

The determining device for a multimedia processing flow provided in the embodiment of the present application can implement each process implemented by the method embodiments of fig. 1 to fig. 2, and in order to avoid repetition, a description is omitted here.

Example IV

As shown in fig. 4, the embodiment of the present application further provides an electronic device 400, which includes a processor 401, a memory 402, and a program or an instruction stored in the memory 402 and capable of running on the processor 401, where the program or the instruction implements each process of the above-mentioned embodiment of the method for determining a multimedia processing flow when executed by the processor 401, and the same technical effects can be achieved, and for avoiding repetition, a description is omitted herein.

The electronic device in the embodiment of the application includes the mobile electronic device and the non-mobile electronic device.

Example five

The embodiment of the application also provides a readable storage medium, on which a program or an instruction is stored, which when executed by a processor, implements each process of the above-mentioned method embodiment for determining a multimedia processing flow, and can achieve the same technical effect, so that repetition is avoided, and no further description is given here.

Wherein the processor is a processor in the electronic device described in the above embodiment. The readable storage medium includes a computer readable storage medium such as a Read-Only Memory (ROM), a random access Memory (Random Access Memory, RAM), a magnetic disk or an optical disk, and the like.

Example six

The embodiment of the application further provides a chip, the chip comprises a processor and a communication interface, the communication interface is coupled with the processor, the processor is used for running programs or instructions, the various processes of the embodiment of the method for determining the multimedia processing flow can be realized, the same technical effect can be achieved, and the repetition is avoided, and the description is omitted here.

It should be understood that the chips referred to in the embodiments of the present application may also be referred to as system-on-chip chips, chip systems, or system-on-chip chips, etc.

It should be noted that, in this document, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one … …" does not exclude the presence of other like elements in a process, method, article, or apparatus that comprises the element. Furthermore, it should be noted that the scope of the methods and apparatus in the embodiments of the present application is not limited to performing the functions in the order shown or discussed, but may also include performing the functions in a substantially simultaneous manner or in an opposite order depending on the functions involved, e.g., the described methods may be performed in an order different from that described, and various steps may be added, omitted, or combined. Additionally, features described with reference to certain examples may be combined in other examples.

From the above description of the embodiments, it will be clear to those skilled in the art that the above-described embodiment method may be implemented by means of software plus a necessary general hardware platform, but of course may also be implemented by means of hardware, but in many cases the former is a preferred embodiment. Based on such understanding, the technical solution of the present application may be embodied essentially or in a part contributing to the prior art in the form of a computer software product stored in a storage medium (e.g. ROM/RAM, magnetic disk, optical disk) comprising instructions for causing a terminal (which may be a mobile phone, a computer, a server, or a network device, etc.) to perform the method according to the embodiments of the present application.

The embodiments of the present application have been described above with reference to the accompanying drawings, but the present application is not limited to the above-described embodiments, which are merely illustrative and not restrictive, and many forms may be made by those having ordinary skill in the art without departing from the spirit of the present application and the scope of the claims, which are to be protected by the present application.

The foregoing description is only of the preferred embodiments of the application and the technical principles employed. The present application is not limited to the specific embodiments described herein, but is capable of numerous modifications, rearrangements and substitutions as will now become apparent to those skilled in the art without departing from the scope of the application. Therefore, while the application has been described in connection with the above embodiments, the application is not limited to the embodiments, but may be embodied in many other equivalent forms without departing from the spirit of the application, the scope of which is set forth in the following claims.

Claims

1. A method for determining a multimedia processing flow, the method being performed by a recording and playing host, the method comprising:

2. The method of claim 1, wherein determining a processing link based on the flow node to process the multimedia data comprises:

3. The method of claim 1, wherein determining a flow node for processing the multimedia data based on the source and target parameters comprises:

4. A method according to claim 3, wherein determining a flow node of the multimedia data based on the same part and different parts of the target parameter and the source parameter comprises:

5. A method according to claim 3, wherein after determining the flow node of the multimedia data from the same part and different parts of the target parameter and the source parameter, the method further comprises:

if so, multiplexing the created nodes in the node list;

6. The method of claim 1, wherein the step of determining the position of the substrate comprises,

the source parameters are read by a source device based on the multimedia data;

7. A device for determining a multimedia processing flow, wherein the device is configured in a recording and playing host, the device comprising:

8. The apparatus according to claim 7, wherein the processing module is specifically configured to:

9. An electronic device comprising a processor, a memory and a program or instruction stored on the memory and executable on the processor, which when executed by the processor, implements the steps of the method of determining a multimedia process flow according to any one of claims 1-6.

10. A readable storage medium, wherein a program or instructions is stored on the readable storage medium, which when executed by a processor, implements the steps of the method for determining a multimedia process flow according to any one of claims 1-6.