CN113141535A - Streaming media data processing method and device and electronic equipment - Google Patents
Streaming media data processing method and device and electronic equipment Download PDFInfo
- Publication number
- CN113141535A CN113141535A CN202110461684.5A CN202110461684A CN113141535A CN 113141535 A CN113141535 A CN 113141535A CN 202110461684 A CN202110461684 A CN 202110461684A CN 113141535 A CN113141535 A CN 113141535A
- Authority
- CN
- China
- Prior art keywords
- data
- streaming media
- packet loss
- media data
- transmission link
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000003672 processing method Methods 0.000 title claims abstract description 20
- 230000005540 biological transmission Effects 0.000 claims abstract description 62
- 238000001514 detection method Methods 0.000 claims abstract description 36
- 238000000034 method Methods 0.000 claims abstract description 36
- 238000011084 recovery Methods 0.000 claims abstract description 17
- 238000009877 rendering Methods 0.000 claims abstract description 15
- 230000004927 fusion Effects 0.000 claims abstract description 14
- 238000012545 processing Methods 0.000 claims abstract description 14
- 230000004044 response Effects 0.000 claims description 4
- 230000007246 mechanism Effects 0.000 claims description 3
- 239000000126 substance Substances 0.000 claims description 3
- 238000004590 computer program Methods 0.000 description 14
- 238000010586 diagram Methods 0.000 description 8
- 238000004891 communication Methods 0.000 description 6
- 230000006870 function Effects 0.000 description 6
- 238000011156 evaluation Methods 0.000 description 5
- 230000003287 optical effect Effects 0.000 description 4
- 230000008569 process Effects 0.000 description 4
- 238000005516 engineering process Methods 0.000 description 3
- 230000000644 propagated effect Effects 0.000 description 2
- 238000011161 development Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 239000004973 liquid crystal related substance Substances 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 239000013307 optical fiber Substances 0.000 description 1
- 230000008439 repair process Effects 0.000 description 1
- 239000004065 semiconductor Substances 0.000 description 1
- 230000008054 signal transmission Effects 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
- H04N21/433—Content storage operation, e.g. storage operation in response to a pause request, caching operations
- H04N21/4331—Caching operations, e.g. of an advertisement for later insertion during playback
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
- H04N21/44—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream, rendering scenes according to MPEG-4 scene graphs
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/60—Network structure or processes for video distribution between server and client or between remote clients; Control signalling between clients, server and network components; Transmission of management data between server and client, e.g. sending from server to client commands for recording incoming content stream; Communication details between server and client
- H04N21/63—Control signaling related to video distribution between client, server and network components; Network processes for video distribution between server and clients or between remote clients, e.g. transmitting basic layer and enhancement layers over different transmission paths, setting up a peer-to-peer communication via Internet between remote STB's; Communication protocols; Addressing
- H04N21/647—Control signaling between network components and server or clients; Network processes for video distribution between server and clients, e.g. controlling the quality of the video stream, by dropping packets, protecting content from unauthorised alteration within the network, monitoring of network load, bridging between two different networks, e.g. between IP and wireless
- H04N21/64723—Monitoring of network processes or resources, e.g. monitoring of network load
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/60—Network structure or processes for video distribution between server and client or between remote clients; Control signalling between clients, server and network components; Transmission of management data between server and client, e.g. sending from server to client commands for recording incoming content stream; Communication details between server and client
- H04N21/63—Control signaling related to video distribution between client, server and network components; Network processes for video distribution between server and clients or between remote clients, e.g. transmitting basic layer and enhancement layers over different transmission paths, setting up a peer-to-peer communication via Internet between remote STB's; Communication protocols; Addressing
- H04N21/647—Control signaling between network components and server or clients; Network processes for video distribution between server and clients, e.g. controlling the quality of the video stream, by dropping packets, protecting content from unauthorised alteration within the network, monitoring of network load, bridging between two different networks, e.g. between IP and wireless
- H04N21/64746—Control signals issued by the network directed to the server or the client
- H04N21/64761—Control signals issued by the network directed to the server or the client directed to the server
- H04N21/64776—Control signals issued by the network directed to the server or the client directed to the server for requesting retransmission, e.g. of data packets lost or corrupted during transmission from server
Landscapes
- Engineering & Computer Science (AREA)
- Multimedia (AREA)
- Signal Processing (AREA)
- Computer Networks & Wireless Communication (AREA)
- Computer Security & Cryptography (AREA)
- Information Transfer Between Computers (AREA)
Abstract
The embodiment of the disclosure provides a streaming media data processing method, a device and an electronic device, belonging to the technical field of data processing, wherein the method comprises the following steps: performing packet loss detection on streaming media data received through a first transmission link; responding to the packet loss detection result, starting a standby cache at the server, wherein the standby cache is used for receiving packet loss data transmitted based on a second transmission link; performing data fusion on the streaming media data sent by the first transmission link and the packet loss data stored in the standby cache in the main cache to form recovery data; and taking the recovered data as decoding rendering data of the streaming media data for data display. By the processing scheme, the integrity of streaming media transmission data can be ensured.
Description
Technical Field
The present disclosure relates to the field of data processing technologies, and in particular, to a streaming media data processing method and apparatus, and an electronic device.
Background
With the rapid development of intelligent hardware and internet streaming media technologies, better updating experience can be brought to users by better combining the intelligent hardware and the internet streaming media technologies, and the method is great tendency and expected by users.
In a weak network environment, due to the loss of the video streaming media transmission packet, the problems of screen splash previewing and unclear video can occur at a client, and the user experience is influenced. Aiming at the current unmanned aerial vehicle product, under a weak network, the transmission quality of the streaming media data transmitted by the images is greatly influenced by the network, especially under the condition that the network is not good, the integrity and the fluency of the data transmitted by the images are difficult to guarantee, and the fluency and the stability of the audio and video transmission shot by the unmanned aerial vehicle in real time are very important for the experience degree of customers.
How to ensure the quality of real-time transmission of audio and video data, reduce the number of transmission lost packets, how to retransmit lost data, and ensure the integrity, timeliness and fluency of preview pictures of streaming media transmission data in a weak network environment is a problem to be solved.
Disclosure of Invention
In view of this, embodiments of the present disclosure provide a streaming media data processing method, apparatus and electronic device to at least partially solve the problems in the prior art.
In a first aspect, an embodiment of the present disclosure provides a streaming media data processing method, including:
performing packet loss detection on streaming media data received through a first transmission link;
responding to the packet loss detection result, starting a standby cache at the server, wherein the standby cache is used for receiving packet loss data transmitted based on a second transmission link;
performing data fusion on the streaming media data sent by the first transmission link and the packet loss data stored in the standby cache in the main cache to form recovery data;
and taking the recovered data as decoding rendering data of the streaming media data for data display.
According to a specific implementation manner of the embodiment of the present disclosure, before performing packet loss detection on streaming media data received through a first transmission link, the method further includes:
and after the streaming media playing application program is started, starting an SDK module for packet loss detection.
According to a specific implementation manner of the embodiment of the present disclosure, the detecting packet loss of streaming media data received through a first transmission link includes:
in the application program, numbering protocol packets contained in streaming media data;
when the application program executes preview operation, whether the serial numbers of the protocol packets of the streaming media data transmitted in a preset time period are continuously detected through the SDK module;
and determining whether the streaming media data has a packet loss problem or not based on the detection result.
According to a specific implementation manner of the embodiment of the present disclosure, before the server starts the standby cache, the method further includes:
and informing the server by adopting an automatic SDK module triggering mechanism, and simultaneously feeding back the index number of the lost data packet to the server.
According to a specific implementation manner of the embodiment of the present disclosure, the starting, at the server, a standby cache in response to a result of packet loss detection includes:
and the server side finds the data packet to be retransmitted from the standby cache according to the missing data packet index sent by the SDK module to retransmit the data, and stores the retransmitted data in the standby cache.
According to a specific implementation manner of the embodiment of the present disclosure, the performing data fusion on the streaming media data sent by the first transmission link and the packet loss data stored in the standby buffer in the primary buffer includes:
acquiring a packet index uploaded to a server;
searching for missing data in a data packet retransmitted in the second transmission link based on the packet index;
and inserting the searched data packets into the primary cache linked list in sequence to form recovery data.
According to a specific implementation manner of the embodiment of the present disclosure, the displaying the recovery data as the decoding rendering data of the streaming media data includes:
transmitting the recovered data to a graphics decoder;
displaying the recovered data in the graphics decoder.
According to a specific implementation manner of the embodiment of the present disclosure, after the data display is performed on the recovered data as decoded rendering data of streaming media data, the method further includes:
turning off the image transmission decoder.
In a second aspect, an embodiment of the present disclosure provides a streaming media data processing apparatus, including:
the detection module is used for carrying out packet loss detection on the streaming media data received through the first transmission link;
the buffer module is used for responding to the packet loss detection result and starting a standby buffer at the server, wherein the standby buffer is used for receiving packet loss data transmitted based on a second transmission link;
the fusion module is used for carrying out data fusion on the streaming media data sent by the first transmission link and the packet loss data stored in the standby cache in the main cache to form recovery data;
and the display module is used for displaying the data by taking the recovered data as the decoding rendering data of the streaming media data.
In a third aspect, an embodiment of the present disclosure further provides an electronic device, where the electronic device includes:
at least one processor; and the number of the first and second groups,
a memory communicatively coupled to the at least one processor; wherein the content of the first and second substances,
the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to execute the streaming media data processing method in the first aspect or any implementation manner of the first aspect.
In a fourth aspect, the disclosed embodiments also provide a non-transitory computer-readable storage medium storing computer instructions for causing a computer to execute the streaming media data processing method in the first aspect or any implementation manner of the first aspect.
In a fifth aspect, the disclosed embodiments also provide a computer program product, the computer program product comprising a computer program stored on a non-transitory computer-readable storage medium, the computer program comprising program instructions that, when executed by a computer, cause the computer to perform the streaming media data processing method in the foregoing first aspect or any implementation manner of the first aspect.
The streaming media data processing scheme in the embodiment of the disclosure includes performing packet loss detection on streaming media data received through a first transmission link; responding to the packet loss detection result, starting a standby cache at the server, wherein the standby cache is used for receiving packet loss data transmitted based on a second transmission link; performing data fusion on the streaming media data sent by the first transmission link and the packet loss data stored in the standby cache in the main cache to form recovery data; and taking the recovered data as decoding rendering data of the streaming media data for data display. By the processing scheme, the stability of streaming media data transmission is improved.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present disclosure, the drawings needed to be used in the embodiments will be briefly described below, and it is apparent that the drawings in the following description are only some embodiments of the present disclosure, and it is obvious for those skilled in the art that other drawings can be obtained according to the drawings without creative efforts.
Fig. 1 is a flowchart of a streaming media data processing method according to an embodiment of the present disclosure;
fig. 2 is a schematic configuration diagram of a streaming media data processing method according to an embodiment of the present disclosure;
fig. 3 is a flowchart of another streaming media data processing method provided by the embodiment of the disclosure;
fig. 4 is a flowchart of another streaming media data processing method provided by the embodiment of the disclosure;
fig. 5 is a schematic structural diagram of a streaming media data processing apparatus according to an embodiment of the disclosure;
fig. 6 is a schematic diagram of an electronic device provided in an embodiment of the present disclosure.
Detailed Description
The embodiments of the present disclosure are described in detail below with reference to the accompanying drawings.
The embodiments of the present disclosure are described below with specific examples, and other advantages and effects of the present disclosure will be readily apparent to those skilled in the art from the disclosure in the specification. It is to be understood that the described embodiments are merely illustrative of some, and not restrictive, of the embodiments of the disclosure. The disclosure may be embodied or carried out in various other specific embodiments, and various modifications and changes may be made in the details within the description without departing from the spirit of the disclosure. It is to be noted that the features in the following embodiments and examples may be combined with each other without conflict. All other embodiments, which can be derived by a person skilled in the art from the embodiments disclosed herein without making any creative effort, shall fall within the protection scope of the present disclosure.
It is noted that various aspects of the embodiments are described below within the scope of the appended claims. It should be apparent that the aspects described herein may be embodied in a wide variety of forms and that any specific structure and/or function described herein is merely illustrative. Based on the disclosure, one skilled in the art should appreciate that one aspect described herein may be implemented independently of any other aspects and that two or more of these aspects may be combined in various ways. For example, an apparatus may be implemented and/or a method practiced using any number of the aspects set forth herein. Additionally, such an apparatus may be implemented and/or such a method may be practiced using other structure and/or functionality in addition to one or more of the aspects set forth herein.
It should be noted that the drawings provided in the following embodiments are only for illustrating the basic idea of the present disclosure, and the drawings only show the components related to the present disclosure rather than the number, shape and size of the components in actual implementation, and the type, amount and ratio of the components in actual implementation may be changed arbitrarily, and the layout of the components may be more complicated.
In addition, in the following description, specific details are provided to facilitate a thorough understanding of the examples. However, it will be understood by those skilled in the art that the aspects may be practiced without these specific details.
The embodiment of the disclosure provides a streaming media data processing method. The streaming media data processing method provided by the embodiment can be executed by a computing device, the computing device can be implemented as software, or implemented as a combination of software and hardware, and the computing device can be integrally arranged in a server, a client and the like.
Referring to fig. 1, a streaming media data processing method in the embodiment of the present disclosure may include the following steps:
s101, packet loss detection is carried out on streaming media data received through a first transmission link.
Unmanned aerial vehicle is at the in-process of flight, need with the real-time control end (e.g. cell-phone) that transmits the user of the image that unmanned aerial vehicle shot to make the user can watch the picture that unmanned aerial vehicle shot at the control end is real-time, however, because unmanned aerial vehicle flight environment's difference, can have the weak condition of unmanned aerial vehicle transmission signal, at this moment, the video picture of the real-time transmission of unmanned aerial vehicle can appear the condition of lacking the frame, lead to the picture unstable, influenced user's use.
Therefore, the streaming media data transmitted by the user picture can be monitored in real time, and the picture transmission strategy of the unmanned aerial vehicle is adjusted by inquiring whether the streaming media data have the phenomenon of packet loss.
The communication link used by the drone for transmitting streaming media may be a specially configured communication link (first transmission link), and the first transmission link may be an RTSP transmission link, and of course, may also be switched to another suitable signal transmission link according to actual needs.
The packet loss detection for the first transmission link may be performed by using a variety of methods, for example, the streaming media data packet may be continuously encoded, and the number of the received streaming media data packet is checked, for example, when the number of the streaming media data packet is found to be discontinuous, the case of packet loss of the streaming media data packet may be determined.
And S102, responding to the packet loss detection result, starting a standby cache at the server, wherein the standby cache is used for receiving packet loss data transmitted based on the second transmission link.
When the condition that packet loss exists in the streaming media data is found, a standby cache can be started at the server, and the standby cache is used for storing the packet loss data.
Therefore, after the condition that the streaming media data are lost is found, a data packet index is established through coding of the streaming media data, the data packet index is sent to the unmanned aerial vehicle end, and meanwhile, the unmanned aerial vehicle transmits the lost packet data based on the second transmission link again, so that the lost packet data can be transmitted to the server side quickly and stored in a standby cache of the server side.
The second transmission link may be a communication link (e.g., UDP transport protocol) with a large amount of transmission bandwidth, so that the packet loss data to be transmitted can be transmitted to the backup buffer in a short time.
And S103, performing data fusion on the streaming media data sent by the first transmission link and the packet loss data stored in the standby buffer in the main buffer to form recovery data.
The server is provided with a main cache, the main cache is used for receiving the streaming media data sent by the first transmission link, and the data in the main cache and the data in the standby cache are fused, so that complete streaming media data without packet loss data can be formed, namely the data is recovered.
As one case, a packet loss compensation algorithm may be set to generate recovery data, where the packet loss compensation algorithm searches missing data in a retransmission data packet of the second transmission link through a packet index uploaded to the server, and sequentially inserts the found data packets into the main cache linked list, thereby generating complete streaming media data.
And S104, displaying the recovery data as the decoding rendering data of the streaming media data.
The restored data may be input to a decoder, through which the streaming media data is subjected to a decryption rendering operation, thereby generating a final video image.
By means of the above embodiment, whether the packet loss condition exists in the streaming media data can be dynamically monitored, and after the packet loss condition is found, the packet loss data is timely compensated, so that the fluency of the video shot by the unmanned aerial vehicle is ensured.
According to a specific implementation manner of the embodiment of the present disclosure, before performing packet loss detection on streaming media data received through a first transmission link, the method further includes:
and after the streaming media playing application program is started, starting an SDK module for packet loss detection. The streaming media playing application program can be software which is arranged in the unmanned aerial vehicle and used for shooting videos or previewing videos, and the SDK module is a software module which is integrated in the unmanned aerial vehicle software and used for detecting whether packet loss exists in streaming media formed by shooting videos in the process of shooting the videos by the unmanned aerial vehicle. By the above mode, the packet loss detection can be automatically started based on the starting of the streaming media playing application program, and the mobility of the packet loss detection is improved.
Referring to fig. 2, according to a specific implementation manner of the embodiment of the present disclosure, the detecting packet loss of streaming media data received through a first transmission link includes:
s201, in the application program, numbering the protocol packets included in the streaming media data.
The mode that can number according to the order, in the in-process that unmanned aerial vehicle shoots the completion video and forms streaming media data, can number according to the order that the streaming media data package generated in succession, through numbering in succession, can carry out effective detection to follow-up streaming media data whether the packet loss.
S202, when the application program executes the preview operation, whether the protocol packet numbers of the streaming media data transmitted in the preset time period are continuous or not is detected through the SDK module.
When the video shot by the unmanned aerial vehicle is transmitted to the control end of the user, the shot video usually needs to be previewed, and in the previewing process, whether the serial number on the streaming media protocol packet transmitted within a preset time period (for example, 3 seconds) is continuously detected through a preset SDK module, so that whether the situation of data packet loss exists is judged.
S203, determining whether the streaming media data has a packet loss problem or not based on the detection result.
Through the content of the embodiment, the packet loss detection can be started at the preview stage of the video shot by the unmanned aerial vehicle, so that the timeliness of the packet loss detection is improved.
According to a specific implementation manner of the embodiment of the present disclosure, before the server starts the standby cache, the method further includes: and informing the server by adopting an automatic SDK module triggering mechanism, and simultaneously feeding back the index number of the lost data packet to the server.
According to a specific implementation manner of the embodiment of the present disclosure, the starting, at the server, a standby cache in response to a result of packet loss detection includes: and the server side finds the data packet to be retransmitted from the standby cache according to the missing data packet index sent by the SDK module to retransmit the data, and stores the retransmitted data in the standby cache.
Referring to fig. 3, according to a specific implementation manner of the embodiment of the present disclosure, the performing data fusion on the streaming media data sent by the first transmission link and the packet-lost data stored in the standby buffer in the primary buffer includes:
s301, acquiring a packet index uploaded to a server;
s302, based on the packet index, missing data is searched in the data packet retransmitted in the second transmission link;
and S303, inserting the searched data packets into the primary cache linked list in sequence to form recovery data.
By the content of the embodiment, the lost flow data packet can be detected based on the preset data packet number, so that packet loss compensation operation is performed, and the efficiency of packet loss data repair is improved.
Referring to fig. 4, according to a specific implementation manner of the embodiment of the present disclosure, the displaying the recovered data as the decoded rendering data of the streaming media data includes:
s401, transmitting the recovery data to a picture transmission decoder;
s402, displaying the recovery data in the image transmission decoder.
According to a specific implementation manner of the embodiment of the present disclosure, after the data display is performed on the recovered data as decoded rendering data of streaming media data, the method further includes:
turning off the image transmission decoder. By turning off the picture transmission decoder, the resources of the system can be effectively saved.
Corresponding to the above method embodiment, referring to fig. 5, an embodiment of the present disclosure further provides a streaming media data processing apparatus 50, including:
a detecting module 501, configured to perform packet loss detection on streaming media data received through a first transmission link;
a buffer module 502, configured to start a standby buffer at a server in response to a packet loss detection result, where the standby buffer is configured to receive packet loss data transmitted based on a second transmission link;
a fusion module 503, configured to perform data fusion on the streaming media data sent by the first transmission link and the packet-lost data stored in the standby buffer in the primary buffer to form recovered data;
a display module 504, configured to display the recovered data as decoded rendering data of the streaming media data.
For parts not described in detail in this embodiment, reference is made to the contents described in the above method embodiments, which are not described again here.
Referring to fig. 6, an embodiment of the present disclosure also provides an electronic device 60, including:
at least one processor; and the number of the first and second groups,
a memory communicatively coupled to the at least one processor; wherein the content of the first and second substances,
the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to execute the streaming media data processing method in the foregoing method embodiment.
The disclosed embodiments also provide a non-transitory computer-readable storage medium storing computer instructions for causing the computer to execute the streaming media data processing method in the foregoing method embodiments.
The disclosed embodiments also provide a computer program product comprising a computer program stored on a non-transitory computer readable storage medium, the computer program comprising program instructions which, when executed by a computer, cause the computer to perform the streaming media data processing method in the aforementioned method embodiments.
Referring now to FIG. 6, a schematic diagram of an electronic device 60 suitable for use in implementing embodiments of the present disclosure is shown. The electronic devices in the embodiments of the present disclosure may include, but are not limited to, mobile terminals such as mobile phones, notebook computers, digital broadcast receivers, PDAs (personal digital assistants), PADs (tablet computers), PMPs (portable multimedia players), in-vehicle terminals (e.g., car navigation terminals), and the like, and fixed terminals such as digital TVs, desktop computers, and the like. The electronic device shown in fig. 6 is only an example, and should not bring any limitation to the functions and the scope of use of the embodiments of the present disclosure.
As shown in fig. 6, the electronic device 60 may include a processing means (e.g., a central processing unit, a graphics processor, etc.) 601 that may perform various appropriate actions and processes in accordance with a program stored in a Read Only Memory (ROM)602 or a program loaded from a storage means 608 into a Random Access Memory (RAM) 603. In the RAM 603, various programs and data necessary for the operation of the electronic apparatus 60 are also stored. The processing device 601, the ROM602, and the RAM 603 are connected to each other via a bus 604. An input/output (I/O) interface 605 is also connected to bus 604.
Generally, the following devices may be connected to the I/O interface 605: input devices 606 including, for example, a touch screen, touch pad, keyboard, mouse, image sensor, microphone, accelerometer, gyroscope, etc.; output devices 607 including, for example, a Liquid Crystal Display (LCD), a speaker, a vibrator, and the like; storage 608 including, for example, tape, hard disk, etc.; and a communication device 609. The communication means 609 may allow the electronic device 60 to communicate with other devices wirelessly or by wire to exchange data. While the figures illustrate an electronic device 60 having various means, it is to be understood that not all illustrated means are required to be implemented or provided. More or fewer devices may alternatively be implemented or provided.
In particular, according to an embodiment of the present disclosure, the processes described above with reference to the flowcharts may be implemented as computer software programs. For example, embodiments of the present disclosure include a computer program product comprising a computer program embodied on a computer readable medium, the computer program comprising program code for performing the method illustrated in the flow chart. In such an embodiment, the computer program may be downloaded and installed from a network via the communication means 609, or may be installed from the storage means 608, or may be installed from the ROM 602. The computer program, when executed by the processing device 601, performs the above-described functions defined in the methods of the embodiments of the present disclosure.
It should be noted that the computer readable medium in the present disclosure can be a computer readable signal medium or a computer readable storage medium or any combination of the two. A computer readable storage medium may be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the foregoing. More specific examples of the computer readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In the present disclosure, a computer readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device. In contrast, in the present disclosure, a computer readable signal medium may comprise a propagated data signal with computer readable program code embodied therein, either in baseband or as part of a carrier wave. Such a propagated data signal may take many forms, including, but not limited to, electro-magnetic, optical, or any suitable combination thereof. A computer readable signal medium may also be any computer readable medium that is not a computer readable storage medium and that can communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device. Program code embodied on a computer readable medium may be transmitted using any appropriate medium, including but not limited to: electrical wires, optical cables, RF (radio frequency), etc., or any suitable combination of the foregoing.
The computer readable medium may be embodied in the electronic device; or may exist separately without being assembled into the electronic device.
The computer readable medium carries one or more programs which, when executed by the electronic device, cause the electronic device to: acquiring at least two internet protocol addresses; sending a node evaluation request comprising the at least two internet protocol addresses to node evaluation equipment, wherein the node evaluation equipment selects the internet protocol addresses from the at least two internet protocol addresses and returns the internet protocol addresses; receiving an internet protocol address returned by the node evaluation equipment; wherein the obtained internet protocol address indicates an edge node in the content distribution network.
Alternatively, the computer readable medium carries one or more programs which, when executed by the electronic device, cause the electronic device to: receiving a node evaluation request comprising at least two internet protocol addresses; selecting an internet protocol address from the at least two internet protocol addresses; returning the selected internet protocol address; wherein the received internet protocol address indicates an edge node in the content distribution network.
Computer program code for carrying out operations for aspects of the present disclosure may be written in any combination of one or more programming languages, including an object oriented programming language such as Java, Smalltalk, C + +, and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code may execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a Local Area Network (LAN) or a Wide Area Network (WAN), or the connection may be made to an external computer (for example, through the Internet using an Internet service provider).
The flowchart and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments of the present disclosure. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that, in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams and/or flowchart illustration, and combinations of blocks in the block diagrams and/or flowchart illustration, can be implemented by special purpose hardware-based systems which perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.
The units described in the embodiments of the present disclosure may be implemented by software or hardware. Where the name of a unit does not in some cases constitute a limitation of the unit itself, for example, the first retrieving unit may also be described as a "unit for retrieving at least two internet protocol addresses".
It should be understood that portions of the present disclosure may be implemented in hardware, software, firmware, or a combination thereof.
The above description is only for the specific embodiments of the present disclosure, but the scope of the present disclosure is not limited thereto, and any changes or substitutions that can be easily conceived by those skilled in the art within the technical scope of the present disclosure should be covered within the scope of the present disclosure. Therefore, the protection scope of the present disclosure shall be subject to the protection scope of the claims.
Claims (10)
1. A method for processing streaming media data, comprising:
performing packet loss detection on streaming media data received through a first transmission link;
responding to the packet loss detection result, starting a standby cache at the server, wherein the standby cache is used for receiving packet loss data transmitted based on a second transmission link;
performing data fusion on the streaming media data sent by the first transmission link and the packet loss data stored in the standby cache in the main cache to form recovery data;
and taking the recovered data as decoding rendering data of the streaming media data for data display.
2. The method according to claim 1, wherein before performing packet loss detection on the streaming media data received via the first transmission link, the method further comprises:
and after the streaming media playing application program is started, starting an SDK module for packet loss detection.
3. The method according to claim 2, wherein the detecting packet loss of the streaming media data received via the first transmission link comprises:
in the application program, numbering protocol packets contained in streaming media data;
when the application program executes preview operation, whether the serial numbers of the protocol packets of the streaming media data transmitted in a preset time period are continuously detected through the SDK module;
and determining whether the streaming media data has a packet loss problem or not based on the detection result.
4. The method of claim 3, wherein before the server opens the backup buffer, the method further comprises:
and informing the server by adopting an automatic SDK module triggering mechanism, and simultaneously feeding back the index number of the lost data packet to the server.
5. The method of claim 4, wherein the opening a backup buffer at the server in response to the packet loss detection result comprises:
and the server side finds the data packet to be retransmitted from the standby cache according to the missing data packet index sent by the SDK module to retransmit the data, and stores the retransmitted data in the standby cache.
6. The method according to claim 5, wherein the performing data fusion on the streaming media data sent by the first transmission link and the packet loss data stored in the backup buffer in the primary buffer comprises:
acquiring a packet index uploaded to a server;
searching for missing data in a data packet retransmitted in the second transmission link based on the packet index;
and inserting the searched data packets into the primary cache linked list in sequence to form recovery data.
7. The method of claim 6, wherein displaying the recovery data as decoded rendering data of streaming media data comprises:
transmitting the recovered data to a graphics decoder;
displaying the recovered data in the graphics decoder.
8. The method of claim 7, wherein after the data displaying the recovery data as decoded rendering data of streaming media data, the method further comprises:
turning off the image transmission decoder.
9. A streaming media data processing apparatus, comprising:
the detection module is used for carrying out packet loss detection on the streaming media data received through the first transmission link;
the buffer module is used for responding to the packet loss detection result and starting a standby buffer at the server, wherein the standby buffer is used for receiving packet loss data transmitted based on a second transmission link;
the fusion module is used for carrying out data fusion on the streaming media data sent by the first transmission link and the packet loss data stored in the standby cache in the main cache to form recovery data;
and the display module is used for displaying the data by taking the recovered data as the decoding rendering data of the streaming media data.
10. An electronic device, characterized in that the electronic device comprises:
at least one processor; and the number of the first and second groups,
a memory communicatively coupled to the at least one processor; wherein the content of the first and second substances,
the memory stores instructions executable by the at least one processor to enable the at least one processor to perform the streaming media data processing method of any of the preceding claims 1-8.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110461684.5A CN113141535A (en) | 2021-04-27 | 2021-04-27 | Streaming media data processing method and device and electronic equipment |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110461684.5A CN113141535A (en) | 2021-04-27 | 2021-04-27 | Streaming media data processing method and device and electronic equipment |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN113141535A true CN113141535A (en) | 2021-07-20 |
Family
ID=76816224
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110461684.5A Pending CN113141535A (en) | 2021-04-27 | 2021-04-27 | Streaming media data processing method and device and electronic equipment |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113141535A (en) |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101719918A (en) * | 2009-11-27 | 2010-06-02 | 北京交通大学 | Improved multi-joint and multi-path transmission method |
| CN103686446A (en) * | 2013-12-06 | 2014-03-26 | 广州华多网络科技有限公司 | Method and system for retransmitting lost packet in video data transmission |
| CN107135216A (en) * | 2017-05-03 | 2017-09-05 | 深圳市视维科技股份有限公司 | One kind strengthens weak net environment flow-medium transmission method |
| CN108174234A (en) * | 2018-01-12 | 2018-06-15 | 珠海全志科技股份有限公司 | A kind of flow-medium transmission method and system |
| WO2018205688A1 (en) * | 2017-05-12 | 2018-11-15 | 华为技术有限公司 | Method, apparatus and system for data transmission |
| CN109120383A (en) * | 2017-06-26 | 2019-01-01 | 深圳市道通智能航空技术有限公司 | Unmanned plane and its earth station, data transmission method |
-
2021
- 2021-04-27 CN CN202110461684.5A patent/CN113141535A/en active Pending
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101719918A (en) * | 2009-11-27 | 2010-06-02 | 北京交通大学 | Improved multi-joint and multi-path transmission method |
| CN103686446A (en) * | 2013-12-06 | 2014-03-26 | 广州华多网络科技有限公司 | Method and system for retransmitting lost packet in video data transmission |
| CN107135216A (en) * | 2017-05-03 | 2017-09-05 | 深圳市视维科技股份有限公司 | One kind strengthens weak net environment flow-medium transmission method |
| WO2018205688A1 (en) * | 2017-05-12 | 2018-11-15 | 华为技术有限公司 | Method, apparatus and system for data transmission |
| CN109120383A (en) * | 2017-06-26 | 2019-01-01 | 深圳市道通智能航空技术有限公司 | Unmanned plane and its earth station, data transmission method |
| CN108174234A (en) * | 2018-01-12 | 2018-06-15 | 珠海全志科技股份有限公司 | A kind of flow-medium transmission method and system |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111064987B (en) | Information display method and device and electronic equipment | |
| CN110290398B (en) | Video issuing method and device, storage medium and electronic equipment | |
| US11785195B2 (en) | Method and apparatus for processing three-dimensional video, readable storage medium and electronic device | |
| US20220394333A1 (en) | Video processing method and apparatus, storage medium, and electronic device | |
| US20170195384A1 (en) | Video Playing Method and Electronic Device | |
| KR20220144857A (en) | Multimedia data publishing method and apparatus, device and recording medium | |
| CN112770159A (en) | Multi-screen interaction system, method, device, equipment and storage medium | |
| CN114531614B (en) | Video playing method, device, electronic equipment and storage medium | |
| CN109714626B (en) | Information interaction method and device, electronic equipment and computer readable storage medium | |
| CN111246245B (en) | Method and device for pushing video aggregation page, server and terminal equipment | |
| CN110908752A (en) | Control setting method and device, electronic equipment and interaction system | |
| CN114938427A (en) | Method, apparatus, device, storage medium and program product for photographing media content | |
| CN114445600A (en) | Method, device and equipment for displaying special effect prop and storage medium | |
| CN114025225A (en) | Bullet screen control method and device, electronic equipment and storage medium | |
| CN114125551B (en) | Video generation method, device, electronic equipment and computer readable medium | |
| CN113259729B (en) | Data switching method, server, system and storage medium | |
| WO2023197811A1 (en) | Video downloading method and apparatus, video transmission method and apparatus, terminal device, server and medium | |
| WO2023198033A1 (en) | Prompting method and device | |
| CN110083768B (en) | Information sharing method, device, equipment and medium | |
| CN115834918A (en) | Video live broadcast method and device, electronic equipment and readable storage medium | |
| CN113141535A (en) | Streaming media data processing method and device and electronic equipment | |
| CN113766303B (en) | Multi-screen interaction method, device, equipment and storage medium | |
| CN114071028B (en) | Video generation and playing method and device, electronic equipment and storage medium | |
| CN114630157A (en) | Live broadcast starting method, equipment and program product | |
| CN114554292A (en) | Method and device for switching visual angle, electronic equipment, storage medium and program product |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20210720 |
|
| RJ01 | Rejection of invention patent application after publication |