CN110855645A - Streaming media data playing method and device - Google Patents
Streaming media data playing method and device Download PDFInfo
- Publication number
- CN110855645A CN110855645A CN201911060374.1A CN201911060374A CN110855645A CN 110855645 A CN110855645 A CN 110855645A CN 201911060374 A CN201911060374 A CN 201911060374A CN 110855645 A CN110855645 A CN 110855645A
- Authority
- CN
- China
- Prior art keywords
- frame
- decoding
- media data
- streaming media
- information
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L65/00—Network arrangements, protocols or services for supporting real-time applications in data packet communication
- H04L65/60—Network streaming of media packets
- H04L65/75—Media network packet handling
- H04L65/764—Media network packet handling at the destination
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L65/00—Network arrangements, protocols or services for supporting real-time applications in data packet communication
- H04L65/60—Network streaming of media packets
- H04L65/70—Media network packetisation
Landscapes
- Engineering & Computer Science (AREA)
- Multimedia (AREA)
- Computer Networks & Wireless Communication (AREA)
- Signal Processing (AREA)
- Two-Way Televisions, Distribution Of Moving Picture Or The Like (AREA)
- Compression Or Coding Systems Of Tv Signals (AREA)
Abstract
The application provides a streaming media data playing method and device. The method comprises the following steps: the method comprises the steps of obtaining network frame receiving information and decoding frame sending information of streaming media data, inputting the streaming media data into a hardware decoder for decoding to generate decoding frames and performance cache information, determining frame fetching time and frame fetching frames according to the network frame receiving information, the decoding frame sending information and the performance cache information, and extracting the decoding frames from a cache of the hardware decoder for playing according to the frame fetching time and the frame fetching frames. The frame fetching time and the frame fetching frame number are determined in consideration of the condition of the streaming media data and the condition of the hardware decoder decoding the streaming media data, so that the decoded frame generated by decoding can be timely fetched, and the obtained playing file is smoother when the decoded frame is fetched from the cache of the hardware decoder for playing according to the frame fetching time and the frame fetching frame number.
Description
Technical Field
The present application relates to the field of communications technologies, and in particular, to a method and an apparatus for playing streaming media data.
Background
At present, two schemes of downloading and streaming transmission are mainly adopted for transmitting multimedia information such as audio, video and the like on the network, the streaming transmission can be played in the process of file transmission, the transmission and the playing are realized, the work and the life of a user are greatly facilitated, and the streaming media data refers to data transmitted by using a streaming transmission technology.
In the prior art, when streaming media data is played, a frame is usually played at regular time, and only one frame is played each time, so that the streaming media data is often jammed during playing.
Disclosure of Invention
The present application aims to provide a method and an apparatus for playing streaming media data, which can make the playing of the streaming media data smoother.
According to an aspect of an embodiment of the present application, there is provided a streaming media data playing method, including: acquiring network frame receiving information and decoding frame sending information of streaming media data; inputting the stream media data into a hardware decoder for decoding to generate a decoding frame and performance cache information; determining frame taking time and frame taking number according to the network frame receiving information, the decoding frame sending information and the performance cache information; and extracting the decoding frame from the buffer memory of the hardware decoder according to the frame extracting time and the frame extracting frame number for playing.
According to an aspect of an embodiment of the present application, there is provided a streaming media data playing apparatus, including: the information module is used for acquiring network frame receiving information and decoding frame sending information of the streaming media data; the decoding module is used for inputting the streaming media data into a hardware decoder for decoding to generate a decoding frame and performance cache information; the computing module is used for determining frame taking time and frame taking number according to the network frame receiving information, the decoding frame sending information and the performance cache information; and the frame fetching module is used for extracting the decoding frame from the cache of the hardware decoder according to the frame fetching time and the frame fetching number so as to play and obtain the frame.
In some embodiments of the present application, based on the foregoing solution, the information module is configured to: acquiring a coding time stamp corresponding to each frame of coding in the streaming media data through a far-end coder; acquiring the packet receiving frame number of the streaming media data through the far-end encoder; and generating the network frame receiving information according to the encoding timestamp and the packet receiving frame number.
In some embodiments of the present application, based on the foregoing solution, the information module is further configured to: and generating the decoding frame sending information according to the effective frame number to be decoded of the streaming media data.
In some embodiments of the present application, based on the foregoing scheme, the decoding module is configured to: inputting the streaming media data into a hardware decoder for decoding to generate the decoding frame; and in the decoding process, generating the performance cache information according to the performance and the cache information of the hardware decoder.
In some embodiments of the present application, based on the foregoing scheme, the decoding module is further configured to: generating decoding performance according to the decoding frame number generated by the hardware decoder in preset time; acquiring the number of buffer frames of the hardware decoder in a preset time; and generating the performance caching information according to the caching frame number and the decoding performance.
In some embodiments of the present application, based on the foregoing, the calculation module is configured to: determining a real-time frame rate and initial frame taking time according to the packet receiving frame number, the encoding timestamp and the decoding performance; and correcting the initial frame taking time according to the decoding frame sending information to generate the frame taking time.
In some embodiments of the present application, based on the foregoing, the calculation module is further configured to: when the decoding frame sending information is larger than or equal to a preset threshold value, correcting the initial frame taking time through a first correction coefficient to generate the frame taking time; and when the decoding frame sending information is smaller than a preset threshold value, correcting the initial frame taking time through a second correction coefficient to generate the frame taking time.
In some embodiments of the present application, based on the foregoing, the calculation module is further configured to: when the decoding frame sending information is greater than or equal to a preset threshold value and the caching frame number is greater than the caching accumulation frame number, determining the frame taking frame number to be N; wherein N is an integer greater than 1.
In some embodiments of the present application, based on the foregoing, the calculation module is further configured to: when the decoding frame sending information is larger than 1 and smaller than a preset threshold value and the number of the cache frames is larger than 0, determining the number of the frame taking frames to be N; wherein N is an integer greater than 1.
According to an aspect of embodiments of the present application, there is provided a computer-readable program medium storing computer program instructions which, when executed by a computer, cause the computer to perform the method of any one of the above.
According to an aspect of an embodiment of the present application, there is provided an electronic apparatus including: a processor; a memory having computer readable instructions stored thereon which, when executed by the processor, implement the method of any of the above.
The technical scheme provided by the embodiment of the application can have the following beneficial effects:
in the technical solutions provided in some embodiments of the present application, the condition of streaming media data sent to a hardware decoder is known by obtaining network frame receiving information and decoding frame sending information of the streaming media data. The condition of the hardware decoder decoding the streaming media data is known by inputting the streaming media data into the hardware decoder for decoding to generate a decoding frame and performance cache information. And then, determining the frame fetching time and the frame fetching frame number according to the network frame receiving information, the decoding frame sending information and the performance cache information, wherein the determination of the frame fetching time and the frame fetching frame number takes the condition of the streaming media data and the condition of the hardware decoder decoding the streaming media data into consideration, so that the decoded frame generated by decoding can be timely fetched, and further, when the decoded frame is fetched from the cache of the hardware decoder according to the frame fetching time and the frame fetching frame number for playing, the obtained playing file is smoother.
It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the application.
Drawings
The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and together with the description, serve to explain the principles of the application.
FIG. 1 shows a schematic diagram of an exemplary system architecture to which aspects of embodiments of the present application may be applied;
fig. 2 schematically shows a flow chart of a streaming media data playing method according to an embodiment of the present application;
fig. 3 schematically shows a flowchart of a process of determining a frame fetch time according to network frame receive information, decoded frame send information, performance buffering information according to an embodiment of the present application;
fig. 4 schematically shows a flowchart of a process of determining a frame fetching number according to network frame receiving information, decoding frame sending information, and performance buffering information according to an embodiment of the present application;
fig. 5 schematically shows a block diagram of a streaming media data playback apparatus according to an embodiment of the present application;
FIG. 6 is a hardware diagram illustrating an electronic device according to an example embodiment.
Detailed Description
Example embodiments will now be described more fully with reference to the accompanying drawings. Example embodiments may, however, be embodied in many different forms and should not be construed as limited to the examples set forth herein; rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the concept of example embodiments to those skilled in the art.
Furthermore, the described features, structures, or characteristics may be combined in any suitable manner in one or more embodiments. In the following description, numerous specific details are provided to give a thorough understanding of embodiments of the application. One skilled in the relevant art will recognize, however, that the subject matter of the present application can be practiced without one or more of the specific details, or with other methods, components, devices, steps, and so forth. In other instances, well-known methods, devices, implementations, or operations have not been shown or described in detail to avoid obscuring aspects of the application.
The block diagrams shown in the figures are functional entities only and do not necessarily correspond to physically separate entities. I.e. these functional entities may be implemented in the form of software, or in one or more hardware modules or integrated circuits, or in different networks and/or processor means and/or microcontroller means.
The flow charts shown in the drawings are merely illustrative and do not necessarily include all of the contents and operations/steps, nor do they necessarily have to be performed in the order described. For example, some operations/steps may be decomposed, and some operations/steps may be combined or partially combined, so that the actual execution sequence may be changed according to the actual situation.
Fig. 1 shows a schematic diagram of an exemplary system architecture 100 to which the technical solutions of the embodiments of the present application can be applied.
As shown in fig. 1, the system architecture 100 may include a terminal device 101 (which may be one or more of a smartphone, a tablet, a laptop, a desktop computer), a network 102, and a server 103. Network 102 is the medium used to provide communication links between terminal devices 101 and server 103. Network 102 may include various connection types, such as wired communication links, wireless communication links, and so forth.
It should be understood that the number of terminal devices 101, networks 102, and servers 103 in fig. 1 is merely illustrative. There may be any number of terminal devices 101, networks 102, and servers 103, as desired for implementation. For example, the server 103 may be a server cluster composed of a plurality of servers.
In an embodiment of the present application, the server 103 obtains the network frame receiving information and the network frame sending information of the streaming media data, and knows the condition of the streaming media data sent to the hardware decoder. The condition of the hardware decoder decoding the streaming media data is known by inputting the streaming media data into the hardware decoder for decoding to generate a decoding frame and performance cache information. And then determining the frame fetching time and the frame fetching frame number according to the network frame receiving information, the decoding frame sending information and the performance cache information, wherein the determination of the frame fetching time and the frame fetching frame number takes the condition of the streaming media data and the condition of the hardware decoder decoding the streaming media data into consideration, so that the decoded frame generated by decoding can be timely fetched, and further, when the decoded frame is fetched from the cache of the hardware decoder according to the frame fetching time and the frame fetching frame number for playing, the obtained playing file is smooth and not blocked.
It should be noted that the streaming media data playing method provided in the embodiment of the present application is generally executed by the server 103, and accordingly, the streaming media data playing apparatus is generally disposed in the server 103. However, in other embodiments of the present application, the terminal device 101 may also have a similar function as the server 103, so as to execute the streaming media data playing method provided in the embodiments of the present application.
The implementation details of the technical solution of the embodiment of the present application are set forth in detail below:
fig. 2 schematically shows a flowchart of a streaming media data playing method according to an embodiment of the present application, and an execution subject of the streaming media data playing method may be a server, such as the server 103 shown in fig. 1.
Referring to fig. 2, the method for playing streaming media data at least includes steps S210 to S230, which are described in detail as follows:
in step S210, network frame receiving information and decoding frame sending information of the streaming media data are acquired.
In an embodiment of the present application, the streaming media data may be in a format of a streaming media data packet, and after the streaming media data in the format of the streaming media data packet is obtained, the streaming media data packet is decompressed first to obtain a frame to be decoded of the streaming media data that can be decoded.
In an embodiment of the present application, a far-end encoder may obtain a coding timestamp corresponding to each frame of coding in streaming media data, and obtain a packet receiving frame number of the streaming media data; and generating network frame receiving information according to the encoding time stamp and the packet receiving frame number.
In an embodiment of the present application, the network frame receiving information of the streaming media data may be an encoding frame rate of the streaming media data, and the encoding frame rate may be obtained according to the encoding timestamp and the number of packet receiving frames. The encoding frame rate may indicate the encoding speed of the encoder, and when the encoding speed of a frame of streaming media data is fast, it indicates that the time required for encoding the frame of streaming media data is short, and indicates that the number of bytes occupied by the frame is small, then the time required for decoding the frame of streaming media data is short, so that the frame of streaming media data is decoded fast, and in order to enable the decoded frame after the streaming media data is decoded to be taken out in time, it is necessary to accelerate the taking out of the decoded frame obtained by fast decoding. Therefore, when the encoding frame rate of the streaming media data is high, the frame taking interval time can be shortened, and the number of frames taken per time can be increased.
In an embodiment of the present application, the encoding timestamp and the packet receiving frame number in the set time in the streaming media data may be obtained, a difference between the encoding timestamp corresponding to the first frame encoding in the set time period and the encoding timestamp corresponding to the last frame encoding in the set time period is obtained, and the packet receiving frame number is divided by the obtained difference in the timestamps to obtain the network frame receiving information. According to the network frame receiving information, the coding speed in the set time period can be obtained, and then the frame taking time and the frame taking number are adjusted according to the set time period. For example, if the speed of encoding in the first set period is three frames per second and the speed of encoding in the second set period is four frames per second, then correspondingly, when taking frames, the frames can be taken according to three frames per second when taking the decoded frames in the first set period, and the frames can be taken according to four frames per second when taking the decoded frames in the second set period.
In an embodiment of the present application, the network frame receiving information may be a coding timestamp of the streaming media data, a coding speed may be obtained by calculating a difference between two adjacent coding timestamps, processing may be accelerated for the streaming media data with a fast coding speed, a frame fetching interval time may be shortened, and a frame fetching number at each time may be increased.
In an embodiment of the present application, the network frame receiving information may be a packet receiving frame number of the streaming media data, and the processing of the streaming media data with a large packet receiving frame number may be accelerated, so as to shorten a frame fetching interval time, and increase a frame fetching frame number each time.
In an embodiment of the present application, the decoding frame sending information may be generated according to the number of valid frames to be decoded of the streaming media data. When the frames are taken, the decoding frame sending information generated according to the number of the effective frames to be decoded is considered, when the number of the effective frames to be decoded is large, the number of the frames to be decoded which need to be decoded is large, the frame taking can be accelerated, the frame taking interval time can be shortened, and the frame taking number of each time can be increased.
In one embodiment of the present application, the decoding frame information may be the number of frames of the streaming media data that are valid to be decoded.
In an embodiment of the present application, a network hoarding frame may occur in the streaming media data acquired from the network during transmission, and a frame to be decoded of the streaming media data actually entering the hardware decoder may be used as a valid frame to be decoded of the streaming media data.
In an embodiment of the present application, frame damage may occur in streaming media data acquired from a network during transmission, a frame to be decoded of the acquired streaming media data may be detected, and the detected frame to be decoded is used as an effective frame to be decoded.
With continued reference to fig. 2, in step S220, the streaming media data is input into a hardware decoder for decoding to generate a decoded frame and performance buffer information.
In one embodiment of the application, the streaming media data is input into a hardware decoder for decoding to generate a decoding frame; and during the decoding process, generating performance cache information according to the performance of the hardware decoder and the cache information. The performance and the cache information of the hardware decoder take the decoding process into consideration, and the frame taking number is adjusted according to the decoding process to take frames in time, so that the video playing is smoother.
In an embodiment of the present application, the performance of the hardware decoder may be obtained according to the number of decoding frames generated by the hardware decoder in a predetermined time, the number of buffering frames of the hardware decoder in the predetermined time may be obtained, and the performance buffering information may be generated according to the number of buffering frames and the decoding performance.
In an embodiment of the present application, the average time of the hardware decoder generating a frame of decoded frames may be obtained by dividing the number of decoded frames generated within a predetermined time by a predetermined time, and the average time of the hardware decoder generating a frame of decoded frames is taken as the performance of the hardware decoder. The average time of the hardware decoder for generating a frame decoding frame is short, so that the hardware decoder decodes quickly, the frame fetching can be accelerated, and the memory frame of the hardware decoder is avoided.
In an embodiment of the present application, the performance of the hardware decoder may be obtained according to the operating state of the hardware decoder, the decoding speed of the hardware decoder during initialization is slower than the decoding speed of the hardware decoder during normal operation, and the hardware decoder may slow down the frame fetching speed when the hardware decoder starts to operate.
In one embodiment of the present application, the number of buffered frames of the hardware decoder in the predetermined time may be the number of decoded frames generated by the hardware decoder in the predetermined time. When the number of buffer frames is large, the decoding speed of the hardware decoder is high, the frame fetching can be accelerated, the frame fetching interval time can be shortened, and the frame number of each frame fetching can be increased.
In one embodiment of the present application, the number of buffered frames of the hardware decoder within the predetermined time may include a number of decoded frames generated by the hardware decoder within the predetermined time and a number of stocked frames before the predetermined time within the hardware decoder. When the number of decoded frames generated by the hardware decoder within the predetermined time is small, it indicates that the hardware decoder is slow to decode, but if the number of stored frames generated by the hardware decoder before the predetermined time is large, in order to reduce the stored frames, the frame fetching speed may exceed the speed of the decoded frames generated by the decoder, the frame fetching time interval may be smaller than the time interval of the decoded frames generated by the hardware decoder, and the number of frames fetched each time may be larger than the number of decoded frames generated by the hardware decoder each time.
Continuing to refer to fig. 2, in step S230, determining a frame fetching time and a frame fetching frame number according to the network frame receiving information, the decoding frame sending information and the performance cache information; and extracting the decoded frame from the buffer of the hardware decoder according to the frame extracting time and the frame extracting number for playing.
In the technical solution provided by the embodiment of fig. 2, the condition of the streaming media data sent to the hardware decoder is known by acquiring the network frame receiving information and decoding frame sending information of the streaming media data. The condition of the hardware decoder decoding the streaming media data is known by inputting the streaming media data into the hardware decoder for decoding to generate a decoding frame and performance cache information. And then determining the frame fetching time and the frame fetching frame number according to the network frame receiving information, the decoding frame sending information and the performance cache information, wherein the determination of the frame fetching time and the frame fetching frame number takes the condition of the streaming media data and the condition of the hardware decoder decoding the streaming media data into consideration, so that the decoded frame generated by decoding can be timely fetched, and further, when the decoded frame is fetched from the cache of the hardware decoder according to the frame fetching time and the frame fetching frame number for playing, the obtained playing file is smooth and not blocked.
In one embodiment of the present application, the frame fetching time and the frame fetching number may be determined according to one or more of network frame receiving information, decoding frame sending information, and performance buffering information.
In an embodiment of the present application, the real-time frame rate and the initial frame fetching time may be determined according to the number of packet receiving frames, the encoding timestamp, and the decoding performance, and then the initial frame fetching time may be modified according to the decoding frame sending information to generate the frame fetching time.
In an embodiment of the present application, the decoding performance may be obtained by testing under line in advance, the test data may be input into the hardware decoder, and the decoding performance of the hardware decoder may be calculated according to the decoding time and the number of decoding frames of the test data corresponding to the hardware decoder.
In this embodiment, the number of packet receiving frames, the encoding time stamp, and the decoding performance can be obtained before decoding, and the initial frame fetching time and the initial frame fetching number can be estimated before decoding. The real-time frame rate can be obtained according to the number of the packet receiving frames and the encoding time stamp, and the real-time frame rate can be used for decoding and can also be used for estimating the initial frame taking time and the initial frame taking number. And correcting the initial frame taking time and the initial frame taking frame number according to the decoding frame sending information in the decoding process to generate the frame taking time and the frame taking frame number, so that the frames can be taken in time, and the pictures can be played smoothly.
In an embodiment of the present application, after the initial frame fetching time and the initial frame fetching number are corrected according to the decoding frame sending information in the decoding process, the corrected initial frame fetching time and the corrected initial frame fetching number may be corrected again according to the buffer frame number of the hardware decoder to obtain the frame fetching time and the frame fetching number.
In an embodiment of the present application, in the embodiment shown in fig. 2, the process of determining the frame fetching time according to the network frame receiving information, the decoding frame sending information, and the performance buffering information may include steps S310 to S330 shown in fig. 3:
in step S310, an initial frame fetching time is determined according to the number of packet receiving frames, the encoding time stamp, and the decoding performance.
In step S320, it is determined whether the decoded frame information is greater than or equal to a preset threshold.
In step S330, if the decoded frame sending information is greater than or equal to the preset threshold, the initial frame fetching time is modified by the first modification coefficient to generate the frame fetching time.
In step S340, if the decoded frame sending information is smaller than the preset threshold, the initial frame fetching time is modified by the second modification coefficient to generate the frame fetching time.
In one embodiment of the present application, the frame taking time interval obtained by the first correction coefficient may be smaller than the frame taking time interval obtained by the second correction coefficient.
In the embodiment of fig. 3, the decoding frame sending information reflects the decoding process of the hardware decoder, and the initial frame fetching time is corrected according to the decoding process of the hardware decoder, so that the decoded frame can be fetched more timely, and the video playing is smoother.
In an embodiment of the present application, in the embodiment shown in fig. 2, the process of determining the frame number according to the network frame receiving information, the decoding frame sending information, and the performance buffering information may include steps S410 to S490 shown in fig. 4:
in step S410, it is determined whether the decoded frame information is greater than or equal to a predetermined threshold.
In step S420, if the decoded frame sending information is greater than or equal to the predetermined threshold, it is determined whether the number of buffered frames is greater than the number of buffered accumulated frames.
In step S430, if the decoded frame sending information is greater than or equal to the preset threshold and the number of the buffer frames is greater than the number of the buffer accumulation frames, it is determined that the number of the frame fetching frames is N, where N is an integer greater than 1.
In an embodiment of the present application, if the decoding frame sending information is greater than or equal to the preset threshold, it indicates that the number of frames to be decoded sent to the hardware decoder is greater, and the number of buffered frames is greater than the number of buffered frames, it indicates that new decoded frames are sent in the buffer of the hardware decoder in addition to the number of buffered frames, and at this time, frame fetching is to be accelerated, so that the number of frame fetching frames is determined to be N, where N is an integer greater than 1. N can be adjusted in real time according to actual conditions or set according to empirical values.
In step S440, if the decoded frame sending information is greater than or equal to the preset threshold and the number of buffered frames is equal to the number of buffered accumulated frames, it is determined that the number of frame fetching frames is 1.
In an embodiment of the present application, if the decoding frame sending information is greater than or equal to the preset threshold, it indicates that the number of frames to be decoded sent to the hardware decoder is large, but there is a possibility that the hardware decoder has not generated a decoded frame, so that the number of buffered frames is equal to the number of stored frames in the buffer, at this time, the number of frame fetching frames is determined to be 1 frame, and the number of stored frames in the buffer is fetched.
In an embodiment of the present application, if the decoding frame sending information is greater than or equal to the preset threshold and the number of the buffer frames is equal to the number of the buffer bin, it may be determined that the number of the frame fetching frames is N, where N is an integer greater than 1, so as to prevent the hardware decoder from bin the decoded frame to be generated.
In step S450, if the decoded frame sending information is smaller than the preset threshold, it is determined whether the decoded frame sending information is equal to 1.
In step S460, if the decoded frame sending information is equal to 1, it is determined that the number of frame taking frames is 1.
In an embodiment of the present application, the number of frames to be decoded sent to the decoder by the hardware decoder is used as the decoding frame sending information, and if the number of frames to be decoded sent to the decoder is 1, it is determined that the number of frame fetching frames is 1, so that frame accumulation can be avoided.
In step S470, if the decoded frame information is greater than 1, it is determined whether the number of buffer frames is greater than 0.
In step S480, if the decoding frame sending information is greater than 1 and the number of buffering frames is greater than 0, it is determined that the number of frame taking frames is N, where N is an integer greater than 1.
In an embodiment of the present application, if the decoding frame sending information is greater than 1, it indicates that there are still frames to be decoded that are sent to the hardware decoder, and if the number of buffered frames is greater than 0, it indicates that the hardware decoder is still generating decoded frames or there are inherent bin frames in the decoder, and it is determined that the number of frame fetching frames is N, where N is an integer greater than 1, so as to avoid the occurrence of bin frames in the buffer of the hardware decoder or to be able to fetch the inherent bin frames in the decoder.
In step S490, if the decoded frame information is not equal to 1 and the number of buffered frames is equal to 0, it is determined that the number of frame fetching frames is 1.
In an embodiment of the present application, if the number of buffered frames is equal to 0, it indicates that the hardware decoder has not generated a decoded frame, but to prevent a hoarding frame, the frame fetching continues, and the number of frame fetching frames is set to 1, so as to prevent a hoarding frame from occurring in the buffer of the hardware decoder.
In an embodiment of the present application, the streaming media data playing method of the present application can be used in an application program that needs to perform streaming media playing, such as wireless screen casting, QQ, national karaoke, and the like. Taking the application of the streaming media data playing method in the application to wireless screen projection as an example, the screen projection display device obtains streaming media data to be played from the client through the network, and when the existing screen projection display device obtains the streaming media data from the client applying the windows platform, the playing delay of the streaming media data is 180 ms. When the streaming media data playing method is used for acquiring the streaming media data from the client of the application windows platform, the playing delay of the streaming media data is 125 ms. Therefore, compared with the conventional streaming media data playing method, the streaming media data playing method has shorter delay time and faster transmission, and can make the playing of the streaming media data smoother.
The following describes an embodiment of an apparatus of the present application, which may be used to execute a streaming media data playing method in the foregoing embodiment of the present application. For details that are not disclosed in the embodiments of the apparatus of the present application, please refer to the embodiments of the streaming media data playing method described above in the present application.
Fig. 5 schematically shows a block diagram of a streaming media data playback apparatus according to an embodiment of the present application.
Referring to fig. 5, a streaming media data playing apparatus 500 according to an embodiment of the present application includes an information module 501, a decoding module 502, a calculating module 503, and a frame fetching module 504.
In some embodiments of the present application, based on the foregoing solution, the information module 501 is configured to obtain network frame receiving information and decoding frame sending information of streaming media data; the decoding module 502 is configured to input the streaming media data into a hardware decoder for decoding to generate a decoded frame and performance cache information; the calculation module 503 is configured to determine a frame fetching time and a frame fetching frame number according to the network frame receiving information, the decoding frame sending information, and the performance cache information; the frame fetching module 504 is configured to fetch a decoded frame from a buffer of a hardware decoder according to the frame fetching time and the frame fetching number, so as to perform a playing obtaining module.
In some embodiments of the present application, based on the foregoing scheme, the information module 501 is configured to: acquiring a coding time stamp corresponding to each frame of coding in streaming media data through a far-end coder; acquiring a packet receiving frame number of the streaming media data through a far-end encoder; and generating network frame receiving information according to the encoding time stamp and the packet receiving frame number.
In some embodiments of the present application, based on the foregoing solution, the information module 501 is further configured to: and generating decoding frame sending information according to the effective frame number to be decoded of the streaming media data.
In some embodiments of the present application, based on the foregoing scheme, the decoding module 502 is configured to: inputting the stream media data into a hardware decoder for decoding to generate a decoding frame; and in the decoding process, generating performance cache information according to the performance of the hardware decoder and the cache information.
In some embodiments of the present application, based on the foregoing scheme, the decoding module 502 is further configured to: generating decoding performance according to the decoding frame number generated by a hardware decoder in preset time; acquiring the number of buffer frames of a hardware decoder in a preset time; and generating performance cache information according to the number of cache frames and the decoding performance.
In some embodiments of the present application, based on the foregoing solution, the calculation module 503 is configured to: determining a real-time frame rate and initial frame taking time according to the number of packet receiving frames, the encoding timestamp and the decoding performance; and correcting the initial frame taking time according to the decoding frame sending information to generate the frame taking time.
In some embodiments of the present application, based on the foregoing solution, the calculation module 503 is further configured to: when the decoding frame sending information is larger than or equal to a preset threshold value, correcting the initial frame taking time through a first correction coefficient to generate frame taking time; and when the decoding frame sending information is smaller than the preset threshold value, correcting the initial frame taking time through a second correction coefficient to generate the frame taking time.
In some embodiments of the present application, based on the foregoing solution, the calculation module 503 is further configured to: when the decoding frame sending information is greater than or equal to a preset threshold value and the number of cache frames is greater than the number of accumulated frames in the cache, determining the number of frame taking frames as N; wherein N is an integer greater than 1.
In some embodiments of the present application, based on the foregoing solution, the calculation module 503 is further configured to: when the decoding frame sending information is larger than 1 and smaller than a preset threshold value and the number of the buffer frames is larger than 0, determining the number of the frame taking frames to be N; wherein N is an integer greater than 1.
As will be appreciated by one skilled in the art, aspects of the present application may be embodied as a system, method or program product. Accordingly, various aspects of the present application may be embodied in the form of: an entirely hardware embodiment, an entirely software embodiment (including firmware, microcode, etc.) or an embodiment combining hardware and software aspects that may all generally be referred to herein as a "circuit," module "or" system.
An electronic device 60 according to this embodiment of the present application is described below with reference to fig. 6. The electronic device 60 shown in fig. 6 is only an example, and should not bring any limitation to the functions and the scope of use of the embodiments of the present application.
As shown in fig. 6, the electronic device 60 is in the form of a general purpose computing device. The components of the electronic device 60 may include, but are not limited to: the at least one processing unit 61, the at least one memory unit 62, a bus 63 connecting different system components (including the memory unit 62 and the processing unit 61), and a display unit 64.
Wherein the storage unit stores program code executable by the processing unit 61 to cause the processing unit 61 to perform the steps according to various exemplary embodiments of the present application described in the section "example methods" above in this specification.
The storage unit 62 may include readable media in the form of volatile memory units, such as a random access memory unit (RAM)621 and/or a cache memory unit 622, and may further include a read only memory unit (ROM) 623.
The storage unit 62 may also include a program/utility 624 having a set (at least one) of program modules 625, such program modules 625 including, but not limited to: an operating system, one or more application programs, other program modules, and program data, each of which, or some combination thereof, may comprise an implementation of a network environment.
The electronic device 60 may also communicate with one or more external devices (e.g., keyboard, pointing device, bluetooth device, etc.), with one or more devices that enable a user to interact with the electronic device 60, and/or with any devices (e.g., router, modem, etc.) that enable the electronic device 60 to communicate with one or more other computing devices. Such communication may be through an input/output (I/O) interface 65. Also, the electronic device 60 may communicate with one or more networks (e.g., a Local Area Network (LAN), a Wide Area Network (WAN), and/or a public network, such as the Internet) via the network adapter 66. As shown, network adapter 66 communicates with the other modules of electronic device 60 via bus 63. It should be appreciated that although not shown in the figures, other hardware and/or software modules may be used in conjunction with electronic device 60, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems, among others.
Through the above description of the embodiments, those skilled in the art will readily understand that the exemplary embodiments described herein may be implemented by software, or by software in combination with necessary hardware. Therefore, the technical solution according to the embodiments of the present application can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (which can be a CD-ROM, a usb disk, a removable hard disk, etc.) or on a network, and includes several instructions to make a computing device (which can be a personal computer, a server, a terminal device, or a network device, etc.) execute the method according to the embodiments of the present application.
There is also provided, in accordance with an embodiment of the present application, a computer-readable storage medium having stored thereon a program product capable of implementing the above-described method of the present specification. In some possible embodiments, various aspects of the present application may also be implemented in the form of a program product comprising program code for causing a terminal device to perform the steps according to various exemplary embodiments of the present application described in the "exemplary methods" section above of this specification, when the program product is run on the terminal device.
In some embodiments of the present application, a program product for implementing the above method of embodiments of the present application is provided, which may employ a portable compact disc read only memory (CD-ROM) and include program code, and may be run on a terminal device, such as a personal computer. However, the program product of the present application is not limited thereto, and in this document, a readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device.
The program product may employ any combination of one or more readable media. The readable medium may be a readable signal medium or a readable storage medium. A readable storage medium may be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the foregoing. More specific examples (a non-exhaustive list) of the readable storage medium include: an electrical connection having one or more wires, a portable disk, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.
A computer readable signal medium may include a propagated data signal with readable program code embodied therein, for example, in baseband or as part of a carrier wave. Such a propagated data signal may take many forms, including, but not limited to, electro-magnetic, optical, or any suitable combination thereof. A readable signal medium may also be any readable medium that is not a readable storage medium and that can communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device.
Program code embodied on a readable medium may be transmitted using any appropriate medium, including but not limited to wireless, wireline, optical fiber cable, RF, etc., or any suitable combination of the foregoing.
Program code for carrying out operations of the present application may be written in any combination of one or more programming languages, including an object oriented programming language such as Java, C + + or the like and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code may execute entirely on the user's computing device, partly on the user's device, as a stand-alone software package, partly on the user's computing device and partly on a remote computing device, or entirely on the remote computing device or server. In the case of a remote computing device, the remote computing device may be connected to the user computing device through any kind of network, including a Local Area Network (LAN) or a Wide Area Network (WAN), or may be connected to an external computing device (e.g., through the internet using an internet service provider).
Furthermore, the above-described figures are merely schematic illustrations of processes involved in methods according to exemplary embodiments of the present application, and are not intended to be limiting. It will be readily understood that the processes shown in the above figures are not intended to indicate or limit the chronological order of the processes. In addition, it is also readily understood that these processes may be performed synchronously or asynchronously, e.g., in multiple modules.
It will be understood that the present application is not limited to the precise arrangements described above and shown in the drawings and that various modifications and changes may be made without departing from the scope thereof. The scope of the application is limited only by the appended claims.
Claims (10)
1. A method for playing streaming media data, comprising:
acquiring network frame receiving information and decoding frame sending information of streaming media data;
inputting the stream media data into a hardware decoder for decoding to generate a decoding frame and performance cache information;
determining frame taking time and frame taking number according to the network frame receiving information, the decoding frame sending information and the performance cache information; and
and extracting the decoded frame from the buffer of the hardware decoder according to the frame extracting time and the frame extracting frame number for playing.
2. The method for playing streaming media data according to claim 1, wherein the acquiring network frame receiving information and decoding frame sending information of the streaming media data comprises:
acquiring a coding time stamp corresponding to each frame of coding in the streaming media data through a far-end coder;
acquiring the packet receiving frame number of the streaming media data through the far-end encoder; and
and generating the network frame receiving information according to the coding time stamp and the packet receiving frame number.
3. The method for playing streaming media data according to claim 2, wherein the acquiring network frame receiving information and decoding frame sending information of the streaming media data comprises:
and generating the decoding frame sending information according to the effective frame number to be decoded of the streaming media data.
4. The method for playing streaming media data according to claim 3, wherein the inputting of the streaming media data into a hardware decoder for decoding to generate a decoded frame and performance buffering information comprises:
inputting the streaming media data into a hardware decoder for decoding to generate the decoding frame; and
and in the decoding process, generating the performance cache information according to the performance and the cache information of the hardware decoder.
5. The method for playing streaming media data according to claim 4, wherein the generating the performance buffering information according to the performance and buffering information of the hardware decoder comprises:
generating decoding performance according to the decoding frame number generated by the hardware decoder in preset time;
acquiring the number of buffer frames of the hardware decoder in a preset time; and
and generating the performance caching information according to the caching frame number and the decoding performance.
6. The method for playing streaming media data according to claim 3 or 5, wherein the determining a frame fetching time and a frame fetching frame number according to the network frame receiving information, the decoding frame sending information, and the performance buffering information comprises:
determining a real-time frame rate and initial frame taking time according to the packet receiving frame number, the encoding timestamp and the decoding performance;
and correcting the initial frame taking time according to the decoding frame sending information to generate the frame taking time.
7. The method for playing streaming media data according to claim 6, wherein the modifying the initial frame fetching time according to the decoded frame sending information to generate the frame fetching time comprises:
when the decoding frame sending information is larger than or equal to a preset threshold value, correcting the initial frame taking time through a first correction coefficient to generate the frame taking time; and
and when the decoding frame sending information is smaller than a preset threshold value, correcting the initial frame taking time through a second correction coefficient to generate the frame taking time.
8. The method for playing streaming media data according to claim 3 or 5, wherein the determining a frame fetching time and a frame fetching frame number according to the network frame receiving information, the decoding frame sending information, and the performance buffering information comprises:
when the decoding frame sending information is greater than or equal to a preset threshold value and the caching frame number is greater than the caching accumulation frame number, determining the frame taking frame number to be N;
wherein N is an integer greater than 1.
9. The method for playing streaming media data according to claim 3 or 5, wherein the determining a frame fetching time and a frame fetching frame number according to the network frame receiving information, the decoding frame sending information, and the performance buffering information comprises:
when the decoding frame sending information is larger than 1 and smaller than a preset threshold value and the number of the cache frames is larger than 0, determining the number of the frame taking frames to be N;
wherein N is an integer greater than 1.
10. A streaming media data playback apparatus, comprising:
the information module is used for acquiring network frame receiving information and decoding frame sending information of the streaming media data;
the decoding module is used for inputting the streaming media data into a hardware decoder for decoding to generate a decoding frame and performance cache information;
the computing module is used for determining frame taking time and frame taking number according to the network frame receiving information, the decoding frame sending information and the performance cache information; and
and the frame fetching module is used for extracting the decoded frame from the buffer memory of the hardware decoder according to the frame fetching time and the frame fetching number for playing.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911060374.1A CN110855645B (en) | 2019-11-01 | 2019-11-01 | Streaming media data playing method and device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911060374.1A CN110855645B (en) | 2019-11-01 | 2019-11-01 | Streaming media data playing method and device |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN110855645A true CN110855645A (en) | 2020-02-28 |
| CN110855645B CN110855645B (en) | 2021-10-22 |
Family
ID=69599298
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201911060374.1A Active CN110855645B (en) | 2019-11-01 | 2019-11-01 | Streaming media data playing method and device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110855645B (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2023020270A1 (en) * | 2021-08-20 | 2023-02-23 | 腾讯科技（深圳）有限公司 | Decoding processing method and apparatus, computer device, and storage medium |
Citations (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101442586A (en) * | 2008-12-31 | 2009-05-27 | 中兴通讯股份有限公司 | Method and terminal for playing multimedia |
| US20090299505A1 (en) * | 2008-05-30 | 2009-12-03 | Nokia Corporation | Optimizing seek functionality in media content |
| CN101873475A (en) * | 2010-01-07 | 2010-10-27 | 杭州海康威视数字技术股份有限公司 | Control command sending method, data transmission method, monitoring system and device |
| CN102651821A (en) * | 2011-02-28 | 2012-08-29 | 华为技术有限公司 | Method and device for evaluating quality of video |
| US20140049689A1 (en) * | 2011-12-05 | 2014-02-20 | Guangzhou Ucweb Computer Technology Co., Ltd | Method and apparatus for streaming media data processing, and streaming media playback equipment |
| CN109168083A (en) * | 2018-10-23 | 2019-01-08 | 青岛海信电器股份有限公司 | A kind of Streaming Media real time playing method and device |
| CN109936763A (en) * | 2017-12-15 | 2019-06-25 | 腾讯科技（深圳）有限公司 | The processing of video and dissemination method |
| CN110072143A (en) * | 2019-03-18 | 2019-07-30 | 视联动力信息技术股份有限公司 | A kind of method for decoding video stream and device |
| CN110213308A (en) * | 2018-02-28 | 2019-09-06 | 杭州海康威视数字技术股份有限公司 | A kind of method and device of decoding video data |
| US20190335216A1 (en) * | 2017-08-24 | 2019-10-31 | Skitter, Inc. | Method for synchronizing gops and idr-frames on multiple encoders without communication |
-
2019
- 2019-11-01 CN CN201911060374.1A patent/CN110855645B/en active Active
Patent Citations (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20090299505A1 (en) * | 2008-05-30 | 2009-12-03 | Nokia Corporation | Optimizing seek functionality in media content |
| CN101442586A (en) * | 2008-12-31 | 2009-05-27 | 中兴通讯股份有限公司 | Method and terminal for playing multimedia |
| CN101873475A (en) * | 2010-01-07 | 2010-10-27 | 杭州海康威视数字技术股份有限公司 | Control command sending method, data transmission method, monitoring system and device |
| CN102651821A (en) * | 2011-02-28 | 2012-08-29 | 华为技术有限公司 | Method and device for evaluating quality of video |
| US20140049689A1 (en) * | 2011-12-05 | 2014-02-20 | Guangzhou Ucweb Computer Technology Co., Ltd | Method and apparatus for streaming media data processing, and streaming media playback equipment |
| US20190335216A1 (en) * | 2017-08-24 | 2019-10-31 | Skitter, Inc. | Method for synchronizing gops and idr-frames on multiple encoders without communication |
| CN109936763A (en) * | 2017-12-15 | 2019-06-25 | 腾讯科技（深圳）有限公司 | The processing of video and dissemination method |
| CN110213308A (en) * | 2018-02-28 | 2019-09-06 | 杭州海康威视数字技术股份有限公司 | A kind of method and device of decoding video data |
| CN109168083A (en) * | 2018-10-23 | 2019-01-08 | 青岛海信电器股份有限公司 | A kind of Streaming Media real time playing method and device |
| CN110072143A (en) * | 2019-03-18 | 2019-07-30 | 视联动力信息技术股份有限公司 | A kind of method for decoding video stream and device |
Non-Patent Citations (3)
| Title |
|---|
| 单俊丽: "《基于Android的流媒体客户端的研究与设计》", 《中国硕士学位论文全文数据库 信息技术辑》 * |
| 徐红阳: "《基于多媒体处理器的图像视频解码研究》", 《中国硕士学位论文全文数据库 信息技术辑》 * |
| 王瑞刚,李燕: "《IP电话终端设备：原理、电路及应用》", 《西安电子科技大学出版社》 * |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2023020270A1 (en) * | 2021-08-20 | 2023-02-23 | 腾讯科技（深圳）有限公司 | Decoding processing method and apparatus, computer device, and storage medium |
Also Published As
| Publication number | Publication date |
|---|---|
| CN110855645B (en) | 2021-10-22 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| WO2020248909A1 (en) | Video decoding method and apparatus, computer device, and storage medium | |
| CN111135569A (en) | Cloud game processing method and device, storage medium and electronic equipment | |
| US9916837B2 (en) | Methods and apparatuses for transmitting and receiving audio signals | |
| US11057628B2 (en) | Effective intra encoding for screen data | |
| US20200241835A1 (en) | Method and apparatus of audio/video switching | |
| EP3142381A1 (en) | Network video playing method and device | |
| WO2019134499A1 (en) | Method and device for labeling video frames in real time | |
| CN108259998B (en) | Player, play control method and device, electronic equipment and play system | |
| CN104053002A (en) | Video decoding method and device | |
| CN114040245B (en) | Video playing method and device, computer storage medium and electronic equipment | |
| CN110855645B (en) | Streaming media data playing method and device | |
| CN113079386B (en) | Video online playing method and device, electronic equipment and storage medium | |
| CN111343503A (en) | Video transcoding method and device, electronic equipment and storage medium | |
| CN114422799A (en) | Video file decoding method and device, electronic equipment and program product | |
| CN114245175A (en) | Video transcoding method and device, electronic equipment and storage medium | |
| CN113573003A (en) | Weak network-based audio and video real-time communication method, device and equipment | |
| US11272227B1 (en) | Buffer recovery in segmented media delivery applications | |
| CN114339415B (en) | Client video playing method and device, electronic equipment and readable medium | |
| CN114079534B (en) | Encoding method, decoding method, apparatus, medium, and electronic device | |
| CN110798700B (en) | Video processing method, video processing device, storage medium and electronic equipment | |
| CN109788357B (en) | Method and device for playing media file | |
| US20100076944A1 (en) | Multiprocessor systems for processing multimedia data and methods thereof | |
| CN115942000B (en) | H.264 format video stream transcoding method, device, equipment and medium | |
| CN115243089B (en) | Audio and video synchronous rendering method and device and electronic equipment | |
| CN114079535B (en) | Transcoding method, device, medium and electronic equipment |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| REG | Reference to a national code |
Ref country code: HK Ref legal event code: DE Ref document number: 40021980 Country of ref document: HK |
|
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |