WO2017129090A1 - 移动终端的数据传输方法和装置 - Google Patents

移动终端的数据传输方法和装置 Download PDF

Info

Publication number
WO2017129090A1
WO2017129090A1 PCT/CN2017/072195 CN2017072195W WO2017129090A1 WO 2017129090 A1 WO2017129090 A1 WO 2017129090A1 CN 2017072195 W CN2017072195 W CN 2017072195W WO 2017129090 A1 WO2017129090 A1 WO 2017129090A1
Authority
WO
WIPO (PCT)
Prior art keywords
data
cache
code rate
cached data
cache ratio
Prior art date
Application number
PCT/CN2017/072195
Other languages
English (en)
French (fr)
Inventor
余学亮
Original Assignee
腾讯科技（深圳）有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 腾讯科技（深圳）有限公司 filed Critical 腾讯科技（深圳）有限公司
Publication of WO2017129090A1 publication Critical patent/WO2017129090A1/zh
Priority to US15/937,238 priority Critical patent/US10728594B2/en

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
    • H04N21/23—Processing of content or additional data; Elementary server operations; Server middleware
    • H04N21/239—Interfacing the upstream path of the transmission network, e.g. prioritizing client content requests
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
    • H04N21/21—Server components or server architectures
    • H04N21/218—Source of audio or video content, e.g. local disk arrays
    • H04N21/2187—Live feed
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
    • H04N21/23—Processing of content or additional data; Elementary server operations; Server middleware
    • H04N21/231—Content storage operation, e.g. caching movies for short term storage, replicating data over plural servers, prioritizing data for deletion
    • H04N21/23106—Content storage operation, e.g. caching movies for short term storage, replicating data over plural servers, prioritizing data for deletion involving caching operations
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
    • H04N21/23—Processing of content or additional data; Elementary server operations; Server middleware
    • H04N21/238—Interfacing the downstream path of the transmission network, e.g. adapting the transmission rate of a video stream to network bandwidth; Processing of multiplex streams
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
    • H04N21/25—Management operations performed by the server for facilitating the content distribution or administrating data related to end-users or client devices, e.g. end-user or client device authentication, learning user preferences for recommending movies
    • H04N21/266—Channel or content management, e.g. generation and management of keys and entitlement messages in a conditional access system, merging a VOD unicast channel into a multicast channel
    • H04N21/2662—Controlling the complexity of the video stream, e.g. by scaling the resolution or bitrate of the video stream based on the client capabilities
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
    • H04N21/27—Server based end-user applications
    • H04N21/274—Storing end-user multimedia data in response to end-user request, e.g. network recorder
    • H04N21/2743—Video hosting of uploaded data from client
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/41—Structure of client; Structure of client peripherals
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/41—Structure of client; Structure of client peripherals
    • H04N21/414—Specialised client platforms, e.g. receiver in car or embedded in a mobile appliance
    • H04N21/41407—Specialised client platforms, e.g. receiver in car or embedded in a mobile appliance embedded in a portable device, e.g. video client on a mobile phone, PDA, laptop
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
    • H04N21/433—Content storage operation, e.g. storage operation in response to a pause request, caching operations
    • H04N21/4331—Caching operations, e.g. of an advertisement for later insertion during playback
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
    • H04N21/442—Monitoring of processes or resources, e.g. detecting the failure of a recording device, monitoring the downstream bandwidth, the number of times a movie has been viewed, the storage space available from the internal hard disk
    • H04N21/44245—Monitoring the upstream path of the transmission network, e.g. its availability, bandwidth

Definitions

  • the present invention relates to the field of the Internet, and in particular to a data transmission method and apparatus for a mobile terminal.
  • the mobile terminal device plays video, for example, the mobile phone plays video, etc., in the second generation mobile communication network (Second Generation, referred to as 2G) / third generation mobile communication network (Third Generation, referred to as 3G) / fourth generation mobile communication network (Fourth Generation, referred to as 4G) or Wireless Fidelity (WIFI) network environment, due to the complex and variable network status, the stability of the playback content is low.
  • the mobile terminal device shares the recorded content in real time
  • the live content of the mobile terminal device is time-sensitive. If the recorded content is attractive and the user terminal has high attention, it is first necessary to ensure that the played content is not card, thereby being the user terminal. Provide smooth and stable live content to enhance the user experience.
  • the state of the network is largely restricted by the distance of regional base stations and the influence of crowd density.
  • the original 4G mobile network will drop to 3G mobile networks and even drop to 2G mobile networks. In this way, the slow transmission speed of the mobile network will greatly limit the uploading of live content.
  • WIFI hotspot coverage is incomplete. Because the user terminal has mobile characteristics, the strength of the WIFI signal received in each place will be different, which will also affect the stability of the live content upload of the mobile terminal. .
  • the method of discarding a part of the encoded data frame for the video is used to reduce the bit rate data amount in a disguised manner, which satisfies the live broadcast upload, but the disadvantage of this solution is that the original image compression content is lost, the frame rate is low, and the fluency is reduced.
  • the amount of content information is reduced, and the same needs Step video group picture group (Group of Picture, referred to as GOP) data, it is possible to lose 1 to 2 seconds of picture data, if the processing is not good, the lost data will cause the decoding end of the decoding failure, the image is abnormal, and then the viewing end has Jumping, resulting in a discontinuous experience, frequent frame loss processing will give users a very bad sense of jumping.
  • GOP Step video group picture group
  • the encoder is reset according to the current network speed and the audio and video code rate is reset, and the live broadcast is passively restarted, causing the live broadcast to be interrupted to the process of reinitialization, which takes time, and this time period Internally affecting the viewing of the playback terminal, there is a jamming buffer and waiting. If the live broadcast builds a multi-layer complex link, the speed of restoring the playback end becomes very slow, so that the amount of lost users is inevitable, if the number of users who watch the recorded video is inevitable. Very large, such interruption and recovery is also a great challenge to the stability and bandwidth adjustment of the server room.
  • the prior art solution has the following disadvantages: First, the overall quality is controlled by the low code rate priority, and cannot be flexibly adapted anytime and anywhere. Second, even if the dynamic control mode is adopted, it is too simple and rude, and requires a large user. As a cost, experience third, bring operating costs and machine scheduling overhead, increase the probability of accidents, lack of security and robustness.
  • the embodiment of the invention provides a data transmission method and device for a mobile terminal, so as to at least solve the technical problem that the flexibility of the acquisition and encoding of the audio and video in the related art is low.
  • a data transmission method for a mobile terminal includes: acquiring cache data of a mobile application on a mobile terminal, where the cache data is a mobile application for coding synthesis.
  • the stream data The cache ratio of the cached data in the networked state, wherein the cache ratio is the ratio of the cache real-time valid data to the total allocated cache size; determining whether the cache ratio exceeds the preset cache percentage threshold; The outbound cache value exceeds the preset cache ratio threshold, adjusts the code rate of the cached data, and obtains the adjusted code rate; and transmits the buffered data to the receiving terminal according to the adjusted bit rate in the networked state, so that the receiving terminal is configured to use the cached data. Play the live content corresponding to the cached data.
  • a data transmission apparatus of a mobile terminal includes: a first acquiring unit, configured to acquire cached data of the mobile application on the mobile terminal, where the data is cached
  • the second obtaining unit is configured to obtain the cache ratio of the cached data in the networked state, wherein the cache ratio is the cached real-time valid data and The ratio of the total allocation cache size
  • the determining unit is configured to determine whether the cache ratio exceeds a preset cache ratio threshold
  • the adjusting unit is configured to adjust the cache data when determining that the cache ratio exceeds a preset cache ratio threshold
  • the code rate is obtained, and the transmission unit is configured to transmit the buffer data to the receiving terminal according to the adjusted code rate in the networked state, so that the receiving terminal root is used to play the live content corresponding to the cached data according to the cached data.
  • a computer readable storage medium having stored thereon program instructions for performing the data transfer method.
  • the cached data of the mobile application is obtained on the mobile terminal, and the cached proportion of the cached data is obtained in a networked state, and it is determined whether the cached ratio exceeds a preset cache percentage threshold.
  • the buffer ratio is greater than the preset cache ratio threshold, adjusting the code rate of the buffered data, obtaining an adjusted code rate, and transmitting the cached data to the receiving terminal according to the adjusted code rate, where
  • the receiving terminal is configured to play the live content corresponding to the cached data according to the cached data, and achieve the purpose of adjusting the code rate of the cached data according to the cache ratio, thereby improving the flexibility of the audio and video capture and encoding uploading.
  • the technical effect further solves the technical problem that the flexibility of the acquisition and encoding of the audio and video in the related art is low.
  • FIG. 1 is a block diagram showing the hardware structure of a computer terminal of a data transmission method of a mobile terminal according to an embodiment of the present invention
  • FIG. 2 is a flowchart of a data transmission method of a mobile terminal according to an embodiment of the present invention
  • step 204 of FIG. 2 is a flow diagram of the steps involved in step 204 of FIG. 2, in accordance with an embodiment of the present invention
  • FIG. 4 is a schematic diagram showing a relationship between a cache ratio and a code rate of a mobile terminal according to an embodiment of the present invention
  • FIG. 5 is a schematic diagram of a live broadcast user's mobile phone upload cache change and a terminal user play situation as a network scenario changes according to an embodiment of the present invention
  • 6A-6C are schematic diagrams of video I frame and P frame transmission according to an embodiment of the present invention.
  • FIG. 7 is a schematic diagram of a data transmission apparatus of a mobile terminal according to an embodiment of the present invention.
  • FIG. 8 is a schematic diagram of a data transmission apparatus of a mobile terminal according to an embodiment of the present invention.
  • FIG. 9 is a schematic diagram of a data transmission apparatus of a mobile terminal according to an embodiment of the present invention.
  • FIG. 10 is a block diagram showing the structure of a computer terminal according to an embodiment of the present invention.
  • Bit rate Bitrate, the unit is kbps, the amount of audio and video data per second.
  • Video image picture and sound quality live network upload requires the code rate as low as possible, storage local requires the code rate as high as possible, the corresponding quality is as good as possible.
  • Single-frame picture pure audio The video picture is fixed with a still picture, non-dynamic frame sequence, audio reservation, and the information volume and code rate of the entire video stream are very low, which is suitable for transmission in a harsh network environment.
  • Hardware coding an audio and video compression processing logic unit of a mobile phone chip.
  • the general mobile phone device has a H.264 video hardware coding logic unit for video compression coding, which is different from the software coding mode, and the hardware coding does not occupy the device CPU resources.
  • Upload cache The memory area saved by the live stream data.
  • the cache is written from the real-time audio and video coding packaged data.
  • the read/write output is obtained in real time for uploading, writing, and reading speed synchronization.
  • the live broadcast is in the best real-time state. If the network upload is slow, the cache read is slower, the read speed is lower than the write speed, the cache will be backlogged, the proportion will increase, and the live broadcast will be lost.
  • Temporary, the playback buffer will appear in the buffered waiting data state.
  • Compression coding The original large amount of data of audio and video is processed by a series of compression coding algorithms to generate compressed data, which is convenient for transmission and storage. After the player obtains the compressed encoded data, the original image is decoded and restored.
  • Key frame The first frame data of the video compression coding sequence group is used to play and decode the subsequent compressed frame, which serves as a reference function. If there is no key frame, the playback end decoder cannot decode and restore the subsequent image normally, which will result in decoding the image data. Abnormal, resulting in a flower screen phenomenon, and the key frame also plays a role in dragging and positioning the video segment at a certain point in time.
  • GOP Fully referred to as Group of picture, refers to a sequence of frames in which compressed video coding frames are packed into a group. All image compression frames between two key frames form a GOP.
  • QP Quantization parameter, a reference standard for rate control.
  • IPB frame The compressed frame type output after video encoding.
  • the I frame is intraframe coded.
  • the first frame at the beginning of the GOP has a low compression ratio, but it serves as a reference encoding P and B frame.
  • the P frame is a one-way reference compression.
  • the frame is used for compressing the frame estimation motion vector and the residual data with reference to the previous frame, without actually compressing the repeated pixel data of the image, which is higher than the I frame compression rate and lower than the B frame compression rate;
  • the B frame is bidirectional reference compression. There is one more direction reference in the P frame, and the compression ratio is the highest.
  • UGC User-generated content, called User Generated Content, where users display their original content on the Internet platform or provide it to other users.
  • an embodiment of a data transmission method of a mobile terminal there is provided an embodiment of a data transmission method of a mobile terminal, and it is to be noted that the steps illustrated in the flowchart of the accompanying drawings may be executed in a computer system such as a set of computer executable instructions, and Although the logical order is shown in the flowcharts, in some cases the steps shown or described may be performed in a different order than the ones described herein.
  • FIG. 1 is a hardware structural block diagram of a computer terminal of a data transmission method of a mobile terminal according to an embodiment of the present invention.
  • computer terminal 1 may include one or more (only one shown) processor 102 (processor 102 may include, but is not limited to, a processing device such as a microprocessor MCU or a programmable logic device FPGA)
  • processor 102 may include, but is not limited to, a processing device such as a microprocessor MCU or a programmable logic device FPGA)
  • a memory 104 for storing data
  • a transmission module 106 for communication functions.
  • computer terminal 1 may also include more or fewer components than those shown in FIG. 1, or have a different configuration than that shown in FIG.
  • the memory 104 can be used to store software programs and modules of the application software, such as program instructions/modules corresponding to the data transmission method of the mobile terminal in the embodiment of the present invention, and the processor 102 runs the software program and the module stored in the memory 104, thereby The various function applications and data processing are performed, that is, the data transmission method of the mobile terminal described above is implemented.
  • Memory 104 may include high speed random access memory, and may also include non-volatile memory such as one or more magnetic storage devices, flash memory, or other non-volatile solid state memory.
  • memory 104 may further include memory remotely located relative to processor 102, which may be connected to computer terminal 1 via a network. Examples of such networks include, but are not limited to, the Internet, intranets, local area networks, mobile communication networks, and combinations thereof.
  • Transmission device 106 is for receiving or transmitting data via a network.
  • the above specific network example may include a wireless network provided by a communication provider of the computer terminal 1.
  • the transmission device 106 includes a Network Interface Controller (NIC) that can be connected to other network devices through a base station to communicate with the Internet.
  • the transmission device 106 can be a Radio Frequency (RF) module for communicating with the Internet wirelessly.
  • NIC Network Interface Controller
  • RF Radio Frequency
  • the present application provides a data transmission method of the mobile terminal as shown in FIG. 2.
  • the method can be applied to a smart terminal device, and is executed by a processor in the smart terminal device, and the smart terminal device can be a smart phone, a tablet computer, or the like.
  • At least one application is installed in the smart terminal device.
  • the embodiment of the present invention does not limit the type of the application, and may be a system-based application or a software-based application.
  • the application program in the embodiment of the present invention includes at least one application function.
  • FIG. 2 is a flow chart of a data transmission method of a mobile terminal according to an embodiment of the present invention. As shown in FIG. 2, an optional solution of the data transmission method of the mobile terminal includes the following steps:
  • Step S201 Acquire cached data of the mobile application on the mobile terminal.
  • the mobile terminal may be a plurality of smart terminals such as a mobile phone and a tablet computer
  • the mobile application may be a mobile application software installed in the mobile terminal, for example, a mobile application software installed in an Android mobile phone.
  • the mobile application software installed in the Apple mobile phone the cached data is encoded and synthesized stream data.
  • the live-end cache is in the local memory of the app, and is the memory area encapsulated into the data saved by the network stream after the live-end App is encoded.
  • UPC User Generated Content
  • users collect image and sound data through a mobile phone camera device and a microphone device, and share the collected image and sound data in real time through a WeChat application or a QQ application to other mobiles.
  • the mobile terminal where the application is located other network users perform real-time playback viewing through the mobile terminal, for example, real-time playback view on mobile terminals such as tablet computers and mobile terminals Look.
  • the underlying program of the mobile application is transparent to the user, and is all intelligent and automated. It does not have the user interface (User Interface, UI for short).
  • the mobile application caches data on the mobile terminal.
  • the reading speed is smaller than the writing speed of the cache data on the mobile terminal, which affects the smoothness of audio and video playback.
  • Step S202 obtaining a cache ratio of the cached data.
  • the cache ratio is a ratio of the cache real-time valid data to the total allocated cache size, and the cache ratio reflects the speed at which the real-time encoded stream data is written into the memory and the cached data is read from the memory.
  • a measure of the speed to the server For example, when the cache data is read at the same speed as the cache data is written, the amount of cached data is 0, and the recorded content is in the best real-time playback state.
  • the cache ratio of the cache data may be obtained by various methods.
  • the embodiment of the present invention may include other readings indicating the cache data. The relationship between the speed and the write speed of the cache data is not illustrated here.
  • Step S203 determining whether the cache occupancy ratio exceeds a preset cache ratio threshold.
  • the preset cache ratio threshold may be a plurality of critical points, and the plurality of threshold points respectively correspond to different network environments. In different network environments, the data amount of the audio and video uploaded per second is different, that is, the code The rate is different, where the bit rate Bitrate, in kbps, represents the amount of audio and video data per second.
  • the preset cache ratio threshold may be 5%, 10%, 30%, 50%, 80%, and the like, and a plurality of preset preset cache ratio values respectively correspond to the optimal network state to the poor network state.
  • Step S204 if it is determined that the cache occupancy ratio exceeds the preset cache ratio threshold, the code rate of the cached data is adjusted to obtain an adjusted code rate.
  • the recorded audio and video is collected and encoded, and the higher the code rate of the obtained cached data, the smaller the compression of the cached data.
  • the code rate For video images, the larger the code rate, the closer the video image is to the original video image, and the higher the resolution of the video image after decoding.
  • the higher the code rate the closer the audio is to the sound quality of the original audio, and the better the quality of the audio after decoding.
  • the live network upload requires the code rate to be as low as possible, and the storage local requires the code rate to be as high as possible, and the corresponding quality is as good as possible.
  • the code rate of the cached data is adjusted according to the preset cache ratio, so as to adjust the quality of the audio and video data according to the network environment, and to ensure the smoothness of the recorded content playback.
  • Adjusting the code rate of the cache data according to the preset cache ratio threshold may be that the higher the preset cache ratio is, the lower the code rate of the adjusted cache data is.
  • the code rate is the target code rate.
  • the code rate is the target bit rate.
  • the preset buffer critical point is 30% to 80% of the segmentation value, the code The target rate is ⁇ 50%.
  • the preset cache threshold is 80% ⁇ 100%, the black-screen pure audio is replaced by the black-screen pure audio.
  • the interval of the segmentation value reflects the response speed of the switching trigger dynamic adjustment code rate. The tighter the response, the faster the scheduling of the code rate.
  • Step S205 in the networked state, the buffered data is transmitted to the receiving terminal according to the adjusted code rate.
  • the buffered data is transmitted to the receiving terminal according to the adjusted code rate, and the receiving terminal is configured to play the live content corresponding to the cached data according to the cached data.
  • the network is adjusted by a preset adjustment algorithm.
  • the UGC live broadcast end receives a prompt indicating that the quality of the response is degraded, prompting the user to leave the weak network environment as soon as possible or replace the better network environment to restore the original live broadcast.
  • the quality of the definition each time the cache ratio exceeds the threshold of the preset cache ratio, the code rate corresponding to the network environment is matched and the user is prompted, and the adaptation of the code rate and the prompt are one-to-one correspondence, thereby Improve the flexibility of recording and encoding uploading of audio and video.
  • the solution provided by the present invention can obtain the cached data of the cached data in the networked state by obtaining the cached data of the mobile application on the mobile terminal, and determine whether the cache ratio exceeds the preset cache ratio.
  • the critical point if it is determined that the cache ratio exceeds the preset cache percentage threshold, the code rate of the cached data is adjusted to obtain an adjusted code rate; and the buffered data is transmitted to the receiving terminal according to the adjusted code rate, thereby improving the acquisition and acquisition of the audio and video collection and encoding.
  • the flexibility further solves the technical problem of low flexibility in the acquisition and encoding of audio and video in related art.
  • the cached data is mainly stream data synthesized by audio and video compressed data, but currently the video compressed data is the subject with the largest amount of data
  • the cached data obtained by the mobile application on the mobile terminal may be: acquiring mobile The video compression frame sequence of the buffered data is applied, and the code rate of the buffered data is adjusted, and the adjusted code rate is obtained by adjusting the quantization parameter of each frame of the video compression frame sequence, obtaining the adjusted quantization parameter, and adjusting the code rate according to the adjusted quantization parameter, Adjust the code rate.
  • step 204 shown in FIG. 2 may further include the steps shown in FIG. 3:
  • Step S301 adjusting quantization parameters of each frame of the video compression frame sequence to obtain an adjustment quantization parameter.
  • obtaining the cached data of the mobile application on the mobile terminal may be a video compression frame sequence for acquiring cached data of the mobile application, and the quantization parameter QP is a standard for controlling the code rate of the cached data. That is, the code rate is adjusted by the quantization parameter to achieve the purpose of adapting the code rate and the network environment.
  • the quantization parameter of each frame of the video compression frame sequence is adjusted to obtain an adjustment quantization parameter.
  • Step S302 adjusting the code rate according to the adjustment quantization parameter to obtain an adjustment code rate.
  • step S302 of the present application after adjusting the quantization parameter of each frame of the video compression frame sequence to obtain the adjustment quantization parameter, and then adjusting the quantization parameter of each frame to each frame of the video compression frame sequence
  • the code rate is adjusted to obtain an adjustment code rate.
  • the buffered data is then transmitted to the receiving terminal according to the adjusted code rate.
  • determining, by step S203, whether the cache ratio exceeds a preset cache ratio threshold includes determining whether the cache ratio exceeds a first preset cache ratio threshold, the first preset cache.
  • the ratio of the critical point can achieve the real-time guarantee of the playback speed of the live content, the network status is good, and the upload of the cached data is stable.
  • the first preset cache ratio threshold is 0%. If it is determined that the cache occupancy ratio does not exceed the preset cache ratio threshold, the code rate of the cached data is maintained, and the cached data is dynamic image sequence data, and the intraframe coding and the unidirectional reference coding of the dynamic image sequence data are different. The receiving terminal can smoothly play the recorded content.
  • the intra-frame coding of the dynamic image sequence data is an I frame, and the I frame is the first frame at the beginning of the GOP, and the compression ratio is low, but the subsequent P-frame and B-frame are provided with reference coding; the unidirectional reference coding is a P-frame.
  • the compression ratio of the I frame is higher than that of the B frame, and the compression ratio of the B frame is lower; the B frame is Bidirectional reference compression, one more direction reference than P frames, with the highest compression ratio.
  • the GOP of this embodiment refers to a sequence of frames in which compressed video encoded frames are packed into a group, and all image compressed frames between two key frames constitute one GOP.
  • the key frame is the first frame data of the video compression coding sequence group, and is used for playing and decoding the subsequent compressed frame, which serves as a reference function. If there is no key frame, the playback end decoder cannot decode and restore the subsequent image normally, which may result in decoding the image data. Abnormal, showing the flower screen, and the key frame also plays a role in dragging and positioning the video segment at a certain point in time.
  • determining whether the cache cache ratio exceeds the preset cache ratio threshold includes determining whether the cache ratio exceeds the second value.
  • the default cache is the critical point. If the cache data is read from the memory faster than the cache data is written to the memory, the network status is poor, and the network cannot upload the cache data according to the code rate corresponding to the first preset cache ratio. The cache ratio is increased. If the value is high, it is determined whether the cache ratio exceeds the second preset cache ratio threshold.
  • the code rate of the cached data is adjusted to obtain an adjusted code rate, wherein the cached data is dynamic image sequence data, wherein the intra-frame coding of the dynamic image sequence data is The one-way reference code is different.
  • the resolution is reduced for the video and belongs to the range that can be received.
  • whether the judgment cache ratio exceeds the preset cache ratio threshold further includes: determining whether the cache ratio exceeds the third preset cache ratio critical point. If the cache data is read from the memory faster than the cache data is written to the memory, the network status is poor, and the network cannot upload the valid dynamic image conforming code rate according to the code rate corresponding to the second preset cache ratio. If the cache ratio continues to increase, it is determined whether the cache ratio exceeds a third preset cache ratio threshold, and the third preset cache ratio threshold may be a cache ratio corresponding to the black screen pure audio adjustment.
  • the collected original image data is replaced with the still YUV data, and the code rate corresponding to the static YUV data is obtained, wherein the original image data is the original captured image data, and the replacement is performed.
  • the subsequent YUV data needs to be sent to the encoder and the actual code rate data is output after adjusting a series of operations such as QP quantized values, that is, the code rate corresponding to the YUV data is to transmit the YUV image data to the encoder, and the code rate is adjusted.
  • the actual code rate data is output, and the highest intensity compression is performed, and the content of the YUV data is covered by the frame rate according to the input original moving image sequence, wherein the static YUV data is the static image sequence data, which can be a single
  • the frame picture is pure audio, that is, the video picture is fixed with a still picture, the non-dynamic frame sequence, the audio is reserved, and the information volume and code rate of the entire video stream are very low, which is suitable for transmission in a harsh network environment.
  • the width of the still YUV image is consistent with the original sequence frame, and is replaced by a static, single, highly redundant black screen with a prompt copy, the prompt copy is used for the information of the user of the playback end, and the frame of the static image sequence data is
  • the encoding is the same as the one-way reference encoding.
  • YUV is a color coding method adopted by the European television system and belongs to the Pal system (PAL) standard, in which "Y” represents brightness (Luminance or Luma), that is, gray scale value; “U” and “V” represent chrominance (Chrominance or Chroma), which is used to describe the color and saturation of the image and is used to specify the color of the pixel. "Brightness” is established by the RGB input signal and is the color space used by the PAL analog color TV system. In modern color television systems, a three-tube color camera or a color CCD camera is usually used for image taking, and then the acquired color image is taken.
  • PAL Pal system
  • RGB luminance signal
  • B-Y ie, U
  • R-Y ie, V
  • the transmitting end will have brightness and color difference three.
  • the signals are separately encoded and sent out on the same channel. This color representation is the YUV color space representation.
  • transmitting the buffer data to the receiving terminal according to the adjusted code rate in step S205 of the present application includes: transmitting a frame sequence of the still YUV data to the receiving terminal according to the code rate corresponding to the still YUV data, wherein the receiving terminal plays the stationary YUV.
  • the image corresponding to the data and the prompt information corresponding to the network state of the third preset cache ratio threshold point for example, the message “The current network state is very poor, please replace the network environment”.
  • the quantization parameter of the frame sequence of the YUV data is adjusted, and the compression rate of the buffered data is high, so that the buffered data is uploaded with a lower code rate encoding, thereby maximally matching the current network state.
  • the transmitting the buffer data to the receiving terminal according to the adjusted code rate includes: transmitting the audio data to the receiving terminal, and receiving the audio corresponding to the playing audio data of the terminal.
  • the image corresponding to the YUV data is a black screen, that is, the image corresponding to the YUV data is all black pixels, and the amount of the audio data is small, after determining that the cache ratio exceeds the third preset cache percentage threshold, Extracting audio data from the cached data, transmitting the audio data to the receiving terminal, and receiving the audio corresponding to the audio data of the terminal, thereby replacing the recorded content with black screen pure audio in a poor network state, thereby improving the compression ratio and further reducing
  • the bit rate of the audio and video data amount is matched with the current bandwidth, the cost is reduced, and the live broadcast of the recorded content is realized.
  • the black screen pure audio of this embodiment may be a fixed still image, and the background of the prompt image is almost all black pixels, and only a few prompt texts are provided for the user of the playing end to understand the current live broadcast situation.
  • the background is a sequence of non-moving images, only the same picture. In this way, the video compression rate can reach the limit ten-digit kbps code rate without distortion, the image frame compression rate will become very high, the code rate can be reduced to a minimum, and the dynamic video sequence needs to be reversed when the change is severe.
  • the larger code rate is used to maintain the original information amount without being lost, and the amount of data that reduces the code rate of a part of the encoded data frame is discarded when the network jitter is not able to meet the current code rate requirement, and the compressed content of the original image is lost.
  • the frame rate becomes lower, the fluency is lowered, and the amount of content information is reduced.
  • FIG. 4 is a schematic diagram showing a relationship between a cache ratio and a code rate of a mobile terminal according to an embodiment of the present invention.
  • the code rate is not adjusted; when the cache ratio is 5% to 20% of the segmentation value, the code rate is adjusted.
  • the code rate is adjusted to 90% of the target code rate; when the cache ratio is 20% to 30% of the segmentation value, the code rate is adjusted to 70% of the target code rate; when the cache ratio is in the segmentation value
  • 30% ⁇ 80% the code rate is adjusted to 50% of the target code rate; when the cache ratio is 80% ⁇ 100% of the segmentation value, the audio data is transmitted to the receiving terminal and played in black screen pure audio.
  • the cache data upload speed becomes slower, the unread cache data increases, and the mobile application increases the cache data upload cache ratio on the mobile terminal, so the code rate is adjusted to the target code respectively.
  • the rate is 90%, 70%, 50%.
  • the cache ratio is too high, the cached data is transmitted in black screen pure audio.
  • the network condition gradually becomes better, as the cache ratio decreases, the cache data upload becomes faster, the unread cache data decreases, and the code rate is gradually increased to the target code rate, thereby improving the quality of the audio and video data.
  • the storage of the cached data in the embodiment of the present invention may be used to store the encapsulated data of the real-time audio and video of the streaming media.
  • the cached data or other cached data that is only output by the video encoding may be specified.
  • the proportion of the segmentation value and the ratio of the dynamic target source code rate can be dynamically configured according to the actual situation. Only some examples are listed in this embodiment, and those skilled in the art should understand that this is not a kind of the present invention. limit. For example, more preset cache ratios can be set as needed, so there are more different segmentation values of the cache ratio, wherein the segmentation value ratio interval reflects the response speed of the handover trigger dynamic adjustment code rate. The tighter the response, the faster the scheduling and the more frequent it is.
  • FIG. 5 is a schematic diagram of a live broadcast user's mobile phone upload cache change and a terminal user play situation as a network scenario changes according to an embodiment of the present invention.
  • the A live network situation is good, and the mobile phone broadcast live user is an end user, the mobile phone.
  • the upload cache of the live broadcast user has a ratio of 0, and the write and read are synchronized in real time.
  • the target code rate is the original code rate, and no adjustment is needed.
  • the screen displayed by the terminal is smooth and stable.
  • the situation of the B-site network is worse.
  • the upload cache of mobile live users accounts for 10%.
  • the upload read cache speed is slower than the real-time code write speed.
  • the live broadcast rate is adjusted to 70% of the target bit rate.
  • the playback terminal can Play normally, but the clarity has dropped. C site network situation is very poor, the mobile phone user's upload cache accounted for 80%, upload and read cache speed is far slower than real-time code write speed, live broadcast needs to switch static frame pure audio ultra low code rate, the playback terminal can be normal Play, but only the prompt screen and audio, the actual live screen is switched instead.
  • FIG. 6A is a schematic diagram of video I frame and P frame transmission according to an embodiment of the present invention.
  • a video sequence with a video resolution of 1280 ⁇ 720 and a frame number of 25 fps per second is taken as an example.
  • the images of the input frame corresponding to the I frame and the P frame of the dynamic image sequence are different, and the frame of the video compression frame is stable for one second at the time A, and the upload is stable, and the cache ratio is 0%. , can achieve real-time guaranteed live stream playback speed.
  • FIG. 6B is a schematic diagram of video I frame and P frame transmission according to an embodiment of the present invention.
  • the B time code rate is 600 kbps
  • the I frame and The P frame corresponds to the input image
  • the video compression frame sequence is one second in the B time.
  • the network upload can not transmit the 1200 kbps code rate according to the normal speed, and the upload buffer ratio increases.
  • the rate adjustment threshold the code rate is adjusted, that is, the QP quantization parameter of each frame of the video coding is adjusted to achieve the purpose of adaptation, although the resolution may be reduced, but a better tone can be guaranteed.
  • the stability and smoothness of video playback, the quality of audio and video is a range that can be received.
  • FIG. 6C is a schematic diagram of video I frame and P frame transmission according to an embodiment of the present invention.
  • the C time code rate is 50 kbps
  • the average QP 55
  • the still image sequence the I frame and
  • the P frame corresponds to the input image.
  • the original moving image sequence covers the YUV data content according to the frame rate, wherein the size of the static YUV image is consistent with the original sequence frame, and is replaced by a static single, a highly redundant black screen with a prompt copy, and the prompt copy is used for
  • the image is made known to the user of the playing end, and then the QP adjustment is performed with the new frame sequence.
  • the frame sequence has a higher compression ratio and is output with the lowest code rate, thereby maximally matching the upload of the current network state.
  • the proportion of the upload cache is gradually reduced.
  • the threshold of the bit rate is again reversely triggered, the code rate is reversed to restore the effective image video.
  • Cache data to the receiving terminal solves the technical problem of low flexibility of recording and encoding uploading of audio and video in related technologies, ensures the live viewing experience of the terminal playing user, and prioritizes that the live content of high quality, key, and high attention is normal and stable.
  • the smooth transmission is presented to the user, and the real-time fidelity is maximized.
  • there is no more requirement for the operation and the server The low-coupling structure is no longer because the slow live broadcast stream is intermittently disconnected, and even the reconnection is disconnected and the background server program is frequently restarted. And trigger a series of uncontrollable risks brought by the playback side playback logic.
  • the embodiment of the present invention achieves the effect of decreasing the code rate by adjusting the code rate based on the QP operation adjustment and inserting the repeated frame with high single redundancy.
  • the current solution only uses the I frame with the low compression rate and P frame encoding output, but the highest compressed B frame has not been introduced, with the advancement of equipment and hardware and software and optimization technology, mobile devices can use B frame encoding to improve the compression rate and reduce the bit rate. Improve video quality with the same code rate. Since the mobile device is limited by the firmware and performance of the machine, the software encoding quality compression rate is good, but the central processing unit (CPU) is greatly consumed to cause the device to generate heat and power consumption, and at the same time limit the output resolution, resulting in clarity.
  • CPU central processing unit
  • encoding can only output 15fps per second transmission frame only contains I frame and P frame, and can not use B frame high compression coding, In this way, it is more difficult to output the best video quality under the premise of ensuring a smooth experience under a weak network or an unstable network.
  • hardware coding is a kind of audio and video compression processing logic unit of mobile phone chip.
  • mobile phone equipment has H.264 video hardware coding logic unit, which is used for video compression coding, which is different from software coding mode, and hardware coding does not occupy device CPU resources.
  • the hardware coding breaks through the bottleneck limitation of software coding on coding performance, but the hardware coding is not mature enough at present, the IOS formation is relatively good, the Android android array has many manufacturers, the standards are different, the quality is uneven, the complexity is high, and the algorithm is flexible. The degree is not comparable to the mature software coding.
  • hardware coding can fully achieve the same performance quality and high compression rate as software coding. This is real-time video coding and network transmission for mobile devices. Related applications provide protection.
  • the speed of the network environment has a profound impact on the field and technology. The further upgrade of the mobile operator network, the construction of the base station, and the stability of the access will greatly benefit the mobile video live broadcast application, and improve the acquisition and encoding of audio and video.
  • FIG. 7 is a schematic diagram of a data transmission apparatus of a mobile terminal according to an embodiment of the present invention. As shown in FIG. 7, the apparatus includes: a first acquisition unit 10, a second acquisition unit 20, a determination unit 30, an adjustment unit 40, and a transmission unit. 50.
  • the first obtaining unit 10 is configured to obtain cache data of the mobile application on the mobile terminal, where the cache data is encoded and synthesized stream data.
  • the second obtaining unit 20 is configured to obtain a cache ratio of the cached data, where the cache ratio is a ratio of the cache real-time valid data to the total allocated cache size, and the cache ratio reflects the speed and the time that the real-time encoded stream data is written into the memory. A measure of the speed at which cached data is sent to the server in memory.
  • the determining unit 30 is configured to determine whether the cache occupancy value exceeds a preset cache ratio threshold.
  • the adjusting unit 40 is configured to adjust a code rate of the cached data to obtain an adjusted code rate when it is determined that the cache occupancy ratio exceeds a preset cache ratio threshold.
  • the transmitting unit 50 is configured to transmit the buffered data to the receiving terminal according to the adjusted code rate in the networked state, where the receiving terminal is configured to play the live content corresponding to the cached data according to the cached data.
  • the first obtaining unit 10 may be configured to perform step S201 in the embodiment of the present invention
  • the second obtaining unit 20 may be configured to perform step S202 in the embodiment of the present invention
  • the determining unit 30 may be used to perform step S203 in the embodiment of the present invention
  • the adjusting unit 40 is configured to perform step S204 in the embodiment of the present invention
  • the transmitting unit 50 may be used to perform step S205 in the embodiment of the present invention, where For details, see the previous description of the data transmission method.
  • FIG. 8 is a schematic diagram of a data transmission apparatus of a mobile terminal according to another embodiment of the present invention.
  • the apparatus includes: a first acquisition unit 10, and a second acquisition unit 20 , the judging unit 30, the adjusting unit 40, and the transmitting unit 50.
  • the adjustment unit 40 includes a first adjustment module 401 and a second adjustment module 402.
  • the functions of the first obtaining unit 10, the second obtaining unit 20, the determining unit 30, the adjusting unit 40, and the transmitting unit 50 in this embodiment are the same as those in the data transmitting apparatus of the mobile terminal according to the embodiment of the present invention described with reference to FIG. the same.
  • the first adjustment module 401 is configured to adjust a quantization parameter of each frame of the video compression frame sequence to obtain an adjustment quantization parameter, where the quantization parameter is a criterion for controlling a code rate of the buffered data.
  • the second adjusting module 402 is configured to adjust the code rate according to the adjusted quantization parameter to obtain an adjusted code rate.
  • the first adjustment module 401 and the second adjustment module 402 are used to perform operations corresponding to steps 301 and 302 in FIG. 3, respectively, and are not described in detail herein for the sake of brevity.
  • the determining unit 30 is configured to determine whether the cache ratio exceeds a threshold of a first preset cache ratio, where the adjusting unit 40 is configured to: when determining that the cache ratio does not exceed a threshold of a preset cache ratio,
  • the code rate of the buffered data is cached data is dynamic image sequence data, and the intraframe coding of the moving image sequence data is different from the one-way reference coding.
  • the determining unit 30 determines that the cache occupancy ratio exceeds the first preset cache ratio threshold
  • the determining unit 30 is further configured to determine whether the cache ratio exceeds a second preset cache ratio threshold.
  • the adjusting unit 40 is configured to adjust the code rate of the buffered data to obtain an adjusted code rate, where the buffered data is dynamic image sequence data, and intraframe coding of the moving image sequence data is performed. It is different from one-way reference encoding.
  • the determining unit 30 determines that the cache occupancy ratio exceeds the second preset cache ratio threshold
  • the determining unit 30 is further configured to determine whether the cache ratio exceeds a third preset cache ratio threshold.
  • the adjusting unit 40 is configured to replace the collected original image data with the still YUV data to obtain a code rate corresponding to the still YUV data, and the YUV data corresponding to the YUV is YUV.
  • the image data is transmitted to the encoder, and the actual code rate data is output after a series of operations such as rate adjustment parameter quantization, wherein the still YUV data is static image sequence data, and the intra image coding of the static image sequence data is the same as the one-way reference code.
  • the transmission unit is configured to transmit a frame sequence of the stationary YUV data to the receiving terminal according to the code rate corresponding to the stationary YUV data, wherein the receiving terminal plays the image corresponding to the static YUV data and outputs a network corresponding to the third preset buffer ratio threshold point. Status message.
  • FIG. 9 is a schematic diagram of a data transmission apparatus of a mobile terminal according to another embodiment of the present invention.
  • the apparatus includes, in addition to the units shown in FIG. Also included is an extraction unit 60.
  • the extracting unit 60 is configured to extract audio data from the cached data after determining that the cache occupancy value exceeds a third preset cache ratio threshold, wherein the transmitting unit 50 is configured to transmit the audio data to the receiving terminal, and the receiving terminal Play the audio corresponding to the audio data.
  • Embodiments of the present invention also provide a storage medium.
  • the foregoing storage medium may be used to save the program generation executed by the data transmission method of the mobile terminal of the foregoing embodiment. code.
  • the foregoing storage medium may be located in at least one network device of the plurality of network devices of the computer network.
  • the foregoing storage medium may include, but is not limited to, a U disk, a read only memory ROM, a random access memory RAM, a mobile hard disk, a magnetic disk, or an optical disk, and the like, which can store program codes.
  • An embodiment of the present invention further provides a computer terminal, which may be any computer terminal device in a computer terminal group.
  • a computer terminal may also be replaced with a terminal device such as a mobile terminal.
  • the computer terminal may be located in at least one network device of the plurality of network devices of the computer network.
  • FIG. 10 is a structural block diagram of a computer terminal according to an embodiment of the present invention.
  • the computer terminal A may include one or more (only one shown in the figure) processor 101, memory 103, and transmission device 105.
  • the memory 103 can be used to store a software program and a module, such as a data transmission method of the mobile terminal and a program instruction/module corresponding to the device in the embodiment of the present invention.
  • the processor 101 runs the software program and the module stored in the memory 103. Thereby, various functional applications and data processing are performed, that is, the above-described data transmission method of the mobile terminal is implemented.
  • Memory 103 may include high speed random access memory, and may also include non-volatile memory such as one or more magnetic storage devices, flash memory, or other non-volatile solid state memory.
  • memory 103 can further include memory remotely located relative to processor 101, which can be connected to computer terminal A via a network. Examples of such networks include, but are not limited to, the Internet, intranets, local area networks, mobile communication networks, and combinations thereof.
  • the transmission device 105 described above is for receiving or transmitting data via a network.
  • Specific examples of the above network may include a wired network and a wireless network.
  • transmission device 105 includes a network adapter that can be connected to other network devices and routers via a network cable to communicate with the Internet or a local area network.
  • transmission device 105 is a radio frequency module that is used to communicate wirelessly with the Internet.
  • the memory 103 is configured to store preset action conditions and information of the preset rights user, and an application.
  • the processor 101 can call the information and the application stored in the memory 103 through the transmission device to Perform the following steps:
  • the cached data of the mobile application on the mobile terminal is obtained, wherein the cached data is encoded and synthesized stream data.
  • the second step is to obtain a cache ratio of the cached data, wherein the cache ratio is a ratio of the cache real-time valid data to the total allocated cache size, and the cache ratio reflects the speed at which the real-time encoded stream data is written into the memory and is read from the memory. A measure of the speed at which cached data is sent to the server.
  • the third step it is determined whether the cache ratio exceeds the preset cache percentage threshold.
  • the code rate of the cached data is adjusted to obtain an adjusted code rate.
  • the buffered data is transmitted to the receiving terminal according to the adjusted bit rate in the networked state, wherein the receiving terminal is used to play the live content corresponding to the cached data according to the cached data.
  • processor described herein can perform various steps as described in the foregoing data transmission method embodiments described with respect to Figures 2 and 3, and will not be described in detail herein for the sake of brevity.
  • the integrated unit in the above embodiment if implemented in the form of a software functional unit and sold or used as a stand-alone product, may be stored in the above-described computer readable storage medium.
  • the technical solution of the present invention may contribute to the prior art or all or part of the technical solution may be embodied in the form of a software product stored in a storage medium.
  • a number of instructions are included to cause one or more computer devices (which may be a personal computer, server or network device, etc.) to perform all or part of the steps of the methods described in various embodiments of the present invention.
  • the disclosed client may be implemented in other manners.
  • the device embodiments described above are merely illustrative.
  • the division of the unit is only a logical function division.
  • multiple units or components may be combined or may be Integrate into another system, or some features can be ignored or not executed.
  • the mutual coupling or direct coupling or communication connection shown or discussed may be an indirect coupling or communication connection through some interface, unit or module, and may be electrical or otherwise.
  • the unit described as a separate component may or may not be physically separated, and the component displayed as a unit may or may not be a physical unit, that is, may be located in one place, or It can also be distributed to multiple network elements. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of the embodiment.
  • each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.
  • the above integrated unit can be implemented in the form of hardware or in the form of a software functional unit.

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Databases & Information Systems (AREA)
  • Computer Networks & Wireless Communication (AREA)
  • General Engineering & Computer Science (AREA)
  • Two-Way Televisions, Distribution Of Moving Picture Or The Like (AREA)
  • Compression Or Coding Systems Of Tv Signals (AREA)

Abstract

本发明公开了一种移动终端的数据传输方法和装置。该移动终端的数据传输方法包括：获取移动应用在移动终端上的缓存数据，在联网状态下获取缓存数据的缓存占比值，判断缓存占比值是否超过预设缓存占比临界点；如果判断出缓存占比值超过预设缓存占比临界点，调整缓存数据的码率，得到调整码率；在联网状态下根据调整码率传输缓存数据至接收终端，其中，接收终端用于根据缓存数据对缓存数据对应的直播内容进行播放。本发明解决了相关技术中音视频的采集编码上传的灵活性低的技术问题。

Description

移动终端的数据传输方法和装置
本申请要求于2016年1月28日提交中国专利局、申请号为201610058950.9、发明名称为“移动终端的数据传输方法和装置”的中国专利申请的优先权，其全部内容通过引用结合在本申请中。
技术领域
本发明涉及互联网领域，具体而言，涉及一种移动终端的数据传输方法和装置。
背景技术
移动终端设备播放视频，比如，手机播放视频等，在第二代移动通讯网络(Second Generation，简称为2G)/第三代移动通讯网络(Third Generation，简称为3G)/第四代移动通讯网络(Fourth Generation，简称为4G)或无线保真(Wireless Fidelity，简称为WIFI)网络环境下，由于网络状态复杂多变，播放内容的稳定性低。在移动终端设备实时分享录制内容的应用场景下，移动终端设备的直播内容具备时效性，如果录制的内容吸引人，受用户终端的关注度高，首先需要确保播放内容不卡，从而为用户终端提供流畅、稳定的直播内容，进而提升用户的体验效果。在移动网络环境下，网络状态很大程度受地域性基站距离的制约，以及人群密集程度的影响，原本4G移动网络将会下降到3G移动网络，甚至下降到2G移动网络。这样，移动网络的传输速度下降将对直播内容的上传大大限制。同理，在WIFI无线网络环境下，WIFI热点覆盖不全，由于用户终端具备移动特性，在每个地方接收到的WIFI信号的强弱也会不同，也将影响移动终端的直播内容上传的稳定性。
目前，大多数移动应用实时采集编码上传的方案有三种：第一种，在开始对直播内容进行编码上传时优先匹配最低的清晰度和最低的码率，但这种编码上传的方法在直播过程中的播放质量不变，用户终端体验效果较差，在网络状态变好时没办法给出最优画质，网络状态变差也无法进行动态调整；第二种，在网络抖动变差无法满足当前码率要求的上传速度时，对视频采取丢弃一部分编码数据帧的手段变相降低码率数据量，满足直播上传，但这个方案缺点是丢失原本图像压缩内容，帧率变低，流畅度降低，内容信息量减少，而且需要同 步视频关键帧画面组(Group of Picture，简称为GOP)数据，有可能丢掉1～2秒画面数据，如果处理不好，丢掉的数据会引起播放端解码失败，图像异常，其次使观看端有跳变，导致不连续的体验效果，频繁丢帧处理会给用户非常不好的卡顿跳跃感。第三种，在对录制内容直播的过程中根据当前网络速度重置编码器及重置音视频码率，被动重启直播，导致直播被中断到重新初始化的过程，需要消耗时间，而这个时间段内影响播放终端的观看，出现卡顿缓冲、等待，如果直播搭建多层复杂链路，这使得恢复播放端的速度变得非常慢，这样流失用户量是在所难免，如果观看录制视频的用户量非常庞大，这样的中断和恢复对机房服务器的稳定性和带宽调整也是极大的挑战。
现有技术方案存在以下缺点：第一，整体质量通过低码率优先的控制，不能做到灵活随时随地适配，第二，即使采用动态控制方式，也过于简单粗暴，需要用较大的用户体验作为代价，第三，带来运营成本及机器调度的开销，增大了事故发生的概率，安全性、鲁棒性都欠缺。
针对相关技术中音视频的采集编码上传的灵活性低的问题，目前尚未提出有效的解决方案。
发明内容
本发明实施例提供了一种移动终端的数据传输方法和装置，以至少解决相关技术中音视频的采集编码上传的灵活性低的技术问题。
根据本发明实施例的一个方面，提供了一种移动终端的数据传输方法，该移动终端的数据传输方法包括：获取移动应用在移动终端上的缓存数据，其中，缓存数据为移动应用为编码合成的流数据；在联网状态下获取缓存数据的缓存占比值，其中，缓存占比值为缓存实时有效数据与总分配缓存大小的比值；判断缓存占比值是否超过预设缓存占比临界点；如果判断出缓存占比值超过预设缓存占比临界点，调整缓存数据的码率，得到调整码率；以及在联网状态下根据调整码率传输缓存数据至接收终端，以便于接收终端用于根据缓存数据对缓存数据对应的直播内容进行播放。
根据本发明实施例的另一方面，还提供了移动终端的数据传输装置，移动终端的数据传输装置包括：第一获取单元，用于获取移动应用在移动终端上的缓存数据，其中，缓存数据为编码合成的流数据；第二获取单元，用于在联网状态下获取缓存数据的缓存占比值，其中，缓存占比值为缓存实时有效数据与 总分配缓存大小的比值；判断单元，用于判断缓存占比值是否超过预设缓存占比临界点；调整单元，用于在判断出缓存占比值超过预设缓存占比临界点时，调整缓存数据的码率，得到调整码率；以及传输单元，用于在联网状态下根据调整码率传输缓存数据至接收终端，以便于接收终端根用于据缓存数据对缓存数据对应的直播内容进行播放。
根据本发明的另一方面，提供了一种计算机可读存储介质，在所述计算机可读存储介质上存储有用于执行所述的数据传输方法的程序指令。
在本发明实施例中，获取移动应用在移动终端上的缓存数据，在联网状态下获取所述缓存数据的缓存占比值，判断所述缓存占比值是否超过预设缓存占比临界点，如果判断出所述缓存占比值超过所述预设缓存占比临界点，调整所述缓存数据的码率，得到调整码率，以及根据所述调整码率传输所述缓存数据至接收终端，其中，所述接收终端根用于据所述缓存数据对所述缓存数据对应的直播内容进行播放，达到了根据缓存占比值调整缓存数据的码率的目的，从而实现了提高音视频采集编码上传的灵活性的技术效果，进而解决了相关技术中音视频的采集编码上传的灵活性低的技术问题。
附图说明
此处所说明的附图用来提供对本发明的进一步理解，构成本申请的一部分，本发明的示意性实施例及其说明用于解释本发明，并不构成对本发明的不当限定。在附图中：
图1是根据本发明实施例的一种移动终端的数据传输方法的计算机终端的硬件结构框图；
图2是根据本发明实施例的移动终端的数据传输方法的流程图；
图3是根据本发明实施例的图2的步骤204所包括的步骤的流程图；
图4是根据本发明实施例的移动终端的缓存占比值与码率关系的示意图；
图5是根据本发明实施例的随着网络场景变化直播用户手机上传缓存变化和终端用户播放情况的示意图；
图6A至图6C是根据本发明实施例的视频I帧、P帧传输的示意图；
图7是根据本发明实施例的移动终端的数据传输装置的示意图；
图8是根据本发明实施例的移动终端的数据传输装置的示意图；
图9是根据本发明实施例的移动终端的数据传输装置的示意图；以及
图10是根据本发明实施例的计算机终端的结构框图。
具体实施方式
为了使本技术领域的人员更好地理解本发明方案，下面将结合本发明实施例中的附图，对本发明实施例中的技术方案进行清楚、完整的描述，显然，所描述的实施例仅仅是本发明一部分的实施例，而不是全部的实施例。基于本发明中的实施例，本领域普通技术人员在没有做出创造性劳动前提下所获得的所有其他实施例，都应当属于本发明保护的范围。
需要说明的是，本发明的说明书和权利要求书及上述附图中的术语“第一”、“第二”等是用于区别类似的对象，而不必用于描述特定的顺序或先后次序。应该理解这样使用的数据在适当情况下可以互换，以便这里描述的本发明的实施例能够以除了在这里图示或描述的那些以外的顺序实施。此外，术语“包括”和“具有”以及他们的任何变形，意图在于覆盖不排他的包含，例如，包含了一系列步骤或单元的过程、方法、系统、产品或设备不必限于清楚地列出的那些步骤或单元，而是可包括没有清楚地列出的或对于这些过程、方法、产品或设备固有的其它步骤或单元。
首先，在对本发明实施例进行描述的过程中出现的部分名词或术语适用于如下解释：
码率：Bitrate，单位为kbps，每秒钟音视频数据量的大小，数值越大，压缩越小，视频图像解码后清晰度越高，音频声音解码后质量更好，一般越大越接近原始音视频图像画面和音质，直播网络上传则要求码率尽可能低，存储本地则要求码率尽可能高，对应的质量尽可能好。
单帧画面纯音频：视频画面固定一个静止的画面，非动态帧序列，音频保留，整个视频流的信息量和码率非常的低，适合恶劣网络环境下传输使用。
硬件编码：手机芯片的一种音视频压缩处理逻辑单元，一般手机设备具备H.264视频硬件编码逻辑单元，用于视频压缩编码，区别于软件编码方式，硬件编码不占用设备CPU资源。
上传缓存：直播流数据保存的内存区域，该缓存的写入来自实时音视频编码封装后的数据，读写输出是实时获取该数据用于上传发送，写入和读取速度同步，则缓存一直为0，直播处于实时最佳状态，如果网络上传变慢，缓存读取变慢，读取速度比写入速度低，缓存就会积压、占比就会增大，直播就失去实 时性，播放观看端就会出现滞留缓冲等待数据状态。
压缩编码：音视频原始的大数据量经过一系列压缩编码算法逻辑处理后生成压缩数据，方便传输与存储，播放器获取该压缩编码后数据后解码还原原始图像。
关键帧：视频压缩编码序列组的第一个帧数据，用于播放解码后续的压缩帧，起到参照作用，如果没有关键帧，播放端解码器无法正常解码还原后续图像，会导致解码图像数据异常，导致花屏现象，另外关键帧也起到拖拉定位视频段某一时间点作用。
GOP：全称为Group of picture，是指压缩视频编码帧打包成一组的帧序列，两个关键帧之间的所有图像压缩帧组成一个GOP。
QP：量化参数，用于码率控制的一个参考标准。
IPB帧：视频编码后输出的压缩帧类型，I帧是帧内编码，GOP开头的第一帧，压缩率低，但起到后续提供参考编码P和B帧作用；P帧是单向参考压缩帧，用于参考前面的帧估算运动矢量和残差数据进行压缩，而不实际对图像的重复像素数据压缩，较I帧压缩率高，较B帧压缩率低；B帧是双向参考压缩，于P帧而言多了一个方向参考，压缩率最高。
UGC：用户原创内容，全称为User Generated Content，用户将原创内容通过互联网平台运行展示或者提供给其他用户。
根据本发明实施例，提供了一种移动终端的数据传输方法实施例，需要说明的是，在附图的流程图示出的步骤可以在诸如一组计算机可执行指令的计算机系统中执行，并且，虽然在流程图中示出了逻辑顺序，但是在某些情况下，可以以不同于此处的顺序执行所示出或描述的步骤。
本申请实施例一所提供的方法实施例可以在移动终端、计算机终端或者类似的运算装置中执行。以运行在计算机终端上为例，图1是根据本发明实施例的一种移动终端的数据传输方法的计算机终端的硬件结构框图。如图1所示，计算机终端1可以包括一个或多个(图中仅示出一个)处理器102(处理器102可以包括但不限于微处理器MCU或可编程逻辑器件FPGA等的处理装置)、用于存储数据的存储器104、以及用于通信功能的传输模块106。本领域普通技术人员可以理解，图1所示的结构仅为示意，其并不对上述电子装置的结构造成限定。例如，计算机终端1还可包括比图1中所示更多或者更少的组件，或者具有与图1所示不同的配置。
存储器104可用于存储应用软件的软件程序以及模块，如本发明实施例中的移动终端的数据传输方法对应的程序指令/模块，处理器102通过运行存储在存储器104内的软件程序以及模块，从而执行各种功能应用以及数据处理，即实现上述的移动终端的数据传输方法。存储器104可包括高速随机存储器，还可包括非易失性存储器，如一个或者多个磁性存储装置、闪存、或者其他非易失性固态存储器。在一些实例中，存储器104可进一步包括相对于处理器102远程设置的存储器，这些远程存储器可以通过网络连接至计算机终端1。上述网络的实例包括但不限于互联网、企业内部网、局域网、移动通信网及其组合。
传输装置106用于经由一个网络接收或者发送数据。上述的网络具体实例可包括计算机终端1的通信供应商提供的无线网络。在一个实例中，传输装置106包括一个网络适配器(Network Interface Controller，简称为NIC)，其可通过基站与其他网络设备相连从而可与互联网进行通讯。在一个实例中，传输装置106可以为射频(Radio Frequency，简称为RF)模块，其用于通过无线方式与互联网进行通讯。
在上述运行环境下，本申请提供了如图2所示的移动终端的数据传输方法。该方法可以应用于智能终端设备中，由智能终端设备中的处理器执行，智能终端设备可以是智能手机、平板电脑等。智能终端设备中安装有至少一个应用程序，本发明实施例并不限定应用程序的种类，可以为系统类应用程序，也可以为软件类应用程序。本发明实施例中的应用程序包括至少一个应用功能。
图2是根据本发明实施例的移动终端的数据传输方法的流程图。如图2所示，该移动终端的数据传输方法的一种可选的方案包括如下步骤：
步骤S201，获取移动应用在移动终端上的缓存数据。
在本申请上述步骤S201提供的方案中，移动终端可以是手机，平板电脑等多种智能终端，移动应用可以是安装在移动终端中的移动应用软件，例如，安装在安卓手机中的移动应用软件或者安装在苹果手机中的移动应用软件，缓存数据为编码合成的流数据。直播端缓存是在App本地内存当中，是直播端App编码后封装成网络流所保存数据的内存区域。例如，用户原创内容(User Generated Content，简称为UGC)用户通过手机摄像头设备和麦克风设备采集图像和声音数据，实时将采集到的图像和声音数据通过微信应用程序或者QQ应用程序直播分享到其他移动应用所在的移动终端，其他网络用户通过移动终端进行实时播放观看，例如，在平板电脑、移动终端等移动终端进行实时播放观 看。移动应用的底层程序内部对用户透明，全是智能化自动化处理，不具备用户界面(User Interface，简称为UI)的交互操作，当用户进入弱网络环境时，移动应用在移动终端上的缓存数据的读取速度比移动终端上的缓存数据的写入速度小的越来越多，会影响音频视频播放的流畅度。
步骤S202，获取缓存数据的缓存占比值。
在本申请上述步骤S202提供的方案中，缓存占比值为缓存实时有效数据与总分配缓存大小的比值，缓存占比值反映了实时编码流数据写入内存的速度和从内存中读取缓存数据发送到服务器的速度的一个衡量标准。例如，当缓存数据读取的速度和缓存数据写入的速度相同时，则缓存数据量为0，录制的内容处于实时最佳的播放状态。
在该实施例中，获取缓存数据的缓存占比值可以通过多种方法，为了表示缓存数据的读取速度和缓存数据的写入速度的关系，本发明实施例可以包括其它表示缓存数据的读取速度和缓存数据的写入速度的关系，此处不再一一举例说明。
步骤S203，判断缓存占比值是否超过预设缓存占比临界点。
在本申请上述步骤S203提供的方案中，在联网状态下获取缓存数据的缓存占比值之后，判断获取到的缓存占比值是否超过预设缓存占比临界点。其中，预设缓存占比临界点可以是多个临界点，多个临界点分别对应不同的网络环境，在不同的网络环境下，每秒钟上传的音视频的数据量不同，也即，码率不同，其中，码率Bitrate，单位为kbps，表示每秒钟音视频数据量的大小。例如，预设缓存占比临界点可以为5％，10％，30％，50％，80％等多个预先设定的预设缓存占比值，分别对应于优网络状态到差网络状态。
步骤S204，如果判断出缓存占比值超过预设缓存占比临界点，调整缓存数据的码率，得到调整码率。
在本申请上述步骤S204提供的方案中，对录制的音视频进行采集编码，得到的缓存数据的码率越高，对缓存数据的压缩越小。对于视频图像，码率越大，视频图像越接近原始视频图像，视频图像解码后清晰度越高。对于音频，码率越大，音频越接近原始音频的音质，音频解码后质量更好。直播网络上传则要求码率尽可能低，存储本地则要求码率尽可能高，对应的质量尽可能好。同样，当缓存数据量越大时，在缓存数据写入内存和从内存中读取的过程中对网络环境的要求也越高，如果缓存数据的码率越高而网络环境不好，则会出现积压的 缓存数据越来越多，对缓存数据的写入内存的速度和从内存中读取的速度的缓存占比值逐渐增大，播放观看端出现数据滞留缓冲等待的状态，对录制内容的播放就失去了直播的实时性，进而影响用户体验。通过预先确定预设缓存占比临界点，根据预设缓存占比调整缓存数据的码率，以根据网络环境调整音频和视频数据的质量，最大限度地保证录制内容播放的流畅性。根据预设缓存占比临界点调整缓存数据的码率可以是在预设缓存占比值越高时，调整缓存数据的码率也越低。比如，当预设缓存占比值为0～5％的分段数值时，码率为目标码率，当预设缓存占比值为5％～20％的分段数值时，码率为目标码率×90％，当预设缓存占比值为20％～30％的分段数值时，码率为目标码率×70％，预设缓存临界点为30％～80％的分段数值时，码率为目标码率×50％，当预设缓存临界点为80％～100％的分段数值时，由黑屏纯音频代替，分段数值的区间反映了切换触发动态调整码率的响应速度，越紧密响应越快，对码率的调度也越快。
步骤S205，在联网状态下根据调整码率传输缓存数据至接收终端。
在调整缓存数据的码率，得到调整码率之后，根据调整码率将缓存数据传输至接收终端，接收终端用于根据缓存数据对缓存数据对应的直播内容进行播放。优选地，通过预设的调整算法适配网络情况，当录制内容上传变慢时，UGC直播端会收到响应质量下降的提示，提示用户尽快脱离弱网环境或更换较好网络环境恢复原本直播清晰度质量，每次在缓存占比值超过预设缓存占比临界点时，都会适配与网络环境相对应的码率和提示用户，对码率的适配与提示是一一对应的，从而提高了音视频的采集编码上传的灵活性。
通过上述步骤S201至步骤S205，本发明提供的方案可以实现通过获取移动应用在移动终端上的缓存数据，在联网状态下获取缓存数据的缓存占比值，判断缓存占比值是否超过预设缓存占比临界点；如果判断出缓存占比值超过预设缓存占比临界点，调整缓存数据的码率，得到调整码率；根据调整码率传输缓存数据至接收终端，提高了音视频的采集编码上传的灵活性，进而解决了相关技术中音视频的采集编码上传的灵活性低的技术问题。
作为一种可选的实施方式，缓存数据主要是音视频压缩数据所合成的流数据，但目前视频压缩数据为数据量最大的主体，获取移动应用在移动终端上的缓存数据可以是：获取移动应用的缓存数据的视频压缩帧序列，调整缓存数据的码率，得到调整码率包括：调整视频压缩帧序列的每一帧的量化参数，得到调整量化参数，根据调整量化参数调整码率，得到调整码率。
作为示例，图2中所示的步骤204可进一步包括如图3所示的步骤：
步骤S301，调整视频压缩帧序列的每一帧的量化参数，得到调整量化参数。
在本申请上述步骤S301提供的方案中，获取移动应用在移动终端上的缓存数据可以是获取移动应用的缓存数据的视频压缩帧序列，量化参数QP为用于控制缓存数据的码率的标准，也即，通过量化参数调整码率，达到对码率和网络环境进行适配的目的。在获取移动应用在移动终端上的缓存数据的视频压缩帧序列之后，调整视频压缩帧序列的每一帧的量化参数，得到调整量化参数。
步骤S302，根据调整量化参数调整码率，得到调整码率。
在本申请上述步骤S302提供的方案中，在调整视频压缩帧序列的每一帧的量化参数，得到调整量化参数之后，进而通过每一帧的调整量化参数对视频压缩帧序列的每一帧的码率进行调整，得到调整码率。其中，调整量化参数越高时，对数据的压缩率越高，码率越低，数据的失真率越高。然后根据调整码率传输缓存数据至接收终端。
作为一种可选的实施方式，本申请步骤S203判断缓存占比值是否超过预设缓存占比临界点包括：判断缓存占比值是否超过第一预设缓存占比临界点，该第一预设缓存占比临界点可以达到实时保证直播内容的播放速度，网络状态好，使缓存数据的上传稳定。优选地，该第一预设缓存占比临界点为0％。其中，如果判断出缓存占比值没有超过预设缓存占比临界点，保持缓存数据的码率，该缓存数据为动态图像序列数据，动态图像序列数据的帧内编码和单向参考编码不同，此时，接收终端可以流畅地播放录制内容。其中，动态图像序列数据的帧内编码为I帧，I帧是GOP开头的第一帧，压缩率低，但随后续的P帧和B帧提供参考编码的作用；单向参考编码为P帧，用于参考前面的帧估算运动矢量和残差数据进行压缩，但不实际对图像的重复像素数据进行压缩，较I帧的压缩率高些，较B帧的压缩率低些；B帧是双向参考压缩，比P帧多了一个方向的参考，压缩率最高。该实施例的GOP是指压缩视频编码帧打包成一组的帧序列，两个关键帧之间的所有图像压缩帧组成一个GOP。关键帧为视频压缩编码序列组的第一个帧数据，用于播放解码后续的压缩帧，起到参照作用，如果没有关键帧，播放端解码器无法正常解码还原后续图像，会导致解码图像数据异常，展现花屏，另外关键帧也起到拖拉定位视频段某一时间点作用。
作为示例，如果缓存占比值超过第一预设缓存占比临界点，则所述判断缓存占比值是否超过预设缓存占比临界点还包括：判断缓存占比值是否超过第二 预设缓存占比临界点。如果从内存中读取缓存数据的速度比向内存中写入缓存数据的速度慢，网络状态差，网络已经无法按照第一预设缓存占比值对应的码率正常上传缓存数据，缓存占比值升高，则判断缓存占比值是否超过第二预设缓存占比临界点。如果判断出缓存占比值超过第二预设缓存占比临界点，调整缓存数据的码率，得到调整码率，其中，缓存数据为动态图像序列数据，其中，动态图像序列数据的帧内编码和单向参考编码不同，此时，录制内容虽然可以正常播放，但是，对与视频而言，清晰度有所下降，属于可以接收的范围。
作为示例，如果缓存占比值超过第二预设缓存占比临界点，则所述判断缓存占比值是否超过预设缓存占比临界点还包括：判断缓存占比值是否超过第三预设缓存占比临界点。如果从内存中读取缓存数据的速度比向内存中写入缓存数据的速度慢，网络状态很差，网络已经无法按照第二预设缓存占比值对应的码率上传有效的动态图像符合码率的要求，缓存占比值继续升高，则判断缓存占比值是否超过第三预设缓存占比临界点，该第三预设缓存占比临界点可以为黑屏纯音频调整时对应的缓存占比值。如果判断出缓存占比值超过第三预设缓存占比临界点，将采集的原图像数据替换为静止YUV数据，得到静止YUV数据对应的码率，其中，原图像数据为原始采集图像数据，替换后的YUV数据需要发送给编码器以及通过调整QP量化值等一系列操作后输出实际的码率数据，也即，YUV数据对应的码率是将YUV图像数据传给编码器，经过码率调整、参数量化等一系列操作后输出实际的码率数据，进行最高强度压缩，对输入原动态图像序列进行按帧率覆盖YUV数据的内容，其中，静止YUV数据为静态图像序列数据，可以为单帧画面纯音频，也即，视频画面固定一个静止的画面，非动态帧序列，音频保留，整个视频流的信息量和码率非常的低，适合恶劣网络环境下传输使用。该静止YUV图像的宽度大小与原序列帧保持一致，从而替换为静止、单一、冗余度极高的黑屏带提示文案，提示文案用于播放端用户知晓的信息，静态图像序列数据的帧内编码和单向参考编码相同。
在该实施例中，YUV是被欧洲电视系统所采用的一种颜色编码方法，属于帕尔制(PAL)标准，其中“Y”表示明亮度(Luminance或Luma)，也就是灰阶值；而“U”和“V”表示的则是色度(Chrominance或Chroma)，作用是描述影像色彩及饱和度，用于指定像素的颜色。“亮度”是透过RGB输入信号来建立的，是PAL模拟彩色电视制式采用的颜色空间。在现代彩色电视系统中，通常采用三管彩色摄影机或彩色CCD摄影机进行取像，然后把取得的彩色图像 信号经分色、分别放大校正后得到RGB，再经过矩阵变换电路得到亮度信号Y和两个色差信号B－Y(即U)、R－Y(即V)，最后发送端将亮度和色差三个信号分别进行编码，用同一信道发送出去，这种色彩的表示方法就是YUV色彩空间表示。
在这种情况下，在本申请步骤S205中根据调整码率传输缓存数据至接收终端包括：根据静止YUV数据对应的码率传输静止YUV数据的帧序列至接收终端，其中，接收终端播放静止YUV数据对应的图像并输出对应于第三预设缓存占比临界点的网络状态的提示信息，比如，“当前网络状态很差，请更换网络环境”的提示信息。在该实施例中，对YUV数据的帧序列的量化参数进行调整，对缓存数据的压缩率较高，使缓存数据以较低码率的编码进行上传，从而最大程度地匹配当前的网络状态。
在一个可选的实施方式中，在判断出缓存占比值超过第三预设缓存占比临界点之后，此时网络情况很差，视频数据传输受到很大限制，而音频数据对网络速度的要求相对较小，为了保证数据传输的完整，从缓存数据中提取音频数据，其中，根据调整码率传输缓存数据至接收终端包括：将音频数据传输至接收终端，接收终端播放音频数据对应的音频。可选地，YUV数据对应的图像为黑屏，也即，YUV数据对应的图像全黑像素点，而音频数据量较小，在判断出缓存占比值超过第三预设缓存占比临界点之后，从缓存数据中提取音频数据，将音频数据传输至接收终端，接收终端播放音频数据对应的音频，从而在网络状态很差的情况下将录制内容以黑屏纯音频替代，提高了压缩率，进而降低了音视频数据量的码率，与当前的带宽相匹配，降低成本，实现了对录制内容的直播。
该实施例的黑屏纯音频可以是采用固定静止的提示图像，该提示图像的背景几乎全是黑色像素，只有少数提示文字供播放端用户了解当前的直播情况。该背景是非动态图像序列，只有同一个画面图像。这样视频压缩率就可以达到极限十位数kbps的码率大小而不觉得失真，图像帧压缩率就会变得非常高，码率就可以降到极限小，动态视频序列且变化剧烈时相反需要更大码率来维持原本的信息量而不被丢失，避免了在网络抖动变差无法满足当前码率要求时，丢弃一部分编码数据帧降低码率的数据量，导致原本图像的压缩内容丢失，帧率变低，流畅度降低，内容信息量减少的问题。
图4是根据本发明实施例的移动终端的缓存占比值与码率关系的示意图。 如图4所示，当上传数据的缓存占比值在0～5％时，不对码率进行调整；当缓存占比值在占比分段数值5％～20％时，对码率进行调整，将码率调整至目标码率的90％；当缓存占比值在占比分段数值20％～30％时，将码率调整至目标码率的70％；当缓存占比值在占比分段数值30％～80％时，将码率调整至目标码率的50％；当缓存占比值在占比分段数值80％～100％时，将音频数据传输至接收终端，以黑屏纯音频播放。
随着网络状态由好至差的转变，缓存数据上传速度变慢，未读取的缓存数据增多，移动应用在移动终端上的缓存数据上传缓存占比值增加，因此将码率分别调整至目标码率的90％，70％，50％，当缓存占比值过高时，以黑屏纯音频传送缓存数据。当网络情况逐渐变好，随着缓存占比值的减小，缓存数据上传变快，未读取的缓存数据减少，将码率逐渐增加至目标码率，提高了音视频数据的质量。
本发明实施例中的缓存数据的存放可以是指存放流媒体实时音视频的封装数据，在扩展延伸领域也可以指定为仅视频编码输出的缓存数据或其他缓存数据，在本发明实施例中，占比分段数值以及动态调整源目标码率的比率，可以根据实际情况进行动态后台配置，该实施例中只是列出了一些示例，本领域技术人员应该理解，这不作为对本发明的一种限制。例如，可以根据需要设置更多个预设缓存占比值，因此存在更多个不同的缓存占比值的分段数值，其中，分段数值占比区间反映了切换触发动态调整码率的响应速度，越紧密响应越快，调度也越频繁。
图5是根据本发明实施例的随着网络场景变化直播用户手机上传缓存变化和终端用户播放情况的示意图，如图5所示，A现场网络情况较好，手机直播用户为终端用户，该手机直播用户的上传缓存占比为0，写入与读取实时同步，目标码率为原码率，无需进行调整，终端播放用户所看到的画面流畅稳定。B现场网络情况变差，手机直播用户的上传缓存占比为10％，上传读取缓存速度比实时编码写入速度慢，直播的码率调整为目标码率的70％，播放终端的画面能够正常播放，但是清晰度有所下降。C现场网络情况非常差，手机直播用户的上传缓存占比为80％，上传读取缓存速度远远比实时编码写入速度慢，直播需要切换静帧纯音频超低码率，播放终端能够正常播放，但是只有提示画面和音频，实际现场画面被切换代替。
缓存数据主要是音视频压缩数据所合成的流数据，但目前视频压缩数据为 数据量最大的主体。图6A是根据本发明实施例的视频I帧、P帧传输的示意图，如图6A所示，在该实施例中，以视频分辨率为1280x720，每秒传输帧数25fps的视频序列为例，在码率为1200kbps，平均量化参数QP＝36时，该动态图像序列I帧和P帧对应输入的图像各异，在A时刻一秒钟视频压缩帧序列，上传稳定，缓存占比值为0％，可以达到实时保证直播流的播放速度。
图6B是根据本发明实施例的视频I帧、P帧传输的示意图，如图6B所示，在该实施例中，B时刻码率为600kbps，平均QP＝40，动态图像序列，I帧和P帧对应输入图像各异，B时刻内一秒钟视频压缩帧序列，由于此时网络状态变差，网络上传已无法按照正常速度传输1200kbps码率，上传缓存占比值升高，当升高的缓存占比值触发码率调整临界值时，调整码率，也即，对视频编码每一帧的QP量化参数进行调整，达到适配目的，虽然清晰度会有所下降但是可以保证更好的音视频播放的稳定性和流畅度，音视频的质量属于可以接收的范围。
图6C是根据本发明实施例的视频I帧、P帧传输的示意图，如图6C所示，在该实施例中，C时刻码率为50kbps，平均QP＝55，静态图像序列，I帧和P帧对应输入图像一样。C时刻内一秒钟视频压缩帧序列，由于当前的网络状态很差，网络上传已经无法正常传输有效动态图像序列符合的码率要求，上传缓存临界点已触发至黑屏纯音频调整策略，对输入原动态图像序列进行按帧率覆盖YUV数据内容，其中，该静止YUV图像宽高大小与原序列帧保持一致，替换为静止单一，冗余度极高的黑屏带提示文案，提示文案为用于使播放端用户知晓的图像，然后以此新的帧序列进行QP调整，该帧序列压缩率较高，以码率最低的编码输出，从而最大程度地匹配当前网络状态的上传。
当网络环境逐渐变好，上传缓存占比值逐渐降低，再次逆向触发提升码率临界点时，逆行调整码率，以恢复有效图像视频。
本发明实施例通过在联网状态下获取缓存数据的缓存占比值，当判断出缓存占比值超过预设缓存占比临界点时，调整缓存数据的码率，得到调整码率，根据调整码率传输缓存数据至接收终端，解决了相关技术中音视频的采集编码上传的灵活性低的技术问题，保证了终端播放用户的直播观看体验，优先保证优质、重点、关注度高的直播内容正常并稳定流畅地传输展现给用户，实时性保真性达到最大，另外对运营，服务器没有更多的要求，低耦合的结构不再因为上传缓慢直播流断断续续，甚至断开重连而发生频繁重启后台服务器程序及触发播放端播放逻辑所带来的一系列不可控的风险。
本发明实施例通过调整码率基于QP运算调整以及插入静止单一冗余度高的重复帧达到码率下降的效果，但由于硬件限制，目前的方案只使用了压缩率都不高的I帧和P帧编码输出，而最高压缩的B帧却没被引入，随着设备和软硬件及优化技术的进步，移动设备可以像电脑一样，无障碍使用B帧编码进一步提高压缩率降低码率，在同等码率不变情况下提升视频画质。由于移动设备受限于机器固件及性能，软件编码质量压缩率虽好但大大消耗中央处理器(Central Processing Unit，简称为CPU)引发设备发热及电量消耗，同时限制了其输出分辨率，导致清晰度及帧率流畅度都会大打折扣，以IOS设备iphone6为例，1280x720P分辨率视频序列，编码只能输出每秒传输帧数15fps仅包含I帧和P帧，且不能使用B帧高压缩编码，这样在弱网络或不稳定网络下最大限度保证流畅体验前提下输出最好的视频画质则较为困难。目前，硬件编码是手机芯片的一种音视频压缩处理逻辑单元，一般手机设备具备H.264视频硬件编码逻辑单元，用于视频压缩编码，区别于软件编码方式，硬件编码不占用设备CPU资源，该硬件编码突破了软件编码对编码性能的瓶颈限制，但硬件编码目前还不够成熟，IOS阵型相对较好，安卓android阵型厂商众多，标准不一，质量参差不齐，复杂度高的算法和灵活度都比不上成熟的软件编码，未来具有CPU的设备以及GPU芯片技术发展成熟后，硬件编码可以完全实现与软件编码一样的性能质量和高压缩率，这对移动设备视频实时编码及网络传输相关应用提供了保障。另外，网络环境的提速对该领域与技术具有很深远的影响，移动运营商网络的进一步升级，基站的建设，接入的稳定性将大大收益于移动视频直播应用，提高了音视频的采集编码上传的灵活性。
需要说明的是，对于前述的各方法实施例，为了简单描述，故将其都表述为一系列的动作组合，但是本领域技术人员应该知悉，本发明并不受所描述的动作顺序的限制，因为依据本发明，某些步骤可以采用其他顺序或者同时进行。其次，本领域技术人员也应该知悉，说明书中所描述的实施例均属于优选实施例，所涉及的动作和模块并不一定是本发明所必须的。
通过以上的实施方式的描述，本领域的技术人员可以清楚地了解到根据上述实施例的方法可借助软件加必需的通用硬件平台的方式来实现，当然也可以通过硬件，但很多情况下前者是更佳的实施方式。基于这样的理解，本发明的技术方案本质上或者说对现有技术做出贡献的部分可以以软件产品的形式体现出来，该计算机软件产品存储在一个存储介质，如只读存储器(Read-Only  Memory，简称为ROM)/随机存取存储器(Random Access Memory，简称为RAM)、磁碟、光盘中，包括若干指令用以使得一台终端设备(可以是手机，计算机，服务器，或者网络设备等)执行本发明各个实施例所述的方法。
根据本发明实施例，还提供了一种用于实施上述移动终端的数据传输方法的装置。图7是根据本发明实施例的移动终端的数据传输装置的示意图，如图7所示，该装置包括：第一获取单元10，第二获取单元20，判断单元30，调整单元40和传输单元50。
第一获取单元10，用于获取移动应用在移动终端上的缓存数据，其中，缓存数据为编码合成的流数据。
第二获取单元20，用于获取缓存数据的缓存占比值，其中，缓存占比值为缓存实时有效数据与总分配缓存大小的比值，缓存占比值反映了实时编码流数据写入内存的速度和从内存中读取缓存数据发送到服务器的速度的一个衡量标准。
判断单元30，用于判断缓存占比值是否超过预设缓存占比临界点。
调整单元40，用于在判断出缓存占比值超过预设缓存占比临界点时，调整缓存数据的码率，得到调整码率。
传输单元50，用于在联网状态下根据调整码率传输缓存数据至接收终端，其中，接收终端用于根据缓存数据对缓存数据对应的直播内容进行播放。
该实施例的移动终端的数据传输装置中，第一获取单元10可以用于执行本发明实施例中的步骤S201，第二获取单元20可以用于执行本发明实施例中的步骤S202，判断单元30可以用于执行本发明实施例中的步骤S203，调整单元40用于执行本发明实施例中的步骤S204，传输单元50可以用于执行本发明实施例中的步骤S205，其中，各个单元的具体操作可参见前面针对数据传输方法的描述
作为一种可选的实施例，图8是根据本发明另一实施例的移动终端的数据传输装置的示意图，如图8所示，该装置包括：第一获取单元10，第二获取单元20，判断单元30，调整单元40和传输单元50。其中，调整单元40包括第一调整模块401和第二调整模块402。
该实施例中的第一获取单元10，第二获取单元20，判断单元30，调整单元40和传输单元50的作用与参照图7描述的本发明实施例的移动终端的数据传输装置中的作用相同。
第一调整模块401，用于调整视频压缩帧序列的每一帧的量化参数，得到调整量化参数，其中，量化参数为用于控制缓存数据的码率的标准。
第二调整模块402，用于根据调整量化参数调整码率，得到调整码率。
这里的第一调整模块401和第二调整模块402用于执行分别与图3中的步骤301和302相应的操作，为了简便起见，这里不进行详细描述。
可选地，判断单元30用于判断缓存占比值是否超过第一预设缓存占比临界点，其中，调整单元40用于在判断出缓存占比值没有超过预设缓存占比临界点时，保持缓存数据的码率，缓存数据为动态图像序列数据，动态图像序列数据的帧内编码和单向参考编码不同。
可选地，在判断单元30判断出所述缓存占比值超过第一预设缓存占比临界点时，判断单元30还用于判断缓存占比值是否超过第二预设缓存占比临界点，在缓存占比值超过第二预设缓存占比临界点时，调整单元40用于调整缓存数据的码率，得到调整码率，其中，缓存数据为动态图像序列数据，动态图像序列数据的帧内编码和单向参考编码不同。
可选地，在判断单元30判断出所述缓存占比值超过第二预设缓存占比临界点时，判断单元30还用于判断缓存占比值是否超过第三预设缓存占比临界点，在缓存占比值超过第三预设缓存占比临界点时，调整单元40用于将采集的原图像数据替换为静止YUV数据，得到静止YUV数据对应的码率，YUV数据对应的码率是将YUV图像数据传给编码器，经过码率调整参数量化等一系列操作后输出实际的码率数据，其中，静止YUV数据为静态图像序列数据，静态图像序列数据的帧内编码和单向参考编码相同，传输单元用于根据静止YUV数据对应的码率传输静止YUV数据的帧序列至接收终端，其中，接收终端播放静止YUV数据对应的图像并输出对应于第三预设缓存占比临界点的网络状态的提示信息。
作为一种可选的实施例，图9是根据本发明另一实施例的移动终端的数据传输装置的示意图，如图9所示，该装置除了包括如图8中所示的各个单元之外，还包括提取单元60。所述提取单元60用于在判断出缓存占比值超过第三预设缓存占比临界点之后，从缓存数据中提取音频数据，其中，传输单元50用于将音频数据传输至接收终端，接收终端播放音频数据对应的音频。
本发明的实施例还提供了一种存储介质。可选地，在本实施例中，上述存储介质可以用于保存上述实施例的移动终端的数据传输方法所执行的程序代 码。
可选地，在本实施例中，上述存储介质可以位于计算机网络的多个网络设备中的至少一个网络设备。
可选地，在本实施例中，上述存储介质可以包括但不限于：U盘、只读存储器ROM、随机存取存储器RAM、移动硬盘、磁碟或者光盘等各种可以存储程序代码的介质。
本发明的实施例还提供一种计算机终端，该计算机终端可以是计算机终端群中的任意一个计算机终端设备。可选地，在本实施例中，上述计算机终端也可以替换为移动终端等终端设备。
可选地，在本实施例中，上述计算机终端可以位于计算机网络的多个网络设备中的至少一个网络设备。
可选地，图10是根据本发明实施例的一种计算机终端的结构框图。如图10所示，该计算机终端A可以包括：一个或多个(图中仅示出一个)处理器101、存储器103、以及传输装置105。
其中，存储器103可用于存储软件程序以及模块，如本发明实施例中的移动终端的数据传输方法和装置对应的程序指令/模块，处理器101通过运行存储在存储器103内的软件程序以及模块，从而执行各种功能应用以及数据处理，即实现上述的移动终端的数据传输方法。存储器103可包括高速随机存储器，还可以包括非易失性存储器，如一个或者多个磁性存储装置、闪存、或者其他非易失性固态存储器。在一些实例中，存储器103可进一步包括相对于处理器101远程设置的存储器，这些远程存储器可以通过网络连接至计算机终端A。上述网络的实例包括但不限于互联网、企业内部网、局域网、移动通信网及其组合。
上述的传输装置105用于经由一个网络接收或者发送数据。上述的网络具体实例可包括有线网络及无线网络。在一个实例中，传输装置105包括一个网络适配器，其可通过网线与其他网络设备与路由器相连从而可与互联网或局域网进行通讯。在一个实例中，传输装置105为射频模块，其用于通过无线方式与互联网进行通讯。
其中，具体地，存储器103用于存储预设动作条件和预设权限用户的信息、以及应用程序。
处理器101可以通过传输装置调用存储器103存储的信息及应用程序，以 执行下述步骤：
第一步，获取移动应用在移动终端上的缓存数据，其中，缓存数据为编码合成的流数据。
第二步，获取缓存数据的缓存占比值，其中，缓存占比值为缓存实时有效数据与总分配缓存大小的比值，缓存占比值反映了实时编码流数据写入内存的速度和从内存中读取缓存数据发送到服务器的速度的一个衡量标准。
第三步，判断缓存占比值是否超过预设缓存占比临界点。
第四步，如果判断出缓存占比值超过预设缓存占比临界点，调整缓存数据的码率，得到调整码率。
第五步，在联网状态下根据调整码率传输缓存数据至接收终端，其中，接收终端根用于据缓存数据对缓存数据对应的直播内容进行播放。
作为示例，这里所描述的处理器可执行如前面根据图2和图3所描述的数据传输方法实施例中所描述的各种步骤，为了简便起见，这里不再进行详细描述。
上述实施例中的集成的单元如果以软件功能单元的形式实现并作为独立的产品销售或使用时，可以存储在上述计算机可读取的存储介质中。基于这样的理解，本发明的技术方案本质上或者说对现有技术做出贡献的部分或者该技术方案的全部或部分可以以软件产品的形式体现出来，该计算机软件产品存储在存储介质中，包括若干指令用以使得一台或多台计算机设备(可为个人计算机、服务器或者网络设备等)执行本发明各个实施例所述方法的全部或部分步骤。
在本发明的上述实施例中，对各个实施例的描述都各有侧重，某个实施例中没有详述的部分，可以参见其他实施例的相关描述。
在本申请所提供的几个实施例中，应该理解到，所揭露的客户端，可通过其它的方式实现。其中，以上所描述的装置实施例仅仅是示意性的，例如所述单元的划分，仅仅为一种逻辑功能划分，实际实现时可以有另外的划分方式，例如多个单元或组件可以结合或者可以集成到另一个系统，或一些特征可以忽略，或不执行。另一点，所显示或讨论的相互之间的耦合或直接耦合或通信连接可以是通过一些接口，单元或模块的间接耦合或通信连接，可以是电性或其它的形式。
所述作为分离部件说明的单元可以是或者也可以不是物理上分开的，作为单元显示的部件可以是或者也可以不是物理单元，即可以位于一个地方，或者 也可以分布到多个网络单元上。可以根据实际的需要选择其中的部分或者全部单元来实现本实施例方案的目的。
另外，在本发明各个实施例中的各功能单元可以集成在一个处理单元中，也可以是各个单元单独物理存在，也可以两个或两个以上单元集成在一个单元中。上述集成的单元既可以采用硬件的形式实现，也可以采用软件功能单元的形式实现。
以上所述仅是本发明的优选实施方式，应当指出，对于本技术领域的普通技术人员来说，在不脱离本发明原理的前提下，还可以做出若干改进和润饰，这些改进和润饰也应视为本发明的保护范围。

Claims (13)

  1. 一种移动终端的数据传输方法，其特征在于，包括：
    获取移动应用在移动终端上的缓存数据，其中，所述缓存数据为编码合成的流数据；
    在联网状态下获取所述缓存数据的缓存占比值，其中，所述缓存占比值为缓存实时有效数据与总分配缓存大小的比值；
    判断所述缓存占比值是否超过预设缓存占比临界点；
    如果判断出所述缓存占比值超过所述预设缓存占比临界点，调整所述缓存数据的码率，得到调整码率；以及
    在联网状态下根据所述调整码率传输所述缓存数据至接收终端，以便于所述接收终端根据所述缓存数据对所述缓存数据对应的直播内容进行播放。
  2. 根据权利要求1所述的方法，其特征在于，
    所述缓存数据包括视频压缩帧序列，
    所述调整所述缓存数据的码率，得到调整码率包括：调整所述视频压缩帧序列的每一帧的量化参数，得到调整量化参数，其中，所述量化参数为用于控制所述缓存数据的码率的标准；根据所述调整量化参数调整所述码率，得到所述调整码率。
  3. 根据权利要求1或2所述的方法，其特征在于，
    所述判断所述缓存占比值是否超过预设缓存占比临界点包括：判断所述缓存占比值是否超过第一预设缓存占比临界点，
    其中，如果判断出所述缓存占比值没有超过所述第一预设缓存占比临界点，保持所述缓存数据的码率。
  4. 根据权利要求3所述的方法，其特征在于，
    如果判断出所述缓存占比值超过第一预设缓存占比临界点，则所述判断所述缓存占比值是否超过预设缓存占比临界点还包括：判断所述缓存占比值是否超过第二预设缓存占比临界点，所述如果判断出所述缓存占比值超过所述预设缓存占比临界点，调整所述缓存数据的码率，得到调整码率还包括：如果判断 出所述缓存占比值超过所述第二预设缓存占比临界点，则调整所述缓存数据的码率，得到所述调整码率。
  5. 根据权利要求4所述的方法，其特征在于，
    如果判断出所述缓存占比值超过第二预设缓存占比临界点，则所述判断所述缓存占比值是否超过预设缓存占比临界点还包括：判断所述缓存占比值是否超过第三预设缓存占比临界点，
    所述如果判断出所述缓存占比值超过所述预设缓存占比临界点，调整所述缓存数据的码率，得到调整码率还包括：如果判断出所述缓存占比值超过所述第三预设缓存占比临界点，将采集的原图像数据替换为静止YUV数据，得到所述静止YUV数据对应的码率，其中，所述静止YUV数据为静态图像序列数据，
    所述根据所述调整码率传输所述缓存数据至接收终端包括：根据所述静止YUV数据对应的码率传输所述静止YUV数据的帧序列至所述接收终端，其中，所述接收终端播放所述静止YUV数据对应的图像并输出对应于所述第三预设缓存占比临界点的网络状态的提示信息。
  6. 根据权利要求5所述的方法，其特征在于，
    在判断出所述缓存占比值超过所述第三预设缓存占比临界点之后，所述方法还包括：从所述缓存数据中提取音频数据，
    其中，根据所述调整码率传输所述缓存数据至接收终端还包括：将所述音频数据传输至所述接收终端，以便于所述接收终端播放所述音频数据对应的音频。
  7. 一种移动终端的数据传输装置，其特征在于，包括：
    第一获取单元，用于获取移动应用在所述移动终端上的缓存数据，其中，所述缓存数据为编码合成的流数据；
    第二获取单元，用于在联网状态下获取所述缓存数据的缓存占比值，其中，所述缓存占比值为缓存实时有效数据与总分配缓存大小的比值；
    判断单元，用于判断所述缓存占比值是否超过预设缓存占比临界点；
    调整单元，用于在判断出所述缓存占比值超过所述预设缓存占比临界点时，调整所述缓存数据的码率，得到调整码率；以及
    传输单元，用于在联网状态下根据所述调整码率传输所述缓存数据至接收终端，以便于所述接收终端根据所述缓存数据对所述缓存数据对应的直播内容进行播放。
  8. 根据权利要求7所述的装置，其特征在于，
    所述缓存数据包括视频压缩帧序列，
    所述调整单元包括：
    第一调整模块，用于调整所述视频压缩帧序列的每一帧的量化参数，得到调整量化参数，其中，所述量化参数为用于控制所述缓存数据的码率的标准；
    第二调整模块，用于根据所述调整量化参数调整所述码率，得到所述调整码率。
  9. 根据权利要求7或8所述的装置，其特征在于，
    所述判断单元进一步被用于判断所述缓存占比值是否超过第一预设缓存占比临界点，
    其中，在判断出所述缓存占比值没有超过所述第一预设缓存占比临界点时，所述调整单元保持所述缓存数据的码率，所述缓存数据为动态图像序列数据。
  10. 根据权利要求9所述的装置，其特征在于，
    在所述判断单元判断出所述缓存占比值超过第一预设缓存占比临界点时，所述判断单元还用于判断所述缓存占比值是否超过第二预设缓存占比临界点，
    在所述缓存占比值超过所述第二预设缓存占比临界点时，所述调整单元调整所述缓存数据的码率，得到所述调整码率，其中，所述缓存数据为动态图像序列数据。
  11. 根据权利要求10所述的装置，其特征在于，
    在所述判断单元判断出所述缓存占比值超过第二预设缓存占比临界点时，所述判断单元还用于判断所述缓存占比值是否超过第三预设缓存占比临界点，
    在所述缓存占比值超过所述第三预设缓存占比临界点时，所述调整单元将采集的原图像数据替换为静止YUV数据，得到所述静止YUV数据对应的码率， 其中，所述静止YUV数据为静态图像序列数据，
    所述传输单元用于根据所述静止YUV数据对应的码率传输所述静止YUV数据的帧序列至所述接收终端，其中，所述接收终端播放所述静止YUV数据对应的图像并输出对应于所述第三预设缓存占比临界点的网络状态的提示信息。
  12. 根据权利要求11所述的装置，其特征在于，所述装置还包括：
    提取单元，用于在所述缓存占比值超过所述第三预设缓存占比临界点之后，从所述缓存数据中提取音频数据，
    其中，所述传输单元用于将所述音频数据传输至所述接收终端，以便于所述接收终端播放所述音频数据对应的音频。
  13. 一种计算机可读存储介质，在所述计算机可读存储介质上存储有用于执行如权利要求1-6中的任一项所述的方法的程序指令。
PCT/CN2017/072195 2016-01-28 2017-01-23 移动终端的数据传输方法和装置 WO2017129090A1 (zh)

Priority Applications (1)

Application Number Priority Date Filing Date Title
US15/937,238 US10728594B2 (en) 2016-01-28 2018-03-27 Method and apparatus for transmitting data of mobile terminal

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN201610058950.9 2016-01-28
CN201610058950.9A CN105744342B (zh) 2016-01-28 2016-01-28 移动终端的数据传输方法和装置

Related Child Applications (1)

Application Number Title Priority Date Filing Date
US15/937,238 Continuation US10728594B2 (en) 2016-01-28 2018-03-27 Method and apparatus for transmitting data of mobile terminal

Publications (1)

Publication Number Publication Date
WO2017129090A1 true WO2017129090A1 (zh) 2017-08-03

Family

ID=56246869

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2017/072195 WO2017129090A1 (zh) 2016-01-28 2017-01-23 移动终端的数据传输方法和装置

Country Status (3)

Country Link
US (1) US10728594B2 (zh)
CN (1) CN105744342B (zh)
WO (1) WO2017129090A1 (zh)

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN107743275A (zh) * 2017-10-24 2018-02-27 国网冀北电力有限公司 一种视频调节方法及装置
CN109348307A (zh) * 2018-10-23 2019-02-15 安徽慧视金瞳科技有限公司 一种慧视云课堂教学系统音视频低延迟方法
CN112087646A (zh) * 2020-08-26 2020-12-15 腾讯科技（深圳）有限公司 视频播放方法、装置、计算机设备和存储介质

Families Citing this family (39)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US10306284B2 (en) * 2015-12-07 2019-05-28 Net Insight Intellectual Property Ab ABR adjustment for live OTT
CN105744342B (zh) * 2016-01-28 2019-04-12 腾讯科技（深圳）有限公司 移动终端的数据传输方法和装置
CN106303569A (zh) * 2016-08-18 2017-01-04 北京奇虎科技有限公司 一种直播视频流的传输方法、主播端设备
CN106254902B (zh) * 2016-08-19 2019-05-31 恒安嘉新（北京）科技股份公司 一种基于移动互联网视频用户感知和分析的方法及系统
CN106454395B (zh) * 2016-09-20 2018-10-12 北京百度网讯科技有限公司 在服务器中用于自适应提供多码率流媒体的方法及装置
CN106534979A (zh) * 2016-10-31 2017-03-22 深圳市金立通信设备有限公司 一种视频发送方法及终端
CN108259937B (zh) * 2016-12-29 2021-02-02 武汉斗鱼网络科技有限公司 一种直播网络信息的显示方法及系统
CN108271071B (zh) * 2016-12-30 2021-03-19 广州方硅信息技术有限公司 一种提高视频清晰度的方法及装置
CN108668146B (zh) * 2017-03-27 2021-07-16 华为技术有限公司 一种调整流媒体码率的方法及设备
CN107347065A (zh) * 2017-06-29 2017-11-14 上海鋆创信息技术有限公司 一种数据推送方法及装置、一种终端及存储介质
CN107613408B (zh) * 2017-09-21 2019-09-27 烽火通信科技股份有限公司 基于hls协议的媒体数据索引文件请求方法及系统
CN110535810A (zh) * 2018-05-25 2019-12-03 视联动力信息技术股份有限公司 一种视频数据的处理方法和终端
KR102080147B1 (ko) * 2018-06-20 2020-02-24 네이버 주식회사 적응형 데이터 전송을 위한 방법 및 시스템
CN108881972B (zh) * 2018-08-01 2021-06-15 广州视源电子科技股份有限公司 一种码率分配方法、装置、设备和存储介质
CN110972093B (zh) * 2018-09-28 2023-10-24 贵州白山云科技股份有限公司 一种移动办公实现方法及系统
CN111107382B (zh) * 2018-10-29 2022-05-06 阿里巴巴集团控股有限公司 数据流的控制方法、装置和系统
CN109634809A (zh) * 2018-12-05 2019-04-16 腾讯音乐娱乐科技（深圳）有限公司 一种音频播放监控方法及相关设备
CN109587510B (zh) * 2018-12-10 2021-11-02 广州虎牙科技有限公司 一种直播方法、装置、设备和存储介质
CN109714415B (zh) * 2018-12-26 2021-09-21 北京小米移动软件有限公司 数据处理方法及装置
CN109918038B (zh) * 2019-01-14 2022-08-12 珠海慧联科技有限公司 一种音频播放速度同步方法及系统
CN111510703B (zh) * 2019-01-31 2022-04-05 杭州海康威视数字技术股份有限公司 一种视频播放方法、装置及系统
CN110278495B (zh) * 2019-06-25 2020-02-07 重庆紫光华山智安科技有限公司 一种基于mpqm的视频传输网络控制方法及装置
CN110267100B (zh) * 2019-07-12 2022-01-25 北京达佳互联信息技术有限公司 Flv视频的码率切换方法、装置、电子设备及存储介质
CN111132112B (zh) * 2019-12-30 2024-08-20 深圳Tcl数字技术有限公司 蓝牙音频传输调整方法、智能电视及可读存储介质
CN111246278B (zh) * 2020-01-17 2023-02-24 北京达佳互联信息技术有限公司 视频播放方法、装置、电子设备及存储介质
CN113141524B (zh) * 2020-01-17 2023-09-29 北京达佳互联信息技术有限公司 资源传输方法、装置、终端及存储介质
CN111479161B (zh) * 2020-04-15 2022-04-12 广州虎牙科技有限公司 一种直播的质量数据上报方法和装置
CN111600614B (zh) * 2020-06-04 2023-09-05 北京润科通用技术有限公司 基于连续帧的3/4码率的编、译码方法、装置及系统
CN113099594A (zh) * 2020-08-05 2021-07-09 薛亮 结合边缘计算网关的物联网设备管理方法及大数据云平台
CN112511848B (zh) * 2020-11-09 2023-06-20 网宿科技股份有限公司 直播方法、服务端及计算机可读存储介质
CN114911550A (zh) * 2021-02-09 2022-08-16 阿里巴巴集团控股有限公司 云应用界面处理系统、方法、装置及设备
CN115240719B (zh) * 2021-04-23 2024-08-13 Oppo广东移动通信有限公司 缓存容量确定方法、装置、蓝牙播放设备及存储介质
CN113259730B (zh) * 2021-07-06 2021-12-14 北京达佳互联信息技术有限公司 用于直播的码率调整方法及装置
CN113645402B (zh) * 2021-07-27 2022-05-17 深圳市阿达视高新技术有限公司 摄像设备的比特率控制方法、系统、存储介质及摄像设备
CN114640886B (zh) * 2022-02-28 2023-09-15 深圳市宏电技术股份有限公司 自适应带宽的音视频传输方法、装置、计算机设备及介质
CN114896184B (zh) * 2022-05-24 2023-07-25 西安微电子技术研究所 一种dma控制器fpga及基于其的固态存储器
CN117319509A (zh) * 2022-06-24 2023-12-29 中兴通讯股份有限公司 网络数据处理方法、装置和计算机可读存储介质
CN115733981A (zh) * 2022-11-22 2023-03-03 天地伟业技术有限公司 一种码流控制方法、装置及电子设备
CN117579887B (zh) * 2023-11-23 2024-07-26 南月（广州）机器人科技有限公司 一种用于物联网竞技的终端智能控制系统

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2000046997A1 (en) * 1999-02-04 2000-08-10 General Instrument Corporation Video rate-buffer management scheme for mpeg transcoder
CN101252689A (zh) * 2008-02-29 2008-08-27 杭州爱威芯科技有限公司 一种自适应的码率控制方法
CN102378065A (zh) * 2011-10-19 2012-03-14 江西省南城县网信电子有限公司 基于mpeg-4的流媒体服务器端配置缓冲区的方法及系统
CN104486688A (zh) * 2014-12-31 2015-04-01 深圳市华宝电子科技有限公司 一种车载视频传输方法及装置
CN204929081U (zh) * 2015-08-26 2015-12-30 重庆西线科技有限公司 一种视频传输装置
CN105744342A (zh) * 2016-01-28 2016-07-06 腾讯科技（深圳）有限公司 移动终端的数据传输方法和装置

Family Cites Families (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US7016409B2 (en) * 2003-11-12 2006-03-21 Sony Corporation Apparatus and method for use in providing dynamic bit rate encoding
GB0421663D0 (en) * 2004-09-29 2004-10-27 Nokia Corp Transmitting data in a wireless network
US8218657B2 (en) * 2005-09-02 2012-07-10 Netgear, Inc. System and method for automatic adjustment of streaming video bit rate
US7987285B2 (en) * 2007-07-10 2011-07-26 Bytemobile, Inc. Adaptive bitrate management for streaming media over packet networks
CN102413307B (zh) * 2011-11-30 2013-11-27 晶门科技（深圳）有限公司 电视视频聊天的自适应码率调整方法及调整装置
CN103294782B (zh) * 2013-05-15 2018-02-16 小米科技有限责任公司 一种数据展现方法和装置
US9697124B2 (en) * 2015-01-13 2017-07-04 Qualcomm Incorporated Systems and methods for providing dynamic cache extension in a multi-cluster heterogeneous processor architecture

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2000046997A1 (en) * 1999-02-04 2000-08-10 General Instrument Corporation Video rate-buffer management scheme for mpeg transcoder
CN101252689A (zh) * 2008-02-29 2008-08-27 杭州爱威芯科技有限公司 一种自适应的码率控制方法
CN102378065A (zh) * 2011-10-19 2012-03-14 江西省南城县网信电子有限公司 基于mpeg-4的流媒体服务器端配置缓冲区的方法及系统
CN104486688A (zh) * 2014-12-31 2015-04-01 深圳市华宝电子科技有限公司 一种车载视频传输方法及装置
CN204929081U (zh) * 2015-08-26 2015-12-30 重庆西线科技有限公司 一种视频传输装置
CN105744342A (zh) * 2016-01-28 2016-07-06 腾讯科技（深圳）有限公司 移动终端的数据传输方法和装置

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN107743275A (zh) * 2017-10-24 2018-02-27 国网冀北电力有限公司 一种视频调节方法及装置
CN109348307A (zh) * 2018-10-23 2019-02-15 安徽慧视金瞳科技有限公司 一种慧视云课堂教学系统音视频低延迟方法
CN112087646A (zh) * 2020-08-26 2020-12-15 腾讯科技（深圳）有限公司 视频播放方法、装置、计算机设备和存储介质

Also Published As

Publication number Publication date
US10728594B2 (en) 2020-07-28
US20180220170A1 (en) 2018-08-02
CN105744342A (zh) 2016-07-06
CN105744342B (zh) 2019-04-12

Similar Documents

Publication Publication Date Title
WO2017129090A1 (zh) 移动终端的数据传输方法和装置
CN110662114B (zh) 视频处理方法、装置、电子设备及存储介质
JP3801984B2 (ja) 帯域幅スケーラを含むマルチキャスト伝送システム
US10091528B2 (en) Method and system for scalable video compression and transmission
EP2452481B1 (en) System and method of transmitting content from a mobile device to a wireless display
US20160234522A1 (en) Video Decoding
US20150373075A1 (en) Multiple network transport sessions to provide context adaptive video streaming
US20110026591A1 (en) System and method of compressing video content
US10171815B2 (en) Coding manner switching method, transmit end, and receive end
JP2006508574A (ja) 要求に応じたｉ画像の挿入
CN107566918A (zh) 一种视频分发场景下的低延时取流秒开方法
KR20140126372A (ko) 데이터, 멀티미디어 및 비디오 전송 갱신 시스템
CN106162257B (zh) 一种实时视频的自适应网络传输优化方法
JP5592716B2 (ja) 映像送信装置
JP2006197321A (ja) 画像処理方法および装置、並びにプログラム
CN101854520A (zh) 一种无线传输视频监控系统及方法
JP2009124518A (ja) 画像送信装置
CN110351576B (zh) 一种在工业场景下进行实时视频流快速显示的方法及其系统
US8095955B2 (en) Methods and systems for improving network response during channel change
WO2021056575A1 (zh) 一种低延迟信源信道联合编码方法及相关设备
CN116962613A (zh) 数据传输方法及装置、计算机设备、存储介质
JP4994732B2 (ja) デジタルテレビ放送波送出装置
GB2469281A (en) Asynchronous live video transmission based on client side requests for individual frames
CN102256119B (zh) 视频录制的方法及装置
CN101188768A (zh) 基于rgb编解码器发送和接收运动图像的方法和设备

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 17743705

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 17743705

Country of ref document: EP

Kind code of ref document: A1