CN106817614B - Audio and video frame loss device and method - Google Patents

Audio and video frame loss device and method Download PDF

Info

Publication number
CN106817614B
CN106817614B CN201710045496.8A CN201710045496A CN106817614B CN 106817614 B CN106817614 B CN 106817614B CN 201710045496 A CN201710045496 A CN 201710045496A CN 106817614 B CN106817614 B CN 106817614B
Authority
CN
China
Prior art keywords
frame rate
video
frame
time
frames
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN201710045496.8A
Other languages
Chinese (zh)
Other versions
CN106817614A (en
Inventor
马子平
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
ZHEJIANG RUIHUA KANGYUAN TECHNOLOGY Co.,Ltd.
Original Assignee
Zhejiang Ruihua Kangyuan Technology Co ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Zhejiang Ruihua Kangyuan Technology Co ltd filed Critical Zhejiang Ruihua Kangyuan Technology Co ltd
Priority to CN201710045496.8A priority Critical patent/CN106817614B/en
Publication of CN106817614A publication Critical patent/CN106817614A/en
Application granted granted Critical
Publication of CN106817614B publication Critical patent/CN106817614B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
    • H04N21/434—Disassembling of a multiplex stream, e.g. demultiplexing audio and video streams, extraction of additional data from a video stream; Remultiplexing of multiplex streams; Extraction or processing of SI; Disassembling of packetised elementary stream
    • H04N21/4342—Demultiplexing isochronously with video sync, e.g. according to bit-parallel or bit-serial interface formats, as SDI
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
    • H04N21/4302—Content synchronisation processes, e.g. decoder synchronisation
    • H04N21/4305—Synchronising client clock from received content stream, e.g. locking decoder clock with encoder clock, extraction of the PCR packets
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
    • H04N21/4302—Content synchronisation processes, e.g. decoder synchronisation
    • H04N21/4307—Synchronising the rendering of multiple content streams or additional data on devices, e.g. synchronisation of audio on a mobile phone with the video output on the TV screen
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
    • H04N21/44—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs
    • H04N21/4402—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs involving reformatting operations of video signals for household redistribution, storage or real-time display
    • H04N21/440281—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs involving reformatting operations of video signals for household redistribution, storage or real-time display by altering the temporal resolution, e.g. by frame skipping

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Two-Way Televisions, Distribution Of Moving Picture Or The Like (AREA)

Abstract

The invention discloses an audio and video frame loss device, which comprises: the acquisition module is used for acquiring the time difference between the audio frame time and the video frame time at the current playing time in real time or at regular time; the judging module is used for judging whether the video comprises a B frame or not if the absolute value of the time difference is greater than or equal to a preset time length, wherein the preset time length is a preset multiple of the display time length of a unit video frame; the determining module is used for determining the time interval for discarding the B frames in the video according to a first frame rate of a currently played video source file and a second frame rate of the currently played video if the video contains the B frames; and the first discarding module is used for discarding the B frames according to the time interval so as to enable the frame rate of the video source file to reach the frame rate of the current video playing. The invention also discloses an audio and video frame loss method. The invention can realize the output synchronization of the audio and video by discarding the B frame which has no influence on the video playing.

Description

Audio and video frame loss device and method
Technical Field
The invention relates to the field of video playing, in particular to an audio and video frame loss device and method.
Background
In the era of mobile internet, with the popularization of intelligent terminals, users continuously pursue high-quality audio-video experience, high compression ratio, high code rate, high resolution, multiple channels and lossless sound sources gradually become video standard fittings, and the requirements of users on the picture quality of multimedia videos are higher and higher.
When a video player of an intelligent terminal is developed, video sources of different file types and coding types and video sources of the same coding type, different levels and different configurations are generally required to be integrally tested so as to meet the diversity of audio and video sources of users. Due to the constraint of hardware performance, it is difficult to achieve the FPS (frame Per Second, video playing frame rate) required by the source file during video playing, so a frame skipping phenomenon occurs during playing, and the audio and video are not synchronous.
Disclosure of Invention
The invention mainly aims to provide an audio and video frame dropping device and method, and aims to solve the technical problem that in the prior art, a frame skipping phenomenon occurs during playing so that audio and video are not synchronous.
In order to achieve the above object, the present invention provides an audio/video frame loss device, comprising:
the acquisition module is used for acquiring the time difference between the audio frame time and the video frame time at the current playing time in real time or at regular time;
the judging module is used for judging whether the video comprises a B frame or not if the absolute value of the time difference is greater than or equal to a preset time length, wherein the preset time length is a preset multiple of the display time length of a unit video frame;
the determining module is used for determining the time interval for discarding the B frames in the video according to a first frame rate of a currently played video source file and a second frame rate of the currently played video if the video contains the B frames;
and the first discarding module is used for discarding the B frames according to the time interval so as to enable the frame rate of the video source file to reach the frame rate of the current video playing.
Optionally, the determining module includes:
the determining unit is used for determining a third frame rate supported by the terminal according to the performance of the terminal chip;
a judging unit, configured to judge whether the second frame rate is less than the third frame rate;
an adjusting unit, configured to increase a preset increment to the third frame rate to update the third frame rate when the second frame rate is greater than or equal to the third frame rate;
the determining unit is further configured to determine whether a sum of the second frame rate and a preset value is greater than the third frame rate when the second frame rate is less than the third frame rate;
a first calculating unit, configured to calculate an average value of the second frame rate and the third frame rate when a sum of the second frame rate and a preset value is less than or equal to the third frame rate, and calculate the time interval according to the average value and the first frame rate;
the adjusting unit is further configured to subtract the preset increment from the third frame rate to update the third frame rate when the sum of the second frame rate and a preset value is greater than the third frame rate.
Optionally, the apparatus further comprises:
and the playing module is used for playing the next I frame if the video does not have the B frame.
Optionally, the apparatus further comprises:
the comparison module is used for comparing the current system playing time with the actual playing time in real time;
the second discarding module is used for discarding the B frames in the video source file when the time difference between the current system playing time and the actual playing time is greater than the first preset time;
the playing module is further configured to play a next I frame when a time difference between the current system playing time and the actual playing time is greater than a second preset time, where the first preset time is less than the second preset time.
Optionally, the second discarding module includes:
the second calculation unit is used for calculating the number of the B frames needing to be discarded according to the second frame rate of the currently played video and the first frame rate of the currently played video source file;
and the discarding unit is used for discarding the corresponding B frames in the video source file according to the calculated number of the B frames needing to be discarded.
In addition, in order to achieve the above object, the present invention further provides an audio and video frame loss method, which comprises the following steps:
acquiring the time difference between the audio frame time and the video frame time at the current playing time in real time or at regular time;
if the absolute value of the time difference is greater than or equal to a preset time length, judging whether the video contains a B frame, wherein the preset time length is a preset multiple of the display time length of a unit video frame;
if so, determining the time interval for discarding the B frames in the video according to the first frame rate of the currently played video source file and the second frame rate of the currently played video;
and discarding the B frames according to the time interval so as to enable the frame rate of the video source file to reach the frame rate of the current video playing.
Optionally, the step of determining a time interval for discarding B frames in the video according to a first frame rate of a currently playing video source file and a second frame rate of a currently playing video includes:
determining a third frame rate supported by the terminal according to the performance of the terminal chip;
judging whether the second frame rate is smaller than the third frame rate;
when the second frame rate is greater than or equal to the third frame rate, increasing a preset increment to the third frame rate to update the third frame rate, and continuing to execute the step of judging whether the second frame rate is less than the third frame rate;
when the second frame rate is less than the third frame rate, judging whether the sum of the second frame rate and a preset value is greater than the third frame rate;
when the sum of the second frame rate and a preset value is less than or equal to the third frame rate, calculating the average value of the second frame rate and the third frame rate, and calculating the time interval according to the average value and the first frame rate;
and when the sum of the second frame rate and a preset value is greater than the third frame rate, subtracting the preset increment from the third frame rate to update the third frame rate, and continuing to execute the step of judging whether the second frame rate is less than the third frame rate.
Optionally, after the step of determining whether there is a B frame in the video, the method further includes:
and if the video has no B frame, playing the next I frame.
Optionally, the method further comprises:
comparing the current system playing time with the actual playing time in real time;
when the time difference between the current system playing time and the actual playing time is greater than a first preset time, discarding B frames in the video source file;
and when the time difference between the current system playing time and the actual playing time is greater than a second preset time, playing the next I frame, wherein the first preset time is less than the second preset time.
Optionally, the step of discarding B frames in the video source file comprises:
calculating the number of B frames needing to be discarded according to the second frame rate of the currently played video and the first frame rate of the currently played video source file;
and discarding the corresponding B frames in the video source file according to the calculated number of the B frames needing to be discarded.
The audio and video frame loss device and the method thereof provided by the invention acquire the time difference between the audio frame time and the video frame time at the current playing moment in real time or at regular time; when the absolute value of the time difference is greater than or equal to a preset time length, judging whether the video contains a B frame, wherein the preset time length is a preset multiple of the display time length of a unit video frame; when the video comprises B frames, determining the time interval for discarding the B frames in the video according to the first frame rate of the currently played video source file and the second frame rate of the currently played video, discarding the B frames according to the time interval, and enabling the frame rate of the video source file to reach the frame rate of the currently played video by discarding the B frames according to the time interval, thereby realizing the output synchronization of the audio and the video.
Drawings
Fig. 1 is a schematic diagram of an alternative hardware configuration of a mobile terminal implementing various embodiments of the present invention;
FIG. 2 is a diagram of a wireless communication device of the mobile terminal shown in FIG. 1;
fig. 3 is a functional module schematic diagram of a first embodiment of the audio-video frame loss device according to the present invention;
fig. 4 is a schematic diagram of a detailed functional module of a determination module in a third embodiment of the audio/video frame loss device according to the present invention;
fig. 5 is a functional module diagram of an audio-video frame loss device according to a fourth embodiment of the present invention;
fig. 6 is a schematic diagram of a detailed functional module of a second discarding module in a fifth embodiment of the audio/video frame loss device according to the present invention;
fig. 7 is a schematic flow chart of a first embodiment of an audio-video frame loss method according to the present invention;
fig. 8 is a schematic flowchart of a detailed process of the step of determining the time interval for discarding the B frame in the video according to the first frame rate of the currently played video source file and the second frame rate of the currently played video in the third embodiment of the audio/video frame dropping method according to the present invention;
fig. 9 is a schematic flow chart of a fourth embodiment of the audio-video frame loss method according to the present invention;
fig. 10 is a detailed flowchart of the step of discarding the B frame in the video source file in the fifth embodiment of the audio/video frame loss method according to the present invention.
The implementation, functional features and advantages of the objects of the present invention will be further explained with reference to the accompanying drawings.
Detailed Description
It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.
A mobile terminal implementing various embodiments of the present invention will now be described with reference to the accompanying drawings. In the following description, suffixes such as "module", "component", or "unit" used to denote elements are used only for facilitating the explanation of the present invention, and have no specific meaning in themselves. Thus, "module" and "component" may be used in a mixture.
The mobile terminal may be implemented in various forms. For example, the terminal described in the present invention may include a mobile terminal such as a mobile phone, a smart phone, a notebook computer, a digital broadcast receiver, a PDA (personal digital assistant), a PAD (tablet computer), a PMP (portable multimedia player), a navigation device, and the like, and a stationary terminal such as a digital TV, a desktop computer, and the like. In the following, it is assumed that the terminal is a mobile terminal. However, it will be understood by those skilled in the art that the configuration according to the embodiment of the present invention can be applied to a fixed type terminal in addition to elements particularly used for moving purposes.
Fig. 1 is a schematic hardware configuration of a mobile terminal implementing various embodiments of the present invention.
The mobile terminal 100 may include a wireless communication unit 110, an a/V (audio/video) input unit 120, a user input unit 130, a sensing unit 140, an output unit 150, a memory 160, an interface unit 170, a controller 180, and a power supply unit 190, etc. Fig. 1 illustrates a mobile terminal having various components, but it is to be understood that not all illustrated components are required to be implemented. More or fewer components may alternatively be implemented. Elements of the mobile terminal will be described in detail below.
The wireless communication unit 110 typically includes one or more components that allow radio communication between the mobile terminal 100 and a wireless communication device or network. For example, the wireless communication unit may include at least one of a mobile communication module 112, a wireless internet module 113.
The mobile communication module 112 transmits and/or receives radio signals to and/or from at least one of a base station (e.g., access point, node B, etc.), an external terminal, and a server. Such radio signals may include voice call signals, video call signals, or various types of data transmitted and/or received according to text and/or multimedia messages.
The wireless internet access technology to which the module refers may include W L AN (wireless L AN) (Wi-Fi), Wibro (wireless broadband), Wimax (worldwide interoperability for microwave Access), HSDPA (high speed Downlink packet Access), and the like.
The user input unit 130 may generate key input data according to a command input by a user to control various operations of the mobile terminal. The user input unit 130 allows a user to input various types of information, and may include a keyboard, dome sheet, touch pad (e.g., a touch-sensitive member that detects changes in resistance, pressure, capacitance, and the like due to being touched), scroll wheel, joystick, and the like. In particular, when the touch pad is superimposed on the display unit 151 in the form of a layer, a touch screen may be formed.
The interface unit 170 serves as an interface through which at least one external device is connected to the mobile terminal 100. For example, the external device may include a wired or wireless headset port, an external power supply (or battery charger) port, a wired or wireless data port, a memory card port, a port for connecting a device having an identification module, an audio input/output (I/O) port, a video I/O port, an earphone port, and the like. The identification module may store various information for authenticating a user using the mobile terminal 100 and may include a User Identity Module (UIM), a Subscriber Identity Module (SIM), a Universal Subscriber Identity Module (USIM), and the like. In addition, a device having an identification module (hereinafter, referred to as an "identification device") may take the form of a smart card, and thus, the identification device may be connected with the mobile terminal 100 via a port or other connection means. The interface unit 170 may be used to receive input (e.g., data information, power, etc.) from an external device and transmit the received input to one or more elements within the mobile terminal 100 or may be used to transmit data between the mobile terminal and the external device.
In addition, when the mobile terminal 100 is connected with an external cradle, the interface unit 170 may serve as a path through which power is supplied from the cradle to the mobile terminal 100 or may serve as a path through which various command signals input from the cradle are transmitted to the mobile terminal. Various command signals or power input from the cradle may be used as signals for recognizing whether the mobile terminal is accurately mounted on the cradle. The output unit 150 is configured to provide output signals (e.g., audio signals, video signals, alarm signals, vibration signals, etc.) in a visual, audio, and/or tactile manner. The output unit 150 may include a display unit 151, an audio output module 152, and the like.
The display unit 151 may display information processed in the mobile terminal 100. For example, when the mobile terminal 100 is in a phone call mode, the display unit 151 may display a User Interface (UI) or a Graphical User Interface (GUI) related to a call or other communication (e.g., text messaging, multimedia file downloading, etc.). When the mobile terminal 100 is in a video call mode or an image capturing mode, the display unit 151 may display a captured image and/or a received image, a UI or GUI showing a video or an image and related functions, and the like.
Meanwhile, when the display unit 151 and the touch pad are stacked on each other in the form of a layer TO form a touch screen, the display unit 151 may be used as an input device and an output device, the display unit 151 may include at least one of a liquid crystal display (L CD), a thin film transistor L CD (TFT-L CD), an organic light emitting diode (O L ED) display, a flexible display, a three-dimensional (3D) display, and the like, some of these displays may be configured TO be transparent TO allow a user TO view from the outside, which may be referred TO as a transparent display, a typical transparent display may be, for example, a TO L ED (transparent organic light emitting diode) display, and the like, the mobile terminal 100 may include two or more display units (or other display devices) according TO a specific intended embodiment, for example, the mobile terminal may include an external display unit (not shown) and an internal display unit (not shown), and the touch screen may be used TO detect a touch input pressure and a touch input position and a touch input area.
The audio output module 152 may convert audio data received by the wireless communication unit 110 or stored in the memory 160 into an audio signal and output as sound when the mobile terminal is in a call signal reception mode, a call mode, a recording mode, a voice recognition mode, a broadcast reception mode, or the like. Also, the audio output module 152 may provide audio output related to a specific function performed by the mobile terminal 100 (e.g., a call signal reception sound, a message reception sound, etc.). The audio output module 152 may include a microphone, a buzzer, and the like.
The memory 160 may store software programs and the like for processing and controlling operations performed by the controller 180, or may temporarily store data (e.g., a phonebook, messages, still images, videos, and the like) that has been or will be output. Also, the memory 160 may store data regarding various ways of vibration and audio signals output when a touch is applied to the touch screen.
The memory 160 may include at least one type of storage medium including a flash memory, a hard disk, a multimedia card, a card-type memory (e.g., SD or DX memory, etc.), a Random Access Memory (RAM), a Static Random Access Memory (SRAM), a read-only memory (ROM), an electrically erasable programmable read-only memory (EEPROM), a programmable read-only memory (PROM), a magnetic memory, a magnetic disk, an optical disk, and the like. Also, the mobile terminal 100 may cooperate with a network storage device that performs a storage function of the memory 160 through a network connection.
The controller 180 generally controls the overall operation of the mobile terminal. For example, the controller 180 performs control and processing related to voice calls, data communications, video calls, and the like. In addition, the controller 180 may include a multimedia module 181 for reproducing (or playing back) multimedia data, and the multimedia module 181 may be constructed within the controller 180 or may be constructed separately from the controller 180. The controller 180 may perform a pattern recognition process to recognize a handwriting input or a picture drawing input performed on the touch screen as a character or an image.
The power supply unit 190 receives external power or internal power and provides appropriate power required to operate various elements and components under the control of the controller 180.
For a hardware implementation, the embodiments described herein may be implemented using at least one of an Application Specific Integrated Circuit (ASIC), a Digital Signal Processor (DSP), a Digital Signal Processing Device (DSPD), a programmable logic device (P L D), a Field Programmable Gate Array (FPGA), a processor, a controller, a microcontroller, a microprocessor, an electronic unit designed to perform the functions described herein, such an implementation may be implemented in the controller 180 in some cases.
Up to this point, mobile terminals have been described in terms of their functionality. Hereinafter, a slide-type mobile terminal among various types of mobile terminals, such as a folder-type, bar-type, swing-type, slide-type mobile terminal, and the like, will be described as an example for the sake of brevity. Accordingly, the present invention can be applied to any type of mobile terminal, and is not limited to a slide type mobile terminal.
The mobile terminal 100 as shown in fig. 1 may be configured to operate with communication means such as wired and wireless communication means and satellite-based communication means that transmit data via frames or packets.
A communication apparatus in which a mobile terminal according to the present invention is operable will now be described with reference to fig. 2.
For example, the air interfaces used by the communication devices include, for example, Frequency Division Multiple Access (FDMA), Time Division Multiple Access (TDMA), Code Division Multiple Access (CDMA), and universal mobile telecommunications device (UMTS) (particularly Long term evolution (L TE)), Global System for Mobile communications (GSM), and so forth.
Referring to fig. 2, a CDMA wireless communication device may include a plurality of mobile terminals 100, a plurality of Base Stations (BSs) 270, Base Station Controllers (BSCs) 275, and Mobile Switching Centers (MSCs) 280, the MSCs 280 are structured to interface with Public Switched Telephone Networks (PSTNs) 290, the MSCs 280 are also structured to interface with the BSCs 275, which may be coupled to the BSs 270 via backhaul lines, which may be structured according to any of several known interfaces including, for example, E1/T1, ATM, IP, PPP, frame relay, HDS L, ADS L, or xDS L, it will be appreciated that a device as shown in fig. 2 may include a plurality of BSCs 275.
Each BS270 may serve one or more sectors (or regions), each sector covered by a multi-directional antenna or an antenna pointing in a particular direction being radially distant from the BS 270. Alternatively, each partition may be covered by two or more antennas for diversity reception. Each BS270 may be configured to support multiple frequency allocations, with each frequency allocation having a particular frequency spectrum (e.g., 1.25MHz,5MHz, etc.).
The intersection of partitions with frequency allocations may be referred to as a CDMA channel. The BS270 may also be referred to as a base transceiver sub-assembly (BTS) or other equivalent terminology. In such a case, the term "base station" may be used to generically refer to a single BSC275 and at least one BS 270. The base stations may also be referred to as "cells". Alternatively, each sector of a particular BS270 may be referred to as a plurality of cell sites.
As shown in fig. 2, a Broadcast Transmitter (BT)295 transmits a broadcast signal to the mobile terminal 100 operating within the apparatus. A broadcast receiving module 111 as shown in fig. 1 is provided at the mobile terminal 100 to receive a broadcast signal transmitted by the BT 295. In fig. 2, several global positioning device (GPS) satellites 300 are shown. The satellite 300 assists in locating at least one of the plurality of mobile terminals 100.
In fig. 2, a plurality of satellites 300 are depicted, but it is understood that useful positioning information may be obtained with any number of satellites. The location information module 115 as shown in fig. 1 is generally configured to cooperate with the satellites 300 to obtain desired positioning information. Other techniques that can track the location of the mobile terminal may be used instead of or in addition to GPS tracking techniques. In addition, at least one GPS satellite 300 may selectively or additionally process satellite DMB transmission.
As a typical operation of the wireless communication apparatus, the BS270 receives reverse link signals from various mobile terminals 100. The mobile terminal 100 is generally engaged in conversations, messaging, and other types of communications. Each reverse link signal received by a particular BS270 is processed within the particular BS 270. The obtained data is forwarded to the associated BSC 275. The BSC provides call resource allocation and mobility management functions including coordination of soft handoff procedures between BSs 270. The BSCs 275 also route the received data to the MSC280, which provides additional routing services for interfacing with the PSTN 290. Similarly, the PSTN290 interfaces with the MSC280, the MSC interfaces with the BSCs 275, and the BSCs 275 accordingly control the BS270 to transmit forward link signals to the mobile terminal 100.
Based on the hardware structure of the mobile terminal and the structure of the communication device, the device and the method of the invention are provided with various embodiments.
The invention provides an audio and video frame loss device.
Referring to fig. 3, fig. 3 is a functional module schematic diagram of an audio-video frame loss device according to a first embodiment of the present invention.
In this embodiment, the audio/video frame loss device includes:
the obtaining module 10 is configured to obtain a time difference between an audio frame time and a video frame time of a current playing time in real time or at a fixed time.
In this embodiment, when the video played by the mobile terminal is not synchronized with the audio/video playing, and the video image decoding duration is longer than the image display duration, the audio/video output needs to be adjusted. Firstly, triggering an instruction for starting an audio and video frame loss synchronization mechanism, and acquiring the audio frame time and the video frame time of the current playing time in real time or at fixed time by the mobile terminal according to the instruction, and calculating the time difference between the audio frame time and the video frame time of the current playing time. In a video coding sequence, there are mainly three types of coded frames: i frame, P frame, B frame. Wherein: i-frame is Intra-coded picture frame, and is coded by using only the information of the frame without referring to other picture frames. The P frame, i.e., a Predictive-coded picture frame, is inter-frame Predictive-coded using a motion prediction method using a previous I frame or P frame. The B frame, i.e. bidirectional predictive coded image frame, provides the highest compression ratio, and requires both the previous image frame (I frame or P frame) and the subsequent image frame (P frame), and performs inter-frame bidirectional predictive coding by using a motion prediction method, and generally, the B frame is only used for recording the difference between the current frame and the previous and subsequent frames, so that in the video playing process, when the audio and video are not synchronized, the corresponding B frame in the video is generally discarded.
The determining module 20 is configured to determine whether the video includes a B frame if the absolute value of the time difference is greater than or equal to a preset time length, where the preset time length is a preset multiple of a display time length of a unit video frame.
In this embodiment, the absolute value of the time difference between the audio frame time and the video frame time is compared with a preset duration, where the preset duration is a preset multiple of the display duration of a unit video frame. If the absolute value of the time difference is greater than or equal to the preset time length, namely when the audio and video frames differ by the preset time length of displaying a plurality of continuous video frames, the preset time length can be 2 times or multiples of other numerical values, people can feel that the audio and video are not synchronous at the moment, and the audio and video are judged to be not synchronous at the moment. At this time, the output of the audio and video needs to be adjusted, so that whether a B frame which does not affect the video playing exists in the video needs to be judged, so as to discard the B frame.
The determining module 30 is configured to determine, if the video includes B frames, a time interval for discarding the B frames in the video according to a first frame rate of a currently played video source file and a second frame rate of a currently played video.
Specifically, if there are B frames in the video, the time interval for discarding the B frames in the video is determined according to the first frame rate of the currently played video source file and the second frame rate of the currently played video. Specifically, the step of determining a time interval for dropping B frames in the video may include:
step one, determining a third frame rate supported by a terminal according to the performance of a terminal chip;
step two, judging whether the second frame rate is smaller than the third frame rate;
step three, when the second frame rate is greater than or equal to the third frame rate, increasing a preset increment to the third frame rate to update the third frame rate, and continuing to execute the step two;
when the second frame rate is lower than the third frame rate, judging whether the sum of the second frame rate and a preset value is higher than the third frame rate;
step five, when the sum of the second frame rate and a preset value is less than or equal to the third frame rate, calculating the average value of the second frame rate and the third frame rate, and calculating the time interval according to the average value and the first frame rate;
and step six, when the sum of the second frame rate and a preset value is greater than the third frame rate, subtracting the preset increment from the third frame rate to update the third frame rate, and continuing to execute the step one.
The first discarding module 40 is configured to discard B frames according to the time interval, so that the frame rate of the video source file reaches the frame rate of the currently played video.
And after the time interval is obtained through calculation, discarding B frames according to the time interval so as to enable the frame rate of the video source file to reach the frame rate of video playing currently.
The audio and video frame loss device provided by the invention obtains the time difference between the audio frame time and the video frame time at the current playing moment in real time or at regular time; when the absolute value of the time difference is greater than or equal to a preset time length, judging whether the video contains a B frame, wherein the preset time length is a preset multiple of the display time length of a unit video frame; when the video comprises B frames, determining the time interval for discarding the B frames in the video according to the first frame rate of the currently played video source file and the second frame rate of the currently played video, discarding the B frames according to the time interval, and enabling the frame rate of the video source file to reach the frame rate of the currently played video by discarding the B frames according to the time interval, thereby realizing the output synchronization of the audio and the video.
Further, referring to fig. 4, a second embodiment of the audio and video frame loss device of the present invention is proposed based on the first embodiment of the audio and video frame loss device of the present invention.
In this embodiment, the determining module 30 includes:
a determining unit 31, configured to determine a third frame rate supported by the terminal according to the performance of the terminal chip;
a determining unit 32, configured to determine whether the second frame rate is less than the third frame rate;
an adjusting unit 33, configured to increase a preset increment to the third frame rate to update the third frame rate when the second frame rate is greater than or equal to the third frame rate;
the determining unit 32 is further configured to determine whether a sum of the second frame rate and a preset value is greater than the third frame rate when the second frame rate is less than the third frame rate;
a first calculating unit 34, configured to calculate an average value of the second frame rate and the third frame rate when a sum of the second frame rate and a preset value is less than or equal to the third frame rate, and calculate the time interval according to the average value and the first frame rate;
the adjusting unit 33 is further configured to subtract the preset increment from the third frame rate to update the third frame rate when the sum of the second frame rate and a preset value is greater than the third frame rate.
In this embodiment, the specific step of calculating the time interval of the B frame in the discarded video may be:
step one, determining a third frame rate supported by a terminal according to the performance of a terminal chip, specifically, the third frame rate is a frame rate directly determined according to the performance of the terminal chip, and the frame rates supported by different terminal chips are generally different, and are not described herein any more;
step two, judging whether the second frame rate is smaller than the third frame rate;
step three, when the second frame rate is greater than or equal to the third frame rate, increasing a preset increment to the third frame rate to update the third frame rate, and continuing to execute the step two; specifically, the preset increment may be 1, and a value other than 1 may also be adopted in the specific implementation process.
When the second frame rate is lower than the third frame rate, judging whether the sum of the second frame rate and a preset value is higher than the third frame rate; specifically, the preset value may be 5, and a value other than 5 may also be adopted in the specific implementation process.
Step five, when the sum of the second frame rate and a preset value is less than or equal to the third frame rate, calculating the average value of the second frame rate and the third frame rate, and calculating the time interval according to the average value and the first frame rate; specifically, the method for calculating the time interval according to the average value and the first frame rate specifically includes: subtracting the average value from the first frame rate to obtain a difference value, and dividing 1000 by the difference value to obtain the time interval.
And step six, when the sum of the second frame rate and a preset value is greater than the third frame rate, subtracting the preset increment from the third frame rate to update the third frame rate, and continuing to execute the step one.
According to the audio/video frame loss device provided by the embodiment, the time interval of the frame B in the video is calculated through the first frame rate, the second frame rate and the second frame rate, so that the frame B in the video can be accurately and timely discarded in the subsequent playing process according to the time interval, and therefore the accurate and real-time control of discarding the frame B which has no influence on the video playing is realized, and the audio/video synchronous output is adjusted.
Further, a third embodiment of the audio and video frame loss device of the present invention is proposed based on the first embodiment of the audio and video frame loss device of the present invention.
In this embodiment, the apparatus further includes:
and the playing module 50 is configured to play the next I frame if there is no B frame in the video.
In this embodiment, if there is no B frame in the video, the next I frame needs to be played to adjust the output synchronization of the audio and video.
Further, referring to fig. 5, a fourth embodiment of the audio/video frame loss device of the present invention is provided based on the first embodiment of the audio/video frame loss device of the present invention.
In this embodiment, the apparatus further includes:
a comparison module 60, configured to compare the current system playing time with the actual playing time in real time;
a second discarding module 70, configured to discard B frames in the video source file when a time difference between the current system playing time and the actual playing time is greater than a first preset time;
the playing module 50 is further configured to play a next I frame when a time difference between the current system playing time and the actual playing time is greater than a second preset time, where the first preset time is less than the second preset time.
In this embodiment, when the buffering time of the currently played video is too long, the situation of audio and video asynchronism also occurs, so that B frames in the video also need to be discarded, or a next I frame needs to be played, so as to achieve audio and video synchronous output. Specifically, the current system playing time and the actual playing time are compared in real time; when the time difference between the current system playing time and the actual playing time is greater than a first preset time, discarding B frames in the video source file; and when the time difference between the current system playing time and the actual playing time is greater than a second preset time, playing the next I frame, wherein the first preset time is less than the second preset time. The first preset time may be 100ms, the second preset time may be 500ms, and the first preset time and the second preset time may be set to other values in a specific implementation process.
The audio and video frame loss device provided by the embodiment compares the current system playing time with the actual playing time in real time; when the time difference between the current system playing time and the actual playing time is greater than a first preset time, discarding B frames in the video source file; and when the time difference between the current system playing time and the actual playing time is greater than a second preset time, playing the next I frame, wherein the first preset time is less than the second preset time, thereby realizing the synchronous output of the audio and video in the video.
Further, referring to fig. 6, a fifth embodiment of the audio and video frame loss device of the present invention is provided based on the fourth embodiment of the audio and video frame loss device of the present invention.
In this embodiment, the second discarding module 70 may include:
a second calculating unit 71, configured to calculate, according to the second frame rate of the currently playing video and the first frame rate of the currently playing video source file, the number of B frames that need to be discarded;
and a discarding unit 72, configured to discard the corresponding B frames in the video source file according to the calculated number of B frames that need to be discarded.
In this embodiment, the number of B frames that need to be discarded may be calculated by the second frame rate of the currently played video and the first frame rate of the currently played video source file, and then the corresponding B frames are discarded in the video source file according to the calculated number of B frames that need to be discarded. Calculating the number of B frames to be discarded according to the second frame rate of the currently played video and the first frame rate of the currently played video source file is the existing general technology, and is not described herein any more.
The audio/video frame dropping device provided by the embodiment calculates the number of B frames to be dropped according to the second frame rate of the currently played video and the first frame rate of the currently played video source file; and then, discarding the corresponding B frames in the video source file according to the calculated number of the B frames needing to be discarded, so as to avoid the phenomenon of discontinuous playing and pause caused by discarding excessive B frames.
In order to better understand the audio/video frame loss device provided by the present invention, the position of the audio/video frame loss device in the video playing system is briefly described herein, and in the application layer of the video playing system, the application program of the video player and the media player include the media player service in the local framework layer. The specific multimedia playing frame is as follows: and outputting the video data source file to a video file analysis module for analysis, then calling through a multi-channel distributor, then outputting the B frames needing to be discarded, and finally decoding through an audio and video decoder and outputting. The frame loss device for verifying audio and video synchronization provided by the invention can analyze the video file, and discard the B frame in the video, thereby realizing audio and video synchronization output.
The invention further provides an audio and video frame loss method.
Referring to fig. 7, fig. 7 is a schematic flow chart of a first embodiment of audio-video frame loss according to the present invention.
In this embodiment, the method includes:
step S10, the time difference between the audio frame time and the video frame time at the current playing time is obtained in real time or at regular time.
In this embodiment, when the video played by the mobile terminal is not synchronized with the audio/video playing, and the video image decoding duration is longer than the image display duration, the audio/video output needs to be adjusted. Firstly, triggering an instruction for starting an audio and video frame loss synchronization mechanism, and acquiring the audio frame time and the video frame time of the current playing time in real time or at fixed time by the mobile terminal according to the instruction, and calculating the time difference between the audio frame time and the video frame time of the current playing time. In a video coding sequence, there are mainly three types of coded frames: i frame, P frame, B frame. Wherein: i-frame is Intra-coded picture frame, and is coded by using only the information of the frame without referring to other picture frames. The P frame, i.e., a Predictive-coded picture frame, is inter-frame Predictive-coded using a motion prediction method using a previous I frame or P frame. The B frame, i.e. bidirectional predictive coded image frame, provides the highest compression ratio, and requires both the previous image frame (I frame or P frame) and the subsequent image frame (P frame), and performs inter-frame bidirectional predictive coding by using a motion prediction method, and generally, the B frame is only used for recording the difference between the current frame and the previous and subsequent frames, so that in the video playing process, when the audio and video are not synchronized, the corresponding B frame in the video is generally discarded.
Step S20, if the absolute value of the time difference is greater than or equal to a preset time length, whether the video contains the B frame is judged, and the preset time length is a preset multiple of the display time length of the unit video frame.
In this embodiment, the absolute value of the time difference between the audio frame time and the video frame time is compared with a preset duration, where the preset duration is a preset multiple of the display duration of a unit video frame. If the absolute value of the time difference is greater than or equal to the preset time length, namely when the audio and video frames differ by the preset time length of displaying a plurality of continuous video frames, the preset time length can be 2 times or multiples of other numerical values, people can feel that the audio and video are not synchronous at the moment, and the audio and video are judged to be not synchronous at the moment. At this time, the output of the audio and video needs to be adjusted, so that whether a B frame which does not affect the video playing exists in the video needs to be judged, so as to discard the B frame.
Step S30, if the video includes B frames, determining a time interval for discarding the B frames in the video according to a first frame rate of a currently played video source file and a second frame rate of the currently played video.
Specifically, if there are B frames in the video, the time interval for discarding the B frames in the video is determined according to the first frame rate of the currently played video source file and the second frame rate of the currently played video. Specifically, the step of determining a time interval for dropping B frames in the video may include:
step one, determining a third frame rate supported by a terminal according to the performance of a terminal chip;
step two, judging whether the second frame rate is smaller than the third frame rate;
step three, when the second frame rate is greater than or equal to the third frame rate, increasing a preset increment to the third frame rate to update the third frame rate, and continuing to execute the step two;
when the second frame rate is lower than the third frame rate, judging whether the sum of the second frame rate and a preset value is higher than the third frame rate;
step five, when the sum of the second frame rate and a preset value is less than or equal to the third frame rate, calculating the average value of the second frame rate and the third frame rate, and calculating the time interval according to the average value and the first frame rate;
and step six, when the sum of the second frame rate and a preset value is greater than the third frame rate, subtracting the preset increment from the third frame rate to update the third frame rate, and continuing to execute the step one.
Step S40, discarding B frames according to the time interval, so that the frame rate of the video source file reaches the frame rate of the currently played video.
And after the time interval is obtained through calculation, discarding B frames according to the time interval so as to enable the frame rate of the video source file to reach the frame rate of video playing currently.
The audio and video frame loss method provided by the invention obtains the time difference between the audio frame time and the video frame time at the current playing moment in real time or at regular time; when the absolute value of the time difference is greater than or equal to a preset time length, judging whether the video contains a B frame, wherein the preset time length is a preset multiple of the display time length of a unit video frame; when the video comprises B frames, determining the time interval for discarding the B frames in the video according to the first frame rate of the currently played video source file and the second frame rate of the currently played video, discarding the B frames according to the time interval, and enabling the frame rate of the video source file to reach the frame rate of the currently played video by discarding the B frames according to the time interval, thereby realizing the output synchronization of the audio and the video.
Further, referring to fig. 8, a second embodiment of the audio/video frame loss method of the present invention is provided based on the first embodiment of the audio/video frame loss method of the present invention.
In this embodiment, the step S30 includes:
step S31, determining a third frame rate supported by the terminal according to the performance of the terminal chip;
step S32, determining whether the second frame rate is less than the third frame rate;
step S33, when the second frame rate is greater than or equal to the third frame rate, increasing a preset increment to the third frame rate to update the third frame rate, and continuing to execute the step of determining whether the second frame rate is less than the third frame rate;
step S34, when the second frame rate is lower than the third frame rate, determining whether a sum of the second frame rate and a preset value is higher than the third frame rate;
step S35, when the sum of the second frame rate and a preset value is less than or equal to the third frame rate, calculating an average value of the second frame rate and the third frame rate, and calculating the time interval according to the average value and the first frame rate;
step S36, when the sum of the second frame rate and a preset value is greater than the third frame rate, subtracting the preset increment from the third frame rate to update the third frame rate, and continuing to perform the step of determining whether the second frame rate is less than the third frame rate.
In this embodiment, the specific step of calculating the time interval of the B frame in the discarded video may be:
step one, determining a third frame rate supported by a terminal according to the performance of a terminal chip, specifically, the third frame rate is a frame rate directly determined according to the performance of the terminal chip, and the frame rates supported by different terminal chips are generally different, and are not described herein any more;
step two, judging whether the second frame rate is smaller than the third frame rate;
step three, when the second frame rate is greater than or equal to the third frame rate, increasing a preset increment to the third frame rate to update the third frame rate, and continuing to execute the step two; specifically, the preset increment may be 1, and a value other than 1 may also be adopted in the specific implementation process.
When the second frame rate is lower than the third frame rate, judging whether the sum of the second frame rate and a preset value is higher than the third frame rate; specifically, the preset value may be 5, and a value other than 5 may also be adopted in the specific implementation process.
Step five, when the sum of the second frame rate and a preset value is less than or equal to the third frame rate, calculating the average value of the second frame rate and the third frame rate, and calculating the time interval according to the average value and the first frame rate; specifically, the method for calculating the time interval according to the average value and the first frame rate specifically includes: subtracting the average value from the first frame rate to obtain a difference value, and dividing 1000 by the difference value to obtain the time interval.
And step six, when the sum of the second frame rate and a preset value is greater than the third frame rate, subtracting the preset increment from the third frame rate to update the third frame rate, and continuing to execute the step one.
According to the audio/video frame loss method provided by the embodiment, the time interval of the frame B in the video is calculated through the first frame rate, the second frame rate and the second frame rate, so that the frame B in the video can be accurately and timely discarded in the subsequent playing process according to the time interval, and therefore the accurate and real-time control of discarding the frame B which has no influence on the video playing is realized, and the audio/video synchronous output is adjusted.
Further, the audio and video frame loss method of the invention is provided in a third embodiment based on the first embodiment of the audio and video frame loss method of the invention.
In this embodiment, the method further includes:
and if the video has no B frame, playing the next I frame.
In this embodiment, if there is no B frame in the video, the next I frame needs to be played to adjust the output synchronization of the audio and video.
Further, referring to fig. 9, a fourth embodiment of the audio/video frame loss method of the present invention is provided based on the third embodiment of the audio/video frame loss method of the present invention.
In this embodiment, the method further includes:
s50, comparing the current system playing time with the actual playing time in real time;
s60, when the time difference between the current system playing time and the actual playing time is greater than a first preset time, discarding the B frame in the video source file;
and S70, when the time difference between the current system playing time and the actual playing time is greater than a second preset time, playing the next I frame, wherein the first preset time is less than the second preset time.
In this embodiment, when the buffering time of the currently played video is too long, the situation of audio and video asynchronism also occurs, so that B frames in the video also need to be discarded, or a next I frame needs to be played, so as to achieve audio and video synchronous output. Specifically, the current system playing time and the actual playing time are compared in real time; when the time difference between the current system playing time and the actual playing time is greater than a first preset time, discarding B frames in the video source file; and when the time difference between the current system playing time and the actual playing time is greater than a second preset time, playing the next I frame, wherein the first preset time is less than the second preset time. The first preset time may be 100ms, the second preset time may be 500ms, and the first preset time and the second preset time may be set to other values in a specific implementation process.
The audio and video frame loss method provided by the embodiment compares the current system playing time with the actual playing time in real time; when the time difference between the current system playing time and the actual playing time is greater than a first preset time, discarding B frames in the video source file; and when the time difference between the current system playing time and the actual playing time is greater than a second preset time, playing the next I frame, wherein the first preset time is less than the second preset time, thereby realizing the synchronous output of the audio and video in the video.
Further, referring to fig. 10, a fifth embodiment of the audio-video frame loss method of the present invention is provided based on any one of the first to fourth embodiments of the audio-video frame loss method of the present invention.
In this embodiment, the step S60 includes:
step S61, calculating the number of B frames to be discarded according to the second frame rate of the currently played video and the first frame rate of the currently played video source file;
and step S62, discarding the corresponding B frames in the video source file according to the calculated number of the B frames needing to be discarded.
In this embodiment, the number of B frames that need to be discarded may be calculated by the second frame rate of the currently played video and the first frame rate of the currently played video source file, and then the corresponding B frames are discarded in the video source file according to the calculated number of B frames that need to be discarded. Calculating the number of B frames to be discarded according to the second frame rate of the currently played video and the first frame rate of the currently played video source file is the existing general technology, and is not described herein any more.
The audio/video frame loss method provided by the embodiment includes the steps that firstly, the number of B frames needing to be discarded is calculated according to the second frame rate of a currently played video and the first frame rate of a currently played video source file; and then, discarding the corresponding B frames in the video source file according to the calculated number of the B frames needing to be discarded, so as to avoid the phenomenon of discontinuous playing and pause caused by discarding excessive B frames.
It should be noted that, in this document, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising an … …" does not exclude the presence of other like elements in a process, method, article, or apparatus that comprises the element.
The above-mentioned serial numbers of the embodiments of the present invention are merely for description and do not represent the merits of the embodiments.
Through the above description of the embodiments, those skilled in the art will clearly understand that the method of the above embodiments can be implemented by software plus a necessary general hardware platform, and certainly can also be implemented by hardware, but in many cases, the former is a better implementation manner. Based on such understanding, the technical solutions of the present invention may be embodied in the form of a software product, which is stored in a storage medium (e.g., ROM/RAM, magnetic disk, optical disk) and includes instructions for enabling a terminal device (e.g., a mobile phone, a computer, a server, or a network device) to execute the method according to the embodiments of the present invention.
The above description is only a preferred embodiment of the present invention, and not intended to limit the scope of the present invention, and all modifications of equivalent structures and equivalent processes, which are made by using the contents of the present specification and the accompanying drawings, or directly or indirectly applied to other related technical fields, are included in the scope of the present invention.

Claims (10)

1. An audio-video frame loss device, the device comprising:
the acquisition module is used for acquiring the time difference between the audio frame time and the video frame time at the current playing time in real time or at regular time;
the judging module is used for judging whether the video comprises a B frame or not if the absolute value of the time difference is greater than or equal to a preset time length, wherein the preset time length is a preset multiple of the display time length of a unit video frame;
the determining module is used for determining the time interval for discarding the B frames in the video according to a first frame rate of a currently played video source file and a second frame rate of the currently played video if the video contains the B frames; the determining module comprises a determining unit, a judging unit and a first calculating unit;
the determining unit is configured to determine, if the video includes a B frame, a third frame rate supported by the terminal according to a performance of a terminal chip;
the judging unit is used for judging whether the second frame rate is smaller than the third frame rate;
the determining unit is further configured to determine whether a sum of the second frame rate and a preset value is greater than the third frame rate when the second frame rate is less than the third frame rate;
the first calculating unit is configured to calculate an average value of the second frame rate and the third frame rate when a sum of the second frame rate and a preset value is less than or equal to the third frame rate, and calculate a time interval for discarding B frames in a video according to the average value and the first frame rate;
and the first discarding module is used for discarding the B frames according to the time interval so as to enable the frame rate of the video source file to reach the frame rate of the current video playing.
2. The audiovisual frame loss device of claim 1, wherein said determining module further comprises:
an adjusting unit, configured to increase a preset increment to the third frame rate to update the third frame rate when the second frame rate is greater than or equal to the third frame rate;
the adjusting unit is further configured to subtract the preset increment from the third frame rate to update the third frame rate when the sum of the second frame rate and a preset value is greater than the third frame rate.
3. The audiovisual frame loss device of claim 1, further comprising:
and the playing module is used for playing the next I frame if the video does not have the B frame.
4. The audiovisual frame loss device of claim 1, further comprising:
the comparison module is used for comparing the current system playing time with the actual playing time in real time;
the second discarding module is used for discarding the B frames in the video source file when the time difference between the current system playing time and the actual playing time is greater than the first preset time;
the playing module is further configured to play a next I frame when a time difference between the current system playing time and the actual playing time is greater than a second preset time, where the first preset time is less than the second preset time.
5. The audiovisual frame loss device of claim 4, wherein the second dropping module comprises:
the second calculation unit is used for calculating the number of the B frames needing to be discarded according to the second frame rate of the currently played video and the first frame rate of the currently played video source file;
and the discarding unit is used for discarding the corresponding B frames in the video source file according to the calculated number of the B frames needing to be discarded.
6. An audio-video frame loss method is characterized by comprising the following steps:
acquiring the time difference between the audio frame time and the video frame time at the current playing time in real time or at regular time;
if the absolute value of the time difference is greater than or equal to a preset time length, judging whether the video contains a B frame, wherein the preset time length is a preset multiple of the display time length of a unit video frame;
if so, determining the time interval for discarding the B frames in the video according to the first frame rate of the currently played video source file and the second frame rate of the currently played video;
discarding B frames according to the time interval so as to enable the frame rate of the video source file to reach the frame rate of video playing currently;
the step of determining the time interval for discarding the B frames in the video according to the first frame rate of the currently played video source file and the second frame rate of the currently played video includes:
determining a third frame rate supported by the terminal according to the performance of the terminal chip;
judging whether the second frame rate is smaller than the third frame rate;
when the second frame rate is less than the third frame rate, judging whether the sum of the second frame rate and a preset value is greater than the third frame rate;
when the sum of the second frame rate and a preset value is less than or equal to the third frame rate, calculating the average value of the second frame rate and the third frame rate, and calculating the time interval for discarding B frames in the video according to the average value and the first frame rate.
7. The audio-video frame loss method according to claim 6, wherein after the step of determining whether the second frame rate is less than the third frame rate, the method further comprises:
when the second frame rate is greater than or equal to the third frame rate, increasing a preset increment to the third frame rate to update the third frame rate, and continuing to execute the step of judging whether the second frame rate is less than the third frame rate;
after the step of determining whether the sum of the second frame rate and a preset value is greater than the third frame rate, the method further includes:
and when the sum of the second frame rate and a preset value is greater than the third frame rate, subtracting the preset increment from the third frame rate to update the third frame rate, and continuing to execute the step of judging whether the second frame rate is less than the third frame rate.
8. The audio-video frame loss method according to claim 7, wherein the step of determining whether there is a B frame in the video further comprises:
and if the video has no B frame, playing the next I frame.
9. An audiovisual frame loss method according to any of claims 6-8, characterized in that the method further comprises:
comparing the current system playing time with the actual playing time in real time;
when the time difference between the current system playing time and the actual playing time is greater than a first preset time, discarding B frames in the video source file;
and when the time difference between the current system playing time and the actual playing time is greater than a second preset time, playing the next I frame, wherein the first preset time is less than the second preset time.
10. The audiovisual frame loss method of claim 9, wherein said step of dropping B frames in said video source file comprises:
calculating the number of B frames needing to be discarded according to the second frame rate of the currently played video and the first frame rate of the currently played video source file;
and discarding the corresponding B frames in the video source file according to the calculated number of the B frames needing to be discarded.
CN201710045496.8A 2017-01-20 2017-01-20 Audio and video frame loss device and method Active CN106817614B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201710045496.8A CN106817614B (en) 2017-01-20 2017-01-20 Audio and video frame loss device and method

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201710045496.8A CN106817614B (en) 2017-01-20 2017-01-20 Audio and video frame loss device and method

Publications (2)

Publication Number Publication Date
CN106817614A CN106817614A (en) 2017-06-09
CN106817614B true CN106817614B (en) 2020-08-04

Family

ID=59111194

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201710045496.8A Active CN106817614B (en) 2017-01-20 2017-01-20 Audio and video frame loss device and method

Country Status (1)

Country Link
CN (1) CN106817614B (en)

Families Citing this family (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112073823B (en) * 2020-09-02 2022-07-12 深圳创维数字技术有限公司 Frame loss processing method, video playing terminal and computer readable storage medium
CN113473229B (en) * 2021-06-25 2022-04-12 荣耀终端有限公司 Method for dynamically adjusting frame loss threshold and related equipment

Family Cites Families (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US7471337B2 (en) * 2004-06-09 2008-12-30 Lsi Corporation Method of audio-video synchronization
CN103310820B (en) * 2012-03-14 2016-03-09 三星电子（中国）研发中心 A kind of method that multimedia player is optimized
CN103856812B (en) * 2014-03-25 2018-08-07 北京奇艺世纪科技有限公司 A kind of video broadcasting method and device
CN104822008B (en) * 2014-04-25 2019-01-08 腾讯科技（北京）有限公司 video synchronization method and device
CN105933800A (en) * 2016-04-29 2016-09-07 联发科技（新加坡）私人有限公司 Video play method and control terminal

Also Published As

Publication number Publication date
CN106817614A (en) 2017-06-09

Similar Documents

Publication Publication Date Title
CN106507173B (en) Mobile terminal and split-screen display control method
CN106603543B (en) Correct the synchronous method and device of stream medium audio and video
CN105468158B (en) Color adjustment method and mobile terminal
US8494494B2 (en) Mobile terminal and method of controlling mobile terminal
CN108702700B (en) Indication and search method and device for defining cell synchronous broadcast block position and base station
CN106911850B (en) Mobile terminal and screen capturing method thereof
CN108401521B (en) Information generation method and device and signal transmission method and device
CN106157970B (en) Audio identification method and terminal
CN107071263B (en) Image processing method and terminal
EP2117142A2 (en) Mobile terminal and method of selecting brodcast mode therein
CN106547439B (en) Method and device for processing message
CN109792625B (en) Method and device for transmitting configuration information about measurement control
US8406737B2 (en) Mobile terminal and controlling method thereof
US8611963B2 (en) Mobile terminal processing and transmitting information related to data play and playing data according to information
CN106817614B (en) Audio and video frame loss device and method
CN105722246B (en) Network speed superposition device and method
CN106453863B (en) Method and system for controlling terminal and earphone
CN105682007B (en) Mobile terminal and WIFI control method
CN109168029B (en) Method, device and computer-readable storage medium for adjusting resolution
CN106791567B (en) Switching method and terminal
US8116753B2 (en) Mobile terminal and method for playing multimedia objects
CN105744061B (en) Eye print recognition interaction device and method
EP2059033A2 (en) Mobile communications terminal for broadcast reception
CN105959301B (en) Multimedia call optimization method and optimization system thereof
CN109275038B (en) Game live broadcast method, terminal and computer readable storage medium

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
TA01 Transfer of patent application right

Effective date of registration: 20200706

Address after: Yuyao City, Zhejiang province 315400 Ningbo Zhongyi Ningbo ecological park Hing Rd No. 28

Applicant after: ZHEJIANG RUIHUA KANGYUAN TECHNOLOGY Co.,Ltd.

Address before: 518057 Guangdong Province, Shenzhen high tech Zone of Nanshan District City, No. 9018 North Central Avenue's innovation building A, 6-8 layer, 10-11 layer, B layer, C District 6-10 District 6 floor

Applicant before: NUBIA TECHNOLOGY Co.,Ltd.

TA01 Transfer of patent application right
GR01 Patent grant
GR01 Patent grant