WO2024255342A1 - 视频卡顿检测方法及相关装置 - Google Patents

视频卡顿检测方法及相关装置 Download PDF

Info

Publication number
WO2024255342A1
WO2024255342A1 PCT/CN2024/081005 CN2024081005W WO2024255342A1 WO 2024255342 A1 WO2024255342 A1 WO 2024255342A1 CN 2024081005 W CN2024081005 W CN 2024081005W WO 2024255342 A1 WO2024255342 A1 WO 2024255342A1
Authority
WO
WIPO (PCT)
Prior art keywords
electronic device
video
control
interface
event
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/CN2024/081005
Other languages
English (en)
French (fr)
Inventor
张威
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Honor Device Co Ltd
Original Assignee
Honor Device Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Honor Device Co Ltd filed Critical Honor Device Co Ltd
Publication of WO2024255342A1 publication Critical patent/WO2024255342A1/zh
Anticipated expiration legal-status Critical
Ceased legal-status Critical Current

Links

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
    • H04N21/442—Monitoring of processes or resources, e.g. detecting the failure of a recording device, monitoring the downstream bandwidth, the number of times a movie has been viewed, the storage space available from the internal hard disk
    • H04N21/44209—Monitoring of downstream path of the transmission network originating from a server, e.g. bandwidth variations of a wireless network
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
    • H04N21/442—Monitoring of processes or resources, e.g. detecting the failure of a recording device, monitoring the downstream bandwidth, the number of times a movie has been viewed, the storage space available from the internal hard disk
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
    • H04N21/443—OS processes, e.g. booting an STB, implementing a Java virtual machine in an STB or power management in an STB
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
    • H04N21/443—OS processes, e.g. booting an STB, implementing a Java virtual machine in an STB or power management in an STB
    • H04N21/4438—Window management, e.g. event handling following interaction with the user interface

Definitions

  • the present application relates to the field of terminal technology, and in particular to a video freeze detection method and related devices.
  • some electronic devices support video playback. Users can watch videos, shoot videos, and start live broadcasts through video applications. However, when the electronic device plays the content of the video application, the interface of the electronic device may become stuck, affecting the user experience.
  • the video freeze detection method and related devices provided in the embodiments of the present application enable the electronic device to perform frame rate refresh detection on drawing controls that do not load local data when detecting freezes in the video application interface, thereby not being affected by controls that load local data, improving the accuracy of determining whether the video application interface is freezed, and further helping to reduce freezes in the video application interface.
  • the video freeze detection method includes:
  • the electronic device displays a video playback interface
  • the video playback interface includes a first control and a second control
  • the first control is used to display a first type of view
  • the second control is used to display a second type of view
  • the first type of view does not include a view loaded locally from the electronic device
  • the second type of view includes a view loaded locally from the electronic device; when the frame interval of the view displayed in the first control is greater than or equal to a preset value, the electronic device determines that the video is stuck; wherein the frame interval of the view displayed in the second control is less than, equal to, or greater than the preset value.
  • the electronic device before the electronic device determines that the video is stuck, the electronic device also includes: the electronic device determines that no preset event occurs in the video playback interface, and the preset event includes one or more of the following: an event in which a progress bar control in the video playback interface is triggered, an event for indicating that the video playback interface is paused, and an event for indicating that the video playback interface is switched to the background. In this way, it can be determined whether the current interface is paused due to an event triggered by the user, and thus it can be more accurately determined whether the interface is stuck.
  • the preset value includes a first threshold value
  • the electronic device determines that the layer texture identifier of the view displayed in the first control has not changed; when the frame interval of the view displayed in the first control is greater than or equal to the preset value, the electronic device determines that the video is stuck, including: when the frame interval of the view displayed in the first control is greater than or equal to the first threshold value, and no event of the progress bar control being triggered occurs in the video playback interface, and/or an event for indicating the pause of the video playback interface occurs, the electronic device determines that the video is stuck. In this way, it can be determined that the interval time between two frames of images exceeds the threshold value due to the user's pause operation, thereby accurately judging whether the video is stuck.
  • the preset value includes a second threshold value
  • the electronic device determines that the layer texture identifier of the view displayed in the first control has changed; when the frame interval of the view displayed in the first control is greater than or equal to the preset value, the electronic device determines that the video is stuck, including: when the frame interval of the view displayed in the first control is greater than or equal to the second threshold value, and no event occurs in the video playback interface to indicate switching the video playback interface to the background, the electronic device determines that the video is stuck.
  • the interval time between two frames of images exceeds the threshold value due to the user's background switching operation, thereby accurately judging whether the video is stuck.
  • the electronic device determines that a video freeze occurs, including: the electronic device records a log related to the freeze, so that further analysis and statistics of the freeze can be performed later.
  • the logs related to the jamming include one or more of the following: scene resource logs, scene event logs.
  • the electronic device records different types of logs to facilitate subsequent different types of analysis of the jamming, thereby specifically analyzing and solving a certain type of problem.
  • the electronic device determines that the video freezes, including: the electronic device performs processing related to preset performance indicators; the preset performance indicators include one or more of the following: process CPU utilization of the electronic device, memory application time of the electronic device, and network signal of the electronic device.
  • the performance of the electronic device can be improved, thereby helping to reduce the freeze of the video interface.
  • the electronic device performs processing related to the preset performance index, including: the electronic device determines that the CPU utilization of the process is abnormal; the electronic device performs detection related to the CPU utilization of the process, and the detection related to the CPU utilization of the process includes one or more of the following: frequency limit detection, scheduling priority detection, and inter-core running time detection; the electronic device records a log related to the CPU utilization of the process; the electronic device increases the CPU running frequency and/or increases the priority of the process; the electronic device determines that the memory application duration is abnormal; the electronic device performs detection related to the memory application duration, and the detection related to the memory application duration includes one or more of the following: available memory detection, slow path application times detection; the electronic device records a log related to the memory application duration; the electronic device cleans up the memory of the electronic device; the electronic device determines that the network signal is abnormal; the electronic device performs detection related to the network signal, and the detection related to the network signal includes one or more of the following: network delay detection, network packet loss detection; the electronic device records
  • the method further includes: when the electronic device determines that a preset condition is met, the electronic device records the frame time; when the electronic device determines that a preset event occurs in the video playback interface, the electronic device sets the frame time to a target value; the electronic device determines that there is buffer data related to the playback video, including: when the frame time is not the target value, the electronic device determines that there is buffer data related to the playback video; wherein, satisfying the preset condition includes any of the following: there is no buffer data related to the playback video; there is buffer data related to the playback video, and the frame interval of the view displayed in the first control is less than the preset value. In this way, the electronic device determines whether there is buffer data related to the playback video, which can reduce unnecessary execution steps and improve the execution efficiency of the code.
  • the first control includes a texture view control or an independent layer view control. In this way, the accuracy of determining whether the video application interface is stuck can be improved, which helps to reduce the stuck of the video application interface.
  • an embodiment of the present application provides a device for detecting video freeze, which may be an electronic device, or a chip or chip system in an electronic device.
  • the device may include a processing unit and a display unit.
  • the processing unit is used to implement the operation performed by the electronic device in the first aspect or any possible implementation of the first aspect. Processing related to any method.
  • the display unit is used to implement any method related to display performed by the electronic device in the first aspect or any possible implementation of the first aspect.
  • the processing unit may be a processor.
  • the device may also include a storage unit, which may be a memory.
  • the storage unit is used to store instructions, and the processing unit executes the instructions stored in the storage unit so that the electronic device implements the method described in the first aspect or any possible implementation of the first aspect.
  • the processing unit may be a processor.
  • the processing unit executes the instructions stored in the storage unit so that the electronic device implements the method described in the first aspect or any possible implementation of the first aspect.
  • the storage unit may be a storage unit in the chip (for example, a register, a cache, etc.), or it may be a storage unit in the electronic device located outside the chip (for example, a read-only memory, a random access memory, etc.).
  • the processing unit is used to determine that the video is stuck.
  • the display unit is used to display the video playback interface.
  • the processing unit is used to determine that no preset event occurs in the video playback interface.
  • the processing unit is used to determine that the layer texture identifier of the view displayed in the first control has not changed, and is also used to determine that a video freeze occurs.
  • the processing unit is used to determine whether a layer texture identifier of a view displayed in a first control has changed, and is also used to determine whether a video freeze has occurred.
  • the processing unit is used to record logs related to the jam.
  • the logs related to lag include one or more of the following: scene resource logs and scene event logs.
  • the processing unit is used to execute processing related to a preset performance indicator.
  • the processing unit is used to determine whether an abnormality has occurred in the CPU utilization of a process, and is also used to perform detections related to the CPU utilization of the process, and is also used to record logs related to the CPU utilization of the process, and is specifically used to increase the operating frequency of the CPU and/or increase the priority of the process; is used to determine whether an abnormality has occurred in the memory application duration, and is also used to perform detections related to the memory application duration, and is also used to record logs related to the memory application duration, and is specifically used to clear the memory of the electronic device; is used to determine whether an abnormality has occurred in the network signal, and is also used to perform detections related to the network signal, and is also used to record logs related to the network signal, and is specifically used to switch the network signal.
  • the processing unit is used to record the frame time, and is also used to set the frame time as a target value, and is specifically used to determine whether there is buffer data related to the playing video.
  • the first control includes a texture view control or an independent layer view control.
  • an embodiment of the present application provides a terminal device, including a processor and a memory, the memory being used to store code instructions, and the processor being used to run the code instructions to execute the method described in the first aspect or any possible implementation of the first aspect.
  • an embodiment of the present application provides a computer-readable storage medium, in which a computer program or instructions are stored.
  • the computer program or instructions When the computer program or instructions are run on a computer, the computer executes the method described in the first aspect or any possible implementation of the first aspect.
  • an embodiment of the present application provides a computer program product including a computer program.
  • the computer program runs on a computer, the computer executes the method described in the first aspect or any possible implementation manner of the first aspect.
  • the present application provides a chip or a chip system, the chip or the chip system comprising at least one A processor and a communication interface, the communication interface and at least one processor are interconnected through a line, and the at least one processor is used to run a computer program or instruction to execute the method described in the first aspect or any possible implementation of the first aspect.
  • the communication interface in the chip can be an input/output interface, a pin or a circuit, etc.
  • the chip or chip system described above in the present application further includes at least one memory, in which instructions are stored.
  • the memory may be a storage unit inside the chip, such as a register, a cache, etc., or a storage unit of the chip (e.g., a read-only memory, a random access memory, etc.).
  • FIG1 is a schematic diagram of an interface of a video application provided in an embodiment of the present application.
  • FIG2 is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application.
  • FIG3 is a schematic diagram of a software structure of an electronic device provided in an embodiment of the present application.
  • FIG4 is a schematic diagram of module interaction of a video freeze detection method provided in an embodiment of the present application.
  • FIG5 is a flow chart of a method for detecting video freeze provided in an embodiment of the present application.
  • FIG6 is a schematic diagram of a video freeze detection method provided in an embodiment of the present application.
  • FIG. 7 is a schematic diagram of the structure of a chip provided in an embodiment of the present application.
  • words such as “first” and “second” are used to distinguish the same or similar items with substantially the same functions and effects.
  • the first chip and the second chip are only used to distinguish different chips, and their order is not limited.
  • words such as “first” and “second” do not limit the quantity and execution order, and words such as “first” and “second” do not necessarily limit them to be different.
  • At least one refers to one or more
  • plural refers to two or more.
  • And/or describes the association relationship of associated objects, indicating that three relationships may exist.
  • a and/or B can represent: A exists alone, A and B exist at the same time, and B exists alone, where A and B can be singular or plural.
  • the character “/” generally indicates that the previous and next associated objects are in an “or” relationship.
  • At least one of the following items or similar expressions refers to any combination of these items, including any combination of single items or plural items.
  • At least one of a, b, or c can represent: a, b, c, a-b, a--c, b-c, or a-b-c, where a, b, c can be single or multiple.
  • the electronic device can detect application jams based on a detection method for software drawing frames.
  • the detection method may include counting the time intervals of frame images drawn on the application interface.
  • the electronic device can call the doframe() method for drawing frame images to determine whether the application is jammed.
  • the electronic device can determine in the doframe() method for drawing frame images whether the time interval between the current frame image and the drawing of the previous frame image exceeds a preset threshold. If it exceeds the preset threshold, it can be considered that the application interface is jammed. Otherwise, it can be considered that the application interface is not jammed.
  • the electronic device can detect application jams based on the layer blending service SurfaceFlinger. For example, when the electronic device cannot send frames stably, the update time interval between the previous layer and the next layer may be unstable for a period of time, causing the application interface to jam.
  • the SurfaceFlinger service can count the update time intervals of the layers. If the update time interval of the layers exceeds the preset value, it can be considered that the application interface is jammed. Otherwise, it can be considered that the application interface is not jammed.
  • the above two implementations of detecting application freezes can be understood as detecting based on the update frequency of the application interface. However, it is impossible to detect the scenario where the application interface is still being refreshed but part of the content in the application interface is not displayed.
  • a of FIG. 1 is an interface of a video application, which includes a video playback control 100 and an avatar display control 101, wherein the avatar display control 101 can display animation effects.
  • the interface of the video application freezes, in a possible scenario, the video playback control 100 has no displayed content, but the avatar display control 101 can still display animation effects, which can be animation effects locally loaded by the video application.
  • the video playback control 100 may not be able to refresh the video at a certain frame rate, thereby causing the video playback control 100 to play stuck.
  • the animation effect of the avatar display control 101 is an animation effect loaded locally by the video application, it does not involve the network environment, and does not require the electronic device to parse the network data packet, so the interface of the video application can still continue to refresh the avatar display control 101 at a certain frame rate, that is, the refresh rate of the video application interface remains unchanged, and the electronic device is still drawing frame pictures at a certain time interval.
  • the detection method determines that the time interval between drawing the frame image of the current video application interface and drawing the previous frame image does not exceed the preset threshold, it is considered that there is no freeze in the interface of the video application.
  • the video freeze detection method provided in the embodiment of the present application can detect the frame rate refresh of the drawing control that does not load local data when the electronic device detects the freeze of the video application interface, so as to avoid The impact of controls that load local data improves the accuracy of judging whether the video application interface is stuck, which helps reduce the stuckness of the video application interface.
  • the electronic device of the embodiment of the present application may also be a terminal device in any form.
  • the electronic device may include a handheld device, a vehicle-mounted device, etc.
  • some electronic devices are: mobile phones, tablet computers, PDAs, laptop computers, mobile internet devices (MID), wearable devices, virtual reality (VR) devices, augmented reality (AR) devices, wireless terminals in industrial control, wireless terminals in self-driving, wireless terminals in remote medical surgery, wireless terminals in smart grids, wireless terminals in transportation safety, wireless terminals in smart cities, etc.
  • the present invention relates to wireless terminals in a smart city, wireless terminals in a smart home, cellular phones, cordless phones, session initiation protocol (SIP) phones, wireless local loop (WLL) stations, personal digital assistants (PDA), handheld devices with wireless communication capabilities, computing devices or other processing devices connected to a wireless modem, vehicle-mounted devices, wearable devices, electronic devices in a 5G network, or electronic devices in a future evolved public land mobile network (PLMN), etc.
  • PLMN public land mobile network
  • the electronic device may also be a wearable device.
  • Wearable devices may also be referred to as wearable smart devices, which are a general term for wearable devices that are intelligently designed and developed using wearable technology for daily wear, such as glasses, gloves, watches, clothing, and shoes.
  • a wearable device is a portable device that is worn directly on the body or integrated into the user's clothes or accessories. Wearable devices are not only hardware devices, but also powerful functions achieved through software support, data interaction, and cloud interaction.
  • wearable smart devices include full-featured, large-sized, and fully or partially independent of smartphones, such as smart watches or smart glasses, as well as devices that only focus on a certain type of application function and need to be used in conjunction with other devices such as smartphones, such as various types of smart bracelets and smart jewelry for vital sign monitoring.
  • the electronic device may also be an electronic device in the Internet of Things (IoT) system.
  • IoT is an important part of the future development of information technology. Its main technical feature is to connect objects to the network through communication technology, thereby realizing an intelligent network that interconnects people and machines and things.
  • the electronic devices in the embodiments of the present application may also be referred to as: user equipment (UE), mobile station (MS), mobile terminal (MT), access terminal, user unit, user station, mobile station, mobile station, remote station, remote terminal, mobile device, user terminal, terminal, wireless communication equipment, user agent or user device, etc.
  • UE user equipment
  • MS mobile station
  • MT mobile terminal
  • access terminal user unit, user station, mobile station, mobile station, remote station, remote terminal, mobile device, user terminal, terminal, wireless communication equipment, user agent or user device, etc.
  • the electronic device or each network device includes a hardware layer, an operating system layer running on the hardware layer, and an application layer running on the operating system layer.
  • the hardware layer includes hardware such as a central processing unit (CPU), a memory management unit (MMU), and a memory (also called main memory).
  • the operating system can be any one or more computer operating systems that implement business processing through processes, such as Linux operating system, Unix operating system, Android operating system, iOS operating system, or Windows operating system.
  • the application layer includes applications such as browsers, address books, word processing software, and instant messaging software.
  • FIG2 shows a schematic structural diagram of an electronic device.
  • the electronic device may include a processor 110, an external memory interface 120, an internal memory 121, a universal serial
  • the present invention comprises a universal serial bus (USB) interface 130, a charging management module 140, a power management module 141, a battery 142, an antenna 1, an antenna 2, a mobile communication module 150, a wireless communication module 160, an audio module 170, a speaker 170A, a receiver 170B, a microphone 170C, an earphone interface 170D, a sensor module 180, a button 190, a motor 191, an indicator 192, a camera 193, a display screen 194, and a subscriber identification module (SIM) card interface 195, etc.
  • USB universal serial bus
  • the sensor module 180 may include a pressure sensor 180A, a gyroscope sensor 180B, an air pressure sensor 180C, a magnetic sensor 180D, an acceleration sensor 180E, a distance sensor 180F, a proximity light sensor 180G, a fingerprint sensor 180H, a temperature sensor 180J, a touch sensor 180K, an ambient light sensor 180L, a bone conduction sensor 180M, etc.
  • the structure illustrated in the embodiment of the present invention does not constitute a specific limitation on the electronic device.
  • the electronic device may include more or fewer components than shown in the figure, or combine certain components, or split certain components, or arrange the components differently.
  • the components shown in the figure may be implemented in hardware, software, or a combination of software and hardware.
  • the processor 110 may include one or more processing units, for example, the processor 110 may include an application processor (AP), a modem processor, a graphics processor (GPU), an image signal processor (ISP), a controller, a video codec, a digital signal processor (DSP), a baseband processor, and/or a neural-network processing unit (NPU), etc.
  • AP application processor
  • GPU graphics processor
  • ISP image signal processor
  • DSP digital signal processor
  • NPU neural-network processing unit
  • Different processing units may be independent devices or integrated in one or more processors.
  • the controller can generate operation control signals according to the instruction operation code and timing signal to complete the control of instruction fetching and execution.
  • the processor 110 may also be provided with a memory for storing instructions and data.
  • the memory in the processor 110 is a cache memory.
  • the memory may store instructions or data that the processor 110 has just used or cyclically used. If the processor 110 needs to use the instruction or data again, it may be directly called from the memory. This avoids repeated access, reduces the waiting time of the processor 110, and thus improves the efficiency of the system.
  • the processor 110 may include one or more interfaces.
  • the interface may include an inter-integrated circuit (I2C) interface, an inter-integrated circuit sound (I2S) interface, a pulse code modulation (PCM) interface, a universal asynchronous receiver/transmitter (UART) interface, a mobile industry processor interface (MIPI), a general-purpose input/output (GPIO) interface, a SIM card interface, and/or a USB interface.
  • I2C inter-integrated circuit
  • I2S inter-integrated circuit sound
  • PCM pulse code modulation
  • UART universal asynchronous receiver/transmitter
  • MIPI mobile industry processor interface
  • GPIO general-purpose input/output
  • SIM card interface SIM card interface
  • USB interface USB interface
  • the interface connection relationship between the modules illustrated in the embodiment of the present invention is only a schematic illustration and does not constitute a structural limitation of the electronic device.
  • the electronic device may also adopt different interface connection methods in the above embodiments, or a combination of multiple interface connection methods.
  • the internal memory 121 can be used to store computer executable program codes, and the executable program codes include instructions.
  • the internal memory 121 can include a program storage area and a data storage area.
  • the program storage area can store an operating system, an application required for at least one function (such as a sound playback function, an image playback function, etc.), etc.
  • the data storage area can store data created during the use of the electronic device (such as audio data, a phone book, etc.), etc.
  • the internal memory 121 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, a universal flash storage (UFS)
  • the processor 110 executes various functional applications and data processing of the electronic device by running instructions stored in the internal memory 121 and/or instructions stored in the memory provided in the processor. For example, the method of the embodiment of the present application may be executed.
  • the electronic device implements the display function through a GPU, a display screen 194, and an application processor.
  • the GPU is a microprocessor for image processing, which is connected to the display screen 194 and the application processor.
  • the GPU is used to perform mathematical and geometric calculations for graphics rendering.
  • the processor 110 may include one or more GPUs that execute program instructions to generate or change display information.
  • the electronic device can implement the shooting function through an ISP, a camera 193, a video codec, a GPU, a display screen 194, and an application processor.
  • FIG3 is a software structure block diagram of an electronic device of an embodiment of the present application.
  • the layered architecture divides the software into several layers, each layer has a clear role and division of labor.
  • the layers communicate with each other through software interfaces.
  • the Android system is divided into five layers, from top to bottom, namely, the application layer, the application framework layer, the Android runtime (Android runtime) and the system library, and the kernel layer.
  • the application layer may include a series of application packages. As shown in Figure 3, the application package may include video, live broadcast, video call and other applications. Applications may include system applications and third-party applications.
  • the application framework layer provides application programming interface (API) and programming framework for the applications in the application layer.
  • API application programming interface
  • the application framework layer includes some predefined functions.
  • the application framework layer may include a view system, a window processing device, a performance logging interface, a content provider, and a resource manager, etc.
  • the application framework layer may also include a drawing management class of a view control, a jamming log system, and the like.
  • the view system can be used for interface drawing and event processing of the application.
  • the view system can include texture view controls and independent layer view controls.
  • the window processing device can be used to obtain relevant data of the electronic device window interface, such as the window name, etc.
  • the window processing device can also detect freeze events.
  • the performance logging interface can be used to call the jamming related interface to write the jamming information into the performance jamming log virtual device of the kernel layer.
  • Content providers are used to implement data sharing between different applications, allowing one program to access data in another program while ensuring the security of the accessed data.
  • the resource manager provides various resources for applications, such as localized strings, icons, images, layout files, video files, and so on.
  • Android runtime includes core libraries and virtual machines. Android runtime is responsible for scheduling and management of the Android system.
  • the core library consists of two parts: one part is the function that needs to be called by the Java language, and the other part is the Android core library.
  • the application layer and the application framework layer run in a virtual machine.
  • the virtual machine executes the java files of the application layer and the application framework layer as binary files.
  • the virtual machine is used to perform functions such as object life cycle management, stack management, thread management, security and exception management, and garbage collection.
  • the virtual machine can be used to perform functions such as drawing interface frame images, detecting jam events, and processing jam events.
  • the system library can also be called the Native layer.
  • the Native layer can include multiple functional modules. For example, function libraries, graphics processing libraries (for example, OpenGL ES), etc.
  • the Native layer can also include Zhiwei Test Diagnostic Services.
  • Log maintenance and diagnosis services may include performance log diagnosis services and resource log collection services, etc.
  • the performance log diagnosis service can be used to detect freeze events in the window interface
  • the resource log collection service can be used to cache system resource logs.
  • the function library provides developers with API interfaces for multiple services, making it easy for developers to quickly integrate and implement various functions.
  • the graphics processing library is used to implement 3D graphics drawing, image rendering, synthesis and layer processing.
  • the kernel layer is the layer between hardware and software.
  • the kernel layer may include various driver modules such as display driver and audio driver, and may also include a performance jam log virtual device.
  • the performance jam log virtual device may be used to save the jam information transmitted by the performance logging interface of the application framework layer into the jam log.
  • FIG. 4 uses FIG. 4 as an example to illustrate the interaction between related modules in the video freeze detection method in an embodiment of the present application.
  • the application layer of the electronic device may include video applications, live broadcast applications, and video call applications.
  • the application can use the view control of the application framework layer to draw the interface.
  • the view control may include a texture view control and an independent layer view control.
  • the texture view control may include a TextureView control
  • the independent layer view control may include a SurfaceView control.
  • the texture view control may also be called a texture control
  • the independent layer view control may also be called a canvas control.
  • the application can use different view controls.
  • the application can use the texture control; in the live broadcast scene, the application can use the canvas control.
  • the application can decide which view control to use, and the embodiment of the present application is not limited.
  • the video scene is used as an example for explanation.
  • the application can call related functions in the texture control to implement interface drawing. For example, the application can call a window-canvas association function, which can bind the canvas to the window interface of the electronic device. The application can also call a canvas frame completion function, which can display the content on the canvas on the window interface of the bound electronic device, thereby achieving the effect of video playback. In addition, the application can also call a window-canvas de-association function to destroy the canvas when no interface drawing is needed.
  • a window-canvas association function which can bind the canvas to the window interface of the electronic device.
  • the application can also call a canvas frame completion function, which can display the content on the canvas on the window interface of the bound electronic device, thereby achieving the effect of video playback.
  • the application can also call a window-canvas de-association function to destroy the canvas when no interface drawing is needed.
  • the application can call related functions in the canvas control to implement interface drawing.
  • the application can call a window-canvas association function, which can bind the canvas to the window interface of the electronic device.
  • the application can also call a layer update function, which can update the content on the layer to the window interface of the electronic device, thereby achieving the effect of video playback.
  • the application can also call a window-canvas de-association function to destroy the canvas when no interface drawing is needed.
  • the above canvas frame completion function or layer update function can also call the video frame processing interface in the window processing device to pass the relevant data of each frame of the video playback to the window processing device. In this way, the window processing device can detect whether the current window interface is stuck.
  • the window processing device may include a texture control switching device, a texture control frame length calculation device, a canvas control frame length calculation device, etc.
  • the texture control switching device and the texture control frame length calculation device may be used to detect a jam event in the texture control, and the canvas control frame length calculation device may be used to detect a jam event in the canvas control.
  • the texture control switching device can be used to detect whether a freeze occurs when the window interface switches, wherein the window interface switching may include the electronic device responding to the user's operation of sliding the video and playing the previous video or the next video; the window interface switching may also include the electronic device responding to the user's operation of switching the application to the background and the video interface being cut to the background.
  • the refresh time of the interface reaches the preset time threshold
  • the texture control switching device detects an event of the user sliding the video, it can be considered that the video switching is currently occurring, the refresh time is acceptable, and the interface is not stuck; if the texture control switching device does not detect an event of the user sliding the video, it can be considered that the video switching is currently not occurring, the refresh time is long, and the interface is stuck.
  • the specific value of the preset time threshold is not limited in the embodiment of the present application.
  • the texture control frame length calculation device can be used to detect whether a certain video is stuck during playback. For example, in the same video, when the refresh time interval between the current and subsequent frames exceeds a preset value, if the texture control frame length calculation device detects that the user clicks to pause playback, it can be considered that the video is not stuck; if the texture control frame length calculation device does not detect that the user clicks to pause playback, it can be considered that the video is stuck.
  • the specific value of the preset value is not limited in the embodiment of the present application.
  • the canvas control frame length calculation device can be used to detect whether a freeze occurs during the playback of a certain video. Similarly, the detection process of the canvas control frame length calculation device can refer to the relevant description of the texture control frame length calculation device above, which will not be repeated.
  • the window processing device can set corresponding jamming identification IDs for different types of jamming scenes.
  • the jamming scene detected by the texture control switching device can correspond to the jamming identification ID1
  • the jamming scene detected by the texture control frame length calculation device can correspond to the jamming identification ID2
  • the jamming scene detected by the canvas control frame length calculation device can correspond to the jamming identification ID3.
  • the window processing device When the window processing device detects that the video interface is stuck, the window processing device can pass the stuck information to the performance logging interface, which can write the stuck information to the stuck log in the performance stuck log virtual device of the kernel layer through the log-related interface of the Native layer and the log-related interface of the kernel layer.
  • the performance log diagnosis service in the log maintenance and diagnosis service can analyze and process the jamming events of the video interface according to the jamming logs mentioned above.
  • the performance log diagnosis service can include a performance log reading device, a texture video playback jamming processor, and an independent layer video playback jamming processor.
  • the performance log reading device can poll and read the jam log in the performance jam log virtual device of the kernel layer.
  • the performance log diagnostic service can determine whether the current jam log is analyzed and processed by the texture class video playback jam processor or the independent layer class video playback jam processor based on the jam identifier in the jam log. For example, when the jam identifier is ID1 or ID2, the current jam log can be analyzed and processed by the texture class video playback jam processor; when the jam identifier is ID3, the current jam log can be analyzed and processed by the independent layer class video playback jam processor.
  • the resource log collection service can be called to obtain the system resource log.
  • the system resource log can record in detail various log information when the electronic device is running, including CPU resource information, input and output information, memory information, network status information and other resource log information. Compared with the system resource log, the above jamming log records relatively less information.
  • the jamming log can record the jamming time, jamming Information related to lag, such as scene and lag degree.
  • Fig. 5 shows a schematic flow chart of a video jam detection method.
  • the video jam detection method may include the following processes: (1) a fault collection process, (2) a video playback jam detection process, and (3) a fault diagnosis and recovery process.
  • the fault collection process can be implemented by a fault collection device, which can collect events reported by some system controls in the electronic device.
  • the fault collection device can be understood as the window processing device of the application framework layer in the embodiment corresponding to Figure 4 above
  • the system control can be understood as the texture control or canvas control in the embodiment corresponding to Figure 4 above.
  • the events reported by the system control may include one or more of the following events: Event 1, event of judging the video scene, Event 2, event of change of visible attribute of the control, Event 3, user click event, Event 4, event of change of foreground and background status of the interface.
  • Event 1 Determine the event of the video scene.
  • the rendering thread signal can be transmitted to the fault collection device.
  • the fault collection device can determine whether the current scene is a video scene according to the rendering thread signal.
  • the fault collection device can determine the name of the interface currently displayed by the electronic device and the application package name corresponding to the interface based on the rendering thread signal. It can be understood that a white list of video scenes is set in the electronic device, and the white list may include application package names and interface names. When the interface name determined by the fault collection device matches the interface name in the white list, and the application package name determined by the fault collection device matches the application package name in the white list, it means that the current scene is a video scene; otherwise, the current scene is not a video scene.
  • the fault acquisition device determines that the current scene is a video scene, it can record the cache time FrameBufferReady and the layer texture identifier SurfaceLayerID of the current frame image, where the layer texture identifier can also be called the layer texture ID.
  • Event 2 The event when the visible property of the control changes.
  • the fault collection device can obtain visible property change events of controls such as progress bars.
  • the visible property change events of the controls can include change events of the visible property Visible of the controls and events of transparency change AlphaChanged.
  • the following uses the progress bar control as an example to illustrate visible property change events and transparency change events.
  • the user can click on the video interface to pause the video.
  • the display state of the progress bar control in the electronic device interface can change.
  • the electronic device can report the change value of the visible attribute Visible of the progress bar control. For example, when the value of Visible is true, it means that the progress bar control is visible, and when the value of Visible is false, it means that the progress bar control is invisible.
  • the electronic device can also report the value of the transparency change of the progress bar control. It can be understood that the transparency value can be in the range of 0 to 255. The smaller the value, the more transparent the display of the progress bar control, and the larger the value, the more opaque the display of the progress bar control.
  • Event 3 User click event.
  • the fault collection device can detect various click events InputEvent of the user on the screen of the electronic device.
  • the click event InputEvent can include a press event ACTION_DOWN, a slide event ACTION_MOVE, a lift event ACTION_UP, etc.
  • the fault collection device can obtain the lift event ACTION_UP and record the time of the lift.
  • the electronic device when the electronic device responds to the user clicking to pause the video playback operation, the progress bar control of the electronic device interface can become visible. At this time, the electronic device has both an event of a change in the visible attribute of the progress bar and an event of a user clicking. In other words, the fault collection device can detect the occurrence of event 2 and event 3.
  • Event 4 the event of the foreground and background status change of the interface.
  • the fault collection device can register the event of the interface switching to the background with the electronic device.
  • the electronic device can report the event of the application switching to the background to the fault collection device.
  • the video playback jam detection process can be implemented by a video playback jam detection device.
  • the detection device can be understood as the texture video playback jamming processing of the Native layer in the embodiment corresponding to FIG. 4 above.
  • the video playback jam detection device can determine whether there is buffered data.
  • the video playback jam detection device can obtain the cache time of the previous frame image, and determine whether there is buffered data based on the cache time.
  • the video playback jam detection device can obtain the cache time of the previous frame image from the cache time FrameBufferReady of the frame image recorded by the fault acquisition device.
  • the cache time before the video playback freeze detection device obtains the frame can be 0, that is, there is no buffered data before the first frame.
  • 0 can also be set to other preset values, as long as the preset value can be used to indicate that there is no buffered data before the first frame.
  • step S506 When the cache time of the previous frame image obtained by the video playback jamming detection device is 0, it means that there is no buffered data, and step S506 can be executed; when the cache time of the previous frame image obtained by the video playback jamming detection device is not 0, it means that there is buffered data, and step S502 can be executed.
  • the video playback jam detection device can further determine whether the video texture ID has changed. If the video texture ID has changed, it means that the current scene is a video switching scene.
  • the video switching scene may include the electronic device responding to the user's operation of sliding the video to play the previous video or the next video, and may also include the electronic device responding to the user's operation of switching the application to the background, and the current video interface switches to the background. If the video texture ID has not changed, it means that the current scene is a video playback scene.
  • step S505 may be performed. If the video texture ID does not change, step S503 may be performed.
  • S503 Determine whether the frame interval time exceeds threshold A.
  • the video playback freeze detection device can determine whether the interval between the first and second frames exceeds a threshold value A.
  • the threshold value A can be preset by the electronic device, for example, the threshold value A can be set to about 80 milliseconds.
  • the specific value of the threshold value A is not limited in the embodiment of the present application.
  • step S506 can be executed.
  • S504 Determine whether the frame interval time exceeds threshold B.
  • the video playback freeze detection device can determine whether the interval between the previous and next two frames exceeds the threshold B.
  • the threshold B can be preset by the electronic device, for example, the threshold B can be set to about 155 milliseconds.
  • the specific value of the threshold B is not limited in the embodiment of the present application.
  • step S505 When the interval between the current and the next two frames exceeds the threshold value B, it means that the refresh time interval between the two frames is relatively long, and the video playback jam detection device needs to further determine whether the current interface is jammed, and then step S505 can be executed.
  • the interval between the current and the next two frames does not exceed the threshold value B, it means that the current interface is refreshed normally and there is no jamming, and then step S506 can be executed.
  • S505 Determine whether the following events occur: a progress bar state change event, a user click event, or an application switching to the background event.
  • step S502 if the video texture ID has not changed, it indicates a video playback scenario.
  • the video playback freeze detection device can further determine whether the interval time between two frames of images exceeds threshold A due to an event in which the visible properties of the control change and/or a user click event.
  • the video playback freeze detection device can determine whether a control visible property change event and/or a user click event occurs within a period of time, wherein the period of time may cover the time when the video texture ID changes.
  • the specific value of the period of time is not limited in the embodiment of the present application.
  • step S506 If the video playback jam detection device detects an event that a control visible property change has occurred, and/or a user click event, it means that the interval between two frames of images exceeds the threshold value A due to the user's pause operation, and it can be considered that the video playback is not jammed, and step S506 can be executed. If the video playback jam detection device does not detect an event that a control visible property change has occurred, and/or a user click event, it means that the video playback is jammed, and step S508 can be executed.
  • the video playback freeze detection device can further determine whether the interval time between two frames of images exceeds threshold B due to an application background switching event.
  • step S506 If the video playback jam detection device detects that an application background switching event has occurred, it means that the interval between two frames of images exceeds the threshold value B due to the user's background switching operation, and it can be considered that the video playback is not jammed, and step S506 can be executed. If the video playback jam detection device does not detect that an application background switching event has occurred, it means that the video playback is jammed, and step S508 can be executed.
  • the video playback jamming detection device can record the frame time, wherein the frame time can be understood as the cache time FrameBufferReady of the frame image reported to the video playback jamming detection device by the above-mentioned fault collection process.
  • the video playback jamming detection device can record the video texture ID, wherein the video texture ID can be understood as the video texture ID reported to the video playback jamming detection device by the above-mentioned fault collection process.
  • the scene resource log may be a partial system resource log obtained by the video playback jam detection device from the cache of the resource log collection service of the Native layer in the embodiment corresponding to the above FIG4.
  • the video playback jam detection device may record the scene resource log to a disk file of the electronic device, and may also package the disk file corresponding to the scene resource log and upload it to the server, so as to facilitate subsequent analysis of the scene resource log.
  • the scene event log may be a partial jam log obtained from the performance jam log virtual device of the kernel layer in the embodiment corresponding to FIG4.
  • the video playback jam detection device may record the scene event log into a disk file of the electronic device, and may also package the disk file corresponding to the scene event log and upload it to the server, so as to facilitate subsequent analysis of the scene event log.
  • the video playback freeze detection device may also include a running fault diagnosis and recovery device.
  • the specific running fault diagnosis and recovery device can refer to the relevant description in the following (3) fault diagnosis and recovery process, which will not be repeated here.
  • the video playback jam detection device when the video playback jam detection device detects a progress bar state change event and/or a user click event, the video playback jam detection device may set the cache time corresponding to the previous frame image to 0. In this way, when the video playback jam detection device obtains the cache time of the previous frame image as 0, it can also indicate that there is no buffered data before the current frame image. Among them, 0 can also be set to other preset values, as long as the preset value can be used to indicate that there is no buffered data before the current frame image.
  • the video playback freeze detection device can set the cache time corresponding to the Nth frame image to 0, indicating that there is no buffered data before the N+1th frame image. This is because when the electronic device responds to the user clicking to pause playback, the electronic device interface content no longer changes, that is, the electronic device does not refresh the interface, and therefore, it can be considered that there is no buffered data before the N+1th frame image.
  • step S505 can be executed to determine the progress bar state change event and/or the user click event again.
  • the video playback jam detection device can consider the occurrence time of the user click event to be the time when the user clicks the screen plus the preset finger lift time. For example, if the user clicks the screen for 110 milliseconds, the preset finger lift time can be 50 milliseconds. In this way, the video playback jam detection device can consider the occurrence time of the user click event to be about 160 milliseconds. Therefore, when the video playback jam detection device determines that the 120 milliseconds of the cache time corresponding to the previous frame of the image is before 160 milliseconds, it can be considered that a click event has occurred, and the cache time corresponding to the previous frame of the image can also be set to 0. This can help improve the accuracy of the video playback jam detection device in determining jam events.
  • the fault diagnosis and recovery process can be implemented by a fault diagnosis and recovery device.
  • the fault diagnosis and recovery device can execute the following processes: (3.1) the processing flow when the CPU utilization of the process is abnormal, (3.2) the processing flow when the memory application time is abnormal. The processing flow of (3.3) and the processing flow when the network signal is abnormal.
  • the process CPU utilization When the process CPU utilization is greater than or equal to the preset CPU utilization threshold, the process CPU utilization can be considered abnormal, for example, the preset CPU utilization threshold can be 40%. It is understandable that the specific value of the preset CPU utilization threshold is not limited in the embodiment of the present application.
  • the fault diagnosis and recovery device can perform one or more of the following tests: (a) frequency limit detection, (b) scheduling priority detection, and (c) inter-core running time detection.
  • Frequency limiting can be understood as a thermal frequency limiting strategy in electronic devices, and the thermal frequency limiting strategy can be understood as a strategy for limiting the CPU frequency of electronic devices according to a preset temperature threshold.
  • the CPU frequency can also be understood as the operating frequency of the CPU.
  • the electronic device when the temperature of the electronic device reaches a preset temperature threshold, the electronic device can reduce the CPU frequency.
  • the preset temperature threshold can be 40 degrees, and the CPU frequency can be reduced to 2.0 GHz.
  • the specific value of the preset temperature threshold and the reduction range of the CPU frequency are not limited in the embodiments of the present application. It can be understood that frequency limiting can reduce the power consumption of the electronic device while roughly maintaining the basic performance of the electronic device, so that the temperature of the electronic device will not be too high, and the battery life of the electronic device will be increased.
  • relevant logs can be printed into a system resource log.
  • a fault diagnosis and recovery device can obtain thermal frequency limiting strategy related logs from the system resource logs, and record the thermal frequency limiting strategy related logs into a scheduling log of a disk file of the electronic device, to facilitate subsequent analysis and statistics of thermal frequency limiting problems.
  • the fault diagnosis and recovery device can detect the priority level of the current video application. If the priority level of the current video application is low, the fault diagnosis and recovery device can send a message to the CPU scheduling module to instruct the CPU scheduling module to increase the priority level of the video application.
  • the fault diagnosis and recovery device can determine whether the priority of the video application is greater than or equal to a preset priority threshold. If the priority of the video application is greater than or equal to the preset priority threshold, it means that the priority level of the video application is low, and the fault diagnosis and recovery device can send a message to the CPU scheduling module to instruct the CPU scheduling module to increase the priority of the video application.
  • the preset priority threshold can be 120, and the specific value of the preset priority threshold is not limited in the embodiment of the present application.
  • the fault diagnosis and recovery device can record the relevant logs of the CPU scheduling priority into the above-mentioned scheduling log to facilitate the subsequent analysis and statistics of the priority scheduling problem.
  • the electronic device may include multiple CPU large cores and multiple CPU small cores, and the fault diagnosis and recovery device may detect the running time and running frequency of each CPU large core or CPU small core.
  • the fault diagnosis and recovery device may send a message to the CPU scheduling module, instructing the CPU scheduling module to increase the running frequency of the CPU large core, thereby reducing the freeze of the application interface.
  • adjusting the running frequency of the CPU large core can also be called frequency modulation.
  • the application process displayed on the current interface is running on a CPU core with a running frequency of 1.3 GHz. Since the running frequency of the CPU core is relatively low, the application interface is stuck.
  • the diagnostic recovery device can send a message to the CPU scheduling module, instructing the CPU scheduling module to increase the operating frequency of the CPU large core.
  • the CPU scheduling module increases the operating frequency of the CPU large core to 2.0 GHz, the lag of the application interface can be reduced.
  • the fault diagnosis and recovery device can record the relevant logs of the inter-core running time into the above-mentioned scheduling log, so as to facilitate the subsequent analysis and statistics of the inter-core running time problem.
  • the execution order of (a) frequency limit detection, (b) scheduling priority detection and (c) inter-core running time detection may not be limited.
  • the memory application duration of the electronic device exceeds the preset application duration threshold within the preset time period, the memory application duration can be considered abnormal.
  • the preset time period can be 100 milliseconds
  • the preset application duration threshold can be 8 milliseconds.
  • the specific values of the preset time period and the preset application duration threshold are not limited in the present embodiment.
  • the fault diagnosis and recovery device may perform one or more of the following tests: (d) available running memory test, (e) slow path application times test.
  • the fault diagnosis and recovery device can determine the remaining space of the running memory of the electronic device. When the remaining space of the running memory is less than the preset running memory threshold, the fault diagnosis and recovery device can perform a background cleanup operation to free up more running memory space.
  • the corresponding preset operating memory threshold may be 800MB.
  • a background cleaning operation may be performed. It is understandable that different electronic devices have different corresponding operating memory sizes, so different electronic devices may have different preset operating memory thresholds.
  • the specific value of the preset operating memory threshold is not limited in the embodiments of the present application.
  • the fault diagnosis and recovery device can record the relevant logs of the available memory detection into the memory log of the disk file of the electronic device, so as to facilitate the subsequent analysis and statistics of the memory problem.
  • the fault diagnosis and recovery device when the fault diagnosis and recovery device determines that the proportion of the number of times the application applies for the slow path is greater than the preset number of times proportion threshold, and/or the proportion of the duration of the application applying for the slow path is greater than the preset duration proportion threshold, the fault diagnosis and recovery device can perform a background cleaning operation.
  • the preset number of times proportion threshold can be 80%
  • the preset duration proportion threshold can be 20%. The specific values of the preset number of times proportion threshold and the preset duration proportion threshold are not limited in the embodiments of the present application.
  • the fault diagnosis and recovery device can record the relevant logs of the number of slow path application detections into the above-mentioned memory logs, so as to facilitate the subsequent analysis and statistics of the slow path application problems.
  • the network signal when the network signal is lower than a preset signal threshold of a full signal, the network signal may be considered abnormal.
  • the preset signal threshold may be 60%. The value is not limited in this embodiment of the present application.
  • the network signal when the network signal strength is not within the preset range of signal strength, the network signal may be considered abnormal.
  • the preset range of signal strength may be -90dBm to -60dBm. It is understood that the specific value of the preset range of signal strength is not limited in the present embodiment.
  • the fault diagnosis and recovery device can perform one or more of the following tests: (f) network delay detection, (g) network packet loss detection.
  • the fault diagnosis and recovery device can perform an operation of switching the network signal.
  • the operation of switching the network signal can include switching the wifi signal to the data signal and switching the data signal to the wifi signal.
  • the preset network delay threshold can be pre-set by the electronic device, for example, the preset network delay threshold can be 100 milliseconds, and the specific value of the preset network delay threshold is not limited in the embodiment of the present application.
  • the fault diagnosis and recovery device can record the relevant logs of network delay detection into the network log of the disk file of the electronic device, so as to facilitate the subsequent analysis and statistics of the network delay problem.
  • the fault diagnosis and recovery device When the fault diagnosis and recovery device detects that the network packet loss rate is greater than a preset packet loss rate threshold, the fault diagnosis and recovery device can perform an operation of switching the network signal.
  • the preset packet loss rate threshold can be pre-set by the electronic device, for example, the preset packet loss rate threshold can be 20%, and the specific value of the preset packet loss rate threshold is not limited in the embodiment of the present application.
  • the fault diagnosis and recovery device can record relevant logs of network packet loss detection into the above-mentioned network log to facilitate subsequent analysis and statistics of the network packet loss problem.
  • the execution order of (3.1) the processing flow when the process CPU utilization is abnormal, (3.2) the processing flow when the memory application duration is abnormal, and (3.3) the processing flow when the network signal is abnormal may not be limited.
  • FIG6 shows a video freeze detection method according to an embodiment of the present application.
  • the method comprises:
  • An electronic device displays a video playback interface, where the video playback interface includes a first control and a second control, where the first control is used to display a first type of view, and the second control is used to display a second type of view, where the first type of view does not include views loaded locally from the electronic device; and the second type of view includes views loaded locally from the electronic device.
  • the first control can be understood as the video playback control 100 in the embodiment corresponding to the above-mentioned Figure 1
  • the second control can be understood as the avatar display control 101 in the embodiment corresponding to the above-mentioned Figure 1
  • the first type of view can be a view of the video playback scene displayed in the first control, a view of the video switching scene, etc.
  • S602 When a frame interval of the view displayed in the first control is greater than or equal to a preset value, the electronic device determines that a video freeze occurs; wherein a frame interval of the view displayed in the second control is less than, equal to or greater than the preset value.
  • the preset value can be pre-set by the electronic device, and the specific value of the preset value is not limited in the embodiment of the present application.
  • the preset value can be understood as the threshold value A or threshold value B in the embodiment corresponding to FIG. 5 above.
  • the electronic device can print a log related to the stuck, and can also The jamming is processed accordingly, which can help reduce the jamming of the video interface.
  • the electronic device can detect the drawing controls that do not load local data without being affected by the controls that load local data, which can improve the accuracy of judging whether the video application interface is freezed, thereby helping to reduce the freeze of the video application interface.
  • the electronic device may also include: the electronic device determines that no preset event occurs in the video playback interface, the preset event includes one or more of the following: an event in which a progress bar control in the video playback interface is triggered, an event for indicating that the video playback interface is paused, and an event for indicating that the video playback interface is switched to the background.
  • the event of the progress bar control in the video playback interface being triggered can be understood as event 2 in the embodiment corresponding to FIG. 5 above
  • the event of the visible attribute change of the control can be understood as event 3 in the embodiment corresponding to FIG. 5 above
  • the user click event and the event for indicating the switching of the video playback interface to the background can be understood as event 4 in the embodiment corresponding to FIG. 5 above
  • the specific preset events can refer to the relevant description in the (1) fault collection process of the embodiment corresponding to FIG. 5 above, and will not be repeated here.
  • the electronic device determines whether a preset event occurs in the video playback interface, so as to judge whether the current interface is paused due to an event triggered by the user, and thus can more accurately judge whether the interface is stuck.
  • the preset value includes a first threshold
  • the electronic device determines that the layer texture identifier of the view displayed in the first control has not changed; when the frame interval of the view displayed in the first control is greater than or equal to the preset value, the electronic device determines that the video is stuck, including: when the frame interval of the view displayed in the first control is greater than or equal to the first threshold, and no event of the progress bar control being triggered occurs in the video playback interface, and/or an event for indicating the pause of the video playback interface occurs, the electronic device determines that the video is stuck.
  • the first threshold value can be understood as the threshold value A in the embodiment corresponding to FIG. 5 above.
  • the specific first threshold value can refer to the relevant description of the threshold value A and will not be repeated here.
  • the layer texture identifier can be understood as the layer texture ID in the embodiment corresponding to FIG. 5 .
  • the specific determination of whether the layer texture identifier has changed can refer to the relevant description in S502 of the embodiment corresponding to FIG. 5 , which will not be repeated here.
  • the electronic device can determine that the video is stuck by referring to the relevant description in S505 of the embodiment corresponding to FIG. 5 , which will not be repeated here.
  • the electronic device determines the event of the progress bar control being triggered and/or the event used to indicate the pause of the video playback interface, and can further determine that the interval time between two frames of images exceeds the threshold A due to the user's pause operation, thereby accurately determining whether the video is stuck.
  • the preset value includes a second threshold
  • the electronic device determines that the layer texture identifier of the view displayed in the first control has changed; when the frame interval of the view displayed in the first control is greater than or equal to the preset value, the electronic device determines that the video is stuck, including: when the frame interval of the view displayed in the first control is greater than or equal to the second threshold, and no event occurs in the video playback interface to indicate switching the video playback interface to the background, the electronic device determines that the video is stuck.
  • the second threshold value can be understood as the threshold value B in the embodiment corresponding to FIG. 5 above.
  • the relevant description of the threshold value B which will not be described in detail.
  • the first threshold value and the second threshold value are The values can be the same or different.
  • the second threshold can be appropriately greater than the first threshold, which can be more in line with the judgment logic of the frame time of the specific scenario.
  • the electronic device may determine that a video freeze occurs by referring to the relevant description in S505 of the embodiment corresponding to FIG. 5 above, and will not repeat them here.
  • the electronic device determines the event for indicating switching the video playback interface to the background, and can further determine that the interval time between two frames of images exceeds the threshold value B due to the user's switching to the background operation, thereby accurately determining whether the video is stuck.
  • the electronic device determining that a video freeze occurs may include: the electronic device recording a log related to the freeze.
  • the logs related to the jamming can refer to the relevant descriptions in S508 and S509 of the embodiment corresponding to FIG. 5 above, and will not be repeated here.
  • Electronic devices can record logs related to lags to facilitate further analysis and statistics of the lags. For example, electronic devices can upload logs to a server, and developers can optimize performance parameters of the electronic devices based on the log information. Electronic devices can also send logs to video application manufacturers, who can further optimize program codes, thereby helping to reduce lags in video playback.
  • the logs related to the jamming include one or more of the following: scene resource logs, scene event logs.
  • the scene resource log can refer to the relevant description in S508 of the embodiment corresponding to FIG5 above, and will not be repeated here.
  • the scene event log can refer to the relevant description in S509 of the embodiment corresponding to FIG5 above, and will not be repeated here.
  • Electronic devices record different types of logs to facilitate subsequent different types of analysis of the jam, thereby analyzing and solving certain types of problems in a targeted manner.
  • the electronic device determining that the video is stuck may include: the electronic device performing processing related to preset performance indicators; the preset performance indicators include one or more of the following: process CPU utilization of the electronic device, memory application duration of the electronic device, and network signal of the electronic device.
  • the electronic device performs processing related to the preset performance indicators, which can refer to the relevant description in the (3) fault diagnosis and recovery process of the embodiment corresponding to FIG. 5 above, and will not be repeated here. After the electronic device performs relevant processing on the above preset performance indicators, the performance of the electronic device can be improved, thereby helping to reduce the freeze of the video interface.
  • the electronic device performs processing related to preset performance indicators, which may include: the electronic device determines that the process CPU utilization is abnormal; the electronic device performs detection related to the process CPU utilization, and the detection related to the process CPU utilization includes one or more of the following: frequency limit detection, scheduling priority detection, and inter-core running time detection; the electronic device records logs related to the process CPU utilization; the electronic device increases the CPU operating frequency and/or increases the priority of the process; the electronic device determines that the memory application duration is abnormal; the electronic device performs detection related to the memory application duration, and the detection related to the memory application duration includes one or more of the following: available memory detection, slow path application times detection; the electronic device records logs related to the memory application duration; the electronic device cleans up the memory of the electronic device; the electronic device determines that a network signal occurs Abnormality; the electronic device performs detection related to the network signal, and the detection related to the network signal includes one or more of the following: network delay detection, network packet loss detection; the electronic device records
  • the electronic device performs processing related to the preset performance indicators, which can refer to the relevant description in the (3) fault diagnosis and recovery process of the embodiment corresponding to FIG. 5 above, and will not be repeated here. After the electronic device performs relevant processing on the above preset performance indicators, the performance of the electronic device can be improved, thereby helping to reduce the freeze of the video interface.
  • the method may also include: when the electronic device determines that a preset condition is met, the electronic device records the frame time; when the electronic device determines that a preset event occurs in the video playback interface, the electronic device sets the frame time to a target value; the electronic device determines that there is buffered data related to playing the video, including: when the frame time is not the target value, the electronic device determines that there is buffered data related to playing the video; wherein, meeting the preset condition includes any one of the following: there is no buffered data related to playing the video; there is buffered data related to playing the video, and the frame interval of the view displayed in the first control is less than the preset value.
  • the target value can be used to indicate that there is no buffered data before the current frame image.
  • the target value can be 0 in step S501 of the embodiment corresponding to Figure 5 above, or it can be other values, which is not limited in the embodiment of the present application.
  • An electronic device determining whether there is buffered data related to playing a video can reduce unnecessary execution steps. For example, in the embodiment corresponding to FIG. 5 above, when the electronic device determines that there is no buffered data related to playing a video, the electronic device may not execute steps S502-S505, etc., which can improve the execution efficiency of the code.
  • the first control includes a texture view (TextureView) control or an independent layer view (SurfaceView) control.
  • TextView TextView
  • SurfaceView independent layer view
  • the texture view (TextureView) control or the independent layer view (SurfaceView) control can be a drawing control that does not load local data.
  • the electronic device can improve the accuracy of judging whether the video application interface is stuck, which helps to reduce the stuck of the video application interface.
  • user information including but not limited to user device information, user personal information, etc.
  • data including but not limited to data used for analysis, stored data, displayed data, etc.
  • user information including but not limited to user device information, user personal information, etc.
  • data including but not limited to data used for analysis, stored data, displayed data, etc.
  • the above mainly introduces the solution provided by the embodiment of the present application from the perspective of the method.
  • it includes hardware structures and/or software modules corresponding to the execution of each function.
  • the present application can be implemented in the form of hardware or a combination of hardware and computer software. Whether a function is executed in the form of hardware or computer software driving hardware depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present application.
  • the embodiment of the present application can divide the functional modules of the device implementing the method according to the above method example.
  • each functional module can be divided according to each function, or two or more functions can be integrated into one.
  • the integrated module can be implemented in the form of hardware or software functional modules. It should be noted that the division of modules in the embodiment of the present application is schematic and is only a logical function division. There may be other division methods in actual implementation.
  • FIG7 is a schematic diagram of the structure of a chip provided in an embodiment of the present application.
  • the chip 700 includes one or more (including two) processors 701 , a communication line 702 , a communication interface 703 and a memory 704 .
  • the memory 704 stores the following elements: executable modules or data structures, or a subset thereof, or an extended set thereof.
  • the method described in the above embodiment of the present application can be applied to the processor 701, or implemented by the processor 701.
  • the processor 701 may be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the above method can be completed by an integrated logic circuit of hardware in the processor 701 or instructions in the form of software.
  • the above processor 701 can be a general-purpose processor (for example, a microprocessor or a conventional processor), a digital signal processor (digital signal processing, DSP), an application specific integrated circuit (application specific integrated circuit, ASIC), a field-programmable gate array (field-programmable gate array, FPGA) or other programmable logic devices, discrete gates, transistor logic devices or discrete hardware components.
  • the processor 701 can implement or execute the methods, steps and logic block diagrams related to each processing disclosed in the embodiment of the present application.
  • the steps of the method disclosed in the embodiment of the present application can be directly embodied as being executed by a hardware decoding processor, or can be executed by a combination of hardware and software modules in the decoding processor.
  • the software module can be located in a mature storage medium in the field such as a random access memory, a read-only memory, a programmable read-only memory, or an electrically erasable programmable read-only memory (EEPROM).
  • the storage medium is located in the memory 704, and the processor 701 reads the information in the memory 704 and completes the steps of the above method in combination with its hardware.
  • the processor 701 , the memory 704 , and the communication interface 703 may communicate with each other via the communication line 702 .
  • the instructions stored in the memory for execution by the processor may be implemented in the form of a computer program product, wherein the computer program product may be pre-written in the memory, or may be downloaded and installed in the memory in the form of software.
  • the embodiment of the present application also provides a computer program product including one or more computer instructions.
  • the computer may be a general-purpose computer, a special-purpose computer, a computer network or other programmable device.
  • the computer instructions may be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium.
  • the computer instructions may be transmitted from one website, computer, server or data center to another website, computer, server or data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means.
  • wired e.g., coaxial cable, optical fiber, digital subscriber line (DSL)
  • wireless e.g., infrared, wireless, microwave, etc.
  • the computer-readable storage medium may be any available medium that a computer can store or a data storage device such as a server or data center that includes one or more available media integrated.
  • the available medium may include a magnetic medium (e.g., a floppy disk, a hard disk or a tape), an optical medium (e.g., a digital versatile disc (DVD)), or a semiconductor medium (e.g., a solid state drive (SSD)), etc.
  • Computer-readable media may include computer storage media and communication media, and may also include any computer program that can be transferred from one place to another.
  • Storage media can be any target medium that can be accessed by a computer.
  • the computer-readable medium may include a compact disc read-only memory (CD-ROM), RAM, ROM, EEPROM or other optical disc storage; the computer-readable medium may include a magnetic disk storage or other magnetic disk storage device.
  • any connecting line may also be appropriately referred to as a computer-readable medium.
  • the software is transmitted from a website, server or other remote source using a coaxial cable, fiber optic cable, twisted pair, DSL or wireless technologies such as infrared, radio and microwave
  • the coaxial cable, fiber optic cable, twisted pair, DSL or wireless technologies such as infrared, radio and microwave are included in the definition of the medium.
  • Disks and discs as used herein include compact discs (CDs), laser discs, optical discs, digital versatile discs (DVDs), floppy disks and Blu-ray discs, where disks typically reproduce data magnetically, while optical discs reproduce data optically using lasers.
  • each process and/or box in the flowchart and/or block diagram and the combination of the process and/or box in the flowchart and/or block diagram can be realized by computer program instructions.
  • These computer program instructions can be provided to the processing unit of a general-purpose computer, a special-purpose computer, an embedded processor or other programmable data processing device to produce a machine, so that the instructions executed by the processing unit of the computer or other programmable data processing device produce a device for realizing the function specified in one process or multiple processes in the flowchart and/or one box or multiple boxes in the block diagram.

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Computer Networks & Wireless Communication (AREA)
  • Software Systems (AREA)
  • Databases & Information Systems (AREA)
  • Human Computer Interaction (AREA)
  • Two-Way Televisions, Distribution Of Moving Picture Or The Like (AREA)

Abstract

本申请实施例提供的视频卡顿检测方法及相关装置，涉及终端技术领域。方法包括：电子设备显示视频播放界面，视频播放界面包括第一控件和第二控件，第一控件用于显示第一类视图，第二控件用于显示第二类视图，第一类视图不包括从电子设备本地加载的视图；第二类视图包括从电子设备本地加载的视图；当第一控件中所显示视图的帧间隔大于或等于预设值时，电子设备确定视频发生卡顿；其中，第二控件中所显示视图的帧间隔小于、等于或大于预设值。这样，可以提高判断视频应用界面是否卡顿的准确率，进而有助于减少视频应用界面的卡顿。

Description

视频卡顿检测方法及相关装置
本申请要求于2023年06月15日提交中国国家知识产权局、申请号为202310715421.1、申请名称为“视频卡顿检测方法及相关装置”的中国专利申请的优先权，其全部内容通过引用结合在本申请中。
技术领域
本申请涉及终端技术领域，尤其涉及视频卡顿检测方法及相关装置。
背景技术
随着电子技术的发展，一些电子设备支持视频播放，用户可以通过视频应用观看视频、拍摄视频，以及开设直播等，然而，电子设备在播放视频应用的内容时，电子设备的界面可能出现卡顿现象，影响用户体验。
发明内容
本申请实施例提供的视频卡顿检测方法及相关装置，电子设备在检测视频应用界面的卡顿时，可以对不加载本地数据的绘制控件进行帧率刷新的检测，从而可以不受加载本地数据的控件的影响，提高判断视频应用界面是否卡顿的准确率，进而有助于减少视频应用界面的卡顿。
第一方面，本申请实施例提供的视频卡顿检测方法，方法包括：
电子设备显示视频播放界面，视频播放界面包括第一控件和第二控件，第一控件用于显示第一类视图，第二控件用于显示第二类视图，第一类视图不包括从电子设备本地加载的视图；第二类视图包括从电子设备本地加载的视图；当第一控件中所显示视图的帧间隔大于或等于预设值时，电子设备确定视频发生卡顿；其中，第二控件中所显示视图的帧间隔小于、等于或大于预设值。这样，可以提高判断视频应用界面是否卡顿的准确率，进而有助于减少视频应用界面的卡顿。
一种可能的实现中，电子设备确定视频发生卡顿之前，还包括：电子设备确定视频播放界面中未发生预设事件，预设事件包括下述的一项或多项：视频播放界面中的进度条控件被触发的事件、用于指示暂停播放视频播放界面的事件、用于指示切换视频播放界面到后台的事件。这样，可以判断当前界面是否由于用户触发的事件而导致界面暂停播放，进而可以更为准确的判断界面是否出现卡顿。
一种可能的实现中，预设值包括第一阈值，电子设备确定视频发生卡顿之前，还包括：电子设备确定第一控件中所显示视图的图层纹理标识未变化；当第一控件中所显示视图的帧间隔大于或等于预设值时，电子设备确定视频发生卡顿，包括：当第一控件中所显示视图的帧间隔大于或等于第一阈值，且视频播放界面中未发生进度条控件被触发的事件，和/或用于指示暂停播放视频播放界面的事件时，电子设备确定视频发生卡顿。这样，可以确定两帧图像的间隔时间超过阈值是由于用户的暂停操作导致的，从而准确判断视频是否发生卡顿。
一种可能的实现中，预设值包括第二阈值，电子设备确定视频发生卡顿之前，还包括：电子设备确定第一控件中所显示视图的图层纹理标识发生变化；当第一控件中所显示视图的帧间隔大于或等于预设值时，电子设备确定视频发生卡顿，包括：当第一控件中所显示视图的帧间隔大于或等于第二阈值，且视频播放界面中未发生用于指示切换视频播放界面到后台的事件时，电子设备确定视频发生卡顿。这样，可以进一步确定两帧图像的间隔时间超过阈值是由于用户的切后台操作导致的，从而准确判断视频是否发生卡顿。
一种可能的实现中，电子设备确定视频发生卡顿，包括：电子设备记录与卡顿相关的日志。这样，可以便于后续对该卡顿作进一步分析和统计。
一种可能的实现中，与卡顿相关的日志包括下述的一项或多项：场景资源日志、场景事件日志。这样，电子设备记录不同类型的日志可以便于后续对该卡顿进行不同类型的分析，从而有针对性的分析和解决某一类型的问题。
一种可能的实现中，电子设备确定视频发生卡顿，包括：电子设备执行与预设性能指标相关的处理；预设性能指标包括下述的一项或多项：电子设备的进程CPU利用率、电子设备的内存申请时长、电子设备的网络信号。这样，电子设备对上述预设性能指标进行相关处理后，可以提升电子设备的性能，从而有助于减少视频界面的卡顿。
一种可能的实现中，电子设备执行与预设性能指标相关的处理，包括：电子设备确定进程CPU利用率发生异常；电子设备执行与进程CPU利用率相关的检测，与进程CPU利用率相关的检测包括下述的一项或多项：限频检测、调度优先级检测、核间运行时长检测；电子设备记录与进程CPU利用率相关的日志；电子设备提高CPU的运行频率，和/或提升进程的优先级；电子设备确定内存申请时长发生异常；电子设备执行与内存申请时长相关的检测，与内存申请时长相关的检测包括下述的一项或多项：可用内存检测、申请慢速路径次数检测；电子设备记录与内存申请时长相关的日志；电子设备清理电子设备的内存；电子设备确定网络信号发生异常；电子设备执行与网络信号相关的检测，与网络信号相关的检测包括下述的一项或多项：网络延迟检测、网络丢包检测；电子设备记录与网络信号相关的日志；电子设备切换网络信号。这样，可以提升电子设备的性能，从而有助于减少视频界面的卡顿。
一种可能的实现中，方法还包括：当电子设备确定满足预设条件时，电子设备记录帧时间；当电子设备确定视频播放界面中发生预设事件时，电子设备设置帧时间为目标值；电子设备确定存在与播放视频有关的缓冲数据，包括：当帧时间不为目标值时，电子设备确定存在与播放视频有关的缓冲数据；其中，满足预设条件包括下述的任一项：不存在与播放视频有关的缓冲数据；存在与播放视频有关的缓冲数据、且第一控件中所显示视图的帧间隔小于预设值。这样，电子设备确定是否存在与播放视频有关的缓冲数据可以减少不必要的执行步骤，提升代码的执行效率。
一种可能的实现中，第一控件包括纹理视图控件或独立图层视图控件。这样，可以提高判断视频应用界面是否卡顿的准确率，有助于减少视频应用界面的卡顿。
第二方面，本申请实施例提供一种视频卡顿检测的装置，该装置可以是电子设备，也可以是电子设备内的芯片或者芯片系统。该装置可以包括处理单元和显示单元。处理单元用于实现第一方面或第一方面的任意一种可能的实现方式中电子设备执行的与 处理相关的任意方法。显示单元用于实现第一方面或第一方面的任意一种可能的实现方式中电子设备执行的与显示相关的任意方法。当该装置是电子设备时，该处理单元可以是处理器。该装置还可以包括存储单元，该存储单元可以是存储器。该存储单元用于存储指令，该处理单元执行该存储单元所存储的指令，以使该电子设备实现第一方面或第一方面的任意一种可能的实现方式中描述的方法。当该装置是电子设备内的芯片或者芯片系统时，该处理单元可以是处理器。该处理单元执行存储单元所存储的指令，以使该电子设备实现第一方面或第一方面的任意一种可能的实现方式中描述的方法。该存储单元可以是该芯片内的存储单元(例如，寄存器、缓存等)，也可以是该电子设备内的位于该芯片外部的存储单元(例如，只读存储器、随机存取存储器等)。
示例性的，处理单元，用于确定视频发生卡顿。显示单元，用于显示视频播放界面。
一种可能的实现方式中，处理单元，用于确定视频播放界面中未发生预设事件。
一种可能的实现方式中，处理单元，用于确定第一控件中所显示视图的图层纹理标识未变化，还用于确定视频发生卡顿。
一种可能的实现方式中，处理单元，用于确定第一控件中所显示视图的图层纹理标识发生变化，还用于确定视频发生卡顿。
一种可能的实现方式中，处理单元，用于记录与卡顿相关的日志。
一种可能的实现方式中，与卡顿相关的日志包括下述的一项或多项：场景资源日志、场景事件日志。
一种可能的实现方式中，处理单元，用于执行与预设性能指标相关的处理。
一种可能的实现方式中，处理单元，用于确定进程CPU利用率发生异常，还用于执行与进程CPU利用率相关的检测，还用于记录与进程CPU利用率相关的日志，具体还用于提高CPU的运行频率，和/或提升进程的优先级；用于确定内存申请时长发生异常，还用于执行与内存申请时长相关的检测，还用于记录与内存申请时长相关的日志，具体还用于清理电子设备的内存；用于确定网络信号发生异常，还用于执行与网络信号相关的检测，还用于记录与网络信号相关的日志，具体还用于切换网络信号。
一种可能的实现方式中，处理单元，用于记录帧时间，还用于设置帧时间为目标值，具体还用于确定存在与播放视频有关的缓冲数据。
一种可能的实现方式中，第一控件包括纹理视图控件或独立图层视图控件。
第三方面，本申请实施例提供一种终端设备，包括处理器和存储器，存储器用于存储代码指令，处理器用于运行代码指令，以执行第一方面或第一方面的任意一种可能的实现方式中描述的方法。
第四方面，本申请实施例提供一种计算机可读存储介质，计算机可读存储介质中存储有计算机程序或指令，当计算机程序或指令在计算机上运行时，使得计算机执行第一方面或第一方面的任意一种可能的实现方式中描述的方法。
第五方面，本申请实施例提供一种包括计算机程序的计算机程序产品，当计算机程序在计算机上运行时，使得计算机执行第一方面或第一方面的任意一种可能的实现方式中描述的方法。
第六方面，本申请提供一种芯片或者芯片系统，该芯片或者芯片系统包括至少一 个处理器和通信接口，通信接口和至少一个处理器通过线路互联，至少一个处理器用于运行计算机程序或指令，以执行第一方面或第一方面的任意一种可能的实现方式中描述的方法。其中，芯片中的通信接口可以为输入/输出接口、管脚或电路等。
在一种可能的实现中，本申请中上述描述的芯片或者芯片系统还包括至少一个存储器，该至少一个存储器中存储有指令。该存储器可以为芯片内部的存储单元，例如，寄存器、缓存等，也可以是该芯片的存储单元(例如，只读存储器、随机存取存储器等)。
应当理解的是，本申请的第二方面至第六方面与本申请的第一方面的技术方案相对应，各方面及对应的可行实施方式所取得的有益效果相似，不再赘述。
附图说明
图1为本申请实施例提供的一种视频应用的界面的示意图；
图2为本申请实施例提供的一种电子设备的结构示意图；
图3为本申请实施例提供的一种电子设备的软件结构示意图；
图4为本申请实施例提供的一种视频卡顿检测方法的模块交互示意图；
图5为本申请实施例提供的一种视频卡顿检测方法的流程示意图；
图6为本申请实施例提供的一种视频卡顿检测方法的示意图；
图7为本申请实施例提供的一种芯片的结构示意图。
具体实施方式
为了便于清楚描述本申请实施例的技术方案，以下，对本申请实施例中所涉及的部分术语和技术进行简单介绍：
1、术语
在本申请的实施例中，采用了“第一”、“第二”等字样对功能和作用基本相同的相同项或相似项进行区分。例如，第一芯片和第二芯片仅仅是为了区分不同的芯片，并不对其先后顺序进行限定。本领域技术人员可以理解“第一”、“第二”等字样并不对数量和执行次序进行限定，并且“第一”、“第二”等字样也并不限定一定不同。
需要说明的是，本申请实施例中，“示例性的”或者“例如”等词用于表示作例子、例证或说明。本申请中被描述为“示例性的”或者“例如”的任何实施例或设计方案不应被解释为比其他实施例或设计方案更优选或更具优势。确切而言，使用“示例性的”或者“例如”等词旨在以具体方式呈现相关概念。
本申请实施例中，“至少一个”是指一个或者多个，“多个”是指两个或两个以上。“和/或”，描述关联对象的关联关系，表示可以存在三种关系，例如，A和/或B，可以表示：单独存在A，同时存在A和B，单独存在B的情况，其中A，B可以是单数或者复数。字符“/”一般表示前后关联对象是一种“或”的关系。“以下至少一项(个)”或其类似表达，是指的这些项中的任意组合，包括单项(个)或复数项(个)的任意组合。例如，a，b，或c中的至少一项(个)，可以表示：a，b，c，a-b，a--c，b-c，或a-b-c，其中a，b，c可以是单个，也可以是多个。
随着电子技术的发展，一些电子设备支持视频播放，用户可以通过视频应用观看 视频、拍摄视频，以及开设直播等，然而，电子设备在播放视频应用的内容时，电子设备的界面可能出现卡顿，影响用户体验。因此电子设备也越来越重视对视频应用的卡顿检测，这样可以在检测到应用发生卡顿时电子设备能够进行相应处理，从而减少视频应用的卡顿。
一种可能检测应用卡顿的实现中，电子设备可以基于软件绘制框架的检测方法对应用进行卡顿检测。该检测方法可以包括对应用界面绘制的帧图片进行时间间隔统计，例如电子设备可以调用绘制帧图片的方法doframe()来判断应用是否出现卡顿。电子设备可以在绘制帧图片的doframe()方法中判断当前帧图片距离绘制上一帧图片的时间间隔是否超出预设阈值，若是超出预设阈值，则可以认为应用界面出现卡顿，否则，可以认为应用界面没有出现卡顿。
另一种可能检测应用卡顿的实现中，电子设备可以基于图层混合服务SurfaceFlinger对应用进行卡顿检测。例如当电子设备不能稳定的送帧时，可能会出现在一段时间内，前一图层和后一图层的更新时间间隔不稳定，从而导致应用界面出现卡顿。SurfaceFlinger服务可以统计图层的更新时间间隔，若图层的更新时间间隔超出预设值，则可以认为应用界面出现卡顿，否则，可以认为应用界面没有出现卡顿。
然而，上述检测应用卡顿的两种实现可以理解为是基于应用界面的更新频率来检测的。但是，对于应用界面仍在不断刷新，而应用界面中的部分内容不显示的场景无法进行检测。
示例性的，图1的a为某个视频应用的界面，该界面中包括视频播放控件100和头像显示控件101，其中，头像显示控件101可以展示动画效果。当该视频应用的界面出现卡顿时，可能的场景中，视频播放控件100没有显示的内容，但是头像显示控件101仍然可以显示动画效果，该动画效果可以为视频应用本地加载的动画效果。
可以理解的是，如图1的b所示，在上述的卡顿场景中，由于网络信号差、电子设备的CPU没资源去解析网络数据包、电子设备的编解码器出现问题等原因可能导致视频播放控件100不能以一定的帧率刷新视频，从而使得视频播放控件100出现播放卡顿。但是由于头像显示控件101的动画效果为视频应用本地加载的动画效果，不涉及网络环境，也不需要电子设备进行网络数据包解析，因此视频应用的界面仍然可以继续以一定的帧率刷新头像显示控件101，也就是说，该视频应用界面的刷新率不变，电子设备仍在以一定的时间间隔绘制帧图片。
当采用上述基于软件绘制框架的检测方法对该视频应用进行卡顿检测时，由于该检测方法判断绘制当前视频应用界面的帧图片距离绘制上一帧图片的时间间隔未超出预设阈值，因此会认为视频应用的界面没有出现卡顿。
当采用上述基于图层混合服务SurfaceFlinger的检测方法对该视频应用进行卡顿检测时，由于该检测方法判断该视频应用界面的前一图层和后一图层的更新时间间隔未超出预设值，因此会认为视频应用界面没有出现卡顿。
由此可知，采用上述的两种检测方法无法准确的检测到视频应用的界面是否出现卡顿，进而不会对卡顿进行处理，使得视频应用界面的卡顿较为频繁。
有鉴于此，本申请实施例提供的视频卡顿检测方法，电子设备在检测视频应用界面的卡顿时，可以对不加载本地数据的绘制控件进行帧率刷新的检测，从而可以不受 加载本地数据的控件的影响，提高判断视频应用界面是否卡顿的准确率，进而有助于减少视频应用界面的卡顿。
可以理解的是，本申请实施例的电子设备也可以为任意形式的终端设备，例如，电子设备可以包括手持式设备、车载设备等。例如，一些电子设备为：手机(mobile phone)、平板电脑、掌上电脑、笔记本电脑、移动互联网设备(mobile internet device，MID)、可穿戴设备，虚拟现实(virtual reality，VR)设备、增强现实(augmented reality，AR)设备、工业控制(industrial control)中的无线终端、无人驾驶(self driving)中的无线终端、远程手术(remote medical surgery)中的无线终端、智能电网(smart grid)中的无线终端、运输安全(transportation safety)中的无线终端、智慧城市(smart city)中的无线终端、智慧家庭(smart home)中的无线终端、蜂窝电话、无绳电话、会话启动协议(session initiation protocol，SIP)电话、无线本地环路(wireless local loop，WLL)站、个人数字助理(personal digital assistant，PDA)、具有无线通信功能的手持设备、计算设备或连接到无线调制解调器的其它处理设备、车载设备、可穿戴设备，5G网络中的电子设备或者未来演进的公用陆地移动通信网络(public land mobile network，PLMN)中的电子设备等，本申请实施例对此并不限定。
作为示例而非限定，在本申请实施例中，该电子设备还可以是可穿戴设备。可穿戴设备也可以称为穿戴式智能设备，是应用穿戴式技术对日常穿戴进行智能化设计、开发出可以穿戴的设备的总称，如眼镜、手套、手表、服饰及鞋等。可穿戴设备即直接穿在身上，或是整合到用户的衣服或配件的一种便携式设备。可穿戴设备不仅仅是一种硬件设备，更是通过软件支持以及数据交互、云端交互来实现强大的功能。广义穿戴式智能设备包括功能全、尺寸大、可不依赖智能手机实现完整或者部分的功能，例如：智能手表或智能眼镜等，以及只专注于某一类应用功能，需要和其它设备如智能手机配合使用，如各类进行体征监测的智能手环、智能首饰等。
此外，在本申请实施例中，电子设备还可以是物联网(internet of things，IoT)系统中的电子设备，IoT是未来信息技术发展的重要组成部分，其主要技术特点是将物品通过通信技术与网络连接，从而实现人机互连，物物互连的智能化网络。
本申请实施例中的电子设备也可以称为：用户设备(user equipment，UE)、移动台(mobile station，MS)、移动终端(mobile terminal，MT)、接入终端、用户单元、用户站、移动站、移动台、远方站、远程终端、移动设备、用户终端、终端、无线通信设备、用户代理或用户装置等。
在本申请实施例中，电子设备或各个网络设备包括硬件层、运行在硬件层之上的操作系统层，以及运行在操作系统层上的应用层。该硬件层包括中央处理器(central processing unit，CPU)、内存管理单元(memory management unit，MMU)和内存(也称为主存)等硬件。该操作系统可以是任意一种或多种通过进程(process)实现业务处理的计算机操作系统，例如，Linux操作系统、Unix操作系统、Android操作系统、iOS操作系统或windows操作系统等。该应用层包含浏览器、通讯录、文字处理软件、即时通信软件等应用。
示例性的，图2示出了电子设备的结构示意图。
电子设备可以包括处理器110，外部存储器接口120，内部存储器121，通用串行 总线(universal serial bus，USB)接口130，充电管理模块140，电源管理模块141，电池142，天线1，天线2，移动通信模块150，无线通信模块160，音频模块170，扬声器170A，受话器170B，麦克风170C，耳机接口170D，传感器模块180，按键190，马达191，指示器192，摄像头193，显示屏194，以及用户标识模块(subscriber identification module，SIM)卡接口195等。其中传感器模块180可以包括压力传感器180A，陀螺仪传感器180B，气压传感器180C，磁传感器180D，加速度传感器180E，距离传感器180F，接近光传感器180G，指纹传感器180H，温度传感器180J，触摸传感器180K，环境光传感器180L，骨传导传感器180M等。
可以理解的是，本发明实施例示意的结构并不构成对电子设备的具体限定。在本申请另一些实施例中，电子设备可以包括比图示更多或更少的部件，或者组合某些部件，或者拆分某些部件，或者不同的部件布置。图示的部件可以以硬件，软件或软件和硬件的组合实现。
处理器110可以包括一个或多个处理单元，例如：处理器110可以包括应用处理器(application processor，AP)，调制解调处理器，图形处理器(graphics processing unit，GPU)，图像信号处理器(image signal processor，ISP)，控制器，视频编解码器，数字信号处理器(digital signal processor，DSP)，基带处理器，和/或神经网络处理器(neural-network processing unit，NPU)等。其中，不同的处理单元可以是独立的器件，也可以集成在一个或多个处理器中。
控制器可以根据指令操作码和时序信号，产生操作控制信号，完成取指令和执行指令的控制。
处理器110中还可以设置存储器，用于存储指令和数据。在一些实施例中，处理器110中的存储器为高速缓冲存储器。该存储器可以保存处理器110刚用过或循环使用的指令或数据。如果处理器110需要再次使用该指令或数据，可从上述存储器中直接调用。避免了重复存取，减少了处理器110的等待时间，因而提高了系统的效率。
在一些实施例中，处理器110可以包括一个或多个接口。接口可以包括集成电路(inter-integrated circuit，I2C)接口，集成电路内置音频(inter-integrated circuit sound，I2S)接口，脉冲编码调制(pulse code modulation，PCM)接口，通用异步收发传输器(universal asynchronous receiver/transmitter，UART)接口，移动产业处理器接口(mobile industry processor interface，MIPI)，通用输入输出(general-purpose input/output，GPIO)接口，SIM卡接口，和/或USB接口等。
可以理解的是，本发明实施例示意的各模块间的接口连接关系，只是示意性说明，并不构成对电子设备的结构限定。在本申请另一些实施例中，电子设备也可以采用上述实施例中不同的接口连接方式，或多种接口连接方式的组合。
内部存储器121可以用于存储计算机可执行程序代码，可执行程序代码包括指令。内部存储器121可以包括存储程序区和存储数据区。其中，存储程序区可存储操作系统，至少一个功能所需的应用程序(比如声音播放功能，图像播放功能等)等。存储数据区可存储电子设备使用过程中所创建的数据(比如音频数据，电话本等)等。此外，内部存储器121可以包括高速随机存取存储器，还可以包括非易失性存储器，例如至少一个磁盘存储器件，闪存器件，通用闪存存储器(universal flash storage，UFS) 等。处理器110通过运行存储在内部存储器121的指令，和/或存储在设置于处理器中的存储器的指令，执行电子设备的各种功能应用以及数据处理。例如，可以执行本申请实施例的方法。
电子设备通过GPU，显示屏194，以及应用处理器等实现显示功能。GPU为图像处理的微处理器，连接显示屏194和应用处理器。GPU用于执行数学和几何计算，用于图形渲染。处理器110可包括一个或多个GPU，其执行程序指令以生成或改变显示信息。电子设备可以通过ISP，摄像头193，视频编解码器，GPU，显示屏194以及应用处理器等实现拍摄功能。
图3是本申请实施例的电子设备的软件结构框图。分层架构将软件分成若干个层，每一层都有清晰的角色和分工。层与层之间通过软件接口通信。在一些实施例中，将Android系统分为五层，从上至下分别为应用程序层，应用程序框架层，安卓运行时(Android runtime)和系统库，以及内核层。
应用程序层可以包括一系列应用程序包。如图3所示，应用程序包可以包括视频类、直播类、视频通话类等应用程序。应用程序可以包括系统应用和三方应用。
应用程序框架层为应用程序层的应用程序提供应用编程接口(application programming interface，API)和编程框架。应用程序框架层包括一些预先定义的函数。
如图3所示，应用程序框架层可以包括视图系统、窗口处理装置、性能日志记录接口、内容提供器和资源管理器等。从外，应用程序框架层还可以包括视图控件的绘制管理类、卡顿日志系统等。
视图系统可以用于应用程序的界面绘制和事件处理。该视图系统中可以包括纹理视图控件和独立图层视图控件。
窗口处理装置可以用于获取电子设备窗口界面的相关数据，例如窗口名称等。窗口处理装置还可以进行卡顿事件的检测。
性能日志记录接口可以用于调用卡顿相关接口将卡顿信息写入内核层的性能卡顿日志虚拟设备中。
内容提供器用于在不同的应用程序之间实现数据共享的功能，允许一个程序访问另一个程序中的数据，同时还能保证被访问的数据的安全性。
资源管理器为应用程序提供各种资源，例如本地化字符串、图标、图片、布局文件、视频文件等等。
Android runtime包括核心库和虚拟机。Android runtime负责安卓系统的调度和管理。
核心库包含两部分：一部分是java语言需要调用的功能函数，另一部分是安卓的核心库。
应用程序层和应用程序框架层运行在虚拟机中。虚拟机将应用程序层和应用程序框架层的java文件执行为二进制文件。虚拟机用于执行对象生命周期的管理，堆栈管理，线程管理，安全和异常的管理，以及垃圾回收等功能。例如本申请实施例中，虚拟机可以用于执行界面帧图像的绘制、卡顿事件的检测、卡顿事件的处理等功能。
系统库也可以称为Native层，Native层可以包括多个功能模块。例如：函数库(function libraries)、图形处理库(例如：OpenGL ES)等。Native层还可以包括日 志维测诊断服务。
日志维测诊断服务可以包括性能日志诊断服务和资源日志采集服务等。其中，性能日志诊断服务可以用于检测窗口界面的卡顿事件，资源日志采集服务可以用于缓存系统资源日志。
函数库为开发者提供多种服务的API接口，方便开发者快速集成并实现各种功能。
图形处理库用于实现三维图形绘图、图像渲染、合成和图层处理等。
内核层是硬件和软件之间的层。内核层可以包括显示驱动、音频驱动等各种驱动模块，还可以包括性能卡顿日志虚拟设备。该性能卡顿日志虚拟设备可以用于将应用程序框架层的性能日志记录接口传递的卡顿信息保存到卡顿日志中。
需要说明的是，本申请实施例仅以安卓系统举例来说明，在其他操作系统中(例如Windows系统，IOS系统等)，只要各个功能模块实现的功能和本申请的实施例类似，也能实现本申请的方案。
下面以图4为例说明本申请实施例的视频卡顿检测方法中相关模块间的交互。
电子设备的应用程序层中可以包括视频类应用、直播类应用、和视频通话类应用等。当应用想要指示电子设备显示某个界面时，该应用可以使用应用程序框架层的视图控件实现界面的绘制。该视图控件可以包括纹理视图控件和独立图层视图控件，例如纹理视图控件可以包括TextureView控件，独立图层视图控件可以包括SurfaceView控件。其中，该纹理视图控件也可以称为纹理控件，该独立图层视图控件也可以称为画布控件。
可以理解的是，针对不同的视频场景，应用可以使用不同的视图控件，例如视频播放场景下，应用可以使用纹理控件；直播场景下，应用可以使用画布控件，应用可以自行决定使用哪种视图控件，本申请实施例不作限定。为了便于描述，后续以视频场景为例进行说明。
当应用使用纹理控件时，该应用可以调用纹理控件中的相关函数实现界面的绘制。示例性的，应用可以调用窗口-画布关联函数，该函数可以将画布和电子设备的窗口界面进行绑定。应用还可以调用画布帧完成函数，该函数可以将画布上的内容展示到已绑定的电子设备的窗口界面上，从而实现视频播放的效果。此外，应用还可以在不需要进行界面绘制时，调用窗口-画布解关联函数，将画布销毁。
类似的，当应用使用画布控件时，该应用可以调用画布控件中的相关函数实现界面的绘制。示例性的，应用可以调用窗口-画布关联函数，该函数可以将画布和电子设备的窗口界面进行绑定。应用还可以调用图层更新函数，该函数可以将图层上的内容更新到电子设备的窗口界面上，从而实现视频播放的效果。此外，应用还可以在不需要进行界面绘制时，调用窗口-画布解关联函数，将画布销毁。
此外，上述的画布帧完成函数或图层更新函数还可以调用窗口处理装置中的视频帧处理接口，将视频播放的每帧图像的相关数据传递到窗口处理装置中。这样，该窗口处理装置可以检测当前窗口界面是否发生卡顿。
该窗口处理装置可以包括纹理控件切换装置、纹理控件帧长计算装置、画布控件帧长计算装置等。其中，纹理控件切换装置和纹理控件帧长计算装置可以用于检测纹理控件中的卡顿事件，画布控件帧长计算装置可以用于检测画布控件中的卡顿事件。
其中，纹理控件切换装置可以用于检测窗口界面发生切换时是否出现卡顿，其中，窗口界面发生切换可以包括电子设备响应于用户滑动视频的操作，播放上个视频或者下个视频；窗口界面发生切换还可以包括电子设备响应于用户将应用切换到后台的操作，视频界面被切到后台。
示例性的，在界面的刷新时间达到预设时间阈值时，若纹理控件切换装置检测到用户滑动视频的事件，则可以认为当前发生视频切换，该刷新时间是可以接收的，界面没有发生卡顿；若纹理控件切换装置没有检测到用户滑动视频的事件，则可以认为当前没有发生视频切换，该刷新时间较长，界面发生了卡顿。其中，本申请实施例对预设时间阈值的具体值不作限定。
纹理控件帧长计算装置可以用于检测在某个视频的播放过程中是否发生卡顿。示例性的，在同一个视频中，当前、后两帧图像的刷新时间间隔超过预设值时，如果纹理控件帧长计算装置检测到用户点击暂停播放的事件时，可以认为该视频没有出现卡顿；如果纹理控件帧长计算装置没有检测到用户点击暂停播放的事件时，可以认为该视频出现卡顿。其中，本申请实施例对预设值的具体值不作限定。
画布控件帧长计算装置可以用于检测在某个视频的播放过程中是否发生卡顿。类似的，该画布控件帧长计算装置的检测流程可以参考上述纹理控件帧长计算装置的相关描述，不再赘述。
可以理解的是，窗口处理装置可以为不同类型的卡顿场景设置对应的卡顿标识ID，例如纹理控件切换装置检测的卡顿场景可以对应有卡顿标识ID1，纹理控件帧长计算装置检测的卡顿场景可以对应有卡顿标识ID2，画布控件帧长计算装置检测的卡顿场景可以对应有卡顿标识ID3。
当窗口处理装置检测到视频界面发生卡顿时，窗口处理装置可以将卡顿相关的信息传递给性能日志记录接口，该性能日志记录接口可以通过Native层的日志相关接口和内核层的日志相关接口将卡顿相关的信息写入到内核层的性能卡顿日志虚拟设备中的卡顿日志中。
在Native层中，日志维测诊断服务中的性能日志诊断服务可以根据上述的卡顿日志对视频界面的卡顿事件进行分析和处理。其中，性能日志诊断服务可以包括性能日志读取装置、纹理类视频播放卡顿处理器、独立图层类视频播放卡顿处理器。
可能的实现中，性能日志读取装置可以轮询读取内核层的性能卡顿日志虚拟设备中的卡顿日志。性能日志诊断服务可以根据该卡顿日志中的卡顿标识确定当前的卡顿日志由纹理类视频播放卡顿处理器进行分析和处理，还是由独立图层类视频播放卡顿处理器进行分析和处理。例如，当卡顿标识为ID1或者ID2时，当前的卡顿日志可以由纹理类视频播放卡顿处理器进行分析和处理；当卡顿标识为ID3时，当前的卡顿日志可以由独立图层类视频播放卡顿处理器进行分析和处理。
当纹理类视频播放卡顿处理器或者独立图层类视频播放卡顿处理器对卡顿事件进行分析和处理时，可以调用资源日志采集服务获取系统资源日志。可以理解的是，该系统资源日志可以详细的记录电子设备运行时的各种日志信息，包括CPU资源信息、输入和输出信息、内存信息、网络状态信息等各种资源日志信息。相对于该系统资源日志，上述的卡顿日志记录的信息则相对较少，该卡顿日志可以记录卡顿时间、卡顿 场景、卡顿程度等与卡顿相关的信息。
具体纹理类视频播放卡顿处理器或者独立图层类视频播放卡顿处理器对于卡顿事件的分析和处理流程，可以参照下述图5对应的实施例的(2)视频播放卡顿检测流程中的相关描述，不再赘述。
图5示出了视频卡顿检测方法的流程示意图。该视频卡顿检测方法可以包括下述的流程(1)故障采集流程，(2)视频播放卡顿检测流程，以及(3)故障诊断恢复流程。
(1)故障采集流程。
故障采集流程可以由故障采集装置实现，该故障采集装置可以采集到电子设备中的一些系统控件上报的事件。其中，该故障采集装置可以理解为上述图4对应的实施例中应用程序框架层的窗口处理装置，该系统控件可以理解为上述图4对应的实施例中的纹理控件或者画布控件。该系统控件上报的事件可以包括下述一项或多项事件：事件1、判断视频场景的事件，事件2、控件可见属性变更的事件，事件3、用户点击事件，事件4、界面前后台状态变更的事件。
事件1、判断视频场景的事件。
在电子设备的界面渲染过程中，系统控件在收到应用的渲染线程信号时，可以将该渲染线程信号传递给故障采集装置。该故障采集装置可以根据该渲染线程信号判断当前是否为视频场景。
可能的实现中，故障采集装置可以根据渲染线程信号确定当前电子设备显示的界面名称，以及该界面对应的应用包名。可以理解的是，电子设备中设置有视频场景的白名单，该白名单中可以包括应用包名和界面名称。当故障采集装置确定的界面名称和该白名单中的界面名称匹配，以及故障采集装置确定的应用包名和该白名单中的应用包名匹配时，说明当前为视频场景；否则，当前不是视频场景。
若故障采集装置确定当前为视频场景，则可以记录当前该帧图像的缓存时间FrameBufferReady和图层纹理标识SurfaceLayerID，其中，该图层纹理标识也可以称为图层纹理ID。
事件2、控件可见属性变更的事件。
故障采集装置可以获取进度条等控件的可见属性变化事件。该控件的可见属性变化事件可以包括控件的可见属性Visible的变化事件和透明度变化AlphaChanged的事件。为了便于描述，下面以进度条控件为例，说明可见属性的变化事件和透明度变化的事件。
可能的实现中，用户可以点击视频界面暂停播放视频，电子设备响应于用户的点击事件，电子设备界面中进度条控件的显示状态可以发生变化，此时，电子设备可以上报该进度条控件的可见属性Visible的变化值，例如当Visible取值为true时，说明进度条控件可见，当Visible取值为false时，说明进度条控件不可见。此外，电子设备还可以上报该进度条控件的透明度变化的值。可以理解的是，该透明度的值可以在0～255取值范围内，取值越小，该进度条控件的显示越透明，取值越大，该进度条控件的显示越不透明。
事件3、用户点击事件。
故障采集装置可以检测到用户对电子设备屏幕的各种点击事件InputEvent，例如点击事件InputEvent可以包括按下事件ACTION_DOWN、滑动事件ACTION_MOVE、抬起事件ACTION_UP等。当用户按下屏幕后再将手指抬起时，故障采集装置可以获取到抬起事件ACTION_UP，并可以记录抬起的时间。
可以理解的是，当电子设备响应于用户点击暂停播放视频的操作时，电子设备界面的进度条控件可以变为可见状态。此时，电子设备既发生了进度条可见属性变更的事件，也发生了用户点击事件。也就是说，故障采集装置可以检测到事件2和事件3的发生。
事件4，界面前后台状态变更的事件。
故障采集装置可以向电子设备注册界面切换后台的事件，当应用切换到后台时，电子设备可以向故障采集装置上报应用切换到后台的事件。
(2)视频播放卡顿检测流程。
视频播放卡顿检测流程可以由视频播放卡顿检测装置实现，该视频播放卡顿检
测装置可以理解为上述图4对应的实施例中Native层的纹理类视频播放卡顿处理
器和独立图层类视频播放卡顿处理器。S501、判断是否有缓冲数据。
本申请实施例中，视频播放卡顿检测装置可以判断是否有缓冲数据。可能的实现中，视频播放卡顿检测装置可以获取上一帧图像的缓存时间，根据该缓存时间判断是否有缓冲数据。其中，视频播放卡顿检测装置可以从故障采集装置记录的帧图像的缓存时间FrameBufferReady中获取上一帧图像的缓存时间。
例如，若当前为第一帧图像，则视频播放卡顿检测装置获取该帧之前的缓存时间可以为0，也就是说，在第一帧图像之前没有缓冲数据。其中，0也可以设置为其他的预设值，只要该预设值能用于表示第一帧图像之前没有缓冲数据即可。
当视频播放卡顿检测装置获取的上一帧图像的缓存时间为0时，说明没有缓冲数据，则可以执行步骤S506；当视频播放卡顿检测装置获取的上一帧图像的缓存时间不为0时，说明有缓冲数据，则可以执行步骤S502。
S502、判断视频纹理ID是否发生变化。
当视频播放卡顿检测装置判断有缓冲数据时，视频播放卡顿检测装置可以进一步判断视频纹理ID是否发生变化。若视频纹理ID发生变化，说明当前为视频切换场景。例如视频切换场景可以包括电子设备响应于用户滑动视频的操作，播放上个视频或者下个视频，还可以包括电子设备响应于用户将应用切换到后台的操作，当前视频界面切换到后台。若视频纹理ID未发生变化，说明当前为视频播放场景。
若视频纹理ID发生变化，则可以执行步骤S505。若视频纹理ID未发生变化，则可以执行步骤S503。
S503、判断帧间隔时间是否超过阈值A。
当视频纹理ID未发生变化时，说明当前为视频播放场景，视频播放卡顿检测装置可以判断前、后两帧图像的间隔时间是否超过阈值A。其中，阈值A可以由电子设备预先设置，例如阈值A可以设置为80毫秒左右。对于阈值A的具体取值，本申请实施例不作限定。
当前、后两帧图像的间隔时间超过阈值A时，说明两帧图像的刷新时间间隔相对 较长，视频播放卡顿检测装置需要进一步判断当前界面是否出现卡顿，则可以执行步骤S505。当前、后两帧图像的间隔时间没有超过阈值A时，说明前界面刷新正常，没有出现卡顿，则可以执行步骤S506。
S504、判断帧间隔时间是否超过阈值B。
当视频纹理ID发生变化时，说明当前为视频切换场景，视频播放卡顿检测装置可以判断前、后两帧图像的间隔时间是否超过阈值B。其中，阈值B可以由电子设备预先设置，例如阈值B可以设置为155毫秒左右。对于阈值B的具体取值，本申请实施例不作限定。
当前、后两帧图像的间隔时间超过阈值B时，说明两帧图像的刷新时间间隔相对较长，视频播放卡顿检测装置需要进一步判断当前界面是否出现卡顿，则可以执行步骤S505。当前、后两帧图像的间隔时间没有超过阈值B时，说明当前界面刷新正常，没有出现卡顿，则可以执行步骤S506。
S505、判断是否有以下事件：进度条状态变化事件、用户点击事件、应用切后台事件。
一种可能的场景中，在上述步骤S502中，若视频纹理ID未发生变化，说明为视频播放场景，则视频播放卡顿检测装置可以进一步判断是否由于发生了控件可见属性变更的事件，和/或用户点击事件，使得两帧图像的间隔时间超过阈值A。
可以理解的是，视频播放卡顿检测装置可以判断在一段时间内是否发生了控件可见属性变更的事件，和/或用户点击事件，其中，该段时间可以涵盖视频纹理ID发生变化的时间。本申请实施例对该段时间的具体值不作限定。
若视频播放卡顿检测装置检测到发生了控件可见属性变更的事件，和/或用户点击事件，则说明两帧图像的间隔时间超过阈值A是由于用户的暂停操作导致的，可以不认为视频播放出现卡顿，则可以执行步骤S506。若视频播放卡顿检测装置未检测到发生控件可见属性变更的事件，和/或用户点击事件，说明视频播放出现卡顿，则可以执行步骤S508。
另一种可能的场景中，在上述步骤S502中，若视频纹理ID发生变化，说明为视频切换场景，则视频播放卡顿检测装置可以进一步判断是否由于发生应用切后台事件，使得两帧图像的间隔时间超过阈值B。
若视频播放卡顿检测装置检测到发生了应用切后台事件，则说明两帧图像的间隔时间超过阈值B是由于用户的切后台操作导致的，可以不认为视频播放出现卡顿，则可以执行步骤S506。若视频播放卡顿检测装置未检测到发生应用切后台事件，说明视频播放出现卡顿，则可以执行步骤S508。
S506、记录帧时间。
视频播放卡顿检测装置可以记录帧时间，其中，该帧时间可以理解为上述故障采集流程向视频播放卡顿检测装置上报的帧图像的缓存时间FrameBufferReady。
S507、记录视频纹理ID。
视频播放卡顿检测装置可以记录视频纹理ID，其中，该视频纹理ID可以理解为上述故障采集流程向视频播放卡顿检测装置上报的视频纹理ID。
S508、记录场景资源日志。
该场景资源日志可以为视频播放卡顿检测装置从上述图4对应的实施例中Native层的资源日志采集服务的缓存中获取的部分系统资源日志。视频播放卡顿检测装置可以将该场景资源日志记录到电子设备的磁盘文件中，还可以将该场景资源日志对应的磁盘文件打包上传给服务器，便于后续对该场景资源日志进行分析。
S509、记录场景事件日志。
该场景事件日志可以为从上述图4对应的实施例中内核层的性能卡顿日志虚拟设备中获取的部分卡顿日志。视频播放卡顿检测装置可以将该场景事件日志记录到电子设备的磁盘文件中，还可以将该场景事件日志对应的磁盘文件打包上传给服务器，便于后续对该场景事件日志进行分析。
S510、运行故障诊断恢复装置。
视频播放卡顿检测装置中还可以包括运行故障诊断恢复装置，具体运行故障诊断恢复装置可以参照下述(3)故障诊断恢复流程中的相关描述，不再赘述。
可选的，在上述的步骤S501中，当视频播放卡顿检测装置检测到发生进度条状态变化事件和/或用户点击事件时，视频播放卡顿检测装置可以将上一帧图像对应的缓存时间设置为0。这样，当视频播放卡顿检测装置获取上一帧图片的缓存时间为0时，也可以说明当前帧图像之前没有缓冲数据。其中，0也可以设置为其他的预设值，只要该预设值能用于表示当前帧图像之前没有缓冲数据即可。
示例性的，若第N+1帧图像不为第一帧图像，假设第N帧图像的缓存时间为100毫秒，第N+1帧图像的缓存时间为200毫秒。当在110毫秒时发生了进度条状态变化事件，和/或用户点击事件时，视频播放卡顿检测装置可以将第N帧图像对应的缓存时间设置为0，表示第N+1帧图像之前没有缓冲数据。这是由于电子设备响应于用户点击暂停播放的操作时，电子设备界面内容不再变化，也就是说电子设备不进行界面的刷新，因此，可以认为第N+1帧图像之前没有缓冲数据。
可能的场景中，当在110毫秒时发生了进度条状态变化事件，和/或用户点击事件时，视频播放卡顿检测装置将上一帧图像对应的缓存时间设置为0。但是由于电子设备送帧速度较快，在120毫秒时电子设备又多送了一帧图像，导致上一帧图像对应的缓存时间变为120毫秒，使得视频播放卡顿检测装置判断当前有缓冲数据，执行步骤S502。当视频播放卡顿检测装置检测帧间隔时间超过阈值时，可以执行步骤S505，再次判断进度条状态变化事件，和/或用户点击事件。
可以理解的是，视频播放卡顿检测装置可以认为用户点击事件的发生时间为用户点击屏幕的时间加上预设的手指抬起时间，例如用户点击屏幕的时间为110毫秒，预设的手指抬起时间可以为50毫秒，这样，视频播放卡顿检测装置可以认为用户点击事件的发生时间为160毫秒左右。因此，当视频播放卡顿检测装置判断上一帧图像对应的缓存时间的120毫秒在160毫秒之前时，可以认为在发生了点击事件，还可以将上一帧图像对应的缓存时间设置为0，这样，可以有助于提升视频播放卡顿检测装置判断卡顿事件的准确性。
(3)故障诊断恢复流程。
故障诊断恢复流程可以由故障诊断恢复装置实现，该故障诊断恢复装置可以执行的流程包括(3.1)进程CPU利用率异常时的处理流程，(3.2)内存申请时长异常时 的处理流程，和(3.3)网络信号异常时的处理流程。
(3.1)进程CPU利用率异常时的处理流程。
当进程CPU利用率大于或等于预设CPU利用率阈值时，可以认为进程CPU利用率异常，例如预设CPU利用率阈值可以为40％。可以理解的是，对于预设CPU利用率阈值的具体取值，本申请实施例不作限定。
当进程CPU利用率异常时，故障诊断恢复装置可以执行下述一项或多项检测：(a)限频检测、(b)调度优先级检测、(c)核间运行时长检测。
(a)限频检测。
限频可以理解为电子设备中的热限频策略，该热限频策略可以理解为电子设备根据预设温度阈值进行CPU频点的限制策略。其中，CPU频点也可以理解为CPU的工作频率。
示例性的，当电子设备的温度达到预设温度阈值时，电子设备可以降低CPU频点。例如预设温度阈值可以为40度，CPU频点可以将到2.0GHz。其中，对于预设温度阈值的具体取值，以及CPU频点的降低幅度，本申请实施例不作限定。可以理解的是，限频可以使得在大体维持电子设备基本性能的情况下，能够降低电子设备的功耗，从而使得电子设备的温度不会太高，增加电子设备的续航时间。
当电子设备执行热限频策略时，可以将相关日志打印到系统资源日志中，故障诊断恢复装置从系统资源日志中可以获取到热限频策略相关日志，并将该热限频策略相关日志记录到电子设备的磁盘文件的调度日志中，便于后续对热限频问题进行分析和统计。
(b)调度优先级检测。
故障诊断恢复装置可以检测当前视频应用的优先级，若当前视频应用的优先级等级较低，则故障诊断恢复装置可以向CPU调度模块发送消息，指示CPU调度模块将该视频应用的优先级调高。
可能的实现中，故障诊断恢复装置可以判断该视频应用的优先级是否大于或等于预设优先级阈值。若该视频应用的优先级大于或等于预设优先级阈值，说明该视频应用的优先级等级较低，则故障诊断恢复装置可以向CPU调度模块发送消息，指示CPU调度模块将该视频应用的优先级调高。其中，预设优先级阈值可以为120，对于预设优先级阈值的具体取值，本申请实施例不作限定。
可以理解的是，故障诊断恢复装置可以将CPU调度优先级的相关日志记录到上述的调度日志中，便于后续对优先级调度问题进行分析和统计。
(c)核间运行时长检测。
电子设备中可以包括多个CPU大核和多个CPU小核，故障诊断恢复装置可以检测各个CPU大核或CPU小核的运行时间，以及运行频率。为了便于后续描述，以CPU大核为例，当由于某个CPU大核的运行频率较低而导致应用界面出现卡顿时，故障诊断恢复装置可以向CPU调度模块发送消息，指示CPU调度模块提高该CPU大核的运行频率，从而减少应用界面的卡顿。其中，调整CPU大核的运行频率也可以称为调频。
例如，当前界面显示的应用进程运行在某个CPU大核上，该CPU大核的运行频率为1.3GHz。由于该CPU大核的运行频率比较低导致应用界面出现了卡顿，则故障 诊断恢复装置可以向CPU调度模块发送消息，指示CPU调度模块提高该CPU大核的运行频率，当CPU调度模块将该CPU大核的运行频率提升到2.0GHz时，则可以减少应用界面的卡顿。
可以理解的是，故障诊断恢复装置可以将核间运行时长的相关日志记录到上述的调度日志中，便于后续对核间运行时长问题进行分析和统计。
本申请实施例中，(a)限频检测、(b)调度优先级检测和(c)核间运行时长检测的执行顺序可以不作限定。
(3.2)内存申请时长异常时的处理流程。
当在预设时间段内，电子设备的内存申请时长超过预设申请时长阈值时，可以认为内存申请时长异常，例如预设时间段可以为100毫秒，预设申请时长阈值可以为8毫秒。其中，对于预设时间段和预设申请时长阈值的具体取值，本申请实施例不作限定。
当内存申请时长异常时，故障诊断恢复装置可以执行下述一项或多项检测：(d)可用运行内存检测、(e)申请慢速路径次数检测。
(d)可用运行内存检测。
当内存申请时长异常时，故障诊断恢复装置可以判断电子设备的运行内存的剩余空间大小。当运行内存的剩余空间小于预设运行内存阈值时，故障诊断恢复装置可以执行清理后台的操作，从而释放更多运行内存的空间。
示例性的，对于运行内存为8G的电子设备，对应的预设运行内存阈值可以为800MB。当故障诊断恢复装置判断运行内存的剩余空间小于800MB时，可以执行清理后台的操作。可以理解的是，不同的电子设备对应的运行内存大小不同，因此不同的电子设备可以由不同的预设运行内存阈值。对于预设运行内存阈值的具体值，本申请实施例不作限定。
可以理解的是，故障诊断恢复装置可以将可用内存检测的相关日志记录到电子设备的磁盘文件的内存日志中，便于后续对内存问题进行分析和统计。
(e)申请慢速路径次数检测。
当应用向电子设备申请内存时，如果电子设备没有足够的内存空间，则电子设备需要先进行内存清理，回收部分内存后，才能为该应用分配内存，这样分配内存的方式可以理解为申请慢速路径。
可能的实现中，当故障诊断恢复装置判断应用申请慢速路径次数占比大于预设次数占比阈值，和/或应用申请慢速路径的时长占比大于预设时长占比阈值时，故障诊断恢复装置可以执行清理后台的操作。示例性的，预设次数占比阈值可以为80％，预设时长占比阈值可以为20％。对于预设次数占比阈值和预设时长占比阈值的具体值，本申请实施例不作限定。
可以理解的是，故障诊断恢复装置可以将申请慢速路径次数检测的相关日志记录到上述的内存日志中，便于后续对申请慢速路径问题进行分析和统计。
(3.3)网络信号异常时的处理流程。
一种可能的实现中，当网络信号低于满格信号的预设信号阈值时，可以认为网络信号异常，例如预设信号阈值可以为60％。可以理解的是，对于预设信号阈值的具体 取值，本申请实施例不作限定。
另一种可能的实现中，当网络信号强度不在信号强度的预设范围内时，可以认为网络信号异常，例如信号强度的预设范围可以为-90dBm～-60dBm。可以理解的是，对于信号强度的预设范围的具体取值，本申请实施例不作限定。
当网络信号异常时，故障诊断恢复装置可以执行下述一项或多项检测：(f)网络延迟检测、(g)网络丢包检测。
(f)网络延迟检测。
当故障诊断恢复装置检测到网络延迟大于预设网络延迟阈值时，故障诊断恢复装置可以执行切换网络信号的操作。其中，切换网络信号的操作可以包括wifi信号切换数据信号和数据信号切换wifi信号。预设网络延迟阈值可以由电子设备预先设置，例如预设网络延迟阈值可以为100毫秒，对于预设网络延迟阈值的具体值，本申请实施例不作限定。
可以理解的是，故障诊断恢复装置可以将网络延迟检测的相关日志记录到电子设备的磁盘文件的网络日志中，便于后续对网络延迟问题进行分析和统计。
(g)网络丢包检测。
当故障诊断恢复装置检测到网络丢包率大于预设丢包率阈值时，故障诊断恢复装置可以执行切换网络信号的操作。其中，预设丢包率阈值可以由电子设备预先设置，例如预设丢包率阈值可以为20％，对于预设丢包率阈值的具体值，本申请实施例不作限定。
可以理解的是，故障诊断恢复装置可以将网络丢包检测的相关日志记录到上述网络日志中，便于后续对网络丢包问题进行分析和统计。
本申请实施例中，(3.1)进程CPU利用率异常时的处理流程，(3.2)内存申请时长异常时的处理流程，和(3.3)网络信号异常时的处理流程的执行顺序可以不作限定。
下面通过具体的实施例对本申请实施例的方法进行详细说明。下面的实施例可以相互结合或独立实施，对于相同或相似的概念或过程可能在某些实施例中不再赘述。
图6示出了本申请实施例的视频卡顿检测方法。方法包括：
S601、电子设备显示视频播放界面，视频播放界面包括第一控件和第二控件，第一控件用于显示第一类视图，第二控件用于显示第二类视图，第一类视图不包括从电子设备本地加载的视图；第二类视图包括从电子设备本地加载的视图。
本申请实施例中，第一控件可以理解为上述图1对应的实施例中的视频播放控件100，第二控件可以理解为上述图1对应的实施例中的头像显示控件101，第一类视图可以为第一控件中显示的视频播放场景的视图、视频切换场景的视图等。
S602、当第一控件中所显示视图的帧间隔大于或等于预设值时，电子设备确定视频发生卡顿；其中，第二控件中所显示视图的帧间隔小于、等于或大于预设值。
本申请实施例中，预设值可以由电子设备预先设置，对于预设值的具体取值，本申请实施例不作限定。例如，预设值可以理解为上述图5对应的实施例中的阈值A或阈值B。
电子设备确定视频发生卡顿时，电子设备可以打印该卡顿相关的日志，还可以对 该卡顿进行相关的处理，从而可以有助于减少视频界面的卡顿。
电子设备在检测视频应用界面的卡顿时，可以对不加载本地数据的绘制控件进行检测，而不受加载本地数据的控件的影响，可以提高判断视频应用界面是否卡顿的准确率，进而有助于减少视频应用界面的卡顿。
可选的，在图6对应的实施例的基础上，电子设备确定视频发生卡顿之前，还可以包括：电子设备确定视频播放界面中未发生预设事件，预设事件包括下述的一项或多项：视频播放界面中的进度条控件被触发的事件、用于指示暂停播放视频播放界面的事件、用于指示切换视频播放界面到后台的事件。
本申请实施例中，视频播放界面中的进度条控件被触发的事件可以理解为上述图5对应的实施例中的事件2、控件可见属性变更的事件，用于指示暂停播放视频播放界面的事件可以理解为上述图5对应的实施例中的事件3、用户点击事件，用于指示切换视频播放界面到后台的事件可以理解为上述图5对应的实施例中的事件4、界面前后台状态变更的事件。具体预设事件可以参照上述图5对应的实施例的(1)故障采集流程中的相关描述，不再赘述。
电子设备确定视频播放界面中是否发生预设事件可以判断当前界面是否由于用户触发的事件而导致界面暂停播放，进而可以更为准确的判断界面是否出现卡顿。
可选的，在图6对应的实施例的基础上，预设值包括第一阈值，电子设备确定视频发生卡顿之前，还可以包括：电子设备确定第一控件中所显示视图的图层纹理标识未变化；当第一控件中所显示视图的帧间隔大于或等于预设值时，电子设备确定视频发生卡顿，包括：当第一控件中所显示视图的帧间隔大于或等于第一阈值，且视频播放界面中未发生进度条控件被触发的事件，和/或用于指示暂停播放视频播放界面的事件时，电子设备确定视频发生卡顿。
本申请实施例中，第一阈值可以理解为上述图5对应的实施例中的阈值A，具体第一阈值可以参照阈值A的相关描述，不再赘述。
图层纹理标识可以理解为上述图5对应的实施例中的图层纹理ID，具体判断图层纹理标识是否变化可以参照上述图5对应的实施例的S502中的相关描述，不再赘述。此外，电子设备确定视频发生卡顿可以参照上述图5对应的实施例的S505中的相关描述，不再赘述。
电子设备判断进度条控件被触发的事件，和/或用于指示暂停播放视频播放界面的事件，可以进一步确定两帧图像的间隔时间超过阈值A是由于用户的暂停操作导致的，从而准确判断视频是否发生卡顿。
可选的，在图6对应的实施例的基础上，预设值包括第二阈值，电子设备确定视频发生卡顿之前，还可以包括：电子设备确定第一控件中所显示视图的图层纹理标识发生变化；当第一控件中所显示视图的帧间隔大于或等于预设值时，电子设备确定视频发生卡顿，包括：当第一控件中所显示视图的帧间隔大于或等于第二阈值，且视频播放界面中未发生用于指示切换视频播放界面到后台的事件时，电子设备确定视频发生卡顿。
本申请实施例中，第二阈值可以理解为上述图5对应的实施例中的阈值B，具体第二阈值可以参照阈值B的相关描述，不再赘述。可以理解的是，第一阈值和第二阈 值的取值可以相同，也可以不同。
可能的场景中，由于电子设备切换上个视频或者下个视频，或者切换视频到后台时帧图像刷新的时间相对于同一个视频播放时帧图像刷新的时间较长，因此，第二阈值可以适当大于第一阈值，这样可以更为符合对具体场景的帧时间的判断逻辑。
此外，电子设备确定视频发生卡顿可以参照上述图5对应的实施例的S505中的相关描述，不再赘述。
电子设备判断用于指示切换视频播放界面到后台的事件，可以进一步确定两帧图像的间隔时间超过阈值B是由于用户的切后台操作导致的，从而准确判断视频是否发生卡顿。
可选的，在图6对应的实施例的基础上，电子设备确定视频发生卡顿，可以包括：电子设备记录与卡顿相关的日志。
本申请实施例中，记录与卡顿相关的日志可以参照上述图5对应的实施例的S508和S509中的相关描述，不再赘述。
电子设备记录与卡顿相关的日志可以便于后续对该卡顿作进一步分析和统计，例如，电子设备可以将日志上传给服务器，开发人员可以根据日志信息优化电子设备的性能参数等；电子设备还可以将日志发送给视频应用厂商，视频应用厂商可以对程序代码进一步优化，从而有助于减少视频播放的卡顿。
可选的，在图6对应的实施例的基础上，与卡顿相关的日志包括下述的一项或多项：场景资源日志、场景事件日志。
本申请实施例中，场景资源日志可以参照上述图5对应的实施例的S508中的相关描述，不再赘述。场景事件日志可以参照上述图5对应的实施例的S509中的相关描述，不再赘述。
电子设备记录不同类型的日志可以便于后续对该卡顿进行不同类型的分析，从而有针对性的分析和解决某一类型的问题。
可选的，在图6对应的实施例的基础上，电子设备确定视频发生卡顿，可以包括：电子设备执行与预设性能指标相关的处理；预设性能指标包括下述的一项或多项：电子设备的进程CPU利用率、电子设备的内存申请时长、电子设备的网络信号。
本申请实施例中，电子设备执行与预设性能指标相关的处理可以参照上述图5对应的实施例的(3)故障诊断恢复流程中的相关描述，不再赘述。电子设备对上述预设性能指标进行相关处理后，可以提升电子设备的性能，从而有助于减少视频界面的卡顿。
可选的，在图6对应的实施例的基础上，电子设备执行与预设性能指标相关的处理，可以包括：电子设备确定进程CPU利用率发生异常；电子设备执行与进程CPU利用率相关的检测，与进程CPU利用率相关的检测包括下述的一项或多项：限频检测、调度优先级检测、核间运行时长检测；电子设备记录与进程CPU利用率相关的日志；电子设备提高CPU的运行频率，和/或提升进程的优先级；电子设备确定内存申请时长发生异常；电子设备执行与内存申请时长相关的检测，与内存申请时长相关的检测包括下述的一项或多项：可用内存检测、申请慢速路径次数检测；电子设备记录与内存申请时长相关的日志；电子设备清理电子设备的内存；电子设备确定网络信号发生 异常；电子设备执行与网络信号相关的检测，与网络信号相关的检测包括下述的一项或多项：网络延迟检测、网络丢包检测；电子设备记录与网络信号相关的日志；电子设备切换网络信号。
本申请实施例中，电子设备执行与预设性能指标相关的处理可以参照上述图5对应的实施例的(3)故障诊断恢复流程中的相关描述，不再赘述。电子设备对上述预设性能指标进行相关处理后，可以提升电子设备的性能，从而有助于减少视频界面的卡顿。
可选的，在图6对应的实施例的基础上，方法还可以包括：当电子设备确定满足预设条件时，电子设备记录帧时间；当电子设备确定视频播放界面中发生预设事件时，电子设备设置帧时间为目标值；电子设备确定存在与播放视频有关的缓冲数据，包括：当帧时间不为目标值时，电子设备确定存在与播放视频有关的缓冲数据；其中，满足预设条件包括下述的任一项：不存在与播放视频有关的缓冲数据；存在与播放视频有关的缓冲数据、且第一控件中所显示视图的帧间隔小于预设值。
本申请实施例中，目标值可以用于指示当前帧图像之前没有缓冲数据，例如，目标值可以为上述图5对应的实施例的步骤S501中的0，也可以其他值，本申请实施例不作限定。
电子设备确定是否存在与播放视频有关的缓冲数据可以减少不必要的执行步骤，例如上述图5对应的实施例，当电子设备确定不存在与播放视频有关的缓冲数据，则电子设备可以不执行步骤S502-步骤S505等步骤，这样可以提升代码的执行效率。
可选的，在图6对应的实施例的基础上，第一控件包括纹理视图(TextureView)控件或独立图层视图(SurfaceView)控件。
本申请实施例中，纹理视图(TextureView)控件或独立图层视图(SurfaceView)控件可以为不加载本地数据的绘制控件。电子设备通过对不加载本地数据的绘制控件进行帧率刷新的检测，可以提高判断视频应用界面是否卡顿的准确率，有助于减少视频应用界面的卡顿。
需要说明的是，本申请所涉及的用户信息(包括但不限于用户设备信息、用户个人信息等)和数据(包括但不限于用于分析的数据、存储的数据、展示的数据等)，均为经用户授权或者经过各方充分授权的信息和数据，并且相关数据的收集、使用和处理需要遵守相关国家和地区的相关法律法规和标准，并提供有相应的操作入口，供用户选择授权或者拒绝。
上述主要从方法的角度对本申请实施例提供的方案进行了介绍。为了实现上述功能，其包含了执行各个功能相应的硬件结构和/或软件模块。本领域技术人员应该很容易意识到，结合本文中所公开的实施例描述的各示例的方法步骤，本申请能够以硬件或硬件和计算机软件的结合形式来实现。某个功能究竟以硬件还是计算机软件驱动硬件的方式来执行，取决于技术方案的特定应用和设计约束条件。专业技术人员可以对每个特定的应用来使用不同方法来实现所描述的功能，但是这种实现不应认为超出本申请的范围。
本申请实施例可以根据上述方法示例对实现该方法的装置进行功能模块的划分，例如可以对应各个功能划分各个功能模块，也可以将两个或两个以上的功能集成在一 个处理模块中。集成的模块既可以采用硬件的形式实现，也可以采用软件功能模块的形式实现。需要说明的是，本申请实施例中对模块的划分是示意性的，仅仅为一种逻辑功能划分，实际实现时可以有另外的划分方式。
如图7示为本申请实施例提供的一种芯片的结构示意图。芯片700包括一个或两个以上(包括两个)处理器701、通信线路702、通信接口703和存储器704。
在一些实施方式中，存储器704存储了如下的元素：可执行模块或者数据结构，或者他们的子集，或者他们的扩展集。
上述本申请实施例描述的方法可以应用于处理器701中，或者由处理器701实现。处理器701可能是一种集成电路芯片，具有信号的处理能力。在实现过程中，上述方法的各步骤可以通过处理器701中的硬件的集成逻辑电路或者软件形式的指令完成。上述的处理器701可以是通用处理器(例如，微处理器或常规处理器)、数字信号处理器(digital signal processing，DSP)、专用集成电路(application specific integrated circuit，ASIC)、现成可编程门阵列(field-programmable gate array，FPGA)或者其他可编程逻辑器件、分立门、晶体管逻辑器件或分立硬件组件，处理器701可以实现或者执行本申请实施例中的公开的各处理相关的方法、步骤及逻辑框图。
结合本申请实施例所公开的方法的步骤可以直接体现为硬件译码处理器执行完成，或者用译码处理器中的硬件及软件模块组合执行完成。其中，软件模块可以位于随机存储器、只读存储器、可编程只读存储器或带电可擦写可编程存储器(electrically erasable programmable read only memory，EEPROM)等本领域成熟的存储介质中。该存储介质位于存储器704，处理器701读取存储器704中的信息，结合其硬件完成上述方法的步骤。
处理器701、存储器704以及通信接口703之间可以通过通信线路702进行通信。
在上述实施例中，存储器存储的供处理器执行的指令可以以计算机程序产品的形式实现。其中，计算机程序产品可以是事先写入在存储器中，也可以是以软件形式下载并安装在存储器中。
本申请实施例还提供一种计算机程序产品包括一个或多个计算机指令。在计算机上加载和执行计算机程序指令时，全部或部分地产生按照本申请实施例的流程或功能。计算机可以是通用计算机、专用计算机、计算机网络或者其他可编程装置。计算机指令可以存储在计算机可读存储介质中，或者从一个计算机可读存储介质向另一计算机可读存储介质传输，例如，计算机指令可以从一个网站站点、计算机、服务器或数据中心通过有线(例如同轴电缆、光纤、数字用户线(digital subscriber line，DSL)或无线(例如红外、无线、微波等)方式向另一个网站站点、计算机、服务器或数据中心进行传输。计算机可读存储介质可以是计算机能够存储的任何可用介质或者是包括一个或多个可用介质集成的服务器、数据中心等数据存储设备。例如，可用介质可以包括磁性介质(例如，软盘、硬盘或磁带)、光介质(例如，数字通用光盘(digital versatile disc，DVD))、或者半导体介质(例如，固态硬盘(solid state disk，SSD))等。
本申请实施例还提供一种计算机可读存储介质。上述实施例中描述的方法可以全部或部分地通过软件、硬件、固件或者其任意组合来实现。计算机可读介质可以包括计算机存储介质和通信介质，还可以包括任何可以将计算机程序从一个地方传送到另 一个地方的介质。存储介质可以是可由计算机访问的任何目标介质。
作为一种可能的设计，计算机可读介质可以包括紧凑型光盘只读储存器(compact disc read-only memory，CD-ROM)、RAM、ROM、EEPROM或其它光盘存储器；计算机可读介质可以包括磁盘存储器或其它磁盘存储设备。而且，任何连接线也可以被适当地称为计算机可读介质。例如，如果使用同轴电缆，光纤电缆，双绞线，DSL或无线技术(如红外，无线电和微波)从网站，服务器或其它远程源传输软件，则同轴电缆，光纤电缆，双绞线，DSL或诸如红外，无线电和微波之类的无线技术包括在介质的定义中。如本文所使用的磁盘和光盘包括光盘(CD)，激光盘，光盘，数字通用光盘(digital versatile disc，DVD)，软盘和蓝光盘，其中磁盘通常以磁性方式再现数据，而光盘利用激光光学地再现数据。
本申请实施例是参照根据本申请实施例的方法、设备(系统)、和计算机程序产品的流程图和/或方框图来描述的。应理解可由计算机程序指令实现流程图和/或方框图中的每一流程和/或方框、以及流程图和/或方框图中的流程和/或方框的结合。可提供这些计算机程序指令到通用计算机、专用计算机、嵌入式处理机或其他可编程数据处理设备的处理单元以产生一个机器，使得通过计算机或其他可编程数据处理设备的处理单元执行的指令产生用于实现在流程图一个流程或多个流程和/或方框图一个方框或多个方框中指定的功能的装置。

Claims (13)

  1. 一种视频卡顿检测方法，其特征在于，所述方法包括：
    电子设备显示视频播放界面，所述视频播放界面包括第一控件和第二控件，所述第一控件用于显示第一类视图，所述第二控件用于显示第二类视图，所述第一类视图不包括从所述电子设备本地加载的视图；所述第二类视图包括从所述电子设备本地加载的视图；
    当所述第一控件中所显示视图的帧间隔大于或等于预设值时，所述电子设备确定视频发生卡顿；其中，所述第二控件中所显示视图的帧间隔小于、等于或大于所述预设值。
  2. 根据权利要求1所述的方法，其特征在于，所述电子设备确定视频发生卡顿之前，还包括：
    所述电子设备确定所述视频播放界面中未发生预设事件，所述预设事件包括下述的一项或多项：所述视频播放界面中的进度条控件被触发的事件、用于指示暂停播放所述视频播放界面的事件、用于指示切换所述视频播放界面到后台的事件。
  3. 根据权利要求1或2所述的方法，其特征在于，所述预设值包括第一阈值，所述电子设备确定视频发生卡顿之前，还包括：
    所述电子设备确定所述第一控件中所显示视图的图层纹理标识未变化；
    当所述第一控件中所显示视图的帧间隔大于或等于预设值时，所述电子设备确定视频发生卡顿，包括：
    当所述第一控件中所显示视图的帧间隔大于或等于所述第一阈值，且所述视频播放界面中未发生所述进度条控件被触发的事件，和/或所述用于指示暂停播放所述视频播放界面的事件时，所述电子设备确定视频发生卡顿。
  4. 根据权利要求1或2所述的方法，其特征在于，所述预设值包括第二阈值，所述电子设备确定视频发生卡顿之前，还包括：
    所述电子设备确定所述第一控件中所显示视图的图层纹理标识发生变化；
    当所述第一控件中所显示视图的帧间隔大于或等于预设值时，所述电子设备确定视频发生卡顿，包括：
    当所述第一控件中所显示视图的帧间隔大于或等于所述第二阈值，且所述视频播放界面中未发生所述用于指示切换所述视频播放界面到后台的事件时，所述电子设备确定视频发生卡顿。
  5. 根据权利要求1-4任一项所述的方法，其特征在于，所述电子设备确定视频发生卡顿，包括：
    所述电子设备记录与所述卡顿相关的日志。
  6. 根据权利要求5所述的方法，其特征在于，所述与所述卡顿相关的日志包括下述的一项或多项：场景资源日志、场景事件日志。
  7. 根据权利要求1-6任一项所述的方法，其特征在于，所述电子设备确定视频发生卡顿，包括：
    所述电子设备执行与所述预设性能指标相关的处理；所述预设性能指标包括下述的一项或多项：所述电子设备的进程CPU利用率、所述电子设备的内存申请时长、所 述电子设备的网络信号。
  8. 根据权利要求7所述的方法，其特征在于，所述电子设备执行与所述预设性能指标相关的处理，包括：
    所述电子设备确定所述进程CPU利用率发生异常；
    所述电子设备执行与所述进程CPU利用率相关的检测，所述与所述进程CPU利用率相关的检测包括下述的一项或多项：限频检测、调度优先级检测、核间运行时长检测；
    所述电子设备记录与所述进程CPU利用率相关的日志；
    所述电子设备提高CPU的运行频率，和/或提升所述进程的优先级；
    所述电子设备确定所述内存申请时长发生异常；
    所述电子设备执行与所述内存申请时长相关的检测，所述与所述内存申请时长相关的检测包括下述的一项或多项：可用内存检测、申请慢速路径次数检测；
    所述电子设备记录与所述内存申请时长相关的日志；
    所述电子设备清理所述电子设备的内存；
    所述电子设备确定所述网络信号发生异常；
    所述电子设备执行与所述网络信号相关的检测，所述与所述网络信号相关的检测包括下述的一项或多项：网络延迟检测、网络丢包检测；
    所述电子设备记录与所述网络信号相关的日志；
    所述电子设备切换网络信号。
  9. 根据权利要求2-8任一项所述的方法，其特征在于，所述方法还包括：
    当所述电子设备确定满足预设条件时，所述电子设备记录帧时间；
    当所述电子设备确定所述视频播放界面中发生所述预设事件时，所述电子设备设置所述帧时间为目标值；
    所述电子设备确定存在与播放视频有关的缓冲数据，包括：
    当所述帧时间不为所述目标值时，所述电子设备确定存在与播放视频有关的缓冲数据；
    其中，所述满足预设条件包括下述的任一项：
    不存在与播放视频有关的缓冲数据；
    存在与播放视频有关的缓冲数据、且所述第一控件中所显示视图的帧间隔小于所述预设值。
  10. 根据权利要求1-9任一项所述的方法，其特征在于，所述第一控件包括纹理视图控件或独立图层视图控件。
  11. 一种电子设备，其特征在于，包括：存储器和处理器，所述存储器用于存储计算机程序，所述处理器用于执行所述计算机程序，以执行如权利要求1-10任一项所述的方法。
  12. 一种计算机可读存储介质，其特征在于，所述计算机可读存储介质存储有指令，当所述指令被执行时，使得计算机执行如权利要求1-10任一项所述的方法。
  13. 一种计算机程序产品，其特征在于，包括计算机程序，当所述计算机程序被运行时，使得电子设备执行如权利要求1-10任一项所述的方法。
PCT/CN2024/081005 2023-06-15 2024-03-11 视频卡顿检测方法及相关装置 Ceased WO2024255342A1 (zh)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202310715421.1A CN119155513B (zh) 2023-06-15 2023-06-15 视频卡顿检测方法及相关装置
CN202310715421.1 2023-06-15

Publications (1)

Publication Number Publication Date
WO2024255342A1 true WO2024255342A1 (zh) 2024-12-19

Family

ID=93808350

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2024/081005 Ceased WO2024255342A1 (zh) 2023-06-15 2024-03-11 视频卡顿检测方法及相关装置

Country Status (2)

Country Link
CN (1) CN119155513B (zh)
WO (1) WO2024255342A1 (zh)

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN106973321A (zh) * 2017-03-31 2017-07-21 广州酷狗计算机科技有限公司 确定视频卡顿的方法及装置
CN110996094A (zh) * 2019-11-01 2020-04-10 北京奇艺世纪科技有限公司 检测视频卡顿的方法、装置、计算机设备及存储介质
CN112511821A (zh) * 2021-02-03 2021-03-16 腾讯科技（深圳）有限公司 视频卡顿检测方法、装置和存储介质
WO2021159609A1 (zh) * 2020-02-11 2021-08-19 深圳壹账通智能科技有限公司 一种视频卡顿识别方法、装置及终端设备
CN114449255A (zh) * 2022-01-12 2022-05-06 杭州网易云音乐科技有限公司 视频卡顿检测方法、装置、计算设备及介质

Family Cites Families (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110300326B (zh) * 2018-03-22 2021-11-26 杭州萤石软件有限公司 一种视频卡顿的检测方法、装置、电子设备及存储介质

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN106973321A (zh) * 2017-03-31 2017-07-21 广州酷狗计算机科技有限公司 确定视频卡顿的方法及装置
CN110996094A (zh) * 2019-11-01 2020-04-10 北京奇艺世纪科技有限公司 检测视频卡顿的方法、装置、计算机设备及存储介质
WO2021159609A1 (zh) * 2020-02-11 2021-08-19 深圳壹账通智能科技有限公司 一种视频卡顿识别方法、装置及终端设备
CN112511821A (zh) * 2021-02-03 2021-03-16 腾讯科技（深圳）有限公司 视频卡顿检测方法、装置和存储介质
CN114449255A (zh) * 2022-01-12 2022-05-06 杭州网易云音乐科技有限公司 视频卡顿检测方法、装置、计算设备及介质

Also Published As

Publication number Publication date
CN119155513B (zh) 2025-09-09
CN119155513A (zh) 2024-12-17

Similar Documents

Publication Publication Date Title
CN109343902B (zh) 音频处理组件的运行方法、装置、终端及存储介质
CN114077529B (zh) 日志上传方法、装置、电子设备及计算机可读存储介质
CN108647051A (zh) 优化策略获取方法、提供方法、装置及设备
CN107196794A (zh) 一种卡顿异常的分析方法及装置
CN107544842A (zh) 应用程序处理方法和装置、计算机设备、存储介质
CN106973321B (zh) 确定视频卡顿的方法及装置
CN111124668B (zh) 内存释放方法、装置、存储介质及终端
US20230042070A1 (en) Recommendation content resource acquisition method and apparatus, device, medium, and program product
CN116196621B (zh) 应用处理方法及相关装置
CN114257502B (zh) 一种日志上报方法及装置
WO2017202175A1 (zh) 一种视频压缩方法、装置及电子设备
CN110750743B (zh) 一种动画播放方法、装置、设备及存储介质
WO2023087875A1 (zh) 进程调度方法和终端设备
CN118433440A (zh) 视频播放方法、装置、电子设备及计算机可读存储介质
CN108197013B (zh) 日志处理方法、装置、终端设备及存储介质
CN109933537B (zh) 一种卡顿检测方法、相关装置、设备及计算机可读介质
US20250335118A1 (en) Method for Adjusting Program Storage Location and Related Apparatus
CN107491349B (zh) 应用程序处理方法和装置、计算机设备、存储介质
CN107688498B (zh) 应用程序处理方法和装置、计算机设备、存储介质
CN119155513B (zh) 视频卡顿检测方法及相关装置
CN116916095B (zh) 一种云视频的平滑显示方法、装置、设备及存储介质
CN114638742B (zh) 场景画面处理方法和电子设备
WO2022262472A1 (zh) 帧率处理方法、装置、存储介质以及终端
CN119003065B (zh) 显示方法、电子设备及系统
CN108235144B (zh) 播放内容获取方法、装置及计算设备

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 24822300

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE