CN113407436A - Play component compatibility detection method and device, computer equipment and storage medium - Google Patents
Play component compatibility detection method and device, computer equipment and storage medium Download PDFInfo
- Publication number
- CN113407436A CN113407436A CN202011213931.1A CN202011213931A CN113407436A CN 113407436 A CN113407436 A CN 113407436A CN 202011213931 A CN202011213931 A CN 202011213931A CN 113407436 A CN113407436 A CN 113407436A
- Authority
- CN
- China
- Prior art keywords
- video frame
- detection
- screen rendering
- rendering environment
- playing
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/36—Preventing errors by testing or debugging software
- G06F11/3668—Software testing
- G06F11/3672—Test management
- G06F11/3684—Test management for test design, e.g. generating new test cases
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Computer Hardware Design (AREA)
- Quality & Reliability (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Two-Way Televisions, Distribution Of Moving Picture Or The Like (AREA)
Abstract
The application relates to the technical field of artificial intelligence, and provides a method and a device for detecting compatibility of a playing component, computer equipment and a storage medium. The method comprises the following steps: loading a playing component to a detection process, creating an off-screen rendering environment, wherein the detection process is a process started when an application program is switched to a background, decoding a target video corresponding to the application program to the off-screen rendering environment through the playing component, extracting a video frame to be detected from the off-screen rendering environment, inputting the video frame to be detected into a preset image detection model, obtaining a video frame detection result, wherein the image detection model is obtained based on image set training including abnormal images, and determining the compatibility detection result of the playing component according to the video frame detection result. The detection process does not depend on the actual operation and feedback of the user, the detection efficiency is improved, and the compatibility detection can be completed under the condition that the user does not sense the compatibility.
Description
Technical Field
The present application relates to the field of artificial intelligence technologies, and in particular, to a method and an apparatus for detecting compatibility of a playback component, a computer device, and a storage medium.
Background
The playing component is an important component in the intelligent device and is playing software for playing multimedia. The system generally configures a system playing component at the time of factory shipment, for example, MediaPlayer in the Android system, that is, a player of the Android system. The system playing component is used as a basic component of the system, and needs to satisfy the function of playing videos in various application programs, so that the compatibility of the playing component needs to be detected.
However, in the conventional method, the detection of the playing component basically depends on feedback information of a user after an actual operation process, and a dynamic detection technology is lacked, so that the efficiency of the compatibility test of the playing component is low.
Disclosure of Invention
In view of the foregoing, it is desirable to provide a method, an apparatus, a computer device and a storage medium for detecting compatibility of a playback component, which can improve testing efficiency.
A play component compatibility detection method comprises the following steps:
loading the playing component to a detection process, and creating an off-screen rendering environment, wherein the detection process is a process started when the application program is switched to a background;
decoding a target video corresponding to the application program to an off-screen rendering environment through a playing component;
extracting a video frame to be detected from an off-screen rendering environment, inputting the video frame to be detected into a preset image detection model to obtain a video frame detection result, wherein the image detection model is obtained based on image set training including abnormal images;
and determining a compatibility detection result of the playing component according to the video frame detection result.
A playback component compatibility detection apparatus, the apparatus comprising:
the loading module is used for loading the playing component to a detection process and creating an off-screen rendering environment, wherein the detection process is a process started when the application program is switched to a background;
the decoding module is used for decoding the target video corresponding to the application program to an off-screen rendering environment through the playing component;
the detection module is used for extracting a video frame to be detected from an off-screen rendering environment, inputting the video frame to be detected into a preset image detection model and obtaining a video frame detection result, wherein the image detection model is obtained based on image set training including abnormal images;
and the compatibility determining module is used for determining the compatibility detection result of the playing component according to the video frame detection result.
A computer device comprising a memory and a processor, the memory storing a computer program, the processor implementing the following steps when executing the computer program:
loading the playing component to a detection process, and creating an off-screen rendering environment, wherein the detection process is a process started when the application program is switched to a background;
decoding a target video corresponding to the application program to an off-screen rendering environment through a playing component;
extracting a video frame to be detected from an off-screen rendering environment, inputting the video frame to be detected into a preset image detection model to obtain a video frame detection result, wherein the image detection model is obtained based on image set training including abnormal images;
and determining a compatibility detection result of the playing component according to the video frame detection result.
A computer-readable storage medium, on which a computer program is stored which, when executed by a processor, carries out the steps of:
loading the playing component to a detection process, and creating an off-screen rendering environment, wherein the detection process is a process started when the application program is switched to a background;
decoding a target video corresponding to the application program to an off-screen rendering environment through a playing component;
extracting a video frame to be detected from an off-screen rendering environment, inputting the video frame to be detected into a preset image detection model to obtain a video frame detection result, wherein the image detection model is obtained based on image set training including abnormal images;
and determining a compatibility detection result of the playing component according to the video frame detection result.
The method, the device, the computer equipment and the storage medium for detecting the compatibility of the playing component load the playing component to the detection process based on the detection process started when the application program is switched to the background, load and start the playing component in the form of the background process, decode the target video corresponding to the application program to the off-screen rendering environment through the playing component by creating the off-screen rendering environment, so as to conveniently realize the video decoding of the playing component under the condition that a user does not sense the video decoding, and because the image detection model is obtained by training based on the image set comprising abnormal images, extract the video frame to be detected from the decoded data of the off-screen rendering environment, input the video frame to be detected into a preset image detection model, judge whether the video frame is abnormal or not, thereby obtaining the compatibility detection result of the playing component based on the video frame detection result, and realizing the compatibility detection of the playing component in the background running process of the application program, the detection process does not depend on the actual operation and feedback of the user, and the detection efficiency is improved.
Drawings
FIG. 1 is a diagram of an exemplary implementation of a method for compatibility detection of playback components;
FIG. 2 is a flowchart illustrating a method for detecting compatibility of playback components in one embodiment;
FIG. 3 is a flowchart illustrating a method for detecting compatibility of a playback component in another embodiment;
FIG. 4 is a flowchart illustrating a method for detecting compatibility of a playback component in accordance with yet another embodiment;
FIG. 5 is a flowchart illustrating a method for detecting compatibility of a playback component in a further embodiment;
FIG. 6 is a flowchart illustrating a method for detecting compatibility of a playback component in another embodiment;
FIG. 7 is a flowchart illustrating a method for detecting compatibility of a playback component in another embodiment;
FIG. 8 is a diagram illustrating a relationship between a main process and a sub-process in a method for detecting compatibility of a playback component according to an embodiment;
FIG. 9 is a flowchart illustrating a method for detecting compatibility of a playback component in another embodiment;
FIG. 10 is a block diagram of a method for Player component compatibility detection in one embodiment;
FIG. 11 is a block diagram showing the structure of a playback component compatibility detection apparatus according to an embodiment;
FIG. 12 is a diagram illustrating an internal structure of a computer device according to an embodiment.
Detailed Description
In order to make the objects, technical solutions and advantages of the present application more apparent, the present application is described in further detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the present application and are not intended to limit the present application.
The play component compatibility detection method provided by the application can be applied to the application environment shown in fig. 1. The terminal 102 and the server 104 communicate through a network, wherein the terminal 102 loads a playing component to a detection process and creates an off-screen rendering environment, the detection process is a process started when an application program is switched to a background, a target video corresponding to the application program is decoded to the off-screen rendering environment through the playing component, a video frame to be detected is extracted from the off-screen rendering environment, the video frame to be detected is input into a preset image detection model, a video frame detection result is obtained, the image detection model is obtained by the server 104 based on an image set including abnormal images and is sent to the terminal 102, and the terminal 102 determines a playing component compatibility detection result according to the video frame detection result.
In other embodiments, the terminal 102 may send the video frame to be detected to the server 104, and the server 104 inputs the video frame to be detected into a preset image detection model to obtain a video frame detection result, and determines a play component compatibility detection result according to the video frame detection result. The obtained play component compatibility detection result may also be fed back to the terminal 102 by the server 104.
The terminal 102 may be, but is not limited to, various personal computers, notebook computers, smart phones, tablet computers, and portable wearable devices. The server 104 may be implemented as a stand-alone server or as a server cluster comprised of multiple servers.
With the research and progress of artificial intelligence technology, the artificial intelligence technology is developed and researched in multiple fields, the scheme provided by the embodiment relates to the technical field of computer vision of artificial intelligence, the computer vision is a science for researching how to enable a machine to be seen, and further, the scheme refers to that a camera and a computer are used for replacing human eyes to carry out machine vision such as identification, tracking and measurement on a target, and further graphic processing is carried out, so that the computer processing becomes an image which is more suitable for human eyes to observe or is transmitted to an instrument to detect. As a scientific discipline, computer vision research-related theories and techniques attempt to build artificial intelligence systems that can capture information from images or multidimensional data. Computer vision techniques typically include image processing, image Recognition, image semantic understanding, image retrieval, OCR (Optical Character Recognition), video processing, video semantic understanding, and the like. The following examples are intended to illustrate in particular:
in one embodiment, as shown in fig. 2, a method for detecting compatibility of a play component is provided, which is described by taking the method as an example for being applied to the terminal in fig. 1, and includes the following steps 202 to 208.
Step 202, loading the playing component to a detection process, and creating an off-screen rendering environment, wherein the detection process is a process started when the application program is switched to the background.
The playing component refers to an object to be tested for decoding the multimedia data to play the multimedia data, and in an embodiment, the playing component may be a system playing component or a playing component downloaded by a third party. For example, the Android system has a Player, MediaPlayer, and for example, QQPlayer, Adobe Flash Player, etc. downloaded from an application store or an official website.
The detection process refers to a sub-process outside the main process independent of the application. And when the main process of the application program enters a background running state, starting the sub-process to perform compatibility detection.
When the detection process is started, the playing component can be loaded to the detection process in a mode of pulling up the playing component. And in the process of loading the playing component, determining the output environment of the video frame decoded by the playing component by creating an off-screen rendering environment.
Off-screen rendering is an off-screen rendering mode, i.e., the rendered result is not directly presented to the current screen. In a normal video playing process, generally, video frames are sequentially rendered to a current screen in a current screen rendering manner, at this time, a rendering operation of a GPU (Graphics Processing Unit) is performed in a current screen buffer area for display, and in an off-screen rendering process, the GPU opens up a buffer area other than the current screen buffer area for rendering operation, and does not render to the current screen.
The application program refers to an application program for playing multimedia data, which needs to use a playing component. The application program may be an object configured by a user as needed for performing compatibility detection on the playing component, or an object determined by the user based on a default policy of the terminal, for example, each default user needs to detect the compatibility of the playing component as a detection object when newly installing the application program.
Background running refers to that an operating system does not display a running process on a front-end display interface, background running is relative to front-end running such as a desktop, a user can directly perceive when the desktop is executed, and a background cannot be seen by the user. Switching an application to the background is the process of switching a running program that is directly visible on the desktop to a state that is not directly visible. For example, a program executing on the desktop is minimized or hidden so that the program page no longer appears on the desktop. For another example, the display content on the current desktop is covered by opening other application programs, so that the program page no longer appears on the desktop.
In one embodiment, the terminal detects the running states of the applications, and starts a detection process when detecting that one of the application states is switched to a background running state and receiving a detection command for the application. The detection command for the application program may be generated based on the background based on a user operation, or may be generated by triggering a terminal default policy (e.g., when the application program is newly installed). By generating the detection command under different conditions, the comprehensive compatibility detection of the playing component can be realized for different application programs.
And step 204, decoding the target video corresponding to the application program to an off-screen rendering environment through the playing component.
The target video corresponding to the application program is a video for performing compatibility detection, wherein the target video can be obtained according to a video file address or a specified video file.
Decoding is a process of restoring a digital code to its contents or converting an electric pulse signal, an optical signal, a radio wave, etc. into information, data, etc. represented by it in a specific way. Decoding is the process by which the recipient restores the received symbol or code to information, corresponding to the encoding process. The playing component can convert video data into continuous video frames in an image format by decoding the target video, and the playing of the video can be realized by rendering the continuous video frames on the interface in sequence.
In the detection process, a playing component is started to decode a target video to obtain a video frame, the obtained video frame is rendered to an off-screen rendering environment, direct rendering in a current screen is avoided, and video decoding and off-screen playing under the condition that a user does not sense are achieved.
And step 206, extracting the video frame to be detected from the off-screen rendering environment, and inputting the video frame to be detected into a preset image detection model to obtain a video frame detection result.
The playing component decodes the target video to the off-screen rendering environment, and the video frames to be detected can be all video frames in the off-screen rendering environment or partial video frames in the off-screen rendering environment. By taking all video frames as the video frames to be detected, all video frames of the target video can be comprehensively detected, and omission of abnormal pictures is avoided. However, because the number of the video frames corresponding to the target video is huge, the video frames can be screened, and part of the video frames are used as the video frames to be detected, so that the detection speed is increased. In an embodiment, the screening manner may be random screening, or screening according to a set rule, for example, screening according to a set proportion, screening according to a set frequency, or the like, where screening according to the set proportion means determining the number of video frames to be screened based on the number of total video frames, and then obtaining the number of video frames to be tested from the total video frames. The screening according to the set frequency refers to screening out one video frame as a video frame to be tested every N video frames according to the time sequence.
The image detection model refers to a neural network model for detecting a video frame to be tested. In an embodiment, the image detection model may include image set training of the anomaly images.
Specifically, a training service of machine learning is built in the background, a large number of materials such as a flower screen, a green screen and a black screen are provided for training, and an image detection model is obtained. The pattern screen, the green screen and the black screen belong to abnormal image display, and the pattern screen refers to the situation that stripes, spots or color blocks different from normal colors are displayed on the display screen, or the situation that the positions are reversed, disordered, the screen is shaken and distorted and the like exists. The green screen indicates that local or all green stripes, spots or color blocks are displayed on the display screen, and the black screen indicates that the display result on the display screen is completely black.
The training process of the image detection model comprises the following processes: taking normal images as positive samples, taking abnormal images such as a flower screen, a green screen, a black screen and the like as negative samples, and training an initial image detection model; and obtaining an image detection model which can be used for carrying out image detection on the video frame to be detected until the model evaluation parameters meet the set conditions.
In the detection process, a video frame to be detected is input into a preset image detection model, and a video frame detection result output by the image detection model is obtained.
And step 208, determining a compatibility detection result of the playing component according to the video frame detection result.
The video frame detection result comprises two conditions of an abnormal video frame with abnormal conditions such as a screen splash, a green screen and a black screen and a normal video frame without abnormal conditions such as a screen splash, a green screen and a black screen.
And when the video frame detection result indicates that the abnormal video frame exists, obtaining a detection result that the playing component is incompatible with the application program. And when the video frame detection result indicates that no abnormal video frame exists, obtaining a detection result that the playing component is compatible with the application program.
The method for detecting the compatibility of the playing component loads the playing component to the detection process based on the detection process started when the application program is switched to the background, loads and starts the playing component in the background process mode, decodes the target video corresponding to the application program to the off-screen rendering environment through the playing component by creating the off-screen rendering environment, is convenient for realizing the video decoding of the playing component under the condition of no perception of a user, can judge whether the video frame is abnormal or not because the image detection model is obtained by training based on an image set comprising abnormal images, extracts the video frame to be detected from the decoding data of the off-screen rendering environment, inputs the video frame to be detected into a preset image detection model, thereby obtaining the compatibility detection result of the playing component based on the video frame detection result and realizing the compatibility detection of the playing component in the background running process of the application program, the detection process does not depend on the actual operation and feedback of the user, the detection efficiency is improved, and the compatibility detection can be completed under the condition that the user does not sense the compatibility.
In one embodiment, as shown in fig. 3, decoding, by the playback component, the target video corresponding to the application program to the off-screen rendering environment, step 204, includes steps 302 to 304:
step 302, starting a playing component, and decoding the target video through the playing component to obtain a decoded video frame.
Step 304, rendering the video frame to an off-screen rendering environment according to the image rendering medium configured in the playing component.
In the embodiment, taking an Android system as an example, the image rendering medium may be a Surface, which is a handle of an image buffer of the Android system and is used as a medium for interaction between a playing component and a display View (View), and the player may draw a decoded image of the player onto a video (View) through the Surface.
The video frame is rendered to the off-screen rendering environment based on the image rendering medium which is associated with the off-screen rendering environment and configured on the playing assembly, and the targeted output of the video frame according to the specified rendering environment is realized by configuring the image rendering medium, so that the direct rendering to the current screen is avoided, and the user-friendly decoding and video frame rendering are realized.
In one embodiment, as shown in fig. 4, steps 402 to 406 are further included before rendering the video frames to the off-screen rendering environment according to the image rendering medium configured in the playback component.
Step 402, according to the off-screen rendering environment, constructing a texture identifier corresponding to the off-screen rendering environment.
Step 404, determining an image rendering medium corresponding to the texture object according to the texture object represented by the texture identifier.
In step 406, the image rendering medium is configured to the playback component.
The texture identifier, i.e., an Identity document ID (identification code), is based on texture mapping information that is randomly generated by the off-screen rendering environment and used for representing an association relationship with the off-screen rendering environment. In an embodiment, the texture identifier may be constructed by EGL, where EGL refers to OpenGL for Android extension, and is equivalent to an Application Programming Interface (API).
And generating a texture object SurfaceTexture based on the texture identifier, generating an image rendering medium Surface through the SurfaceTexture, and determining the data output environment of the play component after the Surface is set to the play component so that the play component can be normally started.
In an embodiment, the texture identifier may be randomly generated based on the off-screen rendering environment, different texture identifiers correspond to different texture objects, image rendering mediums corresponding to different texture objects may also be different, but ultimately all correspond to the same off-screen rendering environment, and the image rendering medium is configured to the playing component to implement association between the playing component and the off-screen rendering environment.
In this embodiment, by constructing the texture identifier and determining the image rendering medium based on the texture object represented by the texture identifier, different image rendering media can be constructed based on different texture identifiers, so that the extension of the associated path between the playing component and the off-screen rendering environment is realized, and the synchronous execution of a plurality of detection processes is facilitated.
In one embodiment, extracting the video frame to be detected from the off-screen rendering environment comprises: and according to a preset frequency, performing frame extraction processing on the video frame in the off-screen rendering environment to obtain the video frame to be detected.
In an embodiment, the EGL may continuously read data in the Surface of the image rendering medium to obtain the video frames decoded by the playing component. The video frame is subjected to frame extraction processing according to a certain frequency and input to an image detection model, for example, 1 frame is sent every 5 frames.
By extracting frames according to the set frequency, the number of the video frames to be tested is reduced, the video frames to be tested can be screened in a mirror image mode according to the same interval, the uniform distribution of screening results in the total video frames is ensured, and accurate and reliable detection results can be obtained.
In one embodiment, as shown in FIG. 5, decoding the target video corresponding to the application program to the off-screen rendering environment through the playback component, step 204, includes step 502 and step 504.
Step 502, when the play mode of the off-screen rendering environment is mute play, starting a play component to decode the target video, and rendering the decoded video frame to the off-screen rendering environment.
Step 504, when the playing mode of the off-screen rendering environment is non-mute playing, the playing mode is adjusted to be mute mode, the playing component is started to decode the target video, and the video frame obtained by decoding is rendered to the off-screen rendering environment.
The play modes of the off-screen rendering environment include mute play and non-mute play. The method comprises the steps that a play mode is configuration data of an off-screen rendering environment, the sound effect of other processes on a terminal cannot be influenced, specifically, when the play mode of the off-screen rendering environment is mute play, a video frame obtained by decoding is rendered from the off-screen rendering environment, the effect of silent rendering is achieved, when the play mode of the off-screen rendering environment is non-mute play, the video frame obtained by decoding is rendered from the off-screen rendering environment, audio data can be synchronously played based on sound effect setting parameters of the terminal, if the video frame is rendered according to the size of multimedia sound currently set by the terminal, and video playing is carried out in the off-screen rendering environment.
By acquiring the play mode of the off-screen rendering environment, when the play mode of the off-screen rendering environment is mute play, starting the play component to decode the target video, rendering the decoded video frame to the off-screen rendering environment, when the play mode of the off-screen rendering environment is non-mute play, adjusting the play mode to the mute mode, starting the play component to decode the target video, rendering the decoded video frame to the off-screen rendering environment, and realizing the silent play in the off-screen rendering environment.
In one embodiment, as shown in FIG. 6, steps 602-604 are included before loading the playback component into the detection process and creating the off-screen rendering environment, step 202.
Step 602, when it is monitored that the application program is switched to the background, recording the running duration of the background.
And step 604, starting a detection process when the background running time reaches a preset time.
After the application program is switched to the background by a user, the detection process is started after a certain waiting time, such as 2 seconds, and the detection process is started when the running time of the background reaches a preset time, so that a stable starting environment is provided for the detection process, and the detection process is prevented from being frequently started due to the fact that the user quickly switches the foreground and the background.
In one embodiment, the method for detecting compatibility of playing components further includes: step 702, when it is monitored that the application program is switched from the background to the front end, the detection process is ended.
In an embodiment, as shown in fig. 8, the running process of the application is a main process, the detection program is a compatibility detection sub-process corresponding to the main process, the sub-process and the main process are executed independently, and the starting and the closing of the sub-process depend on the running state of the main process and the execution result of the sub-process. Specifically, when the application program is switched from the front-end operation to the background operation, the compatibility detection sub-process is started, the compatibility detection sub-process has multiple ending modes, and the first mode is to end the compatibility detection sub-process in advance, for example, if a user switches the application program from the background to the front end, the sub-process is closed. And ending the compatibility detection subprocess after the second detection is finished, for example, the compatibility detection subprocess executes a specific time, for example, the set detection time is 5 seconds, the detection is finished after the execution result is recorded after the execution is finished, and the subprocess is closed, and for example, if an abnormal image appears in the detection result, the detection of the detection result can be recorded and the subprocess is closed.
By monitoring the running state of the application program, when the application program is switched from the background to the front end, the detection process is ended, so that the influence on the normal use of a user is avoided.
In an embodiment, when the execution is finished or the image is abnormal, a compatibility detection result of the playing component on the application program may be obtained, and the application program may be marked as a detected application program. When it is monitored that the application program is switched from the background to the front end, the application program can be marked as the application program to be detected after the detection process is finished, and when it is monitored that the running state of the application program meets the detection condition next time, the detection process can be restarted for detection again until a compatibility detection result of the playing component on the application program is obtained. And marking the application programs which are not detected as the application programs to be detected for detection again so as to realize comprehensive compatibility detection.
In one embodiment, as shown in fig. 9, a play component compatibility detection method is provided, which includes the following steps 902 to 926.
And step 902, recording the running time of the background when the situation that the application program is switched to the background is monitored.
And 904, starting a detection process when the background running time reaches a preset time.
Step 906, load the playback component to the detection process and create an off-screen rendering environment.
Step 908, when the playback mode of the off-screen rendering environment is non-mute playback, the playback mode is adjusted to mute mode.
Step 910, start the playing component, and decode the target video through the playing component to obtain the decoded video frame.
Step 912, according to the off-screen rendering environment, a texture identifier corresponding to the off-screen rendering environment is constructed.
Step 914, according to the texture object represented by the texture identifier, determining an image rendering medium corresponding to the texture object.
At step 916, the image rendering medium is configured to the playback component.
Step 918, render the video frame to an off-screen rendering environment according to the image rendering medium.
And 920, performing frame extraction processing on the video frame in the off-screen rendering environment according to a preset frequency to obtain a video frame to be detected.
And step 922, inputting the video frame to be detected into a preset image detection model to obtain a video frame detection result.
Step 924, determining a compatibility detection result of the playing component according to the video frame detection result.
In step 926, when it is detected that the application program is switched from the background to the front end, the detection process is ended.
The application also provides an application scene, and the application scene applies the play component compatibility detection method. Specifically, the method for detecting compatibility of the playing component is applied to compatibility detection of a MediaPlayer of a player of an Android system, and the application in the application scene is as follows:
firstly, a training service of machine learning needs to be established in the background, a large number of image materials such as a flower screen, a green screen and a black screen are provided to train an initial image detection model, and the image detection model for detecting a video frame is obtained. Then, detection is carried out based on an image detection model, and the detection process comprises two parts:
the first part is to build an environment of an independent process, and the second part is to run a detection program in the independent process and obtain the result. In the first part, when the terminal acquires a detection command of the background, a detection task for detecting the APP is recorded, the running state of the APP is monitored, the detection command can be from a default strategy issued by the background or required to be detected by a new user installed at the terminal every time, and the detection command contains a video playing address or a video file required to be detected. When it is monitored that the user switches the App to the background, the detection process is started immediately after a certain waiting time, such as 2 seconds, so that the frequent starting of the process caused by the quick switching of the foreground and the background of the user is prevented. The detection process has two ending modes, the first mode is that the user switches the App back to the front end, and the other mode is that the App can be destroyed after the process is executed for a certain time, for example, 5 seconds, and the execution result is recorded after the process is executed.
The second part is to run the detection program in an independent process and obtain the result, and the second part is divided into two stages when the player is loaded and when the player is actually started. In the stage of loading the player, as shown in fig. 9, it is necessary to create an OpenGL off-screen rendering environment by using EGL, generate a texture ID by means of glBindTexture, generate a Surface texture by using the texture ID, generate a Surface by using the Surface texture, configure the Surface to the player, and render the decoded video frame to the off-screen rendering environment by using the Surface. The method is realized without any modification to the system player MediaPlayer, and the compatibility test result is not influenced.
When the player is actually started, the mute needs to be set first, and then the player needs to be started. After the player starts decoding, the decoded data will be output to the Surface configured for the player, and the EGL can continuously read the data in the Surface to obtain the decoded data of the player. And then, the data is transmitted to an inference module according to a certain frequency, wherein the frequency can be 1 frame every 5 frames, and the inference module infers and obtains an inference result according to a trained image detection model after receiving the data and mainly judges whether the image is in abnormal conditions such as screen blooming, green screen, black screen and the like. If abnormal conditions such as screen splash, green screen, black screen and the like occur, the detection result can be recorded and the whole program is stopped; otherwise, the operation is normally exited after a certain time. If the system player has an exception, the system player has compatibility problems, otherwise, the system player does not have the compatibility problems. The whole process adopts off-screen rendering and mute playing modes, so that the user cannot be disturbed. Scenes using a large number of system players, such as playing of a TV platform, can be well solved.
It should be understood that, although the steps in the flowcharts shown in the above embodiments are shown in sequence as indicated by the arrows, the steps are not necessarily executed in sequence as indicated by the arrows. The steps are not performed in the exact order shown and described, and may be performed in other orders, unless explicitly stated otherwise. Moreover, at least a part of the steps in each flowchart involved in the above embodiments may include multiple steps or multiple stages, which are not necessarily performed at the same time, but may be performed at different times, and the order of performing the steps or stages is not necessarily sequential, but may be performed alternately or alternately with other steps or at least a part of the steps or stages in other steps.
In one embodiment, as shown in fig. 11, there is provided a playback component compatibility detection apparatus 1100, which may be a part of a computer device using a software module or a hardware module, or a combination of the two, and specifically includes: a loading module 1102, a decoding module 1104, a detection module 1106, and a result determination module 1108, wherein:
the loading module 1102 is configured to load the playback component to the detection process, create an off-screen rendering environment, and detect the process as a process started when the application is switched to the background.
And the decoding module 1104 is used for decoding the target video corresponding to the application program to the off-screen rendering environment through the playing component.
The detection module 1106 is configured to extract a video frame to be detected from an off-screen rendering environment, input the video frame to be detected into a preset image detection model, and obtain a video frame detection result, where the image detection model is obtained based on image set training including abnormal images.
The compatibility determining module 1108 is configured to determine a play component compatibility detection result according to the video frame detection result.
In one embodiment, the decoding module is further configured to start the playing component, and decode the target video through the playing component to obtain a decoded video frame; and rendering the video frame to an off-screen rendering environment according to an image rendering medium configured on the playing component, wherein the image rendering medium is associated with the off-screen rendering environment.
In one embodiment, the playback component compatibility detection apparatus further includes an image rendering medium configuration module, configured to construct, according to the off-screen rendering environment, a texture identifier corresponding to the off-screen rendering environment; determining an image rendering medium corresponding to the texture object according to the texture object represented by the texture identifier; and configuring the image rendering medium to a playing component.
In one embodiment, the detection module is further configured to perform frame extraction processing on the video frame in the off-screen rendering environment according to a preset frequency to obtain the video frame to be detected.
In one embodiment, the decoding module is further configured to, when the play mode of the off-screen rendering environment is silent play, start the play component to decode the target video, and render the decoded video frame to the off-screen rendering environment; and when the playing mode of the off-screen rendering environment is non-mute playing, adjusting the playing mode to be a mute mode, starting the playing component to decode the target video, and rendering the video frame obtained by decoding to the off-screen rendering environment.
In one embodiment, the decoding module is further configured to record a background running duration when it is monitored that the application program is switched to the background; and starting a detection process when the background running time reaches a preset time.
In one embodiment, the monitoring module is further configured to end the detection process when it is monitored that the application program switches from the background to the front end.
The play component compatibility detection device loads the play component to the detection process based on the detection process started when the application program is switched to the background, loads and starts the play component in the background process, decodes the target video corresponding to the application program to the off-screen rendering environment through the play component by creating the off-screen rendering environment, is convenient for realizing the video decoding of the play component under the condition of no perception of a user, can judge whether the video frame is abnormal or not by inputting the video frame to be detected into a preset image detection model because the image detection model is obtained by training based on an image set comprising abnormal images, thereby obtaining the play component compatibility detection result based on the video frame detection result and realizing the play component compatibility detection in the background running process of the application program, the detection process does not depend on the actual operation and feedback of the user, and the compatibility detection efficiency of the playing component is improved.
For the specific definition of the playback component compatibility detection apparatus, reference may be made to the above definition of the playback component compatibility detection method, which is not described herein again. All or part of the modules in the playing component compatibility detection device can be realized by software, hardware and a combination thereof. The modules can be embedded in a hardware form or independent from a processor in the computer device, and can also be stored in a memory in the computer device in a software form, so that the processor can call and execute operations corresponding to the modules.
In one embodiment, a computer device is provided, which may be a terminal, and its internal structure diagram may be as shown in fig. 12. The computer device includes a processor, a memory, a communication interface, a display screen, and an input device connected by a system bus. Wherein the processor of the computer device is configured to provide computing and control capabilities. The memory of the computer device comprises a nonvolatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of an operating system and computer programs in the non-volatile storage medium. The communication interface of the computer device is used for carrying out wired or wireless communication with an external terminal, and the wireless communication can be realized through WIFI, an operator network, NFC (near field communication) or other technologies. The computer program is executed by a processor to implement a playback component compatibility detection method. The display screen of the computer equipment can be a liquid crystal display screen or an electronic ink display screen, and the input device of the computer equipment can be a touch layer covered on the display screen, a key, a track ball or a touch pad arranged on the shell of the computer equipment, an external keyboard, a touch pad or a mouse and the like.
Those skilled in the art will appreciate that the architecture shown in fig. 12 is merely a block diagram of some of the structures associated with the disclosed aspects and is not intended to limit the computing devices to which the disclosed aspects apply, as particular computing devices may include more or less components than those shown, or may combine certain components, or have a different arrangement of components.
In one embodiment, a computer device is further provided, which includes a memory and a processor, the memory stores a computer program, and the processor implements the steps of the above method embodiments when executing the computer program.
In an embodiment, a computer-readable storage medium is provided, in which a computer program is stored which, when being executed by a processor, carries out the steps of the above-mentioned method embodiments.
In one embodiment, a computer program product or computer program is provided that includes computer instructions stored in a computer-readable storage medium. The computer instructions are read by a processor of a computer device from a computer-readable storage medium, and the computer instructions are executed by the processor to cause the computer device to perform the steps in the above-mentioned method embodiments.
It will be understood by those skilled in the art that all or part of the processes of the methods of the embodiments described above can be implemented by hardware related to instructions of a computer program, which can be stored in a non-volatile computer-readable storage medium, and when executed, can include the processes of the embodiments of the methods described above. Any reference to memory, storage, database or other medium used in the embodiments provided herein can include at least one of non-volatile and volatile memory. Non-volatile Memory may include Read-Only Memory (ROM), magnetic tape, floppy disk, flash Memory, optical storage, or the like. Volatile Memory can include Random Access Memory (RAM) or external cache Memory. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM), among others.
The technical features of the above embodiments can be arbitrarily combined, and for the sake of brevity, all possible combinations of the technical features in the above embodiments are not described, but should be considered as the scope of the present specification as long as there is no contradiction between the combinations of the technical features.
The above examples only express several embodiments of the present application, and the description thereof is more specific and detailed, but not construed as limiting the scope of the invention. It should be noted that, for a person skilled in the art, several variations and modifications can be made without departing from the concept of the present application, which falls within the scope of protection of the present application. Therefore, the protection scope of the present patent shall be subject to the appended claims.
Claims (10)
1. A method for detecting compatibility of a playing component, the method comprising:
loading a playing component to a detection process, and creating an off-screen rendering environment, wherein the detection process is a process started when an application program is switched to a background;
decoding a target video corresponding to the application program to the off-screen rendering environment through the playing component;
extracting a video frame to be detected from the off-screen rendering environment, inputting the video frame to be detected into a preset image detection model to obtain a video frame detection result, wherein the image detection model is obtained based on image set training including abnormal images;
and determining the compatibility detection result of the playing component according to the video frame detection result.
2. The method of claim 1, wherein the decoding, by the playback component, the target video corresponding to the application to the off-screen rendering environment comprises:
starting the playing component, and decoding the target video through the playing component to obtain a decoded video frame;
rendering the video frame to the off-screen rendering environment according to an image rendering medium configured to the playing component, wherein the image rendering medium is associated with the off-screen rendering environment.
3. The method of claim 2, wherein the rendering the video frame to the off-screen rendering environment according to an image rendering medium configured to the playback component further comprises:
constructing a texture identifier corresponding to the off-screen rendering environment according to the off-screen rendering environment;
determining an image rendering medium corresponding to the texture object according to the texture object represented by the texture identifier;
configuring the image rendering medium to the playback component.
4. The method of claim 1, wherein extracting the video frame to be detected from the off-screen rendering environment comprises:
and according to a preset frequency, performing frame extraction processing on the video frame in the off-screen rendering environment to obtain a video frame to be detected.
5. The method of claim 1, wherein the decoding, by the playback component, the target video corresponding to the application to the off-screen rendering environment comprises:
when the playing mode of the off-screen rendering environment is mute playing, starting the playing component to decode the target video, and rendering the decoded video frame to the off-screen rendering environment;
and when the playing mode of the off-screen rendering environment is non-mute playing, adjusting the playing mode to be a mute mode, starting the playing component to decode the target video, and rendering the decoded video frame to the off-screen rendering environment.
6. The method of claim 1, wherein before loading the playback component into the detection process and creating the off-screen rendering environment, further comprising:
when the situation that the application program is switched to a background is monitored, recording the running duration of the background;
and starting a detection process when the background running time reaches a preset time.
7. The method of claim 1, further comprising:
and when the application program is monitored to be switched from the background to the front end, ending the detection process.
8. A playback component compatibility detection apparatus, the apparatus comprising:
the loading module is used for loading the playing component to a detection process and creating an off-screen rendering environment, wherein the detection process is started when the application program is switched to a background;
the decoding module is used for decoding the target video corresponding to the application program to the off-screen rendering environment through the playing component;
the detection module is used for extracting a video frame to be detected from the off-screen rendering environment, inputting the video frame to be detected into a preset image detection model and obtaining a video frame detection result, wherein the image detection model is obtained based on image set training including abnormal images;
and the compatibility determining module is used for determining the compatibility detection result of the playing component according to the video frame detection result.
9. A computer device comprising a memory and a processor, the memory storing a computer program, characterized in that the processor, when executing the computer program, implements the steps of the method of any of claims 1 to 7.
10. A computer-readable storage medium, in which a computer program is stored which, when being executed by a processor, carries out the steps of the method according to any one of claims 1 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011213931.1A CN113407436A (en) | 2020-11-04 | 2020-11-04 | Play component compatibility detection method and device, computer equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011213931.1A CN113407436A (en) | 2020-11-04 | 2020-11-04 | Play component compatibility detection method and device, computer equipment and storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN113407436A true CN113407436A (en) | 2021-09-17 |
Family
ID=77677431
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202011213931.1A Pending CN113407436A (en) | 2020-11-04 | 2020-11-04 | Play component compatibility detection method and device, computer equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113407436A (en) |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114339415A (en) * | 2021-12-23 | 2022-04-12 | 天翼云科技有限公司 | Client video playing method and device, electronic equipment and readable medium |
| CN115086603A (en) * | 2022-05-12 | 2022-09-20 | 南方电网深圳数字电网研究院有限公司 | Video processing method and device based on compatible player |
| CN116471446A (en) * | 2023-06-19 | 2023-07-21 | 北京麟卓信息科技有限公司 | Video playing optimization method crossing running environments |
| CN115086603B (en) * | 2022-05-12 | 2024-05-14 | 南方电网数字平台科技(广东)有限公司 | Video processing method and device based on compatibility player |
-
2020
- 2020-11-04 CN CN202011213931.1A patent/CN113407436A/en active Pending
Cited By (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114339415A (en) * | 2021-12-23 | 2022-04-12 | 天翼云科技有限公司 | Client video playing method and device, electronic equipment and readable medium |
| CN114339415B (en) * | 2021-12-23 | 2024-01-02 | 天翼云科技有限公司 | Client video playing method and device, electronic equipment and readable medium |
| CN115086603A (en) * | 2022-05-12 | 2022-09-20 | 南方电网深圳数字电网研究院有限公司 | Video processing method and device based on compatible player |
| CN115086603B (en) * | 2022-05-12 | 2024-05-14 | 南方电网数字平台科技(广东)有限公司 | Video processing method and device based on compatibility player |
| CN116471446A (en) * | 2023-06-19 | 2023-07-21 | 北京麟卓信息科技有限公司 | Video playing optimization method crossing running environments |
| CN116471446B (en) * | 2023-06-19 | 2023-08-11 | 北京麟卓信息科技有限公司 | Video playing optimization method crossing running environments |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN107801096B (en) | Video playing control method and device, terminal equipment and storage medium | |
| CN108521608A (en) | Processing method, device, terminal and the storage medium of video file | |
| CN111277910B (en) | Bullet screen display method and device, electronic equipment and storage medium | |
| CN109474850B (en) | Motion pixel video special effect adding method and device, terminal equipment and storage medium | |
| CN110072047B (en) | Image deformation control method and device and hardware device | |
| CN111095939A (en) | Identifying previously streamed portions of a media item to avoid repeated playback | |
| CN113407436A (en) | Play component compatibility detection method and device, computer equipment and storage medium | |
| CN109729429A (en) | Video broadcasting method, device, equipment and medium | |
| CN105929941B (en) | Information processing method and device and terminal equipment | |
| CN114567693B (en) | Video generation method and device and electronic equipment | |
| CN115941869A (en) | Audio processing method and device and electronic equipment | |
| KR20170043944A (en) | Display apparatus and method of controlling thereof | |
| US20210289266A1 (en) | Video playing method and apparatus | |
| CN114004726A (en) | Watermark display method, watermark display device, computer equipment and storage medium | |
| CN110401865B (en) | Method and device for realizing video interaction function | |
| CN114677738A (en) | MV recording method, MV recording device, electronic equipment and computer readable storage medium | |
| CN114157881A (en) | Multimedia processing method, device, electronic equipment and storage medium | |
| CN113641286A (en) | Screen capturing method, electronic equipment and computer storage medium | |
| CN112511890A (en) | Video image processing method and device and electronic equipment | |
| CN111107385A (en) | Live video processing method and device | |
| CN114341946A (en) | Identification method, identification device, electronic equipment and storage medium | |
| CN110704656A (en) | Picture processing method and device, storage medium and terminal | |
| CN110602559A (en) | Human-computer interaction and television operation control method, device, equipment and storage medium | |
| CN112118410B (en) | Service processing method, device, terminal and storage medium | |
| CN112100075B (en) | User interface playback method, device, equipment and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| REG | Reference to a national code |
Ref country code: HK Ref legal event code: DE Ref document number: 40051388 Country of ref document: HK |
|
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| TA01 | Transfer of patent application right | ||
| TA01 | Transfer of patent application right |
Effective date of registration: 20221114 Address after: 1402, Floor 14, Block A, Haina Baichuan Headquarters Building, No. 6, Baoxing Road, Haibin Community, Xin'an Street, Bao'an District, Shenzhen, Guangdong 518100 Applicant after: Shenzhen Yayue Technology Co.,Ltd. Address before: 518000 Tencent Building, No. 1 High-tech Zone, Nanshan District, Shenzhen City, Guangdong Province, 35 Floors Applicant before: TENCENT TECHNOLOGY (SHENZHEN) Co.,Ltd. |