CN111611151B - Method and device for detecting overdrawing of user interface - Google Patents
Method and device for detecting overdrawing of user interface Download PDFInfo
- Publication number
- CN111611151B CN111611151B CN201910138522.0A CN201910138522A CN111611151B CN 111611151 B CN111611151 B CN 111611151B CN 201910138522 A CN201910138522 A CN 201910138522A CN 111611151 B CN111611151 B CN 111611151B
- Authority
- CN
- China
- Prior art keywords
- user interface
- view
- picture data
- determining
- pixel points
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/36—Preventing errors by testing or debugging software
- G06F11/362—Software debugging
- G06F11/3624—Software debugging by performing operations on the source code, e.g. via a compiler
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/36—Preventing errors by testing or debugging software
- G06F11/362—Software debugging
- G06F11/3636—Software debugging by tracing the execution of the program
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/36—Preventing errors by testing or debugging software
- G06F11/362—Software debugging
- G06F11/3644—Software debugging by instrumenting at runtime
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T11/00—2D [Two Dimensional] image generation
- G06T11/40—Filling a planar surface by adding surface attributes, e.g. colour or texture
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T15/00—3D [Three Dimensional] image rendering
- G06T15/005—General purpose rendering architectures
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/90—Determination of colour characteristics
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2200/00—Indexing scheme for image data processing or generation, in general
- G06T2200/24—Indexing scheme for image data processing or generation, in general involving graphical user interfaces [GUIs]
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
- Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management
Abstract
The invention discloses a method and a device for detecting overdrawing of a user interface. The method comprises the following steps: acquiring picture data of a top layer view rendered in a user interface refreshing process, wherein the picture data is rendered according to a drawing level of the top layer view; judging whether the pixel points are excessively drawn according to the colors of the pixel points in the picture data; determining the ratio of the number of over-drawn pixel points to the total number of pixel points in the picture data, and judging whether the ratio is larger than a preset ratio threshold value or not; if yes, determining that the user interface is overdrawn. The automatic detection and the overdrawing can be realized, the labor cost is saved, and the detection efficiency and the accuracy are improved; the method can better help research personnel to find problems, and finally improves the user experience of the application.
Description
Technical Field
The present invention relates to the field of application detection technologies, and in particular, to a method and an apparatus for detecting excessive user interface drawing.
Background
Over-rendering of User Interfaces (UIs) often occurs in operating systems, such as Applications (APPs) of the android system. Overdrawing means that a certain pixel area on the screen is drawn multiple times within the same frame. In order to maintain the smoothness of vision, the screen refresh frequency of the mobile phone is usually set to 60hz, that is, one frame is updated in 1000/60=16.67 ms, and excessive drawing wastes a large amount of CPU and GPU resources, so that the UI cannot be updated to the next frame in a set frequency to cause frame dropping and UI clamping.
In the prior art, an android system provides a tool for detecting excessive drawing of a user interface, and in a developer mode of a mobile phone, a 'debugging GPU excessive drawing' is opened, a 'display GPU excessive drawing region' in the debugging GPU excessive drawing region is selected, and after the debugging GPU excessive drawing region is opened, the system can render different colors on elements with different rendering complexity, specifically: one layer of the overdrawn display is blue, two layers of the overdrawn display is green, three layers of the overdrawn display is light red, and four layers of the overdrawn display is dark red. And then the developer can perform screenshot on the mobile phone screen, analyze the screenshot, visually check the area size of the area rendered into red in the screenshot and the rendering degree of the red, judge the degree of overdrawing and determine whether the original code needs to be adjusted.
According to the technical scheme, time and labor are wasted in judging through a manual visual inspection method, errors are prone to occur in manual subjective judgment, and particularly under the condition that detection personnel are tired in vision.
Disclosure of Invention
The present invention has been made in view of the above problems, and has as its object to provide a method and apparatus for detecting overdrawing of a user interface that overcomes or at least partially solves the above problems.
In a first aspect, an embodiment of the present invention provides a method for detecting overdrawing of a user interface, including:
acquiring picture data of a top-level view rendered in a user interface refreshing process, wherein the picture data is rendered according to a drawing level of the top-level view;
judging whether the pixel points are excessively drawn according to the colors of the pixel points in the picture data;
determining the ratio of the number of over-drawn pixel points to the total number of pixel points in the picture data, and judging whether the ratio is larger than a preset ratio threshold value or not;
if yes, determining that the user interface is overdrawn.
In some optional embodiments, determining whether the pixel is overdrawn according to the color of the pixel in the picture data includes:
converting RGB values of pixel point colors into HSI values;
judging whether the H component value of the HSI value is larger than or equal to a preset H component minimum threshold value and smaller than or equal to a preset H component maximum threshold value;
if yes, determining that the pixel point is overdrawn.
In some alternative embodiments, acquiring picture data of a top-level view rendered during a user interface refresh process includes:
and acquiring picture data of the top-level view of the user interface rendered in the user interface refreshing process through a pre-inserted probe, wherein the probe is an execution code for acquiring the picture data.
In some alternative embodiments, the probe is pre-inserted by:
traversing the source codes to search all sub-categories of the Activity in the source code compiling process of the application program, wherein each sub-category corresponds to one user interface;
and inserting probes into the refreshing method part of the top view source codes of the subclasses.
In some alternative embodiments, before the probe is inserted into the refresh method portion of the top-level view source code of the subclass, the method further includes:
acquiring the identification of the top view of the subclass;
if the acquisition fails, setting the identifier of the top view in the source code according to a set rule, and then executing the probe insertion part of the refreshing method of the top view source code in the subclass;
if the acquisition is successful, searching whether the top-level view identifier exists in a pre-established view list without inspection; if not, executing the refreshing method part of the top view source codes of the subclasses to insert probes.
In some alternative embodiments, obtaining an identification of a top-level view of the sub-class includes:
and acquiring the identification of the top layer view of the subclass from the onStart method source code of the subclass.
In some alternative embodiments, in the source code compiling process of the application program, the method further includes:
determining an HSI value of the pixel point according to the RGB value of the pixel point of the top-level view;
judging whether the H component value of the HSI value is larger than or equal to the preset H component minimum threshold value and smaller than or equal to the preset H component maximum threshold value;
if yes, the RGB value of the pixel point is modified, so that the H component value corresponding to the modified RGB value is larger than the preset maximum H component threshold or smaller than the preset minimum H component threshold.
In some alternative embodiments, after determining that the user interface is overdrawn, further comprising:
the following information of the top layer view of the user interface is recorded:
and the identification of the top-layer view, the H component of the over-drawn pixels in the top-layer view, the number of the over-drawn pixels in the top-layer view, and the ratio of the number of the over-drawn pixels in the top-layer view to the total number of the pixels.
In a second aspect, an embodiment of the present invention provides a device for detecting overdrawing of a user interface, including:
the device comprises an acquisition module, a display module and a display module, wherein the acquisition module is used for acquiring picture data of a top-level view rendered in a user interface refreshing process, and the picture data is rendered according to a drawing level of the top-level view;
the first judging module is used for judging whether the pixel points are excessively drawn according to the colors of the pixel points in the picture data acquired by the acquiring module;
the first determining module is used for determining the proportion of the number of the over-drawn pixel points in the picture data judged by the first judging module to the total number of the pixel points;
the second judging module is used for judging whether the proportion determined by the first determining module is larger than a preset proportion threshold value or not;
and the second determining module is used for determining that the user interface is overdrawn when the second judging module judges that the user interface is overdrawn.
In a third aspect, embodiments of the present invention provide a computer-readable storage medium having stored thereon computer instructions that, when executed by a processor, implement the above-described method of detecting user interface overdrawing.
In a fourth aspect, an embodiment of the present invention provides a server, including: the device comprises a memory, a processor and a computer program stored in the memory and capable of running on the processor, and is characterized in that the processor realizes the method for detecting the overdrawing of the user interface when executing the program.
The technical scheme provided by the embodiment of the invention has the beneficial effects that at least:
acquiring picture data of a top layer view rendered in a user interface refreshing process, wherein the picture data is rendered according to a drawing level of the top layer view; judging whether the pixel points are excessively drawn according to the colors of the pixel points in the picture data; determining the ratio of the number of over-drawn pixel points to the total number of pixel points in the picture data, and judging whether the ratio is larger than a preset ratio threshold value or not; if yes, determining that the user interface is overdrawn. The automatic detection and the overdrawing can be realized, the labor cost is saved, and the detection efficiency and the accuracy are improved; the method can better help research personnel to find problems, and finally improves the user experience of the application.
Additional features and advantages of the invention will be set forth in the description which follows, and in part will be obvious from the description, or may be learned by practice of the invention. The objectives and other advantages of the invention will be realized and attained by the structure particularly pointed out in the written description and claims thereof as well as the appended drawings.
The technical scheme of the invention is further described in detail through the drawings and the embodiments.
Drawings
The accompanying drawings are included to provide a further understanding of the invention and are incorporated in and constitute a part of this specification, illustrate the invention and together with the embodiments of the invention, serve to explain the invention. In the drawings:
FIG. 1 is a flow chart of a method for detecting user interface overdrawing in an embodiment of the invention;
FIG. 2 is a flow chart of a method of probe pre-insertion according to an embodiment of the present invention;
fig. 3 is a schematic structural diagram of a detection device for overdrawing a user interface according to an embodiment of the invention.
Detailed Description
Exemplary embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. While exemplary embodiments of the present disclosure are shown in the drawings, it should be understood that the present disclosure may be embodied in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the scope of the disclosure to those skilled in the art.
In order to solve the problems that excessive drawing is time-consuming and labor-consuming and error-prone in manual screenshot detection through system rendering in the prior art, the embodiment of the invention provides a detection method for excessive drawing of a user interface, which can automatically detect excessive drawing, saves labor cost and improves detection efficiency and accuracy.
The embodiment of the invention provides a method for detecting overdrawing of a user interface, which is shown in a figure 1, and comprises the following steps:
step S11: and acquiring the picture data of the top-level view rendered in the refreshing process of the user interface.
The picture data is rendered according to the drawing hierarchy of the top-level view.
For example, after the android system detects that an installed application program is detected, after a "debugging GPU overdrawing switch" in a "developer mode" is turned on according to a corresponding command (sometimes called "displaying GPU overdrawing"), in a process of starting to execute opening and traversing of each user interface in the application program, taking a user interface as an example, in a refreshing process of the user interface, the system may render a top layer view into different colors according to rendering complexity of the top layer view, which may specifically be: the rendering hierarchy is first-level rendering to blue, the rendering hierarchy is second-level rendering to green, the rendering hierarchy is three-level rendering to light red, and the rendering hierarchy is four-level rendering to dark red. Alternatively, other rendering methods may be used.
The command may be obtained by obtaining a command input or clicked by a user, or may be obtained from a script command calculated by an upper computer. That is, the operation of turning on the "debug GPU overdrawing switch" in the "developer mode" may be a manual intervention operation, or may be automatically performed by the system, without manual intervention.
Step S12: judging whether the pixel points are excessively drawn according to the colors of the pixel points in the picture data.
In one embodiment, after the image data is obtained, the image data is saved in a foreign storage space, for example, the image data can be saved in an SD card of a mobile phone or an external usb disk.
The picture data is not judged directly in the cache, but is judged asynchronously after being stored in a foreign storage space, so that the memory and the computing resource of the system are saved, and the smooth detection is ensured.
The thumbnail data may be obtained by processing the picture data in the buffer, or the thumbnail data may be obtained by processing the picture data in the buffer and then stored in a local storage space. And judging the thumbnail data in the step.
And processing the picture data to obtain thumbnail data, and judging whether the user interface is excessively drawn according to the thumbnail, so that interference factors can be reduced, and the accuracy and speed of judgment can be improved.
Because the color of the pixel point in the picture data represents the rendering complexity, i.e. the degree of overdrawing, the judgment of whether the pixel point is overdrawed or not may be, in one embodiment, converting the RGB value of the pixel point color into the HSI value; judging whether the H component value of the HSI value is larger than or equal to a preset H component minimum threshold value and smaller than or equal to a preset H component maximum threshold value. When the judgment is negative, determining that the pixel point is not excessively drawn; when the determination is yes, it is determined that the pixel point is overdrawn.
Wherein, RGB value, red, green, blue value, represents the color of three channels of red, green and blue; the HSI value, H, hue, represents the Hue of the color, S, saturation, I, and Intensity of the color.
Step S13: and determining the ratio of the number of the over-drawn pixel points to the total number of the pixel points in the picture data.
Step S14: and judging whether the ratio of the number of the over-drawn pixel points to the total number of the pixel points in the picture data is larger than a preset ratio threshold value.
If yes, executing step S15; and if not, determining that the user interface is not overdrawn.
Step S15: user interface overdrawing is determined.
According to the method for detecting the overdrawing of the user interface, the overdrawing can be automatically detected, the labor cost is saved, and the detection efficiency and accuracy are improved; meanwhile, the method can better help research and development personnel to find problems, and finally improves the user experience of the application.
In step S11, the picture data of the top-level view rendered in the user interface refreshing process may be obtained by a probe inserted in advance, where the probe is an execution code for obtaining the picture data.
Referring to fig. 2, the probe may be inserted in advance as follows:
step S21: in the source code compiling process of the application program, traversing the source codes to search all the sub-categories of the Activity.
Wherein each sub-class of Activity corresponds to a user interface.
The searching method can be searching according to the uniform suffix or prefix name of each subclass; or may be searched based on a common attribute for each subclass.
Step S22: an identification of a top-level view of the sub-category is obtained.
In one embodiment, the identification of the top-level view of the subclass may be obtained in the onStart method source code of the subclass.
In the life cycle of an Activity, the onStart method is called only once, and the onStart method is called possibly a plurality of times, so that the identifier of the top-level view of the subclass is obtained in the onStart method of the subclass instead of the identifier obtained in the onStart method, repeated obtaining of the identifier of the top-level view can be avoided, the calculation amount of a system is reduced, and meanwhile, the detection speed is improved.
If the obtaining of the identifier of the top view of the sub-class fails, that is, the top view of the sub-class has no identifier, executing step S23; if the obtaining of the top view of the sub-category is successful, step S25 is performed.
Step S23: and setting the identification of the top view in the source code according to the set rule.
And for the top-level views without the marks, adding the marks of the top-level views according to the set rules. The method and the device ensure that each top layer view of the detection drawing hierarchy can be obtained finally, and the positions of the specific top layer views in the source codes can be found quickly and conveniently and are modified when research and development personnel want to modify the source codes.
Step S24: and inserting probes into the refreshing method part of the top-level view source codes.
In the refreshing method of the top-level view, probes are inserted into the refreshing method part of the source code of the top-level view, and in an android system, the probes are inserted into the onResume method source code of the source code.
During compiling of the application program, dynamically modifying source codes of the application program, inserting probes, and particularly during onResume of a top layer view of each user interface during operation of the user interface in the application program, the probes can acquire rendered picture data.
Step S25: and searching whether the top-level view is identified in the pre-established non-checking view list.
If not, go to step S24.
If yes, the top-level view, namely the user interface corresponding to the top-level view, is described, detection of the user interface is ignored, probes are not needed to be inserted into source codes of the onResume method, detection of a drawing level is not needed, the calculated amount of monitoring is saved, and the monitoring speed is improved.
The picture data of the top layer view of the user interface rendered in the refreshing process of the user interface is obtained through the pre-inserted probe, the color of each pixel point can be the color rendered by the system according to the drawing degree or the original color, and therefore whether excessive drawing is interfered by the original color is judged according to the color. In one embodiment, the interference of its original color can be avoided by:
in the source code compiling and packaging process of the application, the following operations can be further executed:
determining an HSI value of the pixel point according to the RGB value of the pixel point of the top-level view; judging whether the H component value of the HSI value is larger than or equal to a preset H component minimum threshold value and smaller than or equal to a preset H component maximum threshold value; if yes, the RGB values of the pixel points are modified, so that the H component value corresponding to the modified RGB values is larger than a preset maximum H component threshold value or smaller than a preset minimum H component threshold value. For example, the RGB values of the pixel may be modified to have a corresponding H-value component of blue.
The RGB values of the pixel points are modified according to the set rules, so that the color of the pixel points judged to be excessively drawn in the acquired picture data is rendered by the system instead of the original color of the pixel points. Therefore, judgment interference caused by excessive drawing can be eliminated, and the accuracy and reliability of judgment are improved.
In one embodiment, after determining that one user interface is overdrawn, the following information of the top-level view of the user interface may also be recorded:
the identification of the top-level view, the H component of the over-drawn pixels in the top-level view, the number of over-drawn pixels in the top-level view, and the ratio of the number of over-drawn pixels to the total number of pixels in the top-level view.
After judging whether each user interface of the application program is overdrawn, an overdrawn report of the application program can be generated according to the recorded information of each overdrawn user interface. The method is convenient for a developer to know the concrete situation of overdrawing, can judge whether the source code needs to be adjusted according to the related information, and can quickly position the position needing to be modified in the source code.
Based on the same inventive concept, the embodiment of the invention also provides a device for detecting overdrawing of a user interface, the structure of which is shown in fig. 3, comprising:
the obtaining module 31 is configured to obtain picture data of a top-level view rendered in a user interface refreshing process, where the picture data is rendered according to a drawing hierarchy of the top-level view;
a first judging module 32, configured to judge whether a pixel is overdrawn according to the color of the pixel in the picture data acquired by the acquiring module 31;
a first determining module 33, configured to determine a ratio of the number of pixels excessively drawn in the picture data determined by the first determining module 32 to the total number of pixels;
a second judging module 34, configured to judge whether the ratio determined by the first determining module 33 is greater than a preset ratio threshold;
and a second determining module 35, configured to determine that the user interface is overdrawn when the second judging module 34 judges yes.
In one embodiment, the first determining module 32 is specifically configured to:
converting RGB values of pixel point colors into HSI values; judging whether the H component value of the HSI value is larger than or equal to a preset H component minimum threshold value and smaller than or equal to a preset H component maximum threshold value; if yes, determining that the pixel point is overdrawn.
In one embodiment, the obtaining module 31 is specifically configured to:
and acquiring picture data of the top-level view of the user interface rendered in the user interface refreshing process through a pre-inserted probe, wherein the probe is an execution code for acquiring the picture data.
In one embodiment, the apparatus further comprises:
the inserting module 36 is configured to search all sub-categories of Activity by traversing the source code during compiling the source code of the application program, where each sub-category corresponds to a user interface; and inserting probes into the refreshing method part of the top view source codes of the subclasses.
In one embodiment, the insertion module 36 is further configured to:
before a probe is inserted into a refreshing method part of the top view source code of the subclass, acquiring an identifier of the top view of the subclass; if the acquisition fails, setting the identifier of the top view in the source code according to a set rule, and then executing the probe insertion part of the refreshing method of the top view source code in the subclass; if the acquisition is successful, searching whether the top-level view identifier exists in a pre-established view list without inspection; if not, executing the refreshing method part of the top view source codes of the subclasses to insert probes.
In one embodiment, the insertion module 36 is specifically configured to:
and acquiring the identification of the top layer view of the subclass from the onStart method source code of the subclass.
In one embodiment, the obtaining module 31 is further configured to:
determining an HSI value of the pixel point according to the RGB value of the pixel point of the top-level view; judging whether the H component value of the HSI value is larger than or equal to the preset H component minimum threshold value and smaller than or equal to the preset H component maximum threshold value; if yes, the RGB value of the pixel point is modified, so that the H component value corresponding to the modified RGB value is larger than the preset maximum H component threshold or smaller than the preset minimum H component threshold.
In one embodiment, the apparatus further comprises:
a recording module 37, configured to record the following information of the top-level view of the user interface after determining that the user interface is overdrawn: and the identification of the top-layer view, the H component of the over-drawn pixels in the top-layer view, the number of the over-drawn pixels in the top-layer view, and the ratio of the number of the over-drawn pixels in the top-layer view to the total number of the pixels.
The specific manner in which the various modules perform the operations in the apparatus of the above embodiments have been described in detail in connection with the embodiments of the method, and will not be described in detail herein.
Based on the same inventive concept, the embodiments of the present invention also provide a computer-readable storage medium having stored thereon computer instructions that, when executed by a processor, implement the above-described method of detecting user interface overdrawing.
Based on the same inventive concept, an embodiment of the present invention further provides a server, including: the system comprises a memory, a processor and a computer program stored in the memory and capable of running on the processor, wherein the processor realizes the method for detecting the overdrawing of the user interface when executing the program.
Unless specifically stated otherwise, terms such as processing, computing, calculating, determining, displaying, or the like, may refer to an action and/or process of one or more processing or computing systems, or similar devices, that manipulates and transforms data represented as physical (e.g., electronic) quantities within the processing system's registers or memories into other data similarly represented as physical quantities within the processing system's memories, registers or other such information storage, transmission or display devices. Information and signals may be represented using any of a variety of different technologies and techniques. For example, data, instructions, commands, information, signals, bits, symbols, and chips that may be referenced throughout the above description may be represented by voltages, currents, electromagnetic waves, magnetic fields or particles, optical fields or particles, or any combination thereof.
It should be understood that the specific order or hierarchy of steps in the processes disclosed are examples of exemplary approaches. Based on design preferences, it is understood that the specific order or hierarchy of steps in the processes may be rearranged without departing from the scope of the present disclosure. The accompanying method claims present elements of the various steps in a sample order, and are not meant to be limited to the specific order or hierarchy presented.
In the foregoing detailed description, various features are grouped together in a single embodiment for the purpose of streamlining the disclosure. This method of disclosure is not to be interpreted as reflecting an intention that the claimed embodiments of the subject matter require more features than are expressly recited in each claim. Rather, as the following claims reflect, invention lies in less than all features of a single disclosed embodiment. Thus the following claims are hereby expressly incorporated into this detailed description, with each claim standing on its own as a separate preferred embodiment of this invention.
Those of skill would further appreciate that the various illustrative logical blocks, modules, circuits, and algorithm steps described in connection with the embodiments disclosed herein may be implemented as electronic hardware, computer software, or combinations of both. To clearly illustrate this interchangeability of hardware and software, various illustrative components, blocks, modules, circuits, and steps have been described above generally in terms of their functionality. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the overall system. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present disclosure.
The steps of a method or algorithm described in connection with the embodiments disclosed herein may be embodied directly in hardware, in a software module executed by a processor, or in a combination of the two. A software module may reside in RAM memory, flash memory, ROM memory, EPROM memory, EEPROM memory, registers, hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art. An exemplary storage medium is coupled to the processor such the processor can read information from, and write information to, the storage medium. In the alternative, the storage medium may be integral to the processor. The processor and the storage medium may reside in an ASIC. The ASIC may reside in a user terminal. The processor and the storage medium may reside as discrete components in a user terminal.
For a software implementation, the techniques described herein may be implemented with modules (e.g., procedures, functions, and so on) that perform the functions described herein. These software codes may be stored in memory units and executed by processors. The memory unit may be implemented within the processor or external to the processor, in which case it can be communicatively coupled to the processor via various means as is known in the art.
The foregoing description includes examples of one or more embodiments. It is, of course, not possible to describe every conceivable combination of components or methodologies for purposes of describing the aforementioned embodiments, but one of ordinary skill in the art may recognize that many further combinations and permutations of various embodiments are possible. Accordingly, the embodiments described herein are intended to embrace all such alterations, modifications and variations that fall within the scope of the appended claims. Furthermore, as used in the specification or claims, the term "comprising" is intended to be inclusive in a manner similar to the term "comprising," as interpreted when employed as a transitional word in a claim. Furthermore, any use of the term "or" in the specification of the claims is intended to mean "non-exclusive or".
Claims (7)
1. A method for detecting overdrawing of a user interface, comprising:
acquiring picture data of a top layer view of a user interface rendered in a user interface refreshing process through a pre-inserted probe, wherein the picture data is rendered according to a drawing hierarchy of the top layer view, the probe is an execution code for acquiring the picture data, and the probe is pre-inserted in the following manner: traversing the source codes to search all sub-categories of the Activity in the source code compiling process of the application program, wherein each sub-category corresponds to one user interface; inserting probes into a refreshing method part of the top view source codes of the subclasses; before the probe is inserted into the refreshing method part of the top view source code of the subclass, the method further comprises: acquiring the identification of the top view of the subclass; if the acquisition fails, setting the identifier of the top view in the source code according to a set rule, and then executing the probe insertion part of the refreshing method of the top view source code in the subclass; if the acquisition is successful, searching whether the top-level view identifier exists in a pre-established view list without inspection; if not, executing the probe insertion part of the refreshing method of the top view source codes of the subclasses;
judging whether the pixel points are excessively drawn according to the colors of the pixel points in the picture data;
determining the ratio of the number of over-drawn pixel points to the total number of pixel points in the picture data, and judging whether the ratio is larger than a preset ratio threshold value or not;
if yes, determining that the user interface is overdrawn.
2. The method of claim 1, wherein determining whether a pixel is overdrawn based on a color of the pixel in the picture data comprises:
converting RGB values of pixel point colors into HSI values;
judging whether the H component value of the HSI value is larger than or equal to a preset H component minimum threshold value and smaller than or equal to a preset H component maximum threshold value;
if yes, determining that the pixel point is overdrawn.
3. The method of claim 1, wherein obtaining an identification of a top-level view of the sub-class comprises:
and acquiring the identification of the top layer view of the subclass from the onStart method source code of the subclass.
4. The method of claim 1, further comprising, during source code compilation of the application:
determining an HSI value of the pixel point according to the RGB value of the pixel point of the top-level view;
judging whether the H component value of the HSI value is larger than or equal to the preset H component minimum threshold value and smaller than or equal to the preset H component maximum threshold value;
if yes, the RGB value of the pixel point is modified, so that the H component value corresponding to the modified RGB value is larger than the preset maximum H component threshold or smaller than the preset minimum H component threshold.
5. The method of claim 3, wherein determining that the user interface is overdrawn further comprises:
the following information of the top layer view of the user interface is recorded:
and the identification of the top-layer view, the H component of the over-drawn pixels in the top-layer view, the number of the over-drawn pixels in the top-layer view, and the ratio of the number of the over-drawn pixels in the top-layer view to the total number of the pixels.
6. A user interface overdrawing detection apparatus, comprising:
the acquisition module is used for acquiring picture data of a top layer view of the user interface rendered in a user interface refreshing process through a pre-inserted probe, wherein the picture data is rendered according to a drawing level of the top layer view, the probe is an execution code for acquiring the picture data, and the probe is pre-inserted in the following mode: traversing the source codes to search all sub-categories of the Activity in the source code compiling process of the application program, wherein each sub-category corresponds to one user interface; inserting probes into a refreshing method part of the top view source codes of the subclasses; before the probe is inserted into the refreshing method part of the top view source code of the subclass, the method further comprises: acquiring the identification of the top view of the subclass; if the acquisition fails, setting the identifier of the top view in the source code according to a set rule, and then executing the probe insertion part of the refreshing method of the top view source code in the subclass; if the acquisition is successful, searching whether the top-level view identifier exists in a pre-established view list without inspection; if not, executing the probe insertion part of the refreshing method of the top view source codes of the subclasses;
the first judging module is used for judging whether the pixel points are excessively drawn according to the colors of the pixel points in the picture data acquired by the acquiring module;
the first determining module is used for determining the proportion of the number of the over-drawn pixel points in the picture data judged by the first judging module to the total number of the pixel points;
the second judging module is used for judging whether the proportion determined by the first determining module is larger than a preset proportion threshold value or not;
and the second determining module is used for determining that the user interface is overdrawn when the second judging module judges that the user interface is overdrawn.
7. A computer readable storage medium having stored thereon computer instructions, which when executed by a processor, implement the method of detecting overdrawing of a user interface of any of claims 1-5.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910138522.0A CN111611151B (en) | 2019-02-25 | 2019-02-25 | Method and device for detecting overdrawing of user interface |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910138522.0A CN111611151B (en) | 2019-02-25 | 2019-02-25 | Method and device for detecting overdrawing of user interface |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN111611151A CN111611151A (en) | 2020-09-01 |
| CN111611151B true CN111611151B (en) | 2023-05-16 |
Family
ID=72199927
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910138522.0A Active CN111611151B (en) | 2019-02-25 | 2019-02-25 | Method and device for detecting overdrawing of user interface |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111611151B (en) |
Families Citing this family (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113568840B (en) * | 2021-08-13 | 2024-04-02 | 上海幻电信息科技有限公司 | Method and system for identifying transition drawing |
| CN114332311B (en) * | 2021-12-05 | 2023-08-04 | 北京字跳网络技术有限公司 | Image generation method, device, computer equipment and storage medium |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102650946A (en) * | 2012-03-22 | 2012-08-29 | 杭州梵艺科技有限公司 | Method for loading and displaying graphical interface at hand-held terminal |
| CN105654531A (en) * | 2015-12-30 | 2016-06-08 | 北京金山安全软件有限公司 | Method and device for drawing image contour |
| CN107203383A (en) * | 2017-05-25 | 2017-09-26 | 努比亚技术有限公司 | A kind of user interface method of adjustment and mobile terminal |
| WO2017212686A1 (en) * | 2016-06-08 | 2017-12-14 | 株式会社ソニー・インタラクティブエンタテインメント | Image generation device and image generation method |
| CN108958893A (en) * | 2017-05-23 | 2018-12-07 | 中国移动通信集团重庆有限公司 | Resource control method, device and the computer readable storage medium of high concurrent business |
| CN109165058A (en) * | 2018-07-06 | 2019-01-08 | 北京三快在线科技有限公司 | A kind of image processing method and device |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US6963985B2 (en) * | 2002-02-22 | 2005-11-08 | Intel Corporation | Automatic power down |
-
2019
- 2019-02-25 CN CN201910138522.0A patent/CN111611151B/en active Active
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102650946A (en) * | 2012-03-22 | 2012-08-29 | 杭州梵艺科技有限公司 | Method for loading and displaying graphical interface at hand-held terminal |
| CN105654531A (en) * | 2015-12-30 | 2016-06-08 | 北京金山安全软件有限公司 | Method and device for drawing image contour |
| WO2017212686A1 (en) * | 2016-06-08 | 2017-12-14 | 株式会社ソニー・インタラクティブエンタテインメント | Image generation device and image generation method |
| CN108958893A (en) * | 2017-05-23 | 2018-12-07 | 中国移动通信集团重庆有限公司 | Resource control method, device and the computer readable storage medium of high concurrent business |
| CN107203383A (en) * | 2017-05-25 | 2017-09-26 | 努比亚技术有限公司 | A kind of user interface method of adjustment and mobile terminal |
| CN109165058A (en) * | 2018-07-06 | 2019-01-08 | 北京三快在线科技有限公司 | A kind of image processing method and device |
Non-Patent Citations (1)
| Title |
|---|
| 田师聪.基于实例化的实时三维场景绘制加速研究与实现.中国优秀硕士学位论文全文数据库.2014,全文. * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN111611151A (en) | 2020-09-01 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109086203B (en) | Page detection method and device | |
| CN107025174B (en) | Method, device and readable storage medium for user interface anomaly test of equipment | |
| US10019346B2 (en) | Generating software test script from video | |
| CN108764352B (en) | Method and device for detecting repeated page content | |
| CN109993019A (en) | Two-dimensional code identification method, system and equipment and medium based on connected domain analysis | |
| CN111292302B (en) | Screen detection method and device | |
| CN104978270A (en) | Automatic software testing method and apparatus | |
| US10621066B2 (en) | Automatic repair of scripts | |
| CN104391797A (en) | GUI (graphical user interface) widget identification method and device | |
| CN111611151B (en) | Method and device for detecting overdrawing of user interface | |
| US9600298B2 (en) | Active and efficient monitoring of a graphical user interface | |
| Sun et al. | Ui components recognition system based on image understanding | |
| CN108121648B (en) | Interface error monitoring method | |
| CN103106859B (en) | The detection method of display screen and device | |
| CN111611032B (en) | User interface drawing level and excessive drawing detection method and device | |
| CN104809053A (en) | Control style testing method and device | |
| CN115222653B (en) | Test method and device | |
| CN107818051B (en) | Test case jump analysis method and device and server | |
| CN113805861B (en) | Code generation method based on machine learning, code editing system and storage medium | |
| CN111046878B (en) | Data processing method and device, computer storage medium and computer | |
| CN104572450A (en) | Method and device for detecting resource allocation | |
| CN106155728B (en) | Method and device for detecting fonts of browser supporting canvas | |
| CN112380134A (en) | WebUI automatic testing method based on image recognition | |
| CN111582262A (en) | Segment type liquid crystal picture content identification method, device, equipment and storage medium | |
| CN111538657A (en) | Page over-drawing detection method and device, electronic equipment and readable medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |