CN111611151A - Method and device for detecting excessive drawing of user interface - Google Patents

Method and device for detecting excessive drawing of user interface Download PDF

Info

Publication number
CN111611151A
CN111611151A CN201910138522.0A CN201910138522A CN111611151A CN 111611151 A CN111611151 A CN 111611151A CN 201910138522 A CN201910138522 A CN 201910138522A CN 111611151 A CN111611151 A CN 111611151A
Authority
CN
China
Prior art keywords
user interface
picture data
top view
determining
pixels
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN201910138522.0A
Other languages
Chinese (zh)
Other versions
CN111611151B (en
Inventor
王玉鹏
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Alibaba Group Holding Ltd
Original Assignee
Alibaba Group Holding Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Alibaba Group Holding Ltd filed Critical Alibaba Group Holding Ltd
Priority to CN201910138522.0A priority Critical patent/CN111611151B/en
Publication of CN111611151A publication Critical patent/CN111611151A/en
Application granted granted Critical
Publication of CN111611151B publication Critical patent/CN111611151B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F11/00—Error detection; Error correction; Monitoring
    • G06F11/36—Preventing errors by testing or debugging software
    • G06F11/362—Software debugging
    • G06F11/3624—Software debugging by performing operations on the source code, e.g. via a compiler
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F11/00—Error detection; Error correction; Monitoring
    • G06F11/36—Preventing errors by testing or debugging software
    • G06F11/362—Software debugging
    • G06F11/3636—Software debugging by tracing the execution of the program
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F11/00—Error detection; Error correction; Monitoring
    • G06F11/36—Preventing errors by testing or debugging software
    • G06F11/362—Software debugging
    • G06F11/3644—Software debugging by instrumenting at runtime
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T11/00—2D [Two Dimensional] image generation
    • G06T11/40—Filling a planar surface by adding surface attributes, e.g. colour or texture
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T15/00—3D [Three Dimensional] image rendering
    • G06T15/005—General purpose rendering architectures
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/90—Determination of colour characteristics
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2200/00—Indexing scheme for image data processing or generation, in general
    • G06T2200/24—Indexing scheme for image data processing or generation, in general involving graphical user interfaces [GUIs]
    • Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
    • Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
    • Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Computer Hardware Design (AREA)
  • Quality & Reliability (AREA)
  • General Engineering & Computer Science (AREA)
  • Computer Graphics (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • User Interface Of Digital Computer (AREA)
  • Processing Or Creating Images (AREA)

Abstract

The invention discloses a method and a device for detecting excessive drawing of a user interface. The method comprises the following steps: acquiring picture data of a top view rendered in a user interface refreshing process, wherein the picture data is rendered according to a drawing level of the top view; judging whether the pixel points are excessively drawn according to the colors of the pixel points in the picture data; determining the proportion of the number of pixels excessively drawn in the picture data to the total number of pixels, and judging whether the proportion is greater than a preset proportion threshold value; and if so, determining that the user interface is over-drawn. Excessive drawing can be automatically detected, so that the labor cost is saved, and the detection efficiency and accuracy are improved; the method can help research and development personnel to find problems better, and finally improves the user experience of the application.

Description

Method and device for detecting excessive drawing of user interface
Technical Field
The invention relates to the technical field of application program detection, in particular to a method and a device for detecting excessive drawing of a user interface.
Background
Often, an operating system, such as an Application (APP) of an android system, has an overdraw of a User Interface (UI). Overdraw means that a certain pixel area on the screen is drawn multiple times within the same frame. In order to maintain visual fluency, the screen refresh frequency of the mobile phone is usually set to 60hz, that is, one frame is updated within 16.67ms at 1000/60, and overdraw wastes a lot of CPU and GPU resources, so that sometimes the UI cannot be updated to the next frame within the set frequency, and frame drop occurs, which causes UI deadlock.
In the prior art, the android system provides a detection tool for excessive drawing of a user interface, in a developer mode of a mobile phone, a debugging GPU is turned on to perform excessive drawing, a display GPU excessive drawing area is selected, and different colors can be rendered on elements with different rendering complexity by the system after the system is turned on, specifically: one layer of overdraw is displayed in blue, two layers of overdraw are displayed in green, three layers of overdraw are displayed in light red, and four layers of overdraw are displayed in dark red. And then, the developer can take a screenshot of the mobile phone screen, analyze the screenshot, visually check the area size of a red region rendered in the screenshot and the rendering degree of the red region, judge the over-rendering degree and determine whether the original code needs to be adjusted.
According to the technical scheme, judgment is time-consuming and labor-consuming through a manual visual observation method, and manual subjective judgment is easy to cause errors, especially under the condition that a detector is visually exhausted.
Disclosure of Invention
In view of the above, the present invention has been made to provide a method and apparatus for detecting overdrawing of a user interface that overcomes or at least partially solves the above problems.
In a first aspect, an embodiment of the present invention provides a method for detecting overdraw of a user interface, including:
acquiring picture data of a top view rendered in a user interface refreshing process, wherein the picture data is rendered according to a drawing level of the top view;
judging whether the pixel points are excessively drawn according to the colors of the pixel points in the picture data;
determining the proportion of the number of pixels excessively drawn in the picture data to the total number of pixels, and judging whether the proportion is greater than a preset proportion threshold value;
and if so, determining that the user interface is over-drawn.
In some optional embodiments, determining whether a pixel is over-drawn according to a color of the pixel in the picture data includes:
converting the RGB value of the pixel point color into an HSI value;
judging whether the H component value of the HSI value is greater than or equal to a preset H component minimum threshold value and less than or equal to a preset H component maximum threshold value;
and if so, determining that the pixel point is over-drawn.
In some optional embodiments, acquiring picture data of a top-level view rendered during a user interface refresh process includes:
the method comprises the steps of obtaining picture data of a top view of a user interface rendered in a user interface refreshing process through a pre-inserted probe, wherein the probe is an execution code used for obtaining the picture data.
In some alternative embodiments, the probe is pre-inserted by:
in the process of compiling the source code of the application program, traversing the source code to search out all subclasses of Activity, wherein each subclass corresponds to a user interface;
and inserting probes in the refreshing method part of the top-level view source codes of the subclasses.
In some optional embodiments, before the probe is partially inserted into the refresh method of the top-level view source code of the sub-class, the method further includes:
acquiring an identifier of a top-level view of the subclass;
if the acquisition fails, setting the identifier of the top-level view in the source code according to a set rule, and then executing a part of probe insertion in the refreshing method of the top-level view source code of the subclass;
if the acquisition is successful, searching whether the pre-established inspection-free view list has the identifier of the top view; and if not, partially inserting probes in the refreshing method of the top-level view source codes of the subclasses.
In some optional embodiments, obtaining the identifier of the top-level view of the sub-category includes:
and acquiring the identifier of the top-level view of the subclass in the onStart method source code of the subclass.
In some optional embodiments, during the source code compiling process of the application program, the method further includes:
determining the HSI value of the pixel point according to the RGB value of the pixel point of the top view;
judging whether the H component value of the HSI value is greater than or equal to the preset H component minimum threshold value and less than or equal to the preset H component maximum threshold value;
and if so, modifying the RGB value of the pixel point, so that the H component value corresponding to the modified RGB value is greater than the preset H component maximum threshold value or smaller than the preset H component minimum threshold value.
In some optional embodiments, after determining that the user interface is overdrawn, further comprising:
recording the following information of a top-level view of the user interface:
the top view comprises the identification of the top view, the H component of pixels excessively drawn in the top view, the number of pixels excessively drawn in the top view, and the proportion of the number of pixels excessively drawn in the top view to the total number of pixels.
In a second aspect, an embodiment of the present invention provides an apparatus for detecting overdraw of a user interface, including:
the acquisition module is used for acquiring picture data of a top view rendered in a user interface refreshing process, wherein the picture data is rendered according to a rendering level of the top view;
the first judgment module is used for judging whether the pixel points are excessively drawn according to the colors of the pixel points in the image data acquired by the acquisition module;
the first determining module is used for determining the proportion of the number of pixels which are excessively drawn in the picture data judged by the first judging module to the total number of pixels;
the second judgment module is used for judging whether the proportion determined by the first determination module is larger than a preset proportion threshold value or not;
and the second determination module is used for determining that the user interface is excessively drawn when the second judgment module judges that the user interface is excessively drawn.
In a third aspect, an embodiment of the present invention provides a computer-readable storage medium, on which computer instructions are stored, and when the instructions are executed by a processor, the method for detecting overdraw of a user interface is implemented.
In a fourth aspect, an embodiment of the present invention provides a server, including: the system comprises a memory, a processor and a computer program which is stored on the memory and can run on the processor, and is characterized in that the processor realizes the detection method of the overdrawing of the user interface when executing the program.
The technical scheme provided by the embodiment of the invention has the beneficial effects that at least:
acquiring picture data of a top view rendered in a user interface refreshing process, wherein the picture data is rendered according to a drawing level of the top view; judging whether the pixel points are excessively drawn according to the colors of the pixel points in the picture data; determining the proportion of the number of pixels excessively drawn in the picture data to the total number of pixels, and judging whether the proportion is greater than a preset proportion threshold value; and if so, determining that the user interface is over-drawn. Excessive drawing can be automatically detected, so that the labor cost is saved, and the detection efficiency and accuracy are improved; the method can help research and development personnel to find problems better, and finally improves the user experience of the application.
Additional features and advantages of the invention will be set forth in the description which follows, and in part will be obvious from the description, or may be learned by practice of the invention. The objectives and other advantages of the invention will be realized and attained by the structure particularly pointed out in the written description and claims hereof as well as the appended drawings.
The technical solution of the present invention is further described in detail by the accompanying drawings and embodiments.
Drawings
The accompanying drawings, which are included to provide a further understanding of the invention and are incorporated in and constitute a part of this specification, illustrate embodiments of the invention and together with the description serve to explain the principles of the invention and not to limit the invention. In the drawings:
FIG. 1 is a flow chart of a method for detecting overdrawing of a user interface in an embodiment of the present invention;
FIG. 2 is a flow chart illustrating a method for pre-inserting a probe according to an embodiment of the present invention;
fig. 3 is a schematic structural diagram of a detection apparatus for detecting an over-rendering of a user interface according to an embodiment of the present invention.
Detailed Description
Exemplary embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. While exemplary embodiments of the present disclosure are shown in the drawings, it should be understood that the present disclosure may be embodied in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the scope of the disclosure to those skilled in the art.
In order to solve the problems that manual detection of overdrawing through a screenshot rendered by a system in the prior art is time-consuming, labor-consuming and prone to error, embodiments of the present invention provide a detection method for overdrawing of a user interface, which can automatically detect overdrawing, save labor cost, and improve detection efficiency and accuracy.
The embodiment of the invention provides a method for detecting excessive drawing of a user interface, the flow of which is shown in figure 1, and the method comprises the following steps:
step S11: and acquiring picture data of a top-layer view rendered in the user interface refreshing process.
The picture data is obtained by rendering according to the drawing level of the top view.
For example, in an android system, after an installed application program is detected, after a "debug GPU overdraw switch" (some "display GPU overdraw) in a" developer mode "is turned on according to a corresponding command, in a process of starting and traversing each user interface in the application program, taking one user interface as an example, the system renders a top-level view with different colors according to the rendering complexity of the top-level view in a refreshing process of the user interface, which may specifically be: the drawing level is that the first layer is rendered into blue, the drawing level is that the second layer is rendered into green, the drawing level is that the third layer is rendered into light red, and the drawing level is that the fourth layer is rendered into deep red. Alternatively, other rendering methods may be used.
The command may be obtained by obtaining an instruction input or clicked by a user, or may be obtained from a script command calculated by an upper computer. That is, the operation of turning on the "debug GPU overdraw switch" in the "developer mode" may be a manual intervention operation, or may be automatically performed by the system without manual intervention.
Step S12: and judging whether the pixel points are excessively drawn according to the colors of the pixel points in the picture data.
In one embodiment, after the picture data is acquired, the picture data is saved in a foreign storage space, for example, the picture data may be saved in an SD card of a mobile phone or an external usb disk.
The picture data is not directly judged in the cache, but is asynchronously judged after being stored in the storage space of other places, so that the memory and the computing resource of the system are saved, and the smooth detection is ensured.
Or processing the picture data in the cache to obtain the thumbnail data, and then storing the thumbnail data in a foreign storage space. And judging the thumbnail data in the step.
The picture data is processed to obtain the thumbnail data, and whether the user interface is drawn excessively is judged according to the thumbnail, so that interference factors can be reduced, and the accuracy and the speed of judgment are improved.
Because the color of the pixel point in the image data represents the complexity of rendering, i.e. the degree of overdrawing, the judgment whether the pixel point is overdrawn or not can be, in one embodiment, the RGB value of the pixel point color is converted into the HSI value; and judging whether the H component value of the HSI value is more than or equal to a preset H component minimum threshold value and less than or equal to a preset H component maximum threshold value. When the judgment result is negative, determining that the pixel point is not excessively drawn; and when the judgment result is yes, determining that the pixel point is over-drawn.
Wherein, the RGB values, namely Red, Green and Blue values, represent the colors of the Red, Green and Blue channels; the HSI value, H, Hue, represents the Hue of the color, S, Saturation, represents the Saturation of the color, I, Intensity, represents the Intensity or brightness of the color.
Step S13: and determining the proportion of the number of the pixels excessively drawn in the picture data to the total number of the pixels.
Step S14: and judging whether the ratio of the number of the pixels excessively drawn in the picture data to the total number of the pixels is larger than a preset ratio threshold value or not.
If yes, go to step S15; and when the judgment result is negative, determining that the user interface is not excessively drawn.
Step S15: determining that the user interface is overdrawn.
The detection method for the excessive drawing of the user interface can automatically detect the excessive drawing, so that the labor cost is saved, and the detection efficiency and accuracy are improved; meanwhile, research and development personnel can be better helped to find problems, and finally the user experience degree of the application is improved.
In step S11, the picture data of the top view rendered in the user interface refreshing process may be obtained by using a pre-inserted probe, where the probe is an execution code for obtaining the picture data, and the picture data of the top view rendered in the user interface refreshing process is obtained.
Referring to fig. 2, the probe may be previously inserted in the following manner:
step S21: in the process of compiling the source code of the application program, all the subclasses of Activity are searched out by traversing the source code.
Where each subclass of Activity corresponds to a user interface.
The searching method can be that searching is carried out according to a uniform suffix or prefix name of each subclass; or may be searched based on common attributes of each subclass.
Step S22: an identification of a top level view of the subclass is obtained.
In one embodiment, it may be that the identity of the top-level view of the child is obtained in the onStart method source code of the child.
In an Activity lifecycle, the onStart method is called only once, and the onsroume method may be called many times, so that the identifier of the top-level view of the subclass is obtained in the onStart method of the subclass instead of the onsroume method, thereby avoiding repeated obtaining of the identifier of the top-level view, reducing the calculation amount of the system, and improving the detection speed.
If the obtaining of the identifier of the top-level view of the sub-class fails, that is, the top-level view of the sub-class does not have the identifier, execute step S23; if the obtaining of the identifier of the top-level view of the child class is successful, go to step S25.
Step S23: and setting the identifier of the top-level view in the source code according to the set rule.
And for the top-level view without the identifier, adding the identifier of the top-level view according to a set rule. The method and the device ensure that the detection of which top-level view each top-level view of the drawing level is specifically can be finally obtained, and ensure that when a research and development worker wants to modify the source code, the research and development worker can quickly and conveniently find the position of the specific top-level view in the source code and modify the position.
Step S24: and inserting probes in the refreshing method part of the top-level view source codes.
In the refreshing method of the top-level view, a probe is inserted into a part of the refreshing method of the source code of the top-level view, and in the android system, the probe is inserted into the source code of the onResume method of the source code.
In the process of compiling the application program, dynamically modifying the source code of the application program, inserting a probe, and in the running process of the user interface in the application program, particularly in the onResume process of the top view of each user interface, the probe can acquire rendered picture data.
Step S25: and searching whether the pre-established inspection-free view list has the identification of the top-level view.
If not, go to step S24.
If yes, the top view is described, namely the user interface corresponding to the top view is free from inspection, the detection of the user interface is ignored, a probe is not required to be inserted into the source code of the onResume method, the detection of the drawing level is not required, the calculation amount of monitoring is saved, and the monitoring speed is improved.
The method comprises the steps of obtaining picture data of a top-layer view of a user interface rendered in the user interface refreshing process through a pre-inserted probe, wherein the color of each pixel point may be the color rendered by the system according to the rendering degree of the pixel point or the original color of the pixel point, and therefore whether over-rendering is interfered by the original color is judged according to the color of the pixel point. In one embodiment, interference with its original color may be avoided by:
in the process of compiling and packaging the source code of the application, the following operations can be further executed:
determining the HSI value of the pixel point according to the RGB value of the pixel point of the top view; judging whether the H component value of the HSI value is greater than or equal to a preset H component minimum threshold value and less than or equal to a preset H component maximum threshold value; and if so, modifying the RGB value of the pixel point, so that the H component value corresponding to the modified RGB value is larger than the preset H component maximum threshold value or smaller than the preset H component minimum threshold value. For example, the RGB values of the pixels may be modified to make the corresponding H-value component blue.
And modifying the RGB value of the pixel point according to a set rule, so that the color of the pixel point which is judged to be over-drawn in the acquired picture data is rendered by the system instead of the original color. Therefore, judgment interference caused by over-drawing can be eliminated, and the judgment accuracy and the reliability are improved.
In one embodiment, after determining that a user interface is overdrawn, the following information may also be recorded for the top-level view of the user interface:
the method comprises the steps of identifying a top view, H components of pixels excessively drawn in the top view, the number of the pixels excessively drawn in the top view, and the proportion of the number of the pixels excessively drawn in the top view to the total number of the pixels.
After judging whether each user interface of the application program is over-drawn, an over-drawing report of the application program can be generated according to the recorded information of each over-drawn user interface. The method and the system are convenient for developers to know about the specific situation of over-drawing, can judge whether the source code needs to be adjusted according to the related information, and can quickly position the source code to the position needing to be modified.
Based on the same inventive concept, an embodiment of the present invention further provides a detection apparatus for detecting overdrawing of a user interface, where the structure of the detection apparatus is shown in fig. 3, and the detection apparatus includes:
the acquiring module 31 is configured to acquire picture data of a top view rendered in a user interface refreshing process, where the picture data is rendered according to a rendering level of the top view;
a first determining module 32, configured to determine whether a pixel point is over-drawn according to a color of the pixel point in the image data acquired by the acquiring module 31;
a first determining module 33, configured to determine a ratio of the number of pixels over-drawn in the image data determined by the first determining module 32 to the total number of pixels;
a second judging module 34, configured to judge whether the ratio determined by the first determining module 33 is greater than a preset ratio threshold;
a second determining module 35, configured to determine that the user interface is overdrawn when the second determining module 34 determines yes.
In an embodiment, the first determining module 32 is specifically configured to:
converting the RGB value of the pixel point color into an HSI value; judging whether the H component value of the HSI value is greater than or equal to a preset H component minimum threshold value and less than or equal to a preset H component maximum threshold value; and if so, determining that the pixel point is over-drawn.
In an embodiment, the obtaining module 31 is specifically configured to:
the method comprises the steps of obtaining picture data of a top view of a user interface rendered in a user interface refreshing process through a pre-inserted probe, wherein the probe is an execution code used for obtaining the picture data.
In one embodiment, the above apparatus further comprises:
the inserting module 36 is configured to traverse the source code to search out all subclasses of Activity in a source code compiling process of an application program, where each subclass corresponds to one user interface; and inserting probes in the refreshing method part of the top-level view source codes of the subclasses.
In one embodiment, the insertion module 36 is further configured to:
before a probe is partially inserted into the refreshing method of the top-level view source code of the subclass, acquiring the identifier of the top-level view of the subclass; if the acquisition fails, setting the identifier of the top-level view in the source code according to a set rule, and then executing a part of probe insertion in the refreshing method of the top-level view source code of the subclass; if the acquisition is successful, searching whether the pre-established inspection-free view list has the identifier of the top view; and if not, partially inserting probes in the refreshing method of the top-level view source codes of the subclasses.
In one embodiment, the insertion module 36 is specifically configured to:
and acquiring the identifier of the top-level view of the subclass in the onStart method source code of the subclass.
In one embodiment, the obtaining module 31 is further configured to:
determining the HSI value of the pixel point according to the RGB value of the pixel point of the top view; judging whether the H component value of the HSI value is greater than or equal to the preset H component minimum threshold value and less than or equal to the preset H component maximum threshold value; and if so, modifying the RGB value of the pixel point, so that the H component value corresponding to the modified RGB value is greater than the preset H component maximum threshold value or smaller than the preset H component minimum threshold value.
In one embodiment, the above apparatus further comprises:
a recording module 37, configured to record the following information of the top-level view of the user interface after determining that the user interface is overdrawn: the top view comprises the identification of the top view, the H component of pixels excessively drawn in the top view, the number of pixels excessively drawn in the top view, and the proportion of the number of pixels excessively drawn in the top view to the total number of pixels.
With regard to the apparatus in the above-described embodiment, the specific manner in which each module performs the operation has been described in detail in the embodiment related to the method, and will not be elaborated here.
Based on the same inventive concept, the embodiment of the present invention further provides a computer-readable storage medium, on which computer instructions are stored, and when the instructions are executed by a processor, the method for detecting the overdraw of the user interface is implemented.
Based on the same inventive concept, an embodiment of the present invention further provides a server, including: the system comprises a memory, a processor and a computer program which is stored on the memory and can run on the processor, wherein the processor realizes the detection method for the overdrawing of the user interface when executing the program.
Unless specifically stated otherwise, terms such as processing, computing, calculating, determining, displaying, or the like, may refer to an action and/or process of one or more processing or computing systems or similar devices that manipulates and transforms data represented as physical (e.g., electronic) quantities within the processing system's registers and memories into other data similarly represented as physical quantities within the processing system's memories, registers or other such information storage, transmission or display devices. Information and signals may be represented using any of a variety of different technologies and techniques. For example, data, instructions, commands, information, signals, bits, symbols, and chips that may be referenced throughout the above description may be represented by voltages, currents, electromagnetic waves, magnetic fields or particles, optical fields or particles, or any combination thereof.
It should be understood that the specific order or hierarchy of steps in the processes disclosed is an example of exemplary approaches. Based upon design preferences, it is understood that the specific order or hierarchy of steps in the processes may be rearranged without departing from the scope of the present disclosure. The accompanying method claims present elements of the various steps in a sample order, and are not intended to be limited to the specific order or hierarchy presented.
In the foregoing detailed description, various features are grouped together in a single embodiment for the purpose of streamlining the disclosure. This method of disclosure is not to be interpreted as reflecting an intention that the claimed embodiments of the subject matter require more features than are expressly recited in each claim. Rather, as the following claims reflect, invention lies in less than all features of a single disclosed embodiment. Thus, the following claims are hereby expressly incorporated into the detailed description, with each claim standing on its own as a separate preferred embodiment of the invention.
Those of skill would further appreciate that the various illustrative logical blocks, modules, circuits, and algorithm steps described in connection with the embodiments disclosed herein may be implemented as electronic hardware, computer software, or combinations of both. To clearly illustrate this interchangeability of hardware and software, various illustrative components, blocks, modules, circuits, and steps have been described above generally in terms of their functionality. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the overall system. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present disclosure.
The steps of a method or algorithm described in connection with the embodiments disclosed herein may be embodied directly in hardware, in a software module executed by a processor, or in a combination of the two. A software module may reside in RAM memory, flash memory, ROM memory, EPROM memory, EEPROM memory, registers, hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art. An exemplary storage medium is coupled to the processor such the processor can read information from, and write information to, the storage medium. Of course, the storage medium may also be integral to the processor. The processor and the storage medium may reside in an ASIC. The ASIC may reside in a user terminal. Of course, the processor and the storage medium may reside as discrete components in a user terminal.
For a software implementation, the techniques described herein may be implemented with modules (e.g., procedures, functions, and so on) that perform the functions described herein. The software codes may be stored in memory units and executed by processors. The memory unit may be implemented within the processor or external to the processor, in which case it can be communicatively coupled to the processor via various means as is known in the art.
What has been described above includes examples of one or more embodiments. It is, of course, not possible to describe every conceivable combination of components or methodologies for purposes of describing the aforementioned embodiments, but one of ordinary skill in the art may recognize that many further combinations and permutations of various embodiments are possible. Accordingly, the embodiments described herein are intended to embrace all such alterations, modifications and variations that fall within the scope of the appended claims. Furthermore, to the extent that the term "includes" is used in either the detailed description or the claims, such term is intended to be inclusive in a manner similar to the term "comprising" as "comprising" is interpreted when employed as a transitional word in a claim. Furthermore, any use of the term "or" in the specification of the claims is intended to mean a "non-exclusive or".

Claims (10)

1. A method for detecting overdraw of a user interface, comprising:
acquiring picture data of a top view rendered in a user interface refreshing process, wherein the picture data is rendered according to a drawing level of the top view;
judging whether the pixel points are excessively drawn according to the colors of the pixel points in the picture data;
determining the proportion of the number of pixels excessively drawn in the picture data to the total number of pixels, and judging whether the proportion is greater than a preset proportion threshold value;
and if so, determining that the user interface is over-drawn.
2. The method of claim 1, wherein determining whether a pixel is over-rendered based on a color of the pixel in the picture data comprises:
converting the RGB value of the pixel point color into an HSI value;
judging whether the H component value of the HSI value is greater than or equal to a preset H component minimum threshold value and less than or equal to a preset H component maximum threshold value;
and if so, determining that the pixel point is over-drawn.
3. The method of claim 1, wherein obtaining picture data for a top-level view rendered during a user interface refresh comprises:
the method comprises the steps of obtaining picture data of a top view of a user interface rendered in a user interface refreshing process through a pre-inserted probe, wherein the probe is an execution code used for obtaining the picture data.
4. The method of claim 3, wherein the probe is pre-inserted by:
in the process of compiling the source code of the application program, traversing the source code to search out all subclasses of Activity, wherein each subclass corresponds to a user interface;
and inserting probes in the refreshing method part of the top-level view source codes of the subclasses.
5. The method of claim 4, before the probe is partially inserted into the refresh method of the top-level view source code of the sub-class, further comprising:
acquiring an identifier of a top-level view of the subclass;
if the acquisition fails, setting the identifier of the top-level view in the source code according to a set rule, and then executing a part of probe insertion in the refreshing method of the top-level view source code of the subclass;
if the acquisition is successful, searching whether the pre-established inspection-free view list has the identifier of the top view; and if not, partially inserting probes in the refreshing method of the top-level view source codes of the subclasses.
6. The method of claim 5, wherein obtaining an identification of a top level view of the sub-class comprises:
and acquiring the identifier of the top-level view of the subclass in the onStart method source code of the subclass.
7. The method of claim 4, wherein during source code compilation of the application, further comprising:
determining the HSI value of the pixel point according to the RGB value of the pixel point of the top view;
judging whether the H component value of the HSI value is greater than or equal to the preset H component minimum threshold value and less than or equal to the preset H component maximum threshold value;
and if so, modifying the RGB value of the pixel point, so that the H component value corresponding to the modified RGB value is greater than the preset H component maximum threshold value or smaller than the preset H component minimum threshold value.
8. The method of claim 5 or 6, wherein upon determining that the user interface is overdrawn, further comprising:
recording the following information of a top-level view of the user interface:
the top view comprises the identification of the top view, the H component of pixels excessively drawn in the top view, the number of pixels excessively drawn in the top view, and the proportion of the number of pixels excessively drawn in the top view to the total number of pixels.
9. A user interface overdraw detection apparatus, comprising:
the acquisition module is used for acquiring picture data of a top view rendered in a user interface refreshing process, wherein the picture data is rendered according to a rendering level of the top view;
the first judgment module is used for judging whether the pixel points are excessively drawn according to the colors of the pixel points in the image data acquired by the acquisition module;
the first determining module is used for determining the proportion of the number of pixels which are excessively drawn in the picture data judged by the first judging module to the total number of pixels;
the second judgment module is used for judging whether the proportion determined by the first determination module is larger than a preset proportion threshold value or not;
and the second determination module is used for determining that the user interface is excessively drawn when the second judgment module judges that the user interface is excessively drawn.
10. A computer-readable storage medium having stored thereon computer instructions, which when executed by a processor, implement the method for detecting overdraw in a user interface of any of claims 1-8.
CN201910138522.0A 2019-02-25 2019-02-25 Method and device for detecting overdrawing of user interface Active CN111611151B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201910138522.0A CN111611151B (en) 2019-02-25 2019-02-25 Method and device for detecting overdrawing of user interface

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201910138522.0A CN111611151B (en) 2019-02-25 2019-02-25 Method and device for detecting overdrawing of user interface

Publications (2)

Publication Number Publication Date
CN111611151A true CN111611151A (en) 2020-09-01
CN111611151B CN111611151B (en) 2023-05-16

Family

ID=72199927

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201910138522.0A Active CN111611151B (en) 2019-02-25 2019-02-25 Method and device for detecting overdrawing of user interface

Country Status (1)

Country Link
CN (1) CN111611151B (en)

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113568840A (en) * 2021-08-13 2021-10-29 上海幻电信息科技有限公司 Method and system for recognizing transition drawing
CN114332311A (en) * 2021-12-05 2022-04-12 北京字跳网络技术有限公司 Image generation method and device, computer equipment and storage medium

Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20030167413A1 (en) * 2002-02-22 2003-09-04 Stachura Thomas L. Automatic power down
CN102650946A (en) * 2012-03-22 2012-08-29 杭州梵艺科技有限公司 Method for loading and displaying graphical interface at hand-held terminal
CN105654531A (en) * 2015-12-30 2016-06-08 北京金山安全软件有限公司 Method and device for drawing image contour
CN107203383A (en) * 2017-05-25 2017-09-26 努比亚技术有限公司 A kind of user interface method of adjustment and mobile terminal
WO2017212686A1 (en) * 2016-06-08 2017-12-14 株式会社ソニー・インタラクティブエンタテインメント Image generation device and image generation method
CN108958893A (en) * 2017-05-23 2018-12-07 中国移动通信集团重庆有限公司 Resource control method, device and the computer readable storage medium of high concurrent business
CN109165058A (en) * 2018-07-06 2019-01-08 北京三快在线科技有限公司 A kind of image processing method and device

Patent Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20030167413A1 (en) * 2002-02-22 2003-09-04 Stachura Thomas L. Automatic power down
CN102650946A (en) * 2012-03-22 2012-08-29 杭州梵艺科技有限公司 Method for loading and displaying graphical interface at hand-held terminal
CN105654531A (en) * 2015-12-30 2016-06-08 北京金山安全软件有限公司 Method and device for drawing image contour
WO2017212686A1 (en) * 2016-06-08 2017-12-14 株式会社ソニー・インタラクティブエンタテインメント Image generation device and image generation method
CN108958893A (en) * 2017-05-23 2018-12-07 中国移动通信集团重庆有限公司 Resource control method, device and the computer readable storage medium of high concurrent business
CN107203383A (en) * 2017-05-25 2017-09-26 努比亚技术有限公司 A kind of user interface method of adjustment and mobile terminal
CN109165058A (en) * 2018-07-06 2019-01-08 北京三快在线科技有限公司 A kind of image processing method and device

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
田师聪: "基于实例化的实时三维场景绘制加速研究与实现" *

Cited By (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113568840A (en) * 2021-08-13 2021-10-29 上海幻电信息科技有限公司 Method and system for recognizing transition drawing
CN113568840B (en) * 2021-08-13 2024-04-02 上海幻电信息科技有限公司 Method and system for identifying transition drawing
CN114332311A (en) * 2021-12-05 2022-04-12 北京字跳网络技术有限公司 Image generation method and device, computer equipment and storage medium
CN114332311B (en) * 2021-12-05 2023-08-04 北京字跳网络技术有限公司 Image generation method, device, computer equipment and storage medium

Also Published As

Publication number Publication date
CN111611151B (en) 2023-05-16

Similar Documents

Publication Publication Date Title
CN109086203B (en) Page detection method and device
WO2017005148A1 (en) Automatic software-testing method and device
US8627290B2 (en) Test case pattern matching
US10019346B2 (en) Generating software test script from video
US20110214107A1 (en) Method and system for testing graphical user interfaces
CN109471805B (en) Resource testing method and device, storage medium and electronic equipment
CN110335273B (en) Detection method, detection device, electronic apparatus, and medium
US10176077B2 (en) Generating breakpoints for cross-layer debugging
CN104199767B (en) Test method and device
CN112445693B (en) Page update detection method, device, equipment and storage medium
CN111611151B (en) Method and device for detecting overdrawing of user interface
US20090217259A1 (en) Building Operating System Images Based on Applications
US9529489B2 (en) Method and apparatus of testing a computer program
CN111611032B (en) User interface drawing level and excessive drawing detection method and device
CN112446850B (en) Adaptation test method and device and electronic equipment
JP2013218492A (en) Software test automatic evaluation device and method
CN111488846A (en) Method and equipment for identifying water level
US8291389B2 (en) Automatically detecting non-modifying transforms when profiling source code
CN111538657B (en) Page overdrawing detection method and device, electronic equipment and readable medium
CN106155728B (en) Method and device for detecting fonts of browser supporting canvas
CN104572450A (en) Method and device for detecting resource allocation
CN114512077A (en) Method, device and system for detecting driving time sequence output by receiving card
CN112380134A (en) WebUI automatic testing method based on image recognition
US20160209989A1 (en) Record and replay of operations on graphical objects
CN111611153B (en) Method and device for detecting overdrawing of user interface

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant