CN114691962B - Mobile terminal page crawler method and device and electronic equipment - Google Patents

Mobile terminal page crawler method and device and electronic equipment Download PDF

Info

Publication number
CN114691962B
CN114691962B CN202210438572.2A CN202210438572A CN114691962B CN 114691962 B CN114691962 B CN 114691962B CN 202210438572 A CN202210438572 A CN 202210438572A CN 114691962 B CN114691962 B CN 114691962B
Authority
CN
China
Prior art keywords
crawled
mobile terminal
page
terminal page
elements
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN202210438572.2A
Other languages
Chinese (zh)
Other versions
CN114691962A (en
Inventor
喻纯
张卡尔
史元春
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Tsinghua University
Original Assignee
Tsinghua University
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Tsinghua University filed Critical Tsinghua University
Priority to CN202210438572.2A priority Critical patent/CN114691962B/en
Publication of CN114691962A publication Critical patent/CN114691962A/en
Application granted granted Critical
Publication of CN114691962B publication Critical patent/CN114691962B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/90—Details of database functions independent of the retrieved data types
    • G06F16/95—Retrieval from the web
    • G06F16/951—Indexing; Web crawling techniques
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/90—Details of database functions independent of the retrieved data types
    • G06F16/95—Retrieval from the web
    • G06F16/958—Organisation or management of web site content, e.g. publishing, maintaining pages or automatic linking

Landscapes

  • Engineering & Computer Science (AREA)
  • Databases & Information Systems (AREA)
  • Theoretical Computer Science (AREA)
  • Data Mining & Analysis (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • User Interface Of Digital Computer (AREA)
  • Telephone Function (AREA)

Abstract

The invention provides a mobile terminal page crawler method, a device and electronic equipment, wherein the method utilizes an interface element identification model to process a mobile terminal page obtained by screen capturing operation to obtain element information of all UI elements on the mobile terminal page, then calculates similarity between the mobile terminal page to be crawled and the screen capturing page of the crawled UI elements according to the element information of all the UI elements on the mobile terminal page to be crawled, and finally judges whether to crawl the UI elements in the mobile terminal page to be crawled according to the obtained similarity; and processing the mobile terminal page obtained by screen capturing operation by using the interface element identification model to obtain accurate element information of all UI elements on the mobile terminal page, so that the noise of the obtained UI elements is reduced, and the effective data quantity of the extracted UI elements is improved.

Description

Mobile terminal page crawler method and device and electronic equipment
Technical Field
The invention relates to the technical field of computers, in particular to a mobile terminal page crawler method, a mobile terminal page crawler device and electronic equipment.
Background
At present, a deep learning model based on big data training provides convenience for life of people in aspects of face recognition, machine translation, voice recognition and the like. The web crawler technology provides an important data source for the deep learning model through automatic data collection.
In order to obtain User Interface (UI) elements (such as text, buttons, pictures and other UI elements of different UI element types) on the mobile terminal page, only UI elements displayed on the mobile terminal page can be obtained based on the system API (accessibility API) interface and metadata of the mobile terminal, but the UI elements of the mobile terminal generated based on the API interface and metadata have very loud noise and less effective data.
Disclosure of Invention
In order to solve the problems, an embodiment of the invention aims to provide a mobile terminal page crawler method, a mobile terminal page crawler device and electronic equipment.
In a first aspect, an embodiment of the present invention provides a mobile terminal page crawler method, including:
performing screen capturing operation on a current display page of the mobile terminal to obtain a mobile terminal page to be crawled;
processing the mobile terminal page by using an interface element identification model to obtain element information of all UI elements on the mobile terminal page;
calculating the similarity between the mobile terminal page to be crawled and the screen capturing page of the crawled UI elements according to the element information of all the UI elements on the mobile terminal page to be crawled;
When the calculated similarity is smaller than a similarity threshold, determining the mobile terminal page to be crawled as a screen capturing page of the UI element which is not crawled, and crawling the UI element in the mobile terminal page to be crawled
In a second aspect, an embodiment of the present invention further provides a mobile terminal page crawler apparatus, including:
the screen capturing module is used for carrying out screen capturing operation on a current display page of the mobile terminal to obtain a mobile terminal page to be crawled;
the processing module is used for processing the mobile terminal page by utilizing an interface element identification model to obtain element information of all UI elements on the mobile terminal page;
The computing module is used for computing the similarity between the mobile terminal page to be crawled and the screen capturing page of the crawled UI elements according to the element information of all the UI elements on the mobile terminal page to be crawled;
the crawling module is used for determining the mobile terminal page to be crawled as a screen capturing page of the UI element which is not crawled when the calculated similarity is smaller than a similarity threshold value, and crawling the UI element in the mobile terminal page to be crawled
In a third aspect, embodiments of the present invention also provide a computer-readable storage medium having stored thereon a computer program which, when executed by a processor, performs the steps of the method of the first aspect described above.
In a fourth aspect, an embodiment of the present invention further provides an electronic device, where the electronic device includes a memory, a processor, and one or more programs, where the one or more programs are stored in the memory and configured to execute the steps of the method described in the first aspect by the processor.
In the solutions provided in the first to fourth aspects of the embodiments of the present invention, an interface element identification model is used to process a mobile terminal page obtained by screen capturing operation, so as to obtain element information of all UI elements on the mobile terminal page, then, according to the element information of all UI elements on the mobile terminal page to be crawled, a similarity between the mobile terminal page to be crawled and the screen capturing page of the crawled UI elements is calculated, and finally, according to the obtained similarity, whether to crawl the UI elements in the mobile terminal page to be crawled is determined; compared with the mode that the UI elements displayed on the mobile terminal page can only be obtained based on the system API interface and metadata of the mobile terminal in the related art, the method has the advantages that the interface element identification model is utilized to process the mobile terminal page obtained through screen capturing operation, so that accurate element information of all the UI elements on the mobile terminal page is obtained, noise of the obtained UI elements is reduced, and the effective data quantity of the extracted UI elements is greatly improved; in addition, the UI element can be obtained only by the mobile terminal page obtained through screen capturing operation of the interface element identification model, and is irrelevant to a specific operating system, and a software developer is not required to provide metadata, so that crawling on any mobile terminal operating system with a graphical interface is supported.
In order to make the above objects, features and advantages of the present invention more comprehensible, preferred embodiments accompanied with figures are described in detail below.
Drawings
In order to more clearly illustrate the embodiments of the invention or the technical solutions in the prior art, the drawings that are required in the embodiments or the description of the prior art will be briefly described, it being obvious that the drawings in the following description are only some embodiments of the invention, and that other drawings may be obtained according to these drawings without inventive effort for a person skilled in the art.
Fig. 1 shows a flowchart of a mobile terminal page crawler method provided in embodiment 1 of the present invention;
Fig. 2 shows a schematic structural diagram of a mobile terminal page crawler device according to embodiment 2 of the present invention;
fig. 3 shows a schematic structural diagram of an electronic device according to embodiment 3 of the present invention.
Detailed Description
In the description of the present invention, it should be understood that the terms "center", "longitudinal", "lateral", "length", "width", "thickness", "upper", "lower", "front", "rear", "left", "right", "vertical", "horizontal", "top", "bottom", "inner", "outer", "clockwise", "counterclockwise", etc. indicate orientations or positional relationships based on the orientations or positional relationships shown in the drawings are merely for convenience in describing the present invention and simplifying the description, and do not indicate or imply that the devices or elements referred to must have a specific orientation, be configured and operated in a specific orientation, and thus should not be construed as limiting the present invention.
Furthermore, the terms "first," "second," and the like, are used for descriptive purposes only and are not to be construed as indicating or implying a relative importance or implicitly indicating the number of technical features indicated. Thus, a feature defining "a first" or "a second" may explicitly or implicitly include one or more such feature. In the description of the present invention, the meaning of "a plurality" is two or more, unless explicitly defined otherwise.
At present, a deep learning model based on big data training provides convenience for life of people in aspects of face recognition, machine translation, voice recognition and the like. The web crawler technology provides an important data source for the deep learning model through automatic data collection.
However, implementing crawler technology on mobile terminals is difficult due to the lack of unique identifiers on mobile terminal pages, the lack of metadata that accurately describes interface elements, and so forth. The lack of application metadata of the mobile terminal limits the use scenes of intelligent interaction tools such as voice assistants, screen readers and the like, so that extra manual metadata labeling cost is brought to application developers, and related research is also hindered.
With the development of deep learning technology, the interactive mode of using the mobile terminal by the user is gradually automated, intelligent and simplified. The machine automatic understanding of mobile terminal applications offers more possibilities for intelligent interaction. Recent research uses an interface semantic understanding method based on object detection, page metadata can be predicted only from screen pixel data, and semantic understanding of a machine on a single page is realized. However, few studies have considered the interaction path between pages and solved the problem of machine understanding mobile terminal applications at the multi-page level. Meanwhile, a high-quality data set with a complete page interaction tree is lacking at present, the related automatic crawling method of mobile terminal screen data depends on a system interface or metadata and is limited by an operating system where the mobile terminal screen data is located, the quality of the mobile terminal screen data cannot be ensured, only page information is generally provided, and complete page interaction tree data is not generated.
The quality of the screen data set of the large-scale mobile terminal disclosed at present is not high. Deka et al in 2017 propose a RICO dataset. The RICO dataset is the most widely used public dataset of android screen data currently in standard practice, and is obtained using automated searching and manual mixing methods. Leiva et al have manually examined the RICO dataset in 2020 and found that about 90% of the data had problems with missing marks, mismarks or overlapping each other. They propose ENRICO datasets on the basis of RICO. Nonetheless, advances in the design style of mobile terminal UIs have prevented models trained under the 2017 dataset from functioning well on the current mobile terminal screen shots. To solve this problem, many studies have chosen to expend a great deal of manpower and financial resources, manually annotating mobile terminal screen data.
The current automatic crawling system for the mobile terminal interface can not meet the requirements of the interactive modeling research of the mobile terminal. The automatic crawling system of the mobile terminal is mainly used for automatic testing of the mobile terminal. Such methods rely on the support of barrier-free data or require modification of the test application code. For applications that do not provide source code, have no obstacle data inaccuracy, the information crawled is very noisy and cannot be used for depth model training.
Based on this, this embodiment proposes a mobile terminal page crawler method, device and electronic equipment, which processes a mobile terminal page obtained by screen capturing operation by using an interface element identification model to obtain element information of all UI elements on the mobile terminal page, then calculates similarity between the mobile terminal page to be crawled and a screen capturing page of the crawled UI elements according to the element information of all UI elements on the mobile terminal page to be crawled, and finally determines whether to crawl the UI elements in the mobile terminal page to be crawled according to the obtained similarity; and processing the mobile terminal page obtained by screen capturing operation by using the interface element identification model to obtain accurate element information of all UI elements on the mobile terminal page, so that the noise of the obtained UI elements is reduced, and the effective data quantity of the extracted UI elements is greatly improved.
In order that the above-recited objects, features and advantages of the present application will become more readily apparent, a more particular description of the application will be rendered by reference to the appended drawings and appended detailed description.
Example 1
The execution main body of the mobile terminal page crawler method provided by the embodiment is a mobile terminal.
Referring to a flowchart of a mobile terminal page crawler method shown in fig. 1, this embodiment proposes a mobile terminal page crawler method, which includes the following specific steps:
and 100, performing screen capturing operation on the current display page of the mobile terminal to obtain a mobile terminal page to be crawled.
In the above step 100, the screen capturing operation is mainly responsible for capturing the screen capturing interface of the mobile terminal, and mainly uses MediaProjection API in the operating system that the mobile terminal runs. And obtaining the authority of the application to read and write the local file of the mobile terminal before capturing the screenshot of the page. Since the screen shots involve privacy issues for the user, mediaProjection API must be invoked under a separate Service (Service). Before connecting to the service, the system will pop-up the window to prompt the user of the mobile terminal that the software is about to prepare a screen capture and request the user for consent. After the user agrees, the screen information of the mobile terminal can be read. To prevent screen sticking, the screen capture operation needs to be implemented in a separate thread and the main thread is notified by the object lock when the screen capture data is ready. And after the main thread receives the signal prepared by the screen capturing data, reading the image information in the buffer area, and completing the screen capturing operation.
The above screen capturing operation process is only an example, and any existing screen capturing operation mode may be used for performing the screen capturing operation on the current display page of the mobile terminal, which is not described herein again.
And 102, processing the mobile terminal page by using an interface element identification model to obtain element information of all UI elements on the mobile terminal page.
In the step 102, the implementation principle of the interface element identification model is based on the YOLOv s model, and then the implementation principle of the interface element identification model can be considered as the prior art.
The specific process of processing the mobile terminal page by using the interface element identification model to obtain the element information of all UI elements on the mobile terminal page is the prior art, and is not described herein.
Element information of the UI element, including but not limited to: confidence of the UI element, UI element type and endpoint position coordinates of the UI element and RGB values of pixel points in the UI element.
The RGB values of the pixel point are the R value, G value and B value of the pixel point.
In one embodiment, the RGB value for pixel 1 may be expressed as pixel 1 (123,132,122). Wherein, the R value is 123; the G value is 132; the B value is 122.
Here, the R value, the G value, and the B value may be any value between 0 and 255, and will not be described in detail herein.
In the process of judging whether the images are similar, a popup window page obtained by performing a screen capturing operation after a current screen display page of the mobile terminal is popup is easily identified as a similar page with a mobile terminal page to be crawled obtained after the screen capturing of the current screen display page, so that false identification is caused; however, the content and the function of the popup page are different from those of the mobile terminal page to be crawled, so that the popup page needs to be determined to be an independent page to be processed in a correct mode.
Therefore, in order to determine whether the mobile terminal page to be crawled is a popup page, before executing the step of calculating the similarity between the mobile terminal page to be crawled and the screen capturing page of the crawled UI element according to the element information of all UI elements on the mobile terminal page to be crawled described in the following step 104, the mobile terminal page crawler method provided in this embodiment may execute the following steps (1) to (4):
(1) Acquiring RGB values of pixel points of all UI elements in the mobile terminal page to be crawled;
(2) Counting the number of pixel points of the pixel points in all the UI elements and the number of pixel points with the same RGB value;
(3) When the ratio of the number of the pixel points with the same RGB value to the number of the pixel points is larger than a first proportional threshold and the RGB values of the pixel points with the same RGB value are in a preset numerical range, determining the mobile terminal page to be crawled as a popup page;
(4) When the ratio of the number of pixels with the same RGB value to the number of pixels is not greater than the first proportional threshold or the RGB values of the pixels with the same RGB value are not all within the preset range of values, the step 104 is continuously performed.
In the step (2), in one embodiment, the RGB value of the pixel 2 is the pixel 2 (122,132,123); then pixel 1 and pixel 2 are not pixels with the same RGB value.
If the RGB value of pixel 2 is pixel 2 (123,132,122); then pixel 2 and pixel 1 are pixels with the same RGB value.
In the above step (3), in one embodiment, the first ratio threshold may be set to 0.3.
In one embodiment, the predetermined range of values may be set to [100,127].
And the first proportional threshold and the preset numerical range are both cached in the mobile terminal.
The RGB values of the pixel points with the same RGB value are within a preset numerical range, which indicates that among the pixel points of all UI elements in the mobile terminal page to be crawled, the RGB values of the pixel points with the RGB values greater than the first proportional threshold are the same, and R values, G values and B values in the same RGB values are all in the preset numerical range, for example, RGB values of 30% of partial images in the mobile terminal page to be crawled are (123,102,122); it is indicated that more than 30% of the partial images in the page of the mobile terminal to be crawled are composed of gray pixels. Then, the mobile terminal page to be crawled is determined to be a popup page.
And for the popup window page, the crawling operation of the UI elements can be directly carried out without image similarity calculation, and the UI elements in the mobile terminal page to be crawled can be obtained.
In the step (4), when the ratio of the number of pixels with the same RGB value to the number of pixels is not greater than the first ratio threshold or the RGB values of the pixels with the same RGB value are not all within the preset value range, it is indicated that the gray pixels in the mobile terminal page to be crawled are less than 30% or the pixels reaching the 30% ratio are not gray pixels, and then the mobile terminal page to be crawled is not determined to be a popup window page, so that the following step 104 is continuously performed.
And 104, calculating the similarity between the mobile terminal page to be crawled and the screen capturing page of the crawled UI element according to the element information of all the UI elements on the mobile terminal page to be crawled.
Specifically, in order to calculate the similarity between the mobile terminal page to be crawled and the screen capturing page of the crawled UI element. The above step 104 may be performed as follows steps (1) to (6):
(1) Selecting a first UI element from the screen capturing page of the crawled UI element and forming a UI element pair with a second UI element on the mobile terminal page to be crawled, wherein the first UI element and the second UI element have the same UI element type;
(2) Calculating to obtain the intersection ratio of the first UI element and the second UI element according to the endpoint position coordinates of the first UI element and the endpoint position coordinates of the second UI element in the UI element pair;
(3) When the calculated cross-over ratio is larger than a cross-over ratio threshold, placing the UI element pair into a candidate matching set;
(4) Performing de-duplication processing on the UI element pair with the first UI element and the UI element pair with the second UI element in the candidate matching set to obtain a UI element matching set; wherein the UI element matching set includes: a plurality of UI element pairs;
(5) Calculating the first similarity and the second similarity of the mobile terminal page to be crawled and the screen capturing page of the crawled UI element according to the following formula:
The similarity1 represents a first similarity between a mobile terminal page to be crawled and a screen capturing page of a crawled UI element; similarity2 represents a second similarity of the mobile terminal page to be crawled to the screen capturing page of the crawled UI element; m AB represents a UI element matching set; a i represents a first UI element in an ith UI element pair in the UI element matching set; b i denotes a second UI element in the i-th UI element pair in the UI element matching set; ioU (a i,bi) represents an intersection ratio of a first UI element and a second UI element in an ith UI element pair in the UI element matching set; conf (a i) represents the confidence level of the first UI element in the ith UI element pair in the UI element matching set; conf (b i) represents the confidence level of the second UI element in the ith UI element pair in the UI element matching set; e A represents a set of UI elements of the screenshot page that have crawled the UI elements; a j represents the jth UI element in the set of UI elements of the screenshot page that has crawled UI elements; conf (a j) represents the confidence level of the jth UI element in the set of UI elements of the screenshot page of the UI element that has been crawled; e B represents a UI element set of the mobile terminal page to be crawled; b j is used for showing the j-th UI element in the UI element set of the mobile terminal page to be crawled; conf (b j) represents the confidence level of the j-th UI element in the UI element set of the mobile terminal page to be crawled;
(6) And determining the smaller similarity of the first similarity and the second similarity as the similarity of the mobile terminal page to be crawled and the screen capturing page of the crawled UI element.
In the step (2), a specific process of calculating the intersection ratio of the first UI element and the second UI element according to the endpoint position coordinates of the first UI element and the endpoint position coordinates of the second UI element in the UI element pair is a prior art, and is not described herein.
In the step (3), the cross ratio threshold is cached in the mobile terminal; the cross ratio threshold may be set to any value between 0.5 and 0.55, and will not be described in detail here.
In the step (4), in order to perform the deduplication processing on the UI element pair having the first UI element and the UI element pair having the second UI element in the candidate matching set, the following specific steps may be performed:
And reserving the UI element pair with the largest cross-over ratio in the first UI element pair and the UI element pair with the largest cross-over ratio in the candidate matching set, and completing the de-duplication processing of the UI element pair with the first UI element and the UI element with the second UI element in the candidate matching set.
The UI element pair with the largest cross-over ratio of the first UI element is the UI element pair with the largest cross-over ratio of the first UI element and the second UI element in the UI element pair with the first UI element.
The UI element pair with the largest cross-over ratio of the second UI element is the UI element pair with the largest cross-over ratio of the first UI element and the second UI element in the UI element pair with the second UI element.
In the step (6), if the similarity between the mobile terminal page to be crawled and the screen capturing page of the crawled UI element is greater than the second proportion threshold, it may be determined that the mobile terminal page to be crawled and the screen capturing page of the crawled UI element are the same page.
The second ratio threshold, which is cached in the mobile terminal, may be set to any value between 0.8 and 0.9, which is not described in detail herein.
And if the similarity between the mobile terminal page to be crawled and the screen capturing page of the crawled UI element is larger than a similarity threshold value, determining that the mobile terminal page to be crawled and the screen capturing page of the crawled UI element are similar pages.
The similarity threshold, which is cached in the mobile terminal, may be set to any value between 0.5 and 0.6, which is not described in detail herein.
The mobile terminal page to be crawled does not need to be crawled by the UI element no matter whether the mobile terminal page to be crawled is a similar page or the same page as the screen capturing page of the crawled UI element.
If the similarity between the mobile terminal page to be crawled and the screen capturing page of the crawled UI element is smaller than the similarity threshold, the following step 106 is continued.
And 106, when the calculated similarity is smaller than a similarity threshold, determining that the mobile terminal page to be crawled is a screen capturing page of the UI element which is not crawled, and crawling the UI element in the mobile terminal page to be crawled.
In order to crawl UI elements in the mobile terminal page to be crawled, the above step 106 may perform the following steps (1) to (4):
(1) When the calculated similarity is smaller than a similarity threshold, processing the screen capturing page of the crawled UI element and the mobile terminal page to be crawled by using a displacement correlation method to obtain a displacement difference between the screen capturing page of the crawled UI element and the mobile terminal page to be crawled;
(2) According to the obtained displacement difference, the mobile terminal page to be crawled is displaced, and the displaced mobile terminal page to be crawled is obtained;
(3) Calculating the similarity between the screen capturing page of the crawled UI element and the mobile terminal page to be crawled after displacement;
(4) And when the similarity between the screen capturing page of the crawled UI element and the mobile terminal page to be crawled after displacement is smaller than a similarity threshold, determining that the mobile terminal page to be crawled is a screen capturing page of the UI element which is not crawled, and crawling the UI element in the mobile terminal page to be crawled.
In the step (1), the specific process of obtaining the displacement difference between the screen capturing page of the crawled UI element and the mobile terminal page to be crawled by using the displacement correlation method to process the screen capturing page of the crawled UI element and the mobile terminal page to be crawled is the prior art, and is not described herein.
In the step (3), the similarity between the screen capturing page of the crawled UI element and the displaced mobile terminal page to be crawled is calculated, which is similar to the specific process of calculating the similarity between the mobile terminal page to be crawled and the screen capturing page of the crawled UI element in the step 104, and is not described herein.
In the step (4), when the similarity between the screen capturing page of the crawled UI element and the displaced mobile terminal page to be crawled is smaller than a similarity threshold, it is indicated that the mobile terminal page to be crawled is an independent page, and the crawling of the UI element needs to be performed by using a crawler system.
And the crawler system runs in the mobile terminal and is used for crawling the UI elements on the screen capturing page.
The process of crawling the UI elements on the screen capturing page by the crawler system is in the prior art, and is not described herein.
As can be seen from the processes described in the steps (1) to (4), if the similarity between the mobile terminal page to be crawled and the screenshot page of the crawled UI element is smaller than the similarity threshold, which may be caused by the difference between the positions of the mobile terminal page to be crawled and the screenshot page of the crawled UI element, in step 106, after the mobile terminal page to be crawled is shifted, the difference between the positions of the mobile terminal page to be crawled and the screenshot page of the crawled UI element is eliminated, and then the judgment of whether the screenshot page of the crawled UI element is an independent page is performed again, only if the similarity between the screenshot page of the crawled UI element and the screenshot page of the mobile terminal page to be crawled is determined to be an independent page, the UI element in the mobile terminal page to be crawled is crawled, so that the crawler system can crawl UI elements on the screenshot page of the crawled UI element repeatedly, and the crawler system access efficiency is improved.
After the UI elements in the mobile terminal page are crawled, a leaf node can be created in the interaction tree, and the mobile terminal page, the crawled UI elements and element information of the UI elements are stored.
The data structure in the interaction tree stores basic information of the interaction tree, and pages, UI elements and element information of the UI elements and interaction paths of each screen shot. The data structure of the interaction tree is saved in JSON file format. The crawler system can load the crawled interaction tree file in the JSON format, and support the last crawl result to continue. The crawler system simultaneously supports the functions of page searching and automatic jumping on the interaction tree. The page interaction tree data ensures the timely storage and restorability of the crawler system result.
The interaction tree may be cached in a server that interacts with the mobile terminal.
In one embodiment, in order to improve the execution efficiency of the crawler system of the mobile terminal, the functions implemented in steps 102 to 106 may be performed in the server.
In summary, this embodiment proposes a mobile terminal page crawler method, which processes a mobile terminal page obtained by screen capturing operation by using an interface element identification model to obtain element information of all UI elements on the mobile terminal page, then calculates similarity between the mobile terminal page to be crawled and a screen capturing page of the crawled UI elements according to the element information of all UI elements on the mobile terminal page to be crawled, and finally determines whether to crawl the UI elements in the mobile terminal page to be crawled according to the obtained similarity; compared with the mode that the UI elements displayed on the mobile terminal page can only be obtained based on the system API interface and metadata of the mobile terminal in the related art, the method has the advantages that the interface element identification model is utilized to process the mobile terminal page obtained through screen capturing operation, so that accurate element information of all the UI elements on the mobile terminal page is obtained, noise of the obtained UI elements is reduced, and the effective data quantity of the extracted UI elements is greatly improved; in addition, the UI element can be obtained only by the mobile terminal page obtained through screen capturing operation of the interface element identification model, and is irrelevant to a specific operating system, and a software developer is not required to provide metadata, so that crawling on any mobile terminal operating system with a graphical interface is supported.
Example 2
The present embodiment provides a mobile terminal page crawler device, configured to execute the mobile terminal page crawler method set forth in the foregoing embodiment 1.
Referring to the schematic structural diagram of a mobile terminal page crawler device shown in fig. 2, this embodiment provides a mobile terminal page crawler device, including:
the screen capturing module 200 is configured to perform screen capturing operation on a current display page of the mobile terminal, so as to obtain a mobile terminal page to be crawled;
the processing module 202 is configured to process the mobile terminal page by using an interface element identification model to obtain element information of all UI elements on the mobile terminal page;
the calculating module 204 is configured to calculate, according to element information of all UI elements on the mobile terminal page to be crawled, similarity between the mobile terminal page to be crawled and a screen capturing page of the crawled UI elements;
And the crawling module 206 is configured to determine that the mobile terminal page to be crawled is a screen capturing page of the UI element that is not crawled when the calculated similarity is smaller than the similarity threshold, and crawl the UI element in the mobile terminal page to be crawled.
Element information of the UI element, including: RGB values of pixel points in the UI element.
The mobile terminal page crawler device that this embodiment provided still includes:
the acquisition unit is used for acquiring RGB values of pixel points of all UI elements in the mobile terminal page to be crawled;
The statistics unit is used for counting the number of the pixel points in all the UI elements and the number of the pixel points with the same RGB value;
The first determining unit is used for determining the mobile terminal page to be crawled as a popup page when the ratio of the number of the pixel points with the same RGB value to the number of the pixel points is larger than a first ratio threshold and the RGB values of the pixel points with the same RGB value are in a preset numerical range;
And the second determining unit is used for continuously executing the function realized by the calculating module when the ratio of the number of the pixels with the same RGB value to the number of the pixels is not larger than the first ratio threshold or the RGB values of the pixels with the same RGB value are not in the preset numerical range.
Further, the element information of the UI element includes: confidence of the UI element, UI element type, and endpoint location coordinates of the UI element.
Specifically, the computing module 204 is specifically configured to:
Selecting a first UI element from the screen capturing page of the crawled UI element and forming a UI element pair with a second UI element on the mobile terminal page to be crawled, wherein the first UI element and the second UI element have the same UI element type;
calculating to obtain the intersection ratio of the first UI element and the second UI element according to the endpoint position coordinates of the first UI element and the endpoint position coordinates of the second UI element in the UI element pair;
when the calculated cross-over ratio is larger than a cross-over ratio threshold, placing the UI element pair into a candidate matching set;
Performing de-duplication processing on the UI element pair with the first UI element and the UI element pair with the second UI element in the candidate matching set to obtain a UI element matching set; wherein the UI element matching set includes: a plurality of UI element pairs;
Calculating the first similarity and the second similarity of the mobile terminal page to be crawled and the screen capturing page of the crawled UI element according to the following formula:
The similarity1 represents a first similarity between a mobile terminal page to be crawled and a screen capturing page of a crawled UI element; similarity2 represents a second similarity of the mobile terminal page to be crawled to the screen capturing page of the crawled UI element; m AB represents a UI element matching set; a i represents a first UI element in an ith UI element pair in the UI element matching set; b i denotes a second UI element in the i-th UI element pair in the UI element matching set; ioU (a i,bi) represents an intersection ratio of a first UI element and a second UI element in an ith UI element pair in the UI element matching set; conf (a i) represents the confidence level of the first UI element in the ith UI element pair in the UI element matching set; conf (b i) represents the confidence level of the second UI element in the ith UI element pair in the UI element matching set; e A represents a set of UI elements of the screenshot page that have crawled the UI elements; a j represents the jth UI element in the set of UI elements of the screenshot page that has crawled UI elements; conf (a j) represents the confidence level of the jth UI element in the set of UI elements of the screenshot page of the UI element that has been crawled; e B represents a UI element set of the mobile terminal page to be crawled; b j is used for showing the j-th UI element in the UI element set of the mobile terminal page to be crawled; conf (b j) represents the confidence level of the j-th UI element in the UI element set of the mobile terminal page to be crawled;
And determining the smaller similarity of the first similarity and the second similarity as the similarity of the mobile terminal page to be crawled and the screen capturing page of the crawled UI element.
In summary, this embodiment proposes a mobile terminal page crawler device, which processes a mobile terminal page obtained by screen capturing operation by using an interface element identification model to obtain element information of all UI elements on the mobile terminal page, then calculates similarity between the mobile terminal page to be crawled and a screen capturing page of the crawled UI elements according to the element information of all UI elements on the mobile terminal page to be crawled, and finally determines whether to crawl the UI elements in the mobile terminal page to be crawled according to the obtained similarity; compared with the mode that the UI elements displayed on the mobile terminal page can only be obtained based on the system API interface and metadata of the mobile terminal in the related art, the method has the advantages that the interface element identification model is utilized to process the mobile terminal page obtained through screen capturing operation, so that accurate element information of all the UI elements on the mobile terminal page is obtained, noise of the obtained UI elements is reduced, and the effective data quantity of the extracted UI elements is greatly improved; in addition, the UI element can be obtained only by the mobile terminal page obtained through screen capturing operation of the interface element identification model, and is irrelevant to a specific operating system, and a software developer is not required to provide metadata, so that crawling on any mobile terminal operating system with a graphical interface is supported.
Example 3
The present embodiment proposes a computer readable storage medium having stored thereon a computer program which, when executed by a processor, performs the steps of the mobile terminal page crawler method described in the above embodiment 1. The specific implementation can be referred to method embodiment 1, and will not be described herein.
In addition, referring to the schematic structural diagram of an electronic device shown in fig. 3, the present embodiment also proposes an electronic device, which includes a bus 51, a processor 52, a transceiver 53, a bus interface 54, a memory 55, and a user interface 56. The electronic device includes a memory 55.
In this embodiment, the electronic device further includes: one or more programs stored on memory 55 and executable on processor 52, configured to be executed by the processor for performing steps (1) through (4) below:
(1) Performing screen capturing operation on a current display page of the mobile terminal to obtain a mobile terminal page to be crawled;
(2) Processing the mobile terminal page by using an interface element identification model to obtain element information of all UI elements on the mobile terminal page;
(3) Calculating the similarity between the mobile terminal page to be crawled and the screen capturing page of the crawled UI elements according to the element information of all the UI elements on the mobile terminal page to be crawled;
(4) And when the calculated similarity is smaller than a similarity threshold, determining that the mobile terminal page to be crawled is a screen capturing page of the UI element which is not crawled, and crawling the UI element in the mobile terminal page to be crawled.
A transceiver 53 for receiving and transmitting data under the control of the processor 52.
Where bus architecture (represented by bus 51), bus 51 may comprise any number of interconnected buses and bridges, with bus 51 linking together various circuits, including one or more processors, represented by processor 52, and memory, represented by memory 55. The bus 51 may also link together various other circuits such as peripheral devices, voltage regulators, power management circuits, etc., as are well known in the art, and therefore, will not be described further in connection with this embodiment. Bus interface 54 provides an interface between bus 51 and transceiver 53. The transceiver 53 may be one element or may be a plurality of elements, such as a plurality of receivers and transmitters, providing a means for communicating with various other apparatus over a transmission medium. For example: the transceiver 53 receives external data from other devices. The transceiver 53 is used to transmit the data processed by the processor 52 to other devices. Depending on the nature of the computing system, a user interface 56 may also be provided, such as a keypad, display, speaker, microphone, joystick.
The processor 52 is responsible for managing the bus 51 and general processing, as described above, running a general purpose operating system. And memory 55 may be used to store data used by processor 52 in performing operations.
Alternatively, processor 52 may be, but is not limited to: a central processing unit, a single chip microcomputer, a microprocessor or a programmable logic device.
It will be appreciated that the memory 55 in embodiments of the invention may be either volatile memory or nonvolatile memory, or may include both volatile and nonvolatile memory. The nonvolatile Memory may be a Read-Only Memory (ROM), a Programmable ROM (PROM), an Erasable PROM (EPROM), an Electrically Erasable EPROM (EEPROM), or a flash Memory. The volatile memory may be random access memory (Random Access Memory, RAM) which acts as external cache memory. By way of example, and not limitation, many forms of RAM are available, such as static random access memory (STATIC RAM, SRAM), dynamic random access memory (DYNAMIC RAM, DRAM), synchronous Dynamic Random Access Memory (SDRAM), double data rate Synchronous dynamic random access memory (Double DATA RATE SDRAM, DDRSDRAM), enhanced Synchronous dynamic random access memory (ENHANCED SDRAM, ESDRAM), synchronous link dynamic random access memory (SYNCHLINK DRAM, SLDRAM), and Direct memory bus random access memory (DRRAM). The memory 55 of the system and method described in this embodiment is intended to comprise, without being limited to, these and any other suitable types of memory.
In some implementations, the memory 55 stores the following elements, executable modules or data structures, or a subset thereof, or an extended set thereof: operating system 551 and application programs 552.
The operating system 551 includes various system programs, such as a framework layer, a core library layer, a driver layer, and the like, for implementing various basic services and processing hardware-based tasks. The application programs 552 include various application programs, such as a media player (MEDIA PLAYER), a Browser (Browser), etc., for implementing various application services. A program for implementing the method of the embodiment of the present invention may be included in the application program 552.
In summary, the present embodiment proposes a computer readable storage medium and an electronic device, where an interface element recognition model is used to process a mobile terminal page obtained by screen capturing operation to obtain element information of all UI elements on the mobile terminal page, then calculate similarity between the mobile terminal page to be crawled and a screen capturing page of the crawled UI elements according to the element information of all UI elements on the mobile terminal page to be crawled, and finally determine whether to crawl the UI elements in the mobile terminal page to be crawled according to the obtained similarity; compared with the mode that the UI elements displayed on the mobile terminal page can only be obtained based on the system API interface and metadata of the mobile terminal in the related art, the method has the advantages that the interface element identification model is utilized to process the mobile terminal page obtained through screen capturing operation, so that accurate element information of all the UI elements on the mobile terminal page is obtained, noise of the obtained UI elements is reduced, and the effective data quantity of the extracted UI elements is greatly improved; in addition, the UI element can be obtained only by the mobile terminal page obtained through screen capturing operation of the interface element identification model, and is irrelevant to a specific operating system, and a software developer is not required to provide metadata, so that crawling on any mobile terminal operating system with a graphical interface is supported.
The foregoing is merely illustrative of the present invention, and the present invention is not limited thereto, and any person skilled in the art will readily recognize that variations or substitutions are within the scope of the present invention. Therefore, the protection scope of the present invention shall be subject to the protection scope of the claims.

Claims (6)

1. The mobile terminal page crawler method is characterized by comprising the following steps of:
performing screen capturing operation on a current display page of the mobile terminal to obtain a mobile terminal page to be crawled;
Processing the mobile terminal page by using an interface element identification model to obtain element information of all UI elements on the mobile terminal page, wherein the element information of the UI elements comprises: RGB values of pixel points in the UI element, confidence of the UI element, type of the UI element and endpoint position coordinates of the UI element;
counting the number of pixel points of the pixel points in all the UI elements and the number of pixel points with the same RGB value;
When the ratio of the number of the pixel points with the same RGB value to the number of the pixel points in all the UI elements is larger than a first ratio threshold and the RGB values of the pixel points with the same RGB value are in a preset numerical range, determining the mobile terminal page to be crawled as a popup page;
when the ratio of the number of pixels with the same RGB value to the number of pixels in all the UI elements is not greater than a first ratio threshold or the RGB values of the pixels with the same RGB value are not in a preset numerical range, selecting a first UI element from a screen capturing page of the crawled UI elements and a second UI element on the mobile terminal page to be crawled to form a UI element pair, wherein the first UI element and the second UI element have the same UI element type;
calculating to obtain the intersection ratio of the first UI element and the second UI element according to the endpoint position coordinates of the first UI element and the endpoint position coordinates of the second UI element in the UI element pair;
when the calculated cross-over ratio is larger than a cross-over ratio threshold, placing the UI element pair into a candidate matching set;
Performing de-duplication processing on the UI element pair with the first UI element and the UI element pair with the second UI element in the candidate matching set to obtain a UI element matching set; wherein the UI element matching set includes: a plurality of UI element pairs;
Calculating the first similarity and the second similarity of the mobile terminal page to be crawled and the screen capturing page of the crawled UI element according to the following formula:
；
；
wherein, Representing a first similarity between a mobile terminal page to be crawled and a screen capturing page of a crawled UI element; /(I)Representing a second similarity of the mobile terminal page to be crawled and the screen capturing page of the crawled UI element; representing a UI element matching set; /(I) Representing a first UI element in an ith UI element pair in the UI element matching set; /(I)Representing a second UI element in an ith UI element pair in the UI element matching set; /(I)Representing the intersection ratio of a first UI element and a second UI element in an ith UI element pair in the UI element matching set; /(I)Representing the confidence level of the first UI element in the ith UI element pair in the UI element matching set; /(I)Representing the confidence level of a second UI element in the ith UI element pair in the UI element matching set; /(I)A set of UI elements representing a screenshot page of the UI elements that have been crawled; /(I)A j-th UI element in the UI element set representing the screen capture page of the crawled UI elements; /(I)Representing the confidence level of the j-th UI element in the UI element set of the screen capturing page of the crawled UI elements; /(I)A UI element set representing a mobile terminal page to be crawled; /(I)Representing a j-th UI element in a UI element set of a mobile terminal page to be crawled; /(I)Representing the confidence level of the j-th UI element in the UI element set of the mobile terminal page to be crawled;
determining the smaller similarity of the first similarity and the second similarity as the similarity of the mobile terminal page to be crawled and the screen capturing page of the crawled UI element;
And when the calculated similarity is smaller than a similarity threshold, determining that the mobile terminal page to be crawled is a screen capturing page of the UI element which is not crawled, and crawling the UI element in the mobile terminal page to be crawled.
2. The method according to claim 1, wherein performing deduplication processing on the UI element pair having the first UI element and the UI element pair having the second UI element in the candidate matching set to obtain a UI element matching set includes:
And reserving the UI element pair with the largest cross-over ratio in the first UI element pair and the UI element pair with the largest cross-over ratio in the candidate matching set, and completing the de-duplication processing of the UI element pair with the first UI element and the UI element with the second UI element in the candidate matching set.
3. The method of claim 1, wherein when the calculated similarity is smaller than a similarity threshold, determining the mobile terminal page to be crawled as a screen capturing page of the UI element, crawling the UI element in the mobile terminal page to be crawled, includes:
When the calculated similarity is smaller than a similarity threshold, processing the screen capturing page of the crawled UI element and the mobile terminal page to be crawled by using a displacement correlation method to obtain a displacement difference between the screen capturing page of the crawled UI element and the mobile terminal page to be crawled;
according to the obtained displacement difference, the mobile terminal page to be crawled is displaced, and the displaced mobile terminal page to be crawled is obtained;
Calculating the similarity between the screen capturing page of the crawled UI element and the mobile terminal page to be crawled after displacement;
And when the similarity between the screen capturing page of the crawled UI element and the mobile terminal page to be crawled after displacement is smaller than a similarity threshold, determining that the mobile terminal page to be crawled is a screen capturing page of the UI element which is not crawled, and crawling the UI element in the mobile terminal page to be crawled.
4. A mobile terminal page crawler apparatus, comprising:
the screen capturing module is used for carrying out screen capturing operation on a current display page of the mobile terminal to obtain a mobile terminal page to be crawled;
The processing module is used for processing the mobile terminal page by utilizing an interface element identification model to obtain element information of all the UI elements on the mobile terminal page, wherein the element information of the UI elements comprises: RGB values of pixel points in the UI element, confidence of the UI element, type of the UI element and endpoint position coordinates of the UI element;
The computing module is used for computing the similarity between the mobile terminal page to be crawled and the screen capturing page of the crawled UI elements according to the element information of all the UI elements on the mobile terminal page to be crawled;
The crawling module is used for determining the mobile terminal page to be crawled as a screen capturing page of the UI element which is not crawled when the calculated similarity is smaller than a similarity threshold value, and crawling the UI element in the mobile terminal page to be crawled; the device further comprises:
the acquisition unit is used for acquiring RGB values of pixel points of all UI elements in the mobile terminal page to be crawled;
The statistics unit is used for counting the number of the pixel points in all the UI elements and the number of the pixel points with the same RGB value;
The first determining unit is used for determining the mobile terminal page to be crawled as a popup page when the ratio of the number of the pixel points with the same RGB value to the number of the pixel points in all the UI elements is larger than a first ratio threshold and the RGB values of the pixel points with the same RGB value are in a preset numerical range;
The second determining unit is used for continuously executing the function realized by the calculating module when the ratio of the number of the pixel points with the same RGB value to the number of the pixel points in all the UI elements is not larger than a first ratio threshold or the RGB values of the pixel points with the same RGB value are not in a preset numerical range;
The computing module is specifically configured to:
Selecting a first UI element from the screen capturing page of the crawled UI element and forming a UI element pair with a second UI element on the mobile terminal page to be crawled, wherein the first UI element and the second UI element have the same UI element type;
calculating to obtain the intersection ratio of the first UI element and the second UI element according to the endpoint position coordinates of the first UI element and the endpoint position coordinates of the second UI element in the UI element pair;
when the calculated cross-over ratio is larger than a cross-over ratio threshold, placing the UI element pair into a candidate matching set;
Performing de-duplication processing on the UI element pair with the first UI element and the UI element pair with the second UI element in the candidate matching set to obtain a UI element matching set; wherein the UI element matching set includes: a plurality of UI element pairs;
Calculating the first similarity and the second similarity of the mobile terminal page to be crawled and the screen capturing page of the crawled UI element according to the following formula:
；
；
wherein, Representing a first similarity between a mobile terminal page to be crawled and a screen capturing page of a crawled UI element; /(I)Representing a second similarity of the mobile terminal page to be crawled and the screen capturing page of the crawled UI element; representing a UI element matching set; /(I) Representing a first UI element in an ith UI element pair in the UI element matching set; /(I)Representing a second UI element in an ith UI element pair in the UI element matching set; /(I)Representing the intersection ratio of a first UI element and a second UI element in an ith UI element pair in the UI element matching set; /(I)Representing the confidence level of the first UI element in the ith UI element pair in the UI element matching set; /(I)Representing the confidence level of a second UI element in the ith UI element pair in the UI element matching set; /(I)A set of UI elements representing a screenshot page of the UI elements that have been crawled; /(I)A j-th UI element in the UI element set representing the screen capture page of the crawled UI elements; /(I)Representing the confidence level of the j-th UI element in the UI element set of the screen capturing page of the crawled UI elements; /(I)A UI element set representing a mobile terminal page to be crawled; /(I)A j-th UI element in a UI element set of the mobile terminal page to be crawled is listed; /(I)Representing the confidence level of the j-th UI element in the UI element set of the mobile terminal page to be crawled;
And determining the smaller similarity of the first similarity and the second similarity as the similarity of the mobile terminal page to be crawled and the screen capturing page of the crawled UI element.
5. A computer-readable storage medium, on which a computer program is stored, characterized in that the computer program, when being executed by a processor, performs the steps of the method of any of the preceding claims 1-3.
6. An electronic device comprising a memory, a processor, and one or more programs, wherein the one or more programs are stored in the memory and configured to perform the steps of the method of any of claims 1-3 by the processor.
CN202210438572.2A 2022-04-25 2022-04-25 Mobile terminal page crawler method and device and electronic equipment Active CN114691962B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202210438572.2A CN114691962B (en) 2022-04-25 2022-04-25 Mobile terminal page crawler method and device and electronic equipment

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202210438572.2A CN114691962B (en) 2022-04-25 2022-04-25 Mobile terminal page crawler method and device and electronic equipment

Publications (2)

Publication Number Publication Date
CN114691962A CN114691962A (en) 2022-07-01
CN114691962B true CN114691962B (en) 2024-04-19

Family

ID=82144955

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202210438572.2A Active CN114691962B (en) 2022-04-25 2022-04-25 Mobile terminal page crawler method and device and electronic equipment

Country Status (1)

Country Link
CN (1) CN114691962B (en)

Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN108595583A (en) * 2018-04-18 2018-09-28 平安科技（深圳）有限公司 Dynamic chart class page data crawling method, device, terminal and storage medium
CN112307386A (en) * 2020-10-29 2021-02-02 北京明略昭辉科技有限公司 Information monitoring method, system, electronic device and computer readable storage medium
WO2021184896A1 (en) * 2020-03-20 2021-09-23 支付宝(杭州)信息技术有限公司 Page screenshot method and device

Family Cites Families (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2016013915A1 (en) * 2014-07-25 2016-01-28 오드컨셉 주식회사 Method, apparatus and computer program for displaying search information

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN108595583A (en) * 2018-04-18 2018-09-28 平安科技（深圳）有限公司 Dynamic chart class page data crawling method, device, terminal and storage medium
WO2021184896A1 (en) * 2020-03-20 2021-09-23 支付宝(杭州)信息技术有限公司 Page screenshot method and device
CN112307386A (en) * 2020-10-29 2021-02-02 北京明略昭辉科技有限公司 Information monitoring method, system, electronic device and computer readable storage medium

Also Published As

Publication number Publication date
CN114691962A (en) 2022-07-01

Similar Documents

Publication Publication Date Title
CN109190209B (en) BIM (building information modeling) -based weak association method and system
US20220253631A1 (en) Image processing method, electronic device and storage medium
CN112131499B (en) Image labeling method, device, electronic equipment and storage medium
CN111539412B (en) Image analysis method, system, device and medium based on OCR
CN111368241B (en) Webpage element identification method based on XPath
CN113177559B (en) Image recognition method, system, equipment and medium combining breadth and dense convolutional neural network
CN109711427A (en) Object detection method and Related product
EP3944145A2 (en) Method and device for training image recognition model, equipment and medium
CN113516113A (en) Image content identification method, device, equipment and storage medium
CN111428168A (en) Browser front-end architecture system capable of being separately developed from rear-end architecture
CN113112185A (en) Teacher expressive force evaluation method and device and electronic equipment
CN114691962B (en) Mobile terminal page crawler method and device and electronic equipment
US10957017B1 (en) Synthetic image detector
CN116453125A (en) Data input method, device, equipment and storage medium based on artificial intelligence
CN112116596A (en) Training method of image segmentation model, image segmentation method, medium, and terminal
CN110555352A (en) interest point identification method, device, server and storage medium
CN114067343A (en) Data set construction method, model training method and corresponding device
US20230186664A1 (en) Method for text recognition
CN110955369A (en) Focus judgment method, device and equipment based on click position and storage medium
CN111209501B (en) Picture display method and device, electronic equipment and storage medium
US20220375118A1 (en) Method and apparatus for identifying vehicle cross-line, electronic device and storage medium
CN117934882A (en) Element identification method and system based on image feature matching
CN117671314A (en) Optimization method of image recognition model and computer readable storage medium
Ulm Annotation of screencasts: Distinguishing Between Relevant and Irrelevant Sections
CN117234769A (en) Page white-screen event processing method, device, equipment and storage medium thereof

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant