CN111580902A - Mobile terminal element positioning method and system based on picture analysis - Google Patents
Mobile terminal element positioning method and system based on picture analysis Download PDFInfo
- Publication number
- CN111580902A CN111580902A CN202010311162.2A CN202010311162A CN111580902A CN 111580902 A CN111580902 A CN 111580902A CN 202010311162 A CN202010311162 A CN 202010311162A CN 111580902 A CN111580902 A CN 111580902A
- Authority
- CN
- China
- Prior art keywords
- picture
- image data
- interface
- needing
- positioning
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000000034 method Methods 0.000 title claims abstract description 27
- 238000004458 analytical method Methods 0.000 title claims abstract description 22
- 238000012545 processing Methods 0.000 claims description 14
- 238000004364 calculation method Methods 0.000 claims description 5
- 238000001914 filtration Methods 0.000 claims description 5
- 238000012360 testing method Methods 0.000 abstract description 13
- 230000006870 function Effects 0.000 description 8
- 238000010586 diagram Methods 0.000 description 6
- 230000008569 process Effects 0.000 description 4
- 238000006243 chemical reaction Methods 0.000 description 2
- 238000013461 design Methods 0.000 description 2
- 238000013459 approach Methods 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 239000003795 chemical substances by application Substances 0.000 description 1
- 239000003086 colorant Substances 0.000 description 1
- 230000003993 interaction Effects 0.000 description 1
- 230000002452 interceptive effect Effects 0.000 description 1
- 238000004573 interface analysis Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000000717 retained effect Effects 0.000 description 1
- 230000011218 segmentation Effects 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/44—Arrangements for executing specific programs
- G06F9/451—Execution arrangements for user interfaces
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/36—Preventing errors by testing or debugging software
- G06F11/3668—Software testing
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/10—Segmentation; Edge detection
- G06T7/11—Region-based segmentation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/10—Segmentation; Edge detection
- G06T7/13—Edge detection
Abstract
A mobile terminal element positioning method and system based on picture analysis are disclosed, the method comprises: acquiring a mobile phone screenshot on a user operation interface to obtain a picture A, wherein the picture A comprises an element needing to be positioned, and acquiring a mobile phone screenshot on the user operation interface which has the same background image as the picture A to obtain a picture B, wherein the picture B does not comprise the element needing to be positioned; respectively acquiring image data a of a picture A and image data B of a picture B; subtracting the image data b from the image data a to obtain image data c, wherein the image data c only contains the elements needing to be positioned; and determining the contour of the image data c, determining the central point of the contour, and positioning the element to be positioned according to the coordinate position of the central point. The method fills the blank that the elements on the picture cannot be positioned by using the WebdriverAgent, so that the automation program can normally test all the function points related to the elements on the picture.
Description
Technical Field
The invention relates to the technical field of mobile terminal test automation, in particular to a mobile terminal element positioning method and system based on picture analysis.
Background
In the prior art, a mobile phone operating system mobile test framework WebDriverAgent is a new mobile phone mobile test framework launched by Facebook in the SeleniumConf conference, and the tool can analyze mobile phone device interface elements through a WEB page, wherein a mobile phone screenshot, an interface element tree and attribute information of a single element, including name, position coordinates and the like, are displayed in a contrasting manner.
When the existing test frame analyzes the interface element, the element above the picture cannot acquire the attribute of the element, in a picture of a style of a nine-square by taking a microblog as an example, a label button randomly appears at a certain position on the picture, although the label button is arranged in the screenshot of an interface analysis tool of the frame, the button is not arranged in an interface element analysis tree, and the attribute information of the element cannot be acquired.
In summary, in the prior art, the WebDriverAgent test framework cannot identify elements on a picture, so that an automation program cannot operate some controls, and thus, an automation function point is incomplete.
Disclosure of Invention
In order to achieve the above object, embodiments of the present invention provide a mobile terminal element positioning method and system based on picture analysis, and the method and system provided by the invention solve the problem that a WebDriverAgent test framework cannot identify elements on a picture, so that an automation program cannot operate some controls, and so that automation function points are incomplete.
In one aspect, an embodiment of the present invention provides a mobile terminal element positioning method based on picture analysis, where the method includes:
acquiring a mobile phone screenshot on a user operation interface to obtain a picture A, wherein the picture A comprises an element needing to be positioned, and acquiring a mobile phone screenshot on the user operation interface which has the same background image as the picture A to obtain a picture B, wherein the picture B does not comprise the element needing to be positioned;
respectively acquiring image data a of a picture A and image data B of a picture B;
subtracting the image data b from the image data a to obtain image data c, wherein the image data c only contains the elements needing to be positioned;
and determining the contour of the image data c, determining the central point of the contour, and positioning the element to be positioned according to the coordinate position of the central point.
On the other hand, an embodiment of the present invention further provides a mobile terminal element positioning system based on picture analysis, where the system includes:
the mobile phone positioning device comprises a picture acquisition unit, a picture acquisition unit and a positioning unit, wherein the picture acquisition unit is used for acquiring a mobile phone screenshot on a user operation interface to obtain a picture A, the picture A comprises an element needing to be positioned, the mobile phone screenshot on the user operation interface which has the same background image as the picture A is acquired to obtain a picture B, and the picture B does not comprise the element needing to be positioned;
the picture reading unit is used for respectively acquiring image data a of a picture A and image data B of a picture B;
the data processing unit is used for subtracting the image data b from the image data a to obtain image data c, and the image data c only contains the elements needing to be positioned;
and the coordinate calculation unit is used for determining the contour of the image data c, determining the central point of the contour and positioning the element to be positioned according to the coordinate position of the central point.
The technical scheme has the following beneficial effects: compared with the prior art, the technical scheme of the invention fills the blank that the elements on the picture cannot be positioned by using WebdriverAgent, so that an automation program can normally test all function points related to the elements on the picture, and the UI automation test of the APP to be tested is more comprehensive and complete.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below, it is obvious that the drawings in the following description are only some embodiments of the present invention, and for those skilled in the art, other drawings can be obtained according to the drawings without creative efforts.
FIG. 1 is a flow chart of a mobile terminal element positioning method based on picture analysis according to an embodiment of the present invention;
FIG. 2 is a schematic diagram of a mobile terminal element location system based on picture analysis according to an embodiment of the present invention;
FIG. 3 is a schematic diagram of panel A in an embodiment of the present invention;
FIG. 4 is a schematic diagram of panel B in an embodiment of the present invention;
FIG. 5 is a diagram of a picture A including an element to be located and its background image according to an embodiment of the present invention;
FIG. 6 is a diagram of a background image B that does not include an element to be located in accordance with an embodiment of the present invention;
FIG. 7 is a picture of image data c in an embodiment of the present invention;
FIG. 8 is a picture of image data c after being grayed according to an embodiment of the present invention;
FIG. 9 is a picture of image data c after being grayed and filtered according to an embodiment of the present invention;
FIG. 10 is a photograph of image data c after being outlined in accordance with an embodiment of the present invention;
fig. 11 is a marked picture of an element that needs to be located in an embodiment of the present invention.
Detailed Description
The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
Abbreviations and key terms involved in the present invention are defined as follows:
1. a mobile terminal: the mobile terminal refers to a computer device which can be used in moving, and includes a mobile phone, a tablet computer, a notebook computer, and the like. This is particularly referred to as a cell phone.
2. A user interface: the ui (user interface) refers to the overall design of human-computer interaction, operation logic and beautiful interface of software. The present invention specifically refers to a user operation interface on a mobile terminal.
3. The mobile phone comprises: the operating system of the mobile terminal published by apple in 2007 is mainly operated on ipod, iphone and ipad series products of apple.
4. Picture analysis: objects of interest in the image are detected and measured to obtain their objective information, thereby creating a description of the image.
5. Element positioning: in the UI automatic test, a user interface consists of a plurality of element objects, the process of searching elements through attributes such as names, label labels, value values and the like of the element objects is element positioning, and subsequent automatic test operations such as inputting, clicking, sliding and the like can be carried out only after the elements are positioned.
6. WebDriverAgent: WebDriverAgent is a Web driver agent, WDA for short. It is a mobile terminal test framework pushed by Facebook. The Server of the WebDriver is realized at the mobile phone client, and by means of the Server, the mobile phone equipment can be remotely controlled to test.
7. OpenCv: OpenCv is a cross-platform computer vision library issued on the basis of BSD license, and can run on Linux, Windows, Android and Mac OS operating systems. The method is light and efficient, is composed of a series of C functions and a small number of C + + classes, provides interfaces of languages such as Python, Ruby, MATLAB and the like, and realizes a plurality of general algorithms in the aspects of image processing and computer vision.
As shown in fig. 1, which is a flowchart of a mobile terminal element positioning method based on picture analysis according to an embodiment of the present invention, the present invention is applicable to an IOS system and an android system, and the method includes:
acquiring a mobile phone screenshot on a user operation interface to obtain a picture A, wherein the picture A comprises an element needing to be positioned, and acquiring a mobile phone screenshot on the user operation interface which has the same background image as the picture A to obtain a picture B, wherein the picture B does not comprise the element needing to be positioned;
respectively acquiring image data a and image data B of a picture A and a picture B, specifically, reading the picture A and the picture B to obtain the image data a of the picture A and the image data B of the picture B;
subtracting image data b from the image data a to obtain image data c, wherein the image data c only contains the elements to be positioned, and further performing graying processing and filtering processing on the image data c in sequence, positioning effective content in the processed image data c, and marking the outline of the effective content, wherein the effective content is the elements to be positioned;
and determining the outline of the image data of the element to be positioned, so as to determine the central point of the outline, calculating the coordinates of the central point of the outline according to the coordinates of all the outline points in the outline of the effective content, and positioning the element to be positioned according to the coordinate position of the central point.
Preferably, the acquiring the image data a and the image data b of the picture a and the picture a respectively specifically includes: calling an interface immed provided by an OpenCv library of an open source computer vision library through a Python language, and respectively reading the picture A and the picture A to obtain image data a and image data b.
Preferably, after the subtracting the image data b from the image data a to obtain the image data c, the method further includes: performing graying processing on the image data c through an interface CvtColor provided by an OpenCv library; and filtering out interference pixel points in the image data c subjected to graying processing through an interface threshold provided by the OpenCv library.
Preferably, before the acquiring the image data a and the image data b of the picture a and the picture a, respectively, the method further comprises: cutting the initial picture A to obtain the picture A which not only contains elements needing to be positioned, but also contains a background image; and cutting the initial picture B to obtain the picture B which does not contain elements needing positioning and only contains the background image, namely cutting the picture A, reserving a part of the picture from one fifth to four fifths from top to bottom, cutting the picture B, and reserving a part of the picture from one fifth to four fifths from top to bottom.
Reading a picture A only containing elements and background images thereof and a picture B only containing background images by using a Python language, and respectively obtaining image data a and image data B of the picture A only containing the elements and the background images thereof and the picture B only containing the background images; the image data b is subtracted from the image data a to obtain image data c.
Preferably, the determining the contour of the image data of the element to be located by the search specifically includes: and carrying out contour marking on the image data through a findContours interface provided by an OpenCv library so as to determine the contour of the image data, wherein the boundary point of the contour is marked in the form of a green color point.
As shown in fig. 2, which is a schematic diagram of a mobile terminal element positioning system based on picture analysis according to an embodiment of the present invention, the system includes:
the picture acquiring unit 21 is configured to acquire a mobile phone screenshot on a user operation interface to obtain a picture a, where the picture a includes an element to be positioned, acquire a mobile phone screenshot on the user operation interface that has a same background image as the picture a, and obtain a picture B, where the picture B does not include the element to be positioned;
a picture reading unit 22 for respectively acquiring image data a of picture a and image data B of picture B;
the data processing unit 23 is configured to subtract the image data b from the image data a to obtain image data c, where the image data c only includes the element to be positioned;
and the coordinate calculation unit 24 is configured to perform contour determination on the image data c, determine a center point of the contour, and locate the element to be located at the coordinate position of the center point.
Preferably, the picture reading unit 22 specifically includes:
calling an interface immed provided by an OpenCv library of an open source computer vision library through a Python language, and respectively reading the picture A and the picture A to obtain image data a and image data b.
Preferably, the system further comprises an interference rejection unit configured to:
after the image data a is subtracted from the image data b to obtain image data c, performing graying processing on the image data c through an interface CvtColor provided by an OpenCv library;
and filtering out interference pixel points in the image data c subjected to graying processing through an interface threshold provided by the OpenCv library.
Preferably, the system further comprises a clipping unit for:
before the image data a of the picture A and the image data B of the picture B are respectively obtained, cutting the initial picture A to obtain the picture A which not only contains elements needing to be positioned but also contains a background image;
and cutting the initial picture B to obtain the picture B which does not contain the element needing positioning and only contains the background image.
Preferably, the coordinate calculation unit 24 specifically includes:
the image data c is silhouetted by an interface findContours provided by the OpenCv library, thereby determining the silhouette of the image data c.
According to this embodiment, an exemplary embodiment is as follows:
step 1: when the automation program runs to the interface as shown in FIG. 3, whether the button on the picture can be successfully identified or not is automatically judged by the fact that the element attribute name is 'immediately used', the result is that the button cannot be identified, and then the following process of carrying out element positioning through picture analysis is started;
step 2: the automation program intercepts the cell phone screen to get fig. 3.
And step 3: the automation program locates the button at the lower left corner through the element attribute name of 'pack up' and clicks;
and 4, step 4: the automation program intercepts the cell phone screen to get fig. 4.
And 5: the image of fig. 3 is cut out, and a part of the image from top to bottom, one fifth to four fifths, is reserved to obtain fig. 5.
Step 6: the image of fig. 4 is cut out, and a part of the image from top to bottom, one fifth to four fifths, is retained, so as to obtain fig. 6.
And 7: reading the fig. 5 and fig. 6 by using an interface imread provided by the Python language OpenCv library to obtain an OpenCv picture object 5 and a picture object 6, respectively, where the imread is a function in a computer language and is used to read data in a picture file.
And 8: the picture object 6 is subtracted from the picture object 5 to obtain the picture object 7, as shown in fig. 7.
And step 9: the image object 7 is grayed by using an OpenCv library interface CvtColor to obtain an image object 8, as shown in fig. 8, the CvtColor is a color space conversion function in OpenCv, and can realize conversion from RGB colors to color spaces such as HSV and HSI, and also can be converted into a grayscale image.
Step 10: using OpenCv library interface threshold to filter small interfering pixel points in the picture object 8 to obtain the picture object 9, as shown in fig. 9, the threshold method is to binarize the image information by traversing the middle point of the gray-scale image, and the processed picture has only two color values.
Step 11: an OpenCv library interface findContours is used for finding an outline of effective content in the current picture object 9, and an outline boundary point is drawn on the picture object 9 in the form of a green dot to obtain a picture object 10, as shown in fig. 10, a findContours function is a very useful segmentation function in OpenCv, and provides topological information such as shapes and areas for developers.
Step 12: calculating the coordinates of the center point of the contour according to the coordinates of all the contour points obtained in the previous step, wherein the coordinate position is shown as red point in fig. 11:
step 13: the current element can be clicked according to the coordinates of the red point in the figure 11, so that the problem that the WebdriverAgent cannot locate the element on the picture is solved.
It should be understood that the specific order or hierarchy of steps in the processes disclosed is an example of exemplary approaches. Based upon design preferences, it is understood that the specific order or hierarchy of steps in the processes may be rearranged without departing from the scope of the present disclosure. The accompanying method claims present elements of the various steps in a sample order, and are not intended to be limited to the specific order or hierarchy presented.
The above-mentioned embodiments are intended to illustrate the objects, technical solutions and advantages of the present invention in further detail, and it should be understood that the above-mentioned embodiments are merely exemplary embodiments of the present invention, and are not intended to limit the scope of the present invention, and any modifications, equivalent substitutions, improvements and the like made within the spirit and principle of the present invention should be included in the scope of the present invention.
Claims (10)
1. A mobile terminal element positioning method based on picture analysis is characterized by comprising the following steps:
acquiring a mobile phone screenshot on a user operation interface to obtain a picture A, wherein the picture A comprises an element needing to be positioned, and acquiring a mobile phone screenshot on the user operation interface which has the same background image as the picture A to obtain a picture B, wherein the picture B does not comprise the element needing to be positioned;
respectively acquiring image data a of a picture A and image data B of a picture B;
subtracting the image data b from the image data a to obtain image data c, wherein the image data c only contains the elements needing to be positioned;
and determining the contour of the image data c, determining the central point of the contour, and positioning the element to be positioned according to the coordinate position of the central point.
2. The method according to claim 1, wherein the obtaining of the image data a of the picture a and the image data B of the picture B respectively comprises:
calling an interface immed provided by an OpenCv library of an open source computer vision library through a Python language, and respectively reading a picture A and a picture B to obtain image data a and image data B.
3. The picture analysis-based mobile terminal element positioning method according to claim 1, wherein after said subtracting image data b from image data a to obtain image data c, further comprising:
performing graying processing on the image data c through an interface CvtColor provided by an OpenCv library;
and filtering out interference pixel points in the image data c subjected to graying processing through an interface threshold provided by the OpenCv library.
4. The picture analysis-based mobile terminal element positioning method according to claim 1, further comprising, before said separately obtaining image data a of picture a and image data B of picture B:
cutting the initial picture A to obtain the picture A which not only contains elements needing to be positioned, but also contains a background image;
and cutting the initial picture B to obtain the picture B which does not contain the element needing positioning and only contains the background image.
5. The picture analysis-based mobile terminal element positioning method according to claim 1, wherein said determining the contour of said image data c specifically comprises:
the image data c is silhouetted by an interface findContours provided by the OpenCv library, thereby determining the silhouette of the image data c.
6. A mobile end element positioning system based on picture analysis, the system comprising:
the mobile phone positioning device comprises a picture acquisition unit, a picture acquisition unit and a positioning unit, wherein the picture acquisition unit is used for acquiring a mobile phone screenshot on a user operation interface to obtain a picture A, the picture A comprises an element needing to be positioned, the mobile phone screenshot on the user operation interface which has the same background image as the picture A is acquired to obtain a picture B, and the picture B does not comprise the element needing to be positioned;
the picture reading unit is used for respectively acquiring image data a of a picture A and image data B of a picture B;
the data processing unit is used for subtracting the image data b from the image data a to obtain image data c, and the image data c only contains the elements needing to be positioned;
and the coordinate calculation unit is used for determining the contour of the image data c, determining the central point of the contour and positioning the element to be positioned according to the coordinate position of the central point.
7. The system for mobile terminal element location based on picture analysis as claimed in claim 6, wherein said picture reading unit is specifically configured to:
calling an interface immed provided by an OpenCv library of an open source computer vision library through a Python language, and respectively reading a picture A and a picture B to obtain image data a and image data B.
8. The picture analysis-based mobile terminal element location system of claim 6, wherein the system further comprises an interference rejection unit to:
after the image data a is subtracted from the image data b to obtain image data c, performing graying processing on the image data c through an interface CvtColor provided by an OpenCv library;
and filtering out interference pixel points in the image data c subjected to graying processing through an interface threshold provided by the OpenCv library.
9. The picture analysis-based mobile terminal element positioning system of claim 6, wherein the system further comprises a cropping unit to:
before the image data a of the picture A and the image data B of the picture B are respectively obtained, cutting the initial picture A to obtain the picture A which not only contains elements needing to be positioned but also contains a background image;
and cutting the initial picture B to obtain the picture B which does not contain the element needing positioning and only contains the background image.
10. The system of claim 6, wherein the coordinate calculation unit is specifically configured to:
the image data c is silhouetted by an interface findContours provided by the OpenCv library, thereby determining the silhouette of the image data c.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010311162.2A CN111580902B (en) | 2020-04-20 | 2020-04-20 | Mobile terminal element positioning method and system based on picture analysis |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010311162.2A CN111580902B (en) | 2020-04-20 | 2020-04-20 | Mobile terminal element positioning method and system based on picture analysis |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN111580902A true CN111580902A (en) | 2020-08-25 |
| CN111580902B CN111580902B (en) | 2024-01-26 |
Family
ID=72113081
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010311162.2A Active CN111580902B (en) | 2020-04-20 | 2020-04-20 | Mobile terminal element positioning method and system based on picture analysis |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111580902B (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115033822A (en) * | 2022-06-14 | 2022-09-09 | 壹沓科技(上海)有限公司 | Element positioning method, device and equipment and readable storage medium |
Citations (19)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101099161A (en) * | 2005-04-13 | 2008-01-02 | Scanbuy公司 | Visual code system for camera-equipped mobile devices and applications thereof |
| US20110047488A1 (en) * | 2009-08-24 | 2011-02-24 | Emma Butin | Display-independent recognition of graphical user interface control |
| JP2011081778A (en) * | 2009-08-24 | 2011-04-21 | Kryon Systems Ltd | Method and device for display-independent computerized guidance |
| CN102682082A (en) * | 2012-04-07 | 2012-09-19 | 山东师范大学 | Network Flash searching system and network Flash searching method based on content structure characteristics |
| US20160012611A1 (en) * | 2013-01-07 | 2016-01-14 | Wexenergy Innovations Llc | System And Method Of Measuring Distances Related To An Object Utilizing Ancillary Objects |
| CN105302413A (en) * | 2015-10-14 | 2016-02-03 | 惠州Tcl移动通信有限公司 | UI test method and system for control |
| CN105718370A (en) * | 2016-01-21 | 2016-06-29 | 上海斐讯数据通信技术有限公司 | Android equipment test method and test device |
| CN106815848A (en) * | 2017-01-17 | 2017-06-09 | 厦门可睿特信息科技有限公司 | Portrait background separation and contour extraction method based on grubcut and artificial intelligence |
| CN107066914A (en) * | 2017-03-13 | 2017-08-18 | 广州卓德信息科技有限公司 | Two-dimensional barcode image localization method and system under a kind of complex background |
| CN107451999A (en) * | 2017-08-16 | 2017-12-08 | 中惠创智无线供电技术有限公司 | foreign matter detecting method and device based on image recognition |
| CN107463327A (en) * | 2017-07-20 | 2017-12-12 | 福建网龙计算机网络信息技术有限公司 | A kind of method and terminal for obtaining interface control element position information |
| CN107480036A (en) * | 2017-07-07 | 2017-12-15 | 网易（杭州）网络有限公司 | Automatic test approach, device, storage medium and the terminal of iOS Mobile solutions |
| CN108037871A (en) * | 2017-11-07 | 2018-05-15 | 维沃移动通信有限公司 | Screenshotss method and mobile terminal |
| CN109684997A (en) * | 2018-12-20 | 2019-04-26 | 龙口盛福达食品有限公司 | A kind of image recognition and localization method of the round ripening fruits that is blocked |
| CN109783094A (en) * | 2018-12-15 | 2019-05-21 | 深圳壹账通智能科技有限公司 | Front end page generation method, device, computer equipment and storage medium |
| CN109800165A (en) * | 2019-01-14 | 2019-05-24 | 中国银行股份有限公司 | A kind of front end system test method and device |
| WO2019237747A1 (en) * | 2018-06-14 | 2019-12-19 | 北京微播视界科技有限公司 | Image cropping method and apparatus, and electronic device and computer-readable storage medium |
| CN110599544A (en) * | 2019-08-08 | 2019-12-20 | 佛山科学技术学院 | Workpiece positioning method and device based on machine vision |
| CN110706351A (en) * | 2019-09-30 | 2020-01-17 | 恒信东方文化股份有限公司 | Triangular mesh model generation method and device |
-
2020
- 2020-04-20 CN CN202010311162.2A patent/CN111580902B/en active Active
Patent Citations (19)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101099161A (en) * | 2005-04-13 | 2008-01-02 | Scanbuy公司 | Visual code system for camera-equipped mobile devices and applications thereof |
| US20110047488A1 (en) * | 2009-08-24 | 2011-02-24 | Emma Butin | Display-independent recognition of graphical user interface control |
| JP2011081778A (en) * | 2009-08-24 | 2011-04-21 | Kryon Systems Ltd | Method and device for display-independent computerized guidance |
| CN102682082A (en) * | 2012-04-07 | 2012-09-19 | 山东师范大学 | Network Flash searching system and network Flash searching method based on content structure characteristics |
| US20160012611A1 (en) * | 2013-01-07 | 2016-01-14 | Wexenergy Innovations Llc | System And Method Of Measuring Distances Related To An Object Utilizing Ancillary Objects |
| CN105302413A (en) * | 2015-10-14 | 2016-02-03 | 惠州Tcl移动通信有限公司 | UI test method and system for control |
| CN105718370A (en) * | 2016-01-21 | 2016-06-29 | 上海斐讯数据通信技术有限公司 | Android equipment test method and test device |
| CN106815848A (en) * | 2017-01-17 | 2017-06-09 | 厦门可睿特信息科技有限公司 | Portrait background separation and contour extraction method based on grubcut and artificial intelligence |
| CN107066914A (en) * | 2017-03-13 | 2017-08-18 | 广州卓德信息科技有限公司 | Two-dimensional barcode image localization method and system under a kind of complex background |
| CN107480036A (en) * | 2017-07-07 | 2017-12-15 | 网易（杭州）网络有限公司 | Automatic test approach, device, storage medium and the terminal of iOS Mobile solutions |
| CN107463327A (en) * | 2017-07-20 | 2017-12-12 | 福建网龙计算机网络信息技术有限公司 | A kind of method and terminal for obtaining interface control element position information |
| CN107451999A (en) * | 2017-08-16 | 2017-12-08 | 中惠创智无线供电技术有限公司 | foreign matter detecting method and device based on image recognition |
| CN108037871A (en) * | 2017-11-07 | 2018-05-15 | 维沃移动通信有限公司 | Screenshotss method and mobile terminal |
| WO2019237747A1 (en) * | 2018-06-14 | 2019-12-19 | 北京微播视界科技有限公司 | Image cropping method and apparatus, and electronic device and computer-readable storage medium |
| CN109783094A (en) * | 2018-12-15 | 2019-05-21 | 深圳壹账通智能科技有限公司 | Front end page generation method, device, computer equipment and storage medium |
| CN109684997A (en) * | 2018-12-20 | 2019-04-26 | 龙口盛福达食品有限公司 | A kind of image recognition and localization method of the round ripening fruits that is blocked |
| CN109800165A (en) * | 2019-01-14 | 2019-05-24 | 中国银行股份有限公司 | A kind of front end system test method and device |
| CN110599544A (en) * | 2019-08-08 | 2019-12-20 | 佛山科学技术学院 | Workpiece positioning method and device based on machine vision |
| CN110706351A (en) * | 2019-09-30 | 2020-01-17 | 恒信东方文化股份有限公司 | Triangular mesh model generation method and device |
Non-Patent Citations (1)
| Title |
|---|
| M. OLMEDILLA 等: "Identification of the unique attributes and topics within Smart Things Open Innovation Communities", 《TECHNOLOGICAL FORECASTING AND SOCIAL CHANGE》, vol. 146, pages 133 - 147, XP085747717, DOI: 10.1016/j.techfore.2019.05.004 * |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115033822A (en) * | 2022-06-14 | 2022-09-09 | 壹沓科技(上海)有限公司 | Element positioning method, device and equipment and readable storage medium |
Also Published As
| Publication number | Publication date |
|---|---|
| CN111580902B (en) | 2024-01-26 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| KR101719088B1 (en) | Method for partitioning area, and inspection device | |
| CN111124888B (en) | Method and device for generating recording script and electronic device | |
| EP3734496A1 (en) | Image analysis method and apparatus, and electronic device and readable storage medium | |
| CN114549993B (en) | Method, system and device for grading line segment image in experiment and readable storage medium | |
| CN110363190A (en) | A kind of character recognition method, device and equipment | |
| CN111967490A (en) | Model training method for map detection and map detection method | |
| CN111580902B (en) | Mobile terminal element positioning method and system based on picture analysis | |
| CN113610809A (en) | Fracture detection method, fracture detection device, electronic device, and storage medium | |
| CN112084103B (en) | Interface test method, device, equipment and medium | |
| CN112633341A (en) | Interface testing method and device, computer equipment and storage medium | |
| CN110110660B (en) | Method, device and equipment for analyzing hand operation behaviors | |
| CN115546824B (en) | Taboo picture identification method, apparatus and storage medium | |
| CN110909739B (en) | Picture identification and operation method and device, computer equipment and storage medium | |
| CN112348112B (en) | Training method and training device for image recognition model and terminal equipment | |
| CN115019155A (en) | Reagent automatic identification method, device, equipment and readable storage medium | |
| CN114972500A (en) | Checking method, marking method, system, device, terminal, equipment and medium | |
| JP2016045744A (en) | Image processor, image processing method, and program | |
| US20230237232A1 (en) | Work support device, work support system, and analysis program | |
| CN113760686B (en) | User interface testing method, device, terminal and storage medium | |
| CN115934567A (en) | Interface control detection and identification method and device, electronic equipment and storage medium | |
| CN114998906B (en) | Text detection method, training method and device of model, electronic equipment and medium | |
| CN115471842A (en) | Control positioning method and device, storage medium and computer equipment | |
| CN114627295A (en) | Method, device and equipment for acquiring user interface element and readable storage medium | |
| CN114896148A (en) | Terminal application automatic testing device, system, method, equipment and storage medium based on image recognition capability | |
| CN114679558A (en) | Application operation-based fraud prevention processing method and device, electronic equipment and medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |