CN109358939A - A kind of mobile APP carousel figure automatically selects click method - Google Patents
A kind of mobile APP carousel figure automatically selects click method Download PDFInfo
- Publication number
- CN109358939A CN109358939A CN201811171873.3A CN201811171873A CN109358939A CN 109358939 A CN109358939 A CN 109358939A CN 201811171873 A CN201811171873 A CN 201811171873A CN 109358939 A CN109358939 A CN 109358939A
- Authority
- CN
- China
- Prior art keywords
- carousel
- bufferimage
- picture
- automatically
- sub
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/44—Arrangements for executing specific programs
- G06F9/451—Execution arrangements for user interfaces
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/048—Interaction techniques based on graphical user interfaces [GUI]
- G06F3/0484—Interaction techniques based on graphical user interfaces [GUI] for the control of specific functions or operations, e.g. selecting or manipulating an object, an image or a displayed text element, setting a parameter value or selecting a range
- G06F3/04842—Selection of displayed objects or displayed text elements
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- General Engineering & Computer Science (AREA)
- Software Systems (AREA)
- Human Computer Interaction (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Television Signal Processing For Recording (AREA)
Abstract
The invention discloses a kind of mobile APP carousel figures to automatically select click method, and main process is the parameter recorded carousel graph region and carry out picture recognition, and test is clicked in the identification for then executing carousel automatically.Automatic test carousel figure may be implemented in whole process in the present invention, help realizes APP automatic test, and same the method is adapted to various carousel figures and clicks, can record on one device, it is tested simultaneously on multiple devices, even the advantages that being tested in other operating systems.
Description
Technical field
The present invention relates to image recognitions and picture pixels correlation technique field, and in particular to a kind of mobile APP carousel figure from
Dynamic selection click method.
Background technique
Currently, can effectively realize the scheme that Automated Black Box Testing carousel figure is clicked without one kind.If simple use
Ground is known using carousel figure control is difficult to realize accurate click otherwise, such as: class, ID of the circulation control on carousel figure,
The Xpath of generation is substantially all unanimously, so if the carousel figure for encountering no text text is effectively to identify and click
's.If, first according to region screenshot, then being adjusted and being identified by similarity, most using the image recognition technology of existing open source
After reuse other modes and clicked (such as: generating Monkey script clicked), this process seems very correct, but
Having ignored most carousel figure is that can roll automatically, and the waiting time of the carousel figure of each APP or webpage is
Different, therefore simple image recognition technology is unable to satisfy the carousel figure click of all types.
Summary of the invention
It in view of the deficiencies of the prior art, can the present invention is intended to provide a kind of mobile APP carousel figure automatically selects click method
To effectively improve the speed and accuracy that mobile device carousel figure automatically selects click, the diversity, more of carousel G- Design is realized
Equipment, cross-platform (Android and iOS).
To achieve the goals above, the present invention adopts the following technical scheme:
A kind of mobile APP carousel figure automatically selects click method, includes the following steps:
S1, recording need carousel graph region to be tested, and parameter needed for picture recognition are arranged；The parameter includes:
Carousel figure number: i.e. in how many picture rolled of carousel graph region；
Whether roll automatically: i.e. setting carousel is automatic rolling or does not roll automatically；
Interval time: when being set as rolling automatically, a picture next picture needs from occurring to being automatically scrolled to
How long；
Similarity: i.e. picture comparison needs similarity to be achieved；
Time-out time: needing to complete identification Target Photo in time-out time and do the test of clicking operation, if super
Even if when the time in without complete test automatic test failure, fulfil automatic test ahead of schedule and just terminate in advance；
Picture region: the Target Photo region BufferImage-A_sub click stored when recording；Such as
Fruit setting carousel figure when whether setting rolls automatically is not rolled automatically it is necessary to the record sliding rail on every picture of carousel figure
Mark；
S2, carousel figure is tested on the mobile apparatus: first using whole screen of screenshot tool interception mobile device
Screenshot, then send the end PC for shot image information BufferImage-B；
The end S3, PC passes through the relative position of the need obtained when recording carousel graph region to be tested, calculates
The contrast district BufferImage-B_sub of BufferImage-B, then by opencv by BufferImage-B_sub with
BufferImage-A_sub carries out picture comparison；If BufferImage-B_sub's and BufferImage-A_sub is similar
Degree is greater than or equal to the similarity being arranged in step S1, just records the attribute information of BufferImage-B_sub at this time, so
Afterwards when it is not roll automatically that carousel figure, which is arranged, go to step S5, and otherwise go to step S4；If similarity is less than step
The similarity being arranged in S1, then go to step S6；
S4, the screenshot that screenshot tool intercepts whole screen of mobile device is reused, then by the shot image information
BufferImage-C is sent to the end PC；The end PC passes through the relative position of the need obtained when recording carousel graph region to be tested, calculates
Then the contrast district BufferImage-C_sub of BufferIamge-C out passes through opencv for BufferImage-B_sub
Pixel comparison is carried out with BufferImage-C_sub, if similarity is greater than or equal to the similarity being arranged in step S1, is just remembered
The attribute information of BufferImage-C_sub under record at this time jumps to S5, otherwise re-execute the steps S4；
S5, picture center is clicked, automatic test is completed；
If S6, carousel figure are set as rolling automatically, step S2 is returned to after waiting time to be set；If
Carousel figure is set as not rolling automatically, then according to return step S2 after the sliding trace of picture execution slide.
Further, in step S1, similarity is set as 95 or more percent.
Further, in step S1, the time-out time is at least set as (the carousel figure number * interval time * 2) second.
Further, in step S1, the sliding trace of every picture is made of the coordinate points of multiple connections.
Further, in step S2, the end PC is sent for shot image information BufferImage-B by socket communication；Step
In rapid S4, the end PC is sent for shot image information BufferImage-C by socket communication.
Further, in step S2, screenshot uses the minicap either included screenshot tool of system.
Further, in step S5, by way of the Monkey script of Android or the XCUTest of iOS clicks picture
Center.
Further, in step S6, slide is by way of the Monkey script of Android or the XCUTest of iOS
It executes.
The beneficial effects of the present invention are:
1, automatic test carousel figure may be implemented in whole process, and help realizes APP automatic test, same the method
It is adapted to various carousel figures to click, can record, be tested simultaneously on multiple devices on one device, even in other behaviour
Make the advantages that testing in system；
2, in the prior art, automatic test personnel can not be accurately positioned carousel figure picture and click test, energy of the present invention
Effectively improve the positioning accuracy of picture and accurately realize and click, so as to effectively promote working efficiency, reduce manpower at
This.
Detailed description of the invention
Fig. 1 is the method implementation process diagram in the embodiment of the present invention；
Fig. 2 is picture recognition parameter setting interface schematic diagram in the embodiment of the present invention.
Specific embodiment
Below with reference to attached drawing, the invention will be further described, it should be noted that following embodiment is with this technology
Premised on scheme, the detailed implementation method and specific operation process are given, but protection scope of the present invention is not limited to this
Embodiment.
As shown in Figure 1, a kind of mobile APP carousel figure automatically selects click method, include the following steps:
S1, as shown in Fig. 2, recording needs carousel graph region to be tested, and parameter needed for picture recognition is set；The parameter
Include:
The number of carousel figure: i.e. in how many picture rolled of carousel graph region；
Whether roll automatically: in order to test various types of carousel figures, it is therefore desirable in view of picture will not be automatic
The carousel figure of rolling；
Interval time: when being set as rolling automatically, a picture next picture needs from occurring to being automatically scrolled to
How long, this can be estimated when recording；
Similarity: i.e. picture comparison needs similarity to be achieved, generally requires and is adjusted to 95 or more percent；
Time-out time: can provide default value, generally be at least (the carousel figure number * interval time * 2) second；It needs super
When the time in complete identification Target Photo and do the test of clicking operation, this time, can be when generation step, automatically
An evaluation time is provided, user can also be from main modulation.Even if if surveyed in time-out time without completing test automation
Examination failure, fulfils automatic test ahead of schedule and just terminates in advance；
Picture region: the Target Photo region BufferImage-A_sub click stored when recording；Such as
Fruit setting carousel figure when whether setting rolls automatically is not rolled automatically it is necessary to the record sliding rail on every picture of carousel figure
Mark；Sliding trace can be made of the coordinate points of multiple connections；
Step S1 is carried out in the end PC；
S2, carousel figure is tested on the mobile apparatus: first using whole screen of screenshot tool interception mobile device
Screenshot (such as using minicap either Android, iOS included screenshot tool), then by socket communication general
Shot image information BufferImage-B is sent to the end PC, this process is most slowly less than 500ms.
The end S3, PC passes through the relative position of the need obtained when recording carousel graph region to be tested, calculates
The contrast district BufferImage-B_sub of BufferImage-B, then by opencv by BufferImage-B_sub with
BufferImage-A_sub carries out picture comparison；If BufferImage-B_sub's and BufferImage-A_sub is similar
Degree is greater than or equal to the similarity being arranged in step S1, just records the attribute information of BufferImage-B_sub at this time, so
Afterwards when it is not roll automatically that carousel figure, which is arranged, go to step S5, and otherwise go to step S4；If similarity is less than step
The similarity being arranged in S1, then go to step S6；
S4, the screenshot that screenshot tool intercepts whole screen of mobile device is reused, it then should by socket communication
Shot image information BufferImage-C is sent to the end PC；The end PC passes through the phase of the need obtained when recording carousel graph region to be tested
To position, the contrast district BufferImage-C_sub of BufferIamge-C is calculated, it then will by opencv
BufferImage-B_sub and BufferImage-C_sub carries out pixel comparison, if similarity is greater than or equal in step S1
The similarity of setting just records the attribute information of BufferImage-C_sub at this time, jumps to S5, otherwise re-executes
Step S4；
S5, by way of the Monkey script of Android or the XCUTest of iOS click picture center；
If S6, carousel figure are set as rolling automatically, the time to be set is being waited (can be interval time-one stream
The time that journey consumes, time, that is, picture recognition of step S2-S3 mono- time of process consumption required time consumed) it is heavy later
New return step S2；If carousel figure is set as not rolling automatically, by way of the Monkey script of Android or iOS
XCUTest execute slide after return step S2.
For those skilled in the art, it can be provided various corresponding according to above technical solution and design
Change and modification, and all these change and modification, should be construed as being included within the scope of protection of the claims of the present invention.
Claims (8)
1. a kind of mobile APP carousel figure automatically selects click method, which comprises the steps of:
S1, recording need carousel graph region to be tested, and parameter needed for picture recognition are arranged；The parameter includes:
Carousel figure number: i.e. in how many picture rolled of carousel graph region；
Whether roll automatically: i.e. setting carousel is automatic rolling or does not roll automatically；
Interval time: when being set as rolling automatically, a picture next picture from occurring to being automatically scrolled to need how long
Time；
Similarity: i.e. picture comparison needs similarity to be achieved；
Time-out time: it needs to complete identification Target Photo in time-out time and does the test of clicking operation, if in time-out
Even if interior without completing test automatic test failure, fulfil automatic test ahead of schedule and just terminate in advance；
Picture region: the Target Photo region BufferImage-A_sub click stored when recording；If
Setting carousel figure when whether rolling automatically is arranged not roll automatically it is necessary to record sliding trace on every picture of carousel figure；
S2, carousel figure is tested on the mobile apparatus: first cutting using whole screen of screenshot tool interception mobile device
Figure, then sends the end PC for shot image information BufferImage-B；
The end S3, PC passes through the relative position of the need obtained when recording carousel graph region to be tested, calculates BufferImage-B's
Then contrast district BufferImage-B_sub passes through opencv for BufferImage-B_sub and BufferImage-A_
Sub carries out picture comparison；If the similarity of BufferImage-B_sub and BufferImage-A_sub is greater than or equal to step
The similarity being arranged in rapid S1, just records the attribute information of BufferImage-B_sub at this time, then when setting carousel figure
When not roll automatically, go to step S5, and otherwise go to step S4；If similarity is similar less than being arranged in step S1
Degree, then go to step S6；
S4, the screenshot that screenshot tool intercepts whole screen of mobile device is reused, then by the shot image information
BufferImage-C is sent to the end PC；The end PC passes through the relative position of the need obtained when recording carousel graph region to be tested, calculates
Then the contrast district BufferImage-C_sub of BufferIamge-C out passes through opencv for BufferImage-B_sub
Pixel comparison is carried out with BufferImage-C_sub, if similarity is greater than or equal to the similarity being arranged in step S1, is just remembered
The attribute information of BufferImage-C_sub under record at this time jumps to S5, otherwise re-execute the steps S4；
S5, picture center is clicked, automatic test is completed；
If S6, carousel figure are set as rolling automatically, step S2 is returned to after waiting time to be set；If carousel
Figure is set as not rolling automatically, then according to return step S2 after the sliding trace of picture execution slide.
2. mobile APP carousel figure according to claim 1 automatically selects click method, which is characterized in that in step S1, phase
95 or more percent is set as like degree.
3. mobile APP carousel figure according to claim 1 automatically selects click method, which is characterized in that in step S1, institute
It states time-out time and is at least set as (the carousel figure number * interval time * 2) second.
4. mobile APP carousel figure according to claim 1 automatically selects click method, which is characterized in that in step S1, often
The sliding trace of picture is made of the coordinate points of multiple connections.
5. mobile APP carousel figure according to claim 1 automatically selects click method, which is characterized in that in step S2, lead to
It crosses socket communication and sends the end PC for shot image information BufferImage-B；In step S4, by socket communication by the screenshot
Information BufferImage-C is sent to the end PC.
6. mobile APP carousel figure according to claim 1 automatically selects click method, which is characterized in that in step S2, cut
Figure uses the minicap either included screenshot tool of system.
7. mobile APP carousel figure according to claim 1 automatically selects click method, which is characterized in that in step S5, lead to
The XCUTest of the mode or iOS of crossing the Monkey script of Android clicks picture center.
8. mobile APP carousel figure according to claim 1 automatically selects click method, which is characterized in that sliding in step S6
Dynamic operation is by way of the Monkey script of Android or the XCUTest of iOS is executed.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811171873.3A CN109358939B (en) | 2018-10-08 | 2018-10-08 | Automatic selection and click method for mobile APP carousel map |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811171873.3A CN109358939B (en) | 2018-10-08 | 2018-10-08 | Automatic selection and click method for mobile APP carousel map |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN109358939A true CN109358939A (en) | 2019-02-19 |
| CN109358939B CN109358939B (en) | 2021-06-25 |
Family
ID=65348648
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201811171873.3A Active CN109358939B (en) | 2018-10-08 | 2018-10-08 | Automatic selection and click method for mobile APP carousel map |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN109358939B (en) |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20150086115A1 (en) * | 2013-09-20 | 2015-03-26 | Oracle International Corporation | Screenshot database for application verification |
| CN105227949A (en) * | 2015-09-16 | 2016-01-06 | 成都三零凯天通信实业有限公司 | A kind of Android Set Top Box automated testing method |
| CN107066381A (en) * | 2017-02-27 | 2017-08-18 | 网易（杭州）网络有限公司 | Generate method, device and the equipment of the test analysis report based on image recognition |
| CN108205674A (en) * | 2017-12-22 | 2018-06-26 | 广州爱美互动网络科技有限公司 | Content identification method, electronic equipment, storage medium and the system of social APP |
-
2018
- 2018-10-08 CN CN201811171873.3A patent/CN109358939B/en active Active
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20150086115A1 (en) * | 2013-09-20 | 2015-03-26 | Oracle International Corporation | Screenshot database for application verification |
| CN105227949A (en) * | 2015-09-16 | 2016-01-06 | 成都三零凯天通信实业有限公司 | A kind of Android Set Top Box automated testing method |
| CN107066381A (en) * | 2017-02-27 | 2017-08-18 | 网易（杭州）网络有限公司 | Generate method, device and the equipment of the test analysis report based on image recognition |
| CN108205674A (en) * | 2017-12-22 | 2018-06-26 | 广州爱美互动网络科技有限公司 | Content identification method, electronic equipment, storage medium and the system of social APP |
Also Published As
| Publication number | Publication date |
|---|---|
| CN109358939B (en) | 2021-06-25 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN105988924B (en) | A kind of non-intrusion type embedded software graphic user interface automated testing method | |
| CN104461894B (en) | The operation record device and method of Android software | |
| CN104461855B (en) | A kind of Web automated testing method, system and device | |
| CN105468523B (en) | A kind of automated testing method and device of mobile terminal response time | |
| CN109408384B (en) | Software application testing method and device, processor and electronic device | |
| US9165100B2 (en) | Methods and apparatus to map schematic elements into a database | |
| US9135714B1 (en) | Method and system for integrating a graphical user interface capture for automated test and retest procedures | |
| CN109359043B (en) | Mobile game automatic testing method based on machine learning | |
| CN107665168A (en) | A kind of embedded software GUI automated test devices | |
| CN103744784A (en) | Method and system for automatically testing mobile phone applications | |
| CN104820634A (en) | Method and device for detecting page test track | |
| CN104375937A (en) | Continuous integration method and system of automated testing results | |
| CN106021101A (en) | Method and device for testing mobile terminal | |
| CN104536863A (en) | Application program test method and device | |
| CN103268289A (en) | Method and system for transcribing and applying test script on mobile terminal | |
| CN104268083A (en) | Software automatic testing method and device | |
| CN108446230B (en) | Webdriver-oriented Java test code quality evaluation method | |
| CN104142959A (en) | Automated test device and method based on expert knowledge database | |
| CN105653441A (en) | Method and system for UI traversal testing | |
| CN109358939A (en) | A kind of mobile APP carousel figure automatically selects click method | |
| CN105373446B (en) | It is a kind of based on the system self-repairing method drilled automatically and device | |
| CN112383734A (en) | Video processing method, video processing device, computer equipment and storage medium | |
| CN107729242A (en) | A kind of automated testing method, system, device and computer-readable storage medium | |
| CN105975390A (en) | Test data recovery method based on complex test scene | |
| CN112804040B (en) | Method, module, storage medium, device and system for positioning data position |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |