CN111079730B - Method for determining area of sample graph in interface graph and electronic equipment - Google Patents
Method for determining area of sample graph in interface graph and electronic equipment Download PDFInfo
- Publication number
- CN111079730B CN111079730B CN201911141461.XA CN201911141461A CN111079730B CN 111079730 B CN111079730 B CN 111079730B CN 201911141461 A CN201911141461 A CN 201911141461A CN 111079730 B CN111079730 B CN 111079730B
- Authority
- CN
- China
- Prior art keywords
- sample
- thumbnail
- image
- graph
- target
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 78
- 238000010586 diagram Methods 0.000 claims abstract description 182
- 238000012360 testing method Methods 0.000 claims description 33
- 238000012795 verification Methods 0.000 claims description 30
- 238000004422 calculation algorithm Methods 0.000 claims description 10
- 230000008602 contraction Effects 0.000 claims description 9
- 238000004590 computer program Methods 0.000 claims description 8
- 238000004904 shortening Methods 0.000 claims description 8
- 238000012545 processing Methods 0.000 claims description 7
- 238000012216 screening Methods 0.000 claims description 7
- 238000004364 calculation method Methods 0.000 description 7
- 230000000694 effects Effects 0.000 description 2
- 230000003287 optical effect Effects 0.000 description 2
- 230000012447 hatching Effects 0.000 description 1
- 238000012567 pattern recognition method Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/20—Image preprocessing
- G06V10/22—Image preprocessing by selection of a specific region containing or referencing a pattern; Locating or processing of specific regions to guide the detection or recognition
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/74—Image or video pattern matching; Proximity measures in feature spaces
- G06V10/75—Organisation of the matching processes, e.g. simultaneous or sequential comparisons of image or video features; Coarse-fine approaches, e.g. multi-scale approaches; using context analysis; Selection of dictionaries
- G06V10/751—Comparing pixel values or logical combinations thereof, or feature values having positional relevance, e.g. template matching
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Multimedia (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Health & Medical Sciences (AREA)
- Artificial Intelligence (AREA)
- Computing Systems (AREA)
- Databases & Information Systems (AREA)
- Evolutionary Computation (AREA)
- General Health & Medical Sciences (AREA)
- Medical Informatics (AREA)
- Software Systems (AREA)
- Image Analysis (AREA)
Abstract
The invention discloses a method and electronic equipment for determining an area where a sample image is located in an interface image, which are used for solving the problem that the area where the sample image is located in the interface image is inaccurate. The method and the device for determining the scaled area of the sample graph in the target interface graph are based on the sample graph, the wide-high value of the source graph containing the sample graph and the position of the sample graph in the source graph. The scaling is performed based on the original size of the sample image, so that the scaled sample image can be efficiently matched with the target interface image, and the accuracy of determining the area where the sample image is located is improved. And intercepting the target area diagram from the target interface diagram, and searching the scaled sample diagram from the target area diagram, so that the calculated amount can be reduced, and the efficiency of determining the area where the sample diagram is located can be improved. In addition, the target area map is intercepted by predicting the width and height values of the zoomed sample map and referring to the predicted width and height values of the zoomed sample map and the position of the sample map, so that the zoomed sample map is contained in the target area map, and the situation that the intercepted target area map contains an incomplete zoomed sample map is avoided.
Description
Technical Field
The present invention relates to image recognition, and in particular, to a method and an electronic device for determining an area where a sample image is located in an interface image.
Background
In mobile application automation test, part of the controls are difficult to locate through the control information in the test process due to lack of key control information or different control information on different devices, so that an automation script is difficult to act on the part of the controls.
Because the picture displayed by the control in the interface is often unchanged, the picture displayed by the control in the interface can be used as a sample picture, and the sample picture is identified in the tested interface by an image identification method so as to determine the position of the control in the interface. However, the image recognition method tends to have a large calculation amount and a low recognition efficiency.
How to determine the area of the sample graph in the interface graph is a technical problem to be solved by the method.
Disclosure of Invention
The embodiment of the application aims to provide a method and electronic equipment for determining an area where a sample image is located in an interface image, which are used for solving the problem that the area where the sample image is located in the interface image is inaccurate.
In a first aspect, a method for determining an area where a sample map is located in an interface map is provided, including:
Acquiring a first sample image, a wide-high value of a source image containing the first sample image, a position of the first sample image in the source image and a target interface image containing a second sample image, wherein the second sample image is the scaled first sample image;
determining the estimated width and height values of the second sample according to the width value ratio of the target interface diagram to the source diagram, the height value ratio of the target interface diagram to the source diagram and the width and height values of the first sample;
intercepting a target area diagram in the target interface diagram according to the position of the first sample diagram in the source diagram and the estimated width and height values of the second sample diagram;
and identifying the second sample graph with the estimated width and height values in the target area graph so as to determine the area of the second sample graph in the target interface graph.
In a second aspect, there is provided an electronic device comprising:
the system comprises an acquisition module, a first sample graph, a wide-high value of a source graph containing the first sample graph, a position of the first sample graph in the source graph and a target interface graph containing a second sample graph, wherein the second sample graph is the scaled first sample graph;
the first determining module is used for determining the estimated width and height values of the second sample graph according to the width value ratio of the target interface graph to the source graph, the height value ratio of the target interface graph to the source graph and the width and height values of the first sample graph;
The intercepting module intercepts a target area graph in the target interface graph according to the position of the first sample graph in the source graph and the estimated width and height values of the second sample graph;
and the second determining module is used for identifying the second sample graph with the estimated width and height values in the target area graph so as to determine the area of the second sample graph in the target interface graph.
In a third aspect, there is provided an electronic device comprising a processor, a memory and a computer program stored on the memory and executable on the processor, the computer program implementing the steps of the method as in the first aspect when executed by the processor.
In a fourth aspect, there is provided a computer readable storage medium having stored thereon a computer program which, when executed by a processor, implements the steps of the method as in the first aspect.
In the embodiment of the application, firstly, a width and height value of a source diagram containing a sample diagram, a position of the sample diagram in the source diagram and a target interface diagram containing a scaled sample diagram are obtained, then, an estimated width and height value of the scaled sample diagram is calculated according to a width and height value ratio of the target interface diagram relative to the source diagram, then, a corresponding area is intercepted in the target interface diagram based on the position of the sample diagram in the source diagram, and finally, the scaled sample diagram is identified in the intercepted area so as to determine the area where the sample diagram is located in the target interface diagram. When the width and height values of the target interface diagram and the source diagram are different, the picture displayed in the source diagram is scaled wholly according to the width and height values of the target interface diagram, wherein the sample diagram in the source diagram is scaled correspondingly, so that the size of the scaled sample diagram can be determined according to the width and height values of the source diagram and the target interface diagram, and the identification accuracy is improved. In addition, the method and the device intercept the target area in the target interface diagram according to the position of the sample diagram in the source diagram, search the scaled sample diagram in the target area, improve the recognition accuracy, reduce the calculation amount, and improve the image recognition efficiency without searching the scaled sample diagram in the whole target interface diagram.
Drawings
The accompanying drawings, which are included to provide a further understanding of the invention and are incorporated in and constitute a part of this specification, illustrate embodiments of the invention and together with the description serve to explain the invention and do not constitute a limitation on the invention. In the drawings:
FIG. 1a is a schematic flow chart of a method for determining a region of a sample map in an interface map according to an embodiment of the present disclosure;
FIG. 1b is a schematic diagram of a sample map position of a method for determining a region where the sample map is located in an interface map according to an embodiment of the present disclosure;
FIG. 1c is a schematic diagram of a target area map position of a method for determining an area where a sample map is located in an interface map according to an embodiment of the present disclosure;
FIG. 2a is a second flow chart of a method for determining a region of a sample map in an interface map according to an embodiment of the present disclosure;
FIG. 2b is a schematic diagram of a process for determining the region of a sample map in an interface map according to an embodiment of the present disclosure;
FIG. 3 is a third flow chart of a method for determining a region of a sample map in an interface map according to an embodiment of the present disclosure;
FIG. 4a is a flowchart illustrating a method for determining an area of a sample map in an interface map according to an embodiment of the present disclosure;
FIG. 4b is a schematic flow chart of a method for determining a region where a sample map is located in an interface map according to an embodiment of the present disclosure;
FIG. 5 is a fifth flow chart of a method for determining a region of a sample map in an interface map according to an embodiment of the present disclosure;
FIG. 6a is a flowchart illustrating a method for determining an area of a sample map in an interface map according to an embodiment of the present disclosure;
FIG. 6b is a flowchart of a binarization process of a method for determining a region of a sample map in an interface map according to an embodiment of the present disclosure;
FIG. 6c is a schematic flow chart of a target area diagram after the method for determining the area of the sample diagram in the interface diagram according to the embodiment of the present disclosure is cut out;
FIG. 7 is a flow chart of a method for determining a region of a sample map in an interface map according to an embodiment of the present disclosure;
FIG. 8 is one of the structural schematic diagrams of an electronic device according to the embodiments of the present disclosure;
fig. 9 is a second schematic structural view of an electronic device according to the embodiment of the present disclosure.
Detailed Description
The following description of the embodiments of the present invention will be made clearly and fully with reference to the accompanying drawings, in which it is evident that the embodiments described are some, but not all embodiments of the invention. All other embodiments, which can be made by those skilled in the art based on the embodiments of the invention without making any inventive effort, are intended to be within the scope of the invention. The reference numerals in the present application are only used to distinguish the steps in the scheme, and are not used to limit the execution sequence of the steps, and the specific execution sequence controls the description in the specification.
In mobile application automation test, control information may refer to id, class, text, xpath, etc., and part of the controls to be tested lack critical control information or control information on different devices is inconsistent, so that the controls to be tested cannot be accurately positioned through the control information in the test process.
For the to-be-tested controls lacking key control information, because the pictures displayed by the to-be-tested controls on the display equipment are always the same, the picture displayed by the to-be-tested controls can be identified in the to-be-tested interface by adopting an image identification method, so that the position of the to-be-tested controls is determined according to the identified picture position, and then the test operation can be executed on the position. If the image recognition method is adopted, the images displayed on different devices of the same tested interface have differences due to different sizes and resolutions of the images displayed on different electronic devices. The control to be tested is larger in size displayed on the larger-screen device and smaller in size displayed on the smaller-screen device. For example, when the size of the display of the control to be tested in the tested interface on the electronic device with the screen resolution of 720×1280 is 60×60 pixels, the image of the 60×60 pixels may be a sample image, and when the same tested interface is displayed on the electronic device with the resolution of 1080×1920, the size of the display of the control to be tested may be 90×90 pixels, or may be 85×85 pixels, or other sizes, that is, the sample image is scaled and displayed in the interface.
When the control to be detected is identified in the detected interface, the mode of template matching can be adopted for identification, but the template matching method can only successfully identify the control to be detected under the condition that the width scaling and the height scaling of the sample graph are consistent, and if the width scaling and the height scaling are inconsistent, the mode of template matching is difficult to accurately identify the control to be detected. In addition, the feature point matching method can be used for identifying the features of the sample graph. However, the feature points in the pictures displayed by the simple-style linear control are too few, and accurate identification is difficult to realize by adopting the feature point matching method. In addition, the feature point matching is large in calculation amount, and efficient recognition is difficult to achieve.
In order to solve the problems in the prior art, the present embodiment provides a method for determining an area where a sample map is located in an interface map, as shown in fig. 1a, including the following steps:
s11: acquiring a first sample image, a wide-high value of a source image containing the first sample image, a position of the first sample image in the source image and a target interface image containing a second sample image, wherein the second sample image is the scaled first sample image;
S12: determining the estimated width and height values of the second sample according to the width value ratio of the target interface diagram to the source diagram, the height value ratio of the target interface diagram to the source diagram and the width and height values of the first sample;
s13: intercepting a target area diagram in the target interface diagram according to the position of the first sample diagram in the source diagram and the estimated width and height values of the second sample diagram;
s14: and identifying the second sample graph with the estimated width and height values in the target area graph so as to determine the area of the second sample graph in the target interface graph.
In the automatic test, the tested interface for successfully executing the test script can be recorded to obtain the test video. When performing related tests on different electronic devices, the test may be performed with reference to the recorded test video. In this embodiment, the first sample image may be an icon of a control that successfully executes the test, which is obtained by capturing the first sample image in the test video, or may be an image preset by a tester before executing the test.
In step S11, the acquired first pattern may include information of an image, a size, and the like of the first pattern. The source map containing the first sample map may be a tested interface of successfully executing the test, which is intercepted in the test video. The obtained wide-high value of the source diagram containing the first sample diagram and the position of the first sample diagram in the source diagram can be obtained according to the tested interface, or can be obtained by manual input or other modes. The target interface diagram containing the second sample diagram may be a tested interface to be tested, and the resolution, size and other characteristics of the tested interface may be different from those of the recorded test video.
Since the same tested interface is often displayed with different widths and heights in screens of different resolutions, the second sample is the scaled first sample, and the widths and heights are often different from the first sample. In step S12, the estimated width and height values of the second sample are determined according to the width value ratio of the target interface map to the source map, the height value ratio of the target interface map to the source map, and the width and height values of the first sample. Specifically, the width value ratio and the high value ratio may be determined according to the obtained target interface map and source map, and if the resolution of the source map is 720×1280 and the resolution of the target interface map is 1080×1920, the width value ratio of the target interface map to the source map is 1080/720, i.e. 1.5, and the high value ratio of the target interface map to the source map is 1920/1280, i.e. 1.5. And then, combining the calculated width value ratio, the calculated height value ratio and the calculated width value and height value of the first sample graph to determine the estimated width value and height value of the second sample graph. Assume that the first pattern has a width of 60 pixels and a height of 60 pixels. The width value of the second pattern is calculated to be 60 x 1.5, i.e., 90 pixels, based on the above-mentioned width value ratio, and the height value of the second pattern is calculated to be 60 x 1.5, i.e., 90 pixels, based on the above-mentioned height value ratio.
Subsequently, in step S13, referring to fig. 1b, a coordinate system is first established in the source map, the upper left corner position of the source map is set as the origin O, and the abscissa is established. Subsequently, the position of the first pattern in the source pattern is determined in the established coordinates. Specifically, the position of the first sample graph may refer to the position of any corner point of the first sample graph, and may refer to the position of the center point of the first sample graph. In the present embodiment, the position of the first pattern means that the upper left corner of the first pattern is at A position in the source map, the position coordinates being (X 1 ，Y 1 ). Then, a coordinate system corresponding to the source map is established in the target interface map, and the position (X) in the source map is determined according to the first sample map 1 ，Y 1 ) Determining the position (X) of the upper left corner of the second sample image in the target interface image 2 ，Y 2 ). And intercepting a target area diagram in a target interface based on the estimated width and height values of the second sample diagram and the coordinate positions determined in the step. For example, the second pattern determined in the above step has a size of 90×90 pixels, and the coordinates (X 2 ，Y 2 ) The area of the second sample map in the target interface map may be initially defined, as indicated by the shading in the target interface map in fig. 1 b.
However, in practical applications, other variations in the size of the second pattern in the target interface may occur, for example, the scaling in height is greater than the scaling in width, which may make the size of the second pattern actually displayed different from the size of the second pattern in the ideal case determined in the above step. In order to further determine the area where the second sample image is located in the target interface, accuracy is improved. In this step, the size of the target area map may be determined by expanding a certain area based on the above-described preliminarily determined second pattern. For example, if the size of the first pattern preliminarily determined in the above step is 90×90 pixels, the size may be enlarged as the size of the target area map according to a predetermined ratio in this step, and the size of the truncated target area map may be 100×100 pixels, as shown by the hatching in fig. 1 c. The target area map obtained by interception comprises the second sample map preliminarily determined by the steps, and in practical application, even if the width and the height of the second sample map change to a certain extent, the target area map obtained by interception in the step can also comprise the second sample map so that the complete second sample map can be identified in the target area map later.
Then, in step S14, a second sample image is identified in the intercepted target area image, specifically, an appropriate image identification method may be selected to identify according to the features of the second sample image, and the area of the second sample image in the target area image is determined, so as to determine the area of the second sample image in the target interface image. Since the second sample image is the scaled first sample image, the area determined by the steps in this embodiment is the area where the scaled first sample image is located in the target interface image. In an automated testing process, after determining the region in which the second pattern is located in the target interface pattern, a testing step may be performed on the region to perform a test on the tested interface.
In the embodiment of the application, a source image containing a sample image and a target interface image containing a scaled sample image are firstly obtained, then the width-height value of the scaled sample image is calculated according to the width-height value ratio of the target interface image relative to the source image, then a corresponding area is intercepted in the target interface image based on the position of the sample image in the source image, and finally the scaled sample image is identified in the intercepted area so as to determine the area where the sample image is located in the target interface image. When the width and height values of the target interface diagram and the source diagram are different, the picture displayed in the source diagram is scaled wholly according to the width and height values of the target interface diagram, wherein the sample diagram in the source diagram is scaled correspondingly, so that the size of the scaled sample diagram can be determined according to the width and height values of the source diagram and the target interface diagram, and the identification accuracy is improved. In addition, the method and the device intercept the target area in the target interface diagram according to the position of the sample diagram in the source diagram, search the scaled sample diagram in the target area, improve the recognition accuracy, reduce the calculation amount, and improve the image recognition efficiency without searching the scaled sample diagram in the whole target interface diagram.
The invention discloses a method and electronic equipment for determining an area where a sample image is located in an interface image, which are used for solving the problem that the area where the sample image is located in the interface image is inaccurate. The method and the device for determining the scaled area of the sample graph in the target interface graph are based on the sample graph, the wide-high value of the source graph containing the sample graph and the position of the sample graph in the source graph. The scaling is performed based on the original size of the sample image, so that the scaled sample image can be efficiently matched with the target interface image, and the accuracy of determining the area where the sample image is located is improved. And intercepting the target area diagram from the target interface diagram, and searching the scaled sample diagram from the target area diagram, so that the calculated amount can be reduced, and the efficiency of determining the area where the sample diagram is located can be improved. In addition, the target area map is intercepted by predicting the width and height values of the zoomed sample map and referring to the predicted width and height values of the zoomed sample map and the position of the sample map, so that the zoomed sample map is contained in the target area map, and the situation that the intercepted target area map contains an incomplete zoomed sample map is avoided.
Based on the solution provided in the foregoing embodiment, preferably, as shown in fig. 2a, the step S14 includes:
s21: the target area diagram and the second sample diagram are abbreviated according to a preset abbreviation standard;
S22: identifying the second sample graph after the thumbnail in the target area graph after the thumbnail so as to determine the area of the second sample graph after the thumbnail in the target area graph after the thumbnail;
s23: and determining the region of the second sample image in the target interface image according to a preset thumbnail standard and the region of the second sample image in the target region image after the thumbnail.
In step S21, the preset thumbnail standard may be preset by the tester according to the actual requirement, or may be adjusted according to the actual conditions of the target area map and the second sample map. For example, referring to fig. 2b, the preset thumbnail standard may be 50%, i.e. the width is reduced to 50% and the height is reduced to 50%, and after the thumbnail, the areas of the target area map and the second sample map are 25%. And then, the second thumbnail after the thumbnail is identified in the target area map after the thumbnail can further reduce the calculated amount, shorten the time of the image identification process and improve the overall efficiency of the test.
In addition, since the thumbnail may reduce the accuracy of the identification in the subsequent step, a thumbnail minimum value, that is, a width value and a height value of the target area map after the thumbnail and a width value and a height value of the second sample map are not smaller than the thumbnail minimum value may be set in advance, so as to ensure that the picture contains more features for identification.
For example, the thumbnail minimum is 30 pixels, assuming that the size of the second sample before the thumbnail is 45×45 pixels. Then, if the thumbnail is made according to the 50% preset thumbnail standard, the wide and high values of the second pattern will be less than 30 pixels. In order to ensure that the second thumbnail can contain more features for recognition, and avoid losing the accuracy in the image recognition process, in this embodiment, the size of the second thumbnail may be reduced to 30×30 pixels, so as to obtain the second thumbnail.
Subsequently, in step S22, a second thumbnail image is identified within the target area map after thumbnail. Because the target area diagram and the second sample diagram after the thumbnail are subjected to the thumbnail with a certain proportion, the number of the contained pixels is smaller than that before the thumbnail, and the calculated amount in the image recognition process can be effectively reduced. After the second thumbnail after the thumbnail is identified in the target area map, in step S23, the second thumbnail and the target area map may be enlarged and retracted according to the preset thumbnail standard, so as to determine the area occupied by the second thumbnail in the target area map. Then, the area of the second sample image in the target interface image can be determined based on the position of the target area image in the target interface image.
According to the scheme provided by the embodiment, the second sample image and the target area image are further abbreviated based on the preset thumbnail standard, so that the operation amount in the image recognition process can be further reduced, meanwhile, the better recognition accuracy can be ensured, and the position of the second sample image in the target interface image can be rapidly and efficiently determined.
Based on the solution provided in the foregoing embodiment, preferably, as shown in fig. 3, the step S21 includes:
s31: determining a width value array of the second sample graph after the shortening, wherein an nth item of width value in the width value array meets the following rule:
when n=1, the width W of the second pattern after the shortening 1 For the width value of the second pattern, when n is greater than or equal to 2, the width value W of the second pattern after the shortening n ＝W n-1 +(-1) n X a x (n-1), wherein n is a positive integer, and the preset zooming step a is a positive integer;
s32: determining a high value sequence corresponding to the wide value sequence according to the estimated wide-to-high value ratio of the second sample graph and the wide value sequence;
s33: and the second sample graph is abbreviated according to the wide value sequence and the corresponding high value sequence.
In practical applications, the wide value and the high value of the second pattern may vary according to the actual display requirement, for example, the size of the second pattern is theoretically 90×90 pixels, but in actual display, the high value may vary more than the wide value, and the actual display size may be 90×92 pixels. If the second pattern is recognized in the target area pattern in accordance with the size of 90×90 pixels, it is difficult to accurately recognize the second pattern having the size of 90×92 pixels.
In order to solve the problem that the wide value and the high value change, the embodiment generates a abbreviated wide value sequence of the second sample graph based on the wide value of the second sample graph, and calculates to obtain a corresponding high value sequence. In step S31, the first item of the determined wide value sequence is the wide value of the second sample graph abbreviated according to the preset thumbnail standard. For example, assuming that the width value after the second sample map is abbreviated is 40 pixels, the first term W of the width value series is determined in this step 1 40. Subsequently, based on formula W n ＝W n-1 +(-1) n The other terms of the wide value series are calculated by x a x (n-1). The preset scaling step a is a positive integer, and in this embodiment, the preset scaling step is 2 pixels. The preset scaling step is related to the recognition precision, the larger the scaling step is, the lower the recognition precision is, the smaller the calculated amount is, and the smaller the scaling step is, the higher the precision is, and the calculated amount is. Calculating a wide value sequence according to the above formula, wherein W 2 Is 42, W 3 Is 38, W 4 Is 44, W 5 36 and so on.
In the practical application process, the number of terms of the wide value sequence can be preset to reduce the calculated amount, for example, the wide value sequence is set to be 5 terms, and then W is calculated according to the rule 5 And obtaining a wide value sequence as follows: 40, 42, 38, 44, 36. A threshold value of the wide value sequence may also be set, for example, each item in the wide value sequence is greater than or equal to W 1 -5 and less than or equal to W 1 +5, due to W 6 46, exceeds the threshold of the predetermined sequence of wide values, thus wide valuesThe sequences are five items: 40, 42, 38, 44, 36. In addition, the threshold value of the wide value sequence may be determined according to the wide value of the second thumbnail image or the wide value of the target region image after the thumbnail image, for example, each item in the wide value sequence is set to be smaller than the wide value of the target region image after the thumbnail image, or the like, and specifically may be set according to actual requirements.
After determining the width value sequence of the second sample after the thumbnail, in step S32, a corresponding height value sequence is determined according to the estimated width-to-height value ratio and the width value sequence of the second sample. Specifically, assuming that the size of the second thumbnail is 40×30 pixels, the estimated aspect ratio of the second thumbnail is 4:3. Calculating a corresponding high value for each of the wide value sequences based on the ratio, when the wide value sequences are: 40 At 42, 38, 44, 36, the high value sequence may be: 30 Since the number of pixels is an integer, 42×3/4, 38×3/4, 44×3/4, 36×3/4, and each term in the high-value sequence may be rounded, the high-value sequence may be obtained by: 30, 32, 29, 33, 27.
After obtaining the sequence of wide values and the corresponding sequence of high values, the second pattern may be abbreviated based on the wide values and the corresponding high values in the sequence. When 5 items are included in the high value sequence, 5 kinds of second sample images with different width and height values can be obtained. In a subsequent step, the 5 second sample images with different width and height values can be sequentially identified in the target area images after the thumbnail in sequence until the second sample images after the thumbnail are identified in the target area images after the thumbnail.
By the scheme provided by the embodiment, the width value and the high value of the second sample graph can be further adjusted to obtain the second sample graph after the thumbnail, and the possibility of identifying the second sample graph after the thumbnail in the target area graph after the thumbnail is improved. In the automatic test process, the test success can be further improved.
Based on the solution provided in the foregoing embodiment, preferably, as shown in fig. 4a, the step S22 includes:
s41: template matching is carried out on the target area graph after the thumbnail and the second sample graph after the thumbnail, so that a two-dimensional similarity image is obtained, wherein when any target pixel in the two-dimensional similarity image represents the similarity of the center point of the second sample graph after the thumbnail and the partial area of the target area graph after the thumbnail, which is overlapped, of the second sample graph after the thumbnail is positioned at the target pixel;
S42: and determining the region of the second sample image after the thumbnail in the target region image after the thumbnail according to the pixel point with the highest represented similarity in the similarity two-dimensional image.
In step S41 of this embodiment, any template matching method may be used to identify a second thumbnail image in the target region thumbnail image. Template matching is a pattern recognition method, and can be used for recognizing what area of a target area diagram is located in a thumbnail sample diagram, and further recognizing a control corresponding to a second sample diagram. Specifically, the template matching may include moving the second thumbnail image in the target area image after the thumbnail image is traversed, performing matching recognition on the second thumbnail image and a partial area of the overlapped target area image in the moving process, and determining the matching degree when the second thumbnail image is located in one area of the target area images.
In this embodiment, the number of pixels included in the similarity two-dimensional image obtained by template matching is related to the dimensions of the target region map after the thumbnail and the second sample map after the thumbnail. For example, referring to fig. 4b, assuming that the size of the second thumbnail image is 40×30 pixels and the size of the target area image is 40×40 pixels, in the process of template matching, the second thumbnail image moves in the target area image after the thumbnail image with 1 pixel as a step length, and can cover 10 different positions, and the size of the similarity two-dimensional image obtained after matching is 1×10 pixels. Each pixel may represent a similarity between the second thumbnail and a partial region of the overlapping target region map when the center point of the second thumbnail is located at the pixel, and the similarity may be greater than or equal to 0 and less than or equal to 1. The obtained two-dimensional image with the similarity can also be displayed through a gray scale image, for example, a pixel point with the similarity of 0 is displayed as white, a pixel point with the similarity of 1 is displayed as black, and values between 0 and 1 are displayed as gray with different depths.
After obtaining the two-dimensional image with the similarity, in step S42, a pixel point with the highest similarity in the two-dimensional image with the similarity may be selected first, and then the area of the second sample image after the thumbnail in the target area image after the thumbnail is determined by using the pixel point as the center. When the similarity two-dimensional image is displayed in a gray scale image, the pixel point with the highest similarity can be determined according to the display color of each pixel, and then the area of the second sample image after the thumbnail in the target area image after the thumbnail is determined.
According to the scheme provided by the embodiment, the second thumbnail after the thumbnail is matched in the target thumbnail after the thumbnail is matched in a template matching mode, image recognition is performed based on the similarity between the second thumbnail after the thumbnail and the covered target area map when the second thumbnail after the thumbnail is at different positions, and the area where the second thumbnail after the thumbnail is in the target area map after the thumbnail is determined. The scheme provided by the embodiment has low calculated amount, and the second sample graph in the target area graph can be accurately identified through template matching.
Preferably, a plurality of corresponding second thumbnail images may be obtained based on the wide value sequence and the high value sequence determined in the above embodiment. When the wide value sequence is: 40 The high value sequences are 42, 38, 44, 36: 30 The dimensions of the second pattern after the thumbnail can be obtained at 32, 29, 33, 27: 40×30 pixels, 42×32 pixels, 38×29 pixels, 44×33 pixels, 36×27 pixels. In this embodiment, the template matching may be sequentially performed on the second pattern of different sizes, for example, the template matching may be performed on the second pattern of 40×30 pixels in the target area pattern after the thumbnail, and if the second pattern is not matched in the target area pattern after the thumbnail, the template matching may be performed on the second pattern of 42×32 pixels in the target area pattern after the thumbnail, and so on until the second pattern of the corresponding size is matched in the target area pattern after the thumbnail.
The scheme provided by the embodiment can improve the accuracy of image recognition, and can realize recognition on the second sample graph with different changes of the wide value and the high value in practical application. And the second patterns with different sizes are identified in the identification process, so that the possibility of successful identification can be improved. In the automatic test process, the test success can be improved.
Based on the solution provided in the foregoing embodiment, preferably, as shown in fig. 5, the step S42 includes:
s51: taking the pixel point with highest represented similarity in the similarity two-dimensional image as a center, taking the size of the second sample image after the contraction as a target size, and determining a region image to be verified in the target region image after the contraction;
s52: verifying the region diagram to be verified and the second sample diagram after the thumbnail by a similarity matching algorithm to obtain verification similarity;
s53: when the verification similarity meets a preset similarity standard, determining the region of the region diagram to be verified in the target region diagram as the region of the second sample diagram after the thumbnail in the target region diagram.
In order to further improve the recognition accuracy, in this embodiment, the target area diagram is firstly intercepted based on the pixel point with the highest similarity and the size of the second sample diagram after the thumbnail, the intercepted image is used as the area diagram to be verified, and the similarity verification is performed on the area to be verified through a similarity matching algorithm. In step S51, the pixel point with the highest similarity may represent that when the center of the second thumbnail image is located at the pixel point, the similarity between the second thumbnail image and the overlapping target area image is the highest. And taking the pixel point with the highest represented similarity as a center, and intercepting the target area diagram based on the size of the second sample diagram after the contraction, wherein the obtained area diagram to be verified can be regarded as the area which is the most similar to the second sample diagram after the contraction in the target area diagram after the contraction.
Subsequently, in step S52, the region diagram to be verified is subjected to similarity verification by a similarity matching algorithm, which may include, for example, calculation of similarity using a histogram or calculation using a locally sensitive hash or the like. In practical application, the similarity verification can be performed on the area to be verified through a similarity matching algorithm, so that the verification similarity can be obtained. The region to be verified can be verified for multiple times through multiple different similarity algorithms, and verification similarity can be obtained according to the multiple verification results. Wherein the verification similarity may be revealed by percentage or other forms.
After obtaining the verification similarity, the verification similarity may be compared with a preset similarity criterion. For example, the preset similarity criterion is not less than 90%, and if the verification similarity is less than 90%, such as 87%, it is determined that the verification similarity does not satisfy the preset similarity criterion. If the verification similarity is greater than or equal to 90%, such as 98%, the verification similarity is judged to satisfy the preset similarity standard. When the verification similarity meets a preset similarity standard, determining the region of the region diagram to be verified in the target region diagram as the region of the second sample diagram after the thumbnail in the target region diagram after the thumbnail.
According to the scheme provided by the embodiment, based on the pixel point with the highest similarity and the size of the second thumbnail determined by the steps, the region diagram to be verified is intercepted in the target region diagram, then the similarity verification is carried out on the region diagram to be verified through a similarity matching algorithm, and when the verification similarity meets the preset similarity standard, the region where the second thumbnail is located in the target region diagram after the thumbnail is determined. The scheme of the embodiment can carry out secondary verification on the identified region, ensure the matching degree of the identified region and the second sample graph, and optimize the identification result. In addition, when there are a plurality of pixels with highest similarity in the similarity two-dimensional image, a plurality of areas to be verified can be intercepted based on the pixels with highest similarity and combined with the size of the second sample after the thumbnail. And then calculating the verification similarity of each region to be verified through a similarity matching algorithm, and determining the region of the second sample graph after the thumbnail in the target region graph after the thumbnail based on the region to be verified with the highest verification similarity.
Based on the solution provided in the foregoing embodiment, as shown in fig. 6a, after the step S52, the method further includes:
S61: when the verification similarity does not meet the preset similarity standard, performing binarization processing on the similarity two-dimensional image based on the pixel point with the highest similarity represented in the similarity two-dimensional image and the preset binarization standard to obtain at least one pixel point meeting the preset similarity screening standard;
s62: intercepting the target area graph after the thumbnail containing at least one pixel point meeting the preset similarity screening standard;
s63: and identifying the second thumbnail after the interception in the target area map so as to determine the area of the second thumbnail in the target interface map.
If the verification similarity does not meet the preset similarity standard, the fact that the region which is matched with the second sample image after the thumbnail is not enough in the target region image after the thumbnail is possibly not identified through the scheme is indicated. In this embodiment, binarization processing is performed on the similarity two-dimensional image based on the pixel point with the highest similarity represented in the similarity two-dimensional image. The binarization standard may be a preset binarization value, and pixels larger than or equal to the preset binarization value are processed into a first color during binarization processing, and pixels smaller than the preset binarization value are processed into a second color, so that a binarized image is obtained. The binarization criterion may also be a value related to a preset similarity criterion or a similarity represented by a certain pixel in the two-dimensional image of similarity, and may be specifically set according to practical situations.
For example, as shown in fig. 6b, if the size of the binarized similarity two-dimensional image is 1×10 pixels, the similarity represented by each pixel is shown in the figure, wherein the similarity represented by the pixel with the highest similarity is 0.9, and the preset binarization criterion is-0.03. Then, in the above step S61, a preset binary value may be calculated first, and in this embodiment, the preset binary value is 0.9-0.03, i.e. 0.87. In the binarization processing, the similarity represented by each pixel in the similarity two-dimensional image is compared with 0.87, the pixel with the similarity larger than or equal to 0.87 is processed into a first color (such as white), and the pixel with the similarity smaller than 0.87 is processed into a second color (such as black), so that the binarized similarity two-dimensional image is obtained.
In the binarized similarity two-dimensional image, the pixel point of the first color (white) can represent that when the center point of the second thumbnail is positioned at the pixel, the similarity between the second thumbnail and the covered thumbnail target area image is higher. At least one pixel point of the first color (white) is determined as at least one pixel point meeting a preset similarity screening standard, and then a partial area is cut out from the abbreviated target area diagram based on the pixel point of the first color (white), as shown in fig. 6 c. The truncated region includes the region of the thumbnail target region map covered when the center point of the thumbnail second pixel point is located at these pixels of the first color (white).
And then, continuously identifying the second thumbnail after the shortened target area diagram is intercepted, so that the possibility of identifying the second thumbnail can be improved, the success rate of testing can be improved in the automatic testing process, the calculated amount can be further reduced, and the time length of image identification processing can be shortened. In addition, when the second thumbnail is identified in the intercepted target area diagram after the thumbnail, the size of the identified second thumbnail can be determined according to the wide value sequence and the corresponding high value sequence determined by the scheme, and the image identification flow is optimized.
Based on the solution provided in the foregoing embodiment, preferably, as shown in fig. 7, the step S23 includes:
s71: and restoring the region of the second sample image after the thumbnail in the target region image after the thumbnail according to a preset thumbnail standard to obtain the region of the second sample image in the target interface image.
In this embodiment, the thumbnail standard may be 50%, and the region of the second sample image after thumbnail in the target region image after thumbnail is restored based on the thumbnail standard. For example, the width value of the region determined by the above scheme is doubled, and the high value of the region is doubled, so as to obtain a region with an area four times that of the region where the second sample image after the thumbnail is located in the target region image after the thumbnail. The position of the restored second sample map is defined in the restored target region map based on the position of the second sample map relative to the target region map. The method can be specifically determined according to parameters such as the center point position, the corner point position, the frame line position and the like of the second sample image after the thumbnail relative to the target area image after the thumbnail.
By the scheme provided by the embodiment, the region of the second sample image in the target region image after the thumbnail is restored based on the preset thumbnail standard, so that the region of the second sample image in the target interface image is determined, and the region is the region of the first sample image in the target interface image after scaling.
In order to solve the problems existing in the prior art, the present embodiment provides an electronic device 80, including:
an obtaining module 81, configured to obtain a first sample, a width and height value of a source map including the first sample, a position of the first sample in the source map, and a target interface map including a second sample, where the second sample is the scaled first sample;
a first determining module 82, configured to determine an estimated width-to-height value of the second sample according to a width-to-height ratio of the target interface map to the source map, a height-to-height ratio of the target interface map to the source map, and the width-to-height value of the first sample;
the intercepting module 83 intercepts a target area graph in the target interface graph according to the position of the first sample graph in the source graph and the estimated width and height values of the second sample graph;
the second determining module 84 identifies the second sample map with the estimated width-height value in the target area map, so as to determine the area of the second sample map in the target interface map.
According to the electronic device provided by the embodiment, a source diagram containing a sample diagram and a target interface diagram containing a scaled sample diagram are firstly obtained, then the width-height value of the scaled sample diagram is calculated according to the width-height value ratio of the target interface diagram relative to the source diagram, then a corresponding area is intercepted in the target interface diagram based on the position of the sample diagram in the source diagram, and finally the scaled sample diagram is identified in the intercepted area so as to determine the area where the sample diagram is located in the target interface diagram. When the width and height values of the target interface diagram and the source diagram are different, the picture displayed in the source diagram is scaled wholly according to the width and height values of the target interface diagram, wherein the sample diagram in the source diagram is scaled correspondingly, so that the size of the scaled sample diagram can be determined according to the width and height values of the source diagram and the target interface diagram, and the identification accuracy is improved. In addition, the method and the device intercept the target area in the target interface diagram according to the position of the sample diagram in the source diagram, search the scaled sample diagram in the target area, improve the recognition accuracy, reduce the calculation amount, and improve the image recognition efficiency without searching the scaled sample diagram in the whole target interface diagram.
Based on the electronic device provided in the foregoing embodiment, preferably, the second determining module 84 is configured to:
the target area diagram and the second sample diagram are abbreviated according to a preset abbreviation standard;
identifying the second sample graph after the thumbnail in the target area graph after the thumbnail so as to determine the area of the second sample graph after the thumbnail in the target area graph after the thumbnail;
and determining the region of the second sample image in the target interface image according to a preset thumbnail standard and the region of the second sample image in the target region image after the thumbnail.
Based on the electronic device provided in the foregoing embodiment, preferably, the second determining module 84 is configured to:
determining a width value array of the second sample graph after the shortening, wherein an nth item of width value in the width value array meets the following rule:
when n=1, the width W of the second pattern after the shortening 1 For the width value of the second pattern, when n is greater than or equal to 2, the width value W of the second pattern after the shortening n ＝W n-1 +(-1) n X a x (n-1), wherein n is a positive integer, and the preset zooming step a is a positive integer;
determining a high value sequence corresponding to the wide value sequence according to the estimated wide-to-high value ratio of the second sample graph and the wide value sequence;
And the second sample graph is abbreviated according to the wide value sequence and the corresponding high value sequence.
Based on the electronic device provided in the foregoing embodiment, preferably, the second determining module 84 is configured to:
template matching is carried out on the target area graph after the thumbnail and the second sample graph after the thumbnail, so that a two-dimensional similarity image is obtained, wherein when any target pixel in the two-dimensional similarity image represents the similarity of the center point of the second sample graph after the thumbnail and the partial area of the target area graph after the thumbnail, which is overlapped, of the second sample graph after the thumbnail is positioned at the target pixel;
and determining the region of the second sample image after the thumbnail in the target region image after the thumbnail according to the pixel point with the highest represented similarity in the similarity two-dimensional image.
Based on the electronic device provided in the foregoing embodiment, preferably, the second determining module 84 is configured to:
taking the pixel point with highest represented similarity in the similarity two-dimensional image as a center, taking the size of the second sample image after the contraction as a target size, and determining a region image to be verified in the target region image after the contraction;
verifying the region diagram to be verified and the second sample diagram after the thumbnail by a similarity matching algorithm to obtain verification similarity;
When the verification similarity meets a preset similarity standard, determining the region of the region diagram to be verified in the target region diagram as the region of the second sample diagram after the thumbnail in the target region diagram.
Based on the electronic device provided in the foregoing embodiment, preferably, as shown in fig. 9, the electronic device further includes a verification module 85:
when the verification similarity does not meet the preset similarity standard, performing binarization processing on the similarity two-dimensional image based on the pixel point with the highest similarity represented in the similarity two-dimensional image and the preset binarization standard to obtain at least one pixel point meeting the preset similarity screening standard;
intercepting the target area graph after the thumbnail containing at least one pixel point meeting the preset similarity screening standard;
and identifying the second thumbnail after the interception in the target area map so as to determine the area of the second thumbnail in the target interface map.
Based on the electronic device provided in the foregoing embodiment, preferably, the second determining module 84 is configured to:
and restoring the region of the second sample image after the thumbnail in the target region image after the thumbnail according to a preset thumbnail standard to obtain the region of the second sample image in the target interface image.
Preferably, the embodiment of the present invention further provides a mobile terminal, including a processor, a memory, and a computer program stored in the memory and capable of running on the processor, where the computer program when executed by the processor implements each process of the above embodiment of a method for determining an area where a sample map is located in an interface map, and the same technical effect can be achieved, and for avoiding repetition, a detailed description is omitted herein.
The embodiment of the invention also provides a computer readable storage medium, on which a computer program is stored, which when executed by a processor, implements each process of the above embodiment of the method for determining the area where the sample map is located in the interface map, and can achieve the same technical effect, so that repetition is avoided, and no further description is given here. Wherein the computer readable storage medium is selected from Read-Only Memory (ROM), random access Memory (Random Access Memory, RAM), magnetic disk or optical disk.
It should be noted that, in this document, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one … …" does not exclude the presence of other like elements in a process, method, article, or apparatus that comprises the element.
From the above description of the embodiments, it will be clear to those skilled in the art that the above-described embodiment method may be implemented by means of software plus a necessary general hardware platform, but of course may also be implemented by means of hardware, but in many cases the former is a preferred embodiment. Based on such understanding, the technical solution of the present invention may be embodied essentially or in a part contributing to the prior art in the form of a software product stored in a storage medium (e.g. ROM/RAM, magnetic disk, optical disk) comprising instructions for causing a terminal (which may be a mobile phone, a computer, a server, an air conditioner, or a network device, etc.) to perform the method according to the embodiments of the present invention.
The embodiments of the present invention have been described above with reference to the accompanying drawings, but the present invention is not limited to the above-described embodiments, which are merely illustrative and not restrictive, and many forms may be made by those having ordinary skill in the art without departing from the spirit of the present invention and the scope of the claims, which are to be protected by the present invention.
Claims (8)
1. A method for determining an area of a sample map in an interface map, comprising:
acquiring a first sample image, a wide-high value of a source image containing the first sample image, a position of the first sample image in the source image and a target interface image containing a second sample image, wherein the second sample image is the scaled first sample image;
determining the estimated width and height values of the second sample according to the width value ratio of the target interface diagram to the source diagram, the height value ratio of the target interface diagram to the source diagram and the width and height values of the first sample;
intercepting a target area diagram in the target interface diagram according to the position of the first sample diagram in the source diagram and the estimated width and height values of the second sample diagram; enlarging a certain area according to a preset proportion based on the preliminarily determined second sample image to determine the size of the target area image;
identifying the second sample graph with the estimated width and height values in the target area graph to determine the area of the second sample graph in the target interface graph; the determined area is the area where the zoomed first sample image is located in the target interface image, and in the automatic test process, after the area where the second sample image is located in the target interface image is determined, a test step is executed on the area so as to execute a test on the tested interface;
Identifying the second sample graph with the estimated width and height values in the target area graph to determine the area of the second sample graph in the target interface graph, wherein the method comprises the following steps:
the target area diagram and the second sample diagram are abbreviated according to a preset abbreviation standard; presetting a thumbnail minimum value, namely, the width value and the height value of the target area diagram after the thumbnail and the width value and the height value of the second sample diagram are not smaller than the thumbnail minimum value;
identifying the second sample graph after the thumbnail in the target area graph after the thumbnail so as to determine the area of the second sample graph after the thumbnail in the target area graph after the thumbnail;
determining the region of the second sample image in the target interface image according to a preset thumbnail standard and the region of the second sample image in the target region image after the thumbnail;
the target area diagram and the second sample diagram are abbreviated according to a preset abbreviation standard, and the method comprises the following steps:
determining a width value array of the second sample graph after the shortening, wherein an nth item of width value in the width value array meets the following rule:
when (when)At the time, the second pattern after the thumbnailWide value->For the wide value of the second pattern, when +. >At the time, the width value of the second pattern after the thumbnail +.>Wherein->Is a positive integer, preset a zoom step size +.>Is a positive integer;
determining a high value sequence corresponding to the wide value sequence according to the estimated wide-to-high value ratio of the second sample graph and the wide value sequence;
and the second sample graph is abbreviated according to the wide value sequence and the corresponding high value sequence.
2. The method of claim 1, wherein identifying the second thumbnail image within the target area map after the thumbnail to determine an area of the second thumbnail image within the target area map after the thumbnail comprises:
template matching is carried out on the target area graph after the thumbnail and the second sample graph after the thumbnail, so that a two-dimensional similarity image is obtained, wherein when any target pixel in the two-dimensional similarity image represents the similarity of the center point of the second sample graph after the thumbnail and the partial area of the target area graph after the thumbnail, which is overlapped, of the second sample graph after the thumbnail is positioned at the target pixel;
and determining the region of the second sample image after the thumbnail in the target region image after the thumbnail according to the pixel point with the highest represented similarity in the similarity two-dimensional image.
3. The method of claim 2, wherein determining the region of the second thumbnail image in the target region image after the thumbnail image according to the pixel point with the highest similarity represented in the two-dimensional similarity image comprises:
taking the pixel point with highest represented similarity in the similarity two-dimensional image as a center, taking the size of the second sample image after the contraction as a target size, and determining a region image to be verified in the target region image after the contraction;
verifying the region diagram to be verified and the second sample diagram after the thumbnail by a similarity matching algorithm to obtain verification similarity;
when the verification similarity meets a preset similarity standard, determining the region of the region diagram to be verified in the target region diagram as the region of the second sample diagram after the thumbnail in the target region diagram.
4. The method of claim 3, wherein verifying the region map to be verified and the second thumbnail by a similarity matching algorithm, after obtaining the verified similarity, further comprises:
when the verification similarity does not meet the preset similarity standard, performing binarization processing on the similarity two-dimensional image based on the pixel point with the highest similarity represented in the similarity two-dimensional image and the preset binarization standard to obtain at least one pixel point meeting the preset similarity screening standard;
Intercepting the target area graph after the thumbnail containing at least one pixel point meeting the preset similarity screening standard;
and identifying the second thumbnail after the interception in the target area map so as to determine the area of the second thumbnail in the target interface map.
5. The method according to any one of claims 2 to 4, wherein determining, according to a preset thumbnail criterion and a region of the second thumbnail in the target region map after the thumbnail, a region of the second thumbnail in the target interface map includes:
and restoring the region of the second sample image after the thumbnail in the target region image after the thumbnail according to a preset thumbnail standard to obtain the region of the second sample image in the target interface image.
6. An electronic device, comprising:
the system comprises an acquisition module, a first sample graph, a wide-high value of a source graph containing the first sample graph, a position of the first sample graph in the source graph and a target interface graph containing a second sample graph, wherein the second sample graph is the scaled first sample graph;
the first determining module is used for determining the estimated width and height values of the second sample graph according to the width value ratio of the target interface graph to the source graph, the height value ratio of the target interface graph to the source graph and the width and height values of the first sample graph;
The intercepting module intercepts a target area graph in the target interface graph according to the position of the first sample graph in the source graph and the estimated width and height values of the second sample graph; enlarging a certain area according to a preset proportion based on the preliminarily determined second sample image to determine the size of the target area image;
the second determining module is used for identifying the second sample graph with the estimated width and height values in the target area graph so as to determine the area of the second sample graph in the target interface graph;
wherein the second determining module is configured to:
the target area diagram and the second sample diagram are abbreviated according to a preset abbreviation standard; presetting a thumbnail minimum value, namely, the width value and the height value of the target area diagram after the thumbnail and the width value and the height value of the second sample diagram are not smaller than the thumbnail minimum value;
identifying the second sample graph after the thumbnail in the target area graph after the thumbnail so as to determine the area of the second sample graph after the thumbnail in the target area graph after the thumbnail; the determined area is the area where the zoomed first sample image is located in the target interface image, and in the automatic test process, after the area where the second sample image is located in the target interface image is determined, a test step is executed on the area so as to execute a test on the tested interface;
Determining the region of the second sample image in the target interface image according to a preset thumbnail standard and the region of the second sample image in the target region image after the thumbnail;
the second determining module is further configured to:
determining a width value array of the second sample graph after the shortening, wherein an nth item of width value in the width value array meets the following rule:
when (when)At the time, the width value of the second pattern after the thumbnail +.>For the wide value of the second pattern, when +.>At the time, the width value of the second pattern after the thumbnail +.>Wherein->Is a positive integer, preset a zoom step size +.>Is a positive integer;
determining a high value sequence corresponding to the wide value sequence according to the estimated wide-to-high value ratio of the second sample graph and the wide value sequence;
and the second sample graph is abbreviated according to the wide value sequence and the corresponding high value sequence.
7. An electronic device, comprising: memory, a processor and a computer program stored on the memory and executable on the processor, which when executed by the processor, performs the steps of the method according to any one of claims 1 to 5.
8. A computer-readable storage medium, on which a computer program is stored which, when executed by a processor, implements the steps of the method according to any one of claims 1 to 5.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911141461.XA CN111079730B (en) | 2019-11-20 | 2019-11-20 | Method for determining area of sample graph in interface graph and electronic equipment |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911141461.XA CN111079730B (en) | 2019-11-20 | 2019-11-20 | Method for determining area of sample graph in interface graph and electronic equipment |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN111079730A CN111079730A (en) | 2020-04-28 |
| CN111079730B true CN111079730B (en) | 2023-12-22 |
Family
ID=70311334
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201911141461.XA Active CN111079730B (en) | 2019-11-20 | 2019-11-20 | Method for determining area of sample graph in interface graph and electronic equipment |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111079730B (en) |
Families Citing this family (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111870950B (en) * | 2020-08-10 | 2023-10-31 | 网易（杭州）网络有限公司 | Game control display control method and device and electronic equipment |
| CN112162930B (en) * | 2020-10-21 | 2022-02-08 | 腾讯科技（深圳）有限公司 | Control identification method, related device, equipment and storage medium |
| CN112434641A (en) * | 2020-12-10 | 2021-03-02 | 成都市精卫鸟科技有限责任公司 | Test question image processing method, device, equipment and medium |
| CN112733862A (en) * | 2021-01-05 | 2021-04-30 | 卓望数码技术（深圳）有限公司 | Terminal image automatic matching method, system and storage medium |
Citations (15)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103577809A (en) * | 2013-11-12 | 2014-02-12 | 北京联合大学 | Ground traffic sign real-time detection method based on intelligent driving |
| CN105513038A (en) * | 2014-10-20 | 2016-04-20 | 网易（杭州）网络有限公司 | Image matching method and mobile phone application test platform |
| CN106228194A (en) * | 2016-08-05 | 2016-12-14 | 腾讯科技（深圳）有限公司 | Image lookup method and device |
| CN106920252A (en) * | 2016-06-24 | 2017-07-04 | 阿里巴巴集团控股有限公司 | A kind of image processing method, device and electronic equipment |
| CN107145889A (en) * | 2017-04-14 | 2017-09-08 | 中国人民解放军国防科学技术大学 | Target identification method based on double CNN networks with RoI ponds |
| CN107274442A (en) * | 2017-07-04 | 2017-10-20 | 北京云测信息技术有限公司 | A kind of image-recognizing method and device |
| CN107403443A (en) * | 2017-07-28 | 2017-11-28 | 中南大学 | A kind of more rope multi-lay windings row's rope form state online test method and device based on machine vision |
| CN107562877A (en) * | 2017-09-01 | 2018-01-09 | 北京搜狗科技发展有限公司 | Display methods, device and the device shown for view data of view data |
| CN108182457A (en) * | 2018-01-30 | 2018-06-19 | 百度在线网络技术（北京）有限公司 | For generating the method and apparatus of information |
| CN108985233A (en) * | 2018-07-19 | 2018-12-11 | 常州智行科技有限公司 | One kind being based on the relevant high-precision wireless vehicle tracking of digital picture |
| CN109543701A (en) * | 2018-11-30 | 2019-03-29 | 长沙理工大学 | Vision significance method for detecting area and device |
| CN109684225A (en) * | 2018-12-29 | 2019-04-26 | 广州云测信息技术有限公司 | A kind of method for testing software and device |
| CN109753435A (en) * | 2018-12-29 | 2019-05-14 | 广州云测信息技术有限公司 | A kind of method for testing software and device |
| CN109858504A (en) * | 2017-11-30 | 2019-06-07 | 阿里巴巴集团控股有限公司 | A kind of image-recognizing method, device, system and calculate equipment |
| CN109919164A (en) * | 2019-02-22 | 2019-06-21 | 腾讯科技（深圳）有限公司 | The recognition methods of user interface object and device |
-
2019
- 2019-11-20 CN CN201911141461.XA patent/CN111079730B/en active Active
Patent Citations (15)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103577809A (en) * | 2013-11-12 | 2014-02-12 | 北京联合大学 | Ground traffic sign real-time detection method based on intelligent driving |
| CN105513038A (en) * | 2014-10-20 | 2016-04-20 | 网易（杭州）网络有限公司 | Image matching method and mobile phone application test platform |
| CN106920252A (en) * | 2016-06-24 | 2017-07-04 | 阿里巴巴集团控股有限公司 | A kind of image processing method, device and electronic equipment |
| CN106228194A (en) * | 2016-08-05 | 2016-12-14 | 腾讯科技（深圳）有限公司 | Image lookup method and device |
| CN107145889A (en) * | 2017-04-14 | 2017-09-08 | 中国人民解放军国防科学技术大学 | Target identification method based on double CNN networks with RoI ponds |
| CN107274442A (en) * | 2017-07-04 | 2017-10-20 | 北京云测信息技术有限公司 | A kind of image-recognizing method and device |
| CN107403443A (en) * | 2017-07-28 | 2017-11-28 | 中南大学 | A kind of more rope multi-lay windings row's rope form state online test method and device based on machine vision |
| CN107562877A (en) * | 2017-09-01 | 2018-01-09 | 北京搜狗科技发展有限公司 | Display methods, device and the device shown for view data of view data |
| CN109858504A (en) * | 2017-11-30 | 2019-06-07 | 阿里巴巴集团控股有限公司 | A kind of image-recognizing method, device, system and calculate equipment |
| CN108182457A (en) * | 2018-01-30 | 2018-06-19 | 百度在线网络技术（北京）有限公司 | For generating the method and apparatus of information |
| CN108985233A (en) * | 2018-07-19 | 2018-12-11 | 常州智行科技有限公司 | One kind being based on the relevant high-precision wireless vehicle tracking of digital picture |
| CN109543701A (en) * | 2018-11-30 | 2019-03-29 | 长沙理工大学 | Vision significance method for detecting area and device |
| CN109684225A (en) * | 2018-12-29 | 2019-04-26 | 广州云测信息技术有限公司 | A kind of method for testing software and device |
| CN109753435A (en) * | 2018-12-29 | 2019-05-14 | 广州云测信息技术有限公司 | A kind of method for testing software and device |
| CN109919164A (en) * | 2019-02-22 | 2019-06-21 | 腾讯科技（深圳）有限公司 | The recognition methods of user interface object and device |
Non-Patent Citations (1)
| Title |
|---|
| 刘国华.《HALCON数字图像处理》.西安电子科技大学出版社,2018,第217-223页. * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN111079730A (en) | 2020-04-28 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111079730B (en) | Method for determining area of sample graph in interface graph and electronic equipment | |
| CN111507958B (en) | Target detection method, training method of detection model and electronic equipment | |
| JP3964327B2 (en) | Method and apparatus for determining a region of interest in an image and image transmission method and apparatus | |
| CN111309618B (en) | Page element positioning method, page testing method and related devices | |
| US20170208207A1 (en) | Method and device for correcting document image captured by image pick-up device | |
| US10395393B2 (en) | Method for assessing the quality of an image of a document | |
| CN107480666B (en) | Image capturing device, method and device for extracting scanning target of image capturing device, and storage medium | |
| CN111899270A (en) | Card frame detection method, device and equipment and readable storage medium | |
| CN110414649B (en) | DM code positioning method, device, terminal and storage medium | |
| CN111553302B (en) | Key frame selection method, device, equipment and computer readable storage medium | |
| CN110910360B (en) | Positioning method of power grid image and training method of image positioning model | |
| CN113469921A (en) | Image defect repairing method, system, computer device and storage medium | |
| CN117994250B (en) | Wafer defect detection method, device, equipment and storage medium | |
| JP6106808B2 (en) | Specify barcode placement in document | |
| CN113012128B (en) | Defect characterization method and device | |
| CN117218672A (en) | Deep learning-based medical records text recognition method and system | |
| CN114255493A (en) | Image detection method, face detection device, face detection equipment and storage medium | |
| CN112232390A (en) | Method and system for identifying high-pixel large image | |
| CN111860687A (en) | Image identification method and device, electronic equipment and storage medium | |
| CN115661131B (en) | Image identification method and device, electronic equipment and storage medium | |
| CN116631003A (en) | Equipment identification method and device based on P & ID drawing, storage medium and electronic equipment | |
| CN115115857A (en) | Image matching method and device and computer equipment | |
| CN115249024A (en) | Bar code identification method and device, storage medium and computer equipment | |
| CN114170229A (en) | Method, device and equipment for registering defect images of printed circuit board and storage medium | |
| CN108345893B (en) | Straight line detection method and device, computer storage medium and terminal |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant | ||
| TR01 | Transfer of patent right | ||
| TR01 | Transfer of patent right |
Effective date of registration: 20240417 Address after: 100016 building 4, Dongfang Science Park, 52 Jiuxianqiao Road, Chaoyang District, Beijing Patentee after: BEIJING TESTIN INFORMATION TECHNOLOGY Co.,Ltd. Country or region after: China Address before: 102425 building 31, 69 Yanfu Road, Fangshan District, Beijing Patentee before: Beijing Yunju Intelligent Technology Co.,Ltd. Country or region before: China |