CN109445893A - A kind of method and device of the unique identification of determining operation interface frame - Google Patents
A kind of method and device of the unique identification of determining operation interface frame Download PDFInfo
- Publication number
- CN109445893A CN109445893A CN201811246513.5A CN201811246513A CN109445893A CN 109445893 A CN109445893 A CN 109445893A CN 201811246513 A CN201811246513 A CN 201811246513A CN 109445893 A CN109445893 A CN 109445893A
- Authority
- CN
- China
- Prior art keywords
- piece
- region
- operation interface
- interface frame
- muti
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/44—Arrangements for executing specific programs
- G06F9/451—Execution arrangements for user interfaces
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/70—Determining position or orientation of objects or cameras
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Software Systems (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Human Computer Interaction (AREA)
- General Engineering & Computer Science (AREA)
- User Interface Of Digital Computer (AREA)
Abstract
The application discloses the method and device of determining operation interface frame unique identification a kind of, firstly, obtain operation interface frame to be processed and with the associated human-computer interaction event of operation interface frame to be processed operation interface frame to be processed interaction locations information；Then, one piece is selected in operation interface frame to be processed according to interaction locations information or by the associated muti-piece region of location layout；Finally, by the method for pixel ratio pair, determine relative to operation interface frame it is with uniqueness one piece or by the corresponding image in the associated muti-piece region of location layout be the operation interface frame unique identification.By the unique identification for determining operation interface frame, when so that subsequent execution including the script of the man-machine interactive operation of the unique identification, it can use the relative positional relationship of unique identification and interaction locations pre-recorded in script, the target interaction locations executed on interface are determined, so that analog subscriber operation executes corresponding interactive action automatically at the target interaction locations.
Description
Technical field
This application involves the method for data processing field more particularly to a kind of unique identification of determining operation interface frame and
Device.
Background technique
In the practical application of Internet technology, frequently refers to the data obtained in a system and be entered into another
In one system.For example, enterprise or dealer open up shop on electric business platform, and registration user is logical in e-commerce field
Electric business platform purchase commodity are crossed to generate order data in electric business platform.At this point, just needing to obtain ordering for electric business platform generation
Forms data is simultaneously input to enterprise or ERP (Enterprise Resource Planning, Enterprise Resources Plan) system of dealer
In system or JXC System, so that enterprise or dealer are managed the data such as its inventory, product and finance.
Data transfer between above-mentioned ERP system or JXC System and electric business platform can theoretically pass through
EDI (Electronic Data Interchange, electronic data interchange) integrated technology is realized, i.e., between system and system
The transmission and exchange of structural data are carried out using computer network using standardized format by specific data-interface.
However, need the Development characteristics according to both sides' system due to during development interface, formulate through consultation docking code or
Person's business rule, therefore, the exploitation of specific data interface take a long time with corresponding technical support, development efficiency is low.
Especially when the system being related to has variety classes or multiple versions, needs to develop one for each system and specifically connect
Mouthful, this just more requires a great deal of time, thus more reduces development efficiency.For example, current variety classes or version
ERP system quantity be up to 1500-2000, interface needed for each ERP system and the docking of electric business platform all may be different.
In addition, the developer of ERP or JXC System that many small enterprises use have not existed already or system version is old,
Data-interface cannot be supported at all, thus it is even more extremely difficult for the interface exploitation of this kind of system.
It can be seen that relying on EDI integrated technology solves the problems, such as that the data docking between any not homologous ray is unpractical.It is real
On border, the data docking between current many systems is completed still through the mode of manual entry.Manually will for example, dealer utilizes
The order data that electric business platform generates is entered into the ERP system of oneself.
Based on this, applicant considers to obtain man-machine interactive operation letter based on operation interface frame image when action event occurs
Breath, the man-machine interactive operation information include operation interface frame, the corresponding action message of every width operation interface frame and its execute sequence,
By executing the man-machine interactive operation information automatically, the next data read in a system automatically are simultaneously automatically another by data inputting
In a system.However, how target interaction locations are precisely determined when executing the man-machine operating interactive information in operation interface,
It is current insoluble technical problem.
Summary of the invention
This application provides the method and devices for the mark for determining operation interface frame, to position mesh using the unique identification
Mark interaction locations.
In a first aspect, this application provides the methods of the unique identification of determining operation interface frame a kind of, this method comprises:
Obtain operation interface frame to be processed and with the associated human-computer interaction event of the operation interface frame in operation circle
The interaction locations information of face frame；
One piece is selected in the operation interface frame according to interaction locations information or by the associated multinugget region of location layout
Domain；
Pixel is carried out by described one piece or by the corresponding image in the associated muti-piece region of location layout and operation interface frame
It compares；
According to comparison result, described one piece or by location layout pass with uniqueness is determined relative to operation interface frame
The corresponding image in muti-piece region of connection is the unique identification of operation interface frame.
With reference to first aspect, in a first possible implementation of that first aspect, described to be existed according to interaction locations information
Before selecting one piece in the operation interface frame or passing through the associated muti-piece region of location layout, the method also includes:
Binarization threshold is determined according to the gray value of whole pixels of the operation interface frame；
Binaryzation conversion is carried out to operation interface frame according to the binarization threshold.
With reference to first aspect, in a second possible implementation of that first aspect, described to be existed according to interaction locations information
One piece is selected in the operation interface frame or passes through the associated muti-piece region of location layout, comprising:
Interaction locations are determined according to the interaction locations information；
One piece of region is selected in the predeterminable area of the interaction locations or passes through the associated muti-piece region of location layout；
When the comparison result show selected one piece of region or by the associated muti-piece region of location layout it is corresponding
When image does not have uniqueness relative to operation interface frame, another piece of region is selected in the predeterminable area；Described another piece
Region with selected one piece of region or muti-piece region there is location layout to be associated with.
With reference to first aspect, described one piece is determined in the third possible implementation as steps described below in first aspect
Or whether there is uniqueness relative to the operation interface frame by the corresponding image in the associated muti-piece region of location layout:
If there is image pixel phase corresponding with one piece of selected region in remaining region of the operation interface frame
Same region passes through the associated muti-piece of location layout with selected alternatively, existing in remaining region of the operation interface frame
The identical region of the corresponding image pixel in region, and the position of the identical interregional location layout and the muti-piece region
It sets that layout is identical, determines selected one piece or by the corresponding image in the associated muti-piece region of location layout relative to the behaviour
Make interface frame with uniqueness.
With reference to first aspect or the possible implementation of first aspect first three, in the 4th kind of possible realization of first aspect
In mode, the shape in the selected region is rectangle.
With reference to first aspect, in the 5th kind of possible implementation of first aspect, the determination is relative to operation interface
Frame it is with uniqueness described one piece or by the corresponding image in the associated muti-piece region of location layout be operation interface frame only
After one property mark, the method also includes:
It is the unique identification include one piece or by the associated muti-piece of location layout on the operation interface frame
The corresponding image addition mark in region.
Second aspect, the embodiment of the present application provide the device of the unique identification of determining operation interface frame a kind of, the device
Include:
Acquiring unit, for operation interface frame to be processed and with the associated human-computer interaction event of the operation interface frame in institute
State the interaction locations information of operation interface frame；
Selecting unit, for selecting one piece in the operation interface frame according to interaction locations information or passing through location layout
Associated muti-piece region；
Comparing unit, for by described one piece or by the corresponding image in the associated muti-piece region of location layout and operating boundary
Face frame carries out pixel ratio pair；
Determination unit, for according to comparison result, determine relative to operation interface frame it is with uniqueness described one piece or
It is the unique identification of operation interface frame by the corresponding image in the associated muti-piece region of location layout.
In conjunction with second aspect, in second aspect in the first possible implementation, described device further include:
Image processing unit, the gray value for whole pixels according to the operation interface frame determine binaryzation threshold
Value；
Binaryzation conversion is carried out to operation interface frame according to the binarization threshold.
In conjunction with second aspect, in second of second aspect possible implementation, the selecting unit is specifically used for:
Interaction locations are determined according to the interaction locations information；
One piece of region is selected in the predeterminable area of the interaction locations or passes through the associated muti-piece region of location layout；
When the comparison result show selected one piece of region or by the associated muti-piece region of location layout it is corresponding
When image does not have uniqueness relative to operation interface frame, another piece of region is selected in the predeterminable area；Described another piece
Region with selected one piece of region or muti-piece region there is location layout to be associated with.
In conjunction with second aspect, in second aspect in the third possible implementation, the determination unit is specifically used for:
If there is image pixel phase corresponding with one piece of selected region in remaining region of the operation interface frame
Same region passes through the associated muti-piece of location layout with selected alternatively, existing in remaining region of the operation interface frame
The identical region of the corresponding image pixel in region, and the position of the identical interregional location layout and the muti-piece region
It sets that layout is identical, determines selected one piece or by the corresponding image in the associated muti-piece region of location layout relative to the behaviour
Make interface frame with uniqueness.
From the above technical scheme, the method and dress of the unique identification of determining operation interface frame provided by the present application
It sets, firstly, obtaining operation interface frame to be processed and with the associated human-computer interaction event of operation interface frame to be processed described
The interaction locations information of operation interface frame；Then, selected in operation interface frame to be processed according to interaction locations information one piece or
Pass through the associated muti-piece region of location layout；Finally, determining has only relative to operation interface frame by the method for pixel ratio pair
One piece of one property or by the corresponding image in the associated muti-piece region of location layout be the operation interface frame unique identification.It is logical
The unique identification that the application method and device determines operation interface frame is crossed, so that subsequent execution includes the people of the unique identification
When the script of machine interactive operation, the relative position that can use unique identification and interaction locations pre-recorded in script is closed
System determines the target interaction locations executed on interface, so that analog subscriber operation is automatic at the target interaction locations executes phase
The interactive action answered.Furthermore it is also possible to replace corresponding operation interface frame by storage unique identification, reduce data storage
Amount and data processing amount.
Detailed description of the invention
In order to illustrate more clearly of the technical solution of the application, letter will be made to attached drawing needed in the embodiment below
Singly introduce, it should be apparent that, for those of ordinary skills, without any creative labor,
It is also possible to obtain other drawings based on these drawings.
Fig. 1 is a kind of the application unique identification of determining operation interface frame image shown according to an exemplary embodiment
Method flow diagram；
Fig. 2 is the application scenarios schematic diagram of the application one embodiment；
Fig. 3 (a) is the application scenarios schematic diagram of the application one embodiment；
Fig. 3 (b) is the application scenarios schematic diagram of the application one embodiment；
Fig. 4 is the refinement step flow chart of step S120 shown in the application Fig. 1；
Fig. 5 (a) is the application scenarios schematic diagram of the application one embodiment；
Fig. 5 (b) is the application scenarios schematic diagram of the application one embodiment；
Fig. 6 is the application scenarios schematic diagram of the application one embodiment；
Fig. 7 is the application another uniqueness mark for determining operation interface frame image shown according to an exemplary embodiment
The method flow diagram of knowledge；
Fig. 8 is the uniqueness mark of the application another determination operation interface frame image shown according to an exemplary embodiment
The method flow diagram of knowledge；
Fig. 9 is a kind of the application unique identification of determining operation interface frame image shown according to an exemplary embodiment
Device block diagram；
Figure 10 is the application another mark for determining operation interface frame image shown according to an exemplary embodiment
Device block diagram.
Specific embodiment
Technical solution in order to enable those skilled in the art to better understand the present invention, below in conjunction with of the invention real
The attached drawing in example is applied, technical scheme in the embodiment of the invention is clearly and completely described, it is clear that described implementation
Example is only a part of the embodiment of the present invention, instead of all the embodiments.Based on the embodiments of the present invention, this field is common
Technical staff's every other embodiment obtained without making creative work, all should belong to protection of the present invention
Range.
The method and device of the unique identification of determining operation interface frame provided by the present application, includes but is not limited to apply
During obtaining man-machine interactive operation information.The man-machine interactive operation information is understood that simulation man-machine interactive operation behavior
Script.The process for obtaining man-machine interactive operation information, operation interface frame when referring to based on generation human-computer interaction event
Image, the process of the script of Lai Shengcheng man-machine interactive operation behavior.Operation circle when alternative events occur is stored in the script
Face frame and its corresponding interactive action information of the mark, operation interface frame (position of such as type of action, mark in operation interface frame
Relative positional relationship between interaction locations) and each operation interface frame between execute sequence.By executing the script,
Make computer can be with operation behavior of the analog subscriber on computer operation interface, to replace user's operation frequently repeatedly.
The unique identification of operation interface frame image can be determined by the application method and device, and then is held in computer
When the above-mentioned script of row, according to the mark recorded in script in the position of operation interface frame and the relativeness of interaction locations, determine
The target interaction locations of the operation interface of script execution, to execute corresponding interactive action at the target interaction locations.Separately
Outside, when storing and calling perform script, it can store mark to replace operation interface frame, reduce at data storage capacity and data
Reason amount.
Embodiment one
Refering to fig. 1, the present embodiment provides the methods of the unique identification of determining operation interface frame a kind of, this method comprises:
Step S110 obtains operation interface frame to be processed and with the associated human-computer interaction event of the operation interface frame in institute
State the interaction locations information of operation interface frame；
Human-computer interaction, which refers to, realizes that the information between user and computer is handed over by computer input device and output equipment
Mutually.User according to the output of machine or display equipment provide it is a large amount of for information about and prompt is asked for instructions, by corresponding on machine
Interactive action (response action) input information.In the application, alternative events can be understood as user on machine by corresponding
Interactive action input information event.According to the difference of input mode, the type of the interactive action of alternative events has been triggered
Difference, for example, mouse clicks type (mouse input), keyboard input type etc..
In the present embodiment technology scene, when there is the generation of man-machine alternative events, current operation interface frame, the operation are captured
Interface frame is the operation interface frame to be processed.In general, after full frameization that the operation interface frame is man-machine action pane is shown
Picture, Auto-CAD action pane as shown in Figure 2.The application passes through the human-computer interaction thing that operation interface frame is corresponding
Part is bound, and keeps man-machine alternative events associated with specific operation interface frame.
In the present embodiment, the user response that interaction locations can be understood as triggering alternative events is acted in operation interface frame
Position or region, such as mouse click location, key entry area domain etc. occurs.It is understood that " interaction locations information " is
For characterizing the quantitative information of interaction locations or region, should include " position coordinates ".That is, if " position coordinates " are
If one index, then, interaction locations information is the set of an index.For example, mouse click location is in operation interface frame
In position coordinates or key entry area domain boundary in operation interface frame point coordinate set etc..
Step S120 selects one piece in the operation interface frame according to interaction locations information or is associated with by location layout
Muti-piece region；
In the present embodiment, unique identification is a part of image of operation interface frame, rather than additional addition is on operation circle
Any label or mark on the frame of face, the beneficial effect is that:
It is known in the present embodiment technology scene, be stored in the script of man-machine interactive operation behavior operation interface frame and its
(such as type of action identifies the position in operation interface frame and interacts position for mark, the corresponding interactive action information of operation interface frame
Relative positional relationship between setting) and each operation interface frame between execute sequence.When executing the script on computers,
Firstly the need of the corresponding operation interface frame stored in script according to pending step, is transferred, if the operation interface in script
The frame operation interface current with computer matches, then according to the mark of operation interface frame in script, determines that computer is current
The image-region that operation interface and the mark match, and then according to the relative position in the position and script of the image-region
Relationship positions target interaction locations.
Therefore, in the present embodiment, from a part of image of extraction operation interface frame as the unique of the operation interface frame
Property mark, in order to be matched with computer operation interface when perform script.
In more specifically implementation, unique identification may include the corresponding figure in one piece of region of operation interface frame
Picture, such as B corresponding image in region shown in Fig. 3 (a)；It may also include the corresponding image in muti-piece region of operation interface frame,
At this point, unique identification can be understood as the set of several pieces of small image-regions, such as region B1, B2 and B3 couple shown in Fig. 3 (b)
The image answered.Also, in the present embodiment, shape and size to selected region are without limitation.
It should be noted that the mark is not only several pieces when unique identification includes the corresponding image in muti-piece region
Area image simple combination, the location layout between several pieces of area images are also determining.For example, if having shown in Fig. 3 (b)
Determine that the corresponding image of location layout associated region B1, B2 and B3 is the unique identification of its affiliated operation interface frame, once
Relative positional relationship between region B1, B2 and B3 changes, and will be unable to recognize the mark.
It should be noted that since to be that user asks the response made for instructions according to the prompt that computer exports dynamic for interactive action
Make, so in most of situation, patterned prompt or ask information for instructions and appear near interaction locations, it can be seen that, this
Application embodiment selects one piece in operation interface frame according to interaction locations information or passes through the associated muti-piece region of location layout,
Selectable range can be reduced, data processing amount is not only reduced, data-handling efficiency is improved, has the image of selection more
Representative, image more corresponding than region as selected may be precisely above-mentioned patterned prompt information.
In the concrete realization, according to interaction locations information, the corresponding image of interaction locations near zone is scanned, is obtained
The pixel distribution data for obtaining predeterminable area select feature relatively more by being analyzed and processed to the pixel distribution data
One or more rectangular area.In technical scheme, the region that can preferentially select feature most is more when needing to select
When block region, it can successively be selected according to the sequence that feature distribution amount is successively decreased.It optionally, can also be in technical scheme
A certain piece is randomly choosed in the finite region near interaction locations or muti-piece region, the application do not limit this.
It is worth noting that, when having selected muti-piece region, the location layout in these block regions relative position in other words
Relationship is deterministic, that is to say, that muti-piece region is mutually related by deterministic location layout.When wherein any one
When the position in block region determines, the position of remaining rectangular area determines therewith.
In addition, the application to the size of above-mentioned zone without limitation, in the specific implementation, can with pre-set dimension data,
Dynamic adjustment can also be carried out to it according to pixel distribution data.
Refering to Fig. 4, in some preferred embodiments of the application, the step of step S120 is further refined, includes:
Step S121 determines interaction locations according to the interaction locations information；
Step S122 selects one piece of region or associated more by location layout in the predeterminable area of the interaction locations
Block region；
Step S123, when the comparison result shows one piece of selected region or by the associated multinugget region of location layout
When the corresponding image in domain does not have uniqueness relative to operation interface frame, another piece of region is selected in the predeterminable area；Institute
Stating another piece of region with selected one piece of region or muti-piece region there is location layout to be associated with.
It should be noted that each interaction locations are corresponding with a predeterminable area, which can be understood as interacting
Region near position.The size that the predeterminable area can be coordinate and/or the predeterminable area based on interaction locations is (wide
Degree, length, area or radius etc.) dynamic delimitation.For example, in operation interface frame shown in Fig. 5 (a), the coordinate of interaction locations
For L1(l11, l12), L1Corresponding predeterminable area is A1；For another example in the operation interface frame of Fig. 5 (b), the coordinate of interaction locations is L2
(l21, l22), L2Corresponding predeterminable area is A2.
In above-mentioned steps S122, one piece of region can be selected in the predeterminable area of interaction locations, then goes to step
Pixel comparison process is carried out in S130, and step S123 is triggered by pixel comparison result, it is specific: selected by being shown when comparison result
When the corresponding image in one piece of region selected does not have uniqueness relative to operation interface frame, step S121 is jumped to, i.e., default
Another piece or muti-piece region are selected in region；At this point, selected region is muti-piece, and the muti-piece region passes through determining position
It is associated to set layout relationship.
In above-mentioned steps S122, it can also disposably select to pass through location layout pass in the predeterminable area of interaction locations
The muti-piece region of connection, then goes to progress pixel comparison process in step S130, triggers step S123 by pixel comparison result,
It is specific: when comparison result shows that the corresponding image in selected muti-piece region does not have uniqueness relative to operation interface frame
When, step S121 is jumped to, i.e., selects another piece or muti-piece region in predeterminable area.
Above-mentioned steps S130 specifically: by described one piece or by the corresponding image in the associated muti-piece region of location layout with
Operation interface frame carries out pixel ratio pair；And in step S140, according to comparison result, determining has only relative to operation interface frame
Described one piece of one property or by the corresponding image in the associated muti-piece region of location layout be operation interface frame unique identification.
In the present embodiment, the pixel sequence and operation that include by the corresponding image in one or more region that will be selected
The rest of pixels region (not including the corresponding part in the rectangular area) of interface frame is compared, to judge operation interface frame
In with the presence or absence of image pixel corresponding with favored area be distributed consistent region.
For region individual for one piece, if there is the picture with the region in the rest of pixels region of operation interface frame
The plain consistent region in part, alternatively, if there is the pixel matching degree with the region in the rest of pixels region of operation interface frame
The region for reaching particular preset value illustrates that the corresponding image in the region does not have uniqueness relative to operation interface frame；Otherwise, have
There is uniqueness.
It should be noted that the demand based on different application scenarios, can preset pixel matching value, i.e., above-mentioned spy
Fixed preset value.When actually comparing obtained pixel matching degree and reaching specific preset value, that is, think the pixel in two pieces of regions
It is distributed identical.For example, the corresponding image in one piece of region and operation interface frame are carried out pixel ratio pair, comparison result display operation circle
There is the region for reaching 90% with the block area pixel matching degree in the frame of face, determines that the block region does not have relative to operation interface frame
There is uniqueness；Correspondingly, if pixel ratio is in operation interface frame frame as the result is shown, there is no the pixel matchings with the block region
Degree reaches 90% region, determines that the block region has uniqueness relative to operation interface frame.
For by for the associated muti-piece region of location layout, if the rest of pixels region of operation interface frame exist with
The location layout in aforementioned muti-piece region is identical and the pixel distribution in the corresponding region in position is also consistent, illustrates that the muti-piece region is corresponding
Image relative to operation interface frame do not have uniqueness；Otherwise, there is uniqueness.
Based on this, the present embodiment determines described one piece or as steps described below by the associated muti-piece region pair of location layout
Whether the image answered has uniqueness relative to the operation interface frame:
If there is image pixel phase corresponding with one piece of selected region in remaining region of the operation interface frame
Same region passes through the associated muti-piece of location layout with selected alternatively, existing in remaining region of the operation interface frame
The identical region of the corresponding image pixel in region, and the position of the identical interregional location layout and the muti-piece region
It sets that layout is identical, determines selected one piece or by the corresponding image in the associated muti-piece region of location layout relative to the behaviour
Make interface frame with uniqueness.
For example, selected rectangular area includes being shown by location layout associated C1, C2 and C3 in Fig. 6 refering to Fig. 6
In other regions in operation interface out, there is and location layout consistent with the pixel of C1, C2 and C3 also identical C11, C22
And C33, illustrate that C1, C2 and C3 do not have uniqueness relative to operation interface frame.Conversely, having uniqueness.Preferably, this implementation
In example step S120, the shape in the selected region is rectangle, to be conducive to the reality of pixel ratio equity image processing process
It is existing, save computing resource.
The method of determining operation interface frame unique identification provided by the present application, firstly, obtaining operation interface frame to be processed
With with the associated human-computer interaction event of operation interface frame to be processed the operation interface frame interaction locations information；So
Afterwards, one piece is selected in operation interface frame to be processed according to interaction locations information or by the associated muti-piece region of location layout；
Finally, by the method for pixel ratio pair, one piece or by location layout pass with uniqueness is determined relative to operation interface frame
The corresponding image in muti-piece region of connection is the unique identification of the operation interface frame.It is determined and is operated by the application method and device
The unique identification of interface frame, when so that subsequent execution including the script of the man-machine interactive operation of the unique identification, Ke Yili
With the relative positional relationship of unique identification pre-recorded in script and interaction locations, the target interaction executed on interface is determined
Position, so that analog subscriber operation executes corresponding interactive action automatically at the target interaction locations.Furthermore it is also possible to pass through
Unique identification is stored to replace corresponding operation interface frame, reduces data storage capacity and data processing amount.
Embodiment two
Refering to Fig. 7, the present embodiment provides the method for another unique identification for determining operation interface frame, this method includes
Following step:
Step S210, the operation interface frame and the alternative events when acquisition human-computer interaction event occurs are in operation circle
The interaction locations information of face frame；
Step S220 determines binarization threshold according to the gray value of whole pixels of the operation interface frame；
Step S230 carries out binaryzation conversion to operation interface frame according to the binarization threshold；
Step S240 selects one piece or logical according to interaction locations information out of operation interface frame after binaryzation conversion
Cross the associated muti-piece region of location layout；
Step S250 by described one piece or passes through the corresponding image in the associated muti-piece region of location layout and operation interface frame
Carry out pixel ratio pair；
Step S260 is determined and described one piece or is passed through relative to operation interface frame is with uniqueness according to comparison result
The corresponding image in the associated muti-piece region of location layout is the unique identification of operation interface frame.
The present embodiment two and the difference of above-described embodiment one are only that, before the step S120 of embodiment one, further include
The step of binaryzation conversion process is carried out to the operation interface frame of acquisition.
By pre-processing in advance to operation interface frame image, the calculation amount of image procossing correlated process can be reduced,
Such as later pixel comparison process, while the error of pixel comparison result can be reduced or eliminated.
Embodiment three
Refering to Fig. 8, the present embodiment provides the method for another unique identification for determining operation interface frame, this method includes
Following step:
Step S310, the operation interface frame and the alternative events when acquisition human-computer interaction event occurs are in operation circle
The interaction locations information of face frame；
Step S320 determines binarization threshold according to the gray value of whole pixels of the operation interface frame；
Step S330 carries out binaryzation conversion to operation interface frame according to the binarization threshold；
Step S340 selects one piece in the operation interface frame according to interaction locations information or is associated with by location layout
Muti-piece region；
Step S350 by described one piece or passes through the corresponding image in the associated muti-piece region of location layout and operation interface frame
Carry out pixel ratio pair；
Step S360 is determined and described one piece or is passed through relative to operation interface frame is with uniqueness according to comparison result
The corresponding image in the associated muti-piece region of location layout is the unique identification of operation interface frame；
Step S370 is the unique identification include one piece on the operation interface frame or is closed by location layout
The corresponding image addition mark in the muti-piece region of connection.
Embodiment three and the difference of above-described embodiment two are only that, after the step S260 of embodiment two, further include
The operation interface frame marks the step of mark.
It can be used the place that mark is easily quickly identified at family by that will identify to mark out in operation interface frame
Position is conducive to user and distinguishes to operation interface frame, is also convenient for user according to these marks and establishes these operation interface frames
Between logical relation, so that it is determined that in script each sub- script corresponding with each operation interface frame execution sequence and logic
Relationship.
It should be noted that the logical relation of sub- script or operation interface interframe described in the present embodiment includes judgement, jumps
Turn, branch process etc..For example, putting the picture that "Yes" or "No" jumps when picture pop-up is comprising two prompts of "Yes" and "No"
Face belongs to Liang Ge branch.When picture pops up login page, illustrate that currently logging in possible time-out need to log in again, the login screen
Logical relation between sub- script corresponding with the picture before jumping is to jump.
Example IV
According to the method for the mark of determining operation interface frame provided by the above embodiment, example IV provides a kind of determining behaviour
The device for making the unique identification of interface frame, refering to Fig. 9, which includes:
Acquiring unit U100, for obtain human-computer interaction event occur when operation interface frame and the alternative events in institute
State the interaction locations information of operation interface frame；
Selecting unit U200, for selecting one piece in the operation interface frame according to interaction locations information or passing through position
It is laid out associated muti-piece region；
Comparing unit U300, for and by described one piece or by the corresponding image in the associated muti-piece region of location layout grasp
Make interface frame and carries out pixel ratio pair；
Determination unit U400, for determining relative to operation interface frame with uniqueness described one according to comparison result
Block is the unique identification of operation interface frame by the corresponding image in the associated muti-piece region of location layout.
Refering to fig. 10, it is preferred that described device further include:
Image processing unit U500, the gray value for whole pixels according to the operation interface frame determine binaryzation
Threshold value；
And binaryzation conversion is carried out to operation interface frame according to the binarization threshold.
Preferably, the selecting unit U200 is specifically used for:
Interaction locations are determined according to the interaction locations information；
One piece of region is selected in the predeterminable area of the interaction locations or passes through the associated muti-piece region of location layout；
When the comparison result show selected one piece of region or by the associated muti-piece region of location layout it is corresponding
When image does not have uniqueness relative to operation interface frame, another piece of region is selected in the predeterminable area；Described another piece
Region with selected one piece of region or muti-piece region there is location layout to be associated with.
Preferably, the determination unit U400 is specifically used for:
If there is image pixel phase corresponding with one piece of selected region in remaining region of the operation interface frame
Same region passes through the associated muti-piece of location layout with selected alternatively, existing in remaining region of the operation interface frame
The identical region of the corresponding image pixel in region, and the position of the identical interregional location layout and the muti-piece region
It sets that layout is identical, determines selected one piece or by the corresponding image in the associated muti-piece region of location layout relative to the behaviour
Make interface frame with uniqueness.
Preferably, described device further include:
Unit U600 is marked, for being the unique identification include one piece on the operation interface frame or passing through position
Set the corresponding image addition mark in the associated muti-piece region of layout.
The device of the unique identification for the determination operation interface frame that the present embodiment four provides, firstly, obtaining operation to be processed
Interface frame and with the associated human-computer interaction event of operation interface frame to be processed the operation interface frame interaction locations believe
Breath；Then, one piece is selected in operation interface frame to be processed according to interaction locations information or by the associated muti-piece of location layout
Region；Finally, by the method for pixel ratio pair, determines and one piece or pass through position cloth relative to operation interface frame is with uniqueness
The corresponding image in the associated muti-piece region of office is the unique identification of the operation interface frame.It is determined by the application method and device
The unique identification of operation interface frame can when so that subsequent execution including the script of the man-machine interactive operation of the unique identification
To determine the target executed on interface using the relative positional relationship of unique identification and interaction locations pre-recorded in script
Interaction locations, so that analog subscriber operation executes corresponding interactive action automatically at the target interaction locations.Furthermore it is also possible to
Corresponding operation interface frame is replaced by storage unique identification, reduces data storage capacity and data processing amount.
In the specific implementation, the present invention also provides a kind of computer storage mediums, wherein the computer storage medium can store
There is program, which may include step some or all of in each embodiment of determining method provided by the invention when executing.Institute
The storage medium stated can for magnetic disk, CD, read-only memory (English: read-only memory, referred to as: ROM) or with
Machine storage memory (English: random access memory, referred to as: RAM) etc..
It is required that those skilled in the art can be understood that the technology in the embodiment of the present invention can add by software
The mode of general hardware platform realize.Based on this understanding, the technical solution in the embodiment of the present invention substantially or
Say that the part that contributes to existing technology can be embodied in the form of software products, which can deposit
Storage is in storage medium, such as ROM/RAM, magnetic disk, CD, including some instructions are used so that computer equipment (can be with
It is personal computer, server or the network equipment etc.) execute certain part institutes of each embodiment of the present invention or embodiment
The method stated.
Same and similar part may refer to each other between each embodiment in this specification.Implement especially for device
For example, since it is substantially similar to the method embodiment, so being described relatively simple, related place is referring in embodiment of the method
Explanation.
Invention described above embodiment is not intended to limit the scope of the present invention..
Claims (10)
1. a kind of method of the unique identification of determining operation interface frame, which is characterized in that the described method includes:
Obtain operation interface frame to be processed and with the associated human-computer interaction event of operation interface frame to be processed in the operation
The interaction locations information of interface frame；
One piece is selected in the operation interface frame according to interaction locations information or by the associated muti-piece region of location layout；
Pixel ratio pair is carried out by described one piece or by the corresponding image in the associated muti-piece region of location layout and operation interface frame；
According to comparison result, determine relative to operation interface frame it is with uniqueness described one piece or by location layout it is associated
The corresponding image in muti-piece region is the unique identification of operation interface frame.
2. the method according to claim 1, wherein it is described according to interaction locations information in the operation interface frame
One piece of interior selection or by before the associated muti-piece region of location layout, the method also includes:
Binarization threshold is determined according to the gray value of whole pixels of the operation interface frame；
Binaryzation conversion is carried out to operation interface frame according to the binarization threshold.
3. the method according to claim 1, wherein it is described according to interaction locations information in the operation interface frame
One piece of interior selection passes through the associated muti-piece region of location layout, comprising:
Interaction locations are determined according to the interaction locations information；
One piece of region is selected in the predeterminable area of the interaction locations or passes through the associated muti-piece region of location layout；
When the comparison result shows one piece of selected region or by the corresponding image in the associated muti-piece region of location layout
When not having uniqueness relative to operation interface frame, another piece of region is selected in the predeterminable area；Another piece of region
With selected one piece of region or muti-piece region there is location layout to be associated with.
4. the method according to claim 1, wherein determining described one piece as steps described below or by position cloth
Whether the corresponding image in the associated muti-piece region of office has uniqueness relative to the operation interface frame:
If it is identical to there is image pixel corresponding with one piece of selected region in remaining region of the operation interface frame
Region passes through the associated muti-piece region of location layout with selected alternatively, existing in remaining region of the operation interface frame
The identical region of corresponding image pixel, and the position cloth of the identical interregional location layout and the muti-piece region
Office is identical, determines one piece for selected or by the corresponding image in the associated muti-piece region of location layout relative to operation circle
Face frame has uniqueness.
5. method according to claim 1-4, which is characterized in that the shape in the selected region is rectangle.
6. the method according to claim 1, wherein the determination is with uniqueness relative to operation interface frame
Described one piece or by the corresponding image in the associated muti-piece region of location layout be operation interface frame unique identification after, institute
State method further include:
It is the unique identification include one piece or by the associated muti-piece region of location layout on the operation interface frame
Corresponding image addition mark.
7. a kind of device of the unique identification of determining operation interface frame, which is characterized in that described device includes:
Acquiring unit, for obtaining operation interface frame to be processed and with the associated human-computer interaction event of the operation interface frame in institute
State the interaction locations information of operation interface frame；
Selecting unit, for selecting one piece in the operation interface frame according to interaction locations information or being associated with by location layout
Muti-piece region；
Comparing unit is used for by described one piece or by the corresponding image in the associated muti-piece region of location layout and operation interface frame
Carry out pixel ratio pair；
Determination unit described one piece or passes through for determining according to comparison result relative to operation interface frame is with uniqueness
The corresponding image in the associated muti-piece region of location layout is the unique identification of operation interface frame.
8. device according to claim 7, which is characterized in that described device further include:
Image processing unit, the gray value for whole pixels according to the operation interface frame determine binarization threshold；
Binaryzation conversion is carried out to operation interface frame according to the binarization threshold.
9. device according to claim 7, which is characterized in that the selecting unit is specifically used for:
Interaction locations are determined according to the interaction locations information；
One piece of region is selected in the predeterminable area of the interaction locations or passes through the associated muti-piece region of location layout；
When the comparison result shows one piece of selected region or by the corresponding image in the associated muti-piece region of location layout
When not having uniqueness relative to operation interface frame, another piece of region is selected in the predeterminable area；Another piece of region
With selected one piece of region or muti-piece region there is location layout to be associated with.
10. device according to claim 7, which is characterized in that the determination unit is specifically used for:
If it is identical to there is image pixel corresponding with one piece of selected region in remaining region of the operation interface frame
Region passes through the associated muti-piece region of location layout with selected alternatively, existing in remaining region of the operation interface frame
The identical region of corresponding image pixel, and the position cloth of the identical interregional location layout and the muti-piece region
Office is identical, determines one piece for selected or by the corresponding image in the associated muti-piece region of location layout relative to operation circle
Face frame has uniqueness.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811246513.5A CN109445893A (en) | 2018-10-24 | 2018-10-24 | A kind of method and device of the unique identification of determining operation interface frame |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811246513.5A CN109445893A (en) | 2018-10-24 | 2018-10-24 | A kind of method and device of the unique identification of determining operation interface frame |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN109445893A true CN109445893A (en) | 2019-03-08 |
Family
ID=65547471
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201811246513.5A Pending CN109445893A (en) | 2018-10-24 | 2018-10-24 | A kind of method and device of the unique identification of determining operation interface frame |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN109445893A (en) |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112306885A (en) * | 2020-11-11 | 2021-02-02 | 天津汇商共达科技有限责任公司 | Script debugging method and device |
| CN112306886A (en) * | 2020-11-11 | 2021-02-02 | 天津汇商共达科技有限责任公司 | Script debugging method and device |
| CN112347178A (en) * | 2020-11-11 | 2021-02-09 | 天津汇商共达科技有限责任公司 | Data docking method and device based on human-computer interaction behavior, terminal and server |
| CN112347176A (en) * | 2020-11-11 | 2021-02-09 | 天津汇商共达科技有限责任公司 | Data docking method and device based on human-computer interaction behavior |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104252410A (en) * | 2013-06-27 | 2014-12-31 | 阿里巴巴集团控股有限公司 | Method and equipment for testing control in page |
| CN104899146A (en) * | 2015-06-19 | 2015-09-09 | 安一恒通（北京）科技有限公司 | Image matching technology based software stability test method and device |
| CN104951393A (en) * | 2014-03-25 | 2015-09-30 | 中国电信股份有限公司 | Testing method and device |
| CN106445816A (en) * | 2016-09-07 | 2017-02-22 | 美的智慧家居科技有限公司 | Control execution method and apparatus |
-
2018
- 2018-10-24 CN CN201811246513.5A patent/CN109445893A/en active Pending
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104252410A (en) * | 2013-06-27 | 2014-12-31 | 阿里巴巴集团控股有限公司 | Method and equipment for testing control in page |
| CN104951393A (en) * | 2014-03-25 | 2015-09-30 | 中国电信股份有限公司 | Testing method and device |
| CN104899146A (en) * | 2015-06-19 | 2015-09-09 | 安一恒通（北京）科技有限公司 | Image matching technology based software stability test method and device |
| CN106445816A (en) * | 2016-09-07 | 2017-02-22 | 美的智慧家居科技有限公司 | Control execution method and apparatus |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112306885A (en) * | 2020-11-11 | 2021-02-02 | 天津汇商共达科技有限责任公司 | Script debugging method and device |
| CN112306886A (en) * | 2020-11-11 | 2021-02-02 | 天津汇商共达科技有限责任公司 | Script debugging method and device |
| CN112347178A (en) * | 2020-11-11 | 2021-02-09 | 天津汇商共达科技有限责任公司 | Data docking method and device based on human-computer interaction behavior, terminal and server |
| CN112347176A (en) * | 2020-11-11 | 2021-02-09 | 天津汇商共达科技有限责任公司 | Data docking method and device based on human-computer interaction behavior |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109445893A (en) | A kind of method and device of the unique identification of determining operation interface frame | |
| US8595641B2 (en) | Methods, apparatus and systems for displaying and/or facilitating interaction with secure information via channel grid framework | |
| Plattfaut et al. | Service innovation performance and information technology: An empirical analysis from the dynamic capability perspective | |
| CN109324864B (en) | Method and device for acquiring man-machine interaction operation information | |
| CN104731446B (en) | Spot recovery technique based on prompt | |
| CN111324535B (en) | Control abnormity detection method and device and computer equipment | |
| US10198416B2 (en) | Customizing a form in a model-based system | |
| RU2473179C2 (en) | System and method to control icon on platform of instantaneous messaging | |
| US20120169772A1 (en) | Fractal whiteboarding | |
| CN110045953A (en) | Generate the method and computing device of business rule expression formula | |
| Miikkulainen et al. | Conversion rate optimization through evolutionary computation | |
| CN104182225B (en) | A kind of General Mobile information system adaptation method and device | |
| CN111694615B (en) | Method, device, equipment and storage medium for data configuration | |
| TW200406692A (en) | Semiconductor test data analysis system | |
| CN109492149A (en) | Crawler task processing method and device | |
| US20100094696A1 (en) | Personal style server | |
| CN110020279A (en) | Page data processing method, device and storage medium | |
| CN108804652B (en) | Method and device for generating cover picture, storage medium and electronic device | |
| CN107547614A (en) | The Numerical Methods and device of investment platform | |
| CN109343852B (en) | Method and device for displaying frame pictures of operation interface | |
| US20180349815A1 (en) | Information processing apparatus and information processing method | |
| CN115048302A (en) | Front-end compatibility testing method and device, storage medium and electronic equipment | |
| US10198150B2 (en) | Cross database data selection and correlation interface | |
| WO2017012598A1 (en) | Password setting method and device | |
| CN110221952B (en) | Service data processing method and device and service data processing system |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20190308 |