CN113625924A

CN113625924A - Element selection method, equipment, storage medium and device based on mobile operation

Info

Publication number: CN113625924A
Application number: CN202010391001.9A
Authority: CN
Inventors: 王志超; 高其林; 张杰生
Original assignee: Beijing Qihoo Technology Co Ltd
Current assignee: Beijing Qihoo Technology Co Ltd
Priority date: 2020-05-09
Filing date: 2020-05-09
Publication date: 2021-11-09

Abstract

The invention discloses an element selection method, equipment, a storage medium and a device based on movement operation, wherein the method comprises the steps of detecting cursor movement operation input by a user based on a current interface when an element selection instruction triggered by the user in the current interface is received, determining a corresponding cursor movement track according to the cursor movement operation, generating a closed area based on the cursor movement track, judging whether interface elements in the closed area exist in the current interface, if the interface elements in the closed area exist in the current interface, selecting the interface elements in the closed area, operating the moved closed area according to needs, and adapting to the centralized and complex arrangement of a relational graph, thereby enclosing the required interface elements and improving the frame selection accuracy of the interface elements.

Description

Element selection method, equipment, storage medium and device based on mobile operation

Technical Field

The present invention relates to the field of data processing technologies, and in particular, to a method, a device, a storage medium, and an apparatus for selecting an element based on a move operation.

Background

At present, the method for framing elements in an interface is a rectangular frame, required interface elements are framed by the rectangular frame, however, the arrangement of a relational graph is concentrated and complex, framing selection is performed by the rectangular frame, requirements cannot be met, unnecessary elements are easily framed into the rectangular frame, or required elements can be selected by framing selection for many times, and the framing accuracy of the interface elements is low.

The above is only for the purpose of assisting understanding of the technical aspects of the present invention, and does not represent an admission that the above is prior art.

Disclosure of Invention

The invention mainly aims to provide an element selection method, equipment, a storage medium and a device based on mobile operation, and aims to solve the technical problem that in the prior art, the frame selection accuracy of interface elements is low.

In order to achieve the above object, the present invention provides a method for selecting an element based on a move operation, including the following steps:

when an element selection instruction triggered in a current interface by a user is received, detecting cursor movement operation input by the user based on the current interface;

determining a corresponding cursor movement track according to the cursor movement operation;

generating a closed area based on the cursor movement track;

judging whether interface elements in the closed area exist in the current interface or not;

and if the interface elements in the closed area exist in the current interface, selecting the interface elements in the closed area.

Preferably, the determining whether an interface element in the closed area exists in the current interface specifically includes:

traversing the interface elements in the current interface, and taking the traversed interface elements as the current interface elements;

acquiring a central point of the current interface element;

generating a target ray along a preset direction by taking the central point as a starting point;

acquiring the number of intersection points of the target ray and the edge of the closed area;

and judging whether the current interface element is in the closed area or not according to the number of the intersection points.

Preferably, the determining whether the current interface element is located in the closed area according to the number of the intersection points specifically includes:

judging whether the number of the intersection points is an odd number;

and if the number of the intersection points is an odd number, determining that the current interface element is in the closed area.

traversing the interface elements in the current interface, and taking the traversed interface elements as interface elements to be confirmed;

acquiring the element area of the interface element to be confirmed;

calculating the area ratio of the interface element to be confirmed in the closed region according to the element area;

and judging whether the interface element to be confirmed is in the closed area or not according to the area ratio.

Preferably, the determining, according to the area ratio, whether the interface element to be confirmed is located in the closed region specifically includes:

judging whether the area ratio is larger than a first preset ratio or not;

and if the area ratio is larger than the first preset ratio, determining that the interface element to be confirmed is located in the closed area.

Preferably, after the interface element to be confirmed is determined to be located in the closed region if the area ratio is greater than the first preset ratio, the element selection method based on the moving operation further includes:

judging whether the interface elements to be confirmed with the area ratio smaller than or equal to the first preset ratio and larger than a second preset ratio exist, wherein the second preset ratio is smaller than the first preset ratio;

if the area ratio is smaller than or equal to the first preset ratio and larger than the second preset ratio, the interface element to be confirmed is regarded as an element to be identified;

performing character recognition on the element to be recognized to obtain text information to be recognized;

performing character recognition on the interface elements in the closed area to obtain target text information;

and judging whether the element to be recognized is in the closed area or not according to the text information to be recognized and the target text information.

Preferably, the determining, according to the text information to be recognized and the target text information, whether the element to be recognized is located in the closed area specifically includes:

calculating text similarity between the text information to be recognized and the target text information;

and judging whether the element to be recognized is in the closed area or not according to the text similarity.

Preferably, the calculating the text similarity between the text information to be recognized and the target text information specifically includes:

extracting keywords from the text information to be recognized to obtain text keywords to be recognized;

extracting keywords from the target text information to obtain target text keywords;

and calculating the text similarity between the text information to be recognized and the target text information according to the text key words to be recognized and the target text key words.

Preferably, the interface elements in the closed area are multiple;

the determining, according to the text information to be recognized and the target text information, whether the element to be recognized is located in the closed region specifically includes:

clustering target text information of a plurality of interface elements in the closed area to obtain a clustering center;

calculating the clustering similarity between the text information to be identified and the clustering center;

and judging whether the element to be identified is in the closed region or not according to the clustering similarity.

Preferably, the determining, according to the cluster similarity, whether the element to be identified is located in the closed region specifically includes:

judging whether the clustering similarity exceeds a preset similarity threshold value or not;

and when the clustering similarity exceeds the preset similarity threshold, determining that the element to be recognized is in the closed region.

Preferably, the generating a closed region based on the cursor movement trajectory specifically includes:

judging whether a discontinuity point exists in the cursor moving track;

if a discontinuity point exists in the cursor moving track, taking coordinate points except the discontinuity point in the cursor moving track as target coordinate points;

and connecting the target coordinate points to obtain a closed area.

Preferably, the determining whether there is a discontinuity in the cursor movement trajectory specifically includes:

calculating a coordinate difference value between adjacent coordinate points in the cursor movement track;

and judging whether a catastrophe point exists in the cursor moving track according to the coordinate difference value.

Preferably, the determining whether there is a discontinuity in the cursor movement trajectory according to the coordinate difference specifically includes:

arranging the coordinate difference values in a descending order to obtain an ordered list;

acquiring a median of the sorted list;

calculating a median difference value between each coordinate difference value and the median;

judging whether the median difference value exceeds a preset difference threshold value or not;

and if the median difference value exceeds a preset difference value threshold value, identifying the coordinate point corresponding to the median difference value as a catastrophe point in the cursor moving track.

In addition, to achieve the above object, the present invention further provides a mobile operation-based element selection apparatus, where the mobile operation-based element selection apparatus includes a memory, a processor, and a mobile operation-based element selection program stored in the memory and executable on the processor, and the mobile operation-based element selection program is configured to implement the steps of the mobile operation-based element selection method described above.

In addition, to achieve the above object, the present invention further provides a storage medium, on which an element selection program based on a move operation is stored, and the element selection program based on a move operation, when executed by a processor, implements the steps of the element selection method based on a move operation as described above.

In addition, to achieve the above object, the present invention further provides an element selecting apparatus based on a move operation, including:

the detection module is used for detecting cursor movement operation input by a user based on the current interface when receiving an element selection instruction triggered by the user in the current interface;

the determining module is used for determining a corresponding cursor moving track according to the cursor moving operation;

the generating module is used for generating a closed area based on the cursor moving track;

the judging module is used for judging whether the interface elements in the closed area exist in the current interface or not;

and the selection module is used for selecting the interface elements in the closed area if the interface elements in the closed area exist in the current interface.

Preferably, the determining module is further configured to traverse the interface element in the current interface, and use the traversed interface element as the current interface element; acquiring a central point of the current interface element; generating a target ray along a preset direction by taking the central point as a starting point; acquiring the number of intersection points of the target ray and the edge of the closed area; and judging whether the current interface element is in the closed area or not according to the number of the intersection points.

Preferably, the judging module is further configured to judge whether the number of the intersection points is an odd number; and if the number of the intersection points is an odd number, determining that the current interface element is in the closed area.

Preferably, the determining module is further configured to traverse the interface element in the current interface, and use the traversed interface element as an interface element to be confirmed; acquiring the element area of the interface element to be confirmed; calculating the area ratio of the interface element to be confirmed in the closed region according to the element area; and judging whether the interface element to be confirmed is in the closed area or not according to the area ratio.

Preferably, the generating module is further configured to determine whether there is a discontinuity in the cursor movement trajectory; if a discontinuity point exists in the cursor moving track, taking coordinate points except the discontinuity point in the cursor moving track as target coordinate points; and connecting the target coordinate points to obtain a closed area.

According to the method, when an element selection instruction triggered in a current interface by a user is received, cursor movement operation input by the user based on the current interface is detected, a corresponding cursor movement track is determined according to the cursor movement operation, a closed area is generated based on the cursor movement track, and a cursor can be moved randomly according to needs, so that the method is suitable for complex relational graph arrangement and selects any element in a frame mode; and judging whether the interface elements in the closed area exist in the current interface or not, if so, selecting the interface elements in the closed area, and operating the moved closed area according to needs, so that the centralized and complex arrangement of the relation graph can be adapted, the required interface elements are circled, and the frame selection accuracy of the interface elements is improved.

Drawings

Fig. 1 is a schematic structural diagram of an element selection device based on mobile operation in a hardware execution environment according to an embodiment of the present invention;

FIG. 2 is a flowchart illustrating a first embodiment of a method for selecting elements based on move operations according to the present invention;

FIG. 3 is a schematic diagram of the relationship between interface elements and cursor movement trajectories;

FIG. 4 is a schematic diagram of the relationship between interface elements and closed regions;

FIG. 5 is a flowchart illustrating a second embodiment of a method for selecting elements based on move operations according to the present invention;

FIG. 6 is a flowchart illustrating a third embodiment of a method for selecting elements based on move operations according to the present invention;

FIG. 7 is a flowchart illustrating a fourth embodiment of a method for selecting elements based on move operations according to the present invention;

fig. 8 is a block diagram of a first embodiment of an element selection apparatus based on move operation according to the present invention.

The implementation, functional features and advantages of the objects of the present invention will be further explained with reference to the accompanying drawings.

Detailed Description

It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.

Referring to fig. 1, fig. 1 is a schematic diagram of an element selection device based on mobile operation in a hardware operating environment according to an embodiment of the present invention.

As shown in fig. 1, the element selection device based on the move operation may include: a processor 1001, such as a Central Processing Unit (CPU), a communication bus 1002, a user interface 1003, a network interface 1004, and a memory 1005. Wherein a communication bus 1002 is used to enable connective communication between these components. The user interface 1003 may include a Display screen (Display), and the optional user interface 1003 may further include a standard wired interface and a wireless interface, and the wired interface for the user interface 1003 may be a USB interface in the present invention. The network interface 1004 may optionally include a standard wired interface, a WIreless interface (e.g., a WIreless-FIdelity (WI-FI) interface). The Memory 1005 may be a Random Access Memory (RAM) Memory or a Non-volatile Memory (NVM), such as a disk Memory. The memory 1005 may alternatively be a storage device separate from the processor 1001.

Those skilled in the art will appreciate that the configuration shown in fig. 1 does not constitute a limitation of the element selection device based on movement operations, and may include more or less components than those shown, or some components in combination, or a different arrangement of components.

As shown in fig. 1, a memory 1005, which is a kind of computer storage medium, may include therein an operating system, a network communication module, a user interface module, and an element selection program based on a mobile operation.

In the element selection device based on mobile operation shown in fig. 1, the network interface 1004 is mainly used for connecting to a background server and performing data communication with the background server; the user interface 1003 is mainly used for connecting user equipment; the element selecting device based on the move operation calls an element selecting program based on the move operation stored in the memory 1005 through the processor 1001, and executes the element selecting method based on the move operation according to the embodiment of the present invention.

The move operation based element selection apparatus calls a move operation based element selection program stored in the memory 1005 by the processor 1001, and performs the following operations:

generating a closed area based on the cursor movement track;

Further, the move operation based element selection apparatus calls the move operation based element selection program stored in the memory 1005 by the processor 1001, and further performs the following operations:

acquiring a central point of the current interface element;

judging whether the number of the intersection points is an odd number;

acquiring the element area of the interface element to be confirmed;

judging whether the area ratio is larger than a first preset ratio or not;

Further, the interface elements in the closed area are multiple, and the device for selecting elements based on moving operation calls, through the processor 1001, the program for selecting elements based on moving operation stored in the memory 1005, and further performs the following operations:

judging whether a discontinuity point exists in the cursor moving track;

and connecting the target coordinate points to obtain a closed area.

acquiring a median of the sorted list;

In this embodiment, when an element selection instruction triggered by a user in a current interface is received, a cursor movement operation input by the user based on the current interface is detected, a corresponding cursor movement track is determined according to the cursor movement operation, a closed area is generated based on the cursor movement track, and a cursor can be moved randomly as required, so that complex relational graph arrangement is adapted, and any element is framed and selected; and judging whether the interface elements in the closed area exist in the current interface or not, if so, selecting the interface elements in the closed area, and operating the moved closed area according to needs, so that the centralized and complex arrangement of the relation graph can be adapted, the required interface elements are circled, and the frame selection accuracy of the interface elements is improved.

Based on the hardware structure, the embodiment of the element selection method based on the mobile operation is provided.

Referring to fig. 2, fig. 2 is a flowchart illustrating a method for selecting an element based on a move operation according to a first embodiment of the present invention.

In a first embodiment, the element selection method based on the moving operation includes the following steps:

step S10: when an element selection instruction triggered by a user in the current interface is received, cursor movement operation input by the user based on the current interface is detected.

It should be understood that the execution subject of the embodiment is the mobile operation-based element selection device, and the mobile operation-based element selection device may be an electronic device such as a smartphone, a tablet computer, or a personal computer, which is not limited in this embodiment. When a user clicks a current interface, it can be considered that an element selection instruction triggered in the current interface by the user is received, and a cursor is continuously moved, that is, a cursor movement operation is input based on the current interface. Taking a personal computer as an example for explanation, a user presses a mouse to select a starting point, the starting point is recorded, and a moving coordinate point of a cursor is recorded in real time in a moving process that the user does not loosen the mouse, namely, the user moves the cursor based on current interface input.

Step S20: and determining a corresponding cursor movement track according to the cursor movement operation.

It can be understood that the cursor coordinate points in the cursor moving operation process are recorded, and the cursor coordinate points recorded in the moving operation are sequentially connected according to the sequence to obtain the cursor moving track. As shown in fig. 3, fig. 3 is a schematic diagram of a relationship between an interface element and a cursor movement track.

Step S30: and generating a closed area based on the cursor movement track.

It should be noted that the cursor movement trajectory is connected into a closed region, and coordinate points passing through in the cursor movement process are connected one by one from a starting point to a connection end point to form a closed region, and the closed region is usually an irregular polygon. If the cursor moving track is a closed area, directly taking the cursor moving track as the closed area; if the cursor moving track is not a closed area and the coordinates of the starting point and the ending point are different, connecting coordinate points passing through the cursor moving process from the starting point to the ending point one by one, and connecting the ending point and the starting point in a straight line, so that the closed area is generated. As shown in fig. 4, fig. 4 is a schematic diagram of the relationship between the interface element and the closed region.

Step S40: and judging whether the interface elements in the closed area exist in the current interface or not.

In a specific implementation, the current interface is a display page including a plurality of interface elements, and may be a relationship diagram with complex relationships or other page contents. And judging whether the interface element in the current interface is selected in the closed area, specifically, traversing the interface element in the current interface, taking the traversed interface element as the current interface element, acquiring a central point of the current interface element, and generating a target ray along a preset direction by taking the central point as a starting point, wherein the preset direction comprises a horizontal direction, and if the number of intersection points of the target ray and the edge of the closed area is an odd number, determining that the current interface element is the interface element in the closed area. As shown in fig. 4, if the number of intersections between the center points of interface elements 2 and 5 and the edges of the closed region is 1 and is an odd number, interface elements 2 and 5 are considered as interface elements located in the closed region.

It can be understood that the element area of the current interface element may also be obtained, the area ratio of the current interface element in the closed area is calculated, and whether the area ratio exceeds a first preset ratio is determined, where the first preset ratio is usually set to 0.5, that is, when the area ratio exceeds half of the element area, the current interface element is determined to be the interface element in the closed area. As shown in fig. 4, if the area ratio of the interface element 2 to the interface element 5 exceeds the first preset ratio, the interface element 2 and the interface element 5 are determined to be interface elements in the closed region.

Step S50: and if the interface elements in the closed area exist in the current interface, selecting the interface elements in the closed area.

It should be understood that if an interface element in the closed area exists in the current interface, the interface element in the closed area is considered to be an interface element that the user needs to select, and then the interface element is selected for the next operation. As shown in fig. 4, if the interface elements 2 and 5 are interface elements that the user needs to select, the interface elements 2 and 5 are selected.

Referring to fig. 5, fig. 5 is a flowchart illustrating a second embodiment of the method for selecting an element based on a move operation according to the present invention, and the second embodiment of the method for selecting an element based on a move operation according to the present invention is provided based on the first embodiment illustrated in fig. 2.

In the second embodiment, the step S40 includes:

step S401: and traversing the interface elements in the current interface, and taking the traversed interface elements as the current interface elements.

It should be understood that there are usually a plurality of interface elements in the current interface, taking the current interface as a relationship graph as an example, the interface elements are nodes in the relationship graph, each node in the relationship graph is traversed, and the traversed node is taken as the current interface element.

Step S402: and acquiring the central point of the current interface element.

It can be understood that the center point of the multi-deformation formed by the boundary of the current interface element is the center point of the current interface element. The current interface element is usually presented in the form of a text box, and then the center point of the text box where the current interface element is located may be used as the center point of the current interface element.

Step S403: and generating a target ray along a preset direction by taking the central point as a starting point.

It should be noted that the preset direction includes a horizontal direction, a vertical direction, or any direction, and the target rays are generated along the preset direction with the central point as a starting point, and if the central point is in the closed region, the number of intersection points of the target rays and the closed region is an odd number.

Step S404: and acquiring the number of intersection points of the target ray and the edge of the closed area.

In a specific implementation, the closed region is usually an irregular polygon, an edge of the irregular polygon is obtained, and whether the target ray intersects with the edge of the irregular polygon is determined, so as to obtain the number of intersection points.

Step S405: and judging whether the current interface element is in the closed area or not according to the number of the intersection points.

It should be understood that if the center point is within the closed region, the number of intersection points of the target ray with the closed region is odd; and if the central point is not in the closed region, the number of the intersection points of the target ray and the closed region is an even number. In this embodiment, step S405 includes: judging whether the number of the intersection points is an odd number; and if the number of the intersection points is an odd number, determining that the current interface element is in the closed area.

In this embodiment, the interface element in the current interface is traversed, the traversed interface element is used as the current interface element, a central point of the current interface element is obtained, a target ray is generated along a preset direction with the central point as a starting point, the number of intersection points of the target ray and the edge of the closed area is obtained, whether the current interface element is located in the closed area is judged according to the number of the intersection points, and the accuracy of interface element framing is improved.

Referring to fig. 6, fig. 6 is a flowchart illustrating a third embodiment of the method for selecting an element based on a move operation according to the present invention, and the third embodiment of the method for selecting an element based on a move operation according to the present invention is provided based on the first embodiment illustrated in fig. 2.

In the third embodiment, the step S40 includes:

step S406: and traversing the interface elements in the current interface, and taking the traversed interface elements as interface elements to be confirmed.

It should be understood that there are usually a plurality of interface elements in the current interface, taking the current interface as a relationship diagram as an example, the interface elements are nodes in the relationship diagram, each node in the relationship diagram is traversed, and the traversed node is taken as the interface element to be confirmed.

Step S407: and acquiring the element area of the interface element to be confirmed.

It can be understood that the area of the multi-deformation formed by the boundary of the current interface element is the element area of the current interface element. The current interface element is usually presented in the form of a text box, and the area of the text box where the current interface element is located may be used as the element area of the current interface element.

Step S408: and calculating the area ratio of the interface element to be confirmed in the closed region according to the element area.

It should be noted that, the frame selection area of the interface element to be confirmed in the closed region is obtained, and the area ratio of the interface element to be confirmed in the closed region is calculated according to the frame selection area and the element area. The ratio between the frame area and the element area is calculated, and the ratio is the area ratio.

Step S409: and judging whether the interface element to be confirmed is in the closed area or not according to the area ratio.

In a specific implementation, if the area ratio exceeds a first preset ratio, the interface element to be confirmed is determined to be in the closed region, where the first preset ratio is usually set to 0.5, that is, when the area ratio exceeds half of the area of the element, the current interface element is determined to be the interface element in the closed region. The big data analysis may also be performed, and the first preset ratio is set according to a large number of selected samples, which is not limited in this embodiment. In this embodiment, the step S409 includes: judging whether the area ratio is larger than a first preset ratio or not; and if the area ratio is larger than the first preset ratio, determining that the interface element to be confirmed is located in the closed area.

Further, in this embodiment, if the area ratio is greater than the first preset ratio, after determining that the interface element to be confirmed is located in the closed area, the method further includes:

judging whether the interface elements to be confirmed with the area ratio smaller than or equal to the first preset ratio and larger than a second preset ratio exist, wherein the second preset ratio is smaller than the first preset ratio; if the area ratio is smaller than or equal to the first preset ratio and larger than the second preset ratio, the interface element to be confirmed is regarded as an element to be identified; performing character recognition on the element to be recognized to obtain text information to be recognized; performing character recognition on the interface elements in the closed area to obtain target text information; and judging whether the element to be recognized is in the closed area or not according to the text information to be recognized and the target text information.

It should be understood that the interface elements to be confirmed with the area ratio smaller than or equal to the first preset ratio and larger than a second preset ratio, that is, a part of the interface elements to be confirmed are in the closed region, another part of the interface elements are outside the closed region, the second preset ratio is smaller than the first preset ratio, for example, the first preset ratio is 0.5, the second preset ratio is 0.3, the second preset ratio may be set according to an empirical value, or may be subjected to big data analysis, and the second preset ratio is set according to a large number of frame-selected samples, which is not limited in this embodiment.

It can be understood that if there are interface elements to be confirmed whose area ratio is less than or equal to the first preset ratio and greater than the second preset ratio, the interface elements to be confirmed whose area ratio is less than or equal to the first preset ratio and greater than the second preset ratio are determined as the elements to be identified, and it is necessary to further determine whether the elements to be identified are interface elements that the user wants to select.

It should be noted that the content of the element to be recognized may be recognized to obtain recognition text information corresponding to the element to be recognized, then obtain an interface element in the closed region, perform text recognition on the interface element in the closed region to obtain target text information corresponding to the interface element in the closed region, calculate text similarity between the recognition text information and the target text information, and determine whether the text similarity exceeds a first similarity threshold, where the first similarity threshold may be set according to an empirical value, for example, the first similarity threshold is set to 50%, and if the text similarity exceeds the first similarity threshold, the element to be recognized is determined to be in the closed region, and the element to be recognized is selected; and if the text similarity does not exceed the first similarity threshold, determining that the element to be recognized is located outside the closed region. In this embodiment, the determining, according to the text information to be recognized and the target text information, whether the element to be recognized is located in the closed area specifically includes: calculating text similarity between the text information to be recognized and the target text information; and judging whether the element to be recognized is in the closed area or not according to the text similarity.

Further, in this embodiment, the calculating the text similarity between the text information to be recognized and the target text information specifically includes:

extracting keywords from the text information to be recognized to obtain text keywords to be recognized; extracting keywords from the target text information to obtain target text keywords; and calculating the text similarity between the text information to be recognized and the target text information according to the text key words to be recognized and the target text key words.

In a specific implementation, the text information to be recognized is segmented to obtain all first words of the text information to be recognized, useless words in the first words are filtered to obtain the text keywords to be recognized, and the useless words comprise words without practical meanings such as mood words and auxiliary words. And segmenting the target text information to obtain all second words of the target text information, and filtering useless words in the second words to obtain the target text keywords. Calculating TF-IDF values, TF word frequencies (Term frequencies) and IDF reverse file frequencies (Inverse Document frequencies) of the text keywords to be recognized, and representing the text information to be recognized into a first word vector consisting of the text keywords to be recognized and the corresponding TF-IDF values; calculating TF-IDF values of the target text keywords, and representing the target text information into a second word vector consisting of the target text keywords and the corresponding TF-IDF values; and calculating the cosine distance between the first word vector and the second word vector, and taking the cosine distance as the text similarity between the text information to be recognized and the target text information.

Further, in this embodiment, there are a plurality of interface elements in the closed area;

clustering target text information of a plurality of interface elements in the closed area to obtain a clustering center; calculating the clustering similarity between the text information to be identified and the clustering center; and judging whether the element to be identified is in the closed region or not according to the clustering similarity.

It should be understood that the target text information of a plurality of interface elements in the closed region may be clustered by an Affinity Propagation clustering (AP) algorithm, where the Affinity Propagation clustering algorithm clusters according to similarity between data points, and may be symmetric or asymmetric. The affinity propagation clustering algorithm does not need to determine the number of clusters first, but rather considers all data points as cluster centers in the sense of potential.

It can be understood that the target similarity between each target text message is calculated, a similarity matrix can be constructed according to each target similarity, each target text message is used as a node, the target similarity between each target text message is used as a matrix value, then clustering is performed through an AP algorithm, a cluster is obtained, the cluster comprises a cluster center, the cluster center has the strongest similarity with other target text messages in the corresponding cluster, and the cluster center has the most representativeness of the cluster.

It should be understood that, the clustering similarity between the text information to be recognized and the clustering center is calculated, and whether the clustering similarity exceeds a preset similarity threshold is determined, where the preset similarity threshold may be set according to an empirical value, for example, the preset similarity threshold is set to 50%, or big data analysis may be performed, and the preset similarity threshold is set according to a large number of frame-selected samples, which is not limited in this embodiment. When the clustering similarity exceeds the preset similarity threshold, determining that the element to be identified is in the closed region; and when the clustering similarity does not exceed the preset similarity threshold, determining that the element to be identified is located outside the closed region. In this embodiment, the determining, according to the cluster similarity, whether the element to be identified is located in the closed region specifically includes: judging whether the clustering similarity exceeds a preset similarity threshold value or not; and when the clustering similarity exceeds the preset similarity threshold, determining that the element to be recognized is in the closed region.

In this embodiment, interface elements in the current interface are traversed, the traversed interface elements are used as interface elements to be confirmed, the element areas of the interface elements to be confirmed are obtained, the area ratios of the interface elements to be confirmed in the closed region are calculated according to the element areas, whether the interface elements to be confirmed are in the closed region is judged according to the area ratios, the interface elements with the area ratios exceeding a certain ratio are determined as the interface elements in the closed region, and therefore the problem that whether part of the framed interface elements are selected is solved, and the efficiency and accuracy of framing the interface elements are improved.

Referring to fig. 7, fig. 7 is a flowchart illustrating a third embodiment of the method for selecting an element based on a move operation according to the present invention, and a fourth embodiment of the method for selecting an element based on a move operation according to the present invention is provided based on the first, second, or third embodiments of the methods. The present embodiment will be described based on the first embodiment.

In this embodiment, the step S30 includes:

step S301: and judging whether a catastrophe point exists in the cursor moving track.

It should be understood that, the cursor coordinate points in the cursor moving operation process are recorded, and the cursor coordinate points recorded in the moving operation are sequentially connected according to the sequence, so as to obtain the cursor moving track. The user moves the operation based on the cursor input at the current interface, if the cursor is separated from the normal track due to external factors in the cursor moving operation process, the abrupt change point occurs, for example, the user is impacted by an external force to an arm or other reasons in the cursor moving process through a mouse, so that the cursor moves beyond the original track, and the abrupt change point occurs.

Step S302: and if the cursor moving track has a catastrophe point, taking coordinate points except the catastrophe point in the cursor moving track as target coordinate points.

It can be understood that if there is a discontinuity in the cursor movement trajectory, the discontinuity needs to be filtered, and coordinate points other than the discontinuity in the cursor movement trajectory are retained as target coordinate points.

Step S303: and connecting the target coordinate points to obtain a closed area.

It should be noted that the target coordinate points are sequentially connected according to the recorded time sequence to obtain the closed region.

Further, in this embodiment, the step S301 includes:

calculating a coordinate difference value between adjacent coordinate points in the cursor movement track; and judging whether a catastrophe point exists in the cursor moving track according to the coordinate difference value.

In a specific implementation, each coordinate point in the cursor movement track is obtained, a coordinate difference value between adjacent coordinate points is calculated, the cursor is usually moved gently in the process of moving the cursor, the coordinate difference value is usually within a certain range, a large number of sample cursor movement tracks can be obtained for analysis, a difference value range where the coordinate difference value is located in the gentle movement process is obtained, the coordinate difference value is compared with the difference value range, if the coordinate difference value exceeds the difference value range, whether a mutation point exists in the cursor movement track or not is determined, a coordinate point corresponding to the coordinate difference value exceeding the difference value range is determined as the mutation point, usually the coordinate difference value between the mutation point and the adjacent coordinate point in front of the mutation point is abnormal, and the coordinate difference value between the mutation point and the adjacent coordinate point in back of the mutation point is also abnormal, therefore, a coordinate point whose coordinate difference from the adjacent coordinate points before and after exceeds the difference range can be identified as the mutation point.

Further, in this embodiment, the determining whether there is a discontinuity in the cursor movement trajectory according to the coordinate difference includes:

arranging the coordinate difference values in a descending order to obtain an ordered list; acquiring a median of the sorted list; calculating a median difference value between each coordinate difference value and the median; judging whether the median difference value exceeds a preset difference threshold value or not; and if the median difference value exceeds a preset difference value threshold value, identifying the coordinate point corresponding to the median difference value as a catastrophe point in the cursor moving track.

It should be understood that, in general, during the process of moving the cursor, the cursor is moved gently, the difference between the coordinate difference values is not large, the coordinate difference values are arranged in the descending order to obtain an ordered list, and the median of the ordered list can reflect the normal value of the coordinate difference value, that is, during the process of moving the cursor, the coordinate difference value is usually close to the median.

It can be understood that the preset difference threshold may be set according to an empirical value, or may be analyzed according to big data, so as to obtain a large number of sample cursor movement tracks, and analyze a coordinate difference in the sample cursor movement tracks, thereby setting the preset difference threshold. Calculating a median difference value between each coordinate difference value and the median, if the coordinate point is a coordinate point generated by normal movement, determining that the corresponding coordinate difference value is smaller than the median difference value between the median, comparing the median difference value with a preset difference threshold, determining the coordinate point corresponding to the median difference value as a mutation point in the cursor movement track if the median difference value exceeds the preset difference threshold, and determining the coordinate point corresponding to the median difference value as a target coordinate point in the cursor movement track if the median difference value does not exceed the preset difference threshold, wherein the target coordinate points can be sequentially connected according to a recorded time sequence to obtain the closed area.

In this embodiment, whether a discontinuity point exists in the cursor movement track is judged, if the discontinuity point exists in the cursor movement track, a coordinate point in the cursor movement track except the discontinuity point is used as a target coordinate point, the target coordinate point is connected to obtain a closed area, and the discontinuity point is removed to improve the accuracy of the closed area, so that the accuracy of interface element frame selection is improved.

In addition, an embodiment of the present invention further provides a storage medium, where an element selection program based on a move operation is stored on the storage medium, and when executed by a processor, the element selection program based on the move operation implements the following steps:

generating a closed area based on the cursor movement track;

Further, the element selection program based on the moving operation, when executed by the processor, further implements the following operations:

acquiring a central point of the current interface element;

judging whether the number of the intersection points is an odd number;

acquiring the element area of the interface element to be confirmed;

judging whether the area ratio is larger than a first preset ratio or not;

Further, the interface elements in the closed area are multiple, and when executed by the processor, the element selection program based on the moving operation further implements the following operations:

judging whether a discontinuity point exists in the cursor moving track;

and connecting the target coordinate points to obtain a closed area.

acquiring a median of the sorted list;

In addition, referring to fig. 8, an embodiment of the present invention further provides an element selection apparatus based on a move operation, where the element selection apparatus based on a move operation includes:

the detection module 10 is configured to detect a cursor movement operation input by a user based on a current interface when an element selection instruction triggered in the current interface by the user is received.

And the determining module 20 is configured to determine a corresponding cursor movement track according to the cursor movement operation.

And a generating module 30, configured to generate a closed region based on the cursor movement trajectory.

And the judging module 40 is configured to judge whether an interface element in the closed area exists in the current interface.

And the selecting module 50 is configured to select the interface element in the closed area if the interface element in the closed area exists in the current interface.

Other embodiments or specific implementation manners of the element selection device based on the move operation according to the present invention may refer to the above method embodiments, and are not described herein again.

The above description is only a preferred embodiment of the present invention, and not intended to limit the scope of the present invention, and all modifications of equivalent structures and equivalent processes, which are made by using the contents of the present specification and the accompanying drawings, or directly or indirectly applied to other related technical fields, are included in the scope of the present invention.

The invention discloses A1 and an element selection method based on a moving operation, wherein the element selection method based on the moving operation comprises the following steps:

generating a closed area based on the cursor movement track;

A2, the method for selecting an element based on a move operation as described in a1, where the determining whether an interface element in the closed area exists in the current interface specifically includes:

acquiring a central point of the current interface element;

A3, the method for selecting an element based on a move operation as described in a2, where the determining whether the current interface element is in the closed area according to the number of intersections specifically includes:

judging whether the number of the intersection points is an odd number;

A4, the method for selecting an element based on a move operation as described in a1, where the determining whether an interface element in the closed area exists in the current interface specifically includes:

acquiring the element area of the interface element to be confirmed;

A5, the method for selecting an element based on a move operation as described in a4, where the determining whether the interface element to be confirmed is in the closed region according to the area ratio specifically includes:

judging whether the area ratio is larger than a first preset ratio or not;

A6, the method for selecting an element based on a moving operation as described in a5, wherein if the area ratio is greater than the first preset ratio, the method for selecting an element based on a moving operation further includes:

A7, the element selection method based on moving operation as described in a6, where the determining, according to the text information to be recognized and the target text information, whether the element to be recognized is in the closed area specifically includes:

A8, the element selection method based on moving operation as described in a7, where the calculating of the text similarity between the text information to be recognized and the target text information specifically includes:

A9, the element selection method based on the movement operation as A6, wherein a plurality of interface elements are in the closed area;

A10, the element selection method based on moving operation as described in a9, where the determining, according to the cluster similarity, whether the element to be recognized is in the closed region specifically includes:

A11, in the element selection method based on the movement operation as described in any one of a1 to a10, the generating a closed region based on the cursor movement trajectory specifically includes:

judging whether a discontinuity point exists in the cursor moving track;

and connecting the target coordinate points to obtain a closed area.

A12, the method for selecting an element based on a moving operation as described in a11, where the determining whether there is a discontinuity in the cursor movement trajectory specifically includes:

A13, the element selection method based on moving operation as described in a12, where the determining whether there is a discontinuity in the cursor movement trajectory according to the coordinate difference specifically includes:

acquiring a median of the sorted list;

The invention also discloses B14 and an element selection device based on the mobile operation, wherein the element selection device based on the mobile operation comprises: a memory, a processor and a move operation based element selection program stored on the memory and executable on the processor, the move operation based element selection program when executed by the processor implementing the steps of the move operation based element selection method as claimed in any one of a 1-a 13.

The invention also discloses C15, a storage medium, on which an element selection program based on move operation is stored, which when executed by a processor implements the steps of the element selection method based on move operation as recited in any one of claims A1 to A13.

The invention also discloses D16 and an element selection device based on the moving operation, wherein the element selection device based on the moving operation comprises:

D17, the element selection device based on move operation as recited in D16, wherein the determining module is further configured to traverse the interface element in the current interface, and take the traversed interface element as the current interface element; acquiring a central point of the current interface element; generating a target ray along a preset direction by taking the central point as a starting point; acquiring the number of intersection points of the target ray and the edge of the closed area; and judging whether the current interface element is in the closed area or not according to the number of the intersection points.

D18, the device for selecting elements based on move operation as described in D16, wherein the determining module is further configured to determine whether the number of the intersection points is an odd number; and if the number of the intersection points is an odd number, determining that the current interface element is in the closed area.

D19, the element selection device based on move operation as described in D16, wherein the determining module is further configured to traverse the interface element in the current interface, and use the traversed interface element as the interface element to be confirmed; acquiring the element area of the interface element to be confirmed; calculating the area ratio of the interface element to be confirmed in the closed region according to the element area; and judging whether the interface element to be confirmed is in the closed area or not according to the area ratio.

C20, the element selecting device based on move operation as described in C16, wherein the generating module is further configured to determine whether there is a discontinuity in the cursor movement trajectory; if a discontinuity point exists in the cursor moving track, taking coordinate points except the discontinuity point in the cursor moving track as target coordinate points; and connecting the target coordinate points to obtain a closed area.

Claims

1. An element selection method based on a moving operation is characterized by comprising the following steps:

generating a closed area based on the cursor movement track;

2. The element selection method based on the move operation according to claim 1, wherein the determining whether the interface element in the closed area exists in the current interface specifically includes:

acquiring a central point of the current interface element;

3. The element selection method based on the move operation according to claim 1, wherein the determining whether the interface element in the closed area exists in the current interface specifically includes:

acquiring the element area of the interface element to be confirmed;

4. The element selection method based on the move operation according to claim 3, wherein the determining whether the interface element to be confirmed is located in the closed region according to the area ratio specifically includes:

judging whether the area ratio is larger than a first preset ratio or not;

5. The method as claimed in claim 4, wherein after determining that the interface element to be confirmed is located in the closed region if the area ratio is greater than the first preset ratio, the method further comprises:

6. The element selection method based on the moving operation as claimed in claim 5, wherein the number of the interface elements in the closed area is multiple;

7. The element selection method based on the movement operation as claimed in any one of claims 1 to 6, wherein the generating of the closed region based on the cursor movement trajectory specifically includes:

judging whether a discontinuity point exists in the cursor moving track;

and connecting the target coordinate points to obtain a closed area.

8. An element selection device based on a mobile operation, the element selection device based on the mobile operation comprising: memory, a processor and a move operation based element selection program stored on the memory and executable on the processor, the move operation based element selection program when executed by the processor implementing the steps of the move operation based element selection method according to any one of claims 1 to 7.

9. A storage medium, characterized in that the storage medium stores thereon a move operation based element selection program, which when executed by a processor implements the steps of the move operation based element selection method according to any one of claims 1 to 7.

10. An element selection device based on mobile operation, characterized in that the element selection device based on mobile operation comprises: