WO2022246834A1 - 一种交互方法、电子设备及计算机存储介质 - Google Patents

一种交互方法、电子设备及计算机存储介质 Download PDF

Info

Publication number
WO2022246834A1
WO2022246834A1 PCT/CN2021/096925 CN2021096925W WO2022246834A1 WO 2022246834 A1 WO2022246834 A1 WO 2022246834A1 CN 2021096925 W CN2021096925 W CN 2021096925W WO 2022246834 A1 WO2022246834 A1 WO 2022246834A1
Authority
WO
WIPO (PCT)
Prior art keywords
candidate
user
candidate object
selection icon
selection
Prior art date
Application number
PCT/CN2021/096925
Other languages
English (en)
French (fr)
Inventor
胡风硕
黄光伟
张阳阳
Original Assignee
京东方科技集团股份有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 京东方科技集团股份有限公司 filed Critical 京东方科技集团股份有限公司
Priority to US17/765,832 priority Critical patent/US20240046682A1/en
Priority to PCT/CN2021/096925 priority patent/WO2022246834A1/zh
Priority to CN202180001356.5A priority patent/CN115701298A/zh
Publication of WO2022246834A1 publication Critical patent/WO2022246834A1/zh

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V30/00—Character recognition; Recognising digital ink; Document-oriented image-based pattern recognition
    • G06V30/10—Character recognition
    • G06V30/32—Digital ink
    • G06V30/36—Matching; Classification
    • G06V30/387—Matching; Classification using human interaction, e.g. selection of the best displayed recognition candidate
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/048—Interaction techniques based on graphical user interfaces [GUI]
    • G06F3/0481—Interaction techniques based on graphical user interfaces [GUI] based on specific properties of the displayed interaction object or a metaphor-based environment, e.g. interaction with desktop elements like windows or icons, or assisted by a cursor's changing behaviour or appearance
    • G06F3/0482—Interaction with lists of selectable items, e.g. menus
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/048—Interaction techniques based on graphical user interfaces [GUI]
    • G06F3/0484—Interaction techniques based on graphical user interfaces [GUI] for the control of specific functions or operations, e.g. selecting or manipulating an object, an image or a displayed text element, setting a parameter value or selecting a range
    • G06F3/04842—Selection of displayed objects or displayed text elements
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/048—Interaction techniques based on graphical user interfaces [GUI]
    • G06F3/0484—Interaction techniques based on graphical user interfaces [GUI] for the control of specific functions or operations, e.g. selecting or manipulating an object, an image or a displayed text element, setting a parameter value or selecting a range
    • G06F3/04845—Interaction techniques based on graphical user interfaces [GUI] for the control of specific functions or operations, e.g. selecting or manipulating an object, an image or a displayed text element, setting a parameter value or selecting a range for image manipulation, e.g. dragging, rotation, expansion or change of colour
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/048—Interaction techniques based on graphical user interfaces [GUI]
    • G06F3/0487—Interaction techniques based on graphical user interfaces [GUI] using specific features provided by the input device, e.g. functions controlled by the rotation of a mouse with dual sensing arrangements, or of the nature of the input device, e.g. tap gestures based on pressure sensed by a digitiser
    • G06F3/0488—Interaction techniques based on graphical user interfaces [GUI] using specific features provided by the input device, e.g. functions controlled by the rotation of a mouse with dual sensing arrangements, or of the nature of the input device, e.g. tap gestures based on pressure sensed by a digitiser using a touch-screen or digitiser, e.g. input of commands through traced gestures
    • G06F3/04883—Interaction techniques based on graphical user interfaces [GUI] using specific features provided by the input device, e.g. functions controlled by the rotation of a mouse with dual sensing arrangements, or of the nature of the input device, e.g. tap gestures based on pressure sensed by a digitiser using a touch-screen or digitiser, e.g. input of commands through traced gestures for inputting data by handwriting, e.g. gesture or text
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V30/00—Character recognition; Recognising digital ink; Document-oriented image-based pattern recognition
    • G06V30/10—Character recognition
    • G06V30/32—Digital ink
    • G06V30/333—Preprocessing; Feature extraction
    • G06V30/347—Sampling; Contour coding; Stroke extraction
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V30/00—Character recognition; Recognising digital ink; Document-oriented image-based pattern recognition
    • G06V30/10—Character recognition
    • G06V30/32—Digital ink
    • G06V30/36—Matching; Classification
    • G06V30/373—Matching; Classification using a special pattern or subpattern alphabet
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/048—Interaction techniques based on graphical user interfaces [GUI]
    • G06F3/0481—Interaction techniques based on graphical user interfaces [GUI] based on specific properties of the displayed interaction object or a metaphor-based environment, e.g. interaction with desktop elements like windows or icons, or assisted by a cursor's changing behaviour or appearance
    • G06F3/04817—Interaction techniques based on graphical user interfaces [GUI] based on specific properties of the displayed interaction object or a metaphor-based environment, e.g. interaction with desktop elements like windows or icons, or assisted by a cursor's changing behaviour or appearance using icons

Definitions

  • This article involves but is not limited to the field of computer technology, especially an interactive method, electronic equipment and computer storage media.
  • Electronic devices for paperless office and learning have gradually replaced traditional methods and become the mainstream.
  • Electronic devices such as electronic whiteboards and mobile phones/tablets with touch screens provide users with the ability to use fingers or touch directly on the screen.
  • Pen for writing and drawing functions are used.
  • a series of operations such as search, copy, paste or insert are required to complete, which is inconvenient and inefficient.
  • Embodiments of the present application provide an interaction method, electronic equipment, and a computer storage medium.
  • this embodiment of the application provides an interaction method, including:
  • the track point set includes at least one track point
  • candidate objects according to the set of track points, where the candidate objects include: a first candidate object and a second candidate object;
  • the first candidate object In response to the user's selection operation on the first selection icon, the first candidate object is displayed or the operation corresponding to the first candidate object is executed, and in response to the user's selection operation on the second selection icon, the second candidate object is displayed or the second selection icon is executed.
  • the operation corresponding to the two candidates In response to the user's selection operation on the first selection icon, the first candidate object is displayed or the operation corresponding to the first candidate object is executed, and in response to the user's selection operation on the second selection icon, the second candidate object is displayed or the second selection icon is executed. The operation corresponding to the two candidates.
  • the first candidate object includes a first candidate graphic
  • the second candidate object includes a second candidate graphic
  • the displaying the first candidate object in response to the user's selection operation on the first selection icon, and displaying the second candidate object in response to the user's selection operation on the second selection icon includes:
  • the determining the candidate object according to the track point set includes:
  • the first characteristic information is different from the second characteristic information.
  • the first characteristic information and the second characteristic information respectively include one or more of the following:
  • the first feature information or the second feature information includes one or more character features corresponding to the track;
  • the extraction of the first feature information or the second feature information from the track formed by the set of track points includes:
  • the acquisition of the first/second candidate graphics that successfully match the first/second feature information includes:
  • the label carried by the candidate graphic is used to represent the content of the candidate graphic.
  • the first feature information or the second feature information includes the original line features of the graph formed by the track;
  • the acquisition of the first/second candidate graphics that successfully match the first/second feature information includes:
  • the first characteristic information or the second characteristic information includes the original line characteristics of the graph formed by the trajectory
  • the acquisition of the first/second candidate graphics that successfully match the first/second feature information includes:
  • the subgraph contained in the candidate graph is stored corresponding to the candidate graph.
  • the first feature information or the second feature information includes contour line features of the figure formed by the track;
  • the acquisition of the first/second candidate graphics that successfully match the first/second feature information includes:
  • the first candidate object includes a first candidate instruction
  • the second candidate object includes a second candidate instruction
  • the determining the candidate object according to the track point set includes:
  • the trajectory formed by the set of trajectory points is identified as a standard trajectory, and different candidate instructions are obtained according to the standard trajectory as the first candidate instruction and the second candidate instruction respectively.
  • the first candidate object is a third candidate graphic
  • the second candidate object is a third candidate instruction
  • Displaying the first candidate object in response to the user's selection operation on the first selection icon, and executing the corresponding to the second selection icon in response to the user's selection operation on the second selection icon operations including:
  • the determining the candidate object according to the track point set includes:
  • a trajectory formed by the set of trajectory points is identified as a standard trajectory, and the third candidate instruction is acquired according to the standard trajectory.
  • the determining the candidate object according to the track point set further includes:
  • the trajectory point set In response to failure to identify the trajectory formed by the trajectory point set as a standard trajectory, obtain fourth feature information according to the trajectory formed by the trajectory point set, and acquire a fourth candidate graphic that successfully matches the fourth feature information as the the second candidate object; the fourth characteristic information is different from the third characteristic information.
  • the interaction method is applied to writing software; the acquisition of the track point set input by the user includes:
  • the displaying a first selection icon for representing the first candidate object and a second selection icon for representing the second candidate object includes:
  • displaying the first candidate object or performing an operation corresponding to the first candidate object in response to the user's selection operation on the second selection icon
  • the operation, displaying the second candidate object or performing an operation corresponding to the second candidate object includes:
  • the first candidate object is displayed in the input interface, or the first candidate object is executed in the writing software. corresponding operation;
  • the displaying the first selection icon used to represent the first candidate object and the second selection icon used to represent the second candidate object includes:
  • the first candidate object is a candidate graphic, and the thumbnail of the first candidate object is displayed as the first selection icon; or, the first candidate object is a candidate instruction, and the description information of the first candidate object is displayed as the first selection icon;
  • the second candidate object is a candidate graphic, and the thumbnail of the second candidate object is displayed as the first selection icon; or, the second candidate object is a candidate instruction, and the description information of the second candidate object is displayed as the second selection icon.
  • the display after displaying the first selection icon for representing the first candidate object and the second selection icon for representing the second candidate object, the display further includes:
  • the displayed first selection icon is correspondingly updated; in response to the update of the second candidate object, the displayed second selection icon is correspondingly updated.
  • the acquisition of the track point set input by the user includes:
  • a preset control is displayed; wherein, the detection of the pause of the user's input operation refers to: the time length from the last receipt of the sampling result for the input operation reaches or exceeds the preset duration threshold;
  • an embodiment of the present application provides an electronic device, including: a memory and a processor; the memory is configured to store an interactive program; the processor is configured to read and execute the interactive program, and perform the above interaction method.
  • an embodiment of the present application provides a computer storage medium configured to store an interactive program; when the interactive program is executed, the above-mentioned interactive method is performed.
  • the embodiment of the present application can automatically, efficiently and quickly provide a variety of candidate objects related to the trajectory for the user to choose according to the trajectory formed by the user input, and can display corresponding graphics or perform corresponding operations according to the user's selection, which greatly improves the user experience. user experience.
  • FIG. 1 is a schematic flow diagram of an interaction method in an embodiment of the present application
  • FIG. 2 is a schematic diagram of an electronic device according to an embodiment of the present application.
  • FIG. 3 is a schematic diagram of the implementation process of Example 1;
  • Example 4 is a schematic diagram of a trajectory formed by a set of trajectory points in Example 1;
  • FIG. 5 is a schematic diagram showing selection icons in Example 1.
  • Example 6 is a schematic diagram of the process of calculating the similarity of contour lines in Example 1;
  • Fig. 7 is a schematic diagram when the graph formed by the trajectory in example 1 is similar to the candidate graph subgraph;
  • Fig. 8 is a schematic diagram of candidate graphs composed of subgraphs in Example 1;
  • FIG. 9 is a schematic diagram showing selection icons in Example 2.
  • the terms “installed”, “connected” and “connected” should be interpreted in a broad sense unless otherwise expressly specified and limited. For example, it may be a fixed connection, or a detachable connection, or an integral connection; it may be a mechanical connection, or an electrical connection; it may be a direct connection, or an indirect connection through an intermediate piece, or an internal communication between two components. Those of ordinary skill in the art can understand the meanings of the above terms herein according to the situation.
  • electrically connected includes the case where constituent elements are connected together through an element having some kind of electrical effect.
  • the "element having some kind of electrical function” is not particularly limited as long as it can transmit electrical signals between connected components.
  • Examples of “elements having some kind of electrical function” include not only electrodes and wirings but also switching elements such as transistors, resistors, inductors, capacitors, other elements having one or more functions, and the like.
  • parallel refers to a state in which the angle formed by two straight lines is -10° to 10°, and thus may include a state in which the angle is -5° to 5°.
  • perpendicular refers to a state in which the angle formed by two straight lines is 80° to 100°, and therefore, an angle of 85° to 95° may be included.
  • An embodiment of the present application provides an interactive method, as shown in Figure 1, including steps S110-S140:
  • S120 Determine candidate objects according to the track point set, where the candidate objects include: a first candidate object and a second candidate object;
  • the interaction method of the embodiment of the present application can automatically, efficiently and quickly provide various candidate objects related to the trajectory for the user to choose according to the trajectory formed by the user input, and can display corresponding graphics or perform corresponding operations according to the user's selection, Greatly improved user experience.
  • the above interaction method can be applied to electronic devices including touch screens such as electronic whiteboards, tablet computers, and smart phones;
  • the touch screen can include a touch panel, a sampling unit, a display panel, and a display driver unit; wherein, the touch panel is a transparent board, which is arranged on the upper surface of the display panel;
  • the user uses a finger or a stylus to perform an input operation, and the sampling unit samples the user's touch, and can obtain the finger or touch and touch
  • the real-time coordinates of the track point formed by the touch position of the control board, and the real-time coordinates are sent to the display drive unit, and the track point can be displayed on the display panel corresponding to the position of the real-time coordinates;
  • the user uses a finger or a stylus to touch Sliding continuously on the board to write or draw, through the continuous sampling and sending of the sampling unit,
  • the display panel can display the trajectory formed by the set of track points corresponding to multiple real-time coordinates, and the user's writing or drawing on
  • the above-mentioned interaction method can be applied to a system of a tablet plus a computer or to a computer equipped with a touchpad, and the input operation of the user is received through the tablet or the touchpad, and the input operation forms Track points can be displayed on a computer screen.
  • the user may use a mouse or a keyboard to manipulate the movement of the cursor instead of inputting track points via a touchpad.
  • a set of track points input by the user in a preset input area may be obtained; or may be Obtain a set of track points input by the user during a certain period of time (for example, between two user operation pauses).
  • step S120 one or more first candidate objects and one or more second candidate objects may be determined.
  • the determined candidate objects may further include a third candidate object or more candidate objects.
  • first candidate objects and multiple second candidate objects there are multiple first selection icons and second selection icons, which are respectively the same as the first candidate object and the second selection icon.
  • candidates correspond one-to-one.
  • the category of the candidate object determined in step S120 is a graphic or an instruction corresponding to an operation; the categories of the first candidate object and the second candidate object may both be graphics, or both be instructions, or one may be Graphics and another for instructions.
  • step S140 after the category of the candidate object is a graphic and the corresponding first or second selection icon is clicked, the corresponding graphic can be displayed; the category of the candidate object is an instruction and the corresponding first or second selection icon is clicked After that, the operation corresponding to the instruction can be performed.
  • the first selection icon and the second selection icon displayed in step S130 are for the user to click to select the corresponding first candidate object and second candidate object.
  • the forms adopted by the first and second selection icons can be determined according to the categories of the first and second candidate objects respectively; It is a control that uses the description information of the command as the icon to display the content.
  • the descriptive information may be in the form of graphics or text. For example, if the first candidate object is a "undo" instruction, the first selection icon may be displayed as a counterclockwise arrow; Select the icon to display "del".
  • the first selection icon and the second selection icon may be displayed at the same time, or the first selection icon and the second selection icon may be displayed in sequence; the first and second selection icons may be displayed on the in different regions.
  • the order and area displayed can be in a default manner, or can be set by the user.
  • step S140 the user is only allowed to select one selection icon, and in response to the user's selection operation (such as but not limited to clicking or long-pressing) on a selection icon (the first selection icon or the second selection icon) etc.), the display of the first selection icon and the second selection icon may be stopped, and the candidate object corresponding to the selection icon selected by the user may be displayed or a corresponding operation may be performed according to the corresponding candidate object. That is, the user can select a first candidate object or a second candidate object by selecting the selection icon; according to the category of the candidate object selected by the user, the candidate object is displayed correspondingly, or an operation corresponding to the candidate object is performed.
  • the user's selection operation such as but not limited to clicking or long-pressing
  • the user may be allowed to select two or more selection icons, and the two or more selection icons may be all first/second selection icons, or may be part of the first selection icon icon, part of which is the second choice icon.
  • the user presses the "ctrl" key and selects a first selection icon and a second selection icon.
  • the first and second candidate objects can be displayed at the same time, or the first and second object correspondences can be performed sequentially. , or display one candidate and perform the corresponding action on another candidate.
  • step S140 when displaying the candidate object, it may be displayed at the current position of the cursor, or the candidate object may be used instead of the track formed by the set of track points for display.
  • the first candidate object includes a first candidate graphic
  • the second candidate object includes a second candidate graphic
  • displaying the first candidate object In response to the user's selection operation on the first selection icon, displaying the first candidate object, and in response to the user's selection operation on the second selection icon, displaying the second candidate object, including:
  • Determine candidate objects based on the set of track points including:
  • the first characteristic information is different from the second characteristic information.
  • the first and second candidate graphics may be, but not limited to, graphics in a preset graphics database; there may be one or more graphics databases, which may be set locally, on a server, in the cloud, or the like.
  • the first characteristic information and the second characteristic information respectively include one or more of the following:
  • the original line feature refers to the line itself formed by the track points in the track point set and used to form the graph.
  • the first candidate object and the second candidate object may respectively be candidate graphics that are successfully matched with different types of feature information.
  • the first candidate object is the second candidate graphic that successfully matches the character feature
  • the second candidate object is the second candidate graphic that successfully matches the original line feature.
  • the first feature information or the second feature information includes one or more character features corresponding to the trajectory
  • Acquiring the first/second characteristic information according to the track formed by the set of track points includes:
  • Obtaining the first/second candidate graphics that successfully match the first/second feature information includes:
  • one or more character features corresponding to the track may be extracted, but not limited to, through OCR (Optical Character Recognition, optical character recognition).
  • the character features corresponding to the trajectory can be acquired in other ways, such as extracting stroke features and stroke order features from the trajectory, and comparing them with corresponding features of different characters in the font library, so as to determine the character features corresponding to the trajectory.
  • a candidate graphic may have one or more labels, and the labels may include one character or multiple characters.
  • tags with the same content are searched in the graph database according to the recognized character features; if any tag of a candidate graph is the same as at least one recognized character feature, the candidate graph can be considered to be matched successfully.
  • the standard for successful matching can be set by itself. It can be set that the label must be the same as all character features to be considered a successful match. For example, if the acquired character feature and label are both "umbrella", the match is successful; or It can be set that the label is the same as some character features even if the match is successful. For example, if the acquired character feature is "umbrella" and the label is "umbrella”, it can be judged as a successful match; another example is the acquired character feature is "cat" and the label is "cat”. , can be judged as a successful match.
  • the candidate graphics whose labels and acquired character features are exactly the same can be preferentially selected; Select candidate graphics corresponding to labels with high correlation value or high priority.
  • the correlation value can be used to indicate the degree of correlation between the candidate graphic and the label.
  • the correlation value between the umbrella graphic and the label "umbrella” is 100%; the correlation value between the mushroom graphic and the label "umbrella” 20%. If multiple candidate graphics are successfully matched according to the character feature "umbrella", the umbrella graphics can be selected first.
  • the first characteristic information or the second characteristic information includes the original line characteristics of the graph formed by the trajectory
  • Obtaining the first/second candidate graphics that successfully match the first/second feature information includes:
  • a candidate graphic whose first degree of similarity satisfies the first preset condition is selected as the first/second candidate graphic.
  • the first preset condition may be that the first similarity of the candidate graphics is one of the highest N first similarities; select the candidate graphics whose first similarity satisfies the first preset condition Including: selecting the first N candidate graphics with the highest similarity.
  • N is a preset positive integer, which may be the number of candidate graphics to be displayed for the user to choose.
  • the first preset condition may be that the first similarity exceeds or reaches a preset first similarity threshold.
  • the first similarity between a candidate graphic and the original line feature can be calculated as follows:
  • the Hungarian pairing algorithm is used for pairing; if the number of sampling points in one sampling point set for pairing is less than that of the other sampling point set, the obtained sampling point pair The number is equal to the number of sampling points in the set of sampling points with a small number of sampling points;
  • the distance of each pair of sampling points is calculated separately, and the first similarity between the candidate graphic and the original line feature is determined according to the sum of the distances of all pairs of sampling points.
  • the first interval can be set by itself, for example, if relatively intensive sampling is required, the first interval can be set smaller.
  • the distance between the calculated sampling point pairs can be, but not limited to, Euclidean distance; the smaller the distance, the greater the similarity.
  • the calculation of the first similarity is not limited to the above method, and other methods can be used to determine the first similarity between the original line feature and the candidate graphics, such as calculating the cosine similarity, or calculating the similarity through fingerprint information Wait.
  • the first characteristic information or the second characteristic information includes the original line characteristics of the graph formed by the trajectory
  • Obtaining the first/second candidate graphics that successfully match the first/second feature information includes:
  • the subgraph contained in the candidate graph is stored corresponding to the candidate graph.
  • the largest similarity may be selected as the similarity with the candidate graphic.
  • the second preset condition may be that the second similarity of the candidate graphics is one of the highest N second similarities; select the candidate graphics whose second similarity satisfies the second preset condition Including: selecting N candidate graphics with the second highest similarity.
  • N is a preset positive integer, which may be the number of candidate graphics to be displayed for the user to choose.
  • the second preset condition may be that the second similarity exceeds or reaches a preset second similarity threshold.
  • the implementation details of calculating the similarity between the original line feature and the candidate graph or sub-graph may refer to the method of calculating the first similarity.
  • candidate figures similar to figures formed by parts and tracks can be selected, that is, candidate figures similar to figures formed by "parts" and tracks can be selected.
  • the subgraph can be stored together with the candidate graph composed of the subgraph, or the subgraph can be split from the candidate graph and stored together with the candidate graph.
  • not every candidate graph has a subgraph, and only some of the candidate graphs may have subgraphs.
  • the sum of the subgraphs contained in the candidate graph can be smaller than the candidate graph; the same part in the candidate graph can be used as the same subgraph; for example, if the candidate graph is a house, which includes multiple identical windows, then each window can be Use the same "subplot for windows" without the rest of the "house graphic" as a subplot.
  • the first feature information or the second feature information includes the outline feature of the graph formed by the trajectory
  • Obtaining the first/second candidate graphics that successfully match the first/second feature information includes:
  • a candidate graphic whose third degree of similarity satisfies the third preset condition is selected as the first/second candidate graphic.
  • the third preset condition may be that the third similarity of the candidate graphics is one of the highest N third similarities; select the candidate graphics whose third similarity satisfies the third preset condition Including: selecting the N candidate graphics with the third highest similarity.
  • N is a preset positive integer, which may be the number of candidate graphics to be displayed for the user to choose.
  • the third preset condition may be that the third similarity exceeds or reaches a preset third similarity threshold.
  • the process of calculating the third similarity between a candidate figure and the outline feature may include:
  • the Hungarian pairing algorithm is used for pairing; if the number of sampling points in one sampling point set for pairing is less than that of the other sampling point set, the obtained sampling point pair The number is equal to the number of sampling points in the set of sampling points with a small number of sampling points;
  • the distance of each pair of sampling points is calculated separately, and the third similarity between the contour line feature and the contour line of the candidate figure is determined according to the sum of the distances of all the sampling point pairs.
  • the second spacing can be set by itself, for example, if relatively intensive sampling is required, the second spacing can be set smaller.
  • the calculated distance between the sampling point pairs may be, but not limited to, the Euclidean distance.
  • the calculation of the third similarity is not limited to the above-mentioned method, and any method of determining the similarity between curves can be used to determine the third similarity, for example, the discrete Frescher distance and the Hausdorff distance can be used to determine the third similarity. , Pearson correlation coefficient and other methods to determine the third similarity.
  • the first feature information or the second feature information includes the figure formed by the trajectory or the outline of the figure, and obtaining the first/second feature information may include:
  • the graphics formed by the trajectory are scaled according to multiple preset ratios
  • the degree is calculated according to the scaled graphics of different scales; for different scales, different similarities will be calculated, and the maximum value will be used as the final similarity.
  • first scale IMG 1 according to the preset scaling ratios 0.5, 1, and 2 to obtain IMG 1-1 , IMG 1-2 , and IMG 1-3 .
  • IMG 1-1 and IMG 2 are densely sampled at equal intervals, and then Hungarian pairing is performed on the two sampling point sets obtained, and the distance of each sampling point pair is calculated to obtain the sum of distances L 1-1 ; similarly, for IMG 1-2 and IMG 1-3 obtain the sum of distances L 1-2 and L 1-3 respectively, and determine IMG 1 and IMG 2 according to the minimum value of the three distance sums (corresponding to the maximum similarity) as the final distance The final first similarity of .
  • This embodiment can alleviate the influence of the graph size on distance (or similarity) calculation.
  • the preset ratio can be set to 0.1, 0.2, . . . , 9, 10.
  • the first candidate object includes a first candidate instruction
  • the second candidate object includes a second candidate instruction
  • the determining the candidate object according to the track point set includes:
  • the trajectory formed by the set of trajectory points is identified as a standard trajectory, and different candidate instructions are obtained according to the standard trajectory as the first candidate instruction and the second candidate instruction respectively.
  • the trajectories can be identified as different first standard trajectories and second standard trajectories, the candidate instructions corresponding to the first standard trajectories are acquired as the first candidate instructions, and the candidate instructions corresponding to the second standard trajectories are obtained.
  • Candidate instructions for the second standard instruction can be identified as different first standard trajectories and second standard trajectories, the candidate instructions corresponding to the first standard trajectories are acquired as the first candidate instructions, and the candidate instructions corresponding to the second standard trajectories are obtained.
  • the trajectory may be identified as a standard trajectory, and multiple candidate instructions corresponding to the standard trajectory are obtained as first and second candidate instructions respectively.
  • a variety of standard trajectories are identified, they can be sorted according to the similarity between the multiple standard trajectories and the trajectories formed by the set of trajectory points, and select the standard trajectories ranked first to obtain the first and second trajectories.
  • Candidate instruction if a variety of standard trajectories are identified, they can be sorted according to the similarity between the multiple standard trajectories and the trajectories formed by the set of trajectory points, and select the standard trajectories ranked first to obtain the first and second trajectories.
  • the standard track is a track with a standardized form, because the degree of personalization of the user input track is very high, even if the intention is to input the same track (for example, all want to input track "Z"), the track actually input by different users There will be differences in form, and even the trajectories input twice by the same user will have differences; in order to adapt to this differentiated input, it is necessary to first identify the trajectories input by the user as standard trajectories and then obtain candidate instructions correspondingly, that is, first A "normalized" process.
  • the process of identifying the trajectory as a standard trajectory can be similar to the method of recognizing the characters input by the user in handwriting input: extracting the characteristics of the trajectory and comparing it with the characteristics of the standard trajectory to identify the trajectory as one of the preset standard trajectories . If the feature of the trajectory input by the user cannot be identified as any standard trajectory, it means that the trajectory input by the user does not have a corresponding candidate instruction, and the candidate object whose category is the candidate instruction cannot be obtained according to the trajectory input by the user.
  • multiple standard tracks can be preset, and one or more candidate instructions corresponding to each standard track can be set;
  • the style of the standard track can be any one or more of the following: graphics, symbols, characters.
  • the trajectory input by the user may be recognized as more than one standard trajectory. For example, if the user inputs a slanted "Z", it can be recognized as a standard trajectory "Z” or as a standard trajectory "N".
  • the candidate instructions corresponding to "Z” and “N” are acquired, the candidate instruction corresponding to "Z” is used as a first candidate, and the candidate instruction corresponding to "N” is used as a second candidate.
  • the first and second candidate objects may be different candidate instructions corresponding to a standard trajectory.
  • the standard track "Z" corresponds to two commands: delete the track and clear the dialog box. These two instructions can be used as the first and second candidates respectively.
  • the first candidate object is a third candidate graphic
  • the second candidate object is a third candidate instruction
  • Determining candidate objects based on the set of track points includes:
  • Determine candidate objects based on the set of track points including:
  • a trajectory formed by the set of trajectory points is identified as a standard trajectory, and a third candidate instruction is obtained according to the standard trajectory.
  • determining the candidate object according to the track point set may also include:
  • the fourth feature information is obtained according to the trajectory formed by the trajectory point set, and the fourth candidate graphic that successfully matches the fourth feature information is obtained as the second candidate object; the fourth The characteristic information is different from the third characteristic information.
  • one of the first and second candidate objects is a candidate graphic, and the other is a candidate command; but if the track input by the user cannot be recognized as a standard track, that is, the candidate command cannot be obtained, then the second can be adaptively The second candidate object is changed to a candidate figure that is successfully matched according to the fourth characteristic information different from the third characteristic information.
  • the above-mentioned interaction method can be applied to writing software; obtaining the set of track points input by the user includes:
  • Displaying a first selection icon representing a first candidate object and a second selection icon representing a second candidate object includes:
  • the first candidate object In response to the user's selection operation on the first selection icon, the first candidate object is displayed or the operation corresponding to the first candidate object is executed, and in response to the user's selection operation on the second selection icon, the second candidate object is displayed or the second selection icon is executed.
  • the operations corresponding to the two candidates include:
  • the second candidate object is displayed in the input interface, or an operation corresponding to the second candidate object is executed in the writing software.
  • displaying a first selection icon for representing a first candidate object and a second selection icon for representing a second candidate object includes:
  • the first candidate object is a candidate graphic, and the thumbnail of the first candidate object is displayed as the first selection icon; or, the first candidate object is a candidate instruction, and the description information corresponding to the first candidate object is displayed as the first selection icon ;
  • the second candidate object is a candidate graphic, and the thumbnail of the second candidate object is displayed as the first selection icon; or, the second candidate object is a candidate instruction, and the description information corresponding to the second candidate object is displayed as the second selection icon .
  • the method further includes:
  • the displayed first selection icon is correspondingly updated; in response to the update of the second candidate object, the displayed second selection icon is correspondingly updated.
  • obtaining a track point set input by a user includes:
  • a preset control is displayed; wherein, the detection of the pause of the user's input operation refers to: the time length from the last receipt of the sampling result for the input operation reaches or exceeds the preset duration threshold;
  • a set of track points input in a specific time period (between two operation pauses) is acquired.
  • a set of track points input in a predetermined input area may be acquired.
  • the embodiment of the present application also provides an electronic device, as shown in FIG. 2 , including: a memory 21 and a processor 22;
  • the memory 21 is configured to save the interactive program
  • the processor 22 is configured to read and execute the interactive program, and perform the interactive method in any of the above-mentioned embodiments.
  • the electronic device may further include:
  • a touch panel configured to receive user input operations
  • the real-time coordinates of the touch position are sampled by the sampling unit and sent to the display drive unit, which can be displayed on the display panel according to the user's input operation on the touch panel; among them, the display drive The unit can also display all or part of the matching candidate graphics on the display panel according to the control of the processor.
  • An embodiment of the present application also provides a computer storage medium configured to store an interactive program; when the interactive program is executed, the interactive method in any of the above embodiments is performed.
  • the candidate objects include: first, second, and third candidate objects, all of which are candidate graphics.
  • the processor detects that the user pauses, and recognizes the set of track points formed by the user's writing or drawing to obtain candidate graphics, which are returned to the front end as a recommendation result and displayed in the form of selected icons for the user choose.
  • the way to obtain candidate graphs in this example includes: obtaining three types of feature information for the trajectory formed by the trajectory point set, and matching them with the candidate graphs in the graph database; for:
  • Character matching matches according to the character features corresponding to the trajectory
  • contour line matching this method performs matching according to the contour line characteristics of the graphics formed by the trajectory
  • graphic matching this method performs matching according to the original line characteristics of the graphics formed by the trajectory.
  • the sampling unit may send the sampled real-time coordinates of the user's touch on the electronic whiteboard to the processor; the processor detects that the user pauses and triggers the start of the interactive operation.
  • the detection of user pause may refer to: the time length since the last time the real-time coordinates are received from the sampling unit exceeds a preset time length threshold.
  • the processor displays a dotted frame with an interactive trigger point on the display panel to send an interactive request to the user by controlling the display driving unit.
  • the dotted box frames the trajectory between the last detected pause and the detected pause, as shown in Figure 4; that is, the trajectory in the dotted box is the trajectory formed by the set of acquired trajectory points, including the trajectory from the last time Multiple track points including the first track point received after the pause is detected and the last track point received before the pause is detected this time.
  • the interaction trigger point is set as a dot with a certain area in the lower right corner of the dotted line box, and the trigger method is to click on the dot; during implementation, the interaction trigger point can be in other forms and located at other positions to trigger There can be various methods; for example, the interaction trigger point can be set as an icon in the menu bar, or can be set on the frame line of the dotted line box or in the area inside the dotted line box; Trigger interaction by means of double-click or long-press.
  • the processor determines the user's operation according to one or more actual coordinates sampled after the sampling unit, and performs corresponding processing, including the following five situations:
  • the processor does not process, records the moment of this pause and continues to monitor until the next pause.
  • the processor does not process, records the moment of this pause and continues to monitor until the next pause.
  • the processor changes the coordinates of the track in the dotted line box according to the user's dragging, and sends the changed coordinates to the display driver unit, so that the effect displayed on the display panel is that the track in the dotted line box follows the user's dragging. move.
  • the processor changes the distance between adjacent track points in the track according to the user's dragging, keeps the coordinates of one track point in the track unchanged, and the coordinates of other track points change with the change of distance; the processor will change the The coordinates are sent to the display drive unit, so that the effect displayed on the display panel is that the track in the dotted frame scales as the user drags.
  • the processor obtains three kinds of characteristic information of the trajectory in the dotted line box: character characteristics, original line characteristics and contour line characteristics of the formed graphics, connects to the local or remote graphics database, respectively according to each type of characteristic information obtained, and
  • the candidate graphics contained in the graphics database are matched to obtain candidate objects corresponding to each type of feature information, that is, candidate graphics that are successfully matched according to the type of feature information.
  • the processor returns all or part of the candidate graphics that are successfully matched to the front end as candidate objects for display.
  • the first candidate object is a candidate graphic that is successfully matched according to the character features corresponding to the trajectory
  • the second candidate object is a candidate graphic that is successfully matched according to the outline features of the graphic formed by the trajectory
  • the third candidate object is a candidate graphic that is successfully matched according to the contour line features of the graphic formed by the trajectory.
  • the original line features of the formed graphics match the successful candidate graphics.
  • first selection icons As shown in FIG. 5 , four first selection icons, three second selection icons, and three third selection icons are displayed to the user, which are thumbnails corresponding to the first, second, and third candidate objects, respectively. In this example, the character corresponding to the trace is also displayed.
  • the processor displays the corresponding candidate object in response to the user clicking on any selection icon.
  • One way is to display the candidate object selected by the user in the dotted line box instead of the original track in the dotted line box.
  • the candidate object can be scaled according to the size of the dotted line box; the other way is to display the candidate object selected by the user According to the original size, it is displayed at a predetermined position or at any position, which can be dragged and zoomed by the user later as needed.
  • corresponding candidate objects may be displayed in response to the user's long press on any selection icon.
  • the graphic recommendation area 5 may contain four sub-areas, each displaying different contents:
  • the best recognition result obtained by performing OCR on the trajectory displayed in the OCR result sub-area 51 is used as the character corresponding to the trajectory.
  • the Chinese character "umbrella" is taken as an example.
  • the character graphic recommendation sub-area 52 displays the first selection icon of the first candidate object obtained by matching the OCR result as feature information, that is, finds the corresponding candidate graphic for recommendation according to the acquired character features.
  • Displayed in the similar figure recommendation sub-area 53 is the second selection icon of the second candidate object obtained by matching according to the outline feature of the figure formed by the trajectory.
  • Displayed in the graphic recommendation sub-area 54 is the third selection icon of the third candidate object obtained by matching according to the original line features of the graphic formed by the trajectory.
  • each of the four sub-areas can be marked with a dotted line box; each sub-area can set an interaction trigger point, which is set at the lower right corner of the dotted line box of the sub-area in this example.
  • the user clicks on the interactive trigger point that displays the OCR result sub-area 51 other alternative OCR results will be displayed in this sub-area or in a new area, such as displaying a suboptimal recognition result, which can solve the problem of the user's OCR result. Unsatisfied with the problem that the recognition result needs to be selected independently; if the user selects other OCR results, the first candidate object is reacquired according to the result, and the first selection icon is refreshed.
  • selection icons of more candidate objects may be displayed in the corresponding sub-area.
  • the character corresponding to the trajectory is first recognized through OCR, and the recognition result is used as the acquired character feature.
  • the recognition result can be one character, or can be multiple characters, for example, the recognition result is a word.
  • the characters in this example include Chinese characters, letters, etc.
  • Matching is performed in the graph database according to the recognition results; the candidate graphs contained in the graph database carry labels, and the tags are used to indicate the content of the candidate graphs.
  • the label is the same as at least one of the acquired character features, it is determined that the matching is successful.
  • some or all of the undisplayed successful matching candidate graphics may be selected for display.
  • the similar graphics referred to in this example means that the graphics have similar outlines.
  • the process of recommending similar graphics includes: calculating the similarity between the outline features of the graphics formed by the trajectory and the outlines of each candidate graphic in the graphics database, and obtaining the top three similarities of the outlines.
  • the candidate graphic is used as the second candidate object, and the corresponding second selection icon is displayed.
  • the situation in FIG. 5 is to display the thumbnails of the candidate graphics corresponding to the 1st to 3rd similarities after the similarities are arranged from high to low.
  • the candidate graphics corresponding to the 4th to 6th similarities are reacquired as the second candidate objects, and the displayed No. Two selection icon.
  • the similarity calculation of contour lines includes the following steps 601-604:
  • the contour line features of each candidate graph in the graph database can be extracted and saved in advance, that is, step 603 is completed in advance, so that step 604 can be directly performed after step 602 during matching to save time.
  • candidate graphics in the graphics database are generally standard graphics, so normalization is not required. If they are not standard graphics, they must be normalized after extracting contour features.
  • step 604 the similarity between contour lines can be calculated as follows:
  • contour line features of the candidate graphics and the graphics formed by the trajectory are respectively densely sampled at equal intervals, and the sampling points in the two sampling point sets obtained by sampling are paired in pairs according to the Hungarian matching algorithm, and the distance between each sampling point pair is calculated.
  • the similarity of the contour lines is determined according to the sum of the distances of all sampled point pairs.
  • the calculated distance may be, but not limited to, Euclidean distance.
  • the number of sampling points in the two sampling point sets is different, for example, one is M and the other is N, and N is greater than M, then the obtained sampling point pair is M pair.
  • the meaning of including graphics means that the original line features of the graphics formed by the trajectory are similar to the "parts" in the candidate graphics, that is, the graphics formed by the trajectory are similar in shape to a subgraph in the candidate graphics; As shown in FIG. 7 , the figure formed by the trajectory is similar to the shape of "wings" in the sub-picture 71 of the candidate figure "bird", and the candidate figure "bird” is acquired as the third candidate object.
  • the original line features of the graph formed by the trajectory and the line features in the candidate graph (or the subgraph of the candidate graph) can be densely sampled at equal intervals, and the two sampling point sets obtained by sampling
  • the sampling points are paired in pairs according to the Hungarian matching algorithm, and the distance of each pair of sampling points is calculated, and the similarity of the two graphics is determined according to the sum of the distances of all pairs of sampling points.
  • the calculated distance may be, but not limited to, Euclidean distance.
  • the candidate graphics in the graphics database can have one or more sub-pictures, such as the situation shown in Figure 8, the candidate graphic "animated villain” can have four sub-pictures: head 81, torso 82, arms 83 (same submap for left arm and right arm), 84 for legs (same submap for left and right leg).
  • head 81 the candidate graphic "animated villain” can have four sub-pictures: head 81, torso 82, arms 83 (same submap for left arm and right arm), 84 for legs (same submap for left and right leg).
  • the similarity between the original line features of the graph formed by the trajectory and the original line features of the candidate graph and the subgraph can be calculated respectively, and the maximum similarity is taken among the calculated similarities as The final similarity between the candidate graph and the graph formed by the trajectory.
  • the first candidate object and the second candidate object are the first candidate instruction and the second candidate instruction by default.
  • the displayed first selection icon and second selection icon are descriptive information of the instruction.
  • the trajectory input by the user is identified as one or more of the preset standard trajectories, and the corresponding candidate instructions are acquired as the first and second candidate objects according to the identified standard trajectories, and the corresponding selection icons are displayed, and The corresponding action is performed in response to the user clicking on the selection icon.
  • the selection icon shown to the user is shown in Figure 9, and the track formed by the track point set input by the user is shown in area 91; two standard tracks can be identified for the track point set: "Z” and "Z” N".
  • the candidate instruction corresponding to Z" is used as the second candidate object, and the description information "Z: delete track” corresponding to "Z” is acquired as the second selection icon 922 and displayed in the selection icon area 92 .
  • the descriptive information of the candidate instruction used for selecting the icon in this example may be replaced by an illustration of the candidate instruction or the content of the candidate instruction.
  • one standard trajectory can correspond to multiple candidate instructions.
  • the standard track "N" corresponds to multiple candidate instructions, and the multiple candidate instructions may be obtained as different first candidate objects, and corresponding first selection icons are displayed respectively.
  • one of the first and second candidate objects can be changed to a candidate graphic; all the candidate instructions corresponding to the identified standard trajectories are used as the other candidate object.
  • candidate graphics as candidate objects may be selected by one or more types of feature information.
  • the candidate object can be changed into a candidate graphic adaptively.
  • the trajectory input by the user can only be recognized as a standard trajectory, then the candidate instruction corresponding to the standard trajectory is obtained as the first candidate object, and the feature information of the preset type is extracted from the trajectory to select candidate graphics, and obtained as the second candidate object.
  • candidate graphics may be selected according to different types of feature information as the first and second candidate objects respectively.

Abstract

一种交互方法、电子设备及计算机存储介质；所述交互方法包括：获取用户输入的轨迹点集合，轨迹点集合至少包括一个轨迹点；根据轨迹点集合确定候选对象；候选对象包括：第一候选对象和第二候选对象；显示用于表示第一候选对象的第一选择图标和用于表示第二候选对象的第二选择图标；响应于用户的对第一选择图标的选择操作，显示第一候选对象或者执行第一候选对象所对应的操作，响应于用户的对第二选择图标的选择操作，显示第二候选对象或者执行第二候选对象所对应的操作。

Description

一种交互方法、电子设备及计算机存储介质 技术领域
本文涉及但不限于计算机技术领域，尤指一种交互方法、电子设备及计算机存储介质。
背景技术
通过电子设备进行无纸化办公、学习的方式已逐步取代传统方式，成为主流，电子白板、带有触控屏的手机/平板电脑等电子设备为用户提供了直接在屏幕上用手指或触控笔进行书写、绘画的功能。但是当用户想输入所需要的图形时，需要进行查找、复制、粘贴或插入等一列操作才能完成，不方便且效率低。
发明内容
以下是对本文详细描述的主题的概述。本概述并非是为了限制权利要求的保护范围。
本申请实施例提供了一种交互方法、电子设备及计算机存储介质。
一方面，本申请实施例提供一种交互方法，包括：
获取用户输入的轨迹点集合，轨迹点集合至少包括一个轨迹点；
根据轨迹点集合确定候选对象，候选对象包括：第一候选对象和第二候选对象；
显示用于表示第一候选对象的第一选择图标和用于表示第二候选对象的第二选择图标；
响应于用户的对第一选择图标的选择操作，显示第一候选对象或者执行第一候选对象所对应的操作，响应于用户的对第二选择图标的选择操作，显示第二候选对象或者执行第二候选对象所对应的操作。
一种示例性实施例中，所述第一候选对象包括第一候选图形，所述第二 候选对象包括第二候选图形；
所述响应于用户的对所述第一选择图标的选择操作，显示所述第一候选对象，响应于用户的对所述第二选择图标的选择操作，显示所述第二候选对象，包括：
响应于用户的对所述第一选择图标的选择操作，显示所述第一候选图形，响应于用户的对所述第二选择图标的选择操作，显示所述第二候选图形；
所述根据轨迹点集合确定候选对象，包括：
根据所述轨迹点集合所形成的轨迹获取第一特征信息，获取与所述第一特征信息匹配成功的第一候选图形，
根据所述轨迹点集合所形成的轨迹获取第二特征信息，获取与所述第二特征信息匹配成功的第二候选图形；
所述第一特征信息与所述第二特征信息不同。
一种示例性实施例中，所述第一特征信息、第二特征信息分别包括以下一种或多种：
所述轨迹对应的一个或多个字符特征；
所述轨迹所形成的图形的原始线条特征；
所述轨迹所形成的图形的轮廓线特征。
一种示例性实施例中，所述第一特征信息或第二特征信息包括所述轨迹对应的一个或多个字符特征；
所述根据轨迹点集合所形成的轨迹提取第一特征信息或第二特征信息包括：
提取轨迹点集合所形成的轨迹所对应的一个或多个字符特征；
所述获取与所述第一/第二特征信息匹配成功的第一/第二候选图形包括：
选择所携带的标签和所述第一/第二特征信息中的字符特征至少部分相同的候选图形作为所述第一/第二候选图形；
其中，候选图形所携带的标签用于表示该候选图形的内容。
一种示例性实施例中，所述第一特征信息或所述第二特征信息包括所述 轨迹所形成的图形的原始线条特征；
所述获取与所述第一/第二特征信息匹配成功的第一/第二候选图形包括：
分别计算每个候选图形与所述第一/第二特征信息中的原始线条特征之间的第一相似度；
选择所述第一相似度满足第一预设条件的候选图形作为所述第一/第二候选图形。
一种示例性实施例中，所述第一特征信息或第二特征信息包括所述轨迹所形成的图形的原始线条特征；
所述获取与所述第一/第二特征信息匹配成功的第一/第二候选图形包括：
对于每个候选图形分别进行如下操作：
计算所述第一/第二候选图形中的原始线条特征与该候选图形之间的相似度，以及与该候选图形所包含的子图之间的相似度；以确定与该候选图形之间的第二相似度；
选择所述第二相似度满足第二预设条件的候选图形作为所述第一/第二候选图形；
其中，所述候选图形所包含的子图对应于该候选图形保存。
一种示例性实施例中，所述第一特征信息或第二特征信息包括所述轨迹所形成的图形的轮廓线特征；
所述获取与所述第一/第二特征信息匹配成功的第一/第二候选图形包括：
分别计算每个候选图形的轮廓线与所述第一/第二特征信息中轮廓线特征之间的第三相似度；
选择所述第三相似度满足第三预设条件的候选图形作为所述第一/第二候选图形。
一种示例性实施例中，所述第一候选对象包括第一候选指令，所述第二候选对象包括第二候选指令；
所述响应于用户的对所述第一选择图标的选择操作，执行所述第一候选对象所对应的操作，响应于用户的对所述第二选择图标的选择操作，执行所 述第二候选对象所对应的操作，包括：
响应于用户的对所述第一选择图标的选择操作，执行所述第一候选指令所对应的操作，响应于用户的对所述第二选择图标的选择操作，执行所述第二候选指令所对应的操作；
所述根据轨迹点集合确定候选对象，包括：
将所述轨迹点集合所形成的轨迹识别成标准轨迹，根据所述标准轨迹获取不同的候选指令，分别作为所述第一候选指令和第二候选指令。
一种示例性实施例中，所述第一候选对象是第三候选图形，所述第二候选对象是第三候选指令；
所述响应于用户的对所述第一选择图标的选择操作，显示所述第一候选对象，响应于用户的对所述第二选择图标的选择操作，执行所述第二候选对象所对应的操作，包括：
响应于用户的对所述第一选择图标的选择操作，显示所述第三候选图形，响应于用户的对所述第二选择图标的选择操作，执行所述第三候选指令所对应的操作；
所述根据轨迹点集合确定候选对象，包括：
根据所述轨迹点集合所形成的轨迹获取第三特征信息，获取与所述第三特征信息匹配成功的第三候选图形；
将所述轨迹点集合所形成的轨迹识别成标准轨迹，根据所述标准轨迹获取所述第三候选指令。
一种示例性实施例中，所述根据轨迹点集合确定候选对象还包括：
响应于将所述轨迹点集合形成的轨迹识别成标准轨迹失败，根据所述轨迹点集合所形成的轨迹获取第四特征信息，获取与所述第四特征信息匹配成功的第四候选图形作为所述第二候选对象；所述第四特征信息不同于所述第三特征信息。
一种示例性实施例中，所述交互方法应用于书写软件；所述获取用户输入的轨迹点集合包括：
响应于用户在所述书写软件的输入界面中的输入操作，获取用户在所述输入界面中输入的轨迹点集合；
所述显示用于表示所述第一候选对象的第一选择图标和用于表示所述第二候选对象的第二选择图标包括：
在所述输入界面中显示用于表示所述第一候选对象的第一选择图标和用于表示所述第二候选对象的第二选择图标；
所述响应于用户的对所述第一选择图标的选择操作，显示所述第一候选对象或者执行所述第一候选对象所对应的操作，响应于用户的对所述第二选择图标的选择操作，显示所述第二候选对象或者执行所述第二候选对象所对应的操作包括：
响应于用户在所述输入界面中，对所述第一选择图标的选择操作，在所述输入界面中显示所述第一候选对象，或在所述书写软件中执行所述第一候选对象所对应的操作；
响应于用户在所述输入界面中，对所述第二选择图标的选择操作，在所述输入界面中显示所述第二候选对象，或在所述书写软件中执行所述第二候选对象所对应的操作。
一种示例性实施例中，所述显示用于表示第一候选对象的第一选择图标和用于表示第二候选对象的第二选择图标包括：
第一候选对象为候选图形，以第一候选对象的缩略图作为第一选择图标进行显示；或者，第一候选对象为候选指令，以第一候选对象的描述信息作为第一选择图标进行显示；
第二候选对象为候选图形，以第二候选对象的缩略图作为第一选择图标进行显示；或者，第二候选对象为候选指令，以第二候选对象的描述信息作为第二选择图标进行显示。
一种示例性实施例中，所述显示用于表示第一候选对象的第一选择图标和用于表示第二候选对象的第二选择图标后还包括：
响应于用户的预定操作，更新所确定的第一候选对象和/或第二候选对象；
响应于第一候选对象的更新，相应更新所显示的第一选择图标；响应于 第二候选对象的更新，相应更新所显示的第二选择图标。
一种示例性实施例中，所述获取用户输入的轨迹点集合包括：
响应于用户的输入操作停顿，显示预设控件；其中，检测到用户的输入操作停顿是指：距离上一次收到对于输入操作的采样结果的时间长度达到或超过预设时长阈值；
响应于用户对于所述预设控件的操作，获取本次检测到用户的输入操作停顿和前一次检测到用户的输入操作停顿之间，用户输入操作所对应的轨迹点，保存为所述轨迹点集合。
另一方面，本申请实施例提供一种电子设备，包括：存储器和处理器；存储器设置成保存进行交互的程序；处理器设置成读取执行进行交互的程序，并进行上述交互方法。
再一方面，本申请实施例提供一种计算机存储介质，设置成保存进行交互的程序；进行交互的程序在被执行时，进行上述交互方法。
本申请实施例可以根据用户输入所形成的轨迹，自动、高效、快捷的提供和该轨迹相关的多种候选对象供用户选择，并能够根据用户的选择显示相应图形或进行相应操作，大大改善了用户的体验。
在阅读并理解了附图和详细描述后，可以明白其他方面。
附图说明
附图用来提供对本文技术方案的进一步理解，并且构成说明书的一部分，与本申请实施例一起用于解释本文的技术方案，并不构成对本文的技术方案的限制。附图中一个或多个部件的形状和大小不反映真实比例，目的只是示意说明本文内容。
图1为本申请实施例的交互方法的流程示意图；
图2为本申请实施例的电子设备的示意图；
图3为示例1的实现过程示意图；
图4为示例1中轨迹点集合所形成的轨迹的示意图；
图5为示例1中展示选择图标的示意图；
图6为示例1中计算轮廓线相似度的过程示意图；
图7为示例1中轨迹所形成的图形和候选图形子图相似时的示意图；
图8为示例1中子图构成候选图形的示意图；
图9为示例2中展示选择图标的示意图。
具体实施方式
下面将结合附图对本申请实施例进行详细说明。实施方式可以以多个不同形式来实施。所属技术领域的普通技术人员可以很容易地理解一个事实，就是方式和内容可以在不脱离本文的宗旨及其范围的条件下被变换为一种或多种形式。因此，本文不应该被解释为仅限定在下面的实施方式所记载的内容中。在不冲突的情况下，本文中的实施例及实施例中的特征可以相互任意组合。
在附图中，有时为了明确起见，夸大表示了一个或多个构成要素的大小、层的厚度或区域。因此，本文的一个方式并不一定限定于该尺寸，附图中多个部件的形状和大小不反映真实比例。此外，附图示意性地示出了理想的例子，本文的实现方式不局限于附图所示的形状或数值等。
本文中的“第一”、“第二”、“第三”等序数词是为了避免构成要素的混同而设置，而不是为了在数量方面上进行限定的。本文中的“多个”表示两个或两个以上的数量。
在本文中，为了方便起见，使用“中部”、“上”、“下”、“前”、“后”、“竖直”、“水平”、“顶”、“底”、“内”、“外”等指示方位或位置关系的词句以参照附图说明构成要素的位置关系，仅是为了便于描述本说明书和简化描述，而不是指示或暗示所指的装置或元件必须具有特定的方位、以特定的方位构造和操作，因此不能理解为对本文的限制。构成要素的位置关系根据描述构成要素的方向适当地改变。因此，不局限于在说明书中说明的词句，根据情况可以适当地更换。
在本文中，除非另有明确的规定和限定，术语“安装”、“相连”、“连 接”应做广义理解。例如，可以是固定连接，或可拆卸连接，或一体地连接；可以是机械连接，或电连接；可以是直接相连，或通过中间件间接相连，或两个元件内部的连通。对于本领域的普通技术人员而言，可以根据情况理解上述术语在本文中的含义。
在本文中，“电连接”包括构成要素通过具有某种电作用的元件连接在一起的情况。“具有某种电作用的元件”只要可以进行连接的构成要素间的电信号的传输，就对其没有特别的限制。“具有某种电作用的元件”的例子不仅包括电极和布线，而且还包括晶体管等开关元件、电阻器、电感器、电容器、其它具有一种或多种功能的元件等。
在本文中，“平行”是指两条直线形成的角度为-10°以上且10°以下的状态，因此，可以包括该角度为-5°以上且5°以下的状态。另外，“垂直”是指两条直线形成的角度为80°以上且100°以下的状态，因此，可以包括85°以上且95°以下的角度的状态。
本文中的“约”，是指不严格限定界限，允许工艺和测量误差范围内的数值。
本申请的一个实施例提供了一种交互方法，如图1所示，包括步骤S110-S140：
S110、获取用户输入的轨迹点集合，轨迹点集合至少包括一个轨迹点；
S120、根据轨迹点集合确定候选对象，候选对象包括：第一候选对象和第二候选对象；
S130、显示用于表示第一候选对象的第一选择图标和用于表示第二候选对象的第二选择图标；
S140、响应于用户的对第一选择图标的选择操作，显示第一候选对象或者执行第一候选对象所对应的操作，响应于用户的对第二选择图标的选择操作，显示第二候选对象或者执行第二候选对象所对应的操作。
本申请实施例的交互方法可以根据用户输入所形成的轨迹，自动、高效、快捷的提供和该轨迹相关的多种候选对象供用户选择，并能够根据用户的选 择显示相应图形或进行相应操作，大大改善了用户的体验。
一种示例性实施例中，上述交互方法可以应用在电子白板、平板电脑、智能手机等包含触控屏的电子设备上；触控屏中可包括触控板、采样单元、显示面板以及显示驱动单元；其中，触控板为一个透明板，设置在显示面板上表面；用户使用手指或触控笔进行输入操作，采样单元对该用户的触控进行采样，可以得到手指或触控提与触控板接触位置所形成的轨迹点的实时坐标，将该实时坐标发送给显示驱动单元，就可以在显示面板上对应于该实时坐标的位置显示轨迹点；用户使用手指或触控笔在触控板上连续滑动，从而进行书写或绘画，通过采样单元的不断采样和发送，显示面板上可以显示多个实时坐标对应的轨迹点集合所形成的轨迹，和用户在触控板上书写或绘画的轨迹一致。
一种示例性实施例中，上述交互方法可以应用在手写板加计算机的系统中或应用在配置有触控板的计算机中，通过手写板或触控板接收用户的输入操作，输入操作形成的轨迹点可以显示在计算机的屏幕上。在其它示例性实施例中，用户可以用鼠标或键盘操纵光标的移动，以代替触控板输入轨迹点。
一种示例性实施例中，步骤S110中，可以获取用户在预设的输入区域中(比如对话框，或编辑界面、或预定APP的界面等对应的输入区域)输入的轨迹点集合；或可以获取用户在某个时间段(比如两次用户操作停顿之间)输入的轨迹点集合。
一种示例性实施例中，步骤S120中，可以确定一个或多个第一候选对象、以及一个或多个第二候选对象。
一种示例性实施例中，步骤S120中，所确定的候选对象可以还包括第三候选对象或更多的候选对象。
一种示例性实施例中，第一候选对象和第二候选对象各自有多个，相应地，第一选择图标和第二选择图标也相应各自有多个，分别和第一候选对象、第二候选对象一一对应。
一种示例性实施例中，步骤S120中所确定的候选对象的类别为图形或操作对应的指令；第一候选对象和第二候选对象的类别可以均为图形、或均 为指令、或一个为图形而另一个为指令。相应的，步骤S140中，候选对象的类别为图形且对应的第一或第二选择图标被点击后，可以显示相应图形；候选对象的类别为指令且对应的第一或第二选择图标被点击后，可以执行该指令对应的操作。
一种示例性实施例中，步骤S130中所显示的第一选择图标、第二选择图标是供用户点击以选择相应的第一候选对象、第二候选对象。第一、第二选择图标所采用的形态可以分别根据第一、第二候选对象的类别确定；比如，第一、第二选择图标可以是以图形的缩略图作为图标显示内容的控件，或可以是以指令的描述信息作为图标显示内容的控件。其中，描述信息可以是图形或文字的形式，比如第一候选对象为“撤销”指令，则第一选择图标可显示为逆时针箭头；再比如第一候选对象为“删除”指令，则第一选择图标可显示“del”。
一种示例性实施例中，步骤S130中，可以同时显示第一选择图标和第二选择图标，或可以依次显示第一选择图标、第二选择图标；第一、第二选择图标可以分别显示在不同区域中。显示的顺序、区域可以采用默认方式，或可以由用户自行设定。
一种示例性实施例中，步骤S140中，只允许用户选择一个选择图标，响应于用户对一个选择图标(第一选择图标或第二选择图标)的选择操作(比如但不限于点击或长按等操作)，可以停止显示第一选择图标和第二选择图标，显示用户所选择的选择图标所对应的候选对象或按照所对应的候选对象执行对应操作。即，用户通过对选择图标进行选择，可以选择一个第一候选对象或第二候选对象；根据用户所选择的这一个候选对象的类别，相应显示候选对象，或执行候选对象对应的操作。
一种示例性实施例中，步骤S140中，可以允许用户选择两个或以上的选择图标，这两个或以上的选择图标可以均为第一/第二选择图标，也可以部分是第一选择图标，部分是第二选择图标。比如用户按住“ctrl”键并选择一个第一选择图标，一个第二选择图标，响应于用户该选择操作，可以同时显示第一、第二候选对象、或依次执行第一、第二对象对应的操作、或显示一个候选对象并执行另一个候选对象对应的操作。
一种示例性实施例中，步骤S140中，在显示候选对象时，可以显示在光标当前所在位置，或可以使用候选对象替代轨迹点集合形成的轨迹进行显示。
一种示例性实施例中，第一候选对象包括第一候选图形，第二候选对象包括第二候选图形；
响应于用户的对第一选择图标的选择操作，显示第一候选对象，响应于用户的对第二选择图标的选择操作，显示第二候选对象，包括：
响应于用户的对第一选择图标的选择操作，显示第一候选图形，响应于用户的对第二选择图标的选择操作，显示第二候选图形；
根据轨迹点集合确定候选对象，包括：
根据轨迹点集合所形成的轨迹获取第一特征信息，获取与第一特征信息匹配成功的第一候选图形，
根据轨迹点集合所形成的轨迹获取第二特征信息，获取与第二特征信息匹配成功的第二候选图形；
所述第一特征信息与所述第二特征信息不同。
其中，第一、第二候选图形可以但不限于是预设的图形数据库中的图形；该图形数据库可以有一个或多个，可以设置在本地、服务器、云端等处。
一种示例性实施例中，第一特征信息、第二特征信息分别包括以下一种或多种：
轨迹对应的一个或多个字符特征；
轨迹所形成的图形的原始线条特征；
轨迹所形成的图形的轮廓线特征。
除了上述列举的类型，可以挑选其它类型的特征信息来进行匹配，本申请对此不做限制。
其中，原始线条特征是指轨迹点集合中的轨迹点所构成的、用于形成图形的线条本身。
一种示例性实施例中，第一候选对象和第二候选对象可以分别是与不同 类型的特征信息匹配成功的候选图形。比如第一候选对象是与字符特征匹配成功的第二候选图形，第二候选对象是与原始线条特征匹配成功的第二候选图形。
一种示例性实施例中，第一特征信息或第二特征信息包括轨迹对应的一个或多个字符特征；
根据轨迹点集合所形成的轨迹获取第一/第二特征信息包括：
获取轨迹点集合所形成的轨迹所对应的一个或多个字符特征；
获取与第一/第二特征信息匹配成功的第一/第二候选图形包括：
选择所携带的标签和第一/第二特征信息中的字符特征至少部分相同的候选图形作为第一/第二候选图形；其中，候选图形所携带的标签用于表示该候选图形的内容。
本实施例的一种实施方式中，可以但不限于通过OCR(Optical Character Recognition，光学字符识别)来提取轨迹对应的一个或多个字符特征。或可以通过其它方式来获取轨迹所对应的字符特征，比如从轨迹中提取笔画特征、笔顺特征等，与字库中不同字符的相应特征进行比对，从而确定轨迹对应的字符特征。
本实施例的一种实施方式中，一个候选图形可以具有一个或多个标签，标签可以包括一个字符或多个字符。一个示例中，根据识别出的字符特征在图形数据库中搜索内容相同的标签；如果一个候选图形的任一个标签和识别出的至少一个字符特征相同，则可以认为该候选图形匹配成功。
本实施例的一种实施方式中，可以自行设置匹配成功的标准，可以设置成标签必须和全部字符特征相同才算匹配成功，比如获取的字符特征和标签都为“雨伞”则匹配成功；或可以设置成标签与部分字符特征相同就算匹配成功，比如获取字符特征为“伞”而标签为“雨伞”，可以判定为匹配成功；再比如获取的字符特征为“猫咪”而标签为“猫”，可以判定为匹配成功。
本实施例的一种实施方式中，如果匹配成功的候选图形较多，可以优先挑选标签和获取的字符特征完全相同的候选图形；或者，可以在标签中预先加入相关度数值或优先级，优先挑选相关度数值高或优先级高的标签对应的 候选图形。其中，相关度数值可以用来表示候选图形和该标签之间相关的程度，一个示例中，雨伞图形和标签“伞”的相关度数值为100％；蘑菇图形和标签“伞”的相关度数值为20％，如果根据字符特征“伞”匹配成功多个候选图形，可以优先挑选雨伞图形。
一种示例性实施例中，第一特征信息或第二特征信息包括轨迹所形成的图形的原始线条特征；
获取与第一/第二特征信息匹配成功的第一/第二候选图形包括：
分别计算每个候选图形与第一/第二特征信息中原始线条特征之间的第一相似度；
选择第一相似度满足第一预设条件的候选图形作为第一/第二候选图形。
本实施例的一种实施方式中，第一预设条件可以是候选图形的第一相似度是最高的N个第一相似度之一；选择第一相似度满足第一预设条件的候选图形包括：选择第一相似度最高的N个候选图形。N为预设的正整数，可以是想要展示出来供用户选择的候选图形的个数。
本实施例的另一种实施方式中，第一预设条件可以是第一相似度超过或达到预设的第一相似度阈值。
本实施例的一种实施方式中，一个候选图形与原始线条特征之间的第一相似度可以按照如下方式计算：
对原始线条特征按照第一间距采样得到第一采样点集合，对该候选图形按照第一间距采样得到第二采样点集合；
对第一、第二采样点集合中的采样点，采用匈牙利配对算法进行配对；如果进行配对的一个采样点集合中的采样点个数少于另一个采样点集合，则得到的采样点对的数量等于采样点个数较少的采样点集合中的采样点个数；
分别计算每组采样点对的距离，根据全部采样点对的距离之和确定该候选图形与原始线条特征之间的第一相似度。
其中，第一间距可自行设置，比如需要进行比较密集的采样，可以将第一间距设置的较小。
其中，所计算采样点对的距离可以但不限于是欧式距离；距离越小则相 似度越大。
本实施例中，不限于采用上述方式计算第一相似度，可采用其它方式来确定原始线条特征和候选图形之间的第一相似度，比如计算余弦相似度，再比如通过指纹信息计算相似度等。
一种示例性实施例中，第一特征信息或第二特征信息包括轨迹所形成的图形的原始线条特征；
获取与第一/第二特征信息匹配成功的第一/第二候选图形包括：
对于每个候选图形分别进行如下操作：
计算第一/第二特征信息中的原始线条特征与该候选图形之间的相似度，以及与该候选图形所包含的子图之间的相似度，以确定与该候选图形之间的第二相似度；
选择第二相似度满足第二预设条件的候选图形作为第一/第二候选图形；
其中，候选图形所包含的子图对应于该候选图形保存。
本实施例中，可以是在计算出的与候选图形的相似度、与该候选图形子图的相似度中，挑选最大的相似度，作为与该候选图形之间的相似度。
本实施例的一种实施方式中，第二预设条件可以是候选图形的第二相似度是最高的N个第二相似度之一；选择第二相似度满足第二预设条件的候选图形包括：选择第二相似度最高的N个候选图形。N为预设的正整数，可以是想要展示出来供用户选择的候选图形的个数。
本实施例的另一种实施方式中，第二预设条件可以是第二相似度超过或达到预设的第二相似度阈值。
本实施例中，计算原始线条特征与候选图形或子图的相似度的实现细节可参照计算第一相似度的做法。
本实施例可以挑选出与部分与轨迹所形成的图形相似的候选图形，即挑选出“零件”与轨迹所形成的图形相似的候选图形。
本实施例中，可以在将存储候选图形的过程中，将子图与子图所组成的候选图形一起存储，或可以从候选图形中拆分出子图并和该候选图形一起存 储。
本实施例中，不一定每个候选图形都具有子图，可以仅部分候选图形具有子图。候选图形所包含的子图的总和，可以小于该候选图形；候选图形中相同的部分，可以作为同一个子图；比如候选图形是一个房子，里面包括多个相同的窗户，那么每个窗户都可以使用同一个“窗户的子图”，而“房子图形”的其余部分不作为子图。
一种示例性实施例中，第一特征信息或第二特征信息包括轨迹所形成的图形的轮廓线特征；
获取与第一/第二特征信息匹配成功的第一/第二候选图形包括：
分别计算每个候选图形的轮廓线与第一/第二特征信息中的轮廓线特征之间的第三相似度；
选择第三相似度满足第三预设条件的候选图形作为第一/第二候选图形。
本实施例的一种实施方式中，第三预设条件可以是候选图形的第三相似度是最高的N个第三相似度之一；选择第三相似度满足第三预设条件的候选图形包括：选择第三相似度最高的N个候选图形。N为预设的正整数，可以是想要展示出来供用户选择的候选图形的个数。
本实施例的另一种实施方式中，第三预设条件可以是第三相似度超过或达到预设的第三相似度阈值。
本实施例中，计算一个候选图形和轮廓线特征之间的第三相似度的过程可以包括：
对轨迹所形成的图形提取轮廓线特征，进行归一化处理，按照第二间距采样，得到第三采样点集合；
对该候选图形的轮廓线按照第二间距采样，得到第四采样点集合；
对第三、第四采样点集合中的采样点，采用匈牙利配对算法进行配对；如果进行配对的一个采样点集合中的采样点个数少于另一个采样点集合，则得到的采样点对的数量等于采样点个数较少的采样点集合中的采样点个数；
分别计算每组采样点对的距离，根据全部采样点对的距离之和确定轮廓线特征与该候选图形的轮廓线之间的第三相似度。
其中，第二间距可自行设置，比如需要进行比较密集的采样，可以将第二间距设置的较小。
其中，所计算采样点对的距离可以但不限于是欧式距离。
本实施例中，不限于采用上述方式计算第三相似度，可采用任意一种确定曲线之间相似度的方式来确定第三相似度，比如可以通过离散弗雷歇距离、豪斯多夫距离、pearson相关系数等方式确定第三相似度。
一些示例性的实施例中，第一特征信息或第二特征信息包括轨迹所形成的图形或该图形的轮廓线，获取第一/第二特征信息可以包括：
对轨迹所形成的图形按照预设的多个不同比例进行缩放；
相应地，计算第一/第二特征信息中的原始线条特征和候选图形(或其子图)的相似度、或计算第一/第二特征信息中的轮廓线特征和候选图形轮廓线的相似度，是分别按照不同比例缩放后的图形进行计算；针对不同比例，会计算出不同的相似度，将其中的最大值作为最终的相似度。
比如，要计算原始线条特征IMG 1与候选图形IMG 2的第一相似度，先根据预设的缩放比例0.5、1、2对IMG 1进行缩放，分别得到IMG 1-1、IMG 1-2和IMG 1-3。对IMG 1-1和IMG 2分别按照等间距密集采样，然后对得到的两个采样点集合进行匈牙利配对，计算每个采样点对的距离，得到距离总和L 1-1；类似的，对于IMG 1-2和IMG 1-3分别得到距离总和L 1-2和L 1-3，根据三个距离总和中的最小值(对应于最大的相似度)作为最终距离，来确定IMG 1与IMG 2的最终的第一相似度。
本实施例可以减轻图形大小对距离(或相似度)计算的影响。
本实施例中，预设比例可以设置为0.1、0.2、……、9、10。
一种示例性实施例中，第一候选对象包括第一候选指令，第二候选对象包括第二候选指令；
响应于用户的对第一选择图标的选择操作，执行第一候选对象所对应的操作，响应于用户的对第二选择图标的选择操作，执行第二候选对象所对应的操作，包括：
响应于用户的对第一选择图标的选择操作，执行第一候选指令所对应的 操作，响应于用户的对第二选择图标的选择操作，执行第二候选指令所对应的操作；
所述根据轨迹点集合确定候选对象，包括：
将所述轨迹点集合所形成的轨迹识别成标准轨迹，根据所述标准轨迹获取不同的候选指令，分别作为所述第一候选指令和第二候选指令。
本实施例的一种实施方式中，可以将轨迹识别成不同的第一标准轨迹和第二标准轨迹，获取与第一标准轨迹对应的候选指令作为第一候选指令，获取与第二标准轨迹对应的候选指令作为第二标准指令。
本实施例的另一种实施方式中，可以将轨迹识别成一种标准轨迹，获取该标准轨迹对应的多个候选指令，分别作为第一、第二候选指令。
本实施例中，如果识别出多种标准轨迹，可以按照这多个标准轨迹和轨迹点集合形成的轨迹之间的相似度进行排序，挑选排序靠前的标准轨迹，来获取第一、第二候选指令。
本实施例中，标准轨迹是一种具有标准化形态的轨迹，因为用户输入轨迹的个性化程度非常高，即使意图输入同样的轨迹(比如都想输入轨迹“Z”)，不同用户实际输入的轨迹形态上都会有差异，甚至同一个用户先后两次输入的轨迹也会具有差异；为了适应这种差异化的输入，需要先将用户输入的轨迹识别成标准轨迹再相应获取候选指令，即先进行一个“标准化”的处理。将轨迹识别成标准轨迹的过程可以采用类似于手写输入中识别用户所输入的字符的做法：提取轨迹的特征并和标准轨迹的特征比对，以将轨迹识别为预设的标准轨迹中的一个。如果用户输入的轨迹的特征无法被识别成任一个标准轨迹，则表示用户输入的轨迹并没有对应的候选指令，无法根据用户输入的轨迹获取到类别是候选指令的候选对象。
本实施例中，可以预先设置多个标准轨迹，并设置每个标准轨迹所对应的一个或多个候选指令；标准轨迹的样式可以是以下任一种或多种：图形、符号、字符。
一种示例性实施例中，
本实施例中，用户输入的轨迹可能可以被识别成不止一个标准轨迹，比如用户输入一个斜着的“Z”，可以识别为标准轨迹“Z”，也可以识别为标准轨迹“N”，可以获取“Z”和“N”对应的候选指令，将“Z”对应的候选指令作为第一候选对象，将“N”对应的候选指令作为第二候选对象。
另一示例性实施例中，第一、第二候选对象可以是一个标准轨迹所对应的不同候选指令。比如标准轨迹“Z”对应两个指令：删除轨迹、清空对话框。可以将这两个指令分别作为第一、第二候选对象。
一种示例性实施例中，第一候选对象是第三候选图形，第二候选对象是第三候选指令；
根据轨迹点集合确定候选对象包括：
响应于用户的对第一选择图标的选择操作，显示第一候选对象，响应于用户的对第二选择图标的选择操作，执行第二候选对象所对应的操作，包括：
响应于用户的对第一选择图标的选择操作，显示第三候选图形，响应于用户的对第二选择图标的选择操作，执行第三候选指令所对应的操作；
根据轨迹点集合确定候选对象，包括：
根据轨迹点集合所形成的轨迹获取第三特征信息，获取与第三特征信息匹配成功的第三候选图形；
将所述轨迹点集合所形成的轨迹识别成标准轨迹，根据所述标准轨迹获取第三候选指令。
本实施例的一种实施方式中，根据轨迹点集合确定候选对象还可以包括：
响应于将轨迹点集合形成的轨迹识别成标准轨迹失败，根据轨迹点集合所形成的轨迹获取第四特征信息，获取与第四特征信息匹配成功的第四候选图形作为第二候选对象；第四特征信息和第三特征信息不同。
本实施方式中，默认第一、第二候选对象一个是候选图形，另一个是候选指令；但如果用户输入的轨迹无法被识别为标准轨迹，即无法获得候选指令，则可以自适应的将第二候选对象改为按照不同于第三特征信息的第四特征信息匹配成功的候选图形。
一种示例性实施例中，上述交互方法可应用于书写软件；获取用户输入 的轨迹点集合包括：
响应于用户在书写软件的输入界面中的输入操作，获取用户在输入界面中输入的轨迹点集合；
显示用于表示第一候选对象的第一选择图标和用于表示第二候选对象的第二选择图标包括：
在书写软件的输入界面中显示用于表示第一候选对象的第一选择图标和用于表示第二候选对象的第二选择图标；
响应于用户的对第一选择图标的选择操作，显示第一候选对象或者执行第一候选对象所对应的操作，响应于用户的对第二选择图标的选择操作，显示第二候选对象或者执行第二候选对象所对应的操作包括：
响应于用户在书写软件的输入界面中，对第一选择图标的选择操作，在输入界面中显示第一候选对象，或在书写软件中执行第一候选对象所对应的操作；
响应于用户在输入界面中，对第二选择图标的选择操作，在输入界面中显示第二候选对象，或在书写软件中执行第二候选对象所对应的操作。
一种示例性实施例中，显示用于表示第一候选对象的第一选择图标和用于表示第二候选对象的第二选择图标包括：
第一候选对象为候选图形，以第一候选对象的缩略图作为第一选择图标进行显示；或者，第一候选对象为候选指令，以第一候选对象对应的描述信息作为第一选择图标进行显示；
第二候选对象为候选图形，以第二候选对象的缩略图作为第一选择图标进行显示；或者，第二候选对象为候选指令，以第二候选对象对应的描述信息作为第二选择图标进行显示。
一种示例性实施例中，显示用于表示第一候选对象的第一选择图标和用于表示第二候选对象的第二选择图标后还包括：
响应于用户的预定操作，更新所确定的第一候选对象和/或第二候选对象；
响应于第一候选对象的更新，相应更新所显示的第一选择图标；响应于第二候选对象的更新，相应更新所显示的第二选择图标。
本实施例中，比如用户点击了表示“更多”的控件，或点击了表示“刷新”的图标后，可以获取另一批候选对象(比如已匹配成功但未显示的候选图形、再比如按照其它特征信息重新选择的候选图形、再比如将轨迹识别成其它的标准轨迹重新获取的候选指令等)，并相应更新所显示的选择图标。
一种示例性实施例中，获取用户输入的轨迹点集合包括：
响应于用户的输入操作停顿，显示预设控件；其中，检测到用户的输入操作停顿是指：距离上一次收到对于输入操作的采样结果的时间长度达到或超过预设时长阈值；
响应于用户对于预设控件的操作，获取本次检测到用户的输入操作停顿和前一次检测到用户的输入操作停顿之间，用户输入操作所对应的轨迹点，保存为轨迹点集合。
本实施例中是获取特定时间段(两次操作停顿之间)输入的轨迹点集合，其它实施例中，可以获取预定输入区域中输入的轨迹点集合。
本申请实施例还提供了一种电子设备，如图2所示，包括：存储器21和处理器22；
存储器21设置成保存进行交互的程序；
处理器22设置成读取执行进行交互的程序，并进行上述任一实施例中的交互方法。
一种示例性实施例中，该电子设备可以还包括：
触控板，设置成接收用户的输入操作；
采样单元、显示面板和显示驱动单元；通过采样单元采样触控位置的实时坐标并发送给显示驱动单元，可以在显示面板上根据用户在触控板上的输入操作相应进行显示；其中，显示驱动单元还可以根据处理器的控制，在显示面板上显示全部或部分匹配成功的备选图形。
本申请实施例还提供了一种计算机存储介质，设置成保存进行交互的程序；该进行交互的程序在被执行时，进行上述任一实施例中的交互方法。
下面用两个示例说明本申请实施例。
示例1
本示例应用在电子白板中，该电子白板包括触控屏、存储器和处理器。候选对象包括：第一、第二、第三候选对象，均为候选图形。
本示例的整体实现流程如图3所示：
用户在电子白板上书写或绘画，处理器检测到用户停顿，对用户书写或绘画所形成的轨迹点集合进行识别来获取候选图形，作为推荐结果返回给前端，以选择图标的形式显示出来供用户选择。
本示例中获取候选图形的方式包括：对轨迹点集合形成的轨迹获取三种类型的特征信息，并分别和图形数据库中的候选图形进行匹配；根据所获取的特征信息的类型不同，匹配方式分为：
字符匹配(这个方式根据轨迹对应的字符特征进行匹配)、轮廓线匹配(这个方式根据轨迹形成的图形的轮廓线特征进行匹配)、图形匹配(这个方式根据轨迹形成的图形的原始线条特征进行匹配)。
本示例中，采样单元可以将采样到的用户对于电子白板的触控的实时坐标发送给处理器；处理器检测到用户停顿会触发交互操作开始。其中，检测到用户停顿可以是指：距离上一次从采样单元接收到实时坐标的时间长度超过预设的时长阈值。
本示例中，处理器在交互操作开始后通过控制显示驱动单元，在显示面板上展示一个带有交互触发点的虚线框来向用户发出交互请求。该虚线框将上一次检测到停顿到本次检测到停顿之间的轨迹框出，如图4所示；即该虚线框中的轨迹是所获取的轨迹点集合形成的轨迹，包含从上一次检测到停顿后所接收的第一个轨迹点、到本次检测到停顿前所接收的最后一个轨迹点在内的多个轨迹点。
除了本示例所使用的虚线框外，可以采用其它形式来标识出所获取的轨迹点集合，比如将相应轨迹的颜色改变，或将包含相应轨迹的区域的底色改为高亮等。
本示例中，交互触发点设置成位于虚线框的右下角的一个有一定面积的 圆点，触发方式是点击该圆点；在实施时，交互触发点可以采用其它展现形式、位于其它位置，触发方式可以是多种多样的；比如交互触发点可以设置成菜单栏中的一个图标，或可以设置在虚线框的框线上或虚线框内的区域中；再比如可以不设置交互触点，直接通过双击或长按等方式触发交互。
本示例中，处理器在展示出虚线框后，根据采样单元之后所采样的一个或多个实际坐标确定用户的操作，并进行相应处理，包括以下五种情况：
(1)用户在虚线框框内继续书写或绘画：
处理器不做处理，记录本次停顿的时刻并继续监测直至下次停顿。
(2)用户在虚线框外继续书写或绘画：
处理器不做处理，记录本次停顿的时刻并继续监测直至下次停顿。
(3)用户长按或按住虚线框内任意位置并拖动：
处理器根据用户的拖动相应改变虚线框中轨迹的坐标，并将改变后的坐标发送给显示驱动单元，这样显示面板上展示出的效果是虚线框中的轨迹随着用户的拖动而相应移动。
(4)用户长按或按住虚线框右下角的交互触发点并拖动：
处理器根据用户的拖动相应改变轨迹中相邻轨迹点之间的距离，保持轨迹中一个轨迹点的坐标不变，其它轨迹点的坐标随着距离的变化而变化；处理器将改变后的坐标发送给显示驱动单元，这样显示面板上展示出的效果是虚线框中的轨迹随着用户的拖动而缩放。
(5)用户点击虚线框右下角的交互触发点：
处理器获取虚线框中轨迹的三种特征信息：字符特征、所形成的图形的原始线条特征和轮廓线特征，连接本地或远程的图形数据库，分别根据所获取的每种类型的特征信息，和图形数据库中所包含的候选图形进行匹配，得到每种类型特征信息对应的候选对象，即根据该种类型的特征信息匹配成功的候选图形。处理器向前端返回全部或部分匹配成功的候选图形作为候选对象供显示。本示例中，第一候选对象是根据轨迹对应的字符特征匹配成功的候选图形，第二候选对象是根据轨迹所形成的图形的轮廓线特征匹配成功的候选图形，第三候选对象是根据轨迹所形成的图形的原始线条特征匹配成功 的候选图形。
如图5所示，显示给用户的是四个第一选择图标、三个第二选择图标、三个第三选择图标，分别是第一、第二、第三候选对象对应的缩略图。本示例中，还会显示出轨迹对应的字符。
本示例中，处理器响应于用户对于任一个选择图标的点击，显示相应的候选对象。一种方式是将用户选择的候选对象显示在虚线框中，代替虚线框中原先的轨迹，该实现方式中，可根据虚线框的大小缩放候选对象；另一种方式是将用户选择的候选对象按照原先的大小显示在一个预定位置或任意位置，供用户后续根据需要自行拖动、缩放。
除了点击选择的方式以外，可以响应于用户对于任一个选择图标的长按，来显示相应的候选对象。
图5中，图形推荐区域5可以包含四个子区域，分别显示不同内容：
其中，OCR结果子区域51中显示的对轨迹进行OCR所得到的最佳识别结果，作为轨迹对应的字符，图5中以汉字“伞”举例。
字符图形推荐子区域52中显示的是将OCR结果作为特征信息进行匹配，所得到的第一候选对象的第一选择图标，即根据获取的字符特征，找到对应的候选图形进行推荐。
相似图形推荐子区域53中显示的是根据轨迹所形成的图形的轮廓线特征进行匹配，所得到的第二候选对象的第二选择图标。
包含图形推荐子区域54中显示的是根据轨迹所形成的图形的原始线条特征进行匹配，所得到的第三候选对象的第三选择图标。
本示例中，四个子区域可以各自用虚线框标识出边界；每个子区域可以各自设置交互触发点，本示例中设置在子区域的虚线框的右下角。当用户点击显示OCR结果子区域51的交互触发点时，会在该子区域内或一个新的区域中显示备选的其它OCR结果，比如显示次优的识别结果，这样可以解决用户对OCR结果不满意需要自主选择识别结果的问题；如果用户选择了其它OCR结果，则根据该结果重新获取第一候选对象，并刷新第一选择图标。当用户点击其它三个子区域中任何一个的交互触发点时，可以在相应子区域 中显示更多候选对象的选择图标。
本示例中，按照不同类型的特征信息获取候选图形的细节如下：
(1)字符图形推荐
本示例中，根据轨迹对应的字符特征进行图形推荐时，先通过OCR识别轨迹所对应的字符，将识别结果作为所获取的字符特征。识别结果可以是一个字符，或可以是多个字符，比如识别结果是一个词。本示例中的字符包括汉字、字母等。
根据识别结果在图形数据库中进行匹配；图形数据库中所包含的候选图形携带有标签，该标签用来指示候选图形的内容。当标签和所获取的至少一个字符特征相同，判定匹配成功。
响应于用户点击字符图形推荐子区域52的交互触发点，可以从未显示的匹配成功的候选图形中挑选部分或全部进行显示。
(2)相似图形推荐
本示例中所指的相似图形是指图形具有相似的轮廓线。
考虑到用户手写的不一定是文字，有可能画出一个简笔画图形来寻找图形，此时根据OCR结果推荐无意义，可以用相似图形查找的方法做推荐。
本示例中，进行相似图形推荐的过程包括：计算轨迹形成的图形的轮廓线特征和图形数据库中每个候选图形的轮廓线之间的相似度，获取轮廓线的相似度排在前三位的候选图形作为第二候选对象，显示相应的第二选择图标。比如图5中的情况是显示相似度从高到低排列后，排在第1到第3的相似度对应的候选图形的缩略图。响应于用户点击相似图形推荐子区域53的交互触发点，按照相似度的排列顺序，重新获取排在第4到第6的相似度对应的候选图形作为第二候选对象，相应更新所显示的第二选择图标。
如图6所示，轮廓线的相似度计算包括如下步骤601-604：
601、先将虚线框中轨迹所形成的图形的轮廓线特征提取出来；
602、对于所提取的轮廓线特征进行归一化处理；
对于图形数据库中的每个候选图形分别进行步骤603、604：
603、提取该候选图形的轮廓线特征；
604、对轮廓线特征进行相似度计算。
其中，可以预先对图形数据库中每个候选图形分别提取好轮廓线特征并保存，即提前完成步骤603，这样在匹配时步骤602后就可以直接进行步骤604，以节省时间。图形数据库中的候选图形一般为标准图形，所以不需要归一化，如果不是标准图形，则提取轮廓线特征后也要进行归一化处理。
本示例中，步骤604中可以按照如下方式计算轮廓线之间的相似度：
对候选图形和轨迹形成的图形的轮廓线特征分别进行等间距密集采样，对采样得到的两个采样点集合中的采样点，按照匈牙利匹配算法两两配对，计算每一个采样点对的距离，根据所有采样点对的距离总和确定轮廓线的相似度。
其中，所计算的距离可以但不限于为欧式距离。在配对时，如果两个采样点集合中采样点数量不同，比如一个为M，一个为N，N大于M，则得到的采样点对为M对。
(3)包含图形推荐
本示例中，包含图形的含义是指，轨迹所形成的图形的原始线条特征和候选图形中的“零件”相似，即轨迹所形成的图形，和候选图形中的一个子图的形状相似；如图7所示，轨迹所形成的图形与候选图形“小鸟”的子图71中“翅膀”的形状相似，获取该候选图形“小鸟”作为第三候选对象。
在计算相似度时，可以先对轨迹形成的图形的原始线条特征和候选图形(或候选图形的子图)中的线条特征分别进行等间距密集采样，对采样得到的两个采样点集合中的采样点，按照匈牙利匹配算法两两配对，计算每一个采样点对的距离，根据所有采样点对的距离总和确定这两个图形的相似度。其中，所计算的距离可以但不限于为欧式距离。
本示例中，图形数据库中的候选图形可以具有一个或多个子图，比如图8所示的情况，候选图形“动画小人”可以具有四个子图：分别是头部81、躯干82、双臂83(左臂和右臂采用相同的子图)、双腿84(左腿和右腿采用相同的子图)。当轨迹形成的图形和头部81相似时，可以匹配成功“动画 小人”这个候选图形。
对于具有子图的候选图形，可以分别计算轨迹形成的图形的原始线条特征和该候选图形、子图的原始线条特征各自的相似度，在所计算出的相似度中取最大的相似度，作为该候选图形和轨迹形成的图形之间最终的相似度。
示例2
本示例中，第一候选对象、第二候选对象默认为第一候选指令、第二候选指令。所展示的第一选择图标、第二选择图标为指令的描述信息。本示例中，将用户输入的轨迹识别为预设的标准轨迹中的一个或多个，根据识别出的标准轨迹获取对应的候选指令作为第一、第二候选对象，展示相应的选择图标，并响应于用户对于选择图标的点击执行相应操作。
本示例中，展示给用户的选择图标如图9所示，用户输入的轨迹点集合所形成的轨迹如区域91所示；对该轨迹点集合可识别出两个标准轨迹：“Z”和“N”。获取标准轨迹“N”对应的候选指令作为第一候选对象，并获取“N”对应的描述信息“N：新建文档”作为第一选择图标921，展示在选择图标区域92中；获取标准轨迹“Z”对应的候选指令作为第二候选对象，并获取“Z”对应的描述信息“Z：删除轨迹”作为第二选择图标922，展示在选择图标区域92中。用户点击第一选择图标921则在当前的应用中创建一个新的文档，用户点击第二选择图标922则删除区域91中的轨迹。
本示例中的选择图标所使用的候选指令的描述信息，可以替换为候选指令的图示或候选指令的内容。
本示例中，一个标准轨迹可以对应多个候选指令。比如标准轨迹“N”对应多个候选指令，则可以获取这多个候选指令作为不同的第一候选对象，分别展示相应的第一选择图标。
本示例的一个变形中，可以将第一、第二候选对象中的一个候选对象改为候选图形；所有识别出的标准轨迹对应的候选指令都作为另一个候选对象。作为候选对象的候选图形可以通过一种或多种类型的特征信息进行选择。
本示例中，如果对于用户输入的轨迹，无法识别出标准轨迹，可以自适 应将候选对象改为候选图形。比如对于用户输入的轨迹只能识别成一个标准轨迹，则获取该标准轨迹对应的候选指令作为第一候选对象，另外对轨迹提取预设类型的特征信息，以选择候选图形，获取作为第二候选对象。如果对于用户输入的轨迹无法识别成任何标准轨迹，则可以按照不同类型的特征信息选择候选图形，分别作为第一、第二候选对象。
本文中的附图只涉及本文涉及到的实现方式，其他实现方式可参考设计。在不冲突的情况下，本申请实施例及实施例中的特征可以相互组合以得到新的实施例。
本领域的普通技术人员应当理解，可以对本文的技术方案进行修改或者等同替换，而不脱离本文技术方案的精神和范围，均应涵盖在本文的权利要求的范围当中。

Claims (16)

  1. 一种交互方法，包括：
    获取用户输入的轨迹点集合，所述轨迹点集合至少包括一个轨迹点；
    根据所述轨迹点集合确定候选对象，所述候选对象包括：第一候选对象和第二候选对象；
    显示用于表示所述第一候选对象的第一选择图标和用于表示所述第二候选对象的第二选择图标；
    响应于用户的对所述第一选择图标的选择操作，显示所述第一候选对象或者执行所述第一候选对象所对应的操作，响应于用户的对所述第二选择图标的选择操作，显示所述第二候选对象或者执行所述第二候选对象所对应的操作。
  2. 如权利要求1所述的交互方法，其中，所述第一候选对象包括第一候选图形，所述第二候选对象包括第二候选图形；
    所述响应于用户的对所述第一选择图标的选择操作，显示所述第一候选对象，响应于用户的对所述第二选择图标的选择操作，显示所述第二候选对象，包括：
    响应于用户的对所述第一选择图标的选择操作，显示所述第一候选图形，响应于用户的对所述第二选择图标的选择操作，显示所述第二候选图形；
    所述根据轨迹点集合确定候选对象，包括：
    根据所述轨迹点集合所形成的轨迹获取第一特征信息，获取与所述第一特征信息匹配成功的第一候选图形，
    根据所述轨迹点集合所形成的轨迹获取第二特征信息，获取与所述第二特征信息匹配成功的第二候选图形；
    所述第一特征信息与所述第二特征信息不同。
  3. 如权利要求2所述的交互方法，其中，所述第一特征信息、第二特征信息分别包括以下一种或多种：
    所述轨迹对应的一个或多个字符特征；
    所述轨迹所形成的图形的原始线条特征；
    所述轨迹所形成的图形的轮廓线特征。
  4. 如权利要求3所述的交互方法，其中，所述第一特征信息或第二特征信息包括所述轨迹对应的一个或多个字符特征；所述根据轨迹点集合所形成的轨迹提取第一特征信息或第二特征信息包括：
    提取轨迹点集合所形成的轨迹所对应的一个或多个字符特征；
    所述获取与所述第一/第二特征信息匹配成功的第一/第二候选图形包括：
    选择所携带的标签和所述第一/第二特征信息中的字符特征至少部分相同的候选图形作为所述第一/第二候选图形；
    其中，候选图形所携带的标签用于表示该候选图形的内容。
  5. 如权利要求3所述的交互方法，其中，所述第一特征信息或所述第二特征信息包括所述轨迹所形成的图形的原始线条特征；
    所述获取与所述第一/第二特征信息匹配成功的第一/第二候选图形包括：
    分别计算每个候选图形与所述第一/第二特征信息中的原始线条特征之间的第一相似度；
    选择所述第一相似度满足第一预设条件的候选图形作为所述第一/第二候选图形。
  6. 如权利要求3所述的交互方法，其中，所述第一特征信息或第二特征信息包括所述轨迹所形成的图形的原始线条特征；
    所述获取与所述第一/第二特征信息匹配成功的第一/第二候选图形包括：
    对于每个候选图形分别进行如下操作：
    计算所述第一/第二候选图形中的原始线条特征与该候选图形之间的相似度，以及与该候选图形所包含的子图之间的相似度；以确定与该候选图形之间的第二相似度；
    选择所述第二相似度满足第二预设条件的候选图形作为所述第一/第二候选图形；
    其中，所述候选图形所包含的子图对应于该候选图形保存。
  7. 如权利要求3所述的交互方法，其中，所述第一特征信息或第二特征信息包括所述轨迹所形成的图形的轮廓线特征；
    所述获取与所述第一/第二特征信息匹配成功的第一/第二候选图形包括：
    分别计算每个候选图形的轮廓线与所述第一/第二特征信息中轮廓线特征之间的第三相似度；
    选择所述第三相似度满足第三预设条件的候选图形作为所述第一/第二候选图形。
  8. 如权利要求1所述的交互方法，其中，所述第一候选对象包括第一候选指令，所述第二候选对象包括第二候选指令；
    所述响应于用户的对所述第一选择图标的选择操作，执行所述第一候选对象所对应的操作，响应于用户的对所述第二选择图标的选择操作，执行所述第二候选对象所对应的操作，包括：
    响应于用户的对所述第一选择图标的选择操作，执行所述第一候选指令所对应的操作，响应于用户的对所述第二选择图标的选择操作，执行所述第二候选指令所对应的操作；
    所述根据轨迹点集合确定候选对象，包括：
    将所述轨迹点集合所形成的轨迹识别成标准轨迹，根据所述标准轨迹获取不同的候选指令，分别作为所述第一候选指令和第二候选指令。
  9. 如权利要求1所述的交互方法，其中，所述第一候选对象是第三候选图形，所述第二候选对象是第三候选指令；
    所述响应于用户的对所述第一选择图标的选择操作，显示所述第一候选对象，响应于用户的对所述第二选择图标的选择操作，执行所述第二候选对象所对应的操作，包括：
    响应于用户的对所述第一选择图标的选择操作，显示所述第三候选图形，响应于用户的对所述第二选择图标的选择操作，执行所述第三候选指令所对应的操作；
    所述根据轨迹点集合确定候选对象，包括：
    根据所述轨迹点集合所形成的轨迹获取第三特征信息，获取与所述第三特征信息匹配成功的第三候选图形；
    将所述轨迹点集合所形成的轨迹识别成标准轨迹，根据所述标准轨迹获取所述第三候选指令。
  10. 如权利要求9所述的交互方法，其中，所述根据轨迹点集合确定候选对象还包括：
    响应于将所述轨迹点集合形成的轨迹识别成标准轨迹失败，根据所述轨迹点集合所形成的轨迹获取第四特征信息，获取与所述第四特征信息匹配成功的第四候选图形作为所述第二候选对象；所述第四特征信息不同于所述第三特征信息。
  11. 如权利要求1所述的交互方法，其中，所述交互方法应用于书写软件；所述获取用户输入的轨迹点集合包括：
    响应于用户在所述书写软件的输入界面中的输入操作，获取用户在所述输入界面中输入的轨迹点集合；
    所述显示用于表示所述第一候选对象的第一选择图标和用于表示所述第二候选对象的第二选择图标包括：
    在所述输入界面中显示用于表示所述第一候选对象的第一选择图标和用于表示所述第二候选对象的第二选择图标；
    所述响应于用户的对所述第一选择图标的选择操作，显示所述第一候选对象或者执行所述第一候选对象所对应的操作，响应于用户的对所述第二选择图标的选择操作，显示所述第二候选对象或者执行所述第二候选对象所对应的操作包括：
    响应于用户在所述输入界面中，对所述第一选择图标的选择操作，在所述输入界面中显示所述第一候选对象，或在所述书写软件中执行所述第一候选对象所对应的操作；
    响应于用户在所述输入界面中，对所述第二选择图标的选择操作，在所述输入界面中显示所述第二候选对象，或在所述书写软件中执行所述第二候选对象所对应的操作。
  12. 如权利要求1-11中任一项所述的交互方法，其中，所述显示用于表示第一候选对象的第一选择图标和用于表示第二候选对象的第二选择图标包括：第一候选对象为候选图形，以第一候选对象的缩略图作为第一选择图标进行显示；或者，第一候选对象为候选指令，以第一候选对象的描述信息作为第一选择图标进行显示；
    第二候选对象为候选图形，以第二候选对象的缩略图作为第一选择图标进行显示；或者，第二候选对象为候选指令，以第二候选对象的描述信息作为第二选择图标进行显示。
  13. 如权利要求1-11中任一项所述的交互方法，其中，所述显示用于表示第一候选对象的第一选择图标和用于表示第二候选对象的第二选择图标后还包括：
    响应于用户的预定操作，更新所确定的第一候选对象和/或第二候选对象；
    响应于第一候选对象的更新，相应更新所显示的第一选择图标；响应于第二候选对象的更新，相应更新所显示的第二选择图标。
  14. 如权利要求1-11中任一项所述的交互方法，其中，所述获取用户输入的轨迹点集合包括：
    响应于用户的输入操作停顿，显示预设控件；其中，检测到用户的输入操作停顿是指：距离上一次收到对于输入操作的采样结果的时间长度达到或超过预设时长阈值；
    响应于用户对于所述预设控件的操作，获取本次检测到用户的输入操作停顿和前一次检测到用户的输入操作停顿之间，用户输入操作所对应的轨迹点，保存为所述轨迹点集合。
  15. 一种电子设备，包括：存储器和处理器；
    所述存储器设置成保存进行交互的程序；
    所述处理器设置成读取执行所述进行交互的程序，并进行如权利要求1-14中任一项所述的交互方法。
  16. 一种计算机存储介质，设置成保存进行交互的程序；所述进行交互的程序在被执行时，进行如权利要求1-14中任一项所述的交互方法。
PCT/CN2021/096925 2021-05-28 2021-05-28 一种交互方法、电子设备及计算机存储介质 WO2022246834A1 (zh)

Priority Applications (3)

Application Number Priority Date Filing Date Title
US17/765,832 US20240046682A1 (en) 2021-05-28 2021-05-28 Interaction Method, Electronic Device and Computer Storage Medium
PCT/CN2021/096925 WO2022246834A1 (zh) 2021-05-28 2021-05-28 一种交互方法、电子设备及计算机存储介质
CN202180001356.5A CN115701298A (zh) 2021-05-28 2021-05-28 一种交互方法、电子设备及计算机存储介质

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
PCT/CN2021/096925 WO2022246834A1 (zh) 2021-05-28 2021-05-28 一种交互方法、电子设备及计算机存储介质

Publications (1)

Publication Number Publication Date
WO2022246834A1 true WO2022246834A1 (zh) 2022-12-01

Family

ID=84228364

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2021/096925 WO2022246834A1 (zh) 2021-05-28 2021-05-28 一种交互方法、电子设备及计算机存储介质

Country Status (3)

Country Link
US (1) US20240046682A1 (zh)
CN (1) CN115701298A (zh)
WO (1) WO2022246834A1 (zh)

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101339487A (zh) * 2008-08-29 2009-01-07 飞图科技（北京）有限公司 一种基于手持设备的依靠快捷图形识别调用功能的方法
US20090222770A1 (en) * 2008-02-29 2009-09-03 Inventec Appliances Corp. Method of inputting control instruction and handheld device thereof
CN101551720A (zh) * 2008-04-02 2009-10-07 英华达股份有限公司 手持设备输入控制指令方法及其输入控制指令的手持设备
CN103246437A (zh) * 2012-02-03 2013-08-14 宇龙计算机通信科技(深圳)有限公司 切换输入法的方法和终端
CN104077062A (zh) * 2013-03-25 2014-10-01 联想(北京)有限公司 输入控制方法和装置
CN109213412A (zh) * 2017-07-03 2019-01-15 北京搜狗科技发展有限公司 输入方法、装置和机器可读介质

Family Cites Families (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US9898187B2 (en) * 2013-06-09 2018-02-20 Apple Inc. Managing real-time handwriting recognition
JP2015170150A (ja) * 2014-03-07 2015-09-28 株式会社東芝 電子機器及び方法
JP2020064625A (ja) * 2018-10-15 2020-04-23 株式会社リコー 入力装置、入力方法、プログラム、入力システム

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20090222770A1 (en) * 2008-02-29 2009-09-03 Inventec Appliances Corp. Method of inputting control instruction and handheld device thereof
CN101551720A (zh) * 2008-04-02 2009-10-07 英华达股份有限公司 手持设备输入控制指令方法及其输入控制指令的手持设备
CN101339487A (zh) * 2008-08-29 2009-01-07 飞图科技（北京）有限公司 一种基于手持设备的依靠快捷图形识别调用功能的方法
CN103246437A (zh) * 2012-02-03 2013-08-14 宇龙计算机通信科技(深圳)有限公司 切换输入法的方法和终端
CN104077062A (zh) * 2013-03-25 2014-10-01 联想(北京)有限公司 输入控制方法和装置
CN109213412A (zh) * 2017-07-03 2019-01-15 北京搜狗科技发展有限公司 输入方法、装置和机器可读介质

Also Published As

Publication number Publication date
US20240046682A1 (en) 2024-02-08
CN115701298A (zh) 2023-02-07

Similar Documents

Publication Publication Date Title
RU2702270C2 (ru) Обнаружение выбора рукописного фрагмента
US9665259B2 (en) Interactive digital displays
US7256773B2 (en) Detection of a dwell gesture by examining parameters associated with pen motion
WO2017185575A1 (zh) 一种触摸屏轨迹识别方法及装置
US20110246875A1 (en) Digital whiteboard implementation
US20180067640A1 (en) Formula inputting method and apparatus
JP2003303047A (ja) 画像入力及び表示システム、ユーザインタフェースの利用方法並びにコンピュータで使用可能な媒体を含む製品
US20150154444A1 (en) Electronic device and method
US20150123988A1 (en) Electronic device, method and storage medium
CN103218160A (zh) 人机交互方法及终端
US20140298223A1 (en) Systems and methods for drawing shapes and issuing gesture-based control commands on the same draw grid
CN114365075B (zh) 用于选择图形对象的方法和对应装置
US11144196B2 (en) Operating visual user interface controls with ink commands
US20140270529A1 (en) Electronic device, method, and storage medium
US20160147436A1 (en) Electronic apparatus and method
CN107977155B (zh) 一种手写识别方法、装置、设备和存储介质
US11972097B2 (en) Selecting content in ink documents using a hierarchical data structure
US9811238B2 (en) Methods and systems for interacting with a digital marking surface
WO2022246834A1 (zh) 一种交互方法、电子设备及计算机存储介质
JP4148867B2 (ja) 筆跡処理装置
US20150067592A1 (en) Methods and Systems for Interacting with a Digital Marking Surface
CN115793923B (zh) 一种人机界面运动轨迹识别方法、系统、设备及介质
WO2023000960A1 (zh) 一种电子白板及其图表显示的方法
CN115129218A (zh) 基于书写轨迹进行内容显示的方法、装置和存储介质

Legal Events

Date Code Title Description
WWE Wipo information: entry into national phase

Ref document number: 17765832

Country of ref document: US

121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 21942408

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE