CN104820560A

CN104820560A - Method for selecting character or image and computation equipment

Info

Publication number: CN104820560A
Application number: CN201510259252.0A
Authority: CN
Inventors: 马兰英
Original assignee: Individual
Current assignee: Individual
Priority date: 2015-05-21
Filing date: 2015-05-21
Publication date: 2015-08-05
Also published as: CN105320437A

Abstract

The invention discloses a method for selecting a character or an image and computation equipment. The computation equipment is enabled to execute the following processing: obtaining and recording information relevant to a track indicated on a display by a user, wherein the information comprises position information; judging whether the track has intersections or not, if the track does not have the intersections, judging that the whole track is closed or of a V shape, dividing the track into multiple sections by the intersections if the track has the intersections, and judging whether the each track section is closed or of the V shape or not by aiming at each segmented track; generating a triangular closed area by aiming at the track which is judged to be of the V shape on the basis of the vertex and two end points of the V-shaped track; and if the track is judged to have a closed part or a V-shaped part in shape judgment processing, judging whether the character or the image is selected or not according to the closed area surrounded by the closed track and a relative position relationship between the closed area generated in the generation processing of the closed area and the character or the image.

Description

Select method and the computing equipment of character or image

Technical field

The present invention relates to man-machine interface and computing equipment, in particular to selecting the method for character or image by gesture and employing the computing equipment of this system of selection.

Background technology

User is carrying out in reciprocal process with the computing equipment such as smart mobile phone, often needs to select the character of viewing area display or image etc.In the prior art, user can use the input equipments such as the input equipment of computing equipment such as touch-screen, felt pen (stylus), laser felt pen, infrared touch pen to indicate selected zone intuitively, to select character shown on viewing area or image.

In patent documentation 1, propose the method for following selection text: computing equipment identification user starts the gesture of selection operation, scope specified by the gesture that user starts after selection operation selects text, and user modifies to range of choice by the method such as border changing range of choice.

Patent documentation 1: US Patent No. 12/042,313

Summary of the invention

The problem that invention will solve

Now widely used with in the computing equipment such as smart mobile phone, panel computer, user is when using these computing equipments, and make the to use gesture character that points to intuitively and show in the viewing area of computing equipment or image etc. carry out selection operation.Herein, the pressure etc. when gesture refers to the shape of the track that user indicates with contact or non-contacting mode by finger, felt pen or laser designator, speed (pauses) and indicates gesture with the way of contact.Mainly carry out in the computing equipment operated based on gesture user, there are the following problems in system of selection of the prior art:

First, when selecting character or image etc., needing at least to carry out starting the operation selected and the operation steps of specifying range of choice, namely making multiple gesture (such as, multiple point touching, pressing pause certain hour, instruction track etc.), there is the problem of complex operation.In addition, owing to there is multiple operation steps, therefore, user needs learning and memory for the different operating gesture of different steps.In addition, owing to there is multiple operation steps, user needs to watch viewing area attentively, and carry out next one operation to determine whether to complete an operation, such as, user needs to determine whether to enter preference pattern, then alternative.Namely, user needs cost certain hour to start selection operation, and user also needs at least to judge the operation etc. of when start instruction range of choice, and therefore, user cannot complete selection operation swimmingly, without a break.

The second, when judging to select character or image etc., accurately must indicate range of choice, otherwise there is the problem that the character selected or image do not meet the wish of user.Such as, when selecting text, existing system of selection is selection text from the position that user specifies, until the position of operation is selected in the end that user specifies.And due to the display screen of the handheld devices such as mobile phone less, display interval between character is less, user is often not easy the scope accurately indicating selection to operate, such as, be easy to range of choice to be assigned to the character and previous character or a rear character that will select.In this case, user needs to modify to range of choice.

3rd, there is the problem selecting character inconsistent with selecting the mode of operation of image etc.In computing equipment, the operands such as application program, game object, file often do not show in the mode of text, and usually show in the mode of image.In the prior art, for character and image, often adopt different systems of selection.Such as, as shown in Figure 1A, when selecting text, specifying and selecting initial sum final position.Such as, P1 and P2 in Figure 1A indicates the range of choice of text, and as shown in Figure 1B, user by selecting frame have selected 3 images.

For the means of dealing with problems

For above problem, the present invention proposes following technical scheme.

According to the present invention, provide a kind of method of selection character of being performed by computing equipment or image, described computing equipment is shown by display, and the feature of described method is, described method makes computing equipment perform following process:

Trace information recording processing: obtain and record the information relevant to the track that user indicates on the display, described information comprises positional information;

Shape decision process: judge whether described track exists intersection point, if there is no intersection point, whether whether the shape then judging whole track closed or as V-shaped, if there is intersection point, then utilize described intersection point that described track is divided into multistage, determine whether closed for each section of described track after segmentation or whether be V-shaped;

Enclosed region generating process: for the described track being judged to be V-shaped, based on summit and two end points of the described track of this V-shaped, generates leg-of-mutton enclosed region; And

Select determination processing: if be judged to be the part that there is closed part or there is V-shaped in described shape decision process, then the enclosed region surrounded according to described closed track and the relative position relation of the described enclosed region generated in described enclosed region generating process and described character or image determine whether to select this character or image.

2, the selection character performed by computing equipment according to technical scheme 1 or the method for image, wherein, all right,

Also carry out correction process to described track, described correction process comprises:

Described trajectory corrector existence be interrupted is continuous;

To described track smoothingization process;

It is approximate specification geometric configuration by described trajectory corrector;

Expand or reduce the length on limit of described track or correction V-shaped track; And

When the diameter of closed part is less than defined threshold, this closed part is regarded as non-closed part and removes, and being continuous by the partial correction of described removal.

It is 3, according to the method for the selection character performed by computing equipment in technical scheme 1 ~ 2 described in any one or image, wherein, all right,

Described track is indicated with contact or non-contacting mode on the display by user,

When selecting character or image according to described enclosed region, the character cover described closed part completely or image are as alternative, and the relative position relation of the character covered according to described closed part and non-fully or image selects described character or image.

It is 4, according to the method for the selection character performed by computing equipment in technical scheme 1 ~ 3 described in any one or image, wherein, all right,

Described relative position relation can be evaluated according to level of coverage and departure degree, and wherein, described level of coverage comprises area covering degree, length level of coverage, wherein, all right,

Described area covering degree comprises absolute area level of coverage and relative area level of coverage,

Described length level of coverage comprises absolute growth level of coverage and relative length level of coverage,

Described departure degree comprises absolute departure degree and relative departure degree,

Described character or image is selected according at least one in described absolute area level of coverage, described relative area level of coverage, described absolute growth level of coverage, described relative length level of coverage, described absolute departure degree and described relative departure degree.

It is 5, according to the method for the selection character performed by computing equipment in any one in technical scheme 1 ~ 4 described in any one or image, wherein, all right,

After have selected described character or image, described display is shown in the mode of the described character selected or image and non-selected character or image area being separated,

When being judged to be that described within the predetermined time user does not carry out any operation, cancel the selection to described character or image, described display shown with described character or the non-selected mode of image,

Under being judged to be that described within the predetermined time user indicates the described track situation comprising closed part or V-shaped again, select character or image according to the described track again indicated further.

It is 6, according to the method for the selection character performed by computing equipment in technical scheme 1 ~ 5 described in any one or image, wherein, all right,

When the described character selected or image belong to continuous print multiline text, select the full line text at this character or image place,

Wherein, described text is made up of multiple described character and described image.

It is 7, according to the method for the selection character performed by computing equipment in technical scheme 1 ~ 6 described in any one or image, wherein, all right,

When have selected image, extract the subimage in this image, what show this image reduces all or part of of regional area that figure and/or this image surrounded by described enclosed region and/or described subimage.

It is 8, according to the method for the selection character performed by computing equipment in technical scheme 1 ~ 6 described in any one or image, wherein, all right,

According to the property value of the attribute of the described character selected or image, increase or reduce the described character or image selected.

9, according to the present invention, provide a kind of computing equipment, it is shown by display, and the feature of this computing equipment is to comprise as lower module:

Trace information record processing module: it obtains and records the information relevant to the track that user indicates on the display, and described information comprises positional information;

Shape decision processing module: it judges whether described track exists intersection point, if there is no intersection point, whether whether the shape then judging whole track closed or as V-shaped, if there is intersection point, then utilize described intersection point that described track is divided into multistage, determine whether closed for each section of described track after segmentation or whether be V-shaped;

Enclosed region generating process module: it is for the described track being judged to be V-shaped, based on summit and two end points of the described track of this V-shaped, generates leg-of-mutton enclosed region; And

Select determination processing module: if be judged to be the part that there is closed part or there is V-shaped in described shape decision process, then the relative position relation of the enclosed region that surrounds according to described closed track of this selection determination processing module and the described enclosed region generated in described enclosed region generating process and described character or image determines whether to select this character or image.

With upper module, can realize with the combination of hardware, software or hardware and software.

10, according to the present invention, provide a kind of computing equipment, it is shown by display, and the feature of this computing equipment is to perform following process:

11, according to the present invention, provide a kind of program, it makes computing equipment perform following process:

Invention effect

Pass through the present invention; user is considered as selection operation with finger, stylus or laser designator etc. towards the closed trajectory of character or image instruction or V shape track intuitively over the display; the operation starting operation and the formulation opereating specification selected is united two into one; user is by an operation; selection can be indicated to operate and instruction range of choice; simplify user operation, improve user friendliness.User learning, without the need to remembering multiple mode of operation, operates, without the need to deliberately noticing the transformation of pattern without the need to deliberately separating multiple step when selecting character or image.In addition, in the present invention, increase and decrease chosen content according to by the attribute of the character selected or image, make the wish more being met user by the operand selected.In addition, identical selection mode can be used to character and image, make user not need learning and memory multiple choices mode of operation, decrease the work of learning and memory, improve user friendliness.

Accompanying drawing explanation

Figure 1A is the schematic diagram that the operation selecting text is in the prior art shown.

Figure 1B is the schematic diagram that the operation selecting image is in the prior art shown.

Fig. 2 A, Fig. 2 B, Fig. 2 C, Fig. 2 D are the figure of the relative position relation of the viewing area that track and character or image are shown.

Fig. 3 is the figure of the example that V-shaped track is shown.

Fig. 4 is the figure that two end points of V-shaped track and the example on summit are shown.

Fig. 5 is the figure of the example that the closed trajectory with interruption is shown.

Fig. 6 is the block diagram of the formation that computing equipment 100 is shown.

Fig. 7 is the schematic appearance figure that computing equipment 100 is shown.

Fig. 8 is the process flow diagram that the bulk treatment selecting character or image is shown.

Fig. 9 A, Fig. 9 B, Fig. 9 C are the figure that the example selecting character or image according to closed trajectory is shown.

The figure of Figure 10 A to be figure, Figure 10 B of the example that V-shaped track is shown be example illustrated according to V-shaped Track Pick-up enclosed region.

Figure 11 A, Figure 11 B are the figure that the example selecting multiline text is shown.

Figure 12 A, Figure 12 B are the figure of the example illustrated about subimage etc.

Figure 13 A, Figure 13 B illustrate that the attribute according to the content selected increases/reduce the figure of the example of object.

Figure 14 is the figure of the example that Continuous Selection 2 objects are shown.

Figure 15 is the figure of the example that Continuous Selection 3 objects are shown.

Figure 16 illustrates that the attribute of the content selected is to revise the figure of the treatment step of range of choice.

Figure 17 illustrates that the figure of the example of regional area is selected in display.

Figure 18 illustrates the figure about the example about regional area.

Figure 19 is the figure of the example that the computing equipment indicating track is in a non contact fashion shown.

Embodiment

Embodiment 1

Below, with reference to accompanying drawing, embodiments of the present invention are described in detail.

In addition, in the present description and drawings, identical label is marked for the inscape with the identical functional structure of essence, repeats thus to omit the description.

In addition, in the present description and drawings, after identical label, multiple inscapes with the identical functional structure of essence distinguished in additional different letter sometimes.But, when not needing special differentiation to have multiple inscape of the identical functional structure of essence, identical label is only marked.In addition, content shown in accompanying drawing is only for schematically illustrating, and shown ratio is with actual and inconsistent.

First, the existing knowledge used when judging position relationship is briefly described.

When selecting character or image, user indicates the character or image that will select over the display, compares, determine whether the object that it can be used as and will select thus to the track of user's instruction and the indication range of character or image.In the following description, the track sometimes user indicated in the viewing area of computing equipment is referred to as track or select track.Selecting track to have starting point and terminal, below, when not distinguishing both, being sometimes simply called the end points of end points or selection track.This selection track can be closed, also can be that V-shaped is open.

Fig. 2 A, Fig. 2 B, Fig. 2 C, Fig. 2 D illustrate the relative position relation of the viewing area of track and character or image, and wherein, dotted line A1 represents the selection track that user indicates, and solid line B1 represents the indication range of object.Fig. 3 shows the example of V-shape track, and wherein, dotted line A1 represents the selection track that user indicates, and solid line B1 represents the indication range of object.

User selects in the process of track in instruction because point of short durationly to frame out, laser designator fault, the reason such as touch sensor sensitivity, the track of gained likely becomes to exist and is interrupted.Such as, as shown in Figure 5, the end points of track is T3, T4, selects track cross and close, but on this track, due to touch sensor sensitivity, has place disconnection, if adjacent 2 of track gap are T5, T6.In this case, as long as the ratio of the ultimate range be less than between the distance between defined threshold or and any two points on closed dotted line in the distance of track gap adjacent 2 between T5, T6 is less than fixed threshold, then track can be considered as continuous print at these 2.For can be considered the track that continuous print is interrupted, straight-line segment can be adopted to select 2 connection of gap of track, or trajectory corrector, by 2 connections, is continuous print track by employing curve fitting method, interpolation method etc. thus.

Whether close about selection track, judge by following method: obtain the distance between the end points selecting track, if this distance is less than defined threshold, be then judged to select track to be closed.This threshold value suitably can set according to the resolution of display.

Whether close about selection track, also judge by following method: obtain the distance between the end points selecting track, obtain the ultimate range between any two points on the maximum gauge of track, i.e. track, if the distance between end points is less than defined threshold with the ratio of above-mentioned maximum gauge, be then judged to select track to be closed.This threshold value suitably can set according to the resolution of display.

About selecting whether track is V-shaped, judge by following method: as shown in Figure 4, if two end points selecting track are T1, T2, two end points using two end points T1, T2 as line segment are made line segment, on selection track, this line segment of distance point is farthest as summit, if this summit is T0.If the track between summit T0 and end points T1 is near linear, track between summit T0 and end points T2 is near linear and is less than defined threshold with the line segment that T0 and T1 is end points with the line segment angulation being end points with T0 and T2, be then judged to be that track is V-shaped.This angle threshold suitably can set according to reality.In addition, other method also can be adopted to determine whether V-shaped.

About selecting whether track is near linear, judge by following method: two end points based on track are made line segment, if the length ratio of each point on selection track and the minimum distance of this line segment and line segment is less than predetermined threshold, be then judged to select track to be near linear.This threshold value suitably can set according to reality.In addition, other method also can be adopted to determine whether near linear.

About the length of track between two-end-point, by such as under type calculating: on track, get the multiple points between two-end-point, calculate the length sum of adjacent 2 line segments formed as the course length between end points.Point for calculating is more, then precision is higher, but calculated amount is also larger.Precision and calculated amount can be weighed and suitably determine the quantity of the point of computational length.Also the air line distance between two-end-point can be used as course length between two-end-point approx.

When selecting track closed, under selecting the position relationship of the indication range of track and object to comprise following situation: as shown in Figure 2 A, track is selected to cover the indication range of object completely; As shown in Figure 2 B, the indication range of track local complexity object is selected; As shown in Figure 2 C, track local is selected not overlap completely with the indication range of object; And as shown in Figure 2 D, the indication range of object covers selection track completely, track is selected to be contained in completely in the indication range of object in other words.In the following description, if there is no specified otherwise, then cover the situation comprising covering and local complexity completely.

The position relationship between closed trajectory and the indication range of object is judged: if it is determined that each point of the indication range of object is in the scope that closed trajectory marks, be then considered as selecting track to cover the indication range of object completely by following method.If both there is the point belonged in scope that closed trajectory marks in the indication range judging object, there is again the point do not belonged in scope that closed trajectory marks, be then considered as the indication range selecting track local complexity object.If it is determined that each point of the indication range of object does not belong to the scope that closed trajectory marks, be then considered as both and do not overlap completely.

In addition, when the position relationship of a judgement scope and another scope, also by the relation of the borderline point of a scope and the position relationship of another scope, the position relationship between two scopes can be obtained.Such as, the indication range of object is generally rectangle, circular ellipse, when the indication range of object is generally circular, if the borderline point of the indication range of object is all positioned at the scope that closed trajectory marks, then can be judged to be the indication range covering object in the scope that closed trajectory marks completely.Similarly; when the indication range of object is generally the shape such as rectangle of rule; the position relationship of the scope marked by the summit of indication range and closed trajectory that judge object, judges the position relationship between the scope that the indication range of object and closed trajectory mark.

About the position relationship between a point and closed scope, judge by such as under type: obtain the point by judging and the intersection point on the border of orthogonal two straight lines and closed scope, according to the magnitude relationship between obtained intersection point and the coordinate of point that will judge, judge the position relationship between a point and closed scope.Also can judge the position relationship between a point and closed scope in the following way: obtain the straight line of the point by judging and the intersection point number of closed scope, if intersection point number is even number, then this point is in closed scope, otherwise, outside closed scope.In addition, also can adopt other method to judge.

Above, the decision method of the judgement, the relative position relation such as point and scope, scope and scope etc. of shape is illustrated.But, about the judgement such as judgement, relative position relation of shape, be the common practise of plane geometry, in actual applications, other method also can be adopted to carry out the relative position relation of judgement two scopes.

In the following description, under selection track is not closed curve situation, if not otherwise specified, as long as at least one point on track overlaps with the indication range of object, be then judged to select track to cover the indication range of object.

Below, the computing equipment 100 shown in Fig. 6 is used to realize embodiments of the present invention 1.Herein, the example computing equipment 100 as computing equipment is Handheld multimedia players, but also can be other computing equipments, such as smart mobile phone, panel computer etc.Fig. 6 shows the component part of computing equipment 100.Fig. 7 shows the schematic appearance of computing equipment 100.

As shown in Figure 6, in computing equipment 100, be connected with processor 101, the 1st storer 103, the 2nd storer 104, touch display panel 121, peripheral interface 112 by bus 105, and be connected with button 122, microphone 123, loudspeaker 124, telecommunication circuit 125 and storage card reader 126 by peripheral interface 112.These parts are powered by power supply 131.

Bus 105 connects each component part, transmission command and the data of computing equipment 100.Processor 101 is such as made up of at least 1 CPU (central processing unit) or DSP (digital signal processor), and the 1st storer 103 is such as made up of random access memory, the ephemeral data that temporary transient storage program and program produce in running.2nd storer 104 is such as made up of, for keeping data enduringly one or more disk storage device, flash memory device or other non-volatile solid state memory etc.

The input and output peripherals of computing equipment 100 can be coupled to processor 101 and the 1st storer 103 by peripheral interface 112.Processor 101 loads and performs the operating system and various application software that store in the 2nd storer 104, carrys out each component part of controlling calculation equipment, completes various function.Operating system can adopt OSX, WINDOWS or VxWorks etc.Application software such as comprises browser, video player, music player etc.

Microphone 123 is for computing equipment 100 sound import, and loudspeaker 124 is for externally sounding.Touch display panel 121 such as can be made up of the display panels with touch function, has function that to carry out showing under the control of processor 101 and detects the function etc. that user inputs the touch of touch display panel 121.User can use the objects such as finger, felt pen or utensil to contact with touch display panel 121.In certain embodiments, sometimes carry out work according to the contact pointed and gesture, usually comparatively large owing to pointing the area contacted with touch display panel 121, so in order to improve accuracy, felt pen can be used carry out touch input.When user touches on touch display panel 121, computing equipment can obtain touch any point time positional information and temporal information.

Touch display panel 121 to user's indication example as text, webpage, figure (comprising the user interface object etc. of virtual soft dish), digital picture, video, animation etc.From the angle of user, on touch display panel 121, the content of display can be divided into " char " and " image ".Character includes but not limited to: the letter, numeral, symbol (such as@, *, #), Chinese character, Japanese ideogram etc. of various countries.Image comprises the vector graphics using the generations such as Software tool and the picture etc. utilizing camera or obtained by scanner.Video or animation can regard as a series of image.

Power supply 131 is made up of rechargeable battery or transformer.In figure 6, in order to simplify, eliminate the annexation of power supply 131 and each several part.

Button 122 is such as made up of one or more push switch or inductive switch, has carried out on/off power supply, has called the functions such as master menu.

Telecommunication circuit 125 is such as made up of communication device and antenna, and telecommunication circuit 125 makes computing equipment be connected to the network such as LAN (Local Area Network) or internet etc., carrys out the various data of upload and download.

Should be understood that computing equipment 100 is only an example of computing equipment, in actual applications, can add or reduce parts, two or more parts can be combined, or adopting other arrangement.Various parts shown in Fig. 6 can use the combination of hardware, software or hardware and software to realize.

Touch display panel 121 is equivalent to the example of the display of computing equipment.Computing equipment 100 is examples of computing equipment.Be described based on computing equipment 100 below.

Such as, computing equipment 100 is when manufacturing, in the 2nd storer 104, insert the program had selection function described below, or read in by memory card reader and the program of selection function is installed, or downloaded by telecommunication circuit 125 and the program etc. of selection function is installed.Computing equipment 100 has the program of selection function by performing, carried out selection operation, and gives other program or module by selected character or image transfer, has carried out various function.

Namely, such as when user is touched with finger or stylus etc. towards character or image and indicates on touch display panel 121 by gesture mode intuitively, computing equipment 100 detects the instruction (with the contact of display) of user, start the step performed shown in Fig. 8, obtain the information relevant to track, judge the character that will select or image and select.Such as, user is when indicating track by touch, need contact display (touch display panel 121), when user contacts display, touch display panel 121 can produce interruption, computing equipment 100 is such as processed interruption by operating system, by with to touch relevant positional information, temporal information etc. for the message (event) of parameter and pass to application program.Thus, application program can process the information (position, time, pressure etc.) relevant to touch.This is common practise, no longer carries out more specific description.

Below, be specifically described based on following computing equipment 100, composition graphs 8.

In step S05, computing equipment 100 obtains with track-related information and carries out record by being such as kept at the medium mode of the 1st storer 103, and in this example embodiment, described information comprises the positional informations such as coordinate.In addition, user is when such as indicating track with finger touch, the track being formed and there is width is contacted with display, therefore, this such as can be adopted to have the methods such as the mid point of the track of width, form the track as lines be made up of mid point, that is, select process according to carrying out judgement as the track as lines be made up of continuous print point.That is, after, if not otherwise specified, track refers to the track as lines.When user stops instruction track, that is, when can't detect the instruction of user to display (contact), entering step S10 from step S05, for acquired track, carrying out subsequent treatment.As mentioned above, of short durationly to frame out owing to pointing, laser designator fault, the reason such as touch sensor sensitivity, can cause can't detect user and stop indicating track without being identified as user to the instruction of display.In order to avoid these, can suitable setting-up time threshold value, if user then indicates track in time threshold, be then considered as user and do not stop instruction.Otherwise, be then considered as stopping instruction.

At this, correction process can also be carried out to this track, described correction process comprises following process etc.: the method connected by straight line, curve fitting method or interpolation method etc., will exist be interrupted and the trajectory corrector that this interruption is less than defined threshold is continuous, that is, if there is the interruption being greater than defined threshold, then the process of step S10 ~ S35 is carried out respectively for every section of track, in other words, the track processed in step S10 ~ S35 is the track indicated continuously; To the process of track smoothingization, to remove shake; Being approximate specification geometric configuration by trajectory corrector, such as, when user indicates nonstandard circle or rectangle, is circle or the rectangle of specification by trajectory corrector; Expanding or reduce track, such as, when user indicates circular trace, is the circle of a circle slightly larger than this circle or a slightly little circle by this trajectory corrector, or the length on the limit of correction V-shaped track, such as, be all set to the mean value etc. on two limits; And remove the correction process of less closing section.Specifically, if the maximum gauge of part closed on track is less than defined threshold, then this closed part is regarded as non-closed part and remove, and being continuous by the partial correction removed.This threshold value can according to circumstances suitably set.In addition, also the enclosed region being less than regulation area can be considered as Fei Bie and region and carry out removing and correction process.

Can perform one or more correction process, correction process not necessarily, but preferably carries out correction process, also can carry out other correction process by track.After having carried out correction process, the information relevant to the track after correction has been used to carry out subsequent treatment.About information such as positional information, temporal information, the pressure information etc. of the part be corrected of curve, can be obtained by methods such as interpolation methods.

In step slo, according to the trace information recorded in step S05 or the information correcting rear track, determine whether that track exists intersection point, if there is no intersection point, then directly enter step S20, judge the shape of whole track, if there is intersection point, then enter step S15, utilize described intersection point that this track is divided into multistage, then entering step S20, for splitting each section of track obtaining, judging the whether closed of this section of track respectively or whether as V-shaped etc.

In step S20, according to the positional information of track, judge the shape of described track.Specifically, judge whether this section of track exists closed part, or whether be V-shaped.As mentioned above, if there is intersection point, then can be judged to be that track closes, or, if the length between two of track end points is less than for determining whether the threshold value closed, be then judged to be that track closes.When being judged to be to there is closed part in step 20, being considered as user and carrying out selection operation, entering step S30.When being judged to be that in step S20 this section of track is V-shaped, enter step S25, the triangular closed region that to generate with the summit of V-shaped track and two end points be summit.Such as, the dotted portion in Figure 10 B shows the triangular closed region according to V-shaped Track Pick-up.After generation enclosed region, enter step S30.

In step s 30, for each character and the image of current display, the enclosed region surrounded according to described closed track and the enclosed region generated in described enclosed region generating process determine whether to select this character or image, that is, character or image is selected according to closed part with the relative position relation of described character or image.Specifically, for each character and the image of current display, obtain its positional information (coordinate), according to its positional information, judge whether enclosed region covers this character or image completely, the character that selection covers completely or image, and then, for other character or image, the relative position relation according to enclosed region and character or image determines whether to select.

Such as, relative position relation can be evaluated by level of coverage or departure degree etc.Such as, level of coverage comprises area covering degree, length level of coverage.Wherein, area covering degree comprises absolute area level of coverage and relative area level of coverage.Such as can using closed part to the area coverage (i.e. overlapping area) of this character or image as absolute area level of coverage.The normalization of absolute area level of coverage can be obtained relative area level of coverage.Or, can by the area of the area coverage of this character or image divided by the less side in the viewing area of this enclosed region and this character or image, using the relative area level of coverage of this result of calculation as this character or image.

Length level of coverage comprises absolute growth level of coverage and relative length level of coverage.Wherein, can using track to character or the overlay length of image, i.e. track the length in this character or image as absolute growth level of coverage.This absolute growth level of coverage normalization can be obtained relative length level of coverage, or, relative length level of coverage can also be obtained in the following way: for each character or image, using maximum overlay length as maximal cover length, using overlay length and maximal cover length ratio as relative length level of coverage.

Departure degree refers to that character or image depart from the degree of enclosed region, and absolute departure degree or relative departure degree can be adopted to evaluate departure degree.Such as, using character or the center (center of geometric configuration) of viewing area of image and the air line distance on the border of enclosed region as absolute departure degree, absolute departure degree normalization can be obtained relative departure degree.Wherein, if be negative inside enclosed region, if opposite side is just.If that is, the viewing area of character or image center be positioned at inner side, then above-mentioned air line distance is negative, otherwise is just.Like this, the value of absolute departure degree is less, then near this enclosed region.Thus, can threshold value be set, select absolute departure degree to be character and the image of below threshold value.Also additive method can be adopted to evaluate the degree that character or image depart from enclosed region.Such as, when enclosed region is circular, the distance etc. of image and enclosed region geometric center can be adopted.

Such as, following method for normalizing can be adopted: set the variable that will evaluate as x, the maximal value of variable is max, and minimum value is min, and the variable after normalization is x*, then x*=(x-min)/(max-min).Also other method for normalizing can be adopted.

Or can be combined them select character or image according to any one in relative area level of coverage, absolute area level of coverage, absolute growth level of coverage, relative length level of coverage, definitely departure degree and relative departure degree.

Such as, Fig. 9 A shows following situation: show these 12 images of A, B, C, D, I, J, K, L, R, S, T, U, user indicates the track A1 shown in dotted line, and the end points of track A1 is T1, T2, and track A1 is completely not closed.Herein, if the distance between T1, T2 is less than closed decision threshold and is judged to be that track closes.In this example, interpolation method is adopted to be corrected to continuously by track A1.Fig. 9 B shows the continuous print track after correction.Wherein, the area of the viewing area of image J, K, I, L, B, C, A is respectively 1000,1000,800,300,150,100,40 pixels.

In the example shown in Fig. 9 A, when the process by performing step S30 is carried out judging and selecting, be first judged to be that J, K cover completely and it can be used as alternative.And then, for other object, determine whether to select according to relative area level of coverage.At this, using enclosed region to the area coverage of this character or image as absolute area level of coverage.Such as, the absolute area level of coverage of image I, L, B, C, A is respectively 800,300,150,100,40 pixels.

In the example shown, the area coverage of this character or image is calculated relative area level of coverage divided by the area of the less side in the viewing area of this enclosed region and this character or image.Due to, the area of the viewing area of each image is all less than enclosed region, and therefore, what use the area of viewing area calculates relative area level of coverage.Obtain thus, the relative area level of coverage of image I, L, B, C, A is respectively 0.8,0.3,0.15,0.1 and 0.04.Select character or image according to relative area level of coverage, select relative area to cover character or the image of more than threshold value.Therefore, in this example embodiment, if it is 0.3 that relative area covers threshold value, the result therefore performed is: first, selects image J, K of covering completely, and then, select relative area to cover image I, L of more than threshold value.

In addition, also can select character or image according to absolute area level of coverage, namely absolute area level of coverage is character or the image of selecting absolute area to cover more than threshold value.Such as, if it is 300 that absolute area covers threshold value, then execution result is: first, selects image J, K of covering completely, and then, select absolute area to cover image I, L of more than threshold value.In addition, the relative area level of coverage that normalization also can be adopted to obtain is to select character or image.At this, be no longer described.

Similarly, can carry out judging and selecting according to absolute growth level of coverage and relative length level of coverage.Such as, Figure 10 A shows: show N, P, O, Q tetra-images, user indicates the V-shaped track shown in dotted line.In Figure 10 A, such as, suppose that the overlay length of image O, P, N is respectively 200,100 and 20 pixels.Then the absolute growth level of coverage of image O, P, N is respectively 200,100 and 20 pixels, and the relative length level of coverage after normalization is 1,0.44 and 0.When selecting character or image according to absolute growth level of coverage, selection absolute growth level of coverage is character or the image of more than defined threshold.Such as, if set this threshold value as 120 pixels, then select image O.Also can according to relative length level of coverage, selection relative length level of coverage is character or the image of more than defined threshold.Such as, if set this threshold value as 0.6, then select image O in this example.

In actual applications, according to actual conditions, relative area level of coverage, absolute area level of coverage, absolute growth level of coverage, relative length level of coverage, absolute departure degree and relative departure degree can be combinationally used and carry out judging and selecting.Such as, select relative area level of coverage and be greater than certain value and absolute area level of coverage is greater than the character or image etc. of certain value.Or relative area level of coverage is greater than the character or image etc. that certain value or absolute area level of coverage are greater than certain value.Or relative area level of coverage is greater than certain value, absolute area level of coverage is greater than certain value and absolute growth level of coverage is greater than the character or image etc. of certain value.

After determined the character or image that will select by said method, implement to select operation to described character or image.Such as realize selecting operation by following means: the handle of image J, K, I, L is passed to the module to selecting character or image to process or application program, or, the pointer pointing to the address of image J, K, I, L in internal memory is passed to module or application program.In addition, image can be the link pointing to file etc., and the module or the application program that receive image carry out subsequent treatment to the practical object pointed by image.This does not relate to flesh and blood of the present invention, is no longer described this.

In addition, for user, whether character or image are selected, and normally show in visual mode.Therefore, after selecting an object, change the display mode of display, with make user can visually by select character or image separate with non-selected character or image area.

Such as, as shown in Figure 9 C, can represent by the character selected or image with have frame and the highlighted region shown in dot-and-dash line.User can adjust frame to change selected zone.Fig. 9 C is only example.Can adopt and represent to user the content selected in various manners.Such as frame can adopt the various shapes such as various dotted line, solid line, wave.The frame of various thickness can be adopted, highlightedly can adopt shades of colour, can only adopt frame or only adopt highlighted mode.Frame and highlighted can be static, also dynamically can change the thickness of frame or highlighted color etc.In addition, when user indicates track, show the track of client's instruction in modes such as lines, make the display of track keep certain hour, then make the display of track disappear.Also can expand or show the character or image etc. selected with reducing.

Above, be illustrated for image, also can be selected character or the formatted text etc. that is made up of character and image by same method.

Known by illustrating above, according to the present embodiment, following effect can be obtained: when user indicates closed trajectory or V-shaped track, be considered as user and carry out selection operation, and select character or image according to track.Thus, user when selecting, without the need to first carry out being switched to preference pattern operation after select, only need indicate selection track, selection operation can be completed.Thus, decrease operation steps, save the running time, and can once select multiple object.In addition, owing to decreasing operation steps, decrease the operating gesture that user needs learning and memory, improve user experience.In addition, user operates, without the need to deliberately noticing the transformation of pattern without the need to deliberately separating multiple step when selecting character or image.

In addition, due in the computing equipment with touch display panel, usually by clicking or touch the destination jumping to hyperlink and point to.In the method for existing selection, if just wish to select text or image and do not wish redirect, must operate very carefully to avoid, because selection operation, redirect occurs.On the other hand, in the present embodiment, when selecting with objects such as the hyperlink of text (that is, multiple character) form performance, can, by described above, be undertaken judging and selecting by closed trajectory.Like this, select track directly not contact with selection character or image, avoid and cause maloperation because of touching selection character or image.In addition, the mode of the text selected by instruction V-shaped mode selects text, owing to there is not clicking operation, also can easily avoid above-mentioned maloperation.

In addition, compared with prior art, save and open preference pattern by touch screen certain hour or in conjunction with operations such as other buttons, say from user perspective, by drawing a circle smoothly or drawing hook operating gesture, selection can be completed, improve Consumer's Experience.In other words, so-called smooth and easy, refer to that user indicates the process of track to be basically identical, or can say with roughly consistent speed or with roughly the same acceleration to complete instruction, do not deposit the pause caused because operation is selected in instruction in the prior art.Feel smoothly, be also embodied in user and whether enter preference pattern without the need to noting as prior art.Such as, in the prior art, the operation that some instructions are selected inherently needs user's certain time (such as, Continued depression), or, user needs spended time to think deeply and observes whether enter preference pattern (such as, according to whether occurring that text selecting scope etc. determines whether to enter preference pattern) etc. when selecting, and causes the operation of user to be exist to pause.And in the present embodiment, user only pays close attention to the character or image that need to select, the difference of operation and the operation of instruction range of choice being used to indicate selection (namely entering preference pattern) can be ignored.Therefore, by each embodiment of the present invention and variation thereof, client can be made to carry out selection operation swimmingly, significantly improve Consumer's Experience.

Variation 1

When user have selected character or image, if be the character in continuous print multiline text by the character selected or image, then will character or image spreading be selected to be the full line text at the character place selected.

Such as, as shown in Figure 11 A, dotted line represents selection track.According to this track, according to above-mentioned decision method, have selected character B, C, D, E, F.Owing to by character B, C, D, E, F of selecting being the character in the multiline text arranged continuously up and down, therefore, range of choice is expanded to the full line text at the character place selected.Consequently, when user indicates the track shown in Figure 11 A, have selected multiline text like that as shown in Figure 11 B, in Figure 11 B, by dot-and-dash line, the final content selected is shown.Figure 11 A, Figure 11 B show the example of closed trajectory, for V-shaped track, also can adopt and use the same method to expand chosen content.Thus, user can select multiline text by shirtsleeve operation.

If it is noted that when operated text is the formatted texts such as word file, html web page, text comprises character and picture (that is, image).In this case, also multiline text can be selected according to above-mentioned extended method.And if this style of writing originally comprises character and picture, then the picture in this row is also selected.

Variation 2

In above embodiment, only select character or image according to position relationship, in this embodiment, also correct range of choice according to by the attribute of the character selected or image, specifically, increase or reduce the character or image selected.

Method according to Figure 16, corrects range of choice, specifically, judges whether this character belongs to and finally selected from the character of the character string beginning selected according to track.In step S101, for each property value of each attribute, obtain the property value that there is the maximum character of same alike result value and have, it can be used as " selection property value ".In step 105, select to judge that the variable n of number of times is initialized as 1 by being used for counting.In step s 110, obtain the character that will judge, at this example, obtain n-th character from the starting of the character string selected according to track.In the step s 120, judge whether this character has " selection property value ".If this character has " selection property value ", then terminate to correct range of choice, if this character does not have " selection property value ", then enter step S130, this character is set to non-selection character.In step S135, n is made to add 1.In step S140, judge whether n is less than judgement number of times, if be less than judgement number of times, then proceeds to step S110.If more than the judgement number of times for regulation, namely carry out the judgement judging more than number of times, then terminated to correct range of choice.About judgement number of times, suitably can set according to reality.

In the method shown in Figure 16, from the character of character string beginning, judge whether character belongs to and finally selected.Can also judge from character string end, in this case, in step s 110, obtain selected character string the n-th character reciprocal, the process shown in all the other step with Figure 16 is identical.

Described attribute and property value comprise as inferior situation (wherein, left side is attribute, and right side is property value):

Whether be hyperlink: hyperlink, non-hyperlink;

Whether be numeral: numeral, non-link;

Character types: letter, symbol (as #, * etc.), multibyte character (Chinese character) etc.;

Font name: " Gauss's body ", " Minliu body ", " Song typeface " etc.;

Whether be boldface letter: boldface letter, non-boldface letter;

Whether there is underscore: underscore, without underscore;

Whether be italics: italics, non-italics.

In addition, other attribute can be increased according to actual conditions, or balance calculated amount and calibration result and suitably reduce used attribute.In addition, different weights can be given to attribute, when " selection property value " exists multiple, select weight larger " selection property value " to compare.

Such as, as shown in FIG. 13A, user indicates the indicating area shown in dotted line for shown text.First, according to the system of selection of above-mentioned position-based relation, determine that the character string selected is " to:abc@def.com ".To this character string, obtain the attribute that there is the maximum character of same alike result value and have.In this embodiment, because " abc@def.com " all has the property value of " being hyperlink ", therefore, will " being hyperlink " conduct " selection property value ".According to the method shown in Figure 16, because " to: " in " to:abc@def.com " does not have the attribute of " being hyperlink ", thus by " to: " as non-selection character, the final character string selected is " abc@def.com ".

Figure 16 shows and reduces selected character or the example of image according to attribute, similarly, also by judging that the attribute of object expands selected character or image.In situation, if processed from the beginning of character string, then in step s 110, the n-th character before obtaining character string is to judge its attribute.If processed from the afterbody of character string, then in step s 110, the n-th character after obtaining character string is to judge its attribute.Process shown in all the other step with Figure 16 is identical.

Such as, as shown in Figure 13 B, user indicates the indicating area shown in dotted line for shown text, first, according to the system of selection of above-mentioned position-based relation, selects character string " 23456789 ", and " selection property value " is " being numeral ".According to the method illustrated based on Figure 16, judge the character before and after character string.Due to before and after the character that obtains be respectively 1 and 0, this 1 and 0 and the attribute of character string of current selection be all numeral, therefore, continue to extend as selection character or image using 1 and 0.Next the character obtained and space and ", ", attribute is different from the attribute of the character string of current selection, stops extension.Finally, have selected character string " 1234567890 ".

Thus, when accurately can not indicate track because of the less grade in viewing area, the content meeting user intention can also be selected.

Variation 3

In the above description, after have selected object, indicate by frame or the mode such as highlighted the content selected.In this variation, after user have selected object, if do not do any operation in predetermined time threshold value, then cancel selection.

Such as, after user have selected multiple image by closed trajectory, indicate by frame etc. the image selected.Detect user in predetermined time threshold value whether again to operate.If user does not operate in time threshold, then cancel the selection to image, be used in the disappearances such as the frame of instruction selection, thus, revert to the state before selection operation.

In addition, the different time thresholds for cancelling can being set for different type of gesture, such as, the disappearance of V-shaped track can be made faster than closed trajectory.Such as, the above-mentioned time threshold being used for free track can be set to the above-mentioned time threshold be less than for closed trajectory.Its reason is, closed trajectory mostly is the conscious operation of user, and V-shaped track is likely maloperation.But, these are only example, suitably can set according to time situation for the time threshold cancelling selection.

By the time threshold of setting for cancelling selection, make user when maloperation mark select track, by waiting for the shorter time, without the need to carrying out cancellation operation, can selection be cancelled.Thus, the number of times that client carries out cancellation operation can be reduced, improve the convenience of user.

In addition, when user indicates multiple track continuously, as long as the interval between twice instruction operation of being separated by is less than or equal to predetermined time threshold value, then keeps from the track of most start instruction, namely carry out continuous several times selection.In addition, if the predetermined time the last time after instruction operation does not do any operation in threshold value, then all operations is before this cancelled.Such as, if predetermined time, threshold value was 2 seconds, then user is after indicating the 1st track, if again indicated the 2nd track within 2 seconds, and indicates the 3rd track within after the 2nd track 2 seconds, then finally select character or image according to 3 tracks.

Such as, as shown in figure 14, first user indicates the closed trajectory shown in dotted line, at the appointed time, again indicates the V-shaped track shown in dotted line.Under these circumstances, owing to indicating the time interval of former and later two tracks to be less than predetermined time threshold value, therefore, character or image is selected according to two tracks.Thus, the multiple objects can freely selecting to be positioned at the position be separated by different tracks shape are optimized.

In addition, can make according to repeatedly indicating the display mode of the content selected different, such as, the character selected according to the 1st track or image represented by dashed line, the character selected according to the 2nd track or image dot-and-dash line represent.Similar degree, Figure 15 shows the example of Continuous Selection 3 objects, thus, is convenient to user and distinguishes multiple selection operation.

Variation 4

In this variation, be handled as follows: if selected image, then image is resolved, identify the subimage extracted in image, and show all or part of subimage reduced in figure and/or multiple subimage of image.

As illustrated in fig. 12, user indicates the described track of the V-shaped shown in dotted line to shown image, and computing equipment selects this image according to the described track of V-shaped.Computing equipment is resolved the image selected, and extracts multiple subimage, as Figure 12 B, and all or part of subimage reduced in figure and/or multiple subimage of the image selected by display.In Figure 12 B, for the image img1 that user selects, what show image reduces figure img11, subimage img12 and subimage img13.Thus, user can by select image reduce figure or any one subimage selects character or image more accurately.

About the method extracting subimage, extract subimage by the prior art such as " the target subimage extracting method based on integer wavelet ", " extracting dividing method based on the target image of wavelet transformation ".

Variation 5

In this variation, the method for the regional area selecting image is described.

The display of user to computing equipment when the image shown indicates closed track, according to acquired positional information, select the regional area that in this image, track surrounds.Wherein, cover the situation of multiple image at this track, the regional area in the image selecting area coverage maximum.Such as, as shown in figure 17, computing equipment shows two images, and user indicates the closed trajectory shown in dotted line.In this case, because track is comparatively large to the overlay area of the image on right side, therefore, the regional area covered by track in the image on right side is selected.

In addition, can also be have selected the regional area of image user after, whole image is resolved, identify the subimage in image, show whole image reduce figure, regional area, subimage all or part of.

As shown in figure 18, user indicates the closed trajectory shown in dotted line to shown image, and computing equipment selects the regional area of this image according to closed trajectory.Computing equipment is resolved the image selected, and extracts multiple subimage, and shows all or part of subimage reduced in figure and/or described regional area and/or multiple subimage of whole image.In addition, also can not resolve, and only show image reduce figure and/or described regional area.Or what do not show image reduces figure and/or described regional area, and only carrys out this regional area outstanding in the mode highlighted by this regional area or amplify.

Thus, user can select character or image more accurately by selecting reduce figure, regional area or any one subimage of image.

Other variation

Figure 19 shows another example of computing equipment.Compared with the computing equipment 100 shown in Fig. 6, the computing equipment 100B shown in Figure 19 is only to instead of touch display panel 121 with induction type display panel 121B.Induction type display panel 121B can displaying contents, in addition, when user uses laser pen etc. on induction type display panel 121B during instruction content, the trace information of the laser facula that laser pen can indicate by induction type display panel 121B passes to computing equipment 100B.Thus, user can indicate track in a non-contact manner on induction type display panel 121B, selects character or image.Computing equipment is not limited to above-described computing equipment 100 and computing equipment 100B, as long as user is in the mode of gesture instruction range of choice, point to content that the display of computing equipment shows intuitively and this computing equipment can obtain the information (comprise positional information, temporal information, pressure information etc.) relevant to indicated track, then any computing equipment can be adopted to implement above-mentioned system of selection.

In addition, embodiment 1 shows after user stops instruction track, judge the shape of track and select the example of character or image, but also track can be indicated synchronously to judge the shape of track along with user, and according to being judged to be that the track of closed or V shape selects character and image.

In addition, during user indicates track, the picture of display display there occurs slip or page turning, or when user indicates video or animation, they can be regarded as the set of a series of image, i.e. multiple image, therefore, the time point of user's start instruction track can be obtained and terminate to indicate any two field picture (comprising the frame of start time point) between the time point of track, selecting character or image according to method described above.

In addition, in embodiment 1, show following example: first determine whether to cover completely, select the character that covers completely and image, and then, select the character that do not cover completely and image according to level of coverage.But whether cover completely because level of coverage can reflect, therefore, also can omit the step determining whether to cover completely, and for the character of current display and image, directly determine whether to select according to level of coverage.

In addition, when the track making the track with width become as lines, the track that other point beyond mid point is formed as lines can also be got.

In addition, in the inc situation of track, the track that can only use user to indicate carrys out computational length level of coverage or evaluates departure degree.Such as, when track is V-shaped, in three limits in generated triangular closed region, 2 limits of user's instruction or 2 limits after correcting only are used to carry out computational length level of coverage or evaluate departure degree.Such as, after have selected the character or image that enclosed region covers completely, and then, only select 2 limit local complexity of user's instruction character or image or the character nearer apart from these 2 limits or image.

In addition, if the track of user's instruction is V-shaped and selects character or picture according to length level of coverage, also enclosed region generating process can be omitted.

In addition, in the above description, judge in units of pixel, but also can be converted to centimetre etc., or adopt other dimensions etc.

Above, described the present invention by each embodiment and variation.Each embodiment and variation can independent assortments and obtain more embodiment.

Industry is applied

The present invention can be applied to computer realm, the communications field, consumer electronics field etc.

Claims

1. the selection character performed by computing equipment or a method for image, described computing equipment is shown by display, and the feature of described method is, described method makes computing equipment perform following process:

2. the method for the selection character that performed by computing equipment according to claim 1 or image, wherein,

Described trajectory corrector existence be interrupted is continuous;

To described track smoothingization process;

3. according to the method for the selection character performed by computing equipment in claim 1 or 2 described in any one or image, wherein,

4. according to the method for the selection character performed by computing equipment in claims 1 to 3 described in any one or image, wherein,

Described relative position relation can be evaluated according to level of coverage and departure degree,

Wherein, described level of coverage comprises area covering degree, length level of coverage,

5. according to the method for the selection character performed by computing equipment in any one in claims 1 to 3 described in any one or image, wherein,

6. according to the method for the selection character performed by computing equipment in claims 1 to 3 described in any one or image, wherein,

7. according to the method for the selection character performed by computing equipment in claims 1 to 3 described in any one or image, wherein,

8. according to the method for the selection character performed by computing equipment in claims 1 to 3 described in any one or image, wherein,

9. a computing equipment, it is shown by display, and the feature of this computing equipment is to comprise as lower module:

10. a computing equipment, it is shown by display, and the feature of this computing equipment is to perform following process: