CN113138708B

CN113138708B - Text processing method and system based on gestures

Info

Publication number: CN113138708B
Application number: CN202110540658.1A
Authority: CN
Inventors: 喻纯; 史元春; 杨欢
Original assignee: Interactive Future Beijing Technology Co ltd; Tsinghua University
Current assignee: Interactive Future Beijing Technology Co ltd; Tsinghua University
Priority date: 2021-05-18
Filing date: 2021-05-18
Publication date: 2023-06-06
Anticipated expiration: 2041-05-18
Also published as: CN113138708A

Abstract

The invention provides a text processing method and a text processing system based on gestures, wherein the method comprises the following steps: identifying a target gesture for processing a target text; determining the number of fingers and finger actions that make up the target gesture; determining an operation object and an operation mode corresponding to the target gesture by utilizing the number of fingers and the finger actions forming the target gesture, wherein the operation object is a word, a sentence, a segment, a full text or a cursor in a target text; and responding to the target gesture, and processing the target text based on the operation object and the operation mode corresponding to the target gesture. In the scheme, different operation modes and operation objects with different granularities are set according to different numbers of fingers and finger actions. The method comprises the steps of determining an operation object and an operation mode corresponding to a target gesture, responding to the target gesture to process a target text, providing richer operation modes and finer-granularity operation objects for visually impaired people, meeting the actual operation requirements of visually impaired people, and improving user experience.

Description

Text processing method and system based on gestures

Technical Field

The invention relates to the technical field of text processing, in particular to a text processing method and system based on gestures.

Background

With the development of the internet, electronic products such as smart phones and tablet computers occupy an important part in daily life of people, and in order to make the internet better benefit and visually impaired groups, the electronic products need to be operated by visually impaired people.

When the user touches the screen, the screen reading software reads text or control information at the corresponding position of the screen, and the user of the vision impairment operates the electronic product according to sound feedback. However, when the electronic product is operated, multiple operations are usually involved, and only the reading function is provided, so that the actual requirement of the visually impaired person for operating the electronic product cannot be met, the function of the existing mode for assisting the visually impaired person in operating the electronic product is single, and the user experience is poor.

Disclosure of Invention

In view of the above, the embodiment of the invention provides a gesture-based text processing method and a gesture-based text processing system, which are used for solving the problems of single function, poor user experience and the like of the existing screen reading software.

In order to achieve the above object, the embodiment of the present invention provides the following technical solutions:

an embodiment of the invention discloses a gesture-based text processing method, which comprises the following steps:

Identifying a target gesture for processing target text, the target gesture being comprised of a finger action of a user through at least one finger manipulation screen, the finger action being one or a combination of finger swipe, finger click, and finger pause;

determining a number of fingers and finger movements that make up the target gesture;

determining an operation object and an operation mode corresponding to the target gesture by utilizing the number of fingers and finger actions forming the target gesture, wherein the operation object is a word, a sentence, a segment, a full text or a cursor in the target text;

if the target gesture has an operation object corresponding to the target gesture, responding to the target gesture, and processing the target text based on the operation object and the operation mode corresponding to the target gesture;

and if the target gesture does not have the operation object corresponding to the target gesture, responding to the target gesture, and processing the target text based on the operation mode corresponding to the target gesture.

Preferably, the determining, by using the number of fingers and the finger actions that form the target gesture, the operation object and the operation mode corresponding to the target gesture includes:

determining an operation object and an operation mode corresponding to the target gesture by utilizing a preset gesture operation table and combining the number of fingers and finger actions forming the target gesture, wherein the gesture operation table comprises: the corresponding relation between the combination of different finger numbers and different finger actions and the operation object, and the corresponding relation between the combination of different finger numbers and different finger actions and the operation mode.

if the number of fingers constituting the target gesture is N, and if the finger movement slides in a first designated direction, determining that an operation object corresponding to the target gesture is a word, a sentence or a segment in the target text, and determining that an operation mode corresponding to the target gesture is: reading the last object of the operation object corresponding to the target gesture, wherein N is an integer greater than or equal to 1 and less than or equal to 4, and the operation object corresponding to the target gesture is related to the value of N;

if the number of fingers constituting the target gesture is N, and if the finger movement slides in a second designated direction, determining that the operation object corresponding to the target gesture is a word, a sentence or a segment in the target text, and determining that the operation mode corresponding to the target gesture is: reading an operation object corresponding to the target gesture;

if the number of fingers constituting the target gesture is N, and if the finger movement slides in a third designated direction, determining that the operation object corresponding to the target gesture is a word, a sentence or a segment in the target text, and determining that the operation mode corresponding to the target gesture is: reading the next object of the operation object corresponding to the target gesture;

If the number of fingers forming the target gesture is 1, and if the finger movement is stopped after sliding in a fourth designated direction, determining that the operation object corresponding to the target gesture is the full text in the target text, and determining that the operation mode corresponding to the target gesture is: reading the full text in the target text;

if the number of fingers constituting the target gesture is 1, and if the finger action is single-finger clicking, determining that the target gesture does not have a corresponding operation object, and determining that the operation mode corresponding to the target gesture is as follows: the sentence in which the current cursor is located and all the following contents are read aloud.

if the number of fingers constituting the target gesture is N, and if the finger movement slides in a fifth designated direction, determining that the operation object corresponding to the target gesture is a word, a sentence or a segment in the target text, and determining that the operation mode corresponding to the target gesture is: deleting an operation object corresponding to the target gesture and feeding back a deleting result through voice, wherein N is an integer greater than or equal to 1 and less than or equal to 4, and the operation object corresponding to the target gesture is related to the value of N;

If the number of fingers forming the target gesture is 1, and if the finger movement is stopped after sliding in the sixth designated direction, determining that the operation object corresponding to the target gesture is the full text in the target text, and determining that the operation mode corresponding to the target gesture is: deleting the whole text in the target text and feeding back a deleting result through voice.

if the number of fingers constituting the target gesture is 1, and if the finger movements are single-finger double-click, determining that the operation object corresponding to the target gesture is a sentence in the target text, and determining that the operation mode corresponding to the target gesture is: inserting voice input content of a user at a current cursor of the target text;

if the number of fingers constituting the target gesture is 1, and if the finger gesture slides in a seventh designated direction, determining that the operation object corresponding to the target gesture is a word in the target text, and determining that the operation mode corresponding to the target gesture is: inserting a line feed character at a current cursor of the target text;

If the number of fingers constituting the target gesture is 1, and if the finger movement slides in the eighth designated direction, determining that the operation object corresponding to the target gesture is a word in the target text, and determining that the operation mode corresponding to the target gesture is: inserting commas at the current cursor of the target text;

if the number of fingers constituting the target gesture is 1, and if the finger movement slides in a ninth designated direction, determining that the operation object corresponding to the target gesture is a word in the target text, and determining that the operation mode corresponding to the target gesture is: inserting a period at the current cursor of the target text.

if the number of fingers constituting the target gesture is 1, and if the finger gesture slides in a tenth designated direction, determining that the operation object corresponding to the target gesture is a word, a sentence or a segment in the target text, and determining that the operation mode corresponding to the target gesture is: executing a cancel instruction to cancel a gesture used for processing the target text last time;

If the number of fingers constituting the target gesture is 1, and if the finger movement slides in the eleventh designated direction, determining that the operation object corresponding to the target gesture is a word, a sentence or a segment in the target text, and determining that the operation mode corresponding to the target gesture is: and recovering the content revoked by the last execution of the revocation instruction.

if the number of fingers forming the target gesture is 1, and if the finger movement is stopped after sliding in the twelfth designated direction, determining that the operation object corresponding to the target gesture is a cursor in the target text, and determining that the operation mode corresponding to the target gesture is: moving a cursor to a beginning part in the target text;

if the number of fingers forming the target gesture is 1, and if the finger movement is stopped after sliding in the thirteenth designated direction, determining that the operation object corresponding to the target gesture is a cursor in the target text, and determining that the operation mode corresponding to the target gesture is: and moving a cursor to an ending part in the target text.

if the number of fingers constituting the target gesture is 2, and if the finger movement slides in the fourteenth designated direction, determining that the target gesture does not have a corresponding operation object, and determining that the operation mode corresponding to the target gesture is: switching to a help state;

if the number of fingers constituting the target gesture is 2, and if the finger movement slides in the fifteenth designated direction, determining that the target gesture does not have a corresponding operation object, and determining that the operation mode corresponding to the target gesture is: and exiting the help state.

if the number of fingers constituting the target gesture is 2, and if the finger motion is a long press of two fingers, determining that the operation object of the target gesture is: the last-read content in the target text or the selected content in the target text, and the operation mode corresponding to the target gesture is determined as follows: copying the operation object;

If the number of fingers forming the target gesture is 2, and if the finger motion is double-finger double-click, determining that the operation object of the target gesture is: the copied contents in the target text and the operation modes corresponding to the determined target gestures are as follows: and pasting the operation object to the current position of the cursor.

A second aspect of an embodiment of the present invention discloses a gesture-based text processing system, the system comprising:

a recognition unit for recognizing a target gesture for processing a target text, the target gesture being constituted by a finger action of a user operating a screen through at least one finger, the finger action being one or a combination of finger swipe, finger click, and finger pause;

a first determining unit configured to determine the number of fingers and finger movements constituting the target gesture;

the second determining unit is used for determining an operation object and an operation mode corresponding to the target gesture by utilizing the number of fingers and finger actions forming the target gesture, wherein the operation object is a word, a sentence, a segment, a full text or a cursor in the target text;

the first processing unit is used for responding to the target gesture if the target gesture has an operation object corresponding to the target gesture, and processing the target text based on the operation object and the operation mode corresponding to the target gesture;

And the second processing unit is used for responding to the target gesture and processing the target text based on the operation mode corresponding to the target gesture if the target gesture does not have the operation object corresponding to the target gesture.

Based on the text processing method and system based on the gesture provided by the embodiment of the invention, the method comprises the following steps: identifying a target gesture for processing a target text; determining the number of fingers and finger actions that make up the target gesture; determining an operation object and an operation mode corresponding to the target gesture by utilizing the number of fingers and the finger actions forming the target gesture, wherein the operation object is a word, a sentence, a segment, a full text or a cursor in a target text; if the target gesture has an operation object corresponding to the target gesture, responding to the target gesture, and processing the target text based on the operation object and the operation mode corresponding to the target gesture; and if the target gesture does not have the operation object corresponding to the target gesture, responding to the target gesture, and processing the target text based on the operation mode corresponding to the target gesture. In the scheme, different operation modes and operation objects with different granularities are set according to different numbers of fingers and finger actions. According to the number of fingers and the finger actions forming the target gesture, determining the operation objects and the operation modes corresponding to the target gesture, responding to the target gesture to process the target text, providing richer operation modes and finer-granularity operation objects for visually impaired people, meeting the actual operation requirements of visually impaired people, and improving the user experience.

Drawings

In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings that are required to be used in the embodiments or the description of the prior art will be briefly described below, and it is obvious that the drawings in the following description are only embodiments of the present invention, and that other drawings can be obtained according to the provided drawings without inventive effort for a person skilled in the art.

FIG. 1 is a flowchart of a gesture-based text processing method according to an embodiment of the present invention;

FIG. 2 (a) is a schematic diagram of a single finger left slide; FIG. 2 (b) is a schematic diagram of a two-finger left slide; FIG. 2 (c) is a schematic diagram of a three-finger left slide; FIG. 2 (d) is a schematic diagram of a four-finger left slide;

FIG. 3 (a) is a schematic illustration of single finger downslide; FIG. 3 (b) is a schematic diagram of a two-finger swipe; FIG. 3 (c) is a schematic illustration of three-finger downslide; FIG. 3 (d) is a schematic diagram of a four-finger swipe;

FIG. 4 (a) is a schematic illustration of a single finger right slide; FIG. 4 (b) is a schematic diagram of a two-finger right slide; FIG. 4 (c) is a schematic illustration of a three-finger right slide; FIG. 4 (d) is a schematic diagram of a four-finger right slide;

FIG. 5 (a) is a schematic illustration of a pause after a single finger swipe; FIG. 5 (b) is a schematic diagram of a single finger click;

FIG. 6 (a) is a schematic illustration of single finger up-slide; FIG. 6 (b) is a schematic diagram of a two-finger up-slide; FIG. 6 (c) is a schematic illustration of three-finger up-slide; FIG. 6 (d) is a schematic diagram of a four-finger up-slide, and FIG. 6 (e) is a schematic diagram of a single-finger up-slide pause;

FIG. 7 (a) is a schematic diagram of a single finger double click; FIG. 7 (b) is a schematic view of the single finger sliding down after sliding right;

FIG. 7 (c) is a schematic view of left slide after single finger sliding down; FIG. 7 (d) is a schematic view of right slide after single finger sliding down;

FIG. 8 (a) is a schematic view showing the sliding down of a single finger after sliding up; FIG. 8 (b) is a schematic view showing the upward sliding after the downward sliding;

FIG. 9 (a) is a schematic diagram of a pause after a single finger left slide; FIG. 9 (b) is a schematic diagram of a pause after a single finger right slide;

FIG. 10 (a) is a schematic diagram of a two-finger approach; FIG. 10 (b) is a schematic diagram of a two-finger approach;

FIG. 11 (a) is a schematic diagram of a double finger long press; FIG. 11 (b) is a schematic diagram of a double finger double click;

FIG. 12 is a block diagram illustrating a gesture-based text processing system according to an embodiment of the present invention.

Detailed Description

The following description of the embodiments of the present invention will be made clearly and completely with reference to the accompanying drawings, in which it is apparent that the embodiments described are only some embodiments of the present invention, but not all embodiments. All other embodiments, which can be made by those skilled in the art based on the embodiments of the invention without making any inventive effort, are intended to be within the scope of the invention.

In this application, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one … …" does not exclude the presence of other like elements in a process, method, article, or apparatus that comprises the element.

As known from the background art, when using the electronic product, the visually impaired person relies on the screen reading function completely, and the present screen reading software only provides the function of reading, that is, when the visually impaired person touches the screen with a finger, the screen reading software reads text or control information at the corresponding position of the screen. However, since multiple operations are generally involved in operating the electronic product, only the reading function is provided, the actual requirement of the visually impaired person for operating the electronic product cannot be met, the function of the existing method for assisting the visually impaired person in operating the electronic product is single, and the user experience is poor.

Therefore, the embodiment of the invention provides a gesture-based text processing method and a gesture-based text processing system, which are used for setting different operation modes and operation objects with different granularities according to different numbers of fingers and finger actions. According to the number of fingers and the finger actions forming the target gesture, determining the operation objects and the operation modes corresponding to the target gesture, responding to the target gesture to process the target text, and providing richer operation modes and finer-granularity operation objects for visually impaired people so as to meet the actual operation requirements of visually impaired people and further improve the user experience.

It should be noted that, the gesture-based text processing method and system provided by the embodiment of the invention are suitable for electronic products with touch screens, such as smart phones, tablet computers, and the like, and the specific type of the electronic products is not limited.

Referring to fig. 1, a flowchart of a gesture-based text processing method according to an embodiment of the present invention is shown, where the text processing method includes:

step S101: a target gesture for processing target text is identified.

It should be noted that the target gesture is formed by a finger action of the user operating the screen through at least one finger, and the finger action is one or a combination of finger sliding, finger clicking and finger pausing.

In the specific implementation process of step S101, when the user operates on the screen (touch screen) of the electronic product through at least one finger in combination with a corresponding finger action, the target gesture of the user for processing the target text is identified, and at the same time, the number of fingers and the finger actions forming the target gesture can be obtained after the target gesture is identified.

That is, the number of fingers and finger actions used by the user when operating the screen may constitute a target gesture for processing target text.

Step S102: the number of fingers and finger movements that make up the target gesture are determined.

In the specific implementation process of step S102, after the target gesture is identified, the number of fingers and the finger actions that constitute the target gesture are determined.

Step S103: and determining an operation object and an operation mode corresponding to the target gesture by utilizing the number of fingers and the finger actions forming the target gesture.

It should be noted that the operation object is a word, a sentence, a segment, a full text, or a cursor in the target text.

It should be further noted that, the basic unit of the target text (i.e. the text to be processed) may be divided into a word (including punctuation marks), a word, a sentence, a segment and a full text, the word is distinguished by a character basic data type in the code, the word is distinguished by a default word segmentation result in a natural language processing package (e.g. HanLP), the sentence is distinguished by the punctuation marks, the segment is distinguished by a line-feed character, and the full text is the full text content of the target text.

In the specific implementation process of step S102, a gesture operation table is preset, where the gesture operation table includes: the corresponding relation between the combination of different finger numbers and different finger actions and the operation object, and the corresponding relation between the combination of different finger numbers and different finger actions and the operation mode.

That is, when the number of fingers and the finger motions constituting the target gesture are determined, the operation object and the operation style corresponding to the target gesture can be determined by referring to the gesture operation table using the number of fingers and the finger motions constituting the target gesture.

It will be appreciated that different operation objects may be determined by the number of fingers and finger movements, for example: the single-finger sliding indicates that the operation object is a character, the double-finger sliding indicates that the operation object is a word, the three-finger sliding indicates that the operation object is a sentence, the four-finger sliding indicates that the operation object is a segment, and the single-finger sliding indicates that the operation object is a full text.

The operation object is the current unit (word, sentence, segment, full text or cursor) in the target text which needs to be operated currently, and when the operation object is the word, sentence or segment, the current unit can be determined by the position of the cursor.

In a specific implementation, when the operation object is a word, the word before the cursor is the current unit, for example: for the sentence "today friday", the cursor is between "day" and "star", then "day" is the current unit (i.e., the operation object).

When the operation object is a word, the cursor is positioned in or behind a certain word, and the word is the current unit; when the operation object is a sentence, and the cursor is positioned in or behind a certain sentence, the sentence is the current unit; when the operation object is a segment, and the cursor is positioned in or behind a certain segment, the segment is the current unit;

After the operation object is determined, the operation mode of the target gesture is determined by combining different finger numbers and different combinations of finger actions, and processing such as reading, deleting and inputting is performed on the target text according to the operation mode and the operation object.

That is, the combination of different finger numbers and different finger actions corresponds to different operation objects and operation modes, and how to determine the operation modes and the procedure of the operation objects is described in detail below.

In some embodiments, if the number of fingers constituting the target gesture is N, and if the finger movement is sliding in the first specified direction (e.g. sliding leftwards), determining that the operation object corresponding to the target gesture is a word, a sentence or a segment in the target text, and determining that the operation mode corresponding to the target gesture is: and (3) reading the last object of the operation object corresponding to the target gesture, wherein N is an integer greater than or equal to 1 and less than or equal to 4, and at the moment, the cursor moves to the back of the last object of the operation object.

It should be noted that, the operation object corresponding to the target gesture is related to the value of N, as mentioned above: the single-finger (n=1) slide indicates that the operation object is a word, the double-finger (n=2) slide indicates that the operation object is a word, the three-finger (n=3) slide indicates that the operation object is a sentence, and the four-finger (n=4) slide indicates that the operation object is a segment.

As can be seen from the above, the words, sentences and segments correspond to different numbers of fingers respectively, and when the finger motion is single-finger (n=1) sliding in the first designated direction (for example, single-finger sliding left), the operation object is a word, and the operation mode is the last word of the reading operation object; when the finger motion is a double-finger (n=2) sliding in a first designated direction (for example, double-finger left sliding), the operation object is a word, and the operation mode is a last word of the reading operation object; when the finger motion is three-finger (n=3) sliding in the first designated direction (for example, three-finger sliding left), the operation object is a sentence, and the operation mode is a last sentence of the reading operation object; when the finger motion is a four-finger (n=4) sliding motion in the first designated direction (for example, a four-finger sliding motion is left), the operation object is a segment, and the operation mode is a last segment of the reading operation object.

The specific operation mode of the single-finger left slide is referred to as a schematic diagram of the single-finger left slide provided in fig. 2 (a), the specific operation mode of the double-finger left slide is referred to as a schematic diagram of the double-finger left slide provided in fig. 2 (b), the specific operation mode of the three-finger left slide is referred to as a schematic diagram of the three-finger left slide provided in fig. 2 (c), and the specific operation mode of the four-finger left slide is referred to as a schematic diagram of the four-finger left slide provided in fig. 2 (d).

In some embodiments, if the number of fingers constituting the target gesture is N, and if the finger movement is sliding in the second designated direction (e.g. sliding downward), determining that the operation object corresponding to the target gesture is a word, a sentence or a segment in the target text, and determining that the operation mode corresponding to the target gesture is: and reading the operation object corresponding to the target gesture, and moving the cursor to the back of the operation object.

When the finger motion is a single finger (n=1) sliding in a second designated direction (for example, the single finger slides down), the operation object is a word, and the operation mode is a reading operation object (i.e., reading the current word); when the finger motion is a double-finger (n=2) sliding in the second designated direction (for example, the double-finger sliding down), the operation object is a word, and the operation mode is a reading operation object (i.e., reading the current word); when the finger motion is three-finger (n=3) sliding in the second designated direction (for example, three-finger sliding down), the operation object is a sentence, and the operation mode is a reading operation object (i.e., reading the current sentence); when the finger motion is a four-finger (n=4) sliding in the second designated direction (for example, the four-finger sliding down), the operation object is a segment, and the operation mode is a reading operation object (i.e., reading the current segment).

The specific operation mode of the single-finger sliding refers to the schematic diagram of the single-finger sliding provided in fig. 3 (a), the specific operation mode of the double-finger sliding refers to the schematic diagram of the double-finger sliding provided in fig. 3 (b), the specific operation mode of the three-finger sliding refers to the schematic diagram of the three-finger sliding provided in fig. 3 (c), and the specific operation mode of the four-finger sliding refers to the schematic diagram of the four-finger sliding provided in fig. 3 (d).

In some embodiments, if the number of fingers constituting the target gesture is N, and if the finger movement is sliding in the third designated direction (e.g. sliding rightward), determining that the operation object corresponding to the target gesture is a word, a sentence or a segment in the target text, and determining that the operation mode corresponding to the target gesture is: and reading the next object of the operation object corresponding to the target gesture, and moving the cursor to the back of the next object of the operation object.

When the finger motion is a single finger (n=1) sliding in a third designated direction (for example, a single finger sliding right), the operation object is a word, and the operation mode is to read the next word of the operation object; when the finger motion is a double-finger (n=2) sliding in a third designated direction (for example, double-finger right sliding), the operation object is a word, and the operation mode is the next word of the reading operation object; when the finger motion is three-finger (n=3) sliding in a third designated direction (for example, three-finger sliding right), the operation object is a sentence, and the operation mode is to read the next sentence of the operation object; when the finger motion is a four-finger (n=4) sliding in the third designated direction (for example, a four-finger right sliding), the operation object is a segment, and the operation mode is a next segment of the reading operation object.

The specific operation mode of the single-finger right slide is referred to as a schematic diagram of the single-finger right slide provided in fig. 4 (a), the specific operation mode of the double-finger right slide is referred to as a schematic diagram of the double-finger right slide provided in fig. 4 (b), the specific operation mode of the three-finger right slide is referred to as a schematic diagram of the three-finger right slide provided in fig. 4 (c), and the specific operation mode of the four-finger right slide is referred to as a schematic diagram of the four-finger right slide provided in fig. 4 (d).

In some implementations, if the number of fingers constituting the target gesture is 1, and if the finger movement is a pause after sliding in the fourth designated direction (for example, a pause after sliding down a single finger), determining that the operation object corresponding to the target gesture is the full text in the target text, and determining that the operation mode corresponding to the target gesture is: and reading the full text in the target text, wherein the cursor moves in sentence units, and finally stays at the tail end of the target text.

The specific operation mode of the single-finger post-sliding pause is shown in a schematic diagram of the single-finger post-sliding pause provided in fig. 5 (a).

In some embodiments, if the number of fingers constituting the target gesture is 1, and if the finger motion is a single-finger click, determining that the target gesture does not have a corresponding operation object, and determining that the operation mode corresponding to the target gesture is: the sentence in which the current cursor is located and all the following contents are read aloud.

That is, when the finger action is a single-finger click, the sentence where the current cursor is located is read to the end content of the target text, and the cursor moves in sentence units during the reading process, and finally stays at the end of the target text.

The specific operation manner of the single-finger single click is shown in a schematic diagram of the single-finger single click provided in fig. 5 (b).

It can be understood that, in the above-mentioned embodiments, the related content is read aloud, and in the process of reading aloud, the reading aloud can be stopped by a single-finger single-click mode, and at this time, the cursor moves to the back of the sentence being read aloud.

In some embodiments, if the number of fingers constituting the target gesture is N, and if the finger movement is sliding in the fifth specified direction (e.g. sliding upwards), determining that the operation object corresponding to the target gesture is a word, a sentence or a segment in the target text, and determining that the operation mode corresponding to the target gesture is: deleting the operation object corresponding to the target gesture and feeding back a deleting result (content deleted by voice feedback), wherein N is an integer greater than or equal to 1 and less than or equal to 4, and at the moment, the cursor moves to the back of the deleted operation object.

When the finger motion is a single finger (n=1) sliding in a fifth designated direction (for example, the single finger is sliding upwards), the operation object is a word, the operation mode is to delete the operation object (i.e. delete the current word) and the deleted content is fed back by voice; when the finger motion is a double-finger (n=2) sliding in the fifth designated direction (for example, double-finger up sliding), the operation object is a word, the operation mode is to delete the operation object (i.e., delete the current word) and the deleted content is voice-fed back; when the finger motion is a three-finger (n=3) sliding in the fifth designated direction (for example, the three-finger is sliding upwards), the operation object is a sentence, the operation mode is that the operation object is deleted (i.e., the current sentence is deleted) and the deleted content is voice-fed back; when the finger motion is a four-finger (n=4) sliding in the fifth designated direction (for example, a four-finger up-sliding), the operation object is a segment, and the operation is performed in such a manner that the operation object is deleted (i.e., the current segment is deleted) and the deleted content is voice-fed back.

It can be understood that when deleting the current word, if the current word is a punctuation mark, after deleting the punctuation mark, sentence division is needed to be carried out on the target text again; if the current word is not the punctuation mark, after deleting the current word, the target text needs to be segmented again; if the current cursor is located at the beginning of the target text, the voice prompt "has arrived at the beginning of the text".

When deleting the current word, if all the words of a certain sentence are deleted, punctuation marks at the tail of the sentence are also deleted.

When deleting the current sentence, punctuation marks at the tail of the sentence are also deleted, but the line-feed symbols are not deleted.

The specific operation mode of the single-finger sliding-up refers to the schematic diagram of the single-finger sliding-up provided in fig. 6 (a), the specific operation mode of the double-finger sliding-up refers to the schematic diagram of the double-finger sliding-up provided in fig. 6 (b), the specific operation mode of the three-finger sliding-up refers to the schematic diagram of the three-finger sliding-up provided in fig. 6 (c), and the specific operation mode of the four-finger sliding-up refers to the schematic diagram of the four-finger sliding-up provided in fig. 6 (d).

In some embodiments, if the number of fingers constituting the target gesture is 1, and if the finger movement is a pause after sliding in the sixth designated direction (for example, a pause after sliding upwards), the operation object corresponding to the target gesture is determined to be the full text in the target text, and the operation mode corresponding to the target gesture is determined to be: deleting the full text in the target text and feeding back the deleting result through voice.

That is, when the target gesture is a single-finger pause after sliding up, the operation object corresponding to the target gesture is the full text in the target text, and the operation mode corresponding to the target gesture is as follows: deleting the full text in the target text and feeding back the deleting result through voice.

The specific operation mode of the single finger post-sliding pause is shown in the schematic diagram of the single finger post-sliding pause provided in fig. 6 (e).

In some embodiments, if the number of fingers constituting the target gesture is 1, and if the finger gesture is a single-finger double-click, determining that the operation object corresponding to the target gesture is a sentence in the target text, and determining that the operation mode corresponding to the target gesture is: the user's voice input content is inserted at the current cursor of the target text.

That is, when the target gesture is a single-finger double click, the user's voice input content is inserted at the current cursor of the target text, at which time the cursor moves behind the voice input content.

The specific operation manner of the single-finger double-click is shown in a schematic diagram of the single-finger double-click provided in fig. 7 (a).

In some embodiments, if the number of fingers constituting the target gesture is 1, and if the finger is slid in the seventh designated direction (for example, sliding down after right), the operation object corresponding to the target gesture is determined to be a word in the target text, and the operation mode corresponding to the target gesture is determined to be: a line feed is inserted at the current cursor of the target text.

That is, when the target gesture is a single-finger right-swipe and then a swipe down, a line feed is inserted at the current cursor of the target text, at which time the cursor moves behind the inserted line feed.

The specific operation mode of the single-finger sliding down after the right sliding is shown in a schematic diagram of the single-finger sliding down after the right sliding provided in fig. 7 (b).

In some embodiments, if the number of fingers constituting the target gesture is 1, and if the finger movement is sliding in the eighth designated direction (for example, sliding down and then sliding left), determining that the operation object corresponding to the target gesture is a word in the target text, and determining that the operation mode corresponding to the target gesture is: commas are inserted at the current cursor of the target text.

That is, when the target gesture is a single-finger swipe and then a left swipe, a comma is inserted at the current cursor of the target text, and the cursor moves behind the inserted comma.

The specific operation mode of the single-finger sliding-down and left sliding is shown in a schematic diagram of the single-finger sliding-down and left sliding provided in fig. 7 (c).

In some embodiments, if the number of fingers constituting the target gesture is 1, and if the finger movement is sliding in the ninth designated direction (for example, sliding down and then sliding right), determining that the operation object corresponding to the target gesture is a word in the target text, and determining that the operation mode corresponding to the target gesture is: periods are inserted at the current cursor of the target text.

That is, when the target gesture is a single-finger sliding down and then a right sliding down, a period is inserted at the current cursor of the target text, and the cursor moves to the rear of the inserted period.

The specific operation mode of the single-finger sliding-down and right sliding is shown in a schematic diagram of the single-finger sliding-down and right sliding provided in fig. 7 (d).

In some embodiments, if the number of fingers constituting the target gesture is 1, and if the finger gesture is sliding in the tenth designated direction (for example, sliding up and down), determining that the operation object corresponding to the target gesture is a word, a sentence or a segment in the target text, and determining that the operation mode corresponding to the target gesture is: and executing a cancel instruction to cancel the gesture used for processing the target text last time.

That is, when the target gesture is that the single finger slides up and then slides down, a cancel instruction is executed to cancel the gesture used for processing the target text last time (namely, cancel the previous operation), and if no content can be canceled, the voice prompt is "cancel can not be continued"; an upper limit number of times (for example, an upper limit of 20 times) of execution of the revocation instruction may be set, and when the number of times of execution of the revocation instruction exceeds 20 times, a voice prompt "the upper limit of the number of times of revocation has been reached" may be provided.

The operation mode of sliding down the single finger after sliding up is shown in the schematic diagram of sliding down the single finger after sliding up provided in fig. 8 (a).

In some embodiments, if the number of fingers constituting the target gesture is 1, and if the finger movement is sliding in the eleventh designated direction (for example, sliding down and then sliding up), determining that the operation object corresponding to the target gesture is a word, a sentence or a segment in the target text, and determining that the operation mode corresponding to the target gesture is: and recovering the content revoked by the last execution of the revocation instruction.

That is, when the target gesture is that the single finger slides down and then slides up, the content withdrawn by the last execution of the withdrawal instruction is recovered, and if no recoverable content exists, the voice prompt is "no recovery can be continued".

The operation mode of sliding up after sliding down with a single finger is shown in a schematic diagram of sliding up after sliding down with a single finger provided in fig. 8 (b).

In some embodiments, if the number of fingers constituting the target gesture is 1, and if the finger movement is a pause after sliding in the twelfth designated direction (for example, a pause after sliding left), determining that the operation object corresponding to the target gesture is a cursor in the target text, and determining that the operation mode corresponding to the target gesture is: the cursor is moved to the beginning of the target text.

That is, when the target gesture is a one-finger left swipe, the cursor is moved to the beginning of the target text.

The operation mode of the single-finger left-hand pause is shown in a schematic diagram of the single-finger left-hand pause shown in fig. 9 (a).

In some embodiments, if the number of fingers constituting the target gesture is 1, and if the finger movement is a pause after sliding in the thirteenth designated direction (for example, a pause after sliding right), determining that the operation object corresponding to the target gesture is a cursor in the target text, and determining that the operation mode corresponding to the target gesture is: the cursor is moved to the ending location in the target text.

That is, when the target gesture is a single finger right swipe, the cursor is moved to the ending part in the target text.

The operation mode of the single-finger post-right-slide pause is shown in a schematic diagram of the single-finger post-right-slide pause shown in fig. 9 (b).

In some embodiments, if the number of fingers constituting the target gesture is 2, and if the finger movement is sliding in the fourteenth designated direction (e.g. the two fingers are far away), it is determined that the target gesture does not have a corresponding operation object, and it is determined that the operation mode corresponding to the target gesture is: switch to the help state.

That is, when the target gesture is far away from the two fingers, the user is switched to the help state, and is guided to learn the operation gesture of the electronic product through voice, it can be understood that only the gesture of exiting from the help state is accepted in the help state, and other gestures are not processed.

The specific operation manner of the double-finger separation is shown in a schematic diagram of the double-finger separation provided in fig. 10 (a).

In some embodiments, if the number of fingers constituting the target gesture is 2, and if the finger movement is sliding in the fifteenth designated direction (e.g. two fingers approach), it is determined that the target gesture does not have a corresponding operation object, and it is determined that the operation mode corresponding to the target gesture is: the help state is exited.

That is, when the target gesture is a two-finger approach, the help state is exited.

The specific operation manner of the double-finger approach is shown in a schematic diagram of the double-finger approach provided in fig. 10 (b).

In some embodiments, if the number of fingers constituting the target gesture is 2, and if the finger motion is a long press of two fingers, the operation object of the target gesture is determined as: the last-read content in the target text or the selected content in the target text, and the operation mode corresponding to the determined target gesture is as follows: the operation object is copied.

That is, when the target gesture is a long-press of two fingers, the content in the target text read last time is copied, and if the last gesture is not a gesture for reading, the voice prompt "please select to copy the content" copies the content in the selected target text.

The operation mode of the long double-finger press is shown in a schematic diagram of the long double-finger press provided in fig. 11 (a).

In some embodiments, if the number of fingers constituting the target gesture is 2, and if the finger motion is double-finger double-click, the operation object determining the target gesture is: the copied contents in the target text and the operation modes corresponding to the determined target gestures are as follows: and pasting the operation object to the current position of the cursor.

That is, when the target gesture is a double-finger double-click, the copied content is pasted to the current position of the cursor, at this time, the cursor moves to the back of the pasted content, and if there is no content that can be pasted, the voice prompt "no content that can be pasted" is given.

The operation manner of the double-finger double-click is shown in a schematic diagram of the double-finger double-click provided in fig. 11 (b).

Step S104: and if the target gesture has the operation object corresponding to the target gesture, responding to the target gesture, and processing the target text based on the operation object and the operation mode corresponding to the target gesture.

In the specific implementation process of step S104, if the target gesture has a corresponding operation object, the target text is processed by the operation object and the operation mode of the target gesture determined in the above steps in response to the target gesture, and the specific processing mode can refer to the specific content of each operation mode shown in step S103 and will not be described herein.

Step S105: and if the target gesture does not have the operation object corresponding to the target gesture, responding to the target gesture, and processing the target text based on the operation mode corresponding to the target gesture.

In the specific implementation process of step S105, if the target gesture does not have a corresponding operation object, the target text is processed by the operation mode of the target gesture determined in the above step in response to the target gesture, and the specific processing mode may refer to the specific content of each operation mode shown in step S103, which is not described herein again.

As can be seen from the above content of step S103, by setting the operation objects and operation modes corresponding to the plurality of gestures, a plurality of operation functions are provided to the visually impaired, for example: cancel function, resume function, and cursor movement function.

It can be understood that misoperation of the visually impaired person is common in the process of actually operating the electronic product, so that the visually impaired person does not need to empty all contents and input the contents again when the misoperation occurs by providing the cancel and restore functions, but can continue to process the text under the condition that the visually impaired person does not need to empty all contents by canceling or restoring the functions, and the fault tolerance of the visually impaired person for operating the electronic product is improved.

Similarly, when the visually impaired needs to move the cursor to the beginning or the end of the target text, the user can directly move the cursor to the beginning or the end of the target text only by a simple gesture, and the user can also directly control the cursor through the gesture, so that interaction logic is simplified, and user experience is improved.

When the operation objects are processed, the granularity of the operation objects is divided into characters, words, sentences, segments, full text and cursors, finer operation object division is provided, so that an operator with vision impairment can select the operation objects to be processed according to actual requirements, and the interaction function between the operator with vision impairment and the electronic product is enriched.

In summary, a visually impaired person may operate on a screen of an electronic product through a plurality of target gestures, so as to perform corresponding processing (such as text editing, reading and the like) on target text.

In the embodiment of the invention, different operation modes and operation objects with different granularities are set for different numbers of fingers and finger actions. According to the number of fingers and the finger actions forming the target gesture, determining the operation objects and the operation modes corresponding to the target gesture, responding to the target gesture to process the target text, providing richer operation modes and finer-granularity operation objects for visually impaired people, meeting the actual operation requirements of visually impaired people, and further improving the user experience.

It should be noted that, as can be seen from the above embodiments shown in step S103, there are various operation modes, and different operation modes may be classified and divided into: the method comprises a reading operation mode, a deleting operation mode, an input operation mode, an auxiliary operation mode and a help operation mode, wherein corresponding remark matters exist in each type of operation mode, and the operation modes of each type are explained through a plurality of tables.

The function of the reading operation mode is as follows: and respectively reading a last unit, a current unit and a next unit according to four granularity of the characters, the words, the sentences and the segments, reading the whole text, and starting to read backwards from the current sentence, wherein specific functions and descriptions thereof refer to specific contents of a reading operation mode shown in the table 1.

Table 1:

/>

/>

/>

it can be understood that, when the words are read, the words are read in a word-group manner specifically means that: when a certain word is read, the word corresponding to the word is read, and the visually impaired is assisted to know which word is specifically, for example: when the word "sound" is read, the word "sound" is read in the form of "music" so that the visually impaired people can know that the read word is "sound".

The delete class mode of operation functions as: the current unit is deleted according to the granularity of the characters, words, sentences and segments, the whole text is deleted, and the specific functions and the description thereof refer to the specific contents of the deletion operation modes shown in the table 2.

Table 2:

/>

/>

the input class operation mode has the following functions: the triggering and ending of voice interactions, the entry of spaces, line breaks, commas and periods, specific functions and their description are presented in table 3 with reference to the specific contents of the input class operation mode.

Table 3:

the auxiliary operation mode has the following functions: copy, paste, undo, restore, cursor move to the beginning and cursor move to the end, for specific functions and descriptions thereof see the details of the auxiliary class operation shown in table 4.

Table 4:

/>

/>

the help class operation mode has the following functions: entering the help state and exiting the help state, the specific functions and descriptions thereof are referred to in the details of the help class operation shown in Table 5.

Table 5:

/>

corresponding to the gesture-based text processing method provided in the above embodiment of the present invention, referring to fig. 12, the embodiment of the present invention further provides a block diagram of a gesture-based text processing system, where the text processing system includes: an identification unit 121, a first determination unit 122, a second determination unit 123, a first processing unit 124, and a second processing unit 125;

The recognition unit 121 is configured to recognize a target gesture for processing a target text, where the target gesture is configured by a finger action of a user operating the screen through at least one finger, and the finger action is one or a combination of finger swipe, finger click, and finger pause.

The first determining unit 122 is configured to determine the number of fingers and the finger motion that constitute the target gesture.

The second determining unit 123 is configured to determine, by using the number of fingers and the finger actions that form the target gesture, an operation object and an operation manner corresponding to the target gesture, where the operation object is a word, a sentence, a segment, a full text, or a cursor in the target text.

In a specific implementation, the second determining unit 123 is specifically configured to: determining an operation object and an operation mode corresponding to the target gesture by utilizing a preset gesture operation table and combining the number of fingers and finger actions forming the target gesture, wherein the gesture operation table comprises: the corresponding relation between the combination of different finger numbers and different finger actions and the operation object, and the corresponding relation between the combination of different finger numbers and different finger actions and the operation mode.

In some embodiments, the second determining unit 123 is specifically configured to: if the number of fingers constituting the target gesture is N, and if the finger movement is sliding in the first designated direction, determining that the operation object corresponding to the target gesture is a word, a sentence or a segment in the target text, and determining that the operation mode corresponding to the target gesture is: the last object of the operation object corresponding to the reading target gesture, N is an integer which is more than or equal to 1 and less than or equal to 4, and the operation object corresponding to the target gesture is related to the value of N; if the number of fingers constituting the target gesture is N, and if the finger movement slides in the second designated direction, determining that the operation object corresponding to the target gesture is a word, a sentence or a segment in the target text, and determining that the operation mode corresponding to the target gesture is: reading an operation object corresponding to the target gesture; if the number of fingers constituting the target gesture is N, and if the finger movement is sliding in the third designated direction, determining that the operation object corresponding to the target gesture is a word, a sentence or a segment in the target text, and determining that the operation mode corresponding to the target gesture is: reading the next object of the operation object corresponding to the target gesture; if the number of fingers forming the target gesture is 1, and if the finger movement is stopped after sliding in the fourth designated direction, determining that the operation object corresponding to the target gesture is the whole text in the target text, and determining that the operation mode corresponding to the target gesture is: reading the full text in the target text; if the number of fingers constituting the target gesture is 1, and if the finger motion is single-finger clicking, determining that the target gesture does not have a corresponding operation object, and determining that the operation mode corresponding to the target gesture is as follows: the sentence in which the current cursor is located and all the following contents are read aloud.

In some embodiments, the second determining unit 123 is specifically configured to: if the number of fingers constituting the target gesture is N, and if the finger movement slides in the fifth designated direction, determining that the operation object corresponding to the target gesture is a word, a sentence or a segment in the target text, and determining that the operation mode corresponding to the target gesture is: deleting an operation object corresponding to the target gesture and feeding back a deleting result through voice, wherein N is an integer greater than or equal to 1 and less than or equal to 4, and the operation object corresponding to the target gesture is related to the value of N; if the number of fingers forming the target gesture is 1, and if the finger movement is stopped after sliding in the sixth designated direction, determining that the operation object corresponding to the target gesture is the full text in the target text, and determining that the operation mode corresponding to the target gesture is: deleting the full text in the target text and feeding back the deleting result through voice.

In some embodiments, the second determining unit 123 is specifically configured to: if the number of fingers constituting the target gesture is 1, and if the finger movements are single-finger double-click, determining that the operation object corresponding to the target gesture is a sentence in the target text, and determining that the operation mode corresponding to the target gesture is: inserting voice input content of a user at a current cursor of a target text; if the number of fingers constituting the target gesture is 1, and if the finger gesture is slid in the seventh designated direction, determining that the operation object corresponding to the target gesture is a word in the target text, and determining that the operation mode corresponding to the target gesture is: inserting a line feed character at the current cursor of the target text; if the number of fingers constituting the target gesture is 1, and if the finger movement slides in the eighth designated direction, determining that the operation object corresponding to the target gesture is a word in the target text, and determining that the operation mode corresponding to the target gesture is: inserting commas at the current cursor of the target text; if the number of fingers constituting the target gesture is 1, and if the finger movement is sliding in the ninth designated direction, determining that the operation object corresponding to the target gesture is a word in the target text, and determining that the operation mode corresponding to the target gesture is: periods are inserted at the current cursor of the target text.

In some embodiments, the second determining unit 123 is specifically configured to: if the number of fingers constituting the target gesture is 1, and if the finger gesture is slid in the tenth designated direction, determining that the operation object corresponding to the target gesture is a word, a sentence or a segment in the target text, and determining that the operation mode corresponding to the target gesture is: executing a cancel instruction to cancel a gesture for processing the target text last time; if the number of fingers constituting the target gesture is 1, and if the finger movement slides in the eleventh designated direction, determining that the operation object corresponding to the target gesture is a word, a sentence or a segment in the target text, and determining that the operation mode corresponding to the target gesture is: and recovering the content revoked by the last execution of the revocation instruction.

In some embodiments, the second determining unit 123 is specifically configured to: if the number of fingers constituting the target gesture is 1, and if the finger movement is stopped after sliding in the twelfth designated direction, determining that the operation object corresponding to the target gesture is a cursor in the target text, and determining that the operation mode corresponding to the target gesture is: moving the cursor to a beginning part in the target text; if the number of fingers constituting the target gesture is 1, and if the finger movement is stopped after sliding in the thirteenth designated direction, determining that the operation object corresponding to the target gesture is a cursor in the target text, and determining that the operation mode corresponding to the target gesture is: the cursor is moved to the ending location in the target text.

In some embodiments, the second determining unit 123 is specifically configured to: if the number of fingers constituting the target gesture is 2, and if the finger movement is sliding in the fourteenth designated direction, determining that the target gesture does not have a corresponding operation object, and determining that the operation mode corresponding to the target gesture is: switching to a help state; if the number of fingers constituting the target gesture is 2, and if the finger movement is sliding in the fifteenth designated direction, determining that the target gesture does not have a corresponding operation object, and determining that the operation mode corresponding to the target gesture is: the help state is exited.

In some embodiments, the second determining unit 123 is specifically configured to: if the number of fingers constituting the target gesture is 2, and if the finger motion is a long press of two fingers, determining that the operation object of the target gesture is: the last-read content in the target text or the selected content in the target text, and the operation mode corresponding to the determined target gesture is as follows: copying the operation object; if the number of fingers constituting the target gesture is 2, and if the finger movements are double-finger double-click, determining the operation object of the target gesture is: the copied contents in the target text and the operation modes corresponding to the determined target gestures are as follows: and pasting the operation object to the current position of the cursor.

The first processing unit 124 is configured to, if the target gesture has an operation object corresponding to the target gesture, respond to the target gesture, and process the target text based on the operation object and the operation mode corresponding to the target gesture.

The second processing unit 125 is configured to respond to the target gesture if the target gesture does not have an operation object corresponding to the target gesture, and process the target text based on an operation mode corresponding to the target gesture.

In summary, the embodiments of the present invention provide a gesture-based text processing method and system, which set different operation modes and operation objects with different granularities for different numbers of fingers and finger actions. According to the number of fingers and the finger actions forming the target gesture, determining the operation objects and the operation modes corresponding to the target gesture, responding to the target gesture to process the target text, providing richer operation modes and finer-granularity operation objects for visually impaired people, meeting the actual operation requirements of visually impaired people, and improving the user experience.

In this specification, each embodiment is described in a progressive manner, and identical and similar parts of each embodiment are all referred to each other, and each embodiment mainly describes differences from other embodiments. In particular, for a system or system embodiment, since it is substantially similar to a method embodiment, the description is relatively simple, with reference to the description of the method embodiment being made in part. The systems and system embodiments described above are merely illustrative, wherein the elements illustrated as separate elements may or may not be physically separate, and the elements shown as elements may or may not be physical elements, may be located in one place, or may be distributed over a plurality of network elements. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment. Those of ordinary skill in the art will understand and implement the present invention without undue burden.

Those of skill would further appreciate that the various illustrative elements and algorithm steps described in connection with the embodiments disclosed herein may be implemented as electronic hardware, computer software, or combinations of both, and that the various illustrative elements and steps are described above generally in terms of functionality in order to clearly illustrate the interchangeability of hardware and software. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the solution. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present invention.

The previous description of the disclosed embodiments is provided to enable any person skilled in the art to make or use the present invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the generic principles defined herein may be applied to other embodiments without departing from the spirit or scope of the invention. Thus, the present invention is not intended to be limited to the embodiments shown herein but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A gesture-based text processing method, the method comprising:

determining the number of fingers and finger actions constituting the target gesture, wherein the number of fingers and the finger actions are used for querying a gesture operation table, and the gesture operation table comprises: the corresponding relation between the combination of different finger numbers and different finger actions and the operation object, and the corresponding relation between the combination of different finger numbers and different finger actions and the operation mode;

if the target gesture does not have an operation object corresponding to the target gesture, responding to the target gesture, and processing the target text based on an operation mode corresponding to the target gesture;

the determining the operation object and the operation mode corresponding to the target gesture by using the number of fingers and the finger actions forming the target gesture includes:

if the number of fingers constituting the target gesture is N, and if the finger movement slides in a first designated direction, determining that an operation object corresponding to the target gesture is a word, a sentence or a segment in the target text, and determining that an operation mode corresponding to the target gesture is: reading the last object of the operation object corresponding to the target gesture, wherein N is an integer greater than or equal to 1 and less than or equal to 4, and the operation object corresponding to the target gesture is related to the value of N, wherein the single-finger sliding indicates that the operation object is a word, the double-finger sliding indicates that the operation object is a word, the three-finger sliding indicates that the operation object is a sentence, and the four-finger sliding indicates that the operation object is a segment;

2. The method according to claim 1, wherein determining the operation object and the operation mode corresponding to the target gesture by using the number of fingers and the finger movements constituting the target gesture comprises:

and determining an operation object and an operation mode corresponding to the target gesture by utilizing a preset gesture operation table and combining the number of fingers and the finger actions forming the target gesture.

3. The method according to claim 1, wherein determining the operation object and the operation mode corresponding to the target gesture by using the number of fingers and the finger movements constituting the target gesture comprises:

4. The method according to claim 1, wherein determining the operation object and the operation mode corresponding to the target gesture by using the number of fingers and the finger movements constituting the target gesture comprises:

5. The method according to claim 1, wherein determining the operation object and the operation mode corresponding to the target gesture by using the number of fingers and the finger movements constituting the target gesture comprises:

6. The method according to claim 1, wherein determining the operation object and the operation mode corresponding to the target gesture by using the number of fingers and the finger movements constituting the target gesture comprises:

7. The method according to claim 1, wherein determining the operation object and the operation mode corresponding to the target gesture by using the number of fingers and the finger movements constituting the target gesture comprises:

8. The method according to claim 1, wherein determining the operation object and the operation mode corresponding to the target gesture by using the number of fingers and the finger movements constituting the target gesture comprises:

9. A gesture-based text processing system, the system comprising:

a first determining unit configured to determine the number of fingers and finger actions that constitute the target gesture, the number of fingers and the finger actions being used to query a gesture operation table, the gesture operation table including: the corresponding relation between the combination of different finger numbers and different finger actions and the operation object, and the corresponding relation between the combination of different finger numbers and different finger actions and the operation mode;

The second processing unit is used for responding to the target gesture and processing the target text based on an operation mode corresponding to the target gesture if the target gesture does not have an operation object corresponding to the target gesture;

the second determining unit is specifically configured to determine that, if the number of fingers forming the target gesture is N and if the finger movement is sliding in the first specified direction, the operation object corresponding to the target gesture is a word, a sentence or a segment in the target text, and determine that the operation mode corresponding to the target gesture is: reading the last object of the operation object corresponding to the target gesture, wherein N is an integer greater than or equal to 1 and less than or equal to 4, and the operation object corresponding to the target gesture is related to the value of N, wherein the single-finger sliding indicates that the operation object is a word, the double-finger sliding indicates that the operation object is a word, the three-finger sliding indicates that the operation object is a sentence, and the four-finger sliding indicates that the operation object is a segment; if the number of fingers constituting the target gesture is N, and if the finger movement slides in a second designated direction, determining that the operation object corresponding to the target gesture is a word, a sentence or a segment in the target text, and determining that the operation mode corresponding to the target gesture is: reading an operation object corresponding to the target gesture; if the number of fingers constituting the target gesture is N, and if the finger movement slides in a third designated direction, determining that the operation object corresponding to the target gesture is a word, a sentence or a segment in the target text, and determining that the operation mode corresponding to the target gesture is: reading the next object of the operation object corresponding to the target gesture; if the number of fingers forming the target gesture is 1, and if the finger movement is stopped after sliding in a fourth designated direction, determining that the operation object corresponding to the target gesture is the full text in the target text, and determining that the operation mode corresponding to the target gesture is: reading the full text in the target text; if the number of fingers constituting the target gesture is 1, and if the finger action is single-finger clicking, determining that the target gesture does not have a corresponding operation object, and determining that the operation mode corresponding to the target gesture is as follows: the sentence in which the current cursor is located and all the following contents are read aloud.