CN111198644B

CN111198644B - Screen operation identification method and system of intelligent terminal

Info

Publication number: CN111198644B
Application number: CN201811367066.9A
Authority: CN
Inventors: 冯文康
Original assignee: Xian Yep Telecommunication Technology Co Ltd
Current assignee: Xian Yep Telecommunication Technology Co Ltd
Priority date: 2018-11-16
Filing date: 2018-11-16
Publication date: 2023-10-27
Anticipated expiration: 2038-11-16
Also published as: CN111198644A

Abstract

The invention discloses a method and a system for identifying screen operation of an intelligent terminal, wherein a plurality of distance detection modules are arranged below the screen of the intelligent terminal, and the identification method comprises the following steps: acquiring the corresponding relation between each distance detection module and each coordinate information on the screen; when at least one target object performs a blank operation, detecting different target objects through different distance detection modules and respectively acquiring distance information between each target object and a screen; the target coordinate information of the target object detected by the distance detection module on the screen corresponds to the coordinate information corresponding to the distance detection module one by one; and identifying the operation behavior of the blank operation of each target object on the screen according to the target coordinate information and the distance information. According to the invention, the screen is directly operated in the vertical direction and the horizontal direction in the space operation scene, the gesture operation is not required to be preset, the operation is simple, the practicability is high, and the user experience is improved.

Description

Screen operation identification method and system of intelligent terminal

Technical Field

The invention relates to the technical field of mobile terminals, in particular to a method and a system for identifying screen operation of an intelligent terminal.

Background

At present, besides the touch operation, the screen operation can also be operated by a space-apart gesture. However, the conventional method for operating the screen through the blank gesture has the following drawbacks: 1) The gesture motion recognition method has the advantages that the gesture motion recognition method can only realize the recognition of up-down and left-right operation parallel to the screen, so that the operability is simpler and limited for users, and the practicability is not strong; 2) Even if the application program can be opened, closed, uninstalled, installed, and the like, the operation needs to be preset by a user in advance, for example, the preset gesture "O" corresponds to the opening operation, the preset gesture "C" corresponds to the closing operation, and the like, so that the operation is complex for the user, and the user experience is poor.

Disclosure of Invention

The invention aims to solve the technical problems that operations corresponding to gesture actions in screen operations are required to be preset through blank gestures in the prior art, and the defects of complex operations, low practicality, poor user experience and the like exist.

The invention solves the technical problems by the following technical scheme:

The invention provides a method for identifying screen operation of an intelligent terminal, which comprises the following steps that a plurality of distance detection modules are arranged below the screen of the intelligent terminal:

acquiring the corresponding relation between each distance detection module and each coordinate information on the screen;

when at least one target object is in a blank operation, detecting different target objects through different distance detection modules and respectively acquiring distance information between each target object and the screen;

the distance detection module is used for detecting the coordinate information of the target object on the screen, wherein the coordinate information corresponds to the target object on the screen;

and identifying the operation behavior of the blank operation of each target object on the screen according to the target coordinate information and the distance information.

Optionally, the distance information includes vertical distance information between each of the target objects and the screen;

the step of identifying the operation behavior of the blank operation of each target object on the screen according to the target coordinate information and the distance information comprises the following steps:

And when the target coordinate information is kept unchanged in a first set time period and the vertical distance information is changed at least once in a first set time period and is increased after the first set time period, determining that the operation behavior comprises the clicking operation.

Optionally, when the target coordinate information remains unchanged in the first set duration and the vertical distance information changes from decreasing to increasing in the first set duration, determining that the operation behavior includes a click operation;

and when the target coordinate information is kept unchanged in the first set time period and the vertical distance information is changed in the first set time period after two times of decreasing, determining that the operation behavior comprises double-click operation.

the step of identifying the operation line of the blank operation of each target object on the screen according to the target coordinate information and the distance information comprises the following steps:

and when the target coordinate information is kept unchanged in a second set time period, the vertical distance information is changed in a reduced manner in the second set time period and the reduced distance is kept unchanged in a third set time period, determining that the operation behavior comprises long-press operation.

and when the change range of the vertical distance information is within the set range and the target coordinate information is changed within the fourth set time period, determining that the operation behavior comprises a sliding operation.

Optionally, before the step of identifying the operation behavior of the blank operation of each target object on the screen according to the target coordinate information and the distance information, the step of:

judging whether the target coordinate information and the distance information meet the identification conditions, if so, executing the step of identifying the operation behavior of the blank operation of each target object on the screen according to the target coordinate information and the distance information;

the identification condition comprises that the target coordinate information and the distance information are kept unchanged in a fifth set duration.

The invention also provides a recognition system for screen operation of the intelligent terminal, wherein a plurality of distance detection modules are arranged below the screen of the intelligent terminal, and the recognition system comprises a corresponding relation acquisition module, an information acquisition module and a recognition module;

The corresponding relation acquisition module is used for acquiring the corresponding relation between each distance detection module and each coordinate information on the screen;

the information acquisition module is used for detecting different target objects through different distance detection modules and acquiring distance information between each target object and the screen when at least one target object performs a space-apart operation;

the identification module is used for identifying the operation behavior of the blank operation of each target object on the screen according to the target coordinate information and the distance information.

the identification module is used for determining that the operation behavior comprises the clicking operation when the target coordinate information is kept unchanged in a first set time period and the vertical distance information is changed at least once in the first set time period, wherein the change is performed before the decrease and then the increase.

Optionally, the identifying module is configured to determine that the operation behavior includes a click operation when the target coordinate information remains unchanged in the first set duration and the vertical distance information changes from decreasing to increasing once in the first set duration;

the identification module is further configured to determine that the operation behavior includes a double-click operation when the target coordinate information remains unchanged within the first set duration and the vertical distance information changes from decreasing to increasing twice within the first set duration.

the identification module is used for determining that the operation behavior comprises long-press operation when the target coordinate information is kept unchanged in a second set time period, the vertical distance information is changed in a reduced mode in the second set time period and the reduced distance is kept unchanged in a third set time period.

the identification module is used for determining that the operation behavior comprises sliding operation when the change range of the vertical distance information is in the set range and the target coordinate information is changed in the fourth set duration.

Optionally, the identification system further comprises a judging module;

the judging module is used for judging whether the target coordinate information and the distance information meet the identification conditions, and if so, the identification module is called;

The invention has the positive progress effects that:

the intelligent terminal comprises a screen, a plurality of distance detection modules and a plurality of distance detection modules, wherein the distance detection modules are arranged below the screen of the intelligent terminal; the target coordinate information of the target object on the screen is coordinate information corresponding to a detection module for detecting the target object; and finally, identifying the operation behavior of the blank operation of each target object on the screen according to the target coordinate information and the distance information, thereby realizing the operation (including clicking, long pressing, sliding and the like) of the screen in the vertical direction and the horizontal direction directly in the blank operation scene, having no need of presetting gesture actions, having simple operation and strong practicability and improving the user experience.

Drawings

Fig. 1 is a flowchart of a method for identifying screen operation of an intelligent terminal according to embodiment 1 of the present invention.

Fig. 2 is a schematic diagram of a screen display of a method for identifying a screen operation of the intelligent terminal according to embodiment 1 of the present invention.

Fig. 3 is a flowchart of a method for identifying screen operation of the intelligent terminal according to embodiment 2 of the present invention.

Fig. 4 is a first screen display schematic diagram of a method for identifying a screen operation of the intelligent terminal according to embodiment 2 of the present invention.

Fig. 5 is a second screen display schematic diagram of a method for identifying a screen operation of the intelligent terminal according to embodiment 2 of the present invention.

Fig. 6 is a third screen display schematic diagram of a method for identifying a screen operation of the intelligent terminal according to embodiment 2 of the present invention.

Fig. 7 is a fourth screen display schematic diagram of a method for identifying a screen operation of the intelligent terminal according to embodiment 2 of the present invention.

Fig. 8 is a fifth screen display schematic diagram of a method for identifying a screen operation of the intelligent terminal according to embodiment 2 of the present invention.

Fig. 9 is a sixth screen display schematic diagram of a method for identifying a screen operation of the intelligent terminal according to embodiment 2 of the present invention.

Fig. 10 is a seventh screen display schematic diagram of a method for identifying a screen operation of the intelligent terminal according to embodiment 2 of the present invention.

Fig. 11 is a flowchart of a method for identifying screen operation of the intelligent terminal according to embodiment 3 of the present invention.

Fig. 12 is a flowchart of a method for identifying screen operation of the intelligent terminal according to embodiment 4 of the present invention.

Fig. 13 is a first screen display schematic diagram of a method for identifying a screen operation of the intelligent terminal according to embodiment 4 of the present invention.

Fig. 14 is a second screen display schematic diagram of a method for identifying a screen operation of the intelligent terminal according to embodiment 4 of the present invention.

Fig. 15 is a third screen display schematic diagram of a method for identifying a screen operation of the intelligent terminal according to embodiment 4 of the present invention.

Fig. 16 is a fourth screen display schematic diagram of a method for identifying a screen operation of the intelligent terminal according to embodiment 4 of the present invention.

Fig. 17 is a fifth screen display schematic diagram of a method for identifying a screen operation of the intelligent terminal according to embodiment 4 of the present invention.

Fig. 18 is a sixth screen display schematic diagram of a method for identifying a screen operation of the intelligent terminal according to embodiment 4 of the present invention.

Fig. 19 is a seventh screen display schematic diagram of a method for identifying a screen operation of the intelligent terminal according to embodiment 4 of the present invention.

Fig. 20 is an eighth screen display schematic diagram of a method for identifying a screen operation of the intelligent terminal according to embodiment 4 of the present invention.

Fig. 21 is a first screen display schematic diagram of a method for identifying a screen operation of the intelligent terminal according to embodiment 5 of the present invention.

Fig. 22 is a second screen display schematic diagram of a method for identifying a screen operation of the intelligent terminal according to embodiment 5 of the present invention.

Fig. 23 is a schematic block diagram of an identification system for screen operation of the intelligent terminal according to embodiment 6 of the present invention.

Fig. 24 is a schematic block diagram of an identification system for screen operation of the intelligent terminal according to embodiment 7 of the present invention.

Detailed Description

The invention is further illustrated by means of the following examples, which are not intended to limit the scope of the invention.

Example 1

In this embodiment, a plurality of distance detection modules are disposed below the screen of the intelligent terminal.

According to the actual demand, a plurality of distance detection modules can be arranged on the whole screen below the whole screen, so that the accuracy of the operation of identifying the space-free gestures of the target object (such as a finger and the like) is ensured.

The distance detection module includes a distance sensor, but is not limited to a distance sensor, and may also include any other device that can acquire distance information between each target object and the screen.

The intelligent terminal comprises a mobile phone, a tablet personal computer and other devices.

As shown in fig. 1, the method for identifying the screen operation of the intelligent terminal of the present embodiment includes:

s101, acquiring a corresponding relation between each distance detection module and each coordinate information on a screen;

Specifically, as shown in fig. 2, according to a plurality of distance detection modules set below the whole screen, the screen is divided into areas corresponding to each distance detection module, each area corresponds to one coordinate information, for example, the upper left corner of the screen is taken as origin coordinate information (x 0, y 0), and then the coordinate information (such as application programs APP1, APP2, APP3, APP 4) corresponding to any position in the screen can also be determined.

S102, when at least one target object is in a blank operation, detecting different target objects through different distance detection modules and respectively acquiring distance information between each target object and a screen;

the distance detection module is used for detecting the coordinate information of the target object on the screen, wherein the coordinate information corresponds to the coordinate information corresponding to the distance detection module;

specifically, when a single finger or multiple fingers of a user perform a blank operation on a screen, different distance detection modules detect different fingers and acquire distance information between each finger and the screen, and as the distance detection modules acquiring the distance information between the fingers and the screen have a one-to-one correspondence with each coordinate information on the screen, the specific positions (projection points) of the fingers projected on the screen can be marked by lighting (such as bright spots of a set pattern, cross marks and the like) on the screen and using the positions of the coordinate information corresponding to the distance detection modules for detecting the fingers, so that the user can conveniently know the projection positions of the fingers on the screen, and the operability of the mobile phone by the user is improved.

S103, identifying the operation behavior of the blank operation of each target object on the screen according to the target coordinate information and the distance information.

In the embodiment, a plurality of distance detection modules are arranged below a screen of the intelligent terminal, and when a blank operation is performed, different target objects are detected through the corresponding detection modules respectively and corresponding distance information is acquired through presetting a corresponding relation between each detection module and each coordinate information on the screen; the target coordinate information of the target object on the screen is coordinate information corresponding to a detection module for detecting the target object; and finally, identifying the operation behavior of the blank operation of each target object on the screen according to the target coordinate information and the distance information, thereby realizing the operation of the screen in the vertical direction and the horizontal direction directly in the blank operation scene, having no need of presetting gesture actions, having simple operation and strong practicability and improving the user experience.

Example 2

As shown in fig. 3, the method for identifying the screen operation of the intelligent terminal of the present embodiment is a further improvement of embodiment 1, specifically:

the distance information includes vertical distance information between each target object and the screen.

After step S102, before step S103, the method further comprises:

s1030, judging whether the target coordinate information and the distance information meet the identification conditions, and if so, executing step S103;

When the target coordinate information and the distance information do not meet the recognition condition, the operation of the target object at the moment is indicated to be general movement, and no practical operation significance exists.

Step S103 includes:

s1031, determining that the operation behavior comprises clicking operation when the target coordinate information is kept unchanged in the first set time period and the vertical distance information is changed at least once in a first set time period after being reduced and then increased.

Specifically, when the target coordinate information remains unchanged within the first set time period and the vertical distance information changes from first decrease to last increase within the first set time period, determining that the operation behavior comprises a click operation;

when the target coordinate information remains unchanged within the first set time period and the vertical distance information changes from decreasing to increasing twice within the first set time period, determining that the operation behavior comprises a double-click operation.

For a single click operation:

The marking operation behavior starts when the target coordinate information of the finger on the screen and the vertical distance information between the finger and the screen are kept unchanged for 0.2s (or more than 0.2 s); then, if the corresponding target coordinate information of the finger is unchanged within 0.1s, the vertical distance information is changed by reducing by 2cm (or more than 2 cm) and increasing by 2cm (or more than 2 cm) once, and the marking operation behavior is ended, and the whole operation process of the finger is defined as a clicking operation.

The setting of the time and distance in the present application is only one possible setting manner, and the present embodiment is not limited to the above time and distance values, and may be set according to the system performance and the user requirements.

Taking a single-finger clicking operation as an example, the application program APP3 is opened, and the specific identification process is as follows:

as shown in fig. 4, when the user prepares to start the space operation with a single finger, the distance detection module provided below the screen detects the finger, and the "cross" cursor a1 indicates the projection point of the finger on the screen corresponding at this time.

As shown in fig. 5, when the user's finger starts the blank operation, the distance detection module provided below the screen detects the finger, and the "cross" cursor a2 indicates that the finger is moved onto the icon of the application APP 3; if the target coordinate information of the finger of the user on the screen and the vertical distance information between the finger and the screen are kept unchanged within 0.2s, marking operation behavior is started; then, the distance detection module continuously detects the change of the finger position, if the corresponding target coordinate information of the finger within 0.1s is unchanged, the vertical distance information is reduced by 2cm at first after finishing once, and then the vertical distance information is increased by 2cm within 0.1s, the distance detection module sends the change information of the finger to the conversion module in the intelligent device, and the change information is converted into (0.2 s, -2cm,0.1s, +2cm, x, y), wherein (x, y) represents the corresponding target coordinate information of a single finger.

The conversion module judges that an event corresponding to finger operation is a single-finger single-click event according to the converted change information, and sends the single-finger single-click event and target coordinate information to a processing module of the intelligent terminal, and the processing module controls to open an application program APP3 (shown in FIG. 6 as a display interface of an application program APP3 homepage) according to the single-click event and the target coordinate information, and simultaneously the clicking operation is ended; the distance detection module continues to monitor the change condition of the target coordinate information of the finger on the screen and the distance information between the finger and the screen, and waits for executing the next blank operation.

Taking a double-finger clicking operation as an example, waking up the mobile phone, the specific identification process is as follows:

as shown in fig. 7, when the user starts the blank operation with two fingers while the mobile phone is in the sleep mode (the area a in fig. 7 represents the state of the screen at this time), the distance detection module arranged below the screen detects the fingers, and the target coordinate information of the two fingers of the user on the screen and the vertical distance information between the two fingers and the screen are kept unchanged within 0.2s, at this time, the marking operation behavior starts; then, the distance detection module continuously detects the change of the double-finger position, if the corresponding target coordinate information of the double fingers in 0.1s is unchanged, the vertical distance information is reduced by 2cm at first, and then the vertical distance information in 0.1s is increased by 2cm, the distance detection module sends the change information of the double fingers to the conversion module in the intelligent equipment, and the change information is converted into (0.2 s, -2cm, +2cm, x1, y1, x2, y 2), wherein (x 1, y 1) and (x 2, y 2) respectively represent the corresponding target coordinate information of the double fingers.

The conversion module judges that an event corresponding to finger operation is a double-finger single-click event according to the converted change information, and sends the double-finger single-click event and target coordinate information to a processing module of the intelligent terminal, and the processing module controls the awakening to lighten a mobile phone screen (shown in fig. 8) according to the double-finger single-click event and the target coordinate information, and meanwhile, the clicking operation is finished; the distance detection module continues to monitor the change condition of the target coordinate information of the finger on the screen and the distance information between the finger and the screen, and waits for executing the next blank operation.

Taking a three-finger clicking operation as an example, the function of quickly opening a mobile phone camera is specifically identified as follows:

when a user starts a blank operation by adopting three fingers, a distance detection module arranged below a screen detects the fingers, the target coordinate information of the three fingers of the user on the screen and the distance information between the three fingers and the screen are kept unchanged within 0.2s, and at the moment, marking operation behaviors are started; then, the distance detection module continuously detects the change of the position of the three fingers, if the corresponding target coordinate information of the three fingers within 0.1s is unchanged, the vertical distance information is reduced by 2cm at first and then the vertical distance information is increased by 2cm within 0.1s, the distance detection module sends the change information of the fingers to the conversion module in the intelligent device, and the change information is converted into (0.2 s, -2cm,0.1s, +2cm, x1, y1, x2, y2, x3, y 3), wherein (x 1, y 1), (x 2, y 2) and (x 3, y 3) respectively represent the corresponding target coordinate information of the three fingers.

The conversion module judges that an event corresponding to finger operation is a three-finger single click event according to the converted change information, and sends the three-finger single click event and target coordinate information to a processing module of the intelligent terminal, and the processing module controls to open a mobile phone camera according to the three-finger single click event and the target coordinate information, and meanwhile the click operation is finished; the distance detection module continues to monitor the change condition of the target coordinate information of the finger on the screen and the distance information between the finger and the screen, and waits for executing the next blank operation.

Taking a four-finger clicking operation as an example, a Wi-Fi (wireless local area network) function is quickly opened, and the specific identification process is as follows:

when a user starts a blank operation by adopting four fingers, a distance detection module arranged below a screen detects the fingers, the target coordinate information of the four fingers on the screen of the user and the distance information between the four fingers and the screen are kept unchanged within 0.2s, and at the moment, marking operation behaviors are started; then, the distance detection module continuously detects the change of the four-finger position, if the corresponding target coordinate information of the four fingers within 0.1s is unchanged, the vertical distance information is reduced by 2cm at first, and then the vertical distance information is increased by 2cm within 0.1s, the distance detection module sends the change information of the fingers to the conversion module in the intelligent device, and the change information is converted into (0.2 s, -2cm,0.1s, +2cm, x1, y1, x2, y2, x3, y3, x4, y 4), wherein (x 1, y 1), (x 2, y 2), (x 3, y 3) and (x 4, y 4) respectively represent the corresponding target coordinate information of the four fingers.

The conversion module judges that an event corresponding to finger operation is a four-finger single click event according to the converted change information, and sends the four-finger single click event and target coordinate information to a processing module of the intelligent terminal, and the processing module controls to open a Wi-Fi function according to the four-finger single click event and the target coordinate information, and meanwhile the clicking operation is finished; the distance detection module continues to monitor the change condition of the target coordinate information of the finger on the screen and the distance information between the finger and the screen, and waits for executing the next blank operation.

Taking five-finger clicking operation to rapidly open the Bluetooth function as an example, the specific identification process is as follows:

when a user starts the blank operation by adopting five fingers, a distance detection module arranged below the screen detects the fingers, the target coordinate information of the five fingers of the user on the screen and the distance information between the five fingers of the user and the screen are kept unchanged within 0.2s, and the marking operation behavior starts at the moment; then, the distance detection module continuously detects the change of the position of the five fingers, if the corresponding target coordinate information of the five fingers within 0.1s is unchanged, the vertical distance information is reduced by 2cm at first, and then the vertical distance information is increased by 2cm within 0.1s, the distance detection module sends the change information of the fingers to the conversion module in the intelligent device, and the change information is converted into (0.2 s, -2cm,0.1s, +2cm, x1, y1, x2, y2, x3, y3, x4, y4, x5, y 5), wherein (x 1, y 1), (x 2, y 2), (x 3, y 3), (x 4, y 4) and (x 5, y 5) respectively represent the corresponding target coordinate information of the five fingers.

The conversion module judges that an event corresponding to finger operation is a five-finger clicking event according to the converted change information, and sends the five-finger clicking event and target coordinate information to a processing module of the intelligent terminal, and the processing module controls Bluetooth to be opened according to the five-finger clicking event and the target coordinate information, and meanwhile the clicking operation is finished; the distance detection module continues to monitor the change condition of the target coordinate information of the finger on the screen and the distance information between the finger and the screen, and waits for executing the next blank operation.

For a double click operation:

the marking operation behavior starts when the target coordinate information of the finger on the screen and the vertical distance information between the finger and the screen are kept unchanged for 0.2s (or more than 0.2 s); then, if the corresponding target coordinate information of the finger is unchanged within 0.1s, the vertical distance information is changed by firstly reducing by 2cm (or more than 2 cm) and then increasing by 2cm (or more than 2 cm), and then the change by firstly reducing by 2cm (or more than 2 cm) and then increasing by 2cm (or more than 2 cm) is completed within 0.1s, at the moment, the marking operation behavior is ended, and the whole operation process of the finger is defined as double-click operation.

Taking a single-finger double-click quick deletion application program APP1 as an example, the specific identification process is as follows:

As shown in fig. 9, when the user starts a blank operation with single-finger preparation, the distance detection module provided below the screen detects a finger, and the "cross" cursor b1 indicates that the single finger is moved onto the icon of the application APP1; when the target coordinate information of a single finger of a user on the screen and the distance information between the single finger and the screen are kept unchanged within 0.2s, marking operation behavior starts; then, the distance detection module continuously detects the change of the finger position, if the corresponding target coordinate information of the finger in 0.1s is unchanged, the vertical distance information is changed by firstly decreasing 2cm and then increasing 2cm, and then the vertical distance information is changed by firstly decreasing 2cm and then increasing 2cm in 0.1s, the distance detection module sends the change information of the finger to the conversion module in the intelligent device, and the change information is converted into (0.2 s, -2cm,0.1s, +2cm,0.1s, -2cm, +2cm, x1, y 1), wherein (x 1, y 1) represents the target coordinate information corresponding to the single finger.

The conversion module judges that an event corresponding to finger operation is a single-finger double-click event according to the converted change information, and sends the single-finger double-click event and target coordinate information to a processing module of the intelligent terminal, and the processing module controls deletion of an application program APP1 according to the single-click event and the target coordinate information; as shown in fig. 10, the pop-up window interface that appears after the application APP1 is double-clicked, determines whether to delete the application APP1 again for the user, and the clicking operation is ended at the same time; the distance detection module continues to monitor the change condition of the target coordinate information of the finger on the screen and the distance information between the finger and the screen, and waits for executing the next blank operation.

Taking a double-finger double-click quick entering telephone dialing interface as an example, the specific identification process is as follows:

when a user adopts double fingers to prepare to start the blank operation, a distance detection module arranged below a screen detects the fingers, and when the target coordinate information of the double fingers of the user on the screen and the distance information between the double fingers and the screen are kept unchanged within 0.2s, marking operation behaviors are started; then, the distance detection module continuously detects the change of the finger position, if the corresponding target coordinate information of the finger is unchanged within 0.1s, the vertical distance information is changed by firstly decreasing 2cm and then increasing 2cm, and then the vertical distance information is changed by firstly decreasing 2cm and then increasing 2cm within 0.1s, the distance detection module sends the change information of the finger to the conversion module in the intelligent device, and the change information is converted into (0.2 s, -2cm,0.1s, +2cm,0.1s, -2cm, +2cm, x1, y1, x2, y 2), wherein (x 1, y 1) and (x 2, y 2) respectively represent the corresponding target coordinate information of the double finger.

The conversion module judges that the event corresponding to the finger operation is a double-finger double-click event according to the converted change information, and sends the double-finger double-click event and target coordinate information to the processing module of the intelligent terminal, and the processing module controls to enter a telephone dialing interface according to the single-click event and the target coordinate information, so that a user can dial quickly; at the same time, the clicking operation is ended; the distance detection module continues to monitor the change condition of the target coordinate information of the finger on the screen and the distance information between the finger and the screen, and waits for executing the next blank operation.

Taking a three-finger double-click to realize a rapid screen capturing function as an example, the specific identification process is as follows:

when a user adopts three fingers to prepare to start the blank operation, a distance detection module arranged below a screen detects the fingers, and the target coordinate information of the three fingers of the user on the screen and the distance information between the three fingers and the screen are kept unchanged within 0.2s, so that the marking operation behavior starts; then, the distance detection module continuously detects the change of the finger position, if the corresponding target coordinate information of the finger in 0.1s is unchanged, the vertical distance information is changed by firstly decreasing 2cm and then increasing 2cm, and then the vertical distance information is changed by firstly decreasing 2cm and then increasing 2cm in 0.1s, the distance detection module sends the change information of the finger to the conversion module in the intelligent device, and the change information is converted into (0.2 s, -2cm,0.1s, +2cm,0.1s, -2cm, +2cm, x1, y1, x2, y2, x3 and y 3), wherein (x 1, y 1), (x 2, y 2) and (x 3, y 3) respectively represent the target coordinate information corresponding to three fingers.

The conversion module judges that an event corresponding to finger operation is a three-finger double-click event according to the converted change information, and sends the three-finger double-click event and target coordinate information to a processing module of the intelligent terminal, and the processing module controls screen capturing operation according to the single-click event and the target coordinate information; at the same time, the clicking operation is ended; the distance detection module continues to monitor the change condition of the target coordinate information of the finger on the screen and the distance information between the finger and the screen, and waits for executing the next blank operation.

Taking four-finger double-click to realize the function of quickly opening the recording as an example, the specific identification process is as follows:

when a user adopts four fingers to prepare to start the blank operation, a distance detection module arranged below a screen detects the fingers, and when the target coordinate information of the four fingers on the screen of the user and the distance information between the four fingers and the screen are kept unchanged within 0.2s, the marking operation behavior starts; then, the distance detection module continuously detects the change of the finger position, if the corresponding target coordinate information of the finger is unchanged within 0.1s, the vertical distance information is changed by firstly decreasing 2cm and then increasing 2cm, and then the vertical distance information is changed by firstly decreasing 2cm and then increasing 2cm within 0.1s, the distance detection module sends the change information of the finger to the conversion module in the intelligent device, and the change information is converted into (0.2 s, -2cm,0.1s, +2cm,0.1s, -2cm, +2cm, x1, y1, x2, y2, x3, y3, x4, y 4), wherein (x 1, y 1), (x 2, y 2), (x 3, y 3) and (x 4, y 4) respectively represent the target coordinate information corresponding to four fingers.

The conversion module judges that an event corresponding to finger operation is a four-finger double-click event according to the converted change information, and sends the four-finger double-click event and target coordinate information to a processing module of the intelligent terminal, and the processing module controls to open a recording function interface according to the single-click event and the target coordinate information, so that a user can conveniently perform recording operation; at the same time, the clicking operation is ended; the distance detection module continues to monitor the change condition of the target coordinate information of the finger on the screen and the distance information between the finger and the screen, and waits for executing the next blank operation.

Taking five-finger double-click to realize the function of quickly entering the mobile phone setting interface as an example, the specific identification process is as follows:

when a user adopts five fingers to prepare to start the blank operation, a distance detection module arranged below a screen detects the fingers, and when the target coordinate information of the five fingers of the user on the screen and the distance information between the five fingers and the screen are kept unchanged within 0.2s, the marking operation behavior starts; then, the distance detection module continuously detects the change of the finger position, if the corresponding target coordinate information of the finger is unchanged within 0.1s, the vertical distance information is changed by firstly decreasing 2cm and then increasing 2cm, and then the change of firstly decreasing 2cm and then increasing 2cm is completed within 0.1s, the distance detection module sends the change information of the finger to the conversion module in the intelligent device, and the change information is converted into (0.2 s, -2cm,0.1s, +2cm,0.1s, -2cm, +2cm, x1, y1, x2, y2, x3, y3, x4, y4, x5 and y 5), wherein (x 1, y 1), (x 2, y 2), (x 3, y 3), (x 4, y 4) and (x 5, y 5) respectively represent the target coordinate information corresponding to five fingers.

The conversion module judges that the event corresponding to the finger operation is a five-finger double-click event according to the converted change information, and sends the five-finger double-click event and target coordinate information to the processing module of the intelligent terminal, and the processing module controls to enter a mobile phone setting interface according to the single-click event and the target coordinate information, so that a user can conveniently set functions of the mobile phone; at the same time, the clicking operation is ended; the distance detection module continues to monitor the change condition of the target coordinate information of the finger on the screen and the distance information between the finger and the screen, and waits for executing the next blank operation.

In the embodiment, a plurality of distance detection modules are arranged below a screen of the intelligent terminal, and when a blank operation is performed, different target objects are detected through the corresponding detection modules respectively and corresponding distance information is acquired through presetting a corresponding relation between each detection module and each coordinate information on the screen; the target coordinate information of the target object on the screen is coordinate information corresponding to a detection module for detecting the target object; and finally, recognizing the clicking operation of the blank operation of each target object on the screen according to the change condition of the target coordinate information and the distance information in the first set time length, so that the clicking operation is directly performed on the screen in the vertical direction and the horizontal direction in a blank operation scene, the mobile phone application program and the like are quickly called, gesture actions are not required to be set in advance, the operation is simple, the practicability is high, and the user experience is improved.

Example 3

As shown in fig. 11, the method for identifying the screen operation of the intelligent terminal of the present embodiment is a further improvement of embodiment 1, specifically:

the distance information includes vertical distance information between each target object and the screen;

after step S102, before step S103, the method further comprises:

S1030, judging whether the target coordinate information and the distance information meet the identification conditions, and if so, identifying the operation behavior of the blank operation of each target object on the screen;

Step S103 includes:

s1032, when the target coordinate information is kept unchanged in the second set time period, the vertical distance information is changed in a reduced manner in the second set time period, and the reduced distance is kept unchanged in the third set time period, determining the operation behavior comprises long-press operation.

When the target coordinate information of the finger on the screen and the distance information between the finger and the screen are kept unchanged within 0.2s (or more than 0.2 s), marking operation behavior starts; subsequently, if the vertical distance information between the finger and the screen is reduced by 2cm (or more than 2 cm) and the reduced distance is maintained for 1s (or more than 1 s), the entire operation procedure of the finger is defined as a long press operation.

After the blank operation of the finger is identified as the long press operation, if the vertical distance information between the finger and the screen changes or the target coordinate information, the long press operation is ended.

Taking a single-finger long-press operation as an example, the method for starting the multiple content selection function comprises the following specific identification processes:

when a user adopts a single finger to prepare to start a blank operation, a distance detection module arranged below a screen detects a finger, a projection point corresponding to the finger is displayed on the screen by a cross cursor, and when the target coordinate information of the single finger of the user on the screen and the distance information between the single finger and the screen are kept unchanged within 0.2s, marking operation behaviors are started; then, the distance detection module continuously detects the change of the finger position, and if the vertical distance information between the finger and the screen is reduced by 2cm and the reduced distance is kept unchanged within 1s, the distance detection module sends the change information of the finger to the conversion module in the intelligent device, and converts the change information into (0.2 s, -2cm,1s, x, y), wherein (x, y) represents target coordinate information corresponding to a single finger.

The conversion module judges that an event corresponding to finger operation is a single-finger long-press event according to the converted change information, and sends the single-finger long-press event and target coordinate information to a processing module of the intelligent terminal, and the processing module controls and opens a plurality of content selection functions according to the single-click event and the target coordinate information, so that a user can conveniently select a plurality of items; meanwhile, the single-finger long-press operation is finished; the distance detection module continues to monitor the change condition of the target coordinate information of the finger on the screen and the distance information between the finger and the screen, and waits for executing the next blank operation.

And so on, for other multi-finger long-press operations such as a double-finger long-press operation, a three-finger long-press operation and the like, each long-press operation corresponds to one execution operation, and the specific identification process is the identification process of the single-finger long-press operation, so that the details are not repeated here.

In the embodiment, a plurality of distance detection modules are arranged below a screen of the intelligent terminal, and when a blank operation is performed, different target objects are detected through the corresponding detection modules respectively and corresponding distance information is acquired through presetting a corresponding relation between each detection module and each coordinate information on the screen; the target coordinate information of the target object on the screen is coordinate information corresponding to a detection module for detecting the target object; and finally, recognizing the long-press operation of the blank operation of each target object on the screen according to the change condition of the target coordinate information and the distance information in the second set time length, so that the long-press operation of the screen in the vertical direction and the horizontal direction is directly realized in the blank operation scene, gesture actions are not required to be set in advance, the operation is simple, the practicability is high, and the user experience is improved.

Example 4

As shown in fig. 12, the method for identifying the screen operation of the intelligent terminal of the present embodiment is a further improvement of embodiment 1, specifically:

after step S102, before step S103, the method further comprises:

Step S103 includes:

s1033, when the range of variation of the vertical distance information is within the set range and the target coordinate information is changed within the fourth set period, determining that the operation behavior includes a sliding operation.

When the target coordinate information of the finger on the screen and the distance information between the finger and the screen are kept unchanged within 0.2s (or more than 0.2 s), marking operation behavior starts; subsequently, if the vertical distance information between the finger and the screen varies within a range of ± -0.5cm and the target coordinate information (i.e., the projected point of the finger on the screen) varies, the entire operation procedure of the finger is defined as a sliding operation.

After the finger is identified as a sliding operation, if the finger moves out of the screen (i.e. there is no corresponding projection point on the screen), or the finger stays somewhere for longer than 0.2s after moving for a period of time, or the vertical distance information between the finger and the screen changes, the sliding operation is ended.

When the sliding operation is performed by adopting two fingers, the operation of expanding the screen display by the two fingers and the operation of contracting the screen display by the two fingers are included, and the specific steps are as follows:

when the target coordinate information of the finger on the screen and the distance information between the finger and the screen are kept unchanged within 0.2s (or more than 0.2 s), marking operation behavior starts; then, if the distance between the two fingers is enlarged, the operation of the fingers is positioned as a double-finger enlarging operation;

after the operation of the finger is positioned as the two-finger enlarging operation, if the stay time of the two fingers at the current position exceeds 0.2s or the vertical distance information between the finger and the screen is changed, the two-finger enlarging operation is ended.

When the target coordinate information of the finger on the screen and the distance information between the finger and the screen are kept unchanged within 0.2s (or more than 0.2 s), marking operation behavior starts; then, the distance between the two fingers is reduced, and the operation of the fingers is positioned as a double-finger reduction operation;

after the operation of the finger is positioned as the two-finger zoom-out operation, if the stay time of the two fingers at the current position exceeds 0.2s or the vertical distance information between the finger and the screen changes, the two-finger zoom-out operation is ended.

Taking a single-finger sliding operation as an example, the mobile phone unlocking function is realized, and the specific identification process is as follows:

as shown in fig. 13, when the mobile phone enters the unlocking interface, the unlocking operation needs to be performed according to the gesture password preset by the user. When a user adopts a single finger to prepare to start a blank operation, a distance detection module arranged below a screen detects a finger, and a cross cursor c1 on the screen represents the position of a projection point corresponding to the finger; when the target coordinate information of a single finger of a user on the screen and the vertical distance information between the single finger and the screen are kept unchanged within 0.2s, marking operation behavior starts; then, the finger moves parallel to the mobile phone screen to finish a preset screen unlocking gesture graph, such as 'L' (as shown in fig. 14), the distance detection module sends the change information of the finger to the conversion module in the intelligent device, and converts the change information into (0.2 s, x, y, x+m, x+n, y+m), wherein (x, y) represents the target coordinate information corresponding to the 'cross' cursor c1, (x, y+m) represents the m point in fig. 14, and (x+n, y+m) represents the target coordinate information corresponding to the 'cross' cursor c 2.

The conversion module judges that an event corresponding to finger operation is a single-finger sliding event according to the converted change information, and sends the single-finger sliding event and target coordinate information to a processing module of the intelligent terminal, and the processing module controls sliding unlocking according to the single-click event and the target coordinate information.

When the finger slides to a point 'cross' cursor c2 ending, the vertical distance information between the finger and the screen changes, or the stay time exceeds 0.2s at the point, the single-finger sliding operation is considered to be ended; the distance detection module continues to monitor the change condition of the target coordinate information of the finger on the screen and the distance information between the finger and the screen, and waits for executing the next blank operation.

Taking the example of image amplification realized by double-finger expansion sliding operation, the specific identification process is as follows:

as shown in fig. 15, when the user starts the blank operation using the double-finger preparation, the distance detection module provided below the screen detects the finger, the positions of the projection points corresponding to the double-finger are respectively indicated by the "cross" cursors d1 and d2 on the screen, wherein the target coordinate information corresponding to d1 and d2 is (x 1, y 1) and (x 2, y 2), and the target coordinate information of the double-finger on the screen and the vertical distance information between the double-finger and the screen of the user are all kept unchanged within 0.2s, and the marking operation behavior starts at this time; then, when the two fingers move in parallel to the screen of the mobile phone in opposite directions, the distance detection module sends the change information of the fingers to the conversion module in the intelligent device, and converts the change information into (0.2 s, x1-n1, y1+m1, x2+n2, y2-m 2), wherein, (x 1-n1, y1+m1) represents the corresponding target coordinate information after d1 movement, and (x 2+n2, y2-m 2) represents the corresponding target coordinate information after d2 movement, as shown in fig. 16.

The conversion module judges that an event corresponding to finger operation is a double-finger expansion sliding event according to the converted change information, and sends the double-finger expansion sliding event and target coordinate information to a processing module of the intelligent terminal, and the processing module controls an enlarged image according to the clicking event and the target coordinate information. When the vertical distance information between the finger and the screen changes or the stay time exceeds 0.2s at a certain point, the double-finger expansion sliding operation is considered to be ended; the distance detection module continues to monitor the change condition of the target coordinate information of the finger on the screen and the distance information between the finger and the screen, and waits for executing the next blank operation.

Taking the example of realizing image reduction by double-finger reduction sliding operation, the specific identification process is as follows:

as shown in fig. 17, when the user starts the blank operation using the double-finger preparation, the distance detection module disposed under the screen detects the finger, and the positions of the projection points corresponding to the double-finger are respectively indicated by the "cross" cursors e1 and e2 on the screen, wherein the target coordinate information corresponding to the e1 and e2 is (x 1, y 1) and (x 2, y 2), and the target coordinate information of the double-finger on the screen and the vertical distance information between the double-finger and the screen of the user are all kept unchanged within 0.2s, and the marking operation behavior starts at this time; then, when the two fingers move in parallel to the mobile phone screen in opposite directions, the distance detection module sends the change information of the fingers to the conversion module in the intelligent device, and converts the change information into (0.2 s, x1-n1, y1-m1, x2+n2, y2+m2), wherein as shown in fig. 18, (x 1-n1, y1-m 1) represents the corresponding target coordinate information after e1 movement, and (x2+n2, y2+m2)) represents the corresponding target coordinate information after e2 movement.

The conversion module judges that an event corresponding to finger operation is a double-finger reduced sliding event according to the converted change information, and sends the double-finger reduced sliding event and target coordinate information to a processing module of the intelligent terminal, and the processing module controls a reduced image according to the single-click event and the target coordinate information. When the vertical distance information between the finger and the screen changes or the stay time exceeds 0.2s at a certain point, the double-finger expansion sliding operation is considered to be ended; the distance detection module continues to monitor the change condition of the target coordinate information of the finger on the screen and the distance information between the finger and the screen, and waits for executing the next blank operation.

Taking three-finger sliding operation as an example, the mobile phone volume adjustment is realized, and the specific identification process is as follows:

as shown in fig. 19, when the user starts the space operation with three-finger preparation, the distance detection module disposed below the screen detects the finger, and the "cross" cursors f1, f2, f3 on the screen respectively represent the positions of the projected points corresponding to the two fingers, wherein the target coordinate information corresponding to f1, f2, f3 is (x 1, y 1), (x 2, y 2) and (x 3, y 3), respectively. When the target coordinate information of a single finger of a user on the screen and the distance information between the single finger and the screen are kept unchanged within 0.2s, marking operation behavior starts; then, when the finger moves parallel to the mobile phone screen, the distance detection module sends the change information of the finger to the conversion module in the intelligent device, and converts the change information into (0.2 s, x1+n1, y1+m1, x2+n2, y2+m2, x3+n3, y3+m3), wherein, (x1+n1, y1+m1) represents the corresponding target coordinate information after f1 movement, (x2+n2, y2+m2) represents the corresponding target coordinate information after f2 movement, and (x3+n3, y3+m3) represents the corresponding target coordinate information after f3 movement, as shown in fig. 20.

The conversion module judges that an event corresponding to finger operation is a three-finger sliding event according to the converted change information, and sends the three-finger sliding event and target coordinate information to a processing module of the intelligent terminal, and the processing module controls and adjusts the volume of the mobile phone according to the clicking event and the target coordinate information.

And so on, for other multi-finger sliding operations such as four-finger sliding operation, five-finger sliding operation and the like, each sliding operation corresponds to one execution operation, and the specific identification process is the identification process of the single-finger, double-finger and three-finger long-press operation, so that the description is omitted here.

In the embodiment, a plurality of distance detection modules are arranged below a screen of the intelligent terminal, and when a blank operation is performed, different target objects are detected through the corresponding detection modules respectively and corresponding distance information is acquired through presetting a corresponding relation between each detection module and each coordinate information on the screen; the target coordinate information of the target object on the screen is coordinate information corresponding to a detection module for detecting the target object; and finally, identifying the sliding operation of the blank operation of each target object on the screen according to the change condition of the target coordinate information and the distance information in the fourth set time length, so that the screen is directly operated (including clicking, sliding and the like) in the vertical direction and the horizontal direction in a blank operation scene, a mobile phone application program and the like are quickly called, gesture actions are not required to be set in advance, the operation is simple, the practicability is strong, and the user experience is improved.

Example 5

The method for identifying the screen operation of the intelligent terminal in this embodiment is a further improvement of embodiment 1, specifically:

according to the screen operation identification method of the intelligent terminal, the space gesture operation identification is not only the identification of one operation behavior, but also the identification of various combined operation behaviors (such as any combination of clicking, long-pressing and sliding operations) so as to facilitate the user to finish more operations.

Taking deletion of the application APP1 as an example, the specific identification process is as follows:

as shown in fig. 9, when the user starts the blank operation with a single finger, the distance detection module provided below the screen detects a finger, and the "cross" cursor b1 indicates that the single finger moves onto the icon of the application APP 1; then, as shown in fig. 21, recognizing the operation of the single-finger long-press application program APP1, and displaying a recycle bin icon g on the interface; subsequently, as shown in fig. 22, a one-finger sliding operation is recognized, and the application APP1 is dragged to the recycle bin icon g, so that the deletion operation of the application APP1 is completed, that is, the operation procedure includes both the long press operation and the sliding operation.

In the embodiment, the recognition of the gesture operation in the screen operation of the intelligent terminal is not only the recognition of one operation behavior, but also the recognition of multiple combined operation behaviors, so that gesture operation in a space scene is enriched, more operations are conveniently completed by a user, the practicability is high, and the user experience is improved.

Example 6

As shown in fig. 23, the recognition system of the screen operation of the intelligent terminal of the present embodiment includes a correspondence relation acquisition module 1, an information acquisition module 2, and a recognition module 3.

The corresponding relation acquisition module 1 is used for acquiring the corresponding relation between each distance detection module and each coordinate information on the screen;

The information acquisition module 2 is used for detecting different target objects through different distance detection modules and acquiring distance information between each target object and the screen when at least one target object performs a space-apart operation;

The identification module 3 is used for identifying the operation behavior of the blank operation of each target object on the screen according to the target coordinate information and the distance information.

Example 7

As shown in fig. 24, the recognition system of the screen operation of the intelligent terminal of the present embodiment is a further improvement of embodiment 6, specifically:

the recognition system further comprises a judgment module 4.

The judging module 4 is used for judging whether the target coordinate information and the distance information meet the identification conditions, and if so, the identifying module 3 is called to identify the operation behavior of the blank operation of each target object on the screen;

The identification module 3 is configured to determine that the operation behavior includes a click operation when the target coordinate information remains unchanged for a first set period of time and the vertical distance information changes at least once in a first set period of time.

Specifically, the identification module 3 is configured to determine that the operation behavior includes a click operation when the target coordinate information remains unchanged for a first set period of time and the vertical distance information changes from one decrease to another increase within the first set period of time;

the identification module 3 is further configured to determine that the operation behavior includes a double click operation when the target coordinate information remains unchanged for the first set period of time and the vertical distance information changes from decreasing to increasing twice for the first set period of time.

For a single click operation:

the marking operation behavior starts when the target coordinate information of the finger on the screen and the vertical distance information between the finger and the screen are kept unchanged for 0.2s (or more than 0.2 s); then, if the corresponding target coordinate information of the finger is unchanged within 0.1s, the vertical distance information is changed by reducing by 2cm (or more than 2 cm) and increasing by 2cm (or more than 2 cm) once, and the marking operation behavior is ended, and the whole operation process of the finger is defined as a clicking operation. Taking a single-finger clicking operation as an example, the application program APP3 is opened, and the specific identification process is as follows:

as shown in fig. 7, when the mobile phone is in the sleep mode and the user starts the blank operation by using two fingers, the distance detection module arranged below the screen detects the fingers, and the target coordinate information of the two fingers of the user on the screen and the vertical distance information between the two fingers and the screen are kept unchanged within 0.2s, so that the marking operation behavior starts; then, the distance detection module continuously detects the change of the double-finger position, if the corresponding target coordinate information of the double fingers in 0.1s is unchanged, the vertical distance information is reduced by 2cm at first, and then the vertical distance information in 0.1s is increased by 2cm, the distance detection module sends the change information of the double fingers to the conversion module in the intelligent equipment, and the change information is converted into (0.2 s, -2cm, +2cm, x1, y1, x2, y 2), wherein (x 1, y 1) and (x 2, y 2) respectively represent the corresponding target coordinate information of the double fingers.

For a double click operation:

as shown in fig. 9, when the user starts a blank operation with single-finger preparation, the distance detection module provided below the screen detects a finger, and the "cross" cursor b1 indicates that the single finger is moved onto the icon of the application APP 1; when the target coordinate information of a single finger of a user on the screen and the distance information between the single finger and the screen are kept unchanged within 0.2s, marking operation behavior starts; then, the distance detection module continuously detects the change of the finger position, if the corresponding target coordinate information of the finger in 0.1s is unchanged, the vertical distance information is changed by firstly decreasing 2cm and then increasing 2cm, and then the vertical distance information is changed by firstly decreasing 2cm and then increasing 2cm in 0.1s, the distance detection module sends the change information of the finger to the conversion module in the intelligent device, and the change information is converted into (0.2 s, -2cm,0.1s, +2cm,0.1s, -2cm, +2cm, x1, y 1), wherein (x 1, y 1) represents the target coordinate information corresponding to the single finger.

Example 8

the recognition system further comprises a judgment module 4.

The identification module 3 is configured to determine that the operation behavior includes a long press operation when the target coordinate information remains unchanged for a second set period of time, the vertical distance information undergoes a reduced change for the second set period of time, and the reduced distance remains unchanged for a third set period of time. When the target coordinate information of the finger on the screen and the distance information between the finger and the screen are kept unchanged within 0.2s (or more than 0.2 s), marking operation behavior starts; subsequently, if the vertical distance information between the finger and the screen is reduced by 2cm (or more than 2 cm) and the reduced distance is maintained for 1s (or more than 1 s), the entire operation procedure of the finger is defined as a long press operation.

In the embodiment, a plurality of distance detection modules are arranged below a screen of the intelligent terminal, and when a blank operation is performed, different target objects are detected through the corresponding detection modules respectively and corresponding distance information is acquired through presetting a corresponding relation between each detection module and each coordinate information on the screen; the target coordinate information of the target object on the screen is coordinate information corresponding to a detection module for detecting the target object; and finally, recognizing the long-press operation of the blank operation of each target object on the screen according to the change condition of the target coordinate information and the distance information in the second set time length, so that the long-press operation of the screen in the blank operation scene is realized, the mobile phone application program and the like are quickly and quickly adjusted, gesture actions are not required to be set in advance, the operation is simple, the practicability is strong, and the user experience is improved.

Example 9

the recognition system further comprises a judgment module 4.

The identification module 3 is configured to determine that the operation behavior includes a sliding operation when the range of variation of the vertical distance information is within the set range and the target coordinate information is changed within the fourth set period.

After the operation of the finger is positioned as the two-finger zoom-out operation, if the stay time of the two fingers at the current position exceeds 0.2s or the vertical distance information between the finger and the screen changes, the two-finger zoom-out operation is ended. Taking a single-finger sliding operation as an example, the mobile phone unlocking function is realized, and the specific identification process is as follows:

as shown in fig. 17, when the user starts the blank operation using the double-finger preparation, the distance detection module disposed under the screen detects the finger, and the positions of the projection points corresponding to the double-finger are respectively indicated by the "cross" cursors e1 and e2 on the screen, wherein the target coordinate information corresponding to the e1 and e2 is (x 1, y 1) and (x 2, y 2), and the target coordinate information of the double-finger on the screen and the vertical distance information between the double-finger and the screen of the user are all kept unchanged within 0.2s, and the marking operation behavior starts at this time; then, when the two fingers move in parallel to the mobile phone screen in opposite directions, the distance detection module sends the change information of the fingers to the conversion module in the intelligent device, and converts the change information into (0.2 s, x1-n1, y1-m1, x2+n2, y2+m2), wherein as shown in fig. 17, (x 1-n1, y1-m 1) represents the corresponding target coordinate information after e1 movement, and (x2+n2, y2+m2)) represents the corresponding target coordinate information after e2 movement.

Example 10

The recognition system of screen operation of the intelligent terminal of the present embodiment is a further improvement of embodiment 6, specifically:

the recognition of the space-apart gesture operation in the recognition system of the screen operation of the intelligent terminal in this embodiment is not only to recognize one operation behavior, but also to recognize multiple combined operation behaviors (such as any combination of clicking, long pressing and sliding operations), so as to facilitate the user to complete more operations.

While specific embodiments of the invention have been described above, it will be appreciated by those skilled in the art that these are by way of example only, and the scope of the invention is defined by the appended claims. Various changes and modifications to these embodiments may be made by those skilled in the art without departing from the principles and spirit of the invention, but such changes and modifications fall within the scope of the invention.

Claims

1. The method for identifying the screen operation of the intelligent terminal is characterized in that a plurality of distance detection modules are arranged below the screen of the intelligent terminal, and the method for identifying the screen operation of the intelligent terminal comprises the following steps:

2. The method for recognizing a screen operation of an intelligent terminal according to claim 1, wherein the distance information includes vertical distance information between each of the target objects and the screen;

and when the target coordinate information is kept unchanged in a first set time period and the vertical distance information is changed at least once in a first set time period and then is increased, determining that the operation behavior comprises clicking operation.

3. The method for recognizing a screen operation of an intelligent terminal according to claim 2, wherein the operation behavior is determined to include a click operation when the target coordinate information remains unchanged for the first set period of time and the vertical distance information changes from decrease to increase once within the first set period of time;

4. The method for recognizing a screen operation of an intelligent terminal according to claim 1, wherein the distance information includes vertical distance information between each of the target objects and the screen;

and when the target coordinate information is kept unchanged in a second set time period, the vertical distance information is changed in a reduced manner in the second set time period, and the reduced distance is kept unchanged in a third set time period, determining that the operation behavior comprises long-press operation.

5. The method for recognizing a screen operation of an intelligent terminal according to claim 1, wherein the distance information includes vertical distance information between each of the target objects and the screen;

6. The method for recognizing a screen operation of an intelligent terminal according to claim 1, wherein the step of recognizing an operation behavior of the blank operation of each of the target objects on the screen according to the target coordinate information and the distance information comprises:

7. The identification system for the screen operation of the intelligent terminal is characterized in that a plurality of distance detection modules are arranged below the screen of the intelligent terminal, and the identification system comprises a corresponding relation acquisition module, an information acquisition module and an identification module;

8. The recognition system of screen operation of a smart terminal according to claim 7, wherein the distance information includes vertical distance information between each of the target objects and the screen;

the identification module is used for determining that the operation behavior comprises clicking operation when the target coordinate information is kept unchanged in a first set time period and the vertical distance information is changed at least once in the first set time period, wherein the change is performed before the decrease and then the increase.

9. The recognition system of screen operations of a smart terminal of claim 8, wherein the recognition module is configured to determine that the operation behavior comprises a click operation when the target coordinate information remains unchanged for the first set period of time and the vertical distance information changes from decreasing to increasing once within the first set period of time;

10. The recognition system of screen operation of a smart terminal according to claim 7, wherein the distance information includes vertical distance information between each of the target objects and the screen;

the identification module is used for determining that the operation behavior comprises long-press operation when the target coordinate information is kept unchanged in a second set time period, the vertical distance information is changed in a reduced mode in the second set time period, and the reduced distance is protected unchanged in a third set time period.

11. The recognition system of screen operation of a smart terminal according to claim 7, wherein the distance information includes vertical distance information between each of the target objects and the screen;

12. The recognition system of screen operations of a smart terminal of claim 7, wherein the recognition system further comprises a judgment module;