CN114138110A - Gesture recording method and system based on android system and storage medium - Google Patents

Gesture recording method and system based on android system and storage medium Download PDF

Info

Publication number
CN114138110A
CN114138110A CN202111307003.6A CN202111307003A CN114138110A CN 114138110 A CN114138110 A CN 114138110A CN 202111307003 A CN202111307003 A CN 202111307003A CN 114138110 A CN114138110 A CN 114138110A
Authority
CN
China
Prior art keywords
gesture
recording
action information
actions
transparent layer
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202111307003.6A
Other languages
Chinese (zh)
Inventor
唐小刚
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Chengdu Yingchao Technology Co ltd
Original Assignee
Chengdu Yingchao Technology Co ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Chengdu Yingchao Technology Co ltd filed Critical Chengdu Yingchao Technology Co ltd
Priority to CN202111307003.6A priority Critical patent/CN114138110A/en
Publication of CN114138110A publication Critical patent/CN114138110A/en
Pending legal-status Critical Current

Links

Images

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/017Gesture based interaction, e.g. based on a set of recognized hand gestures
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/011Arrangements for interaction with the human body, e.g. for user immersion in virtual reality

Landscapes

  • Engineering & Computer Science (AREA)
  • General Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Human Computer Interaction (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • User Interface Of Digital Computer (AREA)
  • Telephone Function (AREA)

Abstract

The invention relates to a gesture recording method, a system and a storage medium based on an android system, wherein the method is to start a barrier-free service function and a floating window permission based on the android system; creating a transparent layer in a floating window mode, recording a plurality of gestures in the transparent layer through a barrier-free service function and a floating window authority to obtain a plurality of gesture action information, and collecting the plurality of gesture action information in a list mode to obtain a gesture collection; and analyzing each gesture action information of the gesture set, storing the analyzed gesture action information into the gesture set, and restoring the recording track when all the gesture action information is analyzed. According to the invention, the transparent layer is created through the barrier-free service function and the floating window authority of the android system, and the usability of automatic operation is improved through the form of gesture recording, the operation difficulty of a user and a special scene can be quickly responded, and the consumption of resources such as time, manpower and the like is saved.

Description

Gesture recording method and system based on android system and storage medium
Technical Field
The invention relates to the field of gesture action processing, in particular to a gesture recording method and system based on an android system and a storage medium.
Background
In an android smart phone, instructions such as simulated sliding/clicking are realized by means of a barrier-free service function in an android system, and the coordinate points corresponding to operations such as clicking, sliding and returning each time need to be recorded in the process of realizing automatic operation, and the point positions need to be recorded in the process. In the marking process, the current situations that coordinate points of a multi-screen interface are overlapped and disordered, gesture clicks are not consistent, and automatic operation steps are complicated are caused.
Disclosure of Invention
The invention aims to solve the technical problem of the prior art and provides a gesture recording method and system based on an android system and a storage medium.
The technical scheme for solving the technical problems is as follows: a gesture recording method based on an android system comprises the following steps:
starting a barrier-free service function and a floating window permission based on the android system;
creating a transparent layer in a floating window form, recording a plurality of gesture actions in the transparent layer through the barrier-free service function and the floating window authority to obtain a plurality of gesture action information, wherein the gesture action information comprises gesture actions and coordinates corresponding to the gesture actions, and gathering the gesture action information in a list form to obtain a gesture set;
and analyzing each gesture action information of the gesture set, storing the analyzed gesture action information into the gesture set, and restoring the recording track when all the gesture action information is analyzed.
The invention has the beneficial effects that: the transparent layer is created through the barrier-free service function and the floating window authority of the android system, and the usability of automatic operation is improved through the gesture recording mode, the operation difficulty of a user and a special scene can be responded quickly, and meanwhile, the consumption of resources such as time, manpower and the like is saved.
On the basis of the technical scheme, the invention can be further improved as follows.
Further, the process of creating the transparent layer in the form of the floating window includes:
and calling a windows manager api component in the android system to create a transparent layer in a suspended window form, wherein the length and width areas of the transparent layer are filled in the whole mobile phone screen.
The beneficial effect of adopting the further scheme is that: the floating window is presented in a floating window mode, and the operation of returning to the desktop can be achieved.
Further, the process of recording a plurality of gesture actions in the transparent layer through the barrier-free service function and the floating window permission to obtain a plurality of gesture action information includes:
recording a plurality of click gesture actions in the transparent layer through the barrier-free service function to obtain a plurality of click gesture actions and coordinates corresponding to the click gesture actions,
and recording a plurality of long pressing and sliding gesture actions in the transparent layer through the floating window authority to obtain a plurality of long pressing and sliding gesture actions and coordinates corresponding to the long pressing and sliding gesture actions.
The beneficial effect of adopting the further scheme is that: the current situations that automatic clicking of coordinate points is disordered, and consistency operations such as gestures and clicking are inconvenient to record are solved.
Further, the analyzing each piece of gesture action information of the gesture set, and storing the analyzed gesture action information into the gesture set includes:
and calling an accessibilityService component in the android system to analyze each gesture action information of the gesture set, and storing the analyzed gesture action information into the gesture set according to the analysis sequence.
The beneficial effect of adopting the further scheme is that: through the form of gesture recording, the usability of automatic operation is improved, the operation difficulty of a user and a special scene can be responded quickly, and the operation track of the user is restored.
Another technical solution of the present invention for solving the above technical problems is as follows: a gesture recording system based on an android system, comprising:
the starting module is used for starting a barrier-free service function and a floating window permission based on the android system;
the recording module is used for creating a transparent layer in a floating window form, recording a plurality of gesture actions in the transparent layer through the barrier-free service function and the floating window authority to obtain a plurality of gesture action information, wherein the gesture action information comprises gesture actions and coordinates corresponding to the gesture actions, and gathering the gesture action information in a list form to obtain a gesture set;
and the processing module is used for analyzing each gesture action information of the gesture set, storing the analyzed gesture action information into the gesture set, and restoring the recording track when all the gesture action information is analyzed.
Another technical solution of the present invention for solving the above technical problems is as follows: a gesture recording system based on an android system comprises a memory, a processor and a computer program which is stored in the memory and can run on the processor, and when the computer program is executed by the processor, the gesture recording method based on the android system is realized.
Another technical solution of the present invention for solving the above technical problems is as follows: a computer-readable storage medium storing a computer program which, when executed by a processor, implements the android system based gesture recording method as described above.
Drawings
Fig. 1 is a flowchart of a gesture recording method according to an embodiment of the present invention;
fig. 2 is a block diagram of a gesture recording system according to an embodiment of the present invention.
Detailed Description
The principles and features of this invention are described below in conjunction with the following drawings, which are set forth by way of illustration only and are not intended to limit the scope of the invention.
Example 1:
as shown in fig. 1, a gesture recording method based on the android system includes the following steps:
starting a barrier-free service function and a floating window permission based on the android system;
creating a transparent layer in a floating window form, recording a plurality of gesture actions in the transparent layer through the barrier-free service function and the floating window authority to obtain a plurality of gesture action information, wherein the gesture action information comprises gesture actions (gesture operation) and coordinates corresponding to the gesture actions (gesture operation), and gathering the gesture action information in a list form to obtain a gesture set;
and analyzing each gesture action information of the gesture set, storing the analyzed gesture action information into the gesture set, and restoring the recording track when all the gesture action information is analyzed.
The android system is 7.0 and above. It should be appreciated that the barrier-free service function and floating window permissions native to the android system are initiated in the android system.
In the embodiment, the transparent layer is created through the barrier-free service function and the floating window permission of the android system, and the usability of automatic operation is improved through the gesture recording mode, the operation difficulty of a user and a special scene can be responded quickly, and the consumption of resources such as time, manpower and the like is saved.
Specifically, the process of creating the transparent layer in the form of the floating window includes:
and calling a windows manager api component in the android system to create a transparent layer in a suspended window form, wherein the length and width areas of the transparent layer are filled in the whole mobile phone screen.
In the above embodiment, the operation is also performed by using the floating window to return to the desktop.
Specifically, the process of recording a plurality of gesture actions in the transparent layer through the barrier-free service function and the floating window permission to obtain a plurality of gesture action information includes:
recording a plurality of click gesture actions in the transparent layer through the barrier-free service function to obtain a plurality of click gesture actions and coordinates corresponding to the click gesture actions,
and recording a plurality of long pressing and sliding gesture actions in the transparent layer through the floating window authority to obtain a plurality of long pressing and sliding gesture actions and coordinates corresponding to the long pressing and sliding gesture actions.
The multiple click gesture actions comprise single click, double click and the like.
It should be appreciated that drawing a layer of transparent area in the form of a floating window, up to back to the desktop, may also operate. The view area is used for acquiring and storing a series of gesture actions such as clicking, double clicking, long pressing, sliding and the like, and meanwhile, the current view layer needs to be penetrated and responded, touch events are not intercepted, and normal interaction of the mobile phone is achieved. And after the recording finishes a series of gesture point positions, finishing the recording. And then carrying out area coordinate storage such as clicking/long pressing/double clicking/sliding.
For example, windowmanager, lay particle, FLAG _ NOT _ TOUCHABLE | FLAG _ NOT _ FOCUSABLE, performs a penetration response. And simultaneously, gesture actions such as clicking, double clicking, long pressing, up-down sliding, left-right sliding and the like are obtained through event monitoring, and corresponding x and y axes are recorded and stored in the set in order.
In the embodiment, the current situations that the automatic click coordinate point is disordered, and the consistency operations such as gestures and clicks are inconvenient to record are solved.
Specifically, the process of analyzing each piece of gesture action information of the gesture set and storing the analyzed gesture action information into the gesture set includes:
and calling an accessibilityService component in the android system to analyze each gesture action information of the gesture set, and storing the analyzed gesture action information into the gesture set according to the analysis sequence.
The accessibiliservie component analyzes each gesture action information of the gesture set, and extracts various gestures such as: and the clicked coordinate position, the sliding direction, the distance and the like are stored in the gesture set according to the time sequence, and finally the last operation track of the user is restored.
In the embodiment, the usability of automatic operation is improved through the gesture recording mode, the operation difficulty of a user and a special scene can be quickly responded, and the operation track of the user is restored.
Example 2:
as shown in fig. 2, a gesture recording system based on the android system includes:
the starting module is used for starting a barrier-free service function and a floating window permission based on the android system;
the recording module is used for creating a transparent layer in a floating window form, recording a plurality of gesture actions in the transparent layer through the barrier-free service function and the floating window authority to obtain a plurality of gesture action information, wherein the gesture action information comprises gesture actions and coordinates corresponding to the gesture actions, and gathering the gesture action information in a list form to obtain a gesture set;
and the processing module is used for analyzing each gesture action information of the gesture set, storing the analyzed gesture action information into the gesture set, and restoring the recording track when all the gesture action information is analyzed.
In the embodiment, the transparent layer is created through the barrier-free service function and the floating window permission of the android system, and the usability of automatic operation is improved through the gesture recording mode, the operation difficulty of a user and a special scene can be responded quickly, and the consumption of resources such as time, manpower and the like is saved.
Specifically, in the recording module, the process of creating the transparent layer in the form of the floating window includes:
and calling a windows manager api component in the android system to create a transparent layer in a suspended window form, wherein the length and width areas of the transparent layer are filled in the whole mobile phone screen.
Specifically, in the recording module, a plurality of gesture actions are recorded in the transparent layer through the barrier-free service function and the floating window permission, and a process of obtaining information of the plurality of gesture actions includes:
recording a plurality of click actions in the transparent layer through the barrier-free service function to obtain a plurality of click actions and coordinates corresponding to the click actions,
and recording a plurality of gesture actions in the transparent layer through the floating window authority to obtain a plurality of gesture actions and coordinates corresponding to the gesture actions.
Specifically, in the processing module, the process of analyzing each piece of gesture action information of the gesture set and storing the analyzed gesture action information into the gesture set includes:
and calling an accessibilityService component in the android system to analyze each gesture action information of the gesture set, and storing the analyzed gesture action information into the gesture set according to the analysis sequence.
Example 3:
a gesture recording system based on an android system comprises a memory, a processor and a computer program which is stored in the memory and can run on the processor, and when the computer program is executed by the processor, the gesture recording method based on the android system is realized.
Example 4:
a computer-readable storage medium storing a computer program which, when executed by a processor, implements the android system based gesture recording method as described above.
The invention solves the problems that the automatic click coordinate point is disordered, the consistency operations such as gestures and clicks are inconvenient to record, and the user and scene requirements cannot be rapidly processed. According to the invention, the gesture recording instruction can be rapidly made according to the scene requirements of the user, and the good experience of the user is ensured.
It is clear to those skilled in the art that, for convenience and brevity of description, the specific working processes of the above-described apparatuses and units may refer to the corresponding processes in the foregoing method embodiments, and are not described herein again.
In the several embodiments provided in the present application, it should be understood that the disclosed apparatus and method may be implemented in other ways. For example, the above-described apparatus embodiments are merely illustrative, and for example, a division of a unit is merely a logical division, and an actual implementation may have another division, for example, a plurality of units or components may be combined or integrated into another system, or some features may be omitted, or not executed.
Units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of the embodiment of the present invention.
In addition, functional units in the embodiments of the present invention may be integrated into one processing unit, or each unit may exist alone physically, or two or more units are integrated into one unit. The integrated unit can be realized in a form of hardware, and can also be realized in a form of a software functional unit.
The integrated unit, if implemented in the form of a software functional unit and sold or used as a stand-alone product, may be stored in a computer readable storage medium. Based on such understanding, the technical solution of the present invention essentially or partially contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product stored in a storage medium and including instructions for causing a computer device (which may be a personal computer, a server, or a network device) to execute all or part of the steps of the method according to the embodiments of the present invention. And the aforementioned storage medium includes: a U-disk, a removable hard disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a magnetic disk or an optical disk, and other various media capable of storing program codes.
The technical solutions provided by the embodiments of the present invention are described in detail above, and the principles and embodiments of the present invention are explained in this patent by applying specific examples, and the descriptions of the embodiments above are only used to help understanding the principles of the embodiments of the present invention; the present invention is not limited to the above preferred embodiments, and any modifications, equivalent replacements, improvements, etc. within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims (10)

1. A gesture recording method based on an android system is characterized by comprising the following steps:
starting a barrier-free service function and a floating window permission based on the android system;
creating a transparent layer in a floating window form, recording a plurality of gesture actions in the transparent layer through the barrier-free service function and the floating window authority to obtain a plurality of gesture action information, wherein the gesture action information comprises gesture actions and coordinates corresponding to the gesture actions, and gathering the gesture action information in a list form to obtain a gesture set;
and analyzing each gesture action information of the gesture set, storing the analyzed gesture action information into the gesture set, and restoring the recording track when all the gesture action information is analyzed.
2. The gesture recording method according to claim 1, wherein the process of creating transparent layers in the form of floating windows comprises:
and calling a windows manager api component in the android system to create a transparent layer in a suspended window form, wherein the length and width areas of the transparent layer are filled in the whole mobile phone screen.
3. The gesture recording method according to claim 1, wherein the recording of a plurality of gesture actions in the transparent layer through the barrier-free service function and the floating window permission to obtain a plurality of gesture action information includes:
recording a plurality of click gesture actions in the transparent layer through the barrier-free service function to obtain a plurality of click gesture actions and coordinates corresponding to the click gesture actions,
and recording a plurality of long pressing and sliding gesture actions in the transparent layer through the floating window authority to obtain a plurality of long pressing and sliding gesture actions and coordinates corresponding to the long pressing and sliding gesture actions.
4. The method for recording gestures according to any one of claims 1 to 3, wherein the process of parsing each piece of gesture action information of the gesture set and storing the parsed gesture action information into the gesture set comprises:
and calling an accessibilityService component in the android system to analyze each gesture action information of the gesture set, and storing the analyzed gesture action information into the gesture set according to the analysis sequence.
5. A system is recorded to gesture based on android system, its characterized in that includes:
the starting module is used for starting a barrier-free service function and a floating window permission based on the android system;
the recording module is used for creating a transparent layer in a floating window form, recording a plurality of gesture actions in the transparent layer through the barrier-free service function and the floating window authority to obtain a plurality of gesture action information, wherein the gesture action information comprises gesture actions and coordinates corresponding to the gesture actions, and gathering the gesture action information in a list form to obtain a gesture set;
and the processing module is used for analyzing each gesture action information of the gesture set, storing the analyzed gesture action information into the gesture set, and restoring the recording track when all the gesture action information is analyzed.
6. The gesture recording system according to claim 5, wherein the process of creating the transparent layer in the form of a floating window in the recording module comprises:
and calling a windows manager api component in the android system to create a transparent layer in a suspended window form, wherein the length and width areas of the transparent layer are filled in the whole mobile phone screen.
7. The gesture recording system according to claim 5, wherein in the recording module, a plurality of gesture actions are recorded in the transparent layer through the barrier-free service function and the floating window permission, and a process of obtaining information of the plurality of gesture actions includes:
recording a plurality of click actions in the transparent layer through the barrier-free service function to obtain a plurality of click actions and coordinates corresponding to the click actions,
and recording a plurality of gesture actions in the transparent layer through the floating window authority to obtain a plurality of gesture actions and coordinates corresponding to the gesture actions.
8. The system for recording gestures according to any one of claims 5 to 7, wherein the process of parsing each gesture information of the gesture set and storing the parsed gesture information into the gesture set in the processing module comprises:
and calling an accessibilityService component in the android system to analyze each gesture action information of the gesture set, and storing the analyzed gesture action information into the gesture set according to the analysis sequence.
9. An android system based gesture recording system comprising a memory, a processor and a computer program stored in the memory and executable on the processor, characterized in that when the computer program is executed by the processor, the android system based gesture recording method according to any one of claims 1 to 4 is implemented.
10. A computer-readable storage medium, in which a computer program is stored, which, when being executed by a processor, implements the android system based gesture recording method according to any one of claims 1 to 4.
CN202111307003.6A 2021-11-05 2021-11-05 Gesture recording method and system based on android system and storage medium Pending CN114138110A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202111307003.6A CN114138110A (en) 2021-11-05 2021-11-05 Gesture recording method and system based on android system and storage medium

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202111307003.6A CN114138110A (en) 2021-11-05 2021-11-05 Gesture recording method and system based on android system and storage medium

Publications (1)

Publication Number Publication Date
CN114138110A true CN114138110A (en) 2022-03-04

Family

ID=80392276

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202111307003.6A Pending CN114138110A (en) 2021-11-05 2021-11-05 Gesture recording method and system based on android system and storage medium

Country Status (1)

Country Link
CN (1) CN114138110A (en)

Citations (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20090265671A1 (en) * 2008-04-21 2009-10-22 Invensense Mobile devices with motion gesture recognition
WO2014072734A1 (en) * 2012-11-08 2014-05-15 Rawcliffe Dave Gesture input method and apparatus
CN105224212A (en) * 2014-07-01 2016-01-06 中国移动通信集团公司 A kind of terminal screen unlock method, device and terminal
CN105867751A (en) * 2015-01-20 2016-08-17 腾讯科技(深圳)有限公司 Method and device for processing operation information
CN107197332A (en) * 2017-06-09 2017-09-22 天脉聚源(北京)传媒科技有限公司 The backstage method for pushing and device of news
CN107193794A (en) * 2017-06-28 2017-09-22 广州视源电子科技股份有限公司 The annotation method and device of display content
JP2019008728A (en) * 2017-06-28 2019-01-17 コニカミノルタ株式会社 Document management system, and document management device
CN112947747A (en) * 2021-02-02 2021-06-11 深圳市江元科技(集团)有限公司 Gesture navigation operation method and device, terminal equipment and storage medium

Patent Citations (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20090265671A1 (en) * 2008-04-21 2009-10-22 Invensense Mobile devices with motion gesture recognition
WO2014072734A1 (en) * 2012-11-08 2014-05-15 Rawcliffe Dave Gesture input method and apparatus
CN105224212A (en) * 2014-07-01 2016-01-06 中国移动通信集团公司 A kind of terminal screen unlock method, device and terminal
CN105867751A (en) * 2015-01-20 2016-08-17 腾讯科技(深圳)有限公司 Method and device for processing operation information
CN107197332A (en) * 2017-06-09 2017-09-22 天脉聚源(北京)传媒科技有限公司 The backstage method for pushing and device of news
CN107193794A (en) * 2017-06-28 2017-09-22 广州视源电子科技股份有限公司 The annotation method and device of display content
JP2019008728A (en) * 2017-06-28 2019-01-17 コニカミノルタ株式会社 Document management system, and document management device
CN112947747A (en) * 2021-02-02 2021-06-11 深圳市江元科技(集团)有限公司 Gesture navigation operation method and device, terminal equipment and storage medium

Similar Documents

Publication Publication Date Title
US20210243493A1 (en) Video processing method, video processing device, and storage medium
US10552644B2 (en) Method and apparatus for displaying information content
US10762277B2 (en) Optimization schemes for controlling user interfaces through gesture or touch
WO2018109589A1 (en) Updating data records by adding editing functions to non-editable display elements
US20190026065A1 (en) Systems and methods for enabling replay of internet co-browsing
US20150193549A1 (en) History as a branching visualization
WO2017140242A1 (en) Information processing method and client
WO2015130721A1 (en) Inline and context aware query box
CN104820683A (en) Terminal
CN104796487A (en) Social interaction method and related equipment
CN112073301B (en) Method, device and computer readable medium for deleting chat group members
JP2017515241A (en) Element deletion method and apparatus based on touch panel
CN109726231A (en) Work order data display method, device, computer equipment and storage medium
CN112817817A (en) Buried point information query method and device, computer equipment and storage medium
US20150199105A1 (en) Automatic selection of center of rotation for graphical scenes
CN114138110A (en) Gesture recording method and system based on android system and storage medium
CN104679395A (en) Document presenting method and user terminal
US20160070409A1 (en) Control apparatus, information system, display control method and medium
CN102541487A (en) Method and system for printing screenshots
WO2018194853A1 (en) Enhanced inking capabilities for content creation applications
CN103297315A (en) Information processing method and device
JP2017102847A (en) Information processing system, relay device, method, and program
CN102437969A (en) Method and system for loading hyperlink data in instant messaging tool
CN111552956A (en) Role authority control method and device for background management
CA2939881A1 (en) Asset collection service through capture of content

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination