CN116931735A - AR (augmented reality) glasses display terminal equipment key suspension position identification system and method - Google Patents

AR (augmented reality) glasses display terminal equipment key suspension position identification system and method Download PDF

Info

Publication number
CN116931735A
CN116931735A CN202310968918.4A CN202310968918A CN116931735A CN 116931735 A CN116931735 A CN 116931735A CN 202310968918 A CN202310968918 A CN 202310968918A CN 116931735 A CN116931735 A CN 116931735A
Authority
CN
China
Prior art keywords
frequency
feedback data
finger tip
capacitance
finger
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202310968918.4A
Other languages
Chinese (zh)
Inventor
姜公略
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Beijing Xingzhe Wujiang Technology Co ltd
Original Assignee
Beijing Xingzhe Wujiang Technology Co ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Beijing Xingzhe Wujiang Technology Co ltd filed Critical Beijing Xingzhe Wujiang Technology Co ltd
Priority to CN202310968918.4A priority Critical patent/CN116931735A/en
Publication of CN116931735A publication Critical patent/CN116931735A/en
Pending legal-status Critical Current

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/011—Arrangements for interaction with the human body, e.g. for user immersion in virtual reality
    • A—HUMAN NECESSITIES
    • A63—SPORTS; GAMES; AMUSEMENTS
    • A63F—CARD, BOARD, OR ROULETTE GAMES; INDOOR GAMES USING SMALL MOVING PLAYING BODIES; VIDEO GAMES; GAMES NOT OTHERWISE PROVIDED FOR
    • A63F13/00—Video games, i.e. games using an electronically generated display having two or more dimensions
    • A63F13/20—Input arrangements for video game devices
    • A63F13/21—Input arrangements for video game devices characterised by their sensors, purposes or types
    • A63F13/214—Input arrangements for video game devices characterised by their sensors, purposes or types for locating contacts on a surface, e.g. floor mats or touch pads
    • A63F13/2145—Input arrangements for video game devices characterised by their sensors, purposes or types for locating contacts on a surface, e.g. floor mats or touch pads the surface being also a display device, e.g. touch screens
    • A—HUMAN NECESSITIES
    • A63—SPORTS; GAMES; AMUSEMENTS
    • A63F—CARD, BOARD, OR ROULETTE GAMES; INDOOR GAMES USING SMALL MOVING PLAYING BODIES; VIDEO GAMES; GAMES NOT OTHERWISE PROVIDED FOR
    • A63F13/00—Video games, i.e. games using an electronically generated display having two or more dimensions
    • A63F13/25—Output arrangements for video game devices
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/10—Pre-processing; Data cleansing
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
    • G06F18/213—Feature extraction, e.g. by transforming the feature space; Summarisation; Mappings, e.g. subspace methods
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/03—Arrangements for converting the position or the displacement of a member into a coded form
    • G06F3/041—Digitisers, e.g. for touch screens or touch pads, characterised by the transducing means
    • G06F3/0416—Control or interface arrangements specially adapted for digitisers
    • G06F3/04162—Control or interface arrangements specially adapted for digitisers for exchanging data with external devices, e.g. smart pens, via the digitiser sensing hardware
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/03—Arrangements for converting the position or the displacement of a member into a coded form
    • G06F3/041—Digitisers, e.g. for touch screens or touch pads, characterised by the transducing means
    • G06F3/0416—Control or interface arrangements specially adapted for digitisers
    • G06F3/04166—Details of scanning methods, e.g. sampling time, grouping of sub areas or time sharing with display driving
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/03—Arrangements for converting the position or the displacement of a member into a coded form
    • G06F3/041—Digitisers, e.g. for touch screens or touch pads, characterised by the transducing means
    • G06F3/044—Digitisers, e.g. for touch screens or touch pads, characterised by the transducing means by capacitive means
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/048—Interaction techniques based on graphical user interfaces [GUI]
    • G06F3/0487—Interaction techniques based on graphical user interfaces [GUI] using specific features provided by the input device, e.g. functions controlled by the rotation of a mouse with dual sensing arrangements, or of the nature of the input device, e.g. tap gestures based on pressure sensed by a digitiser
    • G06F3/0488—Interaction techniques based on graphical user interfaces [GUI] using specific features provided by the input device, e.g. functions controlled by the rotation of a mouse with dual sensing arrangements, or of the nature of the input device, e.g. tap gestures based on pressure sensed by a digitiser using a touch-screen or digitiser, e.g. input of commands through traced gestures
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00—Arrangements for program control, e.g. control units
    • G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/44—Arrangements for executing specific programs
    • G06F9/451—Execution arrangements for user interfaces
    • A—HUMAN NECESSITIES
    • A63—SPORTS; GAMES; AMUSEMENTS
    • A63F—CARD, BOARD, OR ROULETTE GAMES; INDOOR GAMES USING SMALL MOVING PLAYING BODIES; VIDEO GAMES; GAMES NOT OTHERWISE PROVIDED FOR
    • A63F2300/00—Features of games using an electronically generated display having two or more dimensions, e.g. on a television screen, showing representations related to the game
    • A63F2300/10—Features of games using an electronically generated display having two or more dimensions, e.g. on a television screen, showing representations related to the game characterized by input arrangements for converting player-generated signals into game device control signals
    • A63F2300/1068—Features of games using an electronically generated display having two or more dimensions, e.g. on a television screen, showing representations related to the game characterized by input arrangements for converting player-generated signals into game device control signals being specially adapted to detect the point of contact of the player on a surface, e.g. floor mat, touch pad
    • A63F2300/1075—Features of games using an electronically generated display having two or more dimensions, e.g. on a television screen, showing representations related to the game characterized by input arrangements for converting player-generated signals into game device control signals being specially adapted to detect the point of contact of the player on a surface, e.g. floor mat, touch pad using a touch screen
    • A—HUMAN NECESSITIES
    • A63—SPORTS; GAMES; AMUSEMENTS
    • A63F—CARD, BOARD, OR ROULETTE GAMES; INDOOR GAMES USING SMALL MOVING PLAYING BODIES; VIDEO GAMES; GAMES NOT OTHERWISE PROVIDED FOR
    • A63F2300/00—Features of games using an electronically generated display having two or more dimensions, e.g. on a television screen, showing representations related to the game
    • A63F2300/30—Features of games using an electronically generated display having two or more dimensions, e.g. on a television screen, showing representations related to the game characterized by output arrangements for receiving control signals generated by the game device
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F2203/00—Indexing scheme relating to G06F3/00 - G06F3/048
    • G06F2203/048—Indexing scheme relating to G06F3/048
    • G06F2203/04804—Transparency, e.g. transparent or translucent windows
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F2218/00—Aspects of pattern recognition specially adapted for signal processing
    • G06F2218/02—Preprocessing
    • G06F2218/04—Denoising
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F2218/00—Aspects of pattern recognition specially adapted for signal processing
    • G06F2218/08—Feature extraction
    • G06F2218/10—Feature extraction by analysing the shape of a waveform, e.g. extracting parameters relating to peaks

Abstract

The invention discloses a key suspension position identification system and method for AR glasses display terminal equipment, wherein the method comprises the steps of initializing and establishing a connection relation; acquiring the display picture of the terminal equipment and capacitance feedback data of a capacitance screen in real time; acquiring a coordinate point of the lowest point of the finger tip through capacitance feedback data; calculating and obtaining real-time vertical distance between the lowest point of the finger tip and the capacitive screen according to the coordinate point of the lowest point of the finger tip; displaying a transparent circular covering layer on the terminal equipment according to the real-time vertical distance and the effective suspension time T1 of the finger tip; and finally fusing the circular mask layer with a display picture of the terminal equipment and synchronizing the fused circular mask layer with the AR glasses.

Description

AR (augmented reality) glasses display terminal equipment key suspension position identification system and method
Technical Field
The invention relates to the field of man-machine interaction, in particular to a key suspension position identification system and method for AR (augmented reality) glasses display terminal equipment.
Background
When a terminal device such as a mobile phone or a palm phone is used for game entertainment, the traditional entertainment mode can cause that part of a screen of the terminal device is blocked by fingers due to clicking or suspension control of the fingers, so that playability is reduced;
After the AR glasses appear, the game picture is displayed through the AR glasses, and the terminal equipment is controlled to be clicked or suspended by fingers, so that the game picture is not blocked, and the entertainment is improved;
however, when the user performs the control, the user may not be skilled in using the terminal device, so that the user may not be aware of the position of the user's finger above the screen of the terminal device (i.e., the floating position of the user's finger may not be clearly known), thereby affecting the operation of the user, further causing incompatibility of the control manner, and making the user feel bad.
Disclosure of Invention
The invention aims to provide a key suspension position identification system and method for AR (augmented reality) glasses display terminal equipment, and solves the technical problems pointed out in the prior art.
The invention provides an AR (augmented reality) glasses display terminal equipment key suspension position identification system, which comprises AR glasses and terminal equipment;
the terminal equipment comprises a capacitive screen and a central processing unit;
the central processing unit comprises a display picture acquisition module, a capacitance feedback data acquisition module, a coordinate point acquisition module, a mask drawing module and a fusion output module;
the display picture acquisition module is used for acquiring the display picture of the terminal equipment in real time;
The capacitance feedback data acquisition module is used for acquiring capacitance feedback data of the capacitance screen in real time;
the coordinate point acquisition module is used for acquiring the coordinate point of the lowest point of the finger tip through the capacitance feedback data;
the cover drawing module is used for calculating and obtaining the vertical distance A between the lowest point of the finger tip and the capacitive screen according to the coordinate point of the lowest point of the finger tip;
presetting a shortest vertical distance threshold S1 between the lowest point of the finger tip and the capacitive screen; judging whether the vertical distance A between the lowest point of the finger tip and the capacitive screen is smaller than or equal to the shortest vertical distance threshold S1 between the lowest point of the finger tip and the capacitive screen;
if the vertical distance A between the lowest point of the finger tip and the capacitive screen is smaller than or equal to the shortest vertical distance threshold S1 between the lowest point of the finger tip and the capacitive screen, acquiring a coordinate point of capacitance feedback data of the coordinate point of the lowest point of the finger tip; calculating and obtaining a display radius by taking a coordinate point of capacitance feedback data of the lowest point coordinate point of the finger tip as a circle center according to a vertical distance A between the lowest point of the finger tip and the capacitance screen, and displaying a transparent circular covering layer on terminal equipment;
Judging whether the vertical distance A between the lowest point of the finger tip and the capacitive screen is 0 or not, if so, judging the area close to the coordinate point, and reducing the transparent circular mask layer to the area close to the coordinate point; and deepen the display color of the circular mask layer;
and the fusion output module is used for fusing and synchronizing the circular mask layer and the display picture of the terminal equipment to the AR glasses.
Preferably, the coordinate point obtaining module is specifically configured to obtain attribute information of the capacitance feedback data; acquiring an electric signal waveform chart according to the attribute information of the capacitance feedback data;
acquiring a plurality of wave peaks of the electric signal waveform diagram;
acquiring a finger crest and a palm crest according to the crest;
hiding the palm wave crest in a blurring way, and acquiring a finger tip wave crest according to the finger wave crest; acquiring a plurality of finger tip coordinate points according to the finger tip wave crest;
presetting a maximum distance threshold S2 between two adjacent finger tip coordinate points; respectively calculating the distance B between two adjacent finger tip coordinate points; grouping the finger tip coordinate points according to a maximum distance threshold S2 between the two adjacent finger tip coordinate points and a distance B between the two adjacent finger tip coordinate points, and obtaining finger tip coordinate points of a plurality of fingers;
And respectively calculating the vertical distance between the finger tip coordinate points of the fingers and the capacitive screen, and determining the finger tip coordinate point of the corresponding finger with the shortest vertical distance as the lowest point coordinate point of the finger tip.
Preferably, the coordinate point obtaining module is configured to digitize the peak during specific operation to obtain a plurality of capacitance feedback data values;
presetting a highest threshold value of the feedback data of the finger capacitance and a lowest threshold value of the feedback data of the palm capacitance; traversing the capacitance feedback data value; if the capacitance feedback data value is smaller than or equal to the highest threshold value of the finger capacitance feedback data value, determining that the capacitance feedback data value is the finger capacitance feedback data value; if the capacitance feedback data value is greater than or equal to the minimum threshold value of the palm capacitance feedback data value, determining that the capacitance feedback data value is the palm capacitance feedback data value;
determining the peak corresponding to the digital value of the finger capacitance feedback data as the finger peak; and determining the peak corresponding to the value of the feedback data of the palm capacitance as the palm peak.
Correspondingly, the invention also provides a key suspension position identification method of the AR glasses display terminal equipment, which comprises the following operation steps:
Initializing and establishing a connection relation; acquiring a display picture of the terminal equipment in real time; acquiring capacitance feedback data of the capacitive screen in real time;
acquiring the coordinate point of the lowest point of the finger tip through the capacitance feedback data;
calculating and obtaining a vertical distance A between the lowest point of the finger tip and the capacitive screen according to the coordinate point of the lowest point of the finger tip;
presetting a shortest vertical distance threshold S1 between the lowest point of the finger tip and the capacitive screen; judging whether the vertical distance A between the lowest point of the finger tip and the capacitive screen is smaller than or equal to the shortest vertical distance threshold S1 between the lowest point of the finger tip and the capacitive screen;
if the vertical distance A between the lowest point of the finger tip and the capacitive screen is smaller than or equal to the shortest vertical distance threshold S1 between the lowest point of the finger tip and the capacitive screen, acquiring a coordinate point of capacitance feedback data of the coordinate point of the lowest point of the finger tip; calculating and obtaining a display radius by taking a coordinate point of capacitance feedback data of the lowest point coordinate point of the finger tip as a circle center according to a vertical distance A between the lowest point of the finger tip and the capacitance screen, and displaying a transparent circular covering layer on terminal equipment;
Judging whether the vertical distance A between the lowest point of the finger tip and the capacitive screen is 0 or not, if so, judging the area close to the coordinate point, and reducing the transparent circular mask layer to the area close to the coordinate point; and deepen the display color of the circular mask layer;
and fusing the round cover layer with the display picture of the terminal equipment and synchronizing the round cover layer and the display picture of the terminal equipment to the AR glasses.
Preferably, the acquiring the coordinate point of the lowest point of the finger tip through the capacitance feedback data includes the following operation steps:
acquiring attribute information of the capacitance feedback data; acquiring an electric signal waveform chart according to the attribute information of the capacitance feedback data; the attribute information of the capacitance feedback data comprises an electric signal waveform diagram;
acquiring a plurality of wave peaks of the electric signal waveform diagram;
acquiring a finger crest and a palm crest according to the crest;
hiding the palm wave crest in a blurring way, and acquiring a finger tip wave crest according to the finger wave crest; acquiring a plurality of finger tip coordinate points according to the finger tip wave crest;
presetting a maximum distance threshold S2 between two adjacent finger tip coordinate points; respectively calculating the distance B between two adjacent finger tip coordinate points; grouping the finger tip coordinate points according to a maximum distance threshold S2 between the two adjacent finger tip coordinate points and a distance B between the two adjacent finger tip coordinate points, and obtaining finger tip coordinate points of a plurality of fingers;
And respectively calculating the vertical distance between the finger tip coordinate points of the fingers and the capacitive screen, and determining the finger tip coordinate point of the corresponding finger with the shortest vertical distance as the lowest point coordinate point of the finger tip.
Preferably, the step of obtaining the finger peak and the palm peak according to the peak includes the following steps:
digitizing the wave crest to obtain a plurality of capacitance feedback data values;
presetting a highest threshold value of the feedback data of the finger capacitance and a lowest threshold value of the feedback data of the palm capacitance; traversing the capacitance feedback data value; if the capacitance feedback data value is smaller than or equal to the highest threshold value of the finger capacitance feedback data value, determining that the capacitance feedback data value is the finger capacitance feedback data value; if the capacitance feedback data value is greater than or equal to the minimum threshold value of the palm capacitance feedback data value, determining that the capacitance feedback data value is the palm capacitance feedback data value;
determining the peak corresponding to the digital value of the finger capacitance feedback data as the finger peak; and determining the peak corresponding to the value of the feedback data of the palm capacitance as the palm peak.
Preferably, the step of digitizing the peak to obtain a plurality of capacitance feedback data values includes the following steps:
Frequency enhancement is carried out on the wave crest to obtain fundamental wave frequency;
denoising the fundamental wave frequency to obtain a denoised frequency;
sampling the frequency after denoising to obtain the frequency after sampling;
performing trend removal operation on the sampled frequency to obtain a plurality of frequency data;
carrying out quantization processing on the frequency data, and respectively and correspondingly acquiring a plurality of capacitance feedback data values;
preferably, the frequency enhancement of the wave crest is performed to obtain a fundamental frequency, which comprises the following operation steps:
acquiring an initial frequency K0 of the wave crest; acquiring an initial phase angle u0 of the initial frequency; obtaining the frequency deviation delta K of the initial frequency and the peak sampling period t0; calculating and acquiring a first phase angle u1 of the initial frequency K0 in a time period [0, t0] according to the frequency deviation delta K and the initial phase angle u0; calculating and obtaining a second phase angle u2 of the initial frequency K0 in a time period [ t0,2t0 ];
the first phase angle u1 and the second phase angle u2 are calculated in the following manners:
u1＝πΔKt0+u0；
u2＝3πΔKt0+u0；
wherein u1 is a first phase angle;
u2 is the second phase angle;
ΔK is the frequency deviation;
t0 is the peak sampling period;
u0 is the initial phase angle;
performing Fourier transformation on the initial frequency according to the first phase angle and the second phase angle, and calculating to obtain an estimated frequency K;
The calculation mode of the estimated frequency K is as follows:
wherein K is an estimated frequency;
calculating and obtaining the absolute value K of the difference between the initial frequency K0 and the estimated frequency K; presetting a difference absolute value threshold K' of an initial frequency K0 and an estimated frequency K; judging whether the absolute value k of the difference value is larger than the absolute value threshold k'; if yes, determining the estimated frequency as fundamental frequency; if not, taking the estimated frequency K as an initial frequency K0, performing first iteration processing operation, recording the current frequency enhancement processing iteration number as N, continuously repeating the iteration processing operation, and adding 1 to the frequency enhancement processing iteration number when each iteration processing operation is completed;
presetting a maximum iteration number threshold R; judging whether the current iteration times are smaller than the maximum threshold R of the iteration times or not; if not, the estimated frequency obtained by the last frequency enhancement processing iteration operation is determined to be the fundamental frequency.
Compared with the prior art, the embodiment of the invention has at least the following technical advantages:
according to analysis of the key suspension position identification system and method for the AR glasses display terminal equipment, when the system and method are applied specifically, firstly, connection relation is initialized and established, and display pictures of the terminal equipment and capacitance feedback data of a capacitance screen are obtained in real time (the terminal equipment obtains the capacitance feedback data, so that subsequent calculation is convenient, a transparent round cover layer is obtained, and finally the transparent round cover layer and the display pictures are fused on the AR glasses to be output);
Acquiring a coordinate point of the lowest point of the finger tip through capacitance feedback data; judging whether to display a transparent round cover layer on the terminal equipment according to the effective suspension time T1 of the finger tip of the coordinate point of the lowest point of the finger tip; finally, the round cover layer and the display picture of the terminal equipment are fused and synchronously displayed on the AR glasses, so that the display picture of the screen is prevented from being blocked by fingers, the operation of a user is facilitated, and the playability of the user is improved;
when the coordinate point of the lowest point of the finger tip is obtained, the wave peaks of the palm and the finger are distinguished according to the characteristics of the wave peaks of the electric signal waveform of the capacitance feedback data of the palm and the finger; when the palm wave crest and the finger wave crest are distinguished, the wave crest is digitized, so that the palm wave crest and the finger wave crest are distinguished more accurately and conveniently, the calculated amount is reduced, and the recognition and judgment speed is improved; in the process of numerical value, operations such as denoising, trend removal and the like are utilized to improve data visualization and enhance the accuracy of distinguishing and identifying; in a specific operation process, the wave crest is subjected to iterative processing of frequency enhancement, so that the judgment and identification accuracy of the wave crest is improved.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings that are needed in the description of the embodiments or the prior art will be briefly described, and it is obvious that the drawings in the description below are some embodiments of the present invention, and other drawings can be obtained according to the drawings without inventive effort for a person skilled in the art.
Fig. 1 is a schematic diagram of an overall architecture of a key suspension position recognition system of an AR glasses display terminal device according to a first embodiment of the present invention;
fig. 2 is a schematic diagram of the physical effect of a key suspension position recognition system of an AR glasses display terminal device according to a first embodiment of the present invention;
fig. 3 is a schematic diagram of a simulation of a system for identifying a key suspension position of an AR glasses display terminal device according to a first embodiment of the present invention, wherein the simulation is performed by acquiring a vertical distance a between a lowest point of a finger tip and the capacitive screen;
fig. 4 is a schematic operation flow diagram of a key suspension position recognition method for an AR glasses display terminal device according to a second embodiment of the present invention;
fig. 5 is a schematic diagram of an operation flow for obtaining a coordinate point of the lowest point of a finger tip in a key suspension position recognition method of an AR glasses display terminal device according to a second embodiment of the present invention;
fig. 6 is a schematic diagram of a simulation of acquiring finger tip coordinate points of a plurality of fingers in a key suspension position recognition method of an AR glasses display terminal device according to a second embodiment of the present invention;
fig. 7 is a schematic diagram of an operation flow for acquiring a finger peak and a palm peak in a key suspension position identification method of an AR glasses display terminal device according to a second embodiment of the present invention;
Fig. 8 is a schematic diagram of an operation flow for obtaining a plurality of values of capacitance feedback data in a key suspension position recognition method of an AR glasses display terminal device according to a second embodiment of the present invention;
fig. 9 is a schematic diagram of an operation flow for obtaining fundamental frequency in a key suspension position recognition method of an AR glasses display terminal device according to a second embodiment of the present invention.
Reference numerals: AR glasses 10; a terminal device 20; a capacitive screen 21; a central processing unit 22; a display screen acquisition module 221; a capacitive feedback data acquisition module 222; a coordinate point acquisition module 223; a mask drawing module 224; a fusion output module 225; a vertical distance A between the lowest point of the finger tip and the capacitive screen.
Detailed Description
The following description of the embodiments of the present invention will be made apparent and fully in view of the accompanying drawings, in which some, but not all embodiments of the invention are shown. All other embodiments, which can be made by those skilled in the art based on the embodiments of the invention without making any inventive effort, are intended to be within the scope of the invention.
The invention will now be described in further detail with reference to specific examples thereof in connection with the accompanying drawings.
Example 1
Referring to fig. 1, and referring to fig. 2 and fig. 3, fig. 1 is a schematic diagram of an overall architecture of a key suspension position recognition system for an AR glasses display terminal device according to a first embodiment of the present invention; fig. 2 is a schematic diagram of the physical effect of a key suspension position recognition system of an AR glasses display terminal device according to a first embodiment of the present invention; fig. 3 is a schematic diagram of a simulation of a system for identifying a key suspension position of an AR glasses display terminal device according to a first embodiment of the present invention, wherein the simulation is performed by acquiring a vertical distance a between a lowest point of a finger tip and the capacitive screen;
the invention provides an AR (augmented reality) glasses display terminal equipment key suspension position identification system, which comprises AR glasses 10 and terminal equipment 20;
the terminal equipment comprises a capacitive screen 21 and a central processing unit 22;
the central processing unit 22 includes a display screen obtaining module 221, a capacitance feedback data obtaining module 222, a coordinate point obtaining module 223, a mask drawing module 224, and a fusion output module 225;
wherein, the display screen obtaining module 221 is configured to obtain a display screen of the terminal device in real time;
the capacitance feedback data obtaining module 222 is configured to obtain capacitance feedback data of the capacitive screen in real time;
The coordinate point obtaining module 223 is configured to obtain a coordinate point of the lowest point of the finger tip according to the capacitance feedback data;
the mask drawing module 224 is configured to calculate and obtain a vertical distance a between the lowest point of the finger tip and the capacitive screen according to the coordinate point of the lowest point of the finger tip;
presetting a shortest vertical distance threshold S1 between the lowest point of the finger tip and the capacitive screen; judging whether the vertical distance A between the lowest point of the finger tip and the capacitive screen is smaller than or equal to the shortest vertical distance threshold S1 between the lowest point of the finger tip and the capacitive screen;
if the vertical distance A between the lowest point of the finger tip and the capacitive screen is smaller than or equal to the shortest vertical distance threshold S1 between the lowest point of the finger tip and the capacitive screen, acquiring a coordinate point of capacitance feedback data of the coordinate point of the lowest point of the finger tip; calculating and obtaining a display radius by taking a coordinate point of capacitance feedback data of the lowest point coordinate point of the finger tip as a circle center according to a vertical distance A between the lowest point of the finger tip and the capacitance screen, and displaying a transparent circular covering layer on terminal equipment;
judging whether the vertical distance A between the lowest point of the finger tip and the capacitive screen is 0 or not, if so, judging the area close to the coordinate point, and reducing the transparent circular mask layer to the area close to the coordinate point; and deepen the display color of the circular mask layer;
The fusion output module 225 is configured to fuse and synchronize the circular mask layer with the display screen of the terminal device to the AR glasses.
Preferably, the coordinate point obtaining module 223 is specifically configured to obtain attribute information of the capacitance feedback data; acquiring an electric signal waveform chart according to the attribute information of the capacitance feedback data;
acquiring a plurality of wave peaks of the electric signal waveform diagram;
acquiring a finger crest and a palm crest according to the crest;
hiding the palm wave crest in a blurring way, and acquiring a finger tip wave crest according to the finger wave crest; acquiring a plurality of finger tip coordinate points according to the finger tip wave crest;
presetting a maximum distance threshold S2 between two adjacent finger tip coordinate points; respectively calculating the distance B between two adjacent finger tip coordinate points; grouping the finger tip coordinate points according to a maximum distance threshold S2 between the two adjacent finger tip coordinate points and a distance B between the two adjacent finger tip coordinate points, and obtaining finger tip coordinate points of a plurality of fingers;
and respectively calculating the vertical distance between the finger tip coordinate points of the fingers and the capacitive screen, and determining the finger tip coordinate point of the corresponding finger with the shortest vertical distance as the lowest point coordinate point of the finger tip.
Preferably, the coordinate point obtaining module 223 is configured to digitize the peak during specific operation to obtain a plurality of capacitance feedback data values;
presetting a highest threshold value of the feedback data of the finger capacitance and a lowest threshold value of the feedback data of the palm capacitance; traversing the capacitance feedback data value; if the capacitance feedback data value is smaller than or equal to the highest threshold value of the finger capacitance feedback data value, determining that the capacitance feedback data value is the finger capacitance feedback data value; if the capacitance feedback data value is greater than or equal to the minimum threshold value of the palm capacitance feedback data value, determining that the capacitance feedback data value is the palm capacitance feedback data value;
determining the peak corresponding to the digital value of the finger capacitance feedback data as the finger peak; and determining the peak corresponding to the value of the feedback data of the palm capacitance as the palm peak.
Preferably, the coordinate point obtaining module 223 is further configured to perform frequency enhancement on the peak to obtain a fundamental frequency;
denoising the fundamental wave frequency to obtain a denoised frequency;
sampling the frequency after denoising to obtain the frequency after sampling;
performing trend removal operation on the sampled frequency to obtain a plurality of frequency data;
And carrying out quantization processing on the frequency data, and respectively and correspondingly acquiring a plurality of capacitance feedback data values.
Preferably, the coordinate point obtaining module 223 is further configured to obtain an initial frequency K0 of the peak; acquiring an initial phase angle u0 of the initial frequency; obtaining the frequency deviation delta K of the initial frequency and the peak sampling period t0; calculating and acquiring a first phase angle u1 of the initial frequency K0 in a time period [0, t0] according to the frequency deviation delta K and the initial phase angle u0; calculating and obtaining a second phase angle u2 of the initial frequency K0 in a time period [ t0,2t0 ];
the first phase angle u1 and the second phase angle u2 are calculated in the following manners:
u1＝πΔKt0+u0；
u2＝3πΔKt0+u0；
wherein u1 is a first phase angle;
u2 is the second phase angle;
ΔK is the frequency deviation;
t0 is the peak sampling period;
u0 is the initial phase angle;
performing Fourier transformation on the initial frequency according to the first phase angle and the second phase angle, and calculating to obtain an estimated frequency K;
the calculation mode of the estimated frequency K is as follows:
wherein K is an estimated frequency;
calculating and obtaining the absolute value K of the difference between the initial frequency K0 and the estimated frequency K; presetting a difference absolute value threshold K' of an initial frequency K0 and an estimated frequency K; judging whether the absolute value k of the difference value is larger than the absolute value threshold k'; if yes, determining the estimated frequency as fundamental frequency; if not, taking the estimated frequency K as an initial frequency K0, performing first iteration processing operation, recording the current frequency enhancement processing iteration number as N, continuously repeating the iteration processing operation, and adding 1 to the frequency enhancement processing iteration number when each iteration processing operation is completed;
Presetting a maximum iteration number threshold R; judging whether the current iteration times are smaller than the maximum threshold R of the iteration times or not; if not, the estimated frequency obtained by the last frequency enhancement processing iteration operation is determined to be the fundamental frequency.
In summary, in the key suspension position identification system for the AR glasses display terminal equipment provided by the invention, during specific operation, the central processing unit is used for acquiring the display picture of the terminal equipment and the capacitance feedback data of the capacitance screen, then the central processing unit is used for analyzing and processing the capacitance feedback plasticizer to obtain the coordinate point of the lowest point of the finger tip, a transparent circular covering layer is displayed in the terminal equipment according to the effective suspension time of the finger tip of the coordinate point of the lowest point of the finger tip, then the circular covering layer and the display picture are fused, and the transparent circular covering layer and the display picture are synchronously output on the AR glasses; when the coordinate point of the lowest point of the finger tip is obtained specifically, the wave peaks of the palm and the finger are distinguished according to the characteristics of the wave peaks of the electric signal waveform of the capacitance feedback data of the palm and the finger; when the palm wave crest and the finger wave crest are distinguished, the wave crest is digitized, so that the palm wave crest and the finger wave crest are distinguished more accurately and conveniently, the calculated amount is reduced, and the recognition and judgment speed is improved; in the process of numerical value, operations such as denoising, trend removal and the like are utilized to improve data visualization and enhance the accuracy of distinguishing and identifying; in a specific operation process, the wave crest is subjected to iterative processing of frequency enhancement, so that the judgment and identification accuracy of the wave crest is improved.
Example two
Correspondingly, as shown in fig. 4, the invention also provides a method for identifying the key suspension position of the AR glasses display terminal device, which comprises the following operation steps:
step S10: initializing and establishing a connection relation; acquiring a display picture of the terminal equipment in real time; acquiring capacitance feedback data of the capacitive screen in real time;
it should be noted that, the above-mentioned initialization establishing connection relationship means that the AR glasses establish connection with the terminal device;
a central processing unit in the terminal equipment acquires a display picture of the terminal equipment, and simultaneously acquires capacitance feedback data of a capacitance screen carried on the terminal equipment, analyzes and processes the capacitance feedback data, and obtains a transparent circular mask layer in subsequent operation; and finally, fusing the transparent round cover layer with a display picture of the terminal equipment and then synchronously outputting the fused round cover layer and the display picture to the AR glasses.
In the method for identifying the key suspension position of the AR glasses display terminal equipment, which is adopted by the embodiment of the invention, a mutual capacitance type capacitive screen is selected, when a human finger approaches the mutual capacitance type capacitive screen, the local capacitance of the mutual capacitance type capacitive screen is reduced, and the coordinate of each touch point can be calculated according to the two-dimensional capacitance variation data of the mutual capacitance type capacitive screen;
Step S20: acquiring the coordinate point of the lowest point of the finger tip through the capacitance feedback data;
it should be noted that, the capacitive feedback data refers to feedback data that when a finger approaches an electric field, the distribution of the electric field is changed due to a certain conductivity of a human body, so as to cause a change of an induced current;
when the capacitive screen is not touched by a finger, the position of the finger can be obtained through electric field induction; in particular, capacitive screens have a series of electrodes arranged on the surface of the screen, between which an electric field is formed; when a finger approaches to the electric field, the distribution of the electric field can be changed due to certain conductivity of the human body, so that the change of induced current is caused; the capacitive screen measures the changes in these induced currents to calculate the finger position;
usually, the capacitive screen adopts an alternating voltage method or a direct current scanning method to detect the position; in the alternating voltage method, a section of alternating voltage is applied between two adjacent electrodes by the capacitive screen, and the magnitude and phase difference of current are measured, so that the position of a finger is calculated; in the direct current scanning method, one electrode is kept at a low level by a capacitive screen, other electrodes are scanned one by one, capacitance change is measured, and finally the finger position is calculated;
In general, the capacitive screen utilizes the electric field induction principle to detect the finger position, and has higher precision and response speed.
Step S30: calculating and obtaining a vertical distance A between the lowest point of the finger tip and the capacitive screen according to the coordinate point of the lowest point of the finger tip;
presetting a shortest vertical distance threshold S1 between the lowest point of the finger tip and the capacitive screen; judging whether the vertical distance A between the lowest point of the finger tip and the capacitive screen is smaller than or equal to the shortest vertical distance threshold S1 between the lowest point of the finger tip and the capacitive screen;
if it is determined that the vertical distance a (real-time vertical distance) between the lowest point of the finger tip and the capacitive screen is less than or equal to the shortest vertical distance threshold S1 between the lowest point of the finger tip and the capacitive screen, acquiring a coordinate point of capacitance feedback data of the coordinate point of the lowest point of the finger tip (the "coordinate point of capacitance feedback data of the coordinate point of the lowest point of the finger tip" refers to a coordinate point of Fang Dianrong screen under the vertical of the lowest point of the finger tip); calculating and obtaining a display radius by taking a coordinate point of capacitance feedback data of the lowest point coordinate point of the finger tip as a circle center according to a vertical distance A between the lowest point of the finger tip and the capacitance screen, and displaying a transparent circular covering layer on terminal equipment;
To explain, in general, the larger the vertical distance from the lowest point of the finger tip to the capacitive screen, the larger the display radius, and the larger the transparent circular mask.
Step S40: judging whether the vertical distance A between the lowest point of the finger tip and the capacitive screen is 0 or not, if so, judging the area close to the coordinate point (the area close to the coordinate point refers to the area formed by all the finger tip coordinate points in the capacitive feedback data of the terminal equipment touched by the finger tip), and reducing the transparent circular mask layer to the area close to the coordinate point; and deepen the display color of the circular mask layer;
step S50: and fusing the round cover layer with the display picture of the terminal equipment and synchronizing the round cover layer and the display picture of the terminal equipment to the AR glasses.
According to the technical scheme adopted by the embodiment of the invention, the display picture and the circular cover layer are fused and output to the AR glasses, so that the display picture of the screen is prevented from being blocked by fingers, the operation of a user is facilitated, and the playability of the user is improved.
In the implementation process of the key suspension position identification method of the AR glasses display terminal equipment provided by the embodiment of the invention, a technician finds that when a palm and a finger are close to a capacitive screen together, coordinate points of the palm and the finger are fed back together by capacitance feedback data, so that the user is influenced by finger suspension position identification, and the influence of the coordinate points of the palm and the finger on a scene where the palm and the finger appear above the capacitive screen together is required to be screened out, so that the coordinate point of the lowest point of the finger tip of the finger is accurately acquired;
Specifically, as shown in fig. 5, in step S20, the coordinate point of the lowest point of the finger tip is obtained by the capacitance feedback data, which includes the following steps:
step S21: acquiring attribute information of the capacitance feedback data; acquiring an electric signal waveform chart according to the attribute information of the capacitance feedback data;
step S22: acquiring a plurality of wave peaks of the electric signal waveform diagram;
step S23: acquiring a finger crest and a palm crest according to the crest;
in the capacitive feedback data, the capacitive feedback intensity and characteristics of the capacitive screen on objects with different thicknesses and areas are different, specifically, the larger the area of the object is, the larger the fluctuation of the capacitive feedback data is, and the larger the thickness of the object is, the stronger the capacitive feedback data is; illustrating: waveform: when a finger approaches the capacitive screen, the sensed capacitive feedback data is an intermittent waveform with a period of about 5ms due to the thin profile; when the palm is close to the capacitive screen, the sensed capacitive feedback data presents a continuous stable waveform;
frequency: when the finger is close to the capacitive screen, the frequency of the sensed capacitive feedback data is high, and when the palm is close to the capacitive screen, the frequency of the sensed capacitive feedback data is lower, so that the capacitive feedback waveform of the palm is more stable, and the fluctuation frequency of the capacitive feedback waveform of the finger is higher;
Amplitude of: when the finger is close to the capacitive screen, the strength of the sensed capacitive feedback data is smaller, and when the palm is close to the capacitive screen, the strength of the sensed capacitive feedback data is larger;
intensity aspect:
thickness: the finger thickness is relatively small, and the capacitance feedback data intensity is also relatively small; conversely, the palm is thicker, and the intensity of capacitance feedback data is also larger;
contact area: the contact area of the finger is small, and the intensity of the capacitance feedback data is relatively small; the contact area of the palm is larger, and the intensity of capacitance feedback data is also relatively larger;
taking thickness as an example, the palm is relatively thick, and the influence of charge distribution close to a capacitive screen on current is stronger when the palm is close to the capacitive screen, so that the sensed capacitive feedback data intensity is higher; in contrast, the finger is relatively thin, the contact area is small when the finger approaches, the influence of charge distribution on current is small, and therefore the sensed capacitance feedback data intensity is relatively small;
in conclusion, the waveform intensity of the capacitance feedback data of the finger is smaller (the wave crest is low) and the fluctuation is larger; the waveform intensity of the capacitance feedback data of the palm is larger (peak height) and the fluctuation is smaller (fluctuation is stable); the embodiment of the application can collect wave crests from all capacitance feedback data waveforms more efficiently by utilizing the technical characteristics, and analyze and identify finger wave crests and palm wave crests from all wave crests.
Step S24: hiding the palm wave crest in a blurring way, and acquiring a finger tip wave crest according to the finger wave crest; acquiring a plurality of finger tip coordinate points according to the finger tip wave crest;
in the finger tip portion, the area and the thickness are changed more than the whole finger, that is, the area is smaller and the thickness is lower, and the frequency of fluctuation of the finger tip peak is higher and the peak is lower, so that the finger tip peak can be distinguished from the finger peak.
Step S25: presetting a maximum distance threshold S2 between two adjacent finger tip coordinate points; respectively calculating the distance B between two adjacent finger tip coordinate points; grouping the finger tip coordinate points according to a maximum distance threshold S2 between the two adjacent finger tip coordinate points and a distance B between the two adjacent finger tip coordinate points, and obtaining finger tip coordinate points of a plurality of fingers;
it should be noted that, for the same finger, the finger tip coordinate points obtained by the capacitance feedback data are relatively uniform, and the distance between two adjacent finger tip coordinate points is smaller, while for two fingers, the distance between two adjacent finger tip coordinate points is larger at two critical positions of the two fingers; grouping the finger tip coordinate points according to the distance between two adjacent finger tip coordinate points (namely, determining which finger tip coordinate points belong to the same finger) so as to obtain finger tip coordinate points of a plurality of fingers;
The distance B between two adjacent finger tip coordinate points refers to a 3D distance and refers to two coordinate points which are adjacent to each other in space; the 3D distance B between two adjacent finger tip coordinate points can be calculated by the euclidean distance formula. Assuming that the coordinate points of two adjacent finger tips are P1 (x 1, y1, z 1) and P2 (x 2, y2, z 2), respectively, the distance B therebetween can be expressed by the following formula:
the formula is that the linear distance between two points is obtained by using the Pythagorean theorem, namely the path length which needs to be walked when one point starts to reach the other point; the calculation manner of the distance B between the two adjacent finger tip coordinate points is in the prior art, and is not a main invention point of the present invention, and the embodiments of the present invention are not described in detail.
Illustrating: as shown in fig. 6, there are currently ten finger tip coordinate points, denoted as a1, a2, a3, a4, a5, a6, a7, a8, a9, a10, respectively;
and, a1 is adjacent to a2, and the distance B1 is 2; a2 is adjacent to a3 and the distance B2 is 3; a3 is adjacent to a4, and B3 is a distance of 4; a4 is adjacent to a5 and the distance B4 is 2; a5 is adjacent to a6 and the distance B5 is 3; a6 is adjacent to a7 and the distance B6 is 4; a7 is adjacent to a8 and the distance B7 is 3; a8 is adjacent to a9 and the distance B8 is 3; a9 is adjacent to a10, and the distance B9 is 3;
Presetting a maximum distance threshold S2 between two adjacent finger tip coordinate points to be 3; because B1, B2, B4, B5, B7, B8, B9 are all less than or equal to S2, while further filtering B3, B6 is greater than S2; then it may be determined that the two adjacent finger tip coordinate points a3 and a4 are not the same coordinate point set; it may be determined that the two adjacent finger tip coordinate points a6 and a7 are not the same coordinate point group; finally, a1, a2 and a3 are divided into a group, namely a coordinate point group on the first finger; dividing a4, a5 and a6 into a group, namely a coordinate point group on a second finger; dividing a7, a8, a9 and a10 into a group, namely a coordinate point group on a third finger;
to sum up, a1, a2, a3, a4, a5, a6, a7, a8, a9, a10; the three groups are divided, and can be described as coordinate point groups of three fingers.
Step S26: and respectively calculating the vertical distance between the finger tip coordinate points of the fingers and the capacitive screen, and determining the finger tip coordinate point of the corresponding finger with the shortest vertical distance as the lowest point coordinate point of the finger tip.
It should be noted that, in the embodiment of the present invention, the peak in the electric signal waveform chart is divided into a palm peak and a finger peak, and then the palm peak is hidden (or screened out); then obtaining a plurality of finger tip coordinate points according to the finger wave crest; because a plurality of fingers can click the screen simultaneously, the finger tip coordinate points are grouped, the finger tip coordinate points with the distance between two adjacent finger tip coordinate points smaller than or equal to the preset distance threshold value of the two adjacent finger tip coordinate points are grouped into a group, and further the finger tip coordinate point with the shortest vertical distance from the capacitive screen in each group is obtained as the finger tip lowest point coordinate point.
According to the technical scheme adopted by the embodiment of the invention, after the palm is close to the distance limit value of the capacitive screen, capacitance feedback data can be obtained due to the conductivity of a human body, then a plurality of capacitance feedback wave peaks are obtained from the capacitance feedback data, the capacitance feedback wave peaks of the fingers are low and have large fluctuation according to the area and thickness of the fingers and the palm, and the capacitance feedback wave peaks of the palm are high and have stable fluctuation, so that the finger wave peaks and the palm wave peaks can be further distinguished; furthermore, the position area and the thickness of the finger tip are changed more greatly, so that the finger tip peak can be further distinguished; according to the feedback peak of the capacitance of the finger tip, a plurality of coordinate points of the finger tip can be obtained through calculation of the terminal equipment, and then the vertical distance between each coordinate point and the terminal equipment is calculated, so that the coordinate point of the lowest point of the finger tip can be obtained.
Specifically, as shown in fig. 7, in step S23, a finger peak and a palm peak are obtained according to the peaks, including the following steps:
step S231: digitizing the wave crest to obtain a plurality of capacitance feedback data values;
step S232: presetting a highest threshold value of the feedback data of the finger capacitance and a lowest threshold value of the feedback data of the palm capacitance; traversing the capacitance feedback data value; if the capacitance feedback data value is smaller than or equal to the highest threshold value of the finger capacitance feedback data value, determining that the capacitance feedback data value is the finger capacitance feedback data value; if the capacitance feedback data value is greater than or equal to the minimum threshold value of the palm capacitance feedback data value, determining that the capacitance feedback data value is the palm capacitance feedback data value;
Step S233: determining the peak corresponding to the digital value of the finger capacitance feedback data as the finger peak; and determining the peak corresponding to the value of the feedback data of the palm capacitance as the palm peak.
The technical scheme adopted by the embodiment of the invention is to digitize the wave crest, so that the visualization of the electric signal frequency of the capacitance feedback data can be improved, the data is distinguished, the calculated amount is reduced, and the speed of identifying and judging the finger wave crest and the palm wave crest is improved.
Specifically, as shown in fig. 8, in step S231, the peak is digitized to obtain a plurality of capacitance feedback data values, including the following steps:
step S2311: frequency enhancement is carried out on the wave crest to obtain fundamental wave frequency;
by the explanation, the wave crest is subjected to frequency enhancement, the obtained fundamental wave frequency can be subjected to frequency enhancement on the tiny electric signal waveform wave crest, and the accuracy of wave crest identification judgment is improved;
step S2312: denoising the fundamental wave frequency to obtain a denoised frequency;
by explanation, the embodiment of the invention removes noise from fundamental wave frequency, eliminates the influence of noise in the waveform of the electric signal, and improves the accuracy of wave crest identification judgment;
Step S2313: sampling the frequency after denoising to obtain the frequency after sampling;
explanation: the embodiment of the invention discretizes the continuous electric signal wave crest into a group of limited data points so as to facilitate the subsequent processing;
step S2314: performing trend removal operation on the sampled frequency to obtain a plurality of frequency data;
it should be noted that, since the peak of the waveform of the electrical signal generally contains some slowly varying trend components, filtering or other methods need to be used to remove these trend components, so as to better analyze and process the transient variation portion of the signal; specifically, some smoothing filtering algorithms, high-pass filtering or differential operation methods can be adopted to realize trending;
step S2315: carrying out quantization processing on the frequency data, and respectively and correspondingly acquiring a plurality of capacitance feedback data values;
in the embodiment of the invention, the frequency enhancement, noise removal, sampling, trend removal and other numerical processing operations are performed on the wave peaks to obtain the capacitance feedback data values, so that the wave peaks of the electric signal waveforms can be visualized, and the wave peak identification judgment processing operation is convenient.
Specifically, as shown in fig. 9, in step S2311, the frequency of the peak is enhanced to obtain a fundamental frequency, which includes the following steps:
Step S23111: acquiring an initial frequency K0 of the wave crest; acquiring an initial phase angle u0 of the initial frequency; obtaining the frequency deviation delta K of the initial frequency and the peak sampling period t0; calculating and acquiring a first phase angle u1 of the initial frequency K0 in a time period [0, t0] according to the frequency deviation delta K and the initial phase angle u0; calculating and obtaining a second phase angle u2 of the initial frequency K0 in a time period [ t0,2t0 ];
the first phase angle u1 and the second phase angle u2 are calculated in the following manners:
u1＝πΔKt0+u0；
u2＝3πΔKt0+u0；
wherein u1 is a first phase angle;
u2 is the second phase angle;
ΔK is the frequency deviation;
t0 is the peak sampling period;
u0 is the initial phase angle;
step S23112: performing Fourier transformation on the initial frequency according to the first phase angle and the second phase angle, and calculating to obtain an estimated frequency K;
the calculation mode of the estimated frequency K is as follows:
wherein K is an estimated frequency;
step S23113: calculating and obtaining the absolute value K of the difference between the initial frequency K0 and the estimated frequency K; presetting a difference absolute value threshold K' of an initial frequency K0 and an estimated frequency K; judging whether the absolute value k of the difference value is larger than the absolute value threshold k'; if yes, determining the estimated frequency as fundamental frequency; if not, taking the estimated frequency K as an initial frequency K0, performing first iteration processing operation, recording the current frequency enhancement processing iteration number as N, continuously repeating the iteration processing operation, and adding 1 to the frequency enhancement processing iteration number when each iteration processing operation is completed;
Step S23114: presetting a maximum iteration number threshold R; judging whether the current iteration times are smaller than the maximum threshold R of the iteration times or not; if not, the estimated frequency obtained by the last frequency enhancement processing iteration operation is determined to be the fundamental frequency.
The maximum threshold value R of the number of iterations is a constant, and is usually 5; because the frequency enhancement precision is higher along with the increase of the iteration times, but under a large number of experiments, the improvement of the precision is very limited after the iteration times exceed three times, so that the maximum threshold value of the winning iteration times in actual operation is set to be 5 times, the frequency enhancement precision is improved, and the processing speed is ensured.
According to the technical scheme adopted by the embodiment of the invention, the frequency enhancement processing is carried out on the wave crest of the tiny electric signal waveform, so that the accuracy of judging and identifying the wave crest can be improved.
In summary, according to the key suspension position recognition system and method for the AR glasses display terminal equipment provided by the embodiment of the invention, firstly, a connection relation is established through initialization, then, a central processing unit on the terminal equipment is utilized to acquire capacitance feedback data of a display picture and a capacitance screen of the terminal equipment, and a coordinate point of the lowest point of a finger tip is acquired through the capacitance feedback data; then, according to the vertical distance between the coordinate point of the lowest point of the finger tip and the capacitive screen and the effective suspension time of the finger tip, a transparent circular covering layer is displayed on the terminal equipment, then the circular covering layer and the display picture are fused, and synchronously output on the AR glasses, so that the display picture of the screen is prevented from being blocked by the finger, the operation of a user is facilitated, and the playability of the user is improved;
When the coordinate point of the lowest point of the finger tip is obtained, the palm wave crest and the finger wave crest are distinguished according to the characteristic of capacitance feedback data of the close palm and the finger by the capacitance screen, then the palm wave crest is hidden and screened out, and a plurality of finger tip coordinate points in the finger wave crest are extracted; the finger coordinate points are further grouped into a plurality of fingers according to the distance between two adjacent finger coordinate points, so that a circular covering layer corresponding to the plurality of fingers can be displayed in the terminal equipment during multi-finger operation, and further the display of the multi-finger operation can be realized;
further, when the palm wave crest and the finger wave crest are distinguished, the electric signal fluctuation wave crest of the capacitance feedback data is digitized, so that the visualization of the electric signal frequency of the capacitance feedback data can be improved, the difference is carried out in a data mode, the calculated amount is reduced, and the speed of identifying and judging the finger wave crest and the palm wave crest is improved;
further, in the process of digitizing, frequency enhancement, noise removal, sampling, trend removal and other digitizing operations are performed on the wave peaks to obtain capacitance feedback data values, so that the wave peaks of the electric signal waveforms can be visualized, and the wave peak identification and judgment processing operations are facilitated;
Further, in the frequency enhancement process, fourier transformation and iterative frequency enhancement operation are performed on the peak frequency, and the accuracy of judging and identifying the peak can be improved by performing frequency enhancement processing on the peak of the tiny electric signal waveform.
Finally, it should be noted that: the above embodiments are only for illustrating the technical solution of the present invention, and not for limiting the same; modifications of the technical solutions described in the foregoing embodiments, or equivalent substitutions of some or all of the technical features thereof, may be made by those of ordinary skill in the art; such modifications and substitutions do not depart from the spirit of the invention.

Claims (10)

1. The key suspension position identification system for the AR glasses display terminal equipment is characterized by comprising AR glasses and terminal equipment;
the terminal equipment comprises a capacitive screen and a central processing unit;
the central processing unit comprises a display picture acquisition module, a capacitance feedback data acquisition module, a coordinate point acquisition module, a mask drawing module and a fusion output module;
the display picture acquisition module is used for acquiring the display picture of the terminal equipment in real time;
The capacitance feedback data acquisition module is used for acquiring capacitance feedback data of the capacitance screen in real time;
the coordinate point acquisition module is used for acquiring the coordinate point of the lowest point of the finger tip through the capacitance feedback data;
the cover drawing module is used for calculating and obtaining the vertical distance A between the lowest point of the finger tip and the capacitive screen according to the coordinate point of the lowest point of the finger tip;
presetting a shortest vertical distance threshold S1 between the lowest point of the finger tip and the capacitive screen; judging whether the vertical distance A between the lowest point of the finger tip and the capacitive screen is smaller than or equal to the shortest vertical distance threshold S1 between the lowest point of the finger tip and the capacitive screen;
if the vertical distance A between the lowest point of the finger tip and the capacitive screen is smaller than or equal to the shortest vertical distance threshold S1 between the lowest point of the finger tip and the capacitive screen, acquiring a coordinate point of capacitance feedback data of the coordinate point of the lowest point of the finger tip; calculating and obtaining a display radius by taking a coordinate point of capacitance feedback data of the lowest point coordinate point of the finger tip as a circle center according to a vertical distance A between the lowest point of the finger tip and the capacitance screen, and displaying a transparent circular covering layer on terminal equipment;
Judging whether the vertical distance A between the lowest point of the finger tip and the capacitive screen is 0 or not, if so, judging the area close to the coordinate point, and reducing the transparent circular mask layer to the area close to the coordinate point; and deepen the display color of the circular mask layer;
and the fusion output module is used for fusing and synchronizing the circular mask layer and the display picture of the terminal equipment to the AR glasses.
2. The AR glasses display terminal device key suspension position identification system according to claim 1, wherein the coordinate point obtaining module is specifically configured to obtain attribute information of the capacitance feedback data; acquiring an electric signal waveform chart according to the attribute information of the capacitance feedback data;
acquiring a plurality of wave peaks of the electric signal waveform diagram;
acquiring a finger crest and a palm crest according to the crest;
hiding the palm wave crest in a blurring way, and acquiring a finger tip wave crest according to the finger wave crest; acquiring a plurality of finger tip coordinate points according to the finger tip wave crest;
presetting a maximum distance threshold S2 between two adjacent finger tip coordinate points; respectively calculating the distance B between two adjacent finger tip coordinate points; grouping the finger tip coordinate points according to a maximum distance threshold S2 between the two adjacent finger tip coordinate points and a distance B between the two adjacent finger tip coordinate points, and obtaining finger tip coordinate points of a plurality of fingers;
And respectively calculating the vertical distance between the finger tip coordinate points of the fingers and the capacitive screen, and determining the finger tip coordinate point of the corresponding finger with the shortest vertical distance as the lowest point coordinate point of the finger tip.
3. The system for identifying the key suspension position of the AR glasses display terminal device according to claim 2, wherein the coordinate point obtaining module is configured to digitize the peak during specific operation to obtain a plurality of capacitance feedback data values;
presetting a highest threshold value of the feedback data of the finger capacitance and a lowest threshold value of the feedback data of the palm capacitance; traversing the capacitance feedback data value; if the capacitance feedback data value is smaller than or equal to the highest threshold value of the finger capacitance feedback data value, determining that the capacitance feedback data value is the finger capacitance feedback data value; if the capacitance feedback data value is greater than or equal to the minimum threshold value of the palm capacitance feedback data value, determining that the capacitance feedback data value is the palm capacitance feedback data value;
determining the peak corresponding to the digital value of the finger capacitance feedback data as the finger peak; and determining the peak corresponding to the value of the feedback data of the palm capacitance as the palm peak.
4. The AR glasses display terminal device key suspension position identification system according to claim 3, wherein the coordinate point obtaining module is further configured to frequency enhance the peak to obtain a fundamental frequency;
denoising the fundamental wave frequency to obtain a denoised frequency;
sampling the frequency after denoising to obtain the frequency after sampling;
performing trend removal operation on the sampled frequency to obtain a plurality of frequency data;
and carrying out quantization processing on the frequency data, and respectively and correspondingly acquiring a plurality of capacitance feedback data values.
5. The system for identifying a key suspension position of an AR glasses display terminal device according to claim 4, wherein the coordinate point obtaining module is further configured to obtain an initial frequency K0 of the peak; acquiring an initial phase angle u0 of the initial frequency; obtaining the frequency deviation delta K of the initial frequency and the peak sampling period t0; calculating and acquiring a first phase angle u1 of the initial frequency K0 in a time period [0, t0] according to the frequency deviation delta K and the initial phase angle u0; calculating and obtaining a second phase angle u2 of the initial frequency K0 in a time period [ t0,2t0 ];
performing Fourier transformation on the initial frequency according to the first phase angle and the second phase angle, and calculating to obtain an estimated frequency K;
Calculating and obtaining the absolute value K of the difference between the initial frequency K0 and the estimated frequency K; presetting a difference absolute value threshold K' of an initial frequency K0 and an estimated frequency K; judging whether the absolute value k of the difference value is larger than the absolute value threshold k'; if yes, determining the estimated frequency as fundamental frequency; if not, taking the estimated frequency K as an initial frequency K0, performing first iteration processing operation, recording the current frequency enhancement processing iteration number as N, continuously repeating the iteration processing operation, and adding 1 to the frequency enhancement processing iteration number when each iteration processing operation is completed;
presetting a maximum iteration number threshold R; judging whether the current iteration times are smaller than the maximum threshold R of the iteration times or not; if not, the estimated frequency obtained by the last frequency enhancement processing iteration operation is determined to be the fundamental frequency.
6. The method for identifying the key suspension position of the AR glasses display terminal equipment is characterized by comprising the following operation steps:
initializing and establishing a connection relation; acquiring a display picture of the terminal equipment in real time; acquiring capacitance feedback data of the capacitive screen in real time;
acquiring the coordinate point of the lowest point of the finger tip through the capacitance feedback data;
Calculating and obtaining a vertical distance A between the lowest point of the finger tip and the capacitive screen according to the coordinate point of the lowest point of the finger tip;
presetting a shortest vertical distance threshold S1 between the lowest point of the finger tip and the capacitive screen; judging whether the vertical distance A between the lowest point of the finger tip and the capacitive screen is smaller than or equal to the shortest vertical distance threshold S1 between the lowest point of the finger tip and the capacitive screen;
if the vertical distance A between the lowest point of the finger tip and the capacitive screen is smaller than or equal to the shortest vertical distance threshold S1 between the lowest point of the finger tip and the capacitive screen, acquiring a coordinate point of capacitance feedback data of the coordinate point of the lowest point of the finger tip; calculating and obtaining a display radius by taking a coordinate point of capacitance feedback data of the lowest point coordinate point of the finger tip as a circle center according to a vertical distance A between the lowest point of the finger tip and the capacitance screen, and displaying a transparent circular covering layer on terminal equipment;
judging whether the vertical distance A between the lowest point of the finger tip and the capacitive screen is 0 or not, if so, judging the area close to the coordinate point, and reducing the transparent circular mask layer to the area close to the coordinate point; and deepen the display color of the circular mask layer;
And fusing the round cover layer with the display picture of the terminal equipment and synchronizing the round cover layer and the display picture of the terminal equipment to the AR glasses.
7. The method for identifying the key suspension position of the AR glasses display terminal device according to claim 6, wherein the step of obtaining the coordinate point of the lowest point of the finger tip through the capacitance feedback data comprises the following operation steps:
acquiring attribute information of the capacitance feedback data; acquiring an electric signal waveform chart according to the attribute information of the capacitance feedback data; the attribute information of the capacitance feedback data comprises an electric signal waveform diagram;
acquiring a plurality of wave peaks of the electric signal waveform diagram;
acquiring a finger crest and a palm crest according to the crest;
hiding the palm wave crest in a blurring way, and acquiring a finger tip wave crest according to the finger wave crest; acquiring a plurality of finger tip coordinate points according to the finger tip wave crest;
presetting a maximum distance threshold S2 between two adjacent finger tip coordinate points; respectively calculating the distance B between two adjacent finger tip coordinate points; grouping the finger tip coordinate points according to a maximum distance threshold S2 between the two adjacent finger tip coordinate points and a distance B between the two adjacent finger tip coordinate points, and obtaining finger tip coordinate points of a plurality of fingers;
And respectively calculating the vertical distance between the finger tip coordinate points of the fingers and the capacitive screen, and determining the finger tip coordinate point of the corresponding finger with the shortest vertical distance as the lowest point coordinate point of the finger tip.
8. The method for identifying the key suspension position of the AR glasses display terminal device according to claim 7, wherein the step of obtaining the finger peak and the palm peak according to the peak comprises the following steps:
digitizing the wave crest to obtain a plurality of capacitance feedback data values;
presetting a highest threshold value of the feedback data of the finger capacitance and a lowest threshold value of the feedback data of the palm capacitance; traversing the capacitance feedback data value; if the capacitance feedback data value is smaller than or equal to the highest threshold value of the finger capacitance feedback data value, determining that the capacitance feedback data value is the finger capacitance feedback data value; if the capacitance feedback data value is greater than or equal to the minimum threshold value of the palm capacitance feedback data value, determining that the capacitance feedback data value is the palm capacitance feedback data value;
determining the peak corresponding to the digital value of the finger capacitance feedback data as the finger peak; and determining the peak corresponding to the value of the feedback data of the palm capacitance as the palm peak.
9. The method for identifying the key suspension position of the AR glasses display terminal device according to claim 8, wherein the step of digitizing the peak to obtain a plurality of capacitance feedback data values includes the following steps:
frequency enhancement is carried out on the wave crest to obtain fundamental wave frequency;
denoising the fundamental wave frequency to obtain a denoised frequency;
sampling the frequency after denoising to obtain the frequency after sampling;
performing trend removal operation on the sampled frequency to obtain a plurality of frequency data;
and carrying out quantization processing on the frequency data, and respectively and correspondingly acquiring a plurality of capacitance feedback data values.
10. The method for identifying the key suspension position of the AR glasses display terminal device according to claim 9, wherein the frequency enhancement of the wave peak is performed to obtain a fundamental frequency, and the method comprises the following operation steps:
acquiring an initial frequency K0 of the wave crest; acquiring an initial phase angle u0 of the initial frequency; obtaining the frequency deviation delta K of the initial frequency and the peak sampling period t0; calculating and acquiring a first phase angle u1 of the initial frequency K0 in a time period [0, t0] according to the frequency deviation delta K and the initial phase angle u0; calculating and obtaining a second phase angle u2 of the initial frequency K0 in a time period [ t0,2t0 ];
The first phase angle u1 and the second phase angle u2 are calculated in the following manners:
u1＝πΔKt0+u0；
u2＝3πΔKt0+u0；
wherein u1 is a first phase angle;
u2 is the second phase angle;
ΔK is the frequency deviation;
t0 is the peak sampling period;
u0 is the initial phase angle;
performing Fourier transformation on the initial frequency according to the first phase angle and the second phase angle, and calculating to obtain an estimated frequency K;
the calculation mode of the estimated frequency K is as follows:
wherein K is an estimated frequency;
calculating and obtaining the absolute value K of the difference between the initial frequency K0 and the estimated frequency K; presetting a difference absolute value threshold K' of an initial frequency K0 and an estimated frequency K; judging whether the absolute value k of the difference value is larger than the absolute value threshold k'; if yes, determining the estimated frequency as fundamental frequency; if not, taking the estimated frequency K as an initial frequency K0, performing first iteration processing operation, recording the current frequency enhancement processing iteration number as N, continuously repeating the iteration processing operation, and adding 1 to the frequency enhancement processing iteration number when each iteration processing operation is completed;
presetting a maximum iteration number threshold R; judging whether the current iteration times are smaller than the maximum threshold R of the iteration times or not; if not, the estimated frequency obtained by the last frequency enhancement processing iteration operation is determined to be the fundamental frequency.
CN202310968918.4A 2023-08-03 2023-08-03 AR (augmented reality) glasses display terminal equipment key suspension position identification system and method Pending CN116931735A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202310968918.4A CN116931735A (en) 2023-08-03 2023-08-03 AR (augmented reality) glasses display terminal equipment key suspension position identification system and method

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202310968918.4A CN116931735A (en) 2023-08-03 2023-08-03 AR (augmented reality) glasses display terminal equipment key suspension position identification system and method

Publications (1)

Publication Number Publication Date
CN116931735A true CN116931735A (en) 2023-10-24

Family

ID=88392345

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202310968918.4A Pending CN116931735A (en) 2023-08-03 2023-08-03 AR (augmented reality) glasses display terminal equipment key suspension position identification system and method

Country Status (1)

Country Link
CN (1) CN116931735A (en)

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN105808086A (en) * 2014-12-30 2016-07-27 Tcl集团股份有限公司 Virtual keyboard input method and device
CN108415654A (en) * 2017-02-10 2018-08-17 上海真曦通信技术有限公司 Virtual input system and correlation technique
CN113448485A (en) * 2021-07-12 2021-09-28 交互未来(北京)科技有限公司 Large-screen window control method and device, storage medium and equipment
JP2022097901A (en) * 2020-12-21 2022-07-01 マクセル株式会社 Space floating video display device
CN115529405A (en) * 2022-09-22 2022-12-27 维沃移动通信有限公司 Image display method and device of front camera

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN105808086A (en) * 2014-12-30 2016-07-27 Tcl集团股份有限公司 Virtual keyboard input method and device
CN108415654A (en) * 2017-02-10 2018-08-17 上海真曦通信技术有限公司 Virtual input system and correlation technique
JP2022097901A (en) * 2020-12-21 2022-07-01 マクセル株式会社 Space floating video display device
CN113448485A (en) * 2021-07-12 2021-09-28 交互未来(北京)科技有限公司 Large-screen window control method and device, storage medium and equipment
CN115529405A (en) * 2022-09-22 2022-12-27 维沃移动通信有限公司 Image display method and device of front camera

Similar Documents

Publication Publication Date Title
US10664156B2 (en) Curve-fitting approach to touch gesture finger pitch parameter extraction
US10175805B2 (en) Bottom-up watershed dataflow method and region-specific segmentation based on historic data to identify patches on a touch sensor panel
TWI419025B (en) A sensing device for detecting contact with or close to an object, and a display device for mounting the same
JP5693729B2 (en) Method for detecting an arbitrary number of touches from a multi-touch device
CN102999198B (en) Touch panel edge holds detection method and the device of touch
Nai et al. Fast hand posture classification using depth features extracted from random line segments
US20120192119A1 (en) Usb hid device abstraction for hdtp user interfaces
US20140176500A1 (en) Piecewise-linear and piecewise-affine subspace transformations for high dimensional touchpad (hdtp) output decoupling and corrections
US20120056846A1 (en) Touch-based user interfaces employing artificial neural networks for hdtp parameter and symbol derivation
JP2013525891A (en) Method and device for determining a user's touch gesture
CN101644984B (en) Method and device for detecting touch screen
CN106681575B (en) Use the sliding block and gesture identification of capacitance sensing
JPH10214346A6 (en) Hand gesture recognition system and method
US20180032170A1 (en) System and method for estimating location of a touch object in a capacitive touch panel
Sezgin et al. Scale-space based feature point detection for digital ink
KR101559502B1 (en) Method and recording medium for contactless input interface with real-time hand pose recognition
US20130265258A1 (en) Method for identifying touch on a touch screen
CN110309726B (en) Micro-gesture recognition method
CN116931735A (en) AR (augmented reality) glasses display terminal equipment key suspension position identification system and method
Hartanto et al. Real time hand gesture movements tracking and recognizing system
CN110442242B (en) Intelligent mirror system based on binocular space gesture interaction and control method
Lee et al. Fast hand and finger detection algorithm for interaction on smart display
CN106502460B (en) A kind of recognition methods of capacitance touching control track noise type
CN209674344U (en) Touch screen controller
CN113807191A (en) Non-invasive visual test script automatic recording method

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination