CN110413111A - A kind of target keyboard tracing system and method - Google Patents
A kind of target keyboard tracing system and method Download PDFInfo
- Publication number
- CN110413111A CN110413111A CN201910615492.8A CN201910615492A CN110413111A CN 110413111 A CN110413111 A CN 110413111A CN 201910615492 A CN201910615492 A CN 201910615492A CN 110413111 A CN110413111 A CN 110413111A
- Authority
- CN
- China
- Prior art keywords
- keyboard
- coordinate
- tracing
- mid
- intelligent glasses
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000000034 method Methods 0.000 title claims abstract description 26
- 239000011521 glass Substances 0.000 claims abstract description 36
- 239000011159 matrix material Substances 0.000 claims abstract description 19
- 230000009466 transformation Effects 0.000 claims abstract description 18
- 230000033001 locomotion Effects 0.000 claims abstract description 10
- 230000003993 interaction Effects 0.000 claims abstract description 8
- 238000000605 extraction Methods 0.000 claims abstract description 4
- 238000001514 detection method Methods 0.000 claims description 9
- 238000012545 processing Methods 0.000 claims description 8
- 239000000284 extract Substances 0.000 claims description 7
- 239000012491 analyte Substances 0.000 claims description 5
- 238000012544 monitoring process Methods 0.000 claims description 5
- 230000006835 compression Effects 0.000 claims description 3
- 238000007906 compression Methods 0.000 claims description 3
- 238000001914 filtration Methods 0.000 claims description 3
- 238000013507 mapping Methods 0.000 claims description 3
- 230000000877 morphologic effect Effects 0.000 claims description 3
- 238000004321 preservation Methods 0.000 claims description 3
- 230000003068 static effect Effects 0.000 claims description 3
- FGUUSXIOTUKUDN-IBGZPJMESA-N C1(=CC=CC=C1)N1C2=C(NC([C@H](C1)NC=1OC(=NN=1)C1=CC=CC=C1)=O)C=CC=C2 Chemical compound C1(=CC=CC=C1)N1C2=C(NC([C@H](C1)NC=1OC(=NN=1)C1=CC=CC=C1)=O)C=CC=C2 FGUUSXIOTUKUDN-IBGZPJMESA-N 0.000 claims description 2
- 230000005540 biological transmission Effects 0.000 claims description 2
- 230000008569 process Effects 0.000 abstract description 7
- 238000006243 chemical reaction Methods 0.000 abstract description 5
- 238000010586 diagram Methods 0.000 description 11
- 238000005265 energy consumption Methods 0.000 description 7
- 230000002452 interceptive effect Effects 0.000 description 5
- 230000003044 adaptive effect Effects 0.000 description 2
- 238000013459 approach Methods 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000019771 cognition Effects 0.000 description 1
- 230000007812 deficiency Effects 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 230000004927 fusion Effects 0.000 description 1
- 238000012806 monitoring device Methods 0.000 description 1
- 230000008447 perception Effects 0.000 description 1
- 238000007639 printing Methods 0.000 description 1
- 230000009467 reduction Effects 0.000 description 1
- 238000010187 selection method Methods 0.000 description 1
- 230000005236 sound signal Effects 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/011—Arrangements for interaction with the human body, e.g. for user immersion in virtual reality
- G06F3/013—Eye tracking input arrangements
Landscapes
- Engineering & Computer Science (AREA)
- General Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Human Computer Interaction (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Position Input By Displaying (AREA)
- User Interface Of Digital Computer (AREA)
Abstract
The invention discloses a kind of target keyboard tracing system and methods, applied in intelligent glasses and paper keyboard the input interaction scenarios of movement；The system includes: keyboard tracing module and self-adoptive trace module；Keyboard tracing module includes the extraction of keyboard, characteristic point to selection, calculating transformation matrix and the conversion of keyboard coordinate；Self-adoptive trace module includes that sensing data obtains, data sectional analysis, judges state and the signal for whether continuing tracking sent.The present invention reduces the time delay of tracking process using multithreading optimisation technique；Real-time keyboard tracking is realized, and meets the keyboard mutuality input demand of user well.
Description
Technical field
The invention belongs to video processing and inertia cognition technology fields, refer specifically to for a kind of target keyboard tracing system and side
Method is applied in the intelligent glasses and paper keyboard input interaction scenarios of movement.
Background technique
In recent years, the shape of mobile device and volume gradually tend to be portable small and exquisite, and people generally compare repulsion and carry heaviness
Huge laptop, and get used to one smart phone of a band, intelligent glasses or smartwatch.But it is small in this youngest
Interacting on Intelligent mobile equipment is a very big challenge, and typical pain spot does not have physical keyboard auxiliary when being exactly typewriting input
It helps, causes input efficiency extremely low.
Current solution mainly has wearable keyboard, on-screen keyboard and the keyboard input based on sound.Wearable key
Disk introduces additional facility load, causes user experience unfriendly；On-screen keyboard generally takes up the source of screen of large area, In
It is disposed on intelligent glasses and smartwatch of today very unrealistic；Though it is negative that the keyboard based on audio signal eliminates additional hardware
Load, but it requires to issue apparent cacophony when user's click keys with finger tip and nail, this unnatural input mode
Also make user experience very poor.At the same time, the built-in sensors in the mobile devices such as smart phone and intelligent glasses are also increasingly
Abundant, they are generally equipped with camera and inertial sensor, and the energy consumption of sensor is very low, this is to generate newly more convenient
Efficient input interactive mode provides possibility.
Summary of the invention
Above-mentioned the deficiencies in the prior art are directed to, the purpose of the present invention is to provide a kind of target keyboard tracing system and sides
Method, to solve the problems, such as that existing keyboard input tape comes.The present invention passes through camera for intelligent glasses and a plain paper printed
Matter keyboard establishes connection, can be provided for user similar physical keyboard input experience, while track input interactive process in by
The keyboard coordinate position caused by user's head is shaken changes, and the motion state of the sensor monitoring device according to low energy consumption
It is converted to complete adaptive tracking, the energy consumption of equipment is reduced in a manner of the fusion of multi-modal perception data, it is final to realize efficiently
The scene application of rate and the input interaction of low energy consumption keyboard.
In order to achieve the above objectives, The technical solution adopted by the invention is as follows:
A kind of target keyboard tracing system of the invention is applied in intelligent glasses equipment, comprising: keyboard tracing module
And self-adoptive trace module, wherein
Keyboard tracing module is used for the coordinate that Detection and Extraction go out keyboard and key, is based on keyboard in continuous two interframe
Intrinsic geometrical characteristic chooses corresponding four groups of characteristic points pair, transformation matrix C is calculated using four groups of characteristic points, to former frame
Coordinate [Xi,Yi, 1] and the coordinate [U of present frame is calculated by premultiplication transformation matrix Ci,Vi,Wi], the side Z is done to current frame coordinate
To compression carry out switching keyboard coordinate；
Self-adoptive trace module acquires the rotary motion data of intelligent glasses equipment by inertial sensor, is based on one
A sliding window calculates analyte sensors data, passes through the mean value and variance to data in sliding window and given empirical value
Compare, to determine whether continuing the real-time tracing of target keyboard, and the judging result data of generation is sent to above-mentioned key
Disk tracing module.
Further, the self-adoptive trace module is specific further include: if it is determined that result be intelligent glasses equipment be in
Motion state, then sending beginning trace signals makes keyboard tracing module continue to track keyboard coordinate；If it is determined that result be intelligence
Glasses device remains static, then sending stopping trace signals making keyboard tracing module stop tracking.
Further, the characteristic point specifically includes selection:
A. it removes the influence that selects characteristic point of hand region: being partitioned into using Face Detection and be likely to occur in frame picture
Then hand is filled with background white to erase hand region；
B. it extracts the convex closure point of keyboard area: the profile of the remaining keyboard of detection after hand region is removed, according to profile results
The convex closure point sequence for extracting keyboard, is denoted as C；
C. the first two convex closure point P in C is first taken outbeginAnd PmidIf distance between the two less than 10, indicates the two very
It is close, the size characteristic of keyboard is not met, updates P at this timemidFor convex closure point next in C, until PbeginAnd PmidBetween away from
From greater than 10；Obtain suitable PbeginAnd PmidAfterwards, then from C it takes out a convex closure point and is denoted as Pend；Ibid, if PmidAnd PendIt
Between distance less than 10, then update PendFor convex closure point next in C, until PmidAnd PendThe distance between be greater than 10；
D. vector is calculatedAnd vectorAngle, if angle be less than 8 degree of threshold value, indicate these three convex closures
Point direction is identical, PmidIt is updated to PendValue, continue to take P in return step c)end；If angle is greater than 8 degree, indicate that these three are convex
Packet point is in PmidPlace, which has, significantly turns, at this time by PmidIt is saved as the characteristic point that detects, and by PbeginIt is updated to Pmid, Pmid
It is updated to Pend, return and continue to take P in Cend；
E. after traversal has analyzed convex closure point set C, the quantity of the convex closure point of preservation is four or five；If four
Then directly as the characteristic point detected, step f) execution is jumped to；It then needs to carry out following processing: five convex closures if five
Line in order is put, is sorted according to the order of magnitude of straight slope；Remove the straight line that slope value is median, remaining straight line
[l is denoted as according to relative rank0,l1,l2,l3], l0And l2、l3Intersection, l1And l2、l3Four characteristic points are obtained in intersection, that is, detect
The characteristic point result arrived；
F. remember that the convex closure point set of two continuous frames is combined into C1And C2, to C1And C2Carry out step b)-e as above) operation
Two groups of characteristic points are obtained, are denoted as (X respectivelyi,Yi) and (Ui,Vi), 0≤i≤3 are sorted according to transverse and longitudinal coordinate value size, by two groups
Characteristic point establishes mapping relations according to the sequence of upper left, upper right, lower-left, bottom right, obtains four characteristic points to (X0,Y0,U0,V0)。
Further, the method for calculating transformation matrix specifically: according to characteristic point to (Xi,Yi,Ui,Vi), 0≤i≤
3, system of linear equations is established, transformation matrix C is calculated:
Further, the switching keyboard sits calibration method specifically: sets the keyboard coordinate of former frame as (Xi,Yi), conversion
For the coordinate (U of the framei,Vi), wherein C is transformation matrix:
A kind of target keyboard method for tracing of the invention, is applied to intelligent glasses equipment and paper keyboard inputs interaction scenarios
Under, it comprises the following steps that
1) friendship is inputted on paper keyboard with the rate acquisition user of 30 frame per second using the camera of intelligent glasses equipment
Mutual video during video acquisition, acquires gyro data with the frequency of 50Hz；
2) to the frame sequence of acquisition smothing filtering and Morphological scale-space in real time, to reduce noise；
3) for the first frame of frame sequence, detect and extract the coordinate of keyboard and each key；
4) for frame sequence later, the angle point of keyboard is chosen and matched between two continuous frames as characteristic point pair；
5) according to the characteristic point of selection to the corresponding transformation matrix of calculating, and the coordinate of former frame is converted into present frame；
6) using inertial sensor acquisition intelligent glasses equipment moving data, monitoring intelligent glasses equipment state, to judge
Whether continue to track keyboard.
Further, intelligent glasses equipment state is monitored using sensor to complete self-adoptive trace in the step 6)
Method are as follows:
61) gyroscope real-time monitoring intelligent glasses equipment state, receiving sensor data are used；
62) mean value and variance that three-axis sensor data are calculated using the sliding window that size is 10, when in sliding window
When data mean value and variance are respectively less than threshold value, indicate that equipment is remain stationary at this time, is sent the signal for stopping tracking, is stopped keyboard
Coordinate tracking,；When in window data mean value or variance be greater than the threshold value when, indicate that equipment at this time is moving, transmission is opened
Begin the signal tracked, and continues the coordinate of real-time tracing keyboard.
The step 1), step 2 to 5), step 6) is respectively adopted individual thread and carrys out parallel processing；Use three threads
Concurrently capturing sequence of frames of video, extracting characteristic point and converting coordinate and analyte sensors data to decide whether to continue
Tracking；By multi-threading parallel process, the processing time of tracking conversion is reduced, realizes real-time keyboard tracking.
Beneficial effects of the present invention:
The present invention can reduce the hardware burden in user's keyboard input process, only need the common paper keyboard of a printing
Connection is established by camera and intelligent glasses equipment；Application scenarios are that user wears the certain typewritings times of intelligent glasses completion
It is engaged in (for example sending message), keyboard tracing module can calculate the conversion process of equipment, switching keyboard in the matrix form in real time
With the coordinate of each key；In addition, the present invention is also using the low energy consumption inertial sensor being equipped in equipment come real-time watch device
Motion state, realize whether adaptive adjustment carries out keyboard tracking, as far as possible the high energy consumption operation of reduction keyboard real-time tracing,
To reduce the whole energy consumption of equipment on the basis of not significantly reducing user's keyboard input precision, while can be brought more to user
Natural smart machine keyboard input experience.
Detailed description of the invention
Fig. 1 is painted present system module design figure.
Fig. 2 is painted characteristic point selection process flow diagram.
Fig. 3 is painted method flow diagram.
Fig. 4 is painted multiple threads schematic diagram.
Fig. 5 a is painted system initialization interface schematic diagram.
Fig. 5 b is painted system and brings into operation interface schematic diagram.
Fig. 5 c is painted system operation interface schematic diagram.
Fig. 5 d is painted system tracks keyboard interface schematic diagram.
Fig. 5 e continues to run interface schematic diagram after being painted system tracks keyboard.
Fig. 5 f is painted system finishing runnable interface schematic diagram.
Specific embodiment
For the ease of the understanding of those skilled in the art, the present invention is made further below with reference to embodiment and attached drawing
Bright, the content that embodiment refers to not is limitation of the invention.
Shown in referring to Fig.1, a kind of target keyboard tracing system of the invention, is applied to intelligent glasses and paper keyboard inputs
Under interaction scenarios, comprising: keyboard tracing module and self-adoptive trace module, wherein
Keyboard tracing module is used for the coordinate that Detection and Extraction go out keyboard and key, is based on keyboard in continuous two interframe
Intrinsic geometrical characteristic chooses corresponding four groups of characteristic points pair, transformation matrix C is calculated using four groups of characteristic points, to former frame
Coordinate [Xi,Yi, 1] and the coordinate [U of present frame is calculated by premultiplication transformation matrix Ci,Vi,Wi], the side Z is done to current frame coordinate
To compression carry out switching keyboard coordinate；
Self-adoptive trace module acquires the rotary motion data of intelligent glasses equipment by inertial sensor, is based on one
A sliding window calculates analyte sensors data, passes through the mean value and variance to data in sliding window and given empirical value
Compare, to determine whether continuing the real-time tracing of target keyboard, and the judging result data of generation is sent to above-mentioned key
Disk tracing module.
If it is determined that result be intelligent glasses equipment be kept in motion, then send start trace signals make keyboard track mould
Block continues to track keyboard coordinate；If it is determined that result be intelligent glasses equipment remain static, then send stopping trace signals
Keyboard tracing module is set to stop tracking keyboard coordinate.
Referring to shown in Fig. 2, the characteristic point selection method based on the intrinsic geometrical characteristic of keyboard between the two continuous frames are as follows:
A. remove the influence that hand region selects characteristic point: user carries out key naturally in the sequence of frames of video of capture
Disk input, but hand can block keyboard and then influence the selection of characteristic point；Being partitioned into frame picture first with Face Detection may
Then the hand of appearance is filled with background white to erase hand region；
B. it extracts the convex closure point of keyboard area: the profile of the remaining keyboard of detection after hand region is removed, according to profile results
The convex closure point sequence for extracting keyboard is denoted as C, and four angle points that keyboard is extracted from C are characterized a little；
C. the first two convex closure point P in C is first taken outbeginAnd PmidIf distance between the two less than 10, indicates the two very
It is close, the size characteristic of keyboard is not met, updates P at this timemidFor convex closure point next in C, until PbeginAnd PmidBetween away from
From greater than 10；Obtain suitable PbeginAnd PmidAfterwards, then from C it takes out a convex closure point and is denoted as Pend；Ibid, if PmidAnd PendIt
Between distance less than 10, then update PendFor convex closure point next in C, until PmidAnd PendThe distance between be greater than 10；
D. vector is calculatedAnd vectorAngle, if angle be less than 8 degree of threshold value, indicate these three convex closures
Point direction is identical, PmidIt is updated to PendValue, continue to take P in return step c)end；If angle is greater than 8 degree, indicate that these three are convex
Packet point is in PmidPlace, which has, significantly turns, at this time by PmidIt is saved as the characteristic point that detects, and by PbeginIt is updated to Pmid, Pmid
It is updated to Pend, return and continue to take P in Cend；
E. after traversal has analyzed convex closure point set C, the quantity of the convex closure point of preservation is four or five；If four
Then directly as last characteristic point；Then need to carry out following processing if five: five convex closure points line in order, according to straight
The order of magnitude of line slope sorts；Because the edge of keyboard be close to it is horizontal and vertical, slope absolute value or
Close to 0 or tend to be very big, removal slope value is the straight line of median, and remaining straight line is denoted as [l according to relative rank0,l1,
l2,l3], l0And l3、l4Intersection, l1And l3、l4Four characteristic points, i.e., last characteristic point result is obtained in intersection；
F. remember that the convex closure point set of two continuous frames is combined into C1And C2；To C1And C2Carry out step b)-e as above) operation
Two groups of characteristic points are obtained, are denoted as (X respectivelyi,Yi) and (Ui,Vi) (0≤i≤3), it sorts according to transverse and longitudinal coordinate value size, by two groups
Characteristic point establishes mapping relations according to the sequence of upper left, upper right, lower-left, bottom right, obtains (Xi,Yi,Ui,Vi)。
Wherein, the calculating transformation matrix and switching keyboard sit calibration method specifically:
A. the keyboard coordinate (X of former frame is seti,Yi) be converted to the frame (Ui,Vi), wherein C is transformation matrix,
B. according to characteristic point to (Xi,Yi) and (Ui,Vi) (0≤i≤3) establish system of linear equations, calculate transformation matrix C:
Referring to shown in Fig. 3, a kind of target keyboard method for tracing of the invention is applied to intelligent glasses equipment and paper keyboard
It inputs under interaction scenarios, comprises the following steps that
1) using the camera of intelligent glasses equipment, with the rate acquisition user of 30 frame per second, in paper keyboard, (one is printed
Common paper keyboard) on input interactive video, during video acquisition, gyro data is acquired with the frequency of 50Hz；
2) to the frame sequence of acquisition smothing filtering and Morphological scale-space in real time, to reduce noise；
3) for the first frame of frame sequence, detect and extract the coordinate of keyboard and each key；
4) for frame sequence later, the angle point of keyboard is chosen and matched between two continuous frames as characteristic point pair；
5) according to the characteristic point of selection to the corresponding transformation matrix of calculating, and the coordinate of former frame is converted into present frame；
6) using inertial sensor acquisition intelligent glasses equipment moving data, monitoring intelligent glasses equipment state, to judge
Whether continue to track keyboard.
Referring to shown in Fig. 4, the step 1), step 2 to 5), step 6) is respectively adopted individual thread and carrys out parallel processing；
Specifically, sequence of frames of video is concurrently captured using three threads, extracts characteristic point and converts coordinate and analyte sensors
Data are to decide whether to continue to track；By multi-threading parallel process, the processing time of tracking conversion is reduced, is realized real-time
Keyboard tracking.
Fig. 5 a- Fig. 5 f is keyboard tracing system interactive interface schematic diagram.When user runs program, into program main interface,
It can be seen that the preview screen of camera, as shown in Figure 5 a.Program starts work after the Capture button in menu bar when the user clicks
Make, keyboard and key extracted to the first frame that captures, and in the frame sequence captured later real-time tracing keyboard key
Coordinate, as shown in Figure 5 b.While real-time tracing, program can be on backstage with the number of the frequency collection gyro sensor of 50Hz
According to Fig. 5 c to Fig. 5 e is the diagram of user's keyboard mutuality scene, and the entry wherein shown in Fig. 5 d " Update keyMap " indicates
The tracking of keyboard is being carried out at this time.After user completes the interactive task of keyboard, again tap on Capture in menu bar by
Button can halt system work.
There are many concrete application approach of the present invention, the above is only a preferred embodiment of the present invention, it is noted that for
For those skilled in the art, without departing from the principle of the present invention, it can also make several improvements, this
A little improve also should be regarded as protection scope of the present invention.
Claims (7)
1. a kind of target keyboard tracing system is applied in intelligent glasses equipment characterized by comprising keyboard tracing module
And self-adoptive trace module, wherein
Keyboard tracing module is used for the coordinate that Detection and Extraction go out keyboard and key, and it is intrinsic to be based on keyboard in continuous two interframe
Geometrical characteristic choose corresponding four groups of characteristic points pair, transformation matrix C are calculated using four groups of characteristic points, to the coordinate of former frame
[Xi,Yi, 1] and the coordinate [U of present frame is calculated by premultiplication transformation matrix Ci,Vi,Wi], Z-direction is done to current frame coordinate
Compression carrys out switching keyboard coordinate；
Self-adoptive trace module acquires the rotary motion data of intelligent glasses equipment by inertial sensor, is based on a cunning
Dynamic window calculation analyte sensors data pass through the mean value and variance to data in sliding window and given empirical value ratio
Compared with to determine whether continuing the real-time tracing of target keyboard, and the judging result data of generation being sent to above-mentioned keyboard
Tracing module.
2. target keyboard tracing system according to claim 1, which is characterized in that the self-adoptive trace module is specifically gone back
Include: if it is determined that result be intelligent glasses equipment be kept in motion, then send start trace signals make keyboard tracing module
Continue to track keyboard coordinate；If it is determined that result be intelligent glasses equipment remain static, then send stop trace signals making
Keyboard tracing module stops tracking keyboard coordinate.
3. target keyboard tracing system according to claim 1, which is characterized in that the characteristic point is to the specific packet of selection
It includes:
A. it removes the influence that hand region selects characteristic point: being partitioned into the hand being likely to occur in frame picture using Face Detection,
Then it is filled with background white to erase hand region；
B. it extracts the convex closure point of keyboard area: removing the profile of the remaining keyboard of detection after hand region, extracted according to profile results
The convex closure point sequence of keyboard, is denoted as C；
C. the first two convex closure point P in C is first taken outbeginAnd PmidIf distance between the two less than 10, indicate both very close to,
The size characteristic for not meeting keyboard, updates P at this timemidFor convex closure point next in C, until PbeginAnd PmidThe distance between be greater than
10；Obtain suitable PbeginAnd PmidAfterwards, then from C it takes out a convex closure point and is denoted as Pend；Ibid, if PmidAnd PendBetween away from
From less than 10, then P is updatedendFor convex closure point next in C, until PmidAnd PendThe distance between be greater than 10；
D. vector is calculatedAnd vectorAngle, if angle be less than 8 degree of threshold value, indicate these three convex closure point sides
To identical, PmidIt is updated to PendValue, continue to take P in return step c)end；If angle is greater than 8 degree, these three convex closure points are indicated
In PmidPlace, which has, significantly turns, at this time by PmidIt is saved as the characteristic point that detects, and by PbeginIt is updated to Pmid, PmidIt updates
For Pend, return and continue to take P in Cend；
E. after traversal has analyzed convex closure point set C, the quantity of the convex closure point of preservation is four or five；It is then straight if four
It connects as the characteristic point detected, jumps to step f) execution；Then need to carry out following processing if five: five convex closures are pressed
Order line sorts according to the order of magnitude of straight slope；Remove slope value be median straight line, remaining straight line according to
Relative rank is denoted as [l0,l1,l2,l3], l0And l2、l3Intersection, l1And l2、l3Four characteristic points are obtained in intersection, that is, detect
Characteristic point result；
F. remember that the convex closure point set of two continuous frames is combined into C1And C2, to C1And C2Carry out step b)-e as above) operation, can be obtained two
Group characteristic point, is denoted as (X respectivelyi,Yi) and (Ui,Vi), 0≤i≤3 are sorted according to transverse and longitudinal coordinate value size, by two groups of characteristic points
Mapping relations are established according to the sequence of upper left, upper right, lower-left, bottom right, obtain four characteristic points to (Xi,Yi,Ui,Vi)。
4. target keyboard tracing system according to claim 1, which is characterized in that the calculating transformation matrix method is specific
Are as follows: according to characteristic point to (Xi,Yi,Ui,Vi), system of linear equations is established in 0≤i≤3, calculate transformation matrix C:
5. target keyboard tracing system according to claim 4, which is characterized in that the switching keyboard sits calibration method tool
Body are as follows: set the keyboard coordinate of former frame as (Xi,Yi), be converted to the coordinate (U of the framei,Vi):
6. a kind of target keyboard method for tracing is applied under intelligent glasses equipment and paper keyboard input interaction scenarios, feature
It is, comprises the following steps that
1) interaction is inputted on paper keyboard with the rate acquisition user of 30 frame per second using the camera of intelligent glasses equipment
Video during video acquisition, acquires gyro data with the frequency of 50Hz；
2) to the frame sequence of acquisition smothing filtering and Morphological scale-space in real time；
3) for the first frame of frame sequence, detect and extract the coordinate of keyboard and each key；
4) for frame sequence later, the angle point of keyboard is chosen and matched between two continuous frames as characteristic point pair；
5) according to the characteristic point of selection to the corresponding transformation matrix of calculating, and the coordinate of former frame is converted into present frame；
6) using inertial sensor acquisition intelligent glasses equipment moving data, monitoring intelligent glasses equipment state, to determine whether
Continue to track keyboard.
7. target keyboard method for tracing according to claim 6, which is characterized in that utilize sensor in the step 6)
Intelligent glasses equipment state is monitored to complete self-adoptive trace method are as follows:
61) gyroscope real-time monitoring intelligent glasses equipment state, receiving sensor data are used；
62) mean value and variance that three-axis sensor data are calculated using the sliding window that size is 10, when data in sliding window
When mean value and variance are respectively less than threshold value, indicate that equipment is remain stationary at this time, is sent the signal for stopping tracking, is stopped the coordinate of keyboard
Tracking；When in window data mean value or variance be greater than the threshold value when, indicate that equipment at this time is moving, transmission start to track
Signal, continue real-time tracing keyboard coordinate.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910615492.8A CN110413111B (en) | 2019-07-09 | 2019-07-09 | Target keyboard tracking system and method |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910615492.8A CN110413111B (en) | 2019-07-09 | 2019-07-09 | Target keyboard tracking system and method |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN110413111A true CN110413111A (en) | 2019-11-05 |
| CN110413111B CN110413111B (en) | 2021-06-01 |
Family
ID=68360748
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910615492.8A Active CN110413111B (en) | 2019-07-09 | 2019-07-09 | Target keyboard tracking system and method |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110413111B (en) |
Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101907954A (en) * | 2010-07-02 | 2010-12-08 | 中国科学院深圳先进技术研究院 | Interactive projection system and interactive projection method |
| US20150293644A1 (en) * | 2014-04-10 | 2015-10-15 | Canon Kabushiki Kaisha | Information processing terminal, information processing method, and computer program |
| CN105229574A (en) * | 2013-01-14 | 2016-01-06 | 纽昂斯通信有限公司 | Reduce the error rate based on the keyboard touched |
| CN106933376A (en) * | 2017-03-23 | 2017-07-07 | 哈尔滨拓博科技有限公司 | A kind of scaling method of smooth projected keyboard |
| CN107066935A (en) * | 2017-01-25 | 2017-08-18 | 网易（杭州）网络有限公司 | Hand gestures method of estimation and device based on deep learning |
| CN206711065U (en) * | 2017-03-23 | 2017-12-05 | 哈尔滨拓博科技有限公司 | Device for nominal light projected keyboard |
| CN107507247A (en) * | 2017-08-28 | 2017-12-22 | 哈尔滨拓博科技有限公司 | A kind of real-time dynamic autoization scaling method of projected keyboard |
| CN108829268A (en) * | 2018-07-11 | 2018-11-16 | 南京邮电大学 | Keyboard and input method based on single RGB camera |
-
2019
- 2019-07-09 CN CN201910615492.8A patent/CN110413111B/en active Active
Patent Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101907954A (en) * | 2010-07-02 | 2010-12-08 | 中国科学院深圳先进技术研究院 | Interactive projection system and interactive projection method |
| CN105229574A (en) * | 2013-01-14 | 2016-01-06 | 纽昂斯通信有限公司 | Reduce the error rate based on the keyboard touched |
| US20150293644A1 (en) * | 2014-04-10 | 2015-10-15 | Canon Kabushiki Kaisha | Information processing terminal, information processing method, and computer program |
| CN107066935A (en) * | 2017-01-25 | 2017-08-18 | 网易（杭州）网络有限公司 | Hand gestures method of estimation and device based on deep learning |
| CN106933376A (en) * | 2017-03-23 | 2017-07-07 | 哈尔滨拓博科技有限公司 | A kind of scaling method of smooth projected keyboard |
| CN206711065U (en) * | 2017-03-23 | 2017-12-05 | 哈尔滨拓博科技有限公司 | Device for nominal light projected keyboard |
| CN107507247A (en) * | 2017-08-28 | 2017-12-22 | 哈尔滨拓博科技有限公司 | A kind of real-time dynamic autoization scaling method of projected keyboard |
| CN108829268A (en) * | 2018-07-11 | 2018-11-16 | 南京邮电大学 | Keyboard and input method based on single RGB camera |
Also Published As
| Publication number | Publication date |
|---|---|
| CN110413111B (en) | 2021-06-01 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN103353935B (en) | A kind of 3D dynamic gesture identification method for intelligent domestic system | |
| US10095033B2 (en) | Multimodal interaction with near-to-eye display | |
| CN103136986B (en) | Sign Language Recognition Method and system | |
| CN104410883B (en) | The mobile wearable contactless interactive system of one kind and method | |
| CN104346816B (en) | Depth determining method and device and electronic equipment | |
| CN101976330B (en) | Gesture recognition method and system | |
| CN102769802A (en) | Man-machine interactive system and man-machine interactive method of smart television | |
| Fillbrandt et al. | Extraction of 3D hand shape and posture from image sequences for sign language recognition | |
| CN102271241A (en) | Image communication method and system based on facial expression/action recognition | |
| CN107632699A (en) | Natural human-machine interaction system based on the fusion of more perception datas | |
| CN102222342A (en) | Tracking method of human body motions and identification method thereof | |
| CN103530892A (en) | Kinect sensor based two-hand tracking method and device | |
| CN103500335A (en) | Photo shooting and browsing method and photo shooting and browsing device based on gesture recognition | |
| CN105069745A (en) | face-changing system based on common image sensor and enhanced augmented reality technology and method | |
| CN106909871A (en) | Gesture instruction recognition methods | |
| CN106055092A (en) | Method and system for implementing interactive projection | |
| CN103000054B (en) | Intelligent teaching machine for kitchen cooking and control method thereof | |
| CN109395375A (en) | A kind of 3d gaming method of interface interacted based on augmented reality and movement | |
| CN111626135A (en) | Three-dimensional gesture recognition system based on depth map | |
| CN104898971A (en) | Mouse pointer control method and system based on gaze tracking technology | |
| CN106774827A (en) | A kind of projection interactive method, projection interactive device and intelligent terminal | |
| CN202068503U (en) | Video communication system | |
| CN110413111A (en) | A kind of target keyboard tracing system and method | |
| CN107479713A (en) | The man-machine interaction method and mobile device of a kind of mobile device | |
| CN113589928B (en) | Gesture recognition method for intelligent television |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |