CN110188669A - A kind of aerial hand-written character track restoration methods based on attention mechanism - Google Patents
A kind of aerial hand-written character track restoration methods based on attention mechanism Download PDFInfo
- Publication number
- CN110188669A CN110188669A CN201910454906.3A CN201910454906A CN110188669A CN 110188669 A CN110188669 A CN 110188669A CN 201910454906 A CN201910454906 A CN 201910454906A CN 110188669 A CN110188669 A CN 110188669A
- Authority
- CN
- China
- Prior art keywords
- input
- sequence
- network
- attention mechanism
- attention
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/017—Gesture based interaction, e.g. based on a set of recognized hand gestures
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/03—Arrangements for converting the position or the displacement of a member into a coded form
- G06F3/033—Pointing devices displaced or positioned by the user, e.g. mice, trackballs, pens or joysticks; Accessories therefor
- G06F3/0346—Pointing devices displaced or positioned by the user, e.g. mice, trackballs, pens or joysticks; Accessories therefor with detection of the device orientation or free movement in a 3D space, e.g. 3D mice, 6-DOF [six degrees of freedom] pointers using gyroscopes, accelerometers or tilt-sensors
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/20—Image preprocessing
- G06V10/30—Noise filtering
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/20—Movements or behaviour, e.g. gesture recognition
- G06V40/28—Recognition of hand or arm movements, e.g. recognition of deaf sign language
Abstract
The aerial hand-written character track restoration methods based on attention mechanism that the invention discloses a kind of; the following steps are included: the inertia sensing signal sequence and plane trajectory coordinates sequence to the pairing got carry out noise reduction filtering, and divide training sample set and test sample collection；A sequence based on attention mechanism is designed to series model, including a coding network and a decoding network；With training sample set training sequence to series model；The inertia sensing signal sequence of test sample collection is input to trained sequence to series model, the result that the sequence that model exports is restored as track.The method of the invention has the advantages that aerial hand-written character track recovery effects are more smooth and it is higher to restore precision.
Description
Technical field
The present invention relates to machine learning and field of artificial intelligence, and in particular to a kind of based on the aerial of attention mechanism
Hand-written character track restoration methods.
Background technique
Based on inertial sensor (accelerometer and gyroscope) it is aerial it is hand-written be newly to rise in computer field in recent years
One of study frontier direction, smart home, automatic Pilot, education, medical treatment, industrial production, in terms of have it is wide
General application.But inertial signal (acceleration and angular speed signal) is the amount for measuring variation, they are readable poor, only with people
Eye observation waveform, it is difficult to recognize specific written contents.This feature of inertial signal is bad sample in the processing stage of data
Cleaning, sample mark, the other sample decomposition of character level bring difficulty, also limit the tuning of aerial handwriting recognition model.Cause
This, hand-written research field has great importance in the sky for the visualization of inertial signal, and inertial signal is reverted to corresponding rail
Mark, undoubtedly a kind of method for visualizing of simple, intuitive.Traditional method, which focuses primarily upon, directly restores track from inertial signal
And the accumulated error of correction motion track, lack the effective use to true handwriting trace, restores precision and practicability day
Cannot gradually user be enabled to please oneself.
Summary of the invention
In view of the deficiencies of the prior art, it is an object of the present invention to provide a kind of aerial handwritten words based on attention mechanism
Track restoration methods are accorded with, sequence of the design based on attention mechanism to series model (Sequence to Sequence is passed through
Model), a kind of completely new aerial hand-written character inertia sensing signal visual means are provided, wherein attention mechanism can have
Effect captures the key message of inertial signal, and sequence to series model can make full use of the inertial signal and planar obit simulation of pairing, learns
The mapping relations of the two are practised, deep learning makes aerial handwriting tracks recovery can achieve outstanding effect, it is extensive to improve track
Multiple precision has good application study value.
The purpose of the present invention can be achieved through the following technical solutions:
A kind of aerial hand-written character track restoration methods based on attention mechanism, the described method comprises the following steps:
S1, the inertia sensing signal sequence and plane trajectory coordinates sequence for obtaining pairing, are filtered smoothing denoising, and draw
Divide training sample set and test sample collection；
S2, design simultaneously train a sequence based on attention mechanism to series model, the specific steps of which are as follows:
One S21, design coding network, network from attention mechanism module and feedforward neural network by constituting；With inertia
Transducing signal sequence be input, by the moment entirely connect mapping after, calculating position coding, then by from attention mechanism module with
Feedforward neural network calculates and exports characteristic pattern；
One S22, design decoding network, network is by exposure mask from attention mechanism module, attention mechanism module and feedforward
Neural network is constituted；It is input with inertia sensing signal sequence, after connecting mapping entirely by the moment, calculating position coding, by covering
Film extracts feature from attention mechanism module, is then input with this feature and by the characteristic pattern of coding network output, by paying attention to
Power mechanism module and feedforward neural network extract feature, and feature progress is connected calculating by the moment entirely, the plane predicted
Trajectory coordinates sequence；
S23, use the inertia sensing signal sequence in training sample set as input, the planar obit simulation coordinate sequence of pairing is made
For target, Training is carried out simultaneously to above-mentioned coding network and decoding network, the planar obit simulation of decoding network prediction is sat
It marks sequence and true planar trajectory coordinates sequence carries out costing bio disturbance, by error back propagation, Optimized Coding Based network and decoding net
The parameter of network；
S3, the inertia sensing signal sequence to input after filtering denoising, are calculated special using trained coding network
Sign figure, is input to trained decoding network for inertia sensing signal sequence, the planar obit simulation being restored together with characteristic pattern.
Further, data acquisition, pretreatment and the partition process in the step S1, the specific steps are as follows:
S11, the aerial hand-written character inertia sensing signal sequence and plane rail that pairing is obtained from public data collection 6DMG
Mark coordinate sequence, 6 dimension inertia sensing signals include 3 dimension acceleration signals and 3 dimension angular velocity signals, 2 dimensional plane trajectory coordinates packets
Include x-axis and y-axis coordinate；The sample of so-called pairing refers to that the inertia in the same aerial hand-written character sample of data set 6DMG passes
Feel signal sequence and plane trajectory coordinates sequence；
S12, fix to the inertia sensing signal sequence and plane trajectory coordinates sequence of acquisition length of window window a length of 5
Moving average filter denoising；
S13, using the inertia sensing signal sequence of pretreated pairing and plane trajectory coordinates sequence as a sample,
Inertia sensing signal sequence is sample input, and planar obit simulation coordinate sequence is target；Sample is divided into training sample set and survey
Try sample set.
Further, if input inertia sensing signal sequence is x (t, i), dimension din, the coding net of the step S21
Network, steps are as follows for concrete operation:
S21 (a), with connected entirely by the moment calculate will input x (t, i) be transformed intoDimension is by dinBecome dmodel；
S21 (b), the position encoded P for calculating transformed input1(t, i), and be superimposed are as follows:
S21 (c), e1It is input to from attention power module and feature e is calculated2；
S21 (d), e2It is input to feedforward neural network and characteristic pattern m is calculated；
The decoding network of the step S22, steps are as follows for concrete operation:
S22 (a), with connected entirely by the moment calculate will input x (t, i) be transformed intoDimension is by dinBecome dmodel；
S22 (b), the position encoded P for calculating transformed input2(t, i), and be superimposed are as follows:
S22 (c), e3It is input to exposure mask and obtains feature e from attention power module4；
S22 (d), e4It is input to the characteristic pattern m of coding network output and notices that feature e is calculated in power module5；
S22 (e), e5It is input to feedforward neural network and feature e is calculated6；
S22 (f), it is calculated e with being connected entirely by the moment6Dimension be transformed into dout, the planar obit simulation that is restored；
Wherein, din=6, dmodel=64, dout=2.
Full connection in S21 (a) and S22 (a) converts not shared parameter, therefore twoNumerical value it is not identical.
Further, in the step S21 and S22, in coding network and decoding network from attention mechanism module, note
Meaning power mechanism module, exposure mask are calculated from the attention in attention mechanism module, formula are as follows:
Wherein Attention (Q, K, V) is the symbol of attention, and Q indicates that index, K indicate key, KTRepresenting matrix K's turns
It sets, V expression value, dkIt is the dimension of K；For connecting transformed input entirelyAnd its position encoded P (t, i), coding network
In from pay attention to power module and decode network in exposure mask from pay attention to power module input The input Q of attention power module in decoding network is that the exposure mask of decoding network pays attention to the output of power module certainly
e4, input K and V is the characteristic pattern m of coding network output.
Further, it in the step S21 (b) and S22 (b), is encoded to transformed input calculating position is connected entirely, position
Set coding formula are as follows:
Wherein, P (t, i) is position encoded symbol, and t indicates the moment, and i indicates dimension, and T and d are full connection transformation respectively
The length and dimension of input afterwards.
Further, the input of exposure mask from attention power module is e in the step S22 (c)3If e3Length be T, exposure mask
Calculating process is as follows:
Wherein QKT(t1,t2) indicate QKTIn (t1,t2) at value, t1And t2All indicate the moment.
Further, in the step S21 and S22, the feedforward neural network in coding network and decoding network, network by
Two layers of full articulamentum is constituted, and the activation primitive of first layer is ReLU, and the second layer does not have activation primitive.
Further, in the step S21 and S22, coding network and decoding network from attention mechanism module, exposure mask
From attention mechanism module, attention mechanism module and feedforward neural network, there are identical mapping, i.e. handle between input and output
The input of module and the output of module add up the output final as module, and identical mapping is specific as follows:
Y=x+f (x)
Wherein, x representation module inputs, the operation of f representation module, and y indicates the module output after identical mapping；
Layer standardization is carried out after identical mapping, specific as follows:
Wherein,Module output after expression layer standardization, α and β be can training parameter, mean (y) indicates to the last of y
One-dimensional to average, std (y) indicates one-dimensional to seek standard deviation to the last of y.
Further, training of the step S23 sequence to series model, loss function formula are as follows:
Wherein, loss is loss function, and y is true planar trajectory coordinates sequence,It is the plane rail predicted by decoding network
Mark coordinate sequence, t indicate the moment, and i indicates dimension, and T is the length of sequence, and d is the dimension of sequence.
Compared with the prior art, the invention has the following advantages and beneficial effects:
1, a kind of aerial hand-written character track restoration methods based on attention mechanism provided by the invention, using based on note
The sequence for power mechanism of anticipating is to series model, and in structure design, it is to sequence by carrying out constantly that the full connection of related input, which calculates,
, therefore there is no limit allow to input the aerial hand-written character inertial signal sequence of random length to the size of input.
2, a kind of aerial hand-written character track restoration methods based on attention mechanism provided by the invention, using based on note
The sequence for power mechanism of anticipating can reach the requirement of real-time recovery to series model, believe an aerial hand-written character inertia sensing
Number sequence does track and restores only to need 30ms.
3, a kind of aerial hand-written character track restoration methods based on attention mechanism provided by the invention are based on attention
The sequence of mechanism extracts the feature of input using attention mechanism to series model, enables list entries each moment
The global characteristics from any other moment are directly received, network can learn the crucial important information of screening, so that in the air
The recovery precision of hand-written character track is higher, and the track of recovery is smoother.
4, a kind of aerial hand-written character track restoration methods based on attention mechanism provided by the invention are based on attention
For the sequence of mechanism to series model, model structure is simple, and expansibility is strong, and ginseng is adjusted to be not difficult.
Detailed description of the invention
Fig. 1 is the training process stream of aerial hand-written character track restoration methods of the embodiment of the present invention based on attention mechanism
Cheng Tu.
Fig. 2 is aerial hand-written character track recovery process flow chart of the sequence of the embodiment of the present invention to series model.
Fig. 3 is aerial hand-written character track recovery effects exemplary diagram of the sequence of the embodiment of the present invention to series model.
Specific embodiment
Present invention will now be described in further detail with reference to the embodiments and the accompanying drawings, but embodiments of the present invention are unlimited
In this.
Embodiment:
Referring to Fig. 1 and Fig. 2, present embodiment discloses a kind of aerial hand-written character track recovery side based on attention mechanism
Method, steps are as follows:
S1, the inertia sensing signal sequence and plane trajectory coordinates sequence for obtaining pairing, are filtered smoothing denoising, and draw
Divide training sample set and test sample collection, the specific steps are as follows:
S11, the aerial hand-written character inertia sensing signal sequence matched from the acquisition in public data collection 6DMG and plane
Trajectory coordinates sequence, 6 dimension inertia sensing signals include 3 dimension acceleration signals and 3 dimension angular velocity signals, 2 dimensional plane trajectory coordinates
Including x-axis and y-axis coordinate.The sample of so-called pairing refers to the inertia in the same aerial hand-written character sample of data set 6DMG
Transducing signal sequence and plane trajectory coordinates sequence；
S12, the moving average filter that window a length of 5 is done to the inertia sensing signal sequence and plane trajectory coordinates sequence of acquisition
Denoising；
S13, using the inertia sensing signal sequence of pretreated pairing and plane trajectory coordinates sequence as a sample,
Inertia sensing signal sequence is sample input, and planar obit simulation coordinate sequence is target.Sample is divided into training sample set and survey
Try sample set；
S2, design simultaneously train a sequence based on attention mechanism to series model, the specific steps of which are as follows:
One S21, design coding network, network from attention mechanism module and feedforward neural network by constituting.With sensing
Signal sequence is input, and after connect mapping entirely by the moment, calculating position is encoded, then by from attention mechanism module and feedforward
Neural computing simultaneously exports characteristic pattern.
If inputting inertia sensing signal sequence is x (t, i), dimension din, specific step is as follows for coding network operation:
S21 (a), with connected entirely by the moment calculate will input x (t, i) be transformed intoDimension is by dinBecome dmodel；
S21 (b), the position encoded P for calculating transformed input1(t, i), and be superimposed are as follows:
S21 (c), e1It is input to from attention power module and feature e is calculated2；
S21 (d), e2It is input to feedforward neural network and characteristic pattern m is calculated.
One S22, design decoding network, network is by exposure mask from attention mechanism module, attention mechanism module and feedforward
Neural network is constituted.It is input with transducing signal sequence, after connecting mapping entirely by the moment, calculating position coding, certainly by exposure mask
Attention mechanism module extracts feature, is then input with this feature and by the characteristic pattern of coding network output, by attention machine
Molding block and feedforward neural network extract feature, and feature progress is connected calculating by the moment entirely, the planar obit simulation predicted
Coordinate sequence.
If inputting inertia sensing signal sequence is x (t, i), dimension din, specific step is as follows for decoding network operation:
S22 (a), with connected entirely by the moment calculate will input x (t, i) be transformed intoDimension is by dinBecome dmodel；
S22 (b), the position encoded P for calculating transformed input2(t, i), and be superimposed are as follows:
S22 (c), e3It is input to exposure mask and obtains feature e from attention power module4；
S22 (d), e4It is input to the characteristic pattern m of coding network output and notices that feature e is calculated in power module5；
S22 (e), e5It is input to feedforward neural network and feature e is calculated6；
S22 (f), it is calculated e with being connected entirely by the moment6Dimension be transformed into dout, the track that is restored.
Have some realization details about step S21 and S22 it may be noted that.
Above-mentioned din=6, dmodel=64, dout=2.
Full connection in S21 (a) and S22 (a) converts not shared parameter, therefore twoNumerical value it is not identical.
The calculating position of step S21 (b) and S22 (b) encodes formula are as follows:
Wherein, P (t, i) is position encoded symbol, and t indicates the moment, and i indicates dimension, and T and d are full connection transformation respectively
The length and dimension of input afterwards.
In step S21 and S22, in coding network and decoding network from attention mechanism module, attention mechanism module,
Exposure mask is from the attention calculation formula in attention mechanism module are as follows:
Wherein Attention (Q, K, V) is the symbol of attention, and Q indicates that index, K indicate key, KTRepresenting matrix turns
It sets, V expression value, dkIt is the dimension of K.It is transformed for connecting entirelyAnd its position encoded P (t, i), in coding network
Pay attention to the input of power module certainly from the exposure mask paid attention in power module and decoding network Solution
The input Q of attention power module in code network is that the exposure mask of decoding network pays attention to the output e of power module certainly4, input K and V is to compile
The characteristic pattern m of code network output.
The input of the exposure mask of step S22 (c) from attention power module is e3If e3Length be T, exposure mask calculating process is as follows:
Wherein QKT(t1,t2) indicate QKTIn (t1,t2) at value, t1And t2All indicate the moment.
In step S21 and S22, the feedforward neural network in coding network and decoding network, network is by two layers of full articulamentum
It constitutes, the activation primitive of first layer is ReLU, and the second layer does not have activation primitive.
In step S21 and S22, from attention mechanism module, exposure mask from attention mechanism module, attention mechanism module and
Feedforward neural network, there is an identical mapping between input and output, i.e., using the input of module and the output of module add up as
The final output of module, identical mapping are specific as follows:
Y=x+f (x)
Wherein, x representation module inputs, the operation of f representation module, and y indicates the module output after identical mapping.
Layer standardization is carried out after identical mapping, specific as follows:
Wherein,Module output after expression layer standardization, α and β be can training parameter, mean (y) indicates to the last of y
One-dimensional to average, std (y) indicates one-dimensional to seek standard deviation to the last of y.
S23, use the inertial signal sequence in training sample set as input, the planar obit simulation coordinate sequence of pairing is as mesh
Mark carries out Training simultaneously to above-mentioned coding network and decoding network, the planar obit simulation coordinate sequence that decoding network is predicted
Column carry out costing bio disturbance with true planar trajectory coordinates sequence, by error back propagation, Optimized Coding Based network and decoding network
Parameter.Loss function formula are as follows:
Wherein, loss is loss function, and y is true planar trajectory coordinates sequence,It is the plane rail predicted by decoding network
Mark coordinate sequence, t indicate the moment, and i indicates dimension, and T is the length of sequence, and d is the dimension of sequence.
S3, the inertia sensing signal sequence to input after filtering denoising, are calculated special using trained coding network
Sign figure, is input to trained decoding network for inertia sensing signal sequence, the planar obit simulation being restored together with characteristic pattern.
As shown in figure 3, the track that the sequence of the invention based on attention mechanism is restored to series model is relatively
True aerial hand-written character planar obit simulation, restores precision and smoothness is all preferable.
In conclusion present invention is mainly used for the inertia sensing signal visualization problem for solving aerial hand-written character, design
One sequence based on attention mechanism extracts feature to series model, using attention mechanism, makes full use of list entries
Global information, using the aerial hand-written character inertia sensing signal sequence and plane trajectory coordinates sequence of pairing have supervision instruct
Practice model, the inertia sensing signal of input is then converted into planar obit simulation, realizes visualization.Compared with prior art, of the invention
Recovery precision and smooth trajectory degree it is more preferable, be worthy to be popularized.
The above, only the invention patent preferred embodiment, but the scope of protection of the patent of the present invention is not limited to
This, anyone skilled in the art is in the range disclosed in the invention patent, according to the present invention the skill of patent
Art scheme and its patent of invention design are subject to equivalent substitution or change, belong to the scope of protection of the patent of the present invention.
Claims (9)
1. a kind of aerial hand-written character track restoration methods based on attention mechanism, which is characterized in that the method includes with
Lower step:
S1, the inertia sensing signal sequence and plane trajectory coordinates sequence for obtaining pairing, are filtered smoothing denoising, and divide instruction
Practice sample set and test sample collection；
S2, design simultaneously train a sequence based on attention mechanism to series model, the specific steps of which are as follows:
One S21, design coding network, network from attention mechanism module and feedforward neural network by constituting；With inertia sensing
Signal sequence is input, and after connect mapping entirely by the moment, calculating position is encoded, then by from attention mechanism module and feedforward
Neural computing simultaneously exports characteristic pattern；
One S22, design decoding network, network is by exposure mask from attention mechanism module, attention mechanism module and feed forward neural
Network is constituted；It is input with inertia sensing signal sequence, after connecting mapping entirely by the moment, calculating position coding, certainly by exposure mask
Attention mechanism module extracts feature, is then input with this feature and by the characteristic pattern of coding network output, by attention machine
Molding block and feedforward neural network extract feature, and feature progress is connected calculating by the moment entirely, the planar obit simulation predicted
Coordinate sequence；
S23, use the inertia sensing signal sequence in training sample set as input, the planar obit simulation coordinate sequence of pairing is as mesh
Mark carries out Training simultaneously to above-mentioned coding network and decoding network, the planar obit simulation coordinate sequence that decoding network is predicted
Column carry out costing bio disturbance with true planar trajectory coordinates sequence, by error back propagation, Optimized Coding Based network and decoding network
Parameter；
S3, the inertia sensing signal sequence to input after filtering denoising, calculate feature using trained coding network
Figure, is input to trained decoding network for inertia sensing signal sequence, the planar obit simulation being restored together with characteristic pattern.
2. a kind of aerial hand-written character track restoration methods based on attention mechanism according to claim 1, feature
It is, data acquisition, pretreatment and the partition process in the step S1, the specific steps are as follows:
S11, the aerial hand-written character inertia sensing signal sequence that pairing is obtained from public data collection 6DMG and planar obit simulation are sat
Sequence is marked, 6 dimension inertia sensing signals include 3 dimension acceleration signals and 3 dimension angular velocity signals, and 2 dimensional plane trajectory coordinates include x-axis
And y-axis coordinate；The sample of so-called pairing refers to the inertia sensing signal in the same aerial hand-written character sample of data set 6DMG
Sequence and plane trajectory coordinates sequence；
The sliding average filter of S12, the length of window that fixes to the inertia sensing signal sequence and plane trajectory coordinates sequence of acquisition
Wave denoising；
S13, using the inertia sensing signal sequence of pretreated pairing and plane trajectory coordinates sequence as a sample, inertia
Transducing signal sequence is sample input, and planar obit simulation coordinate sequence is target；Sample is divided into training sample set and test specimens
This collection.
3. a kind of aerial hand-written character track restoration methods based on attention mechanism according to claim 1, feature
It is, if input inertia sensing signal sequence is x (t, i), dimension din, the coding network of the step S21, concrete operation
Steps are as follows:
S21 (a), with connected entirely by the moment calculate will input x (t, i) be transformed intoDimension is by dinBecome dmodel；
S21 (b), the position encoded P for calculating transformed input1(t, i), and be superimposed are as follows:
S21 (c), e1It is input to from attention power module and feature e is calculated2；
S21 (d), e2It is input to feedforward neural network and characteristic pattern m is calculated；
The decoding network of the step S22, steps are as follows for concrete operation:
S22 (a), with connected entirely by the moment calculate will input x (t, i) be transformed intoDimension is by dinBecome dmodel；
S22 (b), the position encoded P for calculating transformed input2(t, i), and be superimposed are as follows:
S22 (c), e3It is input to exposure mask and obtains feature e from attention power module4；
S22 (d), e4It is input to the characteristic pattern m of coding network output and notices that feature e is calculated in power module5；
S22 (e), e5It is input to feedforward neural network and feature e is calculated6；
S22 (f), it is calculated e with being connected entirely by the moment6Dimension be transformed into dout, the planar obit simulation that is restored；
Wherein, din=6, dmodel=64, dout=2.
4. a kind of aerial hand-written character track restoration methods based on attention mechanism according to claim 3, feature
Be: in the step S21 and S22, in coding network and decoding network from attention mechanism module, attention mechanism mould
Block, exposure mask are calculated from the attention in attention mechanism module, formula are as follows:
Wherein Attention (Q, K, V) is the symbol of attention, and Q indicates that index, K indicate key, KTThe transposition of representing matrix K, V table
Indicating value, dkIt is the dimension of K；For connecting transformed input entirelyAnd its position encoded P (t, i), in coding network from
Pay attention to power module and decodes input of the exposure mask in network from attention power module Decode net
The input Q of attention power module in network is that the exposure mask of decoding network pays attention to the output e of power module certainly4, input K and V is coding net
The characteristic pattern m of network output.
5. a kind of aerial hand-written character track restoration methods based on attention mechanism according to claim 3, feature
It is, in the step S21 (b) and S22 (b), is encoded to transformed input calculating position is connected entirely, position encoded formula
Are as follows:
Wherein, P (t, i) is position encoded symbol, and t indicates the moment, and i indicates dimension, T and d be respectively connect entirely it is transformed
The length and dimension of input.
6. a kind of aerial hand-written character track restoration methods based on attention mechanism according to claim 3, feature
It is, the input of exposure mask from attention power module is e in the step S22 (c)3If e3Length be T, exposure mask calculating process is such as
Under:
Wherein QKT(t1,t2) indicate QKTIn (t1,t2) at value, t1And t2All indicate the moment.
7. a kind of aerial hand-written character track restoration methods based on attention mechanism according to claim 1, feature
Be: in the step S21 and S22, the feedforward neural network in coding network and decoding network, network is by two layers of full articulamentum
It constitutes, the activation primitive of first layer is ReLU, and the second layer does not have activation primitive.
8. a kind of aerial hand-written character track restoration methods based on attention mechanism according to claim 1, feature
Be: in the step S21 and S22, coding network and decoding network from attention mechanism module, exposure mask from attention mechanism
Module, attention mechanism module and feedforward neural network, there is an identical mapping between input and output, i.e., the input of module and
The output of module adds up the output final as module, and identical mapping is specific as follows:
Y=x+f (x)
Wherein, x representation module inputs, the operation of f representation module, and y indicates the module output after identical mapping；
Layer standardization is carried out after identical mapping, specific as follows:
Wherein,Module output after expression layer standardization, α and β be can training parameter, mean (y) indicates to the last one-dimensional of y
It averages, std (y) indicates one-dimensional to seek standard deviation to the last of y.
9. a kind of aerial hand-written character track restoration methods based on attention mechanism according to claim 1, feature
It is, the training of the step S23 sequence to series model, loss function formula are as follows:
Wherein, loss is loss function, and y is true planar trajectory coordinates sequence,It is to be sat by the planar obit simulation of decoding network prediction
Sequence is marked, t indicates the moment, and i indicates dimension, and T is the length of sequence, and d is the dimension of sequence.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910454906.3A CN110188669B (en) | 2019-05-29 | 2019-05-29 | Air handwritten character track recovery method based on attention mechanism |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910454906.3A CN110188669B (en) | 2019-05-29 | 2019-05-29 | Air handwritten character track recovery method based on attention mechanism |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN110188669A true CN110188669A (en) | 2019-08-30 |
| CN110188669B CN110188669B (en) | 2021-01-19 |
Family
ID=67718404
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910454906.3A Active CN110188669B (en) | 2019-05-29 | 2019-05-29 | Air handwritten character track recovery method based on attention mechanism |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110188669B (en) |
Cited By (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111046771A (en) * | 2019-12-05 | 2020-04-21 | 上海眼控科技股份有限公司 | Training method of network model for recovering writing track |
| CN112016245A (en) * | 2020-08-13 | 2020-12-01 | 五邑大学 | Magnetic storm prediction method and device based on self-attention deformation network and storage medium |
| CN112132050A (en) * | 2020-09-24 | 2020-12-25 | 北京计算机技术及应用研究所 | On-line handwritten Chinese character recognition algorithm and visual key stroke evaluation method |
| CN114120337A (en) * | 2021-10-15 | 2022-03-01 | 华南理工大学 | Real-time paper handwriting recovery method, system, equipment and storage medium |
| CN114463760A (en) * | 2022-04-08 | 2022-05-10 | 华南理工大学 | Character image writing track recovery method based on double-stream coding |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107316067A (en) * | 2017-05-27 | 2017-11-03 | 华南理工大学 | A kind of aerial hand-written character recognition method based on inertial sensor |
| CN108805087A (en) * | 2018-06-14 | 2018-11-13 | 南京云思创智信息科技有限公司 | Semantic temporal fusion association based on multi-modal Emotion identification system judges subsystem |
| CN108805089A (en) * | 2018-06-14 | 2018-11-13 | 南京云思创智信息科技有限公司 | Based on multi-modal Emotion identification method |
| US20180329884A1 (en) * | 2017-05-12 | 2018-11-15 | Rsvp Technologies Inc. | Neural contextual conversation learning |
| CN108935188A (en) * | 2018-07-05 | 2018-12-07 | 平安科技（深圳）有限公司 | Pig disease identification method, apparatus and electronic equipment |
| CN109492679A (en) * | 2018-10-24 | 2019-03-19 | 杭州电子科技大学 | Based on attention mechanism and the character recognition method for being coupled chronological classification loss |
-
2019
- 2019-05-29 CN CN201910454906.3A patent/CN110188669B/en active Active
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20180329884A1 (en) * | 2017-05-12 | 2018-11-15 | Rsvp Technologies Inc. | Neural contextual conversation learning |
| CN107316067A (en) * | 2017-05-27 | 2017-11-03 | 华南理工大学 | A kind of aerial hand-written character recognition method based on inertial sensor |
| CN108805087A (en) * | 2018-06-14 | 2018-11-13 | 南京云思创智信息科技有限公司 | Semantic temporal fusion association based on multi-modal Emotion identification system judges subsystem |
| CN108805089A (en) * | 2018-06-14 | 2018-11-13 | 南京云思创智信息科技有限公司 | Based on multi-modal Emotion identification method |
| CN108935188A (en) * | 2018-07-05 | 2018-12-07 | 平安科技（深圳）有限公司 | Pig disease identification method, apparatus and electronic equipment |
| CN109492679A (en) * | 2018-10-24 | 2019-03-19 | 杭州电子科技大学 | Based on attention mechanism and the character recognition method for being coupled chronological classification loss |
Non-Patent Citations (1)
| Title |
|---|
| ASHISH VASWANI,ETC: ""Attention Is All You Need"", 《ARXIV》 * |
Cited By (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111046771A (en) * | 2019-12-05 | 2020-04-21 | 上海眼控科技股份有限公司 | Training method of network model for recovering writing track |
| CN112016245A (en) * | 2020-08-13 | 2020-12-01 | 五邑大学 | Magnetic storm prediction method and device based on self-attention deformation network and storage medium |
| CN112132050A (en) * | 2020-09-24 | 2020-12-25 | 北京计算机技术及应用研究所 | On-line handwritten Chinese character recognition algorithm and visual key stroke evaluation method |
| CN112132050B (en) * | 2020-09-24 | 2024-03-29 | 北京计算机技术及应用研究所 | On-line handwritten Chinese character recognition algorithm and visual key stroke evaluation method |
| CN114120337A (en) * | 2021-10-15 | 2022-03-01 | 华南理工大学 | Real-time paper handwriting recovery method, system, equipment and storage medium |
| CN114120337B (en) * | 2021-10-15 | 2024-04-09 | 华南理工大学 | Method, system, device and storage medium for recovering paper writing in real time |
| CN114463760A (en) * | 2022-04-08 | 2022-05-10 | 华南理工大学 | Character image writing track recovery method based on double-stream coding |
| CN114463760B (en) * | 2022-04-08 | 2022-06-28 | 华南理工大学 | Character image writing track recovery method based on double-stream coding |
Also Published As
| Publication number | Publication date |
|---|---|
| CN110188669B (en) | 2021-01-19 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110188669A (en) | A kind of aerial hand-written character track restoration methods based on attention mechanism | |
| CN106658023B (en) | A kind of end-to-end visual odometry and method based on deep learning | |
| CN103544483B (en) | A kind of joint objective method for tracing based on local rarefaction representation and system thereof | |
| CN110197224B (en) | Method for recovering handwritten character track in air based on feature space depth counterstudy | |
| CN107679462A (en) | A kind of depth multiple features fusion sorting technique based on small echo | |
| Geetha et al. | A vision based dynamic gesture recognition of indian sign language on kinect based depth images | |
| CN106056628A (en) | Target tracking method and system based on deep convolution nerve network feature fusion | |
| CN104615983A (en) | Behavior identification method based on recurrent neural network and human skeleton movement sequences | |
| CN106650687A (en) | Posture correction method based on depth information and skeleton information | |
| CN103003846B (en) | Articulation region display device, joint area detecting device, joint area degree of membership calculation element, pass nodular region affiliation degree calculation element and joint area display packing | |
| CN107292912A (en) | A kind of light stream method of estimation practised based on multiple dimensioned counter structure chemistry | |
| CN103984416A (en) | Gesture recognition method based on acceleration sensor | |
| CN110163108A (en) | Robust sonar target detection method based on dual path Fusion Features network | |
| CN103824049A (en) | Cascaded neural network-based face key point detection method | |
| CN102937848A (en) | Method for editing handwritings and electronic equipment | |
| CN107885327A (en) | A kind of Fingertip Detection based on Kinect depth information | |
| CN108919943A (en) | A kind of real-time hand method for tracing based on depth transducer | |
| CN107146237A (en) | A kind of method for tracking target learnt based on presence with estimating | |
| CN108154066B (en) | Three-dimensional target identification method based on curvature characteristic recurrent neural network | |
| CN111383735A (en) | Unmanned body-building analysis method based on artificial intelligence | |
| CN110189362A (en) | The efficient method for tracking target of network is fought from coding based on multiple-limb | |
| CN107292295B (en) | Gesture segmentation method and device | |
| Xu et al. | A long term memory recognition framework on multi-complexity motion gestures | |
| Juan | Gesture recognition and information recommendation based on machine learning and virtual reality in distance education | |
| CN103839047B (en) | A kind of recognition methods of human body motion track and device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |