CN111354458A - Touch interactive motion user feature extraction method based on general drawing task and auxiliary disease detection system - Google Patents
Touch interactive motion user feature extraction method based on general drawing task and auxiliary disease detection system Download PDFInfo
- Publication number
- CN111354458A CN111354458A CN201811591897.4A CN201811591897A CN111354458A CN 111354458 A CN111354458 A CN 111354458A CN 201811591897 A CN201811591897 A CN 201811591897A CN 111354458 A CN111354458 A CN 111354458A
- Authority
- CN
- China
- Prior art keywords
- over
- track
- points
- task
- parameters
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000000605 extraction Methods 0.000 title claims abstract description 34
- 201000010099 disease Diseases 0.000 title claims abstract description 30
- 208000037265 diseases, disorders, signs and symptoms Diseases 0.000 title claims abstract description 30
- 230000002452 interceptive effect Effects 0.000 title claims abstract description 14
- 238000001514 detection method Methods 0.000 title claims abstract description 10
- 238000003745 diagnosis Methods 0.000 claims abstract description 37
- 238000013145 classification model Methods 0.000 claims abstract description 12
- 238000000034 method Methods 0.000 claims description 38
- 238000005520 cutting process Methods 0.000 claims description 22
- 230000011218 segmentation Effects 0.000 claims description 22
- 238000004422 calculation algorithm Methods 0.000 claims description 21
- 230000006399 behavior Effects 0.000 claims description 14
- 238000005070 sampling Methods 0.000 claims description 14
- 238000007619 statistical method Methods 0.000 claims description 12
- 238000003672 processing method Methods 0.000 claims description 8
- 238000012549 training Methods 0.000 claims description 7
- 238000004364 calculation method Methods 0.000 claims description 5
- 230000036544 posture Effects 0.000 claims description 5
- 239000003205 fragrance Substances 0.000 claims description 4
- 238000012952 Resampling Methods 0.000 claims description 3
- 238000012216 screening Methods 0.000 claims description 2
- 230000003542 behavioural effect Effects 0.000 claims 1
- 208000012902 Nervous system disease Diseases 0.000 abstract description 20
- 230000003993 interaction Effects 0.000 abstract description 5
- 238000005516 engineering process Methods 0.000 abstract description 4
- 238000012544 monitoring process Methods 0.000 abstract description 4
- 241001422033 Thestylus Species 0.000 abstract 1
- 238000012360 testing method Methods 0.000 description 21
- 238000012706 support-vector machine Methods 0.000 description 8
- 230000006870 function Effects 0.000 description 6
- 208000018737 Parkinson disease Diseases 0.000 description 4
- 238000004458 analytical method Methods 0.000 description 4
- 238000010586 diagram Methods 0.000 description 4
- 238000012545 processing Methods 0.000 description 4
- 238000013507 mapping Methods 0.000 description 3
- 230000008569 process Effects 0.000 description 3
- 208000000044 Amnesia Diseases 0.000 description 2
- 208000026139 Memory disease Diseases 0.000 description 2
- 206010044565 Tremor Diseases 0.000 description 2
- 230000001133 acceleration Effects 0.000 description 2
- 208000010877 cognitive disease Diseases 0.000 description 2
- 230000003247 decreasing effect Effects 0.000 description 2
- 208000019622 heart disease Diseases 0.000 description 2
- 238000010801 machine learning Methods 0.000 description 2
- 230000006984 memory degeneration Effects 0.000 description 2
- 208000023060 memory loss Diseases 0.000 description 2
- 210000005036 nerve Anatomy 0.000 description 2
- 201000001119 neuropathy Diseases 0.000 description 2
- 230000007823 neuropathy Effects 0.000 description 2
- 208000033808 peripheral neuropathy Diseases 0.000 description 2
- 230000003068 static effect Effects 0.000 description 2
- 208000024891 symptom Diseases 0.000 description 2
- 239000013598 vector Substances 0.000 description 2
- 208000024827 Alzheimer disease Diseases 0.000 description 1
- 208000035143 Bacterial infection Diseases 0.000 description 1
- 208000012639 Balance disease Diseases 0.000 description 1
- 206010006100 Bradykinesia Diseases 0.000 description 1
- 208000029812 Cerebral Small Vessel disease Diseases 0.000 description 1
- 208000012661 Dyskinesia Diseases 0.000 description 1
- 208000006083 Hypokinesia Diseases 0.000 description 1
- 238000007476 Maximum Likelihood Methods 0.000 description 1
- 206010061296 Motor dysfunction Diseases 0.000 description 1
- 208000002740 Muscle Rigidity Diseases 0.000 description 1
- 208000025966 Neurological disease Diseases 0.000 description 1
- 108010076504 Protein Sorting Signals Proteins 0.000 description 1
- 206010071390 Resting tremor Diseases 0.000 description 1
- 230000002159 abnormal effect Effects 0.000 description 1
- 230000032683 aging Effects 0.000 description 1
- 210000003403 autonomic nervous system Anatomy 0.000 description 1
- 208000022362 bacterial infectious disease Diseases 0.000 description 1
- 210000003169 central nervous system Anatomy 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 238000004140 cleaning Methods 0.000 description 1
- 230000019771 cognition Effects 0.000 description 1
- 230000001149 cognitive effect Effects 0.000 description 1
- 230000003920 cognitive function Effects 0.000 description 1
- 238000013523 data management Methods 0.000 description 1
- 238000013500 data storage Methods 0.000 description 1
- 238000003066 decision tree Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 230000018109 developmental process Effects 0.000 description 1
- 230000004064 dysfunction Effects 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 238000011156 evaluation Methods 0.000 description 1
- 238000001914 filtration Methods 0.000 description 1
- 238000009499 grossing Methods 0.000 description 1
- 230000036541 health Effects 0.000 description 1
- 230000001771 impaired effect Effects 0.000 description 1
- 230000010365 information processing Effects 0.000 description 1
- 238000002372 labelling Methods 0.000 description 1
- 239000004973 liquid crystal related substance Substances 0.000 description 1
- 230000007774 longterm Effects 0.000 description 1
- 238000007726 management method Methods 0.000 description 1
- 230000007659 motor function Effects 0.000 description 1
- 210000000653 nervous system Anatomy 0.000 description 1
- 230000001537 neural effect Effects 0.000 description 1
- 238000010606 normalization Methods 0.000 description 1
- 230000008506 pathogenesis Effects 0.000 description 1
- 230000001575 pathological effect Effects 0.000 description 1
- 210000001428 peripheral nervous system Anatomy 0.000 description 1
- 238000007781 pre-processing Methods 0.000 description 1
- 238000010187 selection method Methods 0.000 description 1
- 238000004513 sizing Methods 0.000 description 1
- 238000000638 solvent extraction Methods 0.000 description 1
- 238000003860 storage Methods 0.000 description 1
- 239000000126 substance Substances 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G16—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR SPECIFIC APPLICATION FIELDS
- G16H—HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA
- G16H50/00—ICT specially adapted for medical diagnosis, medical simulation or medical data mining; ICT specially adapted for detecting, monitoring or modelling epidemics or pandemics
- G16H50/20—ICT specially adapted for medical diagnosis, medical simulation or medical data mining; ICT specially adapted for detecting, monitoring or modelling epidemics or pandemics for computer-aided diagnosis, e.g. based on medical expert systems
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B5/00—Measuring for diagnostic purposes; Identification of persons
- A61B5/40—Detecting, measuring or recording for evaluating the nervous system
- A61B5/4076—Diagnosing or monitoring particular conditions of the nervous system
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/03—Arrangements for converting the position or the displacement of a member into a coded form
- G06F3/033—Pointing devices displaced or positioned by the user, e.g. mice, trackballs, pens or joysticks; Accessories therefor
- G06F3/0354—Pointing devices displaced or positioned by the user, e.g. mice, trackballs, pens or joysticks; Accessories therefor with detection of 2D relative movements between the device, or an operating part thereof, and a plane or surface, e.g. 2D mice, trackballs, pens or pucks
- G06F3/03545—Pens or stylus
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/048—Interaction techniques based on graphical user interfaces [GUI]
- G06F3/0487—Interaction techniques based on graphical user interfaces [GUI] using specific features provided by the input device, e.g. functions controlled by the rotation of a mouse with dual sensing arrangements, or of the nature of the input device, e.g. tap gestures based on pressure sensed by a digitiser
- G06F3/0488—Interaction techniques based on graphical user interfaces [GUI] using specific features provided by the input device, e.g. functions controlled by the rotation of a mouse with dual sensing arrangements, or of the nature of the input device, e.g. tap gestures based on pressure sensed by a digitiser using a touch-screen or digitiser, e.g. input of commands through traced gestures
-
- G—PHYSICS
- G16—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR SPECIFIC APPLICATION FIELDS
- G16H—HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA
- G16H40/00—ICT specially adapted for the management or administration of healthcare resources or facilities; ICT specially adapted for the management or operation of medical equipment or devices
- G16H40/60—ICT specially adapted for the management or administration of healthcare resources or facilities; ICT specially adapted for the management or operation of medical equipment or devices for the operation of medical equipment or devices
- G16H40/63—ICT specially adapted for the management or administration of healthcare resources or facilities; ICT specially adapted for the management or operation of medical equipment or devices for the operation of medical equipment or devices for local operation
Landscapes
- Engineering & Computer Science (AREA)
- Health & Medical Sciences (AREA)
- Biomedical Technology (AREA)
- General Engineering & Computer Science (AREA)
- Medical Informatics (AREA)
- Theoretical Computer Science (AREA)
- Public Health (AREA)
- Life Sciences & Earth Sciences (AREA)
- Physics & Mathematics (AREA)
- General Health & Medical Sciences (AREA)
- Primary Health Care (AREA)
- Epidemiology (AREA)
- Pathology (AREA)
- Neurology (AREA)
- General Physics & Mathematics (AREA)
- Human Computer Interaction (AREA)
- Heart & Thoracic Surgery (AREA)
- Surgery (AREA)
- Animal Behavior & Ethology (AREA)
- Molecular Biology (AREA)
- Biophysics (AREA)
- Veterinary Medicine (AREA)
- Business, Economics & Management (AREA)
- General Business, Economics & Management (AREA)
- Physiology (AREA)
- Neurosurgery (AREA)
- Data Mining & Analysis (AREA)
- Databases & Information Systems (AREA)
- Measurement Of The Respiration, Hearing Ability, Form, And Blood Characteristics Of Living Organisms (AREA)
Abstract
The invention belongs to the field of digital medical treatment, and particularly relates to a touch interactive motion user feature extraction method and an auxiliary disease detection system based on a general drawing task. The user characteristics which are irrelevant to the task are extracted by using the stylus and the drawing board, the hand motion function of a patient with the nervous system disease is evaluated, a general drawing task-based nervous system disease daily monitoring and early warning system is realized, a classification model of the system is analyzed and verified, and the result shows that the system can accurately predict the nervous system disease. The invention discloses a system for automatically identifying nervous system diseases in a general drawing task by using a pen interaction technology, an adopted feature extraction method can be used for constructing a robust and accurate identification model, and the drawing system constructed according to the system can be used for automatic diagnosis in the field including but not limited to nervous system disease detection under the condition of no supervision and no task relevance.
Description
Technical Field
The invention belongs to the field of digital medical treatment, and particularly relates to a diagnostic interactive software system for nervous system diseases, which utilizes a touch interactive motion user characteristic extraction method irrelevant to a specific drawing task and utilizes a touch pen, a drawing screen and a common desktop computer as input equipment.
Background
The nervous system diseases are caused by various causes such as bacterial infection or heredity in the central nervous system, peripheral nervous system, and autonomic nervous system, and the symptoms thereof are usually manifested as various problems such as dyskinesia and cognitive dysfunction, for example, decreased motor ability, decreased language ability, impaired memory, and memory loss. Common nervous system diseases such as Parkinson's disease, Alzheimer's disease and cerebral small vessel disease are clinically manifested as motor dysfunction such as resting tremor, muscular rigidity and balance disorder, and cognitive dysfunction such as memory loss, language and directional dysfunction.
Due to the change of environment and the aging of population, the number of patients suffering from nervous system diseases is increasing, and the diseases largely affect the quality of life of the patients and their families. Due to the complex pathogenesis, complex examination steps and high medical cost of the diseases, an economical and effective daily monitoring scheme is not obtained yet. With the continuous development of computer science and technology and the gradual deepening of human cognition on the relationship between physiological information and diseases, the method enables the physiological information to be obtained and analyzed through a computer, and the detection of the functional diseases of the nervous system becomes feasible.
The pen interaction is used as a mature touch interaction technology and has the characteristics of high interaction efficiency, low cognitive load and the like. The pen-moving state and the graph drawing result can effectively reflect the abnormal information of the nervous system disease patients on the motor and cognitive functions. For example, Atilla et al extract multiple features from handwriting, pressure and inclination of digital pen, and combine them to model train for diagnosing Parkinson's disease (reference: Atilla)
R ü diger Brause, Karsten Krakow.2006.hand writing Analysis for diagnosing and diagnosing Parkinson's disease. in Proceedings of the 7th international relationship on the parkinsons disease using digital pen-extracted information, finding features for quantitative assessment of bradykinesia, font undersize and hand tremor symptoms (reference: heat J Smits, Antti Tonen, Luc clinical, AIkvarls, Bernard A Constrain, Rutger C sizing, U.S. Pat. No. 5. see the relevant neural system of the diagnosis of neuropathy, U.S. Pat. No. 5. the important parameters of the diagnosis of neuropathy, I.S. Pat. No. 5. 7. the important parameters of the diagnosis of the disease, I.S. Pat. No. 5. 7. the important parameters of the diagnosis of the nerve system of the heart, I.S. 5. and 7. the important parameters of the diagnosis of the disease using digital pen-reading test system of the heart, I.S. 5. the diagnosis of the disease using the heart test of the heart test system of the heart and the heart test of the heart disease (reference: the important parameters of the diagnosis of the nerve of the disease, I.S. 7. the diagnosis of the heart, the diagnosis of the disease using heart test of the heart test system of the heart, the diagnosis of the heart, the heart test of the diagnosis of the heart, the diagnosis of the disease using heart, the heart test of the heart disease using the heart test system of the heart, the heart test of the heart, the heart test of the diagnosis of the heart of the diagnosis of the heart of the diagnosis of the heart of the diagnosis of the heart of the disease of the diagnosis of the heart of the diagnosis of the heart of the diagnosis of the heart of the diagnosis of the disease of the heart of the diagnosis of the disease of the diagnosis of the: stuart Hagler, Holly Jimison, Misha Pavel.2014. insulating functional a computer gate. effective journal of biological and Health information, 18(4),1442 and 1452.) the diagnosis effect of the method on different nervous system diseases was evaluated.
However, the above detection methods are specific tasks, the feature extraction method must be modified according to the designated mapping task, and the execution of the above tests must be performed under the supervision of medical staff, and the long-term real-time monitoring of patients in daily life cannot be realized.
Disclosure of Invention
The invention aims to provide a touch interactive motion user feature extraction method and an auxiliary disease detection system based on a general drawing task.
The invention considers the user characteristics irrelevant to the task to evaluate the hand motion function of the patient with nervous system disease, and comprises data acquisition, track segmentation, parameter calculation, characteristic extraction and classification diagnosis.
The technical scheme adopted by the invention is as follows:
a touch interactive motion user feature extraction method based on a general drawing task comprises the following steps:
1) collecting drawing tracks of different drawing tasks of a user;
2) carrying out track segmentation on the collected drawing tracks to form a task-independent sub-track set;
3) calculating motion parameters of all points of any section of sub-track in the sub-track set;
4) track-level features are extracted from the motion parameters and statistical analysis is performed on all sub-tracks to extract graph-level features.
Further, step 1) collecting a drawing track of a user by using a touch pen and a drawing screen; the touch control pen is provided with a pressure sensor and a gyroscope and can acquire X and Y direction positions, touch control point pressure and touch control equipment space postures, and the touch control equipment space postures comprise azimuth angles, elevation angles and self-rotation angles.
Further, the drawing trace of the user collected in step 1) includes:
1.1) time equidistant trajectory, requiring the time interval between two sampling points to be consistent, for calculating time-domain correlation features
1.2) space equidistant trajectory, the space distance between two sampling points is required to be consistent, and the spatial equidistant trajectory is used for calculating the position correlation characteristics.
Further, the track segmentation of step 2) includes:
2.1) segmenting long and complex tracks by using local geometric extreme values;
and 2.2) clustering the dense over-segmentation points by utilizing time and space information, and selecting the optimal segmentation point as a final segmentation point.
Further, step 2.1) performing SER resampling on the track to obtain curvatures of all points and putting the curvatures into a linked list, then obtaining a local extreme value of the curvatures from the linked list by using a Gaussian filter, and screening out the over-cut points through a curvature threshold value and putting the over-cut points into an over-cut point list; and 2.2) traversing all the over-cutting points, recording other over-cutting points in a space-time domain range R of all the over-cutting points in a data structure named nearIndexes, then sorting the over-cutting points in a descending order according to the quantity of the nearIndexes of the over-cutting points, finally traversing the sorted over-cutting points, removing other over-cutting points recorded in the nearIndexes of each over-cutting point from a list of the over-cutting points, and taking the remaining list of the over-cutting points as the final over-cutting points.
Further, the motion parameters in the step 3) include behavior parameters and kinematic parameters; the behavior parameters are parameters directly reflecting the drawing behaviors and decisions of the user; the kinematic parameters are first and second order kinematic parameters obtained by deriving a part of the behavior parameters with respect to time.
Further, extracting track level characteristics by adopting a statistical method, a signal processing method and an information entropy method in the step 4); the statistical method comprises summation, mean, maximum value, minimum value, quartile and standard deviation; the signal processing method comprises a time domain method and a frequency domain method, wherein the time domain method utilizes zero crossing times, zero crossing frequency and root mean square to extract features, and the frequency domain method utilizes dominant frequency, dominant frequency energy and dominant frequency energy ratio to extract features; the information entropy method utilizes the fragrance concentration entropy and the Rayleigh entropy to extract features.
Further, the extracting of the graphics-level features in step 4) is to extract two types of graphics-level features of the whole drawing task: statistical attributes of all plot motion parameters, and statistical attributes of all sub-trajectory features.
A touch interactive motion user feature extraction device based on a general drawing task comprises:
the data acquisition module is responsible for acquiring drawing tracks of different drawing tasks of a user;
the track segmentation module is responsible for carrying out track segmentation on the collected drawing tracks to form a task-independent sub-track set;
the parameter calculation module is responsible for calculating motion parameters of all points of any section of sub-track in the sub-track set;
and the characteristic extraction module is responsible for extracting the track level characteristics from the motion parameters and performing statistical analysis on all the sub-tracks to extract the graph level characteristics.
A general drawing task-based auxiliary disease detection system comprises a general drawing task-based touch interactive motion user feature extraction device, a classification model training module and a disease diagnosis module; the classification model training module screens the characteristics related to the diseases from the characteristics extracted by the characteristic extraction module by using a characteristic selection algorithm to train a classification model; and the disease diagnosis module performs automatic diagnosis of diseases by using the trained classification model.
According to the invention, the touch pen and the drawing board are used for extracting the user characteristics irrelevant to the task, the hand motion function of the patient with the nervous system disease is evaluated, a general drawing task-based nervous system disease daily monitoring and early warning system is realized, the classification model of the system is analyzed and verified, the result shows that the scheme can accurately predict the nervous system disease, and a plurality of results capable of providing inspiration for the diagnosis of the nervous system disease are found.
The system constructed by the invention is the first system for automatically identifying nervous system diseases in a general drawing task by utilizing a pen interaction technology. The given feature extraction method can be used for constructing a robust and accurate recognition model, and various application programs can be constructed according to the method. The mapping system constructed in accordance with the present system can be used in unsupervised, task-independent situations for automated diagnosis including, but not limited to, the field of neurological disease detection.
Drawings
FIG. 1 is a system framework diagram of the present invention.
FIG. 2 is a diagram of hardware used in the present invention.
Fig. 3 is a flowchart of the algorithm workflow (universal trajectory segmentation algorithm workflow).
FIG. 4 is a diagram illustrating a graph level feature.
FIG. 5 is a flowchart of the steps for extracting user features from a drawing task.
Detailed Description
In order to make the aforementioned objects, features and advantages of the present invention comprehensible, the present invention shall be described in further detail with reference to the following detailed description and accompanying drawings.
The system framework of the invention is shown in fig. 1, and the hardware schematic diagram used is shown in fig. 2. And obtaining a feature set of the user irrelevant to a specific drawing task by using the user drawing information collected by the hardware equipment and using a universal track segmentation algorithm, and automatically performing classification diagnosis on diseases by using a machine learning algorithm. The details of the system involved are as follows:
1. data acquisition: the interactive software system for diagnosing the nervous system diseases integrates different drawing tasks, and acquires multi-dimensional drawing and motion information when a tested person performs a test by using a stylus (a digital pen in figure 1) and a drawing screen (a liquid crystal digital screen in figure 1) as input devices. The system is also responsible for the storage, management and preprocessing of these data.
Specifically, the user drawing trace data is acquired based on a stylus equipped with a pressure sensor and a gyroscope. Besides the X, Y directional position that can be obtained by a general touch device, it can also provide touch point pressure (pressure), and spatial attitude of the touch device, including azimuth (azimuth), elevation (elevation), and rotation angle (rotation).
The user drawing trace used includes the following two:
1) time equidistant trace, requiring the time interval between two sampling points to be consistent, for calculating time-domain correlation characteristics
2) And the spatial equidistant trajectory requires that the spatial distance between two sampling points is consistent for calculating the position correlation characteristics.
2. Track segmentation: and carrying out unified track segmentation on data obtained from different drawing tasks by using a universal track segmentation algorithm to form a task-independent sub-track set.
The general track segmentation algorithm: the algorithm mainly comprises two steps: 1) and segmenting the long and complex track by utilizing the local geometric extreme value, and intentionally not smoothing the track. 2) And clustering the dense over-segmentation points by using time and space information, and selecting the optimal segmentation point as a final segmentation point. The following describes the track segmentation algorithm employed in the present invention.
Firstly, an algorithm carries out SER resampling on a track, on the basis, curvatures of all points are obtained and are put into a linked list points _ cur, then, a Gaussian filter is used for obtaining a local extreme value of the curvatures from the points _ cur, and an over-cut point is screened out through a curvature threshold value A and is put into an over-cut point list.
The local extremum is calculated in the same way as for UMC. The related introduction to the UMC method is as follows:
in the track task, any attribute can be regarded as a signal s (n), and it is assumed that the signal includes two components: intentional Motion Component (IMC) i (n) and Unintentional Motion Component (UMC) u (n):
s(n)＝i(n)+u(n)
to obtain the involuntary moving parts, the above equation is transformed:
u(n)＝s(n)-i(n)
where s (n) can be obtained directly from the touch device, and i (n) is unknown. Estimating i (n) using a gaussian filter:
wherein the content of the first and second substances,
representing the estimated IMC at time n, s (n + k) is the signal obtained directly from the device, offset by k from time n, w is the length of half the processing time window, and σ represents the standard deviation in a gaussian filter. Because the original signal is smoothed by the Gaussian filter, the unintentional motion of the user is excluded
Furthermore, the UMC can thus be calculated by subtracting the IMC from the original signal:
secondly, traversing all the over-segmentation points by the algorithm, and recording other over-segmentation points in a space-time domain range R of all the over-segmentation points in a data structure named as nearIndexes of the over-segmentation points;
thirdly, sorting the over-cut points in a descending order according to the quantity of the nearIndexes of the over-cut points by an algorithm;
and finally, traversing the sorted over-cut points, and removing other over-cut points recorded in the nearIndexes of each over-cut point from the over-cut point list. Thus, the remaining list of over-cut points is the final cut point.
The workflow of the algorithm is shown in fig. 3. Wherein (a) the graph is an original stroke, and (b) the graph is an over-segmentation point screened by a UMC local curvature extreme value. The overcut points 1 and 7 are generated directly from the start and end points of the trajectory, and with UMC, some inappropriately large curvature points have been removed, but are still too dense at the several cut points 2, 3, 4, 5. The algorithm then calculates all of these overcut points' neighbors within the domain R (only the spatial neighborhood is considered in the illustration for simplicity), the neighborhood of overcut point 3 containing 2, 4, 5, which is the point with the most neighboring overcut points of all overcut points, and thus its neighborhood list containing 2, 4, 5 is removed, resulting in the final cutpoints 1, 3, 6, 7 as labeled in fig. 3 (c). The numbers 1-7 in the figure represent the cut points of the track, and the dotted circle in the figure represents the neighborhood range of the over-cut point.
3. And (3) parameter calculation: on the segmented track, the motion parameters of each point are calculated by taking track points as units, so that the user characteristics are extracted from the motion parameters by using a method of statistics, signal processing and information entropy in a later step 4. The motion parameters are mainly classified into behavior parameters and kinematics parameters. The behavior parameters are parameters which directly reflect the drawing behaviors and decisions of the user; kinematic parameters refer to first and second order motion parameters obtained by deriving a portion of the behavior parameters with respect to time.
4. Feature extraction: and extracting the drawing features from all the motion parameters by using a drawing feature extraction method irrelevant to the task, wherein the feature extraction comprises a feature extraction method and a feature extraction strategy.
1) The feature extraction method comprises the following steps:
the feature extraction method used in the algorithm comprises a statistical method, a signal processing method and an information entropy method.
a) And (5) a statistical method. Including Sum (Sum), Mean (Mean), maximum (Max), minimum (Min), quartile (Q1, Q2, and Q3), standard deviation STD.
b) Provided is a signal processing method. Including time domain methods and frequency domain methods. In the time domain, three signal processing methods are utilized: number of zero crossing times (NTZ), zero crossing frequency (RTZ), Root Mean Square (RMS). The three processing methods are explained as follows:
NTZ: for a signal which can take a positive value or a negative value, within a period of sampling time, if a value obtained by the t-th sampling is opposite to that obtained by the next sampling, the signal is regarded as a zero crossing point, namely, the NTZ value is increased by 1.
RTZ: the inverse of the value of NTZ is the zero-crossing frequency.
RMS: the calculation formula is as follows:
where N is the length of the signal, siIs the ith signal value in the signal sequence. Generally, RMS reflects the average intensity of a signal, which is related to the amplitude of the signal.
In the frequency domain, a Main Frequency (MF), a main frequency Energy (EMF), and a main frequency Energy Ratio (ER) are utilized. The computation of these three frequency domain features relies on a Fast Fourier Transform (FFT) algorithm. Assuming that the sampling frequency is Fs and the number of samples is N, the result of the FFT is N complex numbers, any one of which is associated with a frequency. More specifically, the relationship between the ith complex number and the frequency f is:
f＝(i-1)×Fs/N
then, the amplitude at frequency f can be calculated by:
wherein z isf＝af+bfi is the complex number associated with frequency f. Then we can obtain the energy at frequency f by squaring the amplitude:
then, EMF can be found from the maximum of the energy in all frequency domains, and correspondingly, MF is the frequency of the frequency domain with the maximum energy, and ER is the ratio of MF to the energy of all frequency domains.
c) An information entropy method. The user features are extracted by using two information entropies, including a fragrance entropy (Shannenentrypy) and a Rayleigh entropy (Rinyi entrypy). The entropy of the aroma of a random signal X is defined as follows:
where p (x) is the probability density function of the value x, which is estimated using MLE (Maximum Likelihood estimation):
where I is an indicator function, and therefore, p (x) reflects the probability that sample x appears in the population.
The Rayleigh entropy is an expansion form of the fragrance entropy, and is defined as follows:
where r ≧ 0 is the order of rayleigh entropy, and the aroma entropy is actually the case where r is 1, and the case where r is 2 is what is commonly called rayleigh entropy.
2) A feature extraction strategy:
the motion parameters are behavior and motion descriptions based on the user drawing task, which are motion information of "point" level, and for each sub-track, motion information of "track" level, i.e. track level feature (track level feature), is extracted by using a feature extraction method of statistics, information processing and information entropy, and finally, motion information of "graph" level, i.e. graph level feature, is extracted by performing statistical analysis on all sub-tracks of the whole graph, as shown in fig. 4. Where pi at a point level feature represents the ith point in the trajectory, di represents the deviation of the point from the point after filtering, ai represents the acceleration of the point, vi represents the velocity of the point, and ri represents the curvature of the point.
The feature extraction methods were divided into three "method groups" (MGs) that made them suitable for processing different types of data, as shown in table 1 below. MG1 includes statistical methods for extracting overall statistical attributes from the data. The MG2 processes vector data with timing information. Scalars with timing information can then be processed simultaneously using two sets of methods. MG3 is used to extract statistical attributes from the overall map data and features.
TABLE 1 all feature extraction methods and method group partitioning
The steps for extracting user features from a drawing task are as follows, and FIG. 5 illustrates these four steps:
(1) and processing the original track of the drawing task by using a general track segmentation algorithm to obtain a sub-track set.
(2) And for any section of sub-track, calculating the behavior parameters and the motion parameters of all points on the track.
(3) And extracting track level characteristics from the behavior parameters and the motion parameters of the track by using the MG1 and the MG 2.
(4) Extracting two types of graphic level features (graphic level features) of the whole drawing task: statistical attributes of all plot motion parameters, and statistical attributes of all sub-trajectory features. For the first type, MG1 is used to extract features from the motion parameters of all drawing points, referred to as SPM (static properties on all sampling points); for the second type, MG3 is used to extract features from all sub-track features, that is, "features of features", referred to as ssf (static properties on all track level features for short).
In step (3) of the feature extraction strategy, the MG1 and the MG2 are used, the types of the motion parameters are considered, and for different motion parameters, a matching method group is established, and all the matching method groups and the calculated feature numbers are shown in table 2 below.
TABLE 2 method set and calculated feature number
The correspondence of motion parameters to symbols in the above table is shown in table 3 below, from different motion parameters, trace-level features are extracted from sub-traces, then graphics-level features 32 × +36 × -940 SSF are extracted from these trace-level features, then statistical attributes are directly calculated from all point sets, graphics-level features 8 × -64 are extracted, and 64+ 940-1004 dimensional features are composed.
TABLE 3 motion parameters and symbols
5. And (3) classified diagnosis: on the obtained total data feature set, a feature selection algorithm is used to screen out features most relevant to diseases, and the features are used as input data (such as but not limited to decision tree (C4.5), Naive Bayes (NB) and linear Support Vector Machine (SVM)) of a classification model to train so as to obtain a classification model for automatic diagnosis of diseases.
1. The system hardware is composed of a touch pen, a displayable drawing board and a common desktop computer, for example, the touch pen is a Wacom drawing pen with the length of 14.4cm, the diameter of 1cm, the weight of 19g and the sampling rate of 100Hz, the model of the displayable drawing board is Wacom Cintiq DTK-1300, the screen size is 13.3 inches, and the resolution is 1280 × 800 pixels, the main reason for selecting the equipment is that besides basic handwriting, the equipment also supports the acquisition of data such as pressure sense and pen body posture, so that more comprehensive hand motion characteristics can be acquired, the software system is installed on the matched computer, is developed by a C # programming language, operates under a NET Framework 4.0, and performs the unified user data storage and management by using SQLite and a file system, the handwriting data structure is adopted by the system, the sampling point data set is an ordered sampling point data structure:
data＝{timestamp,x,y,pressure,azimuth,altitude,rotation,pen_event}
wherein timestamp is a timestamp, and PEN _ event is an enumeration type for marking PEN status, and is used for marking events such as PEN DOWN (PEN DOWN), drawing (PEN MOVE), PEN UP (PEN UP), and the like.
2. After the test is started, the system starts to record user data, and after the user test is completed, the system combines all sample data obtained by 5 tests to obtain a total data set containing 1070 samples (214 samples per test), which is called ODS (orthogonal dataset). Thereafter, the 6 data sets were analyzed separately.
3. All 5 test data are processed by using a task-independent drawing task feature extraction method, 1004-dimensional features are obtained in each sample of each test, feature selection is carried out on the feature set, the most important feature set in the evaluation of the nervous system disease-oriented hand motion function is found out, and the features are analyzed. The feature analysis is carried out on the data sets comprising the ODS and the 5 tests, the feature set which is excellent in different tasks can be obtained through the feature analysis of the ODS, and the feature set which is suitable for different degrees of freedom and different scenes can be obtained through the feature analysis of the 5 test data sets.
4. And (3) carrying out data cleaning and normalization, and specifically comprising the following steps:
1) removing samples lacking more than 30% of feature data; 2) calculating the mean value and the standard deviation of each dimension characteristic in the non-empty data; 3) replacing null value features of corresponding dimensions of the sample data by using the feature mean value of each dimension; 4) replacing feature data of each dimension beyond three times of standard deviation by using the feature mean value of each dimension; 5) the feature vectors extracted for the pathological features were normalized to [0,1] using the maximum minimum.
5. The optimal subset is searched by using a characteristic selection method based on correlation so as to increase the operation speed without losing the most main characteristic information, and the algorithm utilizes the following heuristic indexes to guide the searching process of the optimal subset:
wherein M issIs the goodness of the subset S containing k features,
is the average feature-class correlation coefficient,
is the average feature-feature internal correlation coefficient.
6. The feature selection algorithm picks 27 optimal feature subsets in the ODS, picking the corresponding features. The features are classified in three different ways to analyze the performance of different classes of features in the mapping task. (1) The relevant features are classified into 9 categories of position, UMO, curvature, velocity, acceleration, pressure, azimuth, altitude and rotation angle according to the motion parameter category. (2) The related features are divided into two categories of MG1 (statistics) and MG2 (signal and entropy) according to the track level feature extraction method group. (3) The related features are divided into two categories of SPM (statistical properties of point parameters) and SSF (statistical properties of trace-level features) according to the feature category of the graph level.
7. Training a diagnosis model:
in the stage of diagnosing model training, we can use different classifiers to train, and the implementation process is described in detail below by taking a support vector machine as an example.
Support Vector Machines (SVMs) are a class of supervised learning in machine learning, and can construct a hyperplane, which is used for classification or regression. For training of Support Vector Machines (SVMs), the following scheme is used:
1) and taking the features extracted in the steps as input, and labeling the sample.
2) And performing feature extraction on the collected user drawing tasks, and taking the extracted user drawing tasks as the input of a support vector machine to obtain a final classification result.
The method and system of the present invention can be used not only for the related hardware devices mentioned in the above specific trial solution, but also for deployment and use in, but not limited to, tablet, mobile phone, and interactive device with touch function.
The above embodiments are only intended to illustrate the technical solution of the present invention and not to limit the same, and a person skilled in the art can modify the technical solution of the present invention or substitute the same without departing from the spirit and scope of the present invention, and the scope of the present invention should be determined by the claims.
Claims (10)
1. A touch interactive motion user feature extraction method based on a general drawing task is characterized by comprising the following steps:
1) collecting drawing tracks of different drawing tasks of a user;
2) carrying out track segmentation on the collected drawing tracks to form a task-independent sub-track set;
3) calculating motion parameters of all points of any section of sub-track in the sub-track set;
4) track-level features are extracted from the motion parameters and statistical analysis is performed on all sub-tracks to extract graph-level features.
2. The method of claim 1, wherein step 1) collects a drawing trace of the user using a stylus and a drawing screen; the touch control pen is provided with a pressure sensor and a gyroscope and can acquire X and Y direction positions, touch control point pressure and touch control equipment space postures, and the touch control equipment space postures comprise azimuth angles, elevation angles and self-rotation angles.
3. The method of claim 1, wherein the step 1) of collecting the user's drawing trace comprises:
1.1) time equidistant trajectory, requiring the time interval between two sampling points to be consistent, for calculating time-domain correlation features
1.2) space equidistant trajectory, the space distance between two sampling points is required to be consistent, and the spatial equidistant trajectory is used for calculating the position correlation characteristics.
4. The method of claim 1, wherein step 2) the track slicing comprises:
2.1) segmenting long and complex tracks by using local geometric extreme values;
and 2.2) clustering the dense over-segmentation points by utilizing time and space information, and selecting the optimal segmentation point as a final segmentation point.
5. The method of claim 4, wherein step 2.1) SER resampling the trajectory to obtain curvatures for all points and placing them in a linked list, then using gaussian filter to obtain local extrema of curvature from the linked list, and screening out the over-cut points by a curvature threshold to place them in the over-cut point list; and 2.2) traversing all the over-cutting points, recording other over-cutting points in a space-time domain range R of all the over-cutting points in a data structure named nearIndexes, then sorting the over-cutting points in a descending order according to the quantity of the nearIndexes of the over-cutting points, finally traversing the sorted over-cutting points, removing other over-cutting points recorded in the nearIndexes of each over-cutting point from a list of the over-cutting points, and taking the remaining list of the over-cutting points as the final over-cutting points.
6. The method of claim 1, wherein the motion parameters of step 3) include both behavioral parameters and kinematic parameters; the behavior parameters are parameters directly reflecting the drawing behaviors and decisions of the user; the kinematic parameters are first and second order kinematic parameters obtained by deriving a part of the behavior parameters with respect to time.
7. The method of claim 1, wherein step 4) adopts a statistical method, a signal processing method and an information entropy method to extract track-level features; the statistical method comprises summation, mean, maximum value, minimum value, quartile and standard deviation; the signal processing method comprises a time domain method and a frequency domain method, wherein the time domain method utilizes zero crossing times, zero crossing frequency and root mean square to extract features, and the frequency domain method utilizes dominant frequency, dominant frequency energy and dominant frequency energy ratio to extract features; the information entropy method utilizes the fragrance concentration entropy and the Rayleigh entropy to extract features.
8. The method of claim 1, wherein the extracting of the graphics-level features of step 4) is extracting two types of graphics-level features of the whole drawing task: statistical attributes of all plot motion parameters, and statistical attributes of all sub-trajectory features.
9. A touch interactive motion user feature extraction device based on a general drawing task is characterized by comprising the following steps:
the data acquisition module is responsible for acquiring drawing tracks of different drawing tasks of a user;
the track segmentation module is responsible for carrying out track segmentation on the collected drawing tracks to form a task-independent sub-track set;
the parameter calculation module is responsible for calculating motion parameters of all points of any section of sub-track in the sub-track set;
and the characteristic extraction module is responsible for extracting the track level characteristics from the motion parameters and performing statistical analysis on all the sub-tracks to extract the graph level characteristics.
10. An auxiliary disease detection system based on a general drawing task, which is characterized by comprising the touch interactive motion user feature extraction device based on the general drawing task as claimed in claim 9, and a classification model training module and a disease diagnosis module; the classification model training module screens the characteristics related to the diseases from the characteristics extracted by the characteristic extraction module by using a characteristic selection algorithm to train a classification model; and the disease diagnosis module performs automatic diagnosis of diseases by using the trained classification model.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811591897.4A CN111354458B (en) | 2018-12-20 | 2018-12-20 | Touch interactive motion user feature extraction method and auxiliary disease detection system based on universal drawing task |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811591897.4A CN111354458B (en) | 2018-12-20 | 2018-12-20 | Touch interactive motion user feature extraction method and auxiliary disease detection system based on universal drawing task |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN111354458A true CN111354458A (en) | 2020-06-30 |
| CN111354458B CN111354458B (en) | 2023-11-14 |
Family
ID=71198111
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201811591897.4A Active CN111354458B (en) | 2018-12-20 | 2018-12-20 | Touch interactive motion user feature extraction method and auxiliary disease detection system based on universal drawing task |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111354458B (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| EP4350489A1 (en) * | 2022-10-07 | 2024-04-10 | BIC Violex Single Member S.A. | Writing instrument and method therefor |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20050234309A1 (en) * | 2004-01-07 | 2005-10-20 | David Klapper | Method and apparatus for classification of movement states in Parkinson's disease |
| CN105068743A (en) * | 2015-06-12 | 2015-11-18 | 西安交通大学 | Mobile terminal user identity authentication method based on multi-finger touch behavior characteristics |
| CN106095104A (en) * | 2016-06-20 | 2016-11-09 | 电子科技大学 | Continuous gesture path dividing method based on target model information and system |
| CN107045393A (en) * | 2016-02-05 | 2017-08-15 | 中国科学院软件研究所 | Line test multi-channel data collecting method and system based on digital pen |
| CN107273677A (en) * | 2017-06-08 | 2017-10-20 | 中国科学院软件研究所 | A kind of multi-channel nerve function quantitative evaluation system |
| CN108962397A (en) * | 2018-06-06 | 2018-12-07 | 中国科学院软件研究所 | A kind of multichannel multitask the nervous system disease assistant diagnosis system based on pen and voice |
-
2018
- 2018-12-20 CN CN201811591897.4A patent/CN111354458B/en active Active
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20050234309A1 (en) * | 2004-01-07 | 2005-10-20 | David Klapper | Method and apparatus for classification of movement states in Parkinson's disease |
| CN105068743A (en) * | 2015-06-12 | 2015-11-18 | 西安交通大学 | Mobile terminal user identity authentication method based on multi-finger touch behavior characteristics |
| CN107045393A (en) * | 2016-02-05 | 2017-08-15 | 中国科学院软件研究所 | Line test multi-channel data collecting method and system based on digital pen |
| CN106095104A (en) * | 2016-06-20 | 2016-11-09 | 电子科技大学 | Continuous gesture path dividing method based on target model information and system |
| CN107273677A (en) * | 2017-06-08 | 2017-10-20 | 中国科学院软件研究所 | A kind of multi-channel nerve function quantitative evaluation system |
| CN108962397A (en) * | 2018-06-06 | 2018-12-07 | 中国科学院软件研究所 | A kind of multichannel multitask the nervous system disease assistant diagnosis system based on pen and voice |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| EP4350489A1 (en) * | 2022-10-07 | 2024-04-10 | BIC Violex Single Member S.A. | Writing instrument and method therefor |
Also Published As
| Publication number | Publication date |
|---|---|
| CN111354458B (en) | 2023-11-14 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Tonekaboni et al. | Unsupervised representation learning for time series with temporal neighborhood coding | |
| CN111046731B (en) | Transfer learning method and recognition method for gesture recognition based on surface electromyographic signals | |
| US20180144182A1 (en) | Analyzing digital holographic microscopy data for hematology applications | |
| Exarchos et al. | EEG transient event detection and classification using association rules | |
| Jadhav et al. | Modular neural network based arrhythmia classification system using ECG signal data | |
| CN108511055B (en) | Ventricular premature beat recognition system and method based on classifier fusion and diagnosis rules | |
| Fahimi et al. | On metrics for measuring scanpath similarity | |
| CN112818883B (en) | Deep learning detection and positioning method for interested target based on eye movement signal | |
| CN109817339A (en) | Patient's group technology and device based on big data | |
| Liu et al. | Analyzing outliers cautiously | |
| CN111783887B (en) | Classified lie detection identification method based on fMRI (magnetic resonance imaging) small-world brain network computer | |
| Jasm et al. | Deep image mining for convolution neural network | |
| CN107045624B (en) | Electroencephalogram signal preprocessing and classifying method based on maximum weighted cluster | |
| Raykov et al. | Probabilistic modelling of gait for robust passive monitoring in daily life | |
| Ingle et al. | Lung Cancer Types Prediction Using Machine Learning Approach | |
| CN111053552A (en) | QRS wave detection method based on deep learning | |
| CN108962379B (en) | Mobile phone auxiliary detection system for cranial nerve system diseases | |
| CN111354458A (en) | Touch interactive motion user feature extraction method based on general drawing task and auxiliary disease detection system | |
| Ng et al. | Automated identification of epileptic and alcoholic EEG signals using recurrence quantification analysis | |
| CN116484290A (en) | Depression recognition model construction method based on Stacking integration | |
| CN113171102B (en) | ECG data classification method based on continuous deep learning | |
| CN115358260A (en) | Electroencephalogram sleep staging method and device, electronic equipment and storage medium | |
| CN112022149B (en) | Atrial fibrillation detection method based on electrocardiosignals | |
| Toranjsimin et al. | Robust Low Complexity Framework for Early Diagnosis of Autism Spectrum Disorder Based on Cross Wavelet Transform and Deep Transfer Learning | |
| CN113133742B (en) | Knee joint signal feature extraction and classification method and device based on time domain multidimensional feature fusion |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |