CN115363581B - Method, equipment and medium for predicting dysreading for young children - Google Patents
Method, equipment and medium for predicting dysreading for young children Download PDFInfo
- Publication number
- CN115363581B CN115363581B CN202210997976.5A CN202210997976A CN115363581B CN 115363581 B CN115363581 B CN 115363581B CN 202210997976 A CN202210997976 A CN 202210997976A CN 115363581 B CN115363581 B CN 115363581B
- Authority
- CN
- China
- Prior art keywords
- user
- image set
- eye movement
- movement track
- interest
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title abstract description 54
- 230000004424 eye movement Effects 0.000 claims abstract description 91
- 230000000007 visual effect Effects 0.000 claims abstract description 43
- 208000012661 Dyskinesia Diseases 0.000 claims abstract description 29
- 208000022372 Reading disease Diseases 0.000 claims abstract description 20
- 206010013932 dyslexia Diseases 0.000 claims abstract description 20
- 230000008859 change Effects 0.000 claims abstract description 12
- 238000012549 training Methods 0.000 claims description 31
- 238000012706 support-vector machine Methods 0.000 claims description 15
- 230000004434 saccadic eye movement Effects 0.000 claims description 4
- 230000008569 process Effects 0.000 description 16
- 238000011161 development Methods 0.000 description 12
- 230000018109 developmental process Effects 0.000 description 12
- 230000006870 function Effects 0.000 description 11
- 230000003920 cognitive function Effects 0.000 description 8
- 230000001149 cognitive effect Effects 0.000 description 7
- 238000004590 computer program Methods 0.000 description 7
- 238000010586 diagram Methods 0.000 description 7
- 238000012545 processing Methods 0.000 description 7
- 238000013461 design Methods 0.000 description 6
- 238000012360 testing method Methods 0.000 description 6
- 230000006378 damage Effects 0.000 description 5
- 238000004458 analytical method Methods 0.000 description 4
- 238000005516 engineering process Methods 0.000 description 4
- 230000004438 eyesight Effects 0.000 description 4
- 239000011159 matrix material Substances 0.000 description 4
- 230000000750 progressive effect Effects 0.000 description 4
- 239000000243 solution Substances 0.000 description 4
- 238000013473 artificial intelligence Methods 0.000 description 3
- 230000009286 beneficial effect Effects 0.000 description 3
- 230000004641 brain development Effects 0.000 description 3
- 230000019771 cognition Effects 0.000 description 3
- 230000000694 effects Effects 0.000 description 3
- 230000005540 biological transmission Effects 0.000 description 2
- 239000003086 colorant Substances 0.000 description 2
- 239000012141 concentrate Substances 0.000 description 2
- 238000007596 consolidation process Methods 0.000 description 2
- 238000013480 data collection Methods 0.000 description 2
- 238000003745 diagnosis Methods 0.000 description 2
- 208000037265 diseases, disorders, signs and symptoms Diseases 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 230000003287 optical effect Effects 0.000 description 2
- 238000010422 painting Methods 0.000 description 2
- 230000001575 pathological effect Effects 0.000 description 2
- 230000007170 pathology Effects 0.000 description 2
- 208000020016 psychiatric disease Diseases 0.000 description 2
- 238000011160 research Methods 0.000 description 2
- 230000035945 sensitivity Effects 0.000 description 2
- 238000007619 statistical method Methods 0.000 description 2
- 230000002123 temporal effect Effects 0.000 description 2
- 208000020358 Learning disease Diseases 0.000 description 1
- 208000027418 Wounds and injury Diseases 0.000 description 1
- 230000015572 biosynthetic process Effects 0.000 description 1
- 230000003925 brain function Effects 0.000 description 1
- 238000012512 characterization method Methods 0.000 description 1
- 230000008131 children development Effects 0.000 description 1
- 238000003759 clinical diagnosis Methods 0.000 description 1
- 230000036992 cognitive tasks Effects 0.000 description 1
- 230000001447 compensatory effect Effects 0.000 description 1
- 238000013479 data entry Methods 0.000 description 1
- 238000001514 detection method Methods 0.000 description 1
- 230000001771 impaired effect Effects 0.000 description 1
- 238000002347 injection Methods 0.000 description 1
- 239000007924 injection Substances 0.000 description 1
- 208000014674 injury Diseases 0.000 description 1
- 230000010354 integration Effects 0.000 description 1
- 201000003723 learning disability Diseases 0.000 description 1
- 238000004519 manufacturing process Methods 0.000 description 1
- 239000000463 material Substances 0.000 description 1
- 230000007246 mechanism Effects 0.000 description 1
- 230000001717 pathogenic effect Effects 0.000 description 1
- 230000003068 static effect Effects 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- 208000024891 symptom Diseases 0.000 description 1
- 238000012546 transfer Methods 0.000 description 1
- 230000004382 visual function Effects 0.000 description 1
Images
Classifications
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B5/00—Measuring for diagnostic purposes; Identification of persons
- A61B5/16—Devices for psychotechnics; Testing reaction times ; Devices for evaluating the psychological state
- A61B5/163—Devices for psychotechnics; Testing reaction times ; Devices for evaluating the psychological state by tracking eye movement, gaze, or pupil change
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B5/00—Measuring for diagnostic purposes; Identification of persons
- A61B5/40—Detecting, measuring or recording for evaluating the nervous system
- A61B5/4076—Diagnosing or monitoring particular conditions of the nervous system
- A61B5/4088—Diagnosing of monitoring cognitive diseases, e.g. Alzheimer, prion diseases or dementia
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/011—Arrangements for interaction with the human body, e.g. for user immersion in virtual reality
- G06F3/013—Eye tracking input arrangements
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/20—Image preprocessing
- G06V10/25—Determination of region of interest [ROI] or a volume of interest [VOI]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/40—Extraction of image or video features
- G06V10/56—Extraction of image or video features relating to colour
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/764—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using classification, e.g. of video objects
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B2503/00—Evaluating a particular growth phase or type of persons or animals
- A61B2503/06—Children, e.g. for attention deficit diagnosis
Abstract
The application discloses a method, equipment and medium for predicting dyskinesia for young children, and relates to the field of data identification. The method comprises the following steps: acquiring first eye movement track data of a first interest area of a first image set of a user; the first image set is a set of first images that contain a first region of interest; the eye movement track data at least comprises the number of fixation points and the first fixation frequency; determining a second image set in the first image set according to the number of fixation points and the first fixation frequency; changing the color visual characteristics of the first region of interest to change the first region of interest to a second region of interest; generating a third region of interest in the second image, such that the third region of interest includes color visual features of the first region of interest; acquiring second eye movement track data of a user about a second region of interest in a second image set; and generating a first prediction result of the reading disorder of the user according to the first eye movement track data and the second eye movement track data.
Description
Technical Field
The application relates to the field of data identification, in particular to a method, equipment and medium for predicting dysreading of a young child.
Background
Dyskinesia belongs to one of learning disorders, and at present, the pathological mechanism of dyskinesia is not completely clear in the pathological research of mental disorder. The common technical scheme mainly can be divided into two directions, and the first one belongs to symptoms and functional injury aiming at dysreading, and provides auxiliary techniques for auxiliary training and auxiliary treatment of dysreading, such as: the invention patent of vision-sports integration reading training system, CN114360328A, proposes a drawing tool of a painting training method and a design method of paper materials, which are used for painting treatment of patients with dyskinesia; the invention patent (intelligent device for multi-sense language spelling learning for read-write obstacle children) discloses a device design method for rehabilitation training of read-write obstacle children, which is disclosed in CN 114613206A. The second one pertains to an invention for prediction of dysreading using artificial intelligence means, such as: the invention patent of intelligent identification method, system, equipment and storage medium for developmental dyskinesia, CN113842113A, proposes a design method for jointly detecting teenager dyskinesia equipment by using a phrase article and eye movement equipment under the condition of unclear pathology; the invention patent of a method for predicting Chinese dyskinesia through visual efficiency, CN114209274A, proposes a method for designing dyskinesia cause detection equipment for deducing whether functions such as eyesight, vision adjusting function, vergence function, eye deviation and the like are damaged through visual function parameter acquisition and analysis; the invention discloses a Chinese progressive dysreading prediction system and a prediction method thereof, CN112381287A, which provides a dysreading prediction system design method using demographic information, voice consciousness test and reading accuracy test as input, an artificial intelligent model as an analysis module and voice consciousness, picture naming capability and digital rapid naming capability as output.
The limitations of the above technical solution are: the developing dyskinesia is a developing disorder, in the brain development process of children and teenagers, imbalance and 'window period' of function development exist, and brain development has compensatory effect, but the above technical scheme does not consider the brain development characteristics of different development stages and the development characteristics of corresponding cognitive functions.
Disclosure of Invention
In order to solve the above problems, the present application provides a method, a device and a medium for predicting dysreading for young children, where the method includes:
acquiring first eye movement track data of a first interest area of a first image set of a user; the first image set is a set of first images including a first region of interest; the region of interest is a region where a preset visual feature is located; the eye movement track data at least comprises the number of fixation points and the first fixation frequency; determining a second image set in the first image set according to the number of fixation points and the first fixation frequency; changing color visual characteristics of the first region of interest in the second set of images to change the first region of interest to a second region of interest; generating a third interest area in each second image, wherein the third interest area comprises color visual characteristics of the first interest area; acquiring second eye movement track data of a user about the second region of interest in a second image set through the eye movement instrument; and generating a first prediction result that the user has dyskinesia according to the first eye movement track data and the second eye movement track data.
In one example, the determining a second image set in the first image set according to the number of fixation points and the first fixation frequency specifically includes: according to the number of the fixation points and the first fixation frequency, which correspond to each first image in the first image set, respectively, determining a mean value of the number of the fixation points and a mean value of the first fixation frequency, which correspond to the first image set; according to the gaze point number average value and the first gaze frequency average value, determining standard deviations between the gaze point number corresponding to each first image and the first gaze frequency and the gaze point number average value and the first gaze frequency average value respectively; and selecting the image with the standard deviation larger than a first preset threshold value from the first image set as a second image to generate a second image set.
In one example, the generating a first prediction result that the user has a dysreading according to the first eye movement track data and the second eye movement track data specifically includes: determining a preset support vector machine initial model, and acquiring pre-stored first training data; training the support vector machine initial model through the first training data to obtain a first judgment model; inputting the first eye movement track data and the second eye movement track data into the first judging model to judge whether the first eye movement track data and the second eye movement track data have the same association relation or not; if not, the user has a reading disorder.
In one example, after the generating the first prediction result that the user has a dysreading according to the first eye movement track data and the second eye movement track data, the method further includes: generating a third image set and a fourth image set according to the character pattern features and the character sound features; the font characteristic in the third image set is consistent with the word sound characteristic, and the font characteristic in the fourth image set is inconsistent with the word sound characteristic; acquiring, by the eye tracker, a first length of glance time for the user for the third set of images and a second length of glance time for the fourth set of images; the saccade time length is the saccade time of the user from one side of the image to the other side; and generating a second prediction result of the user having dysreading according to the first glance time length and the second glance time length.
In one example, the generating a second prediction result that the user has dysreading according to the first glance time length and the second glance time length specifically includes: t-checking the first glance time length and the second glance time length to judge whether a difference exists between the first glance time length and the second glance time length; if not, the user has a reading disorder.
In one example, after the generating the first prediction result that the user has a dysreading according to the first eye movement track data and the second eye movement track data, the method further includes: randomly generating a fifth image set, wherein the fifth image contains a preset number of phrases; acquiring a first reading time length of the user for the fifth image set through an eye tracker; determining the coordinates of the point of injection when the user reads the fifth image through an eye tracker; when the annotating point coordinates are overlapped with the first phrase, changing a second phrase adjacent to the first phrase until the user finishes reading so as to obtain a second reading time length; and generating a third prediction result of the reading disorder of the user according to the first reading time length and the second reading time length.
In one example, the generating a third prediction result that the user has a reading disorder according to the first reading time length and the second reading time length specifically includes: determining a preset support vector machine initial model, and acquiring pre-stored second training data; training the support vector machine initial model through the second training data to obtain a second judgment model; inputting the first reading time length and the second reading time length into the second judging model to judge whether the first reading time length and the second reading time length have the same association relation or not; if not, the user has a reading disorder.
In one example, the method further comprises: acquiring the occupied time length of the user in the current flow; and if the occupied time exceeds a preset time threshold, displaying the user flow progress and prompting the user to currently execute the flow.
The application also provides a dyskinesia prediction device for young children, comprising: at least one processor; and a memory communicatively coupled to the at least one processor; wherein the memory stores instructions executable by the at least one processor to enable the at least one processor to perform: acquiring first eye movement track data of a first interest area of a first image set of a user; the first image set is a set of first images including a first region of interest; the region of interest is a region where a preset visual feature is located; the eye movement track data at least comprises the number of fixation points and the first fixation frequency; determining a second image set in the first image set according to the number of fixation points and the first fixation frequency; changing color visual characteristics of the first region of interest in the second set of images to change the first region of interest to a second region of interest; generating a third interest area in each second image, wherein the third interest area comprises color visual characteristics of the first interest area; acquiring second eye movement track data of a user about the second region of interest in a second image set through the eye movement instrument; and generating a first prediction result that the user has dyskinesia according to the first eye movement track data and the second eye movement track data.
The present application also provides a non-volatile computer storage medium storing computer executable instructions, characterized in that the computer executable instructions are configured to: acquiring first eye movement track data of a first interest area of a first image set of a user; the first image set is a set of first images including a first region of interest; the region of interest is a region where a preset visual feature is located; the eye movement track data at least comprises the number of fixation points and the first fixation frequency; determining a second image set in the first image set according to the number of fixation points and the first fixation frequency; changing color visual characteristics of the first region of interest in the second set of images to change the first region of interest to a second region of interest; generating a third interest area in each second image, wherein the third interest area comprises color visual characteristics of the first interest area; acquiring second eye movement track data of a user about the second region of interest in a second image set through the eye movement instrument; and generating a first prediction result that the user has dyskinesia according to the first eye movement track data and the second eye movement track data.
According to the method provided by the application, whether the cognitive function of the young children in different development stages is problematic can be predicted, and whether the reading disorder of the young children is predicted by combining the cognitive function development characteristics.
Drawings
The accompanying drawings, which are included to provide a further understanding of the application and are incorporated in and constitute a part of this application, illustrate embodiments of the application and together with the description serve to explain the application and do not constitute an undue limitation to the application. In the drawings:
fig. 1 is a schematic flow chart of a method for predicting dyskinesia for young children according to an embodiment of the present application;
FIG. 2 is a schematic view of a first image according to an embodiment of the present application;
fig. 3 is a schematic diagram of a device for predicting dyskinesia for young children according to an embodiment of the present application.
Detailed Description
For the purposes, technical solutions and advantages of the present application, the technical solutions of the present application will be clearly and completely described below with reference to specific embodiments of the present application and corresponding drawings. It will be apparent that the described embodiments are only some, but not all, of the embodiments of the present application. All other embodiments, which can be made by one of ordinary skill in the art without undue burden from the present disclosure, are within the scope of the present disclosure.
The following describes in detail the technical solutions provided by the embodiments of the present application with reference to the accompanying drawings.
Aiming at the limitations of the prior art, such as the indistinguishable development stage and the lack of predictive design suitable for the young children and the lack of interpretability of the artificial intelligence analysis model, the invention provides a design scheme of an interpretable artificial intelligence predictive device for the young children, which considers the development stage.
According to the research result of educational neuroscience, the formation of reading ability is mainly based on a triangular loop formed by a frontal area, a temporal occipital area and a temporal cortex area, which are centered on a blossoming area in a brain function network, and a dorsal loop formed by a dorsal loop. The triangle loop processes font-semantic information, font-word sound information and word sound-semantic information respectively. According to the cognitive neuroscience of children's development, the children's reading ability development is firstly to build the association of visual cognition (graphic features, topological features, color features and the like) and character processing, then to build the association of character form-word sound-semantic, and finally to build the direct association of character form or word sound and semantic, and the development process of the above development process corresponding to reading cognition is generally divided into a pre-letter stage, a partial letter stage, a complete letter stage and a letter consolidation stage. In the four development stages, there is a developmental difference in cognitive function in each stage, but the cognitive function may act on the occurrence of dysreading.
Fig. 1 is a flow chart of a method for predicting dyskinesia for young children according to one or more embodiments of the present disclosure. The process may be performed by a corresponding computing device, with some input parameters or intermediate results in the process allowing for manual intervention adjustments to help improve accuracy.
The implementation of the analysis method according to the embodiment of the present application may be a terminal device or a server, which is not particularly limited in this application. For ease of understanding and description, the following embodiments are described in detail with reference to a server. It should be noted that the server may be a single device, or may be a system formed by a plurality of devices, that is, a distributed server, which is not specifically limited in this application.
Note that, the present invention is not limited to the above-described embodiments. The invention is not used for directly providing a direct clinical diagnosis conclusion or diagnosis suggestion for the developing dyskinesia, but provides multidimensional information for diagnosing the developing dyskinesia in pathology for the psychiatrist of children, and because the developing dyskinesia is a mental disorder formed by multiple causes, multiple hypothesis establishment and multiple hypothesis examination are required in the diagnosis process to find the pathogenic cause of a specific patient individual.
As shown in fig. 1, an embodiment of the present application provides a method for predicting dysreading for a young child, including:
s101: acquiring first eye movement track data of a first interest area of a first image set of a user; the first image set is a set of first images including a first region of interest; the region of interest is a region where a preset visual feature is located; the eye movement track data at least comprises the number of fixation points and the first fixation frequency.
The predicted part of the method is the recognition of visual clues in the pre-letter stage and the recognition of cognitive functions of letter recognition. Since the vocabulary can be preferentially noticed during the visual search when the user observes the familiar vocabulary, the features of increased gaze point and increased first gaze frequency are manifested on the eye movement features. In the process of learning, the young children mainly rely on cognitive correlation between visual features such as shapes and colors and words in the starting stage, if semantic characterization is generated, after the visual features such as shapes and colors are changed, the words can be preferentially noticed in the visual search process, and the features of increased fixation points and increased first fixation frequency are presented. Therefore, first, a first image set needs to be displayed on a display screen, and it needs to be explained that the first image set is a set of first images including a first region of interest; the region of interest is a region where a preset visual feature is located; the eye movement track data at least comprises the number of fixation points and the first fixation frequency; the first image is exemplified by a nine-square form of 9 pinyin letters, wherein one group of pinyin letters contains specific-meaning combinations, the other groups are in a disordered order, and the specific-meaning combinations are marked by red bold. As shown in fig. 2, where "M", "a" may be marked in bold red, the first region of interest in this example is the MA region of interest. Then, the child is required to view the letter sequence displayed on the screen, and eye movement track data of the child is acquired through the eye movement instrument, wherein the eye movement track data comprises the number of fixation points NGP (Number of Gaze Point) in the MA interest area and the first fixation frequency NFGP (Number of First Gaze Point).
S102: and determining a second image set in the first image set according to the number of the fixation points and the first fixation frequency.
After the eye movement track data of the user for the first region of interest is acquired, a certain number of pictures need to be selected in the first image set to form a second image set. After the children complete the pinyin combination of watching the first image set, statistical analysis is carried out on the NGP and NFGP data corresponding to each combination, so that the second image set is selected from the first image set.
In one embodiment, when determining the second image set in the first image set, firstly, determining a gaze point number average value and a gaze frequency average value corresponding to the first image set according to the gaze point number and the first gaze frequency corresponding to each first image in the first image set, and then determining standard deviations between the gaze point number and the first gaze frequency corresponding to each first image and the gaze point number average value and the first gaze frequency average value according to the gaze point number average value and the first gaze frequency average value; and selecting the image with the standard deviation larger than a first preset threshold value from the first image set as a second image to generate a second image set. For example, a combination of NGP and NFGP scores greater than 2 standard deviations of the mean is calculated and used as the second image to generate the second image set.
S103: changing color visual characteristics of the first region of interest in the second set of images to change the first region of interest to a second region of interest; and generating a third region of interest in each second image such that the third region of interest contains color visual features of the first region of interest.
When the second image set is displayed, the original region of interest in the image needs to be changed, and a nine-square box is taken as an example, when the region of interest is changed, firstly, the red mark of the original MA region of interest is changed into the blue mark, and then, the region is randomly selected to carry out red marks, such as red marked letters of 'B' and 'U'.
S104: and acquiring second eye movement track data of a user about the second interest area in a second image set through the eye movement instrument.
At this time, the user views the second image set modified by the region of interest on the display screen to obtain another set of eye movement track data.
S105: and generating a first prediction result that the user has dyskinesia according to the first eye movement track data and the second eye movement track data.
After the two groups of eye movement track data are obtained, comparison can be performed according to the two groups of data so as to analyze whether the user has a possibility of dyskinesia.
Specifically, when a first prediction result of the user with reading disorder is generated, a preset support vector machine initial model is first required to be determined, and pre-stored first training data is acquired. And training the support vector machine initial model through the first training data to obtain a first judgment model. And inputting the first eye movement track data and the second eye movement track data into the first judging model to judge whether the first eye movement track data and the second eye movement track data have the same association relation, and if not, the user has reading disorder.
The first training data is data acquired by a preset number of tested users before the eye movement track data of the users are acquired. For example, 200 children already having the primary pinyin reading capability are recruited in the early stage to perform data collection to establish a first judgment model, the children in the sample group are required to complete the visual and cognitive tasks, and the information in the matrix is observed by adopting an 'out-sound thinking method' description, for example: "I see MA, referring to 'mother'; alternatively, I do not see any meaningful pinyin or I do not know what the pinyin represents. Each child completes 20 visual observation tasks, acquires 4000 pieces of original data in total, and filters and eliminates invalid data in the 4000 pieces of original data to acquire 3678 pieces of valid data. Further establishing the relationship between NGP, NFGP and labels in the gazing task, namely reporting the data items of the NGP and the NFGP corresponding to the 'seen valid information/pinyin', and marking the data items as 'yes'; reporting a data entry of "no valid information seen or what is represented" marked "no"; together, the result is data 1822 labeled "yes" and data 1853 labeled "no" forming a first training data set.
After the user finishes looking at the visual task, the first judgment model automatically analyzes the data to obtain a result 'yes', wherein the result 'yes' represents that the child notices the specific pinyin in the image matrix in the task, which indicates that the user has the sensitivity to words in the previous letter stage, namely, the child is inferred that the child has no damage to the functional target point in the stage; the result 'no' indicates that the child cannot notice the specific pinyin in the image matrix in the task, and indicates that the child does not have sensitivity to words in the pre-letter stage, namely the possibility that the child has a functional target point damage in the stage is inferred, and the damage target point is as follows: orthographic vision processing disorder.
In one embodiment, since stable feature bundling between the glyphs and the word-phones is gradually formed in the second stage of the child's language cognitive function development, i.e., from the partial letter stage to the complete letter stage, in other words, the processing path of the semantics can be achieved by the glyphs and the word-phones together. In the test, if the character shape and the character sound feature are consistent, the attention channel does not have larger cognitive load, and if the character shape and the character sound are inconsistent, the attention channel has larger cognitive load. The attention transfer paradigm is used to measure cognitive load in children, where inconsistent word sounds and glyphs take up more cognitive load, requiring the child to turn attention to the gaze point on the other side for a length of glance compared to taking up less cognitive load. Thus, a third image set and a fourth image set may be generated from the visual glyph features and the word sound features; here, the third image concentrates on the character pattern feature to coincide with the character sound feature, and the fourth image concentrates on the character pattern feature to not coincide with the character sound feature. And then acquiring a first scanning time length of the user for the third image set and a second scanning time length of the user for the fourth image set through the eye tracker, wherein the scanning time length is the scanning time of the user from one side of the image to the other side of the image. And then generating a second prediction result of the reading disorder of the user according to the first glance time length and the second glance time length. Namely, the eye tracker is used for recording the scanning time length of a user from left to right, judging whether the scanning time length is different in the two situations of consistent and inconsistent fonts according to a T test technology in automatic statistical analysis of the scanning time length, if so, pushing a judging result to be higher in binding compactness, wherein the situation that the children can integrate visual and voice information in the reading process is represented, and the function is free from obstacle; if there is no difference, the pushing result is "binding compactness is low", which means that the child cannot integrate the visual and voice information in the reading process, and the function may be hindered. The tightness of the binding can then be used to predict whether a user is reading impaired.
In one embodiment, a child is able to perform a simple phrase reading because of the relatively stable reading capabilities that they develop during the letter consolidation phase. In phrase reading, when the first character of the phrase is read, the eye center can preview the next character, so that the reading speed is increased. When the first character is read, if the computer technology is used to replace the second character quickly, the reading speed of the child is reduced. Therefore, according to the prediction of whether the user has reading disorder, a fifth image set is firstly required to be randomly generated, and the fifth image contains a preset number of phrases; a first length of time for the user to read the fifth set of images is then acquired by the eye tracker. After the first reading time length is acquired, determining the point coordinates of a user when reading a fifth image by an eye tracker when testing, and changing a second phrase adjacent to the first phrase when the point coordinates are overlapped with the first phrase until the user finishes reading so as to obtain a second reading time length; and generating a third prediction result of the reading disorder of the user according to the first reading time length and the second reading time length. Briefly, a set of phrases of no more than five words is presented on a display screen, the gaze point of the child is captured using an eye tracker, and when the gaze point coincides with the region corresponding to the first word of the phrase, the eye tracker begins calculating time, and the length of reading time from the first word to the last word is defined as the first length of reading time. In the second test, when the first reading time length of the point of regard is coincident with the region corresponding to the first word of the phrase, the computing system automatically replaces the second word, the eye tracker starts to calculate the time, and the reading time length from the first word to the last word is defined as the second reading time length.
In one example, when a third prediction result of the user with reading disorder is generated according to the first reading time length and the second reading time length, a preset support vector machine initial model is first required to be determined, and pre-stored training data is acquired; and training the support vector machine initial model through pre-stored second training data to obtain a second judgment model. And inputting the first reading time length and the second reading time length into a second judging model to judge whether the first reading time length and the second reading time length have the same association relation or not, and if not, causing the user to have reading disorder.
The second training data are a plurality of pre-stored tested users, and two groups of corresponding reading time lengths of the tested users which are unchanged and changed for the fifth image are respectively collected to be used as the second training data. For example, pre-recruited 200 children who have been provided with primary pinyin reading capability perform data collection to build a pre-trained classifier model that requires children in a sample group to complete the visual-cognitive tasks described above and to use the "out-of-sound thinking" instructions to view information in the matrix, such as: "I feel the words changed; or, I do not feel what the text is not powerful. Each child completes 20 visual observation tasks, acquires 4000 pieces of original data in total, and filters and eliminates invalid data in the 4000 pieces of original data to acquire 3602 pieces of valid data. Further establishing the relationship between the NGP, the NFGP and the label in the gazing task, namely reporting the data items of the NGP and the NFGP corresponding to the 'change of the visible characters', and marking the data items as 'yes'; reporting a data item of 'no change in characters', and marking as 'no'; together, 1702 labeled "yes" and 1900 labeled "no" are obtained, forming a second training data set for training the initial support vector machine model. After the child finishes the vision task, the second judgment model automatically analyzes the data to obtain a result that the child can perceive that the characters are replaced in the task, and the visual cognition function is not damaged by the functional target in the progressive reading task; the result of no indicates that the child cannot perceive that the text is replaced in the task, indicating that the visual cognitive function may have target damage in the progressive reading. And after the result is obtained, displaying the information on a screen respectively.
In one embodiment, if the user does not know the operation task of the current flow, the prediction process will take too long time in the current flow or the prediction result will be inaccurate, so that the occupied time of the user in the current flow can be counted, if the occupied time exceeds the preset time threshold of the current flow, the flow progress corresponding to the current user is displayed, and the user is prompted to execute the current flow.
As shown in fig. 3, based on the same inventive concept, the embodiment of the present application further provides a dyskinesia prediction apparatus for young children, including:
at least one processor; the method comprises the steps of,
a memory communicatively coupled to the at least one processor; wherein,
the memory stores instructions executable by the at least one processor to enable the at least one processor to:
acquiring first eye movement track data of a first interest area of a first image set of a user; the first image set is a set of first images including a first region of interest; the region of interest is a region where a preset visual feature is located; the eye movement track data at least comprises the number of fixation points and the first fixation frequency; determining a second image set in the first image set according to the number of fixation points and the first fixation frequency; changing color visual characteristics of the first region of interest in the second set of images to change the first region of interest to a second region of interest; generating a third interest area in each second image, wherein the third interest area comprises color visual characteristics of the first interest area; acquiring second eye movement track data of a user about the second region of interest in a second image set through the eye movement instrument; and generating a first prediction result that the user has dyskinesia according to the first eye movement track data and the second eye movement track data.
The embodiments also provide a non-volatile computer storage medium storing computer executable instructions configured to:
acquiring first eye movement track data of a first interest area of a first image set of a user; the first image set is a set of first images including a first region of interest; the region of interest is a region where a preset visual feature is located; the eye movement track data at least comprises the number of fixation points and the first fixation frequency; determining a second image set in the first image set according to the number of fixation points and the first fixation frequency; changing color visual characteristics of the first region of interest in the second set of images to change the first region of interest to a second region of interest; generating a third interest area in each second image, wherein the third interest area comprises color visual characteristics of the first interest area; acquiring second eye movement track data of a user about the second region of interest in a second image set through the eye movement instrument; and generating a first prediction result that the user has dyskinesia according to the first eye movement track data and the second eye movement track data.
All embodiments in the application are described in a progressive manner, and identical and similar parts of all embodiments are mutually referred, so that each embodiment mainly describes differences from other embodiments. In particular, for the apparatus and medium embodiments, the description is relatively simple, as it is substantially similar to the method embodiments, with reference to the section of the method embodiments being relevant.
The devices and media provided in the embodiments of the present application are in one-to-one correspondence with the methods, so that the devices and media also have similar beneficial technical effects as the corresponding methods, and since the beneficial technical effects of the methods have been described in detail above, the beneficial technical effects of the devices and media are not described in detail herein.
It will be appreciated by those skilled in the art that embodiments of the present application may be provided as a method, system, or computer program product. Accordingly, the present application may take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present application may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and the like) having computer-usable program code embodied therein.
The present application is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems) and computer program products according to embodiments of the application. It will be understood that each flow and/or block of the flowchart illustrations and/or block diagrams, and combinations of flows and/or blocks in the flowchart illustrations and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
In one typical configuration, a computing device includes one or more processors (CPUs), input/output interfaces, network interfaces, and memory.
The memory may include volatile memory in a computer-readable medium, random Access Memory (RAM) and/or nonvolatile memory, such as Read Only Memory (ROM) or flash memory (flash RAM). Memory is an example of computer-readable media.
Computer readable media, including both non-transitory and non-transitory, removable and non-removable media, may implement information storage by any method or technology. The information may be computer readable instructions, data structures, modules of a program, or other data. Examples of storage media for a computer include, but are not limited to, phase change memory (PRAM), static Random Access Memory (SRAM), dynamic Random Access Memory (DRAM), other types of Random Access Memory (RAM), read Only Memory (ROM), electrically Erasable Programmable Read Only Memory (EEPROM), flash memory or other memory technology, compact disc read only memory (CD-ROM), digital Versatile Discs (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices, or any other non-transmission medium, which can be used to store information that can be accessed by a computing device. Computer-readable media, as defined herein, does not include transitory computer-readable media (transmission media), such as modulated data signals and carrier waves.
It should also be noted that the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one … …" does not exclude the presence of other like elements in a process, method, article or apparatus that comprises the element.
The foregoing is merely exemplary of the present application and is not intended to limit the present application. Various modifications and changes may be made to the present application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc. which are within the spirit and principles of the present application are intended to be included within the scope of the claims of the present application.
Claims (7)
1. A reading disorder prediction apparatus for an young child, comprising:
at least one processor; and a memory communicatively coupled to the at least one processor; wherein,
The memory stores instructions executable by the at least one processor to enable the at least one processor to perform:
acquiring first eye movement track data of a first interest area of a first image set of a user; the first image set is a set of first images including a first region of interest; the region of interest is a region where a preset visual feature is located; the eye movement track data at least comprises the number of fixation points and the first fixation frequency;
determining a second image set in the first image set according to the number of fixation points and the first fixation frequency;
changing color visual characteristics of the first region of interest in the second set of images to change the first region of interest to a second region of interest; generating a third interest area in each second image, wherein the third interest area comprises color visual characteristics of the first interest area;
acquiring second eye movement track data of a user about the second interest region in a second image set;
generating a first prediction result of the user having dyskinesia according to the first eye movement track data and the second eye movement track data;
The determining a second image set in the first image set according to the number of fixation points and the first fixation frequency specifically includes:
according to the number of the fixation points and the first fixation frequency, which correspond to each first image in the first image set, respectively, determining a mean value of the number of the fixation points and a mean value of the first fixation frequency, which correspond to the first image set;
according to the gaze point number average value and the first gaze frequency average value, determining standard deviations between the gaze point number corresponding to each first image and the first gaze frequency and the gaze point number average value and the first gaze frequency average value respectively;
selecting the image with the standard deviation larger than a first preset threshold value from the first image set as a second image to generate a second image set;
the generating a first prediction result that the user has dyskinesia according to the first eye movement track data and the second eye movement track data specifically includes:
determining a preset support vector machine initial model, and acquiring pre-stored first training data;
training the support vector machine initial model through the first training data to obtain a first judgment model;
Inputting the first eye movement track data and the second eye movement track data into the first judging model to judge whether the first eye movement track data and the second eye movement track data have the same association relation or not;
if not, the user has a reading disorder.
2. The apparatus of claim 1, wherein after generating the first prediction result that the user has a dysread from the first eye movement trace data and the second eye movement trace data, the apparatus is further configured to:
generating a third image set and a fourth image set according to the character pattern features and the character sound features; the font characteristic in the third image set is consistent with the word sound characteristic, and the font characteristic in the fourth image set is inconsistent with the word sound characteristic;
acquiring a first glance time length of the user for the third image set and a second glance time length of the user for the fourth image set; the saccade time length is the saccade time of the user from one side of the image to the other side;
and generating a second prediction result of the user having dysreading according to the first glance time length and the second glance time length.
3. The apparatus of claim 2, wherein the generating the second prediction result that the user has dyskinesia according to the first glance time length and the second glance time length specifically includes:
t-checking the first glance time length and the second glance time length to judge whether a difference exists between the first glance time length and the second glance time length;
if not, the user has a reading disorder.
4. The apparatus according to claim 1, wherein after generating a first prediction result that the user has a dysread according to the first eye movement track data and the second eye movement track data, a fifth image set is randomly generated, the fifth image set being a set containing a fifth image, the fifth image containing a preset number of phrases;
acquiring a first reading time length of the user for the fifth image set;
determining the coordinates of the annotating point when the user reads the fifth image;
when the annotating point coordinates are overlapped with the first phrase, changing a second phrase adjacent to the first phrase until the user finishes reading so as to obtain a second reading time length;
And generating a third prediction result of the reading disorder of the user according to the first reading time length and the second reading time length.
5. The apparatus of claim 4, wherein the generating a third prediction result that the user has a dysreading according to the first reading time length and the second reading time length specifically includes:
determining a preset support vector machine initial model, and acquiring pre-stored second training data;
training the support vector machine initial model through the second training data to obtain a second judgment model;
inputting the first reading time length and the second reading time length into the second judging model to judge whether the first reading time length and the second reading time length have the same association relation or not;
if not, the user has a reading disorder.
6. The apparatus of claim 1, wherein the apparatus is further configured to:
acquiring the occupied time length of the user in the current flow;
and if the occupied time exceeds a preset time threshold, displaying the user flow progress and prompting the user to currently execute the flow.
7. A non-transitory computer storage medium storing computer-executable instructions, the computer-executable instructions configured to:
acquiring first eye movement track data of a first interest area of a first image set of a user; the first image set is a set of first images including a first region of interest; the region of interest is a region where a preset visual feature is located; the eye movement track data at least comprises the number of fixation points and the first fixation frequency;
determining a second image set in the first image set according to the number of fixation points and the first fixation frequency;
changing color visual characteristics of the first region of interest in the second set of images to change the first region of interest to a second region of interest; generating a third interest area in each second image, wherein the third interest area comprises color visual characteristics of the first interest area;
acquiring second eye movement track data of a user about the second interest region in a second image set;
generating a first prediction result of the user having dyskinesia according to the first eye movement track data and the second eye movement track data;
The determining a second image set in the first image set according to the number of fixation points and the first fixation frequency specifically includes:
according to the number of the fixation points and the first fixation frequency, which correspond to each first image in the first image set, respectively, determining a mean value of the number of the fixation points and a mean value of the first fixation frequency, which correspond to the first image set;
according to the gaze point number average value and the first gaze frequency average value, determining standard deviations between the gaze point number corresponding to each first image and the first gaze frequency and the gaze point number average value and the first gaze frequency average value respectively;
selecting the image with the standard deviation larger than a first preset threshold value from the first image set as a second image to generate a second image set;
the generating a first prediction result that the user has dyskinesia according to the first eye movement track data and the second eye movement track data specifically includes:
determining a preset support vector machine initial model, and acquiring pre-stored first training data;
training the support vector machine initial model through the first training data to obtain a first judgment model;
Inputting the first eye movement track data and the second eye movement track data into the first judging model to judge whether the first eye movement track data and the second eye movement track data have the same association relation or not;
if not, the user has a reading disorder.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210997976.5A CN115363581B (en) | 2022-08-19 | 2022-08-19 | Method, equipment and medium for predicting dysreading for young children |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210997976.5A CN115363581B (en) | 2022-08-19 | 2022-08-19 | Method, equipment and medium for predicting dysreading for young children |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN115363581A CN115363581A (en) | 2022-11-22 |
| CN115363581B true CN115363581B (en) | 2023-05-05 |
Family
ID=84066136
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210997976.5A Active CN115363581B (en) | 2022-08-19 | 2022-08-19 | Method, equipment and medium for predicting dysreading for young children |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN115363581B (en) |
Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US4838681A (en) * | 1986-01-28 | 1989-06-13 | George Pavlidis | Method and means for detecting dyslexia |
| US4889422A (en) * | 1986-01-28 | 1989-12-26 | George Pavlidis | Method and means for detecting dyslexia |
| WO2001060233A2 (en) * | 2000-02-14 | 2001-08-23 | Levinson Harold N | Network-based detection of dyslexia and cvs disorders |
| CN110464365A (en) * | 2018-05-10 | 2019-11-19 | 深圳先进技术研究院 | A kind of attention rate determines method, apparatus, equipment and storage medium |
| CN113506274A (en) * | 2021-07-15 | 2021-10-15 | 中科人工智能创新技术研究院(青岛)有限公司 | Detection system for human cognitive condition based on visual saliency difference map |
| CN113805695A (en) * | 2021-08-26 | 2021-12-17 | 东北大学 | Reading understanding level prediction method and device, electronic equipment and storage medium |
| CN113842113A (en) * | 2021-07-22 | 2021-12-28 | 陆烁 | Developing reading disorder intelligent identification method, system, equipment and storage medium |
Family Cites Families (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2007070876A2 (en) * | 2005-12-15 | 2007-06-21 | Posit Science Corporation | Cognitive training using visual stimuli |
| EP2334226A4 (en) * | 2008-10-14 | 2012-01-18 | Univ Ohio | Cognitive and linguistic assessment using eye tracking |
| US11334803B2 (en) * | 2016-04-20 | 2022-05-17 | Carnegie Mellon University | Data processing system to detect neurodevelopmental-specific learning disorders |
-
2022
- 2022-08-19 CN CN202210997976.5A patent/CN115363581B/en active Active
Patent Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US4838681A (en) * | 1986-01-28 | 1989-06-13 | George Pavlidis | Method and means for detecting dyslexia |
| US4889422A (en) * | 1986-01-28 | 1989-12-26 | George Pavlidis | Method and means for detecting dyslexia |
| WO2001060233A2 (en) * | 2000-02-14 | 2001-08-23 | Levinson Harold N | Network-based detection of dyslexia and cvs disorders |
| CN110464365A (en) * | 2018-05-10 | 2019-11-19 | 深圳先进技术研究院 | A kind of attention rate determines method, apparatus, equipment and storage medium |
| CN113506274A (en) * | 2021-07-15 | 2021-10-15 | 中科人工智能创新技术研究院(青岛)有限公司 | Detection system for human cognitive condition based on visual saliency difference map |
| CN113842113A (en) * | 2021-07-22 | 2021-12-28 | 陆烁 | Developing reading disorder intelligent identification method, system, equipment and storage medium |
| CN113805695A (en) * | 2021-08-26 | 2021-12-17 | 东北大学 | Reading understanding level prediction method and device, electronic equipment and storage medium |
Non-Patent Citations (1)
| Title |
|---|
| 王海英 等.与视觉相关的阅读障碍.《中国眼镜科技杂志》.2020,第99-101页. * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN115363581A (en) | 2022-11-22 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US8814357B2 (en) | System and method for identifying the existence and position of text in visual media content and for determining a subject's interactions with the text | |
| Majerus et al. | The dorsal attention network reflects both encoding load and top–down control during working memory | |
| Ericsson et al. | Verbal reports on thinking | |
| Stein et al. | Content-specific expectations enhance stimulus detectability by increasing perceptual sensitivity. | |
| Heisz et al. | More efficient scanning for familiar faces | |
| Brockmole et al. | Short article: Recognition and attention guidance during contextual cueing in real-world scenes: Evidence from eye movements | |
| Cohen et al. | The neuropsychology of attention | |
| KR102262889B1 (en) | Apparatus and method for diagnosis of reading ability based on machine learning using eye tracking | |
| Arbel et al. | On the utility of positive and negative feedback in a paired-associate learning task | |
| Shanks et al. | Are there multiple memory systems? Tests of models of implicit and explicit memory | |
| Król et al. | A novel machine learning analysis of eye-tracking data reveals suboptimal visual information extraction from facial stimuli in individuals with autism | |
| KR102262890B1 (en) | Reading ability improvement training apparatus for providing training service to improve reading ability in connection with reading ability diagnosis apparatus based on eye tracking and apparatus for providing service comprising the same | |
| CN114209324A (en) | Psychological assessment data acquisition method based on image visual cognition and VR system | |
| CN115444423A (en) | Prediction system, prediction method, prediction device, prediction equipment and storage medium | |
| Panetta et al. | Software architecture for automating cognitive science eye-tracking data analysis and object annotation | |
| Vergauwe et al. | Evidence for spontaneous serial refreshing in verbal working memory? | |
| Winter et al. | Defining gaze patterns for process model literacy–Exploring visual routines in process models with diverse mappings | |
| Gong et al. | Emotional valence of stimuli modulates false recognition: Using a modified version of the simplified conjoint recognition paradigm | |
| Moshel et al. | Are you for real? Decoding realistic AI-generated faces from neural activity | |
| Megla et al. | Induced forgetting is the result of true forgetting, not shifts in decision-making thresholds | |
| CN115363581B (en) | Method, equipment and medium for predicting dysreading for young children | |
| Mayo et al. | How hard are computer vision datasets? Calibrating dataset difficulty to viewing time | |
| Kelley et al. | Delayed judgments of learning are associated with activation of information from past experiences: A neurobiological examination | |
| Matthews et al. | Sustained conscious access to incidental memories in RSVP | |
| König et al. | Modeling visual exploration in rhesus macaques with bottom-up salience and oculomotor statistics |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |