WO2021109855A1 - 一种基于深度学习的孤独症辅助评估系统和方法 - Google Patents

一种基于深度学习的孤独症辅助评估系统和方法 Download PDF

Info

Publication number
WO2021109855A1
WO2021109855A1 PCT/CN2020/129160 CN2020129160W WO2021109855A1 WO 2021109855 A1 WO2021109855 A1 WO 2021109855A1 CN 2020129160 W CN2020129160 W CN 2020129160W WO 2021109855 A1 WO2021109855 A1 WO 2021109855A1
Authority
WO
WIPO (PCT)
Prior art keywords
neural network
layer
classification result
map
autism
Prior art date
Application number
PCT/CN2020/129160
Other languages
English (en)
French (fr)
Inventor
连重源
燕楠
王岚
Original Assignee
中国科学院深圳先进技术研究院
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 中国科学院深圳先进技术研究院 filed Critical 中国科学院深圳先进技术研究院
Publication of WO2021109855A1 publication Critical patent/WO2021109855A1/zh

Links

Classifications

    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00—Measuring for diagnostic purposes; Identification of persons
    • A61B5/16—Devices for psychotechnics; Testing reaction times ; Devices for evaluating the psychological state
    • A61B5/165—Evaluating the state of mind, e.g. depression, anxiety
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B3/00—Apparatus for testing the eyes; Instruments for examining the eyes
    • A61B3/10—Objective types, i.e. instruments for examining the eyes independent of the patients' perceptions or reactions
    • A61B3/113—Objective types, i.e. instruments for examining the eyes independent of the patients' perceptions or reactions for determining or recording eye movement
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00—Measuring for diagnostic purposes; Identification of persons
    • A61B5/16—Devices for psychotechnics; Testing reaction times ; Devices for evaluating the psychological state
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00—Measuring for diagnostic purposes; Identification of persons
    • A61B5/72—Signal processing specially adapted for physiological signals or for diagnostic purposes
    • A61B5/7235—Details of waveform analysis
    • A61B5/7264—Classification of physiological signals or data, e.g. using neural networks, statistical classifiers, expert systems or fuzzy systems
    • A61B5/7267—Classification of physiological signals or data, e.g. using neural networks, statistical classifiers, expert systems or fuzzy systems involving training the classification device

Definitions

  • the present invention relates to the technical field of autism assessment, in particular to an auxiliary assessment system and method for autism based on deep learning.
  • Autism spectrum disorder Disorder ASD
  • ASD Autism spectrum disorder Disorder
  • ASD Autism spectrum disorder Disorder
  • ASD Autism spectrum disorder Disorder
  • ASD Autism spectrum disorder Disorder
  • ASD Autism spectrum disorder Disorder
  • the diagnostic criteria for children with ASD mainly include the Diagnostic and Statistical Manual of Mental Disorders of the American Psychiatric Association and the diagnostic criteria for mental and behavioral disorders of the World Health Organization.
  • the current mainstream assessment and prediction methods are based on diagnostic criteria, aiming at the three aspects of language communication barriers, social communication barriers and repetitive stereotypes, using the mainstream international diagnostic tools (Autism Diagnostic Observation Scale and Autism Diagnostic Interview Scale). Autism diagnosis was made by combining questionnaire surveys and interviews with the children’s usual growth and development history, medical history, and mental examination. When direct or indirect observation based on behavioral signs and symptoms is no longer applicable, from the perspective of social cognition, the analysis of the visual characteristics of ASD around social disorders, which is the core specific symptom of ASD, is of special significance.
  • autism testers to watch the facial images corresponding to the pre-made video and the changes in facial temperature, heart rate, and breathing when viewing the images; or use a visual camera to assist in judging the subject’s response to the language; or during the process of collecting funny laughs
  • Multi-channel audio and video multi-modal data from multiple RGB-D camera perspectives of subjects, evaluators, and props; or extract the subjects’ brain electrical signals, electromyographic signals, and eye electrical signals in different emotional states based on physiological signals Signals, galvanic skin response signals, body temperature data, respiratory rate, etc., and even directly use the subject’s expression response to extract expression features to assist in judgment.
  • a variety of available multi-modal signals and features have been tried to assist in the diagnosis of children with autism.
  • the ASD eye movement data is used to classify and assist the diagnosis of autism, usually watching static face pictures or static pictures in visual following tasks, with the help of eye trackers or wearing glasses.
  • Eye tracker to obtain the eye movement data or eye gaze data of the observer.
  • SVM machine learning classification model support vector machine
  • BP BP neural network training
  • Facial recognition and emotional perception disorders are the core problems of social disorders in children with ASD, and facial emotion recognition defects that are common in patients with autism spectrum disorder are the core reasons for their social and communication disorders.
  • the eye movement patterns of patients with autism spectrum disorder are significantly different from those of the typical developing population. Patients with autism spectrum disorder pay less attention to the eye area, and avoiding direct gaze may be patients with autism spectrum disorder Reasons for the development of facial emotion recognition defects.
  • the current research on facial emotion recognition disorder of patients with autism spectrum disorder is mainly eye tracking technology. Studies have shown that ASD children have different eye movement patterns in face recognition and emotion perception than normal children. This feature exists in ASD children with mild to severe symptoms.
  • the specific facial processing mode of patients with autism spectrum disorder is mainly to select facial stimuli and extract information, which is sub-optimal for the task of emotion recognition.
  • the existing technical solutions mainly have the following problems: 1)
  • the stimulus material of the eye movement-based solution is too typical and out of touch with real life scenes.
  • the stimulus materials provided are all static pictures, which are out of touch with real life scenes, and cannot truly evaluate the emotion recognition ability of ASD children in real social communication and interaction.
  • Most of the current Autism Spectrum Disorder Scales have age ranges. For autistic patients, there are often problems with difficult questions or age that is not within the scope of the scale.
  • the purpose of the present invention is to overcome the above-mentioned shortcomings of the prior art, and provide a deep learning-based auxiliary assessment system and method for autism, which combines eye movement technology and deep learning to predict and assess autism.
  • an auxiliary evaluation system for autism based on deep learning includes a data acquisition and feature extraction unit, a first neural network, a second neural network, a third neural network, and a result output unit.
  • the data acquisition and feature extraction unit and the result output unit are respectively connected to the first neural network.
  • the network, the second neural network, and the third neural network have a communication connection, wherein: the data collection and feature extraction unit is used to collect eye movement data of the subject watching the video to obtain the corresponding heat map and focus map And a scan path map, the heat map is used to characterize the time and position of the fixation point dynamic changes, the focus map is used to characterize the fixation position, the dynamic change of time, the path scan chart continuously displays the fixation point position and Each gaze time information; the first neural network is used to input the heat map to obtain a first classification result; the second neural network is used to input the focus map to obtain a second classification result; the third neural network The network is used to input the scanning path graph to obtain a third classification result; the result output unit gathers the first classification result, the second classification result, and the third classification result to obtain the subject’s autism detection result.
  • the first neural network, the second neural network, and the third neural network have the same or different structures.
  • the first neural network, the second neural network, and the third neural network have the same structure, including an input layer, a first convolutional layer, a second pooling layer, and a first Three layers of convolutional layer, fourth layer of pooling layer, fifth layer of convolution layer, sixth layer of fully connected layer, seventh layer of fully connected layer and output layer.
  • the first neural network, the second neural network, and the third neural network have a first convolutional layer, a second pooling layer, a third convolutional layer, and a fourth neural network.
  • the activation function of the layer pooling layer, the fifth convolutional layer, and the sixth fully connected layer is the ReLU nonlinear activation function
  • the activation function of the seventh fully connected layer is the Softmax activation function
  • the number of neurons in the output layer is 4
  • the result output unit uses a simple voting method to combine the first classification result, the second classification result, and the third classification result to give a final prediction result.
  • an eye tracker is used to collect eye movement data for each subject in a non-invasive manner.
  • the heat map displays the dynamic changes of the time and position of the fixation point with warm chromaticity
  • the focus map displays the dynamic changes of the fixation position and time with the brightness
  • an auxiliary evaluation method for autism based on deep learning includes the following steps:
  • the heat map is used to characterize the dynamic changes of the time and position of the fixation point
  • the focus map is used to characterize the fixation position.
  • the dynamic change of time, the path scanning diagram continuously displays the location of the gaze point and the information of each gaze time point by point;
  • the distribution of the heat map, the focus map, and the scan path map are input to the trained first neural network, the second neural network, and the third neural network to obtain the first classification result, the second classification result, and the second classification result, respectively.
  • the first classification result, the second classification result, and the third classification result are assembled to obtain the subject's autism detection result.
  • the present invention has the advantages of: starting from the common facial emotion recognition defect in ASD patients, aiming at the research goal of ASD eye movement screening, compared with static picture stimulation, it adopts dynamic and daily speech
  • the expressed scene is used as the stimulus material.
  • the stimulus design of the dynamic video Through the stimulus design of the dynamic video, the emotion recognition response and the eye movement data of the subject in the real social interaction are extracted, thereby improving the true reliability of the auxiliary diagnosis; because the eye movement technology is non-invasive
  • the subject does not need to wear any device, and the stimulating material can be appropriately adjusted according to the age of the subject, which can be suitable for autistic patients of different ages and different developmental levels, especially for children from 6 to 18 months of age.
  • Fig. 1 is a schematic diagram of a deep learning-based autism auxiliary assessment system according to an embodiment of the present invention
  • Fig. 2 is a structure diagram of a neural network according to an embodiment of the present invention.
  • Facial emotion recognition defects are common in patients with autism spectrum disorder, which is the core cause of their social and communication disorders.
  • the current research on facial emotion recognition disorders in patients with autism spectrum disorder mainly involves eye tracking technology. Under the conditions of social and facial stimulation, the eye movement patterns of patients with autism spectrum disorder are significantly different from those of the typical developing population. Patients with autism spectrum disorder pay less attention to the eye area, and avoid direct gaze on the face of patients with autism spectrum disorder. Reasons for the development of emotion recognition defects. There is evidence that the high levels of arousal associated with direct gaze in patients with autism spectrum disorder are related to avoiding direct gaze and are related to more severe impairments in social skills.
  • the specific facial processing mode of patients with autism spectrum disorder is mainly to select facial stimuli and extract information, which is sub-optimal for the task of emotion recognition.
  • the present invention provides an efficient, convenient, non-invasive, and low-cost ASD auxiliary diagnosis method based on eye movement technology for face recognition and emotion perception tasks based on the association between eye movement data characteristics and patients with autism spectrum disorder.
  • the embodiment of the present invention includes the following steps: guide the subject to complete the video viewing task during the data collection process, require the subject to focus on what he hears and see, and use it when the subject is watching the short video.
  • the eye tracker records its eye movement data; for the obtained eye movement image data, the data preprocessing is completed; then, combined with the convolutional neural network (Convolutional Neural Network) in the deep learning method Networks, CNN) perform automatic feature extraction and obtain neural network classifiers through model training, and finally achieve auxiliary diagnosis of ASD.
  • the present invention combines eye tracking data with a deep learning algorithm to effectively extract the specific facial processing mode of autistic patients, thereby realizing auxiliary diagnosis for patients with mild to moderate autism.
  • the autism auxiliary assessment system based on deep learning includes a data collection and feature extraction unit 110, a neural network for classification training, and a result output unit 120 that are sequentially connected, wherein , Shows neural network 1, neural network 2, and neural network 3.
  • the present invention mainly includes three processes, namely data collection and feature extraction, classifier training and result prediction, which will be introduced in detail below.
  • the present invention uses an eye tracker that is convenient for young children and does not require high professionalism for doctors to collect eye movement data with dynamic daily life scenes as stimulus materials for classification.
  • the dynamic video emotion stimulus is selected in the data collection process of the present invention.
  • static image stimuli the use of dynamic video stimuli with daily verbal expressions is more suitable for daily life scenes, and is more able to truly assess the emotion recognition ability of ASD children, because the communication that occurs in real social interactions cannot always be It is well aware, and can dynamically adjust the stimulus materials according to the different age groups of the subjects in order to be suitable for children of different age groups.
  • the stimulus material in the data collection process is composed of the Chinese Natural Emotional Audiovisual Database (Chinese Natural Emotional Audiovisual Database), which aims to provide Chinese resources for the study of multimodal and multimedia interactions.
  • Audio-Visual Database CHEAVD
  • CHEAVD provides 20 movie clips video composition.
  • six typical and relatively complete emotional stimulation videos are selected for analysis. They are composed of three positive emotional videos and three negative emotional videos. The duration of the video is between 3 seconds and 9 seconds.
  • the RED250 eye tracker from SMI of Germany was used to collect the eye movement data of each subject in a non-invasive manner.
  • the device has been integrated into a 22-inch widescreen display panel with a resolution of 1280*1024 pixels.
  • the sampling frequency is 60 Hz, and the accuracy is 0.4 degrees.
  • the subject’s head movement freedom is 40*20 cm at a distance of 70 cm.
  • the experiment design software Experiment Center of SMI company of Germany was used for online eye movement data recording, and the SMI data analysis software BeGaze was used for offline data analysis.
  • the Samsung tablet ST800 Before formally conducting the experimental task, first train the subjects to understand the task in this experiment. After understanding the task, use the Samsung tablet ST800 to conduct pre-experiment to ensure that the experimenter is familiar with the entire process of the experiment. Then ask the participant to sit about 60-80 cm away from the test screen until the eye tracker can stably detect the participant’s pupils. During the experiment, participants cannot be intervened to avoid any attentional bias.
  • first perform a five-point calibration of eye movements the subject is asked to look at the four corners of the screen and the calibration point in the middle one by one.
  • the calibration part is counted only when the average error of all five calibration points does not exceed 1 degree of viewing angle. by.
  • 20 test videos were played randomly. Participants were allowed to watch each video multiple times because they did not understand the content displayed in it.
  • the data analysis software SMI BeGaze processed the output to obtain the heat map, focus map, and scan path of each video clip that the subjects watched.
  • the heat map for example, displays the dynamic change of the time and position of the fixation point in warm chromaticity, that is, the closer to the color on the right side of the color bar in the data analysis software, the longer the time to fix the area.
  • the scan path map continuously displays information such as the location of the gaze point and each gaze time, for example, point by point.
  • the focus map displays the dynamic changes of the gaze position and time with brightness, for example.
  • neural network training and learning are used to obtain a neural network classifier for predicting and evaluating autism.
  • the embodiment of the present invention adopts a convolutional neural network and uses a design similar to the LeNet structure.
  • the structure of the entire neural network includes an input layer (input), three convolutional layers (ie conv1, conv2, conv3), and 2 Maximum pooling layer (namely max pooling1, max pooling2), 2-layer fully connected layer (namely fc1 and fc2) and output layer (output).
  • the first is the data input layer.
  • the input images are heat maps, focus maps, and scan path maps obtained by analysis software of eye movement data, and the size of the input images is uniformly normalized to 1024*1024.
  • the first layer is the convolutional layer
  • the second layer is the maximum pooling layer
  • the third layer is the convolutional layer
  • the fourth layer is the maximum pooling layer
  • the fifth layer is the convolutional layer
  • the sixth and seventh layers are Fully connected layer
  • the eighth layer is the output layer.
  • the activation function of the first six layers is the ReLU nonlinear activation function
  • the activation function of the seventh layer is the Softmax activation function.
  • the number of neurons in the output layer can be set to 4, corresponding to healthy, mild autism symptoms, and moderate autism. There are four categories of closed symptoms and severe autism symptoms.
  • the training of the neural network is used for the auxiliary diagnosis of ASD. Specifically, set the number of nodes in the input layer, hidden layer and output layer neurons and the size of the convolution kernel, and initialize the weight matrix randomly, including the input layer to the hidden layer, the hidden layer to the hidden layer, and the hidden layer to the output layer.
  • the weight matrix between. Input the heat map, focus map, and scan path map of each video clip's eye movement data as the input of the neural network.
  • the neural network is trained according to the forward and back propagation algorithms and the gradient descent method.
  • the weight matrix is the cross-entropy loss function.
  • a cross-validation method is used, and the confusion matrix of the classification result is used according to the receiver operating characteristic curve (Receiver Operating Characteristic Curve).
  • Characteristic, ROC Receiveiver Operating Characteristic Curve
  • AUC rea Under ROC Curve
  • the three convolutional neural network classifiers take the heat map, focus map and scanning path map of the eye tracking data as input, and the output corresponds to the healthy and mild self. There are four categories of autism symptoms, moderate autism symptoms and severe autism symptoms. Then, according to the predicted output of the three classifiers, a simple voting method is used to combine to give the final prediction result. The final four results are still one of the above four categories.
  • the neural networks involved in the embodiments of the present invention can have the same or different structures. For example, more convolutional layers and fully connected layers can be used, and either average pooling or maximum pooling can be used, and The classification results are not limited to the above four categories.
  • the neural network and the result output unit can be implemented by software or hardware, such as a hardware processor or logic circuit.
  • the AOI area of interest
  • AOI aims to measure the areas of interest that the eyes are looking at, usually including the eyes, nose, and mouth, and then count the frequency and time of the eyes looking at these areas.
  • children with ASD have relatively less fixation time and number of fixations in AOI compared to normal children. More specifically, children with ASD spend far more time looking at their bodies and objects than looking at their eyes.
  • the eye movement data The heat map analysis and focus map analysis illustrate the same problem.
  • the present invention performs auxiliary diagnosis of ASD based on different eye movement patterns.
  • the present invention aims at the shortcomings in the current ASD auxiliary diagnosis technology and the main problems in the ASD screening assessment and clinical application, and establishes an efficient, convenient, easy to implement and popularize and suitable for the auxiliary diagnosis of ASD in young children.
  • Technical solutions aims at the shortcomings in the current ASD auxiliary diagnosis technology and the main problems in the ASD screening assessment and clinical application, and establishes an efficient, convenient, easy to implement and popularize and suitable for the auxiliary diagnosis of ASD in young children.
  • the present invention may be a system, a method and/or a computer program product.
  • the computer program product may include a computer-readable storage medium loaded with computer-readable program instructions for enabling a processor to implement various aspects of the present invention.
  • the computer-readable storage medium may be a tangible device that holds and stores instructions used by the instruction execution device.
  • the computer-readable storage medium may include, but is not limited to, an electrical storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination of the foregoing, for example.
  • Computer-readable storage media include: portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM) Or flash memory), static random access memory (SRAM), portable compact disk read-only memory (CD-ROM), digital versatile disk (DVD), memory stick, floppy disk, mechanical encoding device, such as a printer with instructions stored thereon
  • RAM random access memory
  • ROM read-only memory
  • EPROM erasable programmable read-only memory
  • flash memory flash memory
  • SRAM static random access memory
  • CD-ROM compact disk read-only memory
  • DVD digital versatile disk
  • memory stick floppy disk
  • mechanical encoding device such as a printer with instructions stored thereon

Landscapes

  • Health & Medical Sciences (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Veterinary Medicine (AREA)
  • Public Health (AREA)
  • General Health & Medical Sciences (AREA)
  • Animal Behavior & Ethology (AREA)
  • Surgery (AREA)
  • Medical Informatics (AREA)
  • Biophysics (AREA)
  • Molecular Biology (AREA)
  • Biomedical Technology (AREA)
  • Heart & Thoracic Surgery (AREA)
  • Psychiatry (AREA)
  • Artificial Intelligence (AREA)
  • Pathology (AREA)
  • Child & Adolescent Psychology (AREA)
  • Ophthalmology & Optometry (AREA)
  • Hospice & Palliative Care (AREA)
  • Educational Technology (AREA)
  • Developmental Disabilities (AREA)
  • Human Computer Interaction (AREA)
  • Psychology (AREA)
  • Social Psychology (AREA)
  • Evolutionary Computation (AREA)
  • Fuzzy Systems (AREA)
  • Mathematical Physics (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Physiology (AREA)
  • Signal Processing (AREA)
  • Eye Examination Apparatus (AREA)

Abstract

一种基于深度学习的孤独症辅助评估系统和方法，基于深度学习的孤独症辅助评估系统包括数据采集和特征提取单元（110）、第一神经网络（1）、第二神经网络（2）、第三神经网络（3）和结果输出单元（120），数据采集和特征提取单元（110）以及结果输出单元（120）分别与第一神经网络（1）、第二神经网络（2）、第三神经网络（3）具有连接，数据采集和特征提取单元（110）用于采集受试者观看视频的眼动数据，获得热点图、焦点图和扫描路径图；第一神经网络（1）输入热点图，获得第一分类结果；第二神经网络（2）输入焦点图，获得第二分类结果；第三神经网络（3）输入扫描路径图，获得第三分类结果；结果输出单元（120）集合第一分类结果、第二分类结果和第三分类结果获得受试者的孤独症检测结果。基于深度学习的孤独症辅助评估系统和方法提高了孤独症的预测效率和预测准确性。

Description

一种基于深度学习的孤独症辅助评估系统和方法 技术领域
本发明涉及孤独症评估技术领域，尤其涉及一种基于深度学习的孤独症辅助评估系统和方法。
 背景技术
孤独症谱系障碍（autism spectrum disorder，ASD），或称“自闭症谱系障碍”、“孤独症”、“自闭症”是以社交障碍和刻板行为为核心障碍的异质性神经发育障碍。全世界范围内ASD儿童发病率逐年增加，现已成为社会公共卫生问题。根据美国疾病控制与预防中心下属的孤独症和发育障碍监控网络的估计，在美国每68人中就有1人患上孤独症，因此现今社会对ASD的关注在过去几年大幅上升。康复训练是ASD儿童主要的治疗方式，越来越多的实例表明，开始干预的时间越早，预后越好。通过对ASD儿童的早期诊断和干预，借以减轻ASD儿童症状，能够最大程度发挥其潜能，提高其相应功能水平，使其能够和正常儿童一样健康茁壮成长。
然而，目前ASD的诊断缺乏可靠的生化诊断标志物或影像学依据，异质性大。ASD的核心症状社会交流和互动障碍受损程度不同，表现多样，尤其轻-中度ASD儿童的及时识别及正确诊断存在一定的难度，容易漏诊、误诊，使ASD儿童进入主流托幼机构后被贴上“问题儿童”的标签，易受到家长的责怪、老师的责罚和同学的疏远，引发行为、情绪问题，非常不利于孩子的预后。故急需使用方便、可靠性高的方法进行辅助诊断，从而有利于早期正确诊断和及时干预，减轻越来越沉重的孤独症谱系障碍患儿康复费用负担。
ASD儿童的诊断标准主要有美国精神病学会的《精神障碍诊断和统计手册》和世界卫生组织关于精神与行为障碍的诊断标准。目前主流的评估预测方法是根据诊断标准，针对语言交流障碍、社会交往障碍和重复刻板行为三个方面，利用国际上主流的诊断工具（孤独症诊断观察量表和孤独症诊断访谈量表），以问卷调查和访谈的方式结合儿童的平时生长发育史、病史以及精神检查进行孤独症诊断。当基于行为征兆和症状的直接或间接观察不再适用时，从社会认知视角出发，围绕社交障碍这一ASD核心特异性症状，对ASD视觉特点的分析就富有特殊的意义。最近的研究提供了ASD个体不同的眼动模式的证据，通过眼球追踪技术研究了ASD个体如何扫描人脸。这些研究一致认为孤独症个体与正常儿童相比，对面部有更少的视觉注意。目前医学领域已经开始研究眼球凝视行为并将其作为儿童自闭症的诊断标准之一。从该角度出发，形成了一类借助眼动仪追踪ASD的眼球凝视行为，通过ASD的眼动数据进行分析及特征提取从而进行自闭症的评估预测。另一方面，随着脑功能成像技术研究的发展，出现了部分借助脑功能成像技术对ASD进行诊断的方法。通过核磁共振扫描仪得到静息态功能磁共振数据或者借助脑电设备采集得到的脑电信号，对采集得到的数据进行预处理和相应的功能处理得到所需的特征用以训练分类模型进行自闭症和正常儿童的分类。此外，越来越多的研究用工程技术的手段通过来自各个方面的多模态信号提取相关的特征用于自闭症的评估预测。例如，通过自闭症测试者观看预制视频对应的面部图像以及观看图像时的面部温度变化、心率变化和呼吸变化；或者利用视觉摄像头辅助判断受试者对语言的回应；或者采集逗笑试验过程中被试者、评估者和道具的多个RGB-D摄像头视角多声道音视频多模态数据；或者基于生理信号提取受试者在不同情绪状态下的脑电信号、肌电信号、眼电信号、皮肤电反应信号、体温数据、呼吸频率等，甚至直接利用受试者的表情反应提取表情特征来进行辅助判断。总之，多种能够利用的多模态信号及特征都已经被尝试用来进行自闭症儿童的辅助诊断。
在现有技术中，以ASD的眼动数据进行自闭症的分类辅助诊断方法中，通常以观看静态的人脸图片或视觉追随任务中的静态图片为主，借助眼动仪或者佩戴眼镜式眼球追踪器来获取被观察者的眼动数据或眼球凝视数据。并以此进行繁杂的人工特征提取工作辅以传统机器学习分类模型支持向量机（SVM）或者BP神经网络的训练得到性能较优的分类器模型，根据建立的分类模型对儿童进行分类识别。在利用眼动追踪技术进行辅助诊断的方案中大多以简单的静态面孔识别为主要研究方向，而基于面部情绪识别实现ASD患儿的辅助诊断的报道少之又少。面孔识别以及情绪感知障碍是ASD儿童社交障碍的核心问题，而孤独症谱系障碍患者普遍存在的面部情绪识别缺陷是导致其社交、沟通障碍的核心原因。孤独症谱系障碍患者在社交和面部刺激条件下，与典型发展人群的眼球运动模式存在着显著性差异，孤独症谱系障碍患者对眼睛区域的注视更少，避免直接凝视可能是孤独症谱系障碍患者面部情绪识别缺陷发展的原因。目前孤独症谱系障碍患者的面部情绪识别障碍的研究主要是眼动追踪技术。研究表明ASD儿童在面孔识别与情绪感知中存在异于正常儿童的眼动模式，这一特征存在于症状从轻到重的ASD儿童中。孤独症谱系障碍患者的特异性面部加工模式，主要是对面部刺激进行选择并提取信息，这些信息对于情绪识别的任务来说是次优的。
总之，现有的技术方案主要存在以下问题：1）、基于眼动的方案刺激材料过于典型，与实际生活场景脱节。在目前的基于眼动辅助诊断的方案中，提供的刺激材料皆是静态图片，与实际生活场景脱节，无法真实评估ASD儿童在现实社会交流互动中的情感识别能力。2）、不适用于6-18月年龄段儿童的辅助诊断。当前的自闭症谱系障碍量表多有年龄范围，对于自闭症患者而言，经常会出现题目难度大或者年龄不在量表评估范围的问题。而且对于年龄偏小的儿童，无法对其问卷调查和访谈，只能凭借父母长辈的日常观察和医生的访谈及行为核查借以诊断，容易造成漏诊、误诊。目前的用眼镜式眼球追踪器获取眼动数据、利用脑电等多模态数据进行数据采集的方法以及利用表情反应的方法对于6-18个月年龄段的儿童而言太难实现。3）、需要繁琐的人工特征提取工作。当前用于ASD辅助诊断的相关工程技术方案，无论是采取眼动追踪技术获取受试者的眼动数据，还是获取受试者在面孔识别相关任务中的面部特征，甚至是通过获取受试者的脑电信号、肌电信号等多模态信号，都是通过繁杂的人工特征提取工作后进行分类器模块的训练进行诊断分类。对所获得的不同形态的数据必须通过精细繁杂的特征提取才能为后期高准确的分类性能服务。
 技术问题
本发明的目的在于克服上述现有技术的缺陷，提供一种基于深度学习的孤独症辅助评估系统和方法，结合眼动技术和深度学习来预测评估孤独症。
 技术解决方案
根据本发明的第一方面，提供一种基于深度学习的孤独症辅助评估系统。该系统包括数据采集和特征提取单元、第一神经网络、第二神经网络、第三神经网络和结果输出单元，所述数据采集和特征提取单元和所述结果输出单元分别与所述第一神经网络、所述第二神经网络、所述第三神经网络具有通信连接，其中：所述数据采集和特征提取单元用于采集受试者观看视频的眼动数据，获得对应的热点图、焦点图和扫描路径图，所述热点图用于表征注视点的时间和位置的动态变化，所述焦点图用于表征注视位置、时间的动态变化，所述路径扫描图逐点连续显示注视点位置和各注视时间信息；所述第一神经网络用于输入所述热点图，获得第一分类结果；所述第二神经网络用于输入所述焦点图，获得第二分类结果；所述第三神经网络用于输入所述扫描路径图，获得第三分类结果；所述结果输出单元集合所述第一分类结果、所述第二分类结果和所述第三分类结果获得受试者的孤独症检测结果。
在一个实施例中，所述第一神经网络、所述第二神经网络和所述第三神经网络具有相同或不同的结构。
在一个实施例中，所述第一神经网络、所述第二神经网络和所述第三神经网络具有相同的结构，包括输入层、第一层卷积层，第二层池化层，第三层卷积层，第四层池化层，第五层卷积层，第六层全连接层、第七层全连接层和输出层。
在一个实施例中，所述第一神经网络、所述第二神经网络和所述第三神经网络的第一层卷积层、第二层池化层、第三层卷积层、第四层池化层、第五层卷积层、第六层全连接层的激活函数为ReLU非线性激活函数，第七层全连接层的激活函数为Softmax激活函数，输出层的神经元数目为4个，分别对应健康、轻度自闭症症状、中度自闭症症状和重度自闭症症状四个类别。
在一个实施例中，所述结果输出单元使用简单投票法结合所述第一分类结果、所述第二分类结果和所述第三分类结果给出最终的预测结果。
在一个实施例中，使用眼动仪以非侵入方式收集每个受试者的眼动数据。
在一个实施例中，所述热点图以颜色暖色度来显示注视点的时间和位置的动态变化，所述焦点图以亮度显示注视位置、时间的动态变化。
根据本发明的第二方面，提供一种基于深度学习的孤独症辅助评估方法。该方法包括以下步骤：
采集受试者观看视频的眼动数据，获得对应的热点图、焦点图和扫描路径图，所述热点图用于表征注视点的时间和位置的动态变化，所述焦点图用于表征注视位置、时间的动态变化，所述路径扫描图逐点连续显示注视点位置和各注视时间信息；
将所述热点图、所述焦点图和所述扫描路径图分布输入到经训练的第一神经网络、第二神经网络和第三神经网络，分别获得第一分类结果、第二分类结果和第三分类结果；
集合所述第一分类结果、所述第二分类结果和所述第三分类结果获得受试者的孤独症检测结果。
 有益效果
与现有技术相比，本发明的优点在于：从ASD患者普遍存在的面部情绪识别缺陷出发，针对ASD眼动筛查这一研究目标，相较于静态图片刺激，采取动态的、具有日常言语表达的场景作为刺激材料，通过动态视频的刺激设计，提取受试者在真实社交互动中情绪识别反应及其眼动数据，从而提高辅助诊断的真实可靠性；由于眼动技术是非侵入式的且受试者不需佩戴任何装置，同时刺激材料可以根据受试者年龄段进行适当调整，能够适合不同年龄段和不同发育水平的自闭症患者，尤其对6-18月儿童更具适用性，具有更大的适用范围；现有的利用眼动技术方案实现辅助诊断的方法，大部分经过繁琐的人工特征提取工作然后进行传统机器学习分类模型训练进行结果预测，耗时耗力，而本发明在眼动追踪技术的基础上，通过深度学习中卷积神经网络进行自动特征提取进行神经网络分类器的训练进行结果预测，提高了预测效率和预测准确性。
 附图说明
以下附图仅对本发明作示意性的说明和解释，并不用于限定本发明的范围，其中：
图1是根据本发明一个实施例的基于深度学习的孤独症辅助评估系统的示意图；
图2是根据本发明一个实施例的神经网络结构图。
 本发明的实施方式
为了使本发明的目的、技术方案、设计方法及优点更加清楚明了，以下结合附图通过具体实施例对本发明进一步详细说明。应当理解，此处所描述的具体实施例仅用于解释本发明，并不用于限定本发明。
在本文示出和讨论的所有例子中，任何具体值应被解释为仅仅是示例性的，而不是作为限制。因此，示例性实施例的其它例子可以具有不同的值。
对于相关领域普通技术人员已知的技术、方法和设备可能不作详细讨论，但在适当情况下，所述技术、方法和设备应当被视为说明书的一部分。
孤独症谱系障碍患者普遍存在面部情绪识别的缺陷，是导致其社交、沟通障碍的核心原因，目前研究孤独症谱系障碍患者的面部情绪识别障碍的研究主要有眼动追踪技术。孤独症谱系障碍患者在社交和面部刺激条件下，与典型发展人群的眼球运动模式存在着显著性差异，孤独症谱系障碍患者对眼睛区域的注视更少，避免直接凝视是孤独症谱系障碍患者面部情绪识别缺陷发展的原因。有证据显示，孤独症谱系障碍患者与直接注视相关的高水平唤醒与避免直接凝视有关，与社交能力的更严重损伤有关。孤独症谱系障碍患者的特异性面部加工模式，主要是对面部刺激进行选择并提取信息，这些信息对于情绪识别的任务来说是次优的。
本发明根据眼动数据特征与孤独症谱系障碍患者之间的关联，提供一种基于眼动技术面向面孔识别和情绪感知任务的高效便捷、无创、低成本的ASD辅助诊断方法。简言之，本发明实施例包括以下步骤：引导受试者在数据采集过程中完成视频观看任务，要求受试者专注于听到和看到的内容，在受试者观看视频短片的时候利用眼动仪记录其眼动数据；针对获得的眼动图像数据，完成数据预处理工作；然后，结合深度学习方法中的卷积神经网络（Convolutional Neural Networks，CNN）进行自动特征提取并通过模型训练得到神经网络分类器，最终实现对ASD的辅助诊断。本发明运用眼动追踪数据与深度学习算法相结合，可以有效地提取孤独症患者的特异性面部加工模式，从而实现对轻中度孤独症患者的辅助诊断。
具体地，参见图1所示，本发明实施例提供的基于深度学习的孤独症辅助评估系统包括依次连接的数据采集和特征提取单元110、用于分类训练的神经网络和结果输出单元120，其中，示意了神经网络1、神经网络2和神经网络3。
结合图1，本发明主要包括三个过程，分别是数据采集和特征提取，分类器训练和结果预测，下文将分别具体介绍。
1）、数据采集和特征提取
为了实现根据ASD儿童与正常儿童在面部情绪识别过程中相异的眼动模式进行适用于不同年龄段的辅助诊断，同时又能不限于当地医疗水平的影响，在医生专业能力与经验不足情况下，也能够准确的评估自闭症谱系障碍患者。本发明在数据采集过程中，利用方便年幼儿童且对医生专业性要求不高的眼动仪采集以动态的日常生活场景为刺激材料的眼动数据进行分类。
针对ASD儿童社交障碍中的核心问题面孔识别和情绪感知障碍，围绕ASD患者普遍存在的面部情绪识别缺陷，选择能够反映其特异性眼球运动模式的面部刺激。相比于常见于研究中的静态面部表情刺激，本发明中数据采集过程中选取动态视频情绪刺激。与静态图片刺激相比，采取动态的、具有日常言语表达的视频刺激更符合日常生活场景，并且更能够真实评估ASD儿童的情感识别能力，因为在现实社会互动中发生的交流并不总能被很好的察觉，同时可以根据受试者的不同年龄段进行刺激材料的动态调整以期适用于不同年龄段的儿童。
在一个实施例中，数据采集过程中的刺激素材由旨在为研究多模态和多媒体交互作用提供中文资源的中国自然情感视听数据库(Chinese Natural Emotional Audio-Visual Database，CHEAVD)提供的20个影片剪辑视频组成。例如，选择六个典型且相对完整的情绪刺激视频用于分析，它们由三个积极情绪视频和三个负面情绪视频组合而成。视频的持续时间在3秒到9秒之间。数据采集过程使用德国SMI公司的RED250眼动仪以非侵入方式收集每个受试者的眼动数据，该设备已集成到分辨率为1280*1024像素的22英寸宽屏显示器面板中。采样频率为60Hz，精度为0.4度。受试者头部运动的自由度为70厘米距离下移动范围是40*20厘米。采集过程中使用德国SMI公司实验设计软件Experiment Center进行在线眼动数据记录，并使用SMI数据分析软件BeGaze进行离线数据分析。在正式进行实验任务之前，首先对受试者进行培训以使其能够理解本实验中的任务，在理解任务后使用三星平板电脑ST800进行预实验以确保实验者熟悉实验的整个流程。然后要求参与者坐在距离测试屏幕60-80厘米左右的地方，直到眼动仪能够稳定地检测出参与者的瞳孔为止。在实验过程中，不能干预参与者以避免任何注意偏差。具体地，首先进行眼动的五点校准：受试者被要求依次看向屏幕四个角落和中间的校准点，只有当所有五个校准点误差平均不超过1度视角时，校准部分才算通过。在正式实验任务阶段，随机播放20个测试视频，参与者允许多次观看每个视频因为他们不了解其中显示的内容。
在采集到眼动数据后，经由配套数据分析软件SMI BeGaze处理输出得到受试者观看每个视频短片的热点图（Heat Map），焦点图（Focus Map），扫描路径图（Scan Path）。热点图例如以颜色暖色度来显示注视点的时间和位置的动态变化，即越接近于数据分析软件中颜色条右面的颜色，表明注视该区域的时间越长。扫描路径图例如逐点连续显示注视点位置及各个注视时间等信息。焦点图例如以亮度显示注视位置、时间的动态变化。
综上，本发明在任务设计中，围绕ASD儿童社交障碍中的核心问题面孔识别及情绪感知障碍采用动态的、具有日常言语表达的场景作为刺激材料，通过真实的生活化场景更能够反映受试者在社交互动中的自然反应以及眼动数据的真切性。并且，眼动仪追踪是非侵入性的，使用眼动仪进行数据采集时受试者不需要佩戴任何装置，因此不会给受试者造成不适感觉，尤其是年龄较小儿童。而且实验任务刺激可以根据受试者年龄情况进行灵活调整，对于6-18月年龄的儿童也方便操作进行诊断。
2）、分类训练
在提取到反映受试者眼动特征的热点图、焦点图和扫描路径图之后，利用神经网络训练学习后，获得用于预测评估孤独症的神经网络分类器。
参见图2所示，本发明实施例采用卷积神经网络，使用类似LeNet结构的设计，整个神经网络的结构包括输入层（input）、3层卷积层（即conv1、conv2、conv3）、2层最大池化层（即max pooling1、max pooling2）、2层全连接层（即fc1和fc2）和输出层（output）。具体地，首先是数据输入层，输入图像分别为眼动数据经过分析软件获得的热图、焦点图和扫描路径图，输入图像的尺寸统一归一化为1024*1024。第一层为卷积层，第二层为最大池化层，第三层为卷积层，第四层为最大池化层，第五层为卷积层，第六层和第七层为全连接层，第八层是输出层。前六层的激活函数为ReLU非线性激活函数，第七层的激活函数为Softmax激活函数，输出层的神经元数目可设置为4个，分别对应健康、轻度自闭症症状、中度自闭症症状和重度自闭症症状四个类别。
在设计好神经网络结构之后，开始神经网络的训练用于ASD的辅助诊断。具体地，设置好输入层、隐藏层和输出层神经元的节点数目以及卷积核的大小，随机初始化权重矩阵，包括输入层到隐层、隐层到隐层之间以及隐层到输出层之间的权重矩阵。分别输入各个视频短片眼动数据的热图、焦点图以及扫描路径图作为神经网络的输入，基于初始化的权重矩阵根据前向传播和反向传播算法和梯度下降法训练得到神经网络各层之间的权重矩阵。最后输出层的损失函数为交叉熵损失函数。
在测试本发明效果时使用交叉验证的方法，利用分类结果混淆矩阵根据受试者工作特征曲线（Receiver Operating Characteristic，ROC）及ROC曲线下面积AUC（Area Under ROC Curve）来判断神经网络分类器的效果优劣。采用集成学习中类似套袋法（bootstrap aggregating，Bagging）的方法，根据眼动数据的三个输入图像，集合三个分类器的分类结果从而给出最后的分类结果。
3）、分类结果预测
根据训练出的神经网络分类器模型进行ASD的分类预测。首先，根据输入数据的不同，经过训练得到三个卷积神经网络分类器，这三个分类器分别以眼动数据的热图、焦点图和扫描路径图为输入，输出对应健康、轻度自闭症症状、中度自闭症症状和重度自闭症症状四个类别。而后根据三个分类器的预测输出，使用简单投票法进行结合给出最终的预测结果，最终的四个结果仍然是上述四个类别中的一个。
需说明的是，本发明实施例的涉及的神经网络可具有相同或不同的结构，例如，可采用更多的卷积层、全连接层，采用平均池化或最大池化方式均可，并且分类结果不限于上述四类。此外，神经网络和结果输出单元等可采用软件或硬件实现，例如硬件处理器或逻辑电路等。
为了进一步验证本发明的效果，经过十名ASD儿童和十九名正常儿童的对比实验研究，发现本发明的数据分析结果与研究表明结果一致。AOI（area of interest）方法被广泛应用于眼动分析中。AOI旨在测量眼睛注视的感兴趣面部区域，通常包括眼睛、鼻子和嘴，然后统计出眼睛注视这些区域的频率和时间。在本发明的预实验中，ASD儿童相比于正常儿童在AOI中具有相对较少的注视时间和注视次数，更具体的ASD儿童注视身体及物体的时间远远超过注视眼睛，其眼动数据的热图分析和焦点图分析说明了同样的问题。本发明基于不同的眼动模式进行ASD的辅助诊断。
综上所述，本发明针对目前ASD辅助诊断技术中存在的不足以及ASD筛查评估和临床应用中存在的主要问题，建立了一种高效便捷，易于实现推广且适用于年幼儿童ASD辅助诊断的技术方案。围绕ASD儿童社交障碍中核心问题：面孔识别以及情绪感知障碍，针对ASD儿童在面孔识别和情绪感知中存在的异于正常儿童的眼动模式，借助红外视觉追踪技术（眼动技术可针对的年龄越来越小，当前的眼动技术已经能对3个月大的婴儿进行研究），结合深度学习方法，省去繁杂耗时的人工特征提取工作，实现了辅助诊断ASD并提高ASD患者的早期诊断和发现，为治疗打开有效的时间窗，实现对相关患者的早期干预，降低社会和家庭的经济负担，以此实现真正的“早发现，早干预，早治疗”。
需要说明的是，虽然上文按照特定顺序描述了各个步骤，但是并不意味着必须按照上述特定顺序来执行各个步骤，实际上，这些步骤中的一些可以并发执行，甚至改变顺序，只要能够实现所需要的功能即可。
本发明可以是系统、方法和/或计算机程序产品。计算机程序产品可以包括计算机可读存储介质，其上载有用于使处理器实现本发明的各个方面的计算机可读程序指令。
计算机可读存储介质可以是保持和存储由指令执行设备使用的指令的有形设备。计算机可读存储介质例如可以包括但不限于电存储设备、磁存储设备、光存储设备、电磁存储设备、半导体存储设备或者上述的任意合适的组合。计算机可读存储介质的更具体的例子（非穷举的列表）包括：便携式计算机盘、硬盘、随机存取存储器（RAM）、只读存储器（ROM）、可擦式可编程只读存储器（EPROM或闪存）、静态随机存取存储器（SRAM）、便携式压缩盘只读存储器（CD-ROM）、数字多功能盘（DVD）、记忆棒、软盘、机械编码设备、例如其上存储有指令的打孔卡或凹槽内凸起结构、以及上述的任意合适的组合。
以上已经描述了本发明的各实施例，上述说明是示例性的，并非穷尽性的，并且也不限于所披露的各实施例。在不偏离所说明的各实施例的范围和精神的情况下，对于本技术领域的普通技术人员来说许多修改和变更都是显而易见的。本文中所用术语的选择，旨在最好地解释各实施例的原理、实际应用或对市场中的技术改进，或者使本技术领域的其它普通技术人员能理解本文披露的各实施例。

Claims (8)

  1. 一种基于深度学习的孤独症辅助评估系统，其特征在于，包括数据采集和特征提取单元、第一神经网络、第二神经网络和第三神经网络，结果输出单元，所述数据采集和特征提取单元以及所述结果输出单元分别与所述第一神经网络、所述第二神经网络、所述第三神经网络具有通信连接，其中：
    所述数据采集和特征提取单元用于采集受试者观看动态视频的眼动数据，获得对应的热点图、焦点图和扫描路径图，所述热点图用于表征注视点的时间和位置的动态变化，所述焦点图用于表征注视位置、时间的动态变化，所述路径扫描图逐点连续显示注视点位置和各注视时间信息；
    所述第一神经网络用于输入所述热点图，获得第一分类结果；
    所述第二神经网络用于输入所述焦点图，获得第二分类结果；
    所述第三神经网络用于输入所述扫描路径图，获得第三分类结果；
    所述结果输出单元集合所述第一分类结果、所述第二分类结果和所述第三分类结果获得受试者的孤独症检测结果。
  2. 根据权利要求1所述的系统，其特征在于，所述第一神经网络、所述第二神经网络和所述第三神经网络具有相同或不同的结构。
  3. 根据权利要求1所述的系统，其特征在于，所述第一神经网络、所述第二神经网络和所述第三神经网络具有相同的结构，包括输入层、第一层卷积层，第二层池化层，第三层卷积层，第四层池化层，第五层卷积层，第六层全连接层、第七层全连接层和输出层。
  4. 根据权利要求3所述的系统，其特征在于，所述第一神经网络、所述第二神经网络和所述第三神经网络的第一层卷积层、第二层池化层、第三层卷积层、第四层池化层、第五层卷积层、第六层全连接层的激活函数为ReLU非线性激活函数，第七层全连接层的激活函数为Softmax激活函数，输出层的神经元数目为4个，分别对应健康、轻度自闭症症状、中度自闭症症状和重度自闭症症状四个类别。
  5. 根据权利要求1所述的系统，其特征在于，所述结果输出单元使用简单投票法结合所述第一分类结果、所述第二分类结果和所述第三分类结果给出最终的预测结果。
  6. 根据权利要求1所述的系统，其特征在于，使用眼动仪以非侵入方式收集每个受试者的眼动数据。
  7. 根据权利要求1所述的系统，其特征在于，所述热点图以颜色暖色度来显示注视点的时间和位置的动态变化，所述焦点图以亮度显示注视位置、时间的动态变化。
  8. 一种基于深度学习的孤独症辅助评估方法，包括以下步骤：
    采集受试者观看动态视频的眼动数据，获得对应的热点图、焦点图和扫描路径图，所述热点图用于表征注视点的时间和位置的动态变化，所述焦点图用于表征注视位置、时间的动态变化，所述路径扫描图逐点连续显示注视点位置和各注视时间信息；
    将所述热点图、所述焦点图和所述扫描路径图分布输入到经训练的第一神经网络、第二神经网络和第三神经网络，分别获得第一分类结果、第二分类结果和第三分类结果；
    集合所述第一分类结果、所述第二分类结果和所述第三分类结果获得受试者的孤独症检测结果。
PCT/CN2020/129160 2019-12-04 2020-11-16 一种基于深度学习的孤独症辅助评估系统和方法 WO2021109855A1 (zh)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN201911228792.7 2019-12-04
CN201911228792.7A CN112890815A (zh) 2019-12-04 2019-12-04 一种基于深度学习的孤独症辅助评估系统和方法

Publications (1)

Publication Number Publication Date
WO2021109855A1 true WO2021109855A1 (zh) 2021-06-10

Family

ID=76111103

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2020/129160 WO2021109855A1 (zh) 2019-12-04 2020-11-16 一种基于深度学习的孤独症辅助评估系统和方法

Country Status (2)

Country Link
CN (1) CN112890815A (zh)
WO (1) WO2021109855A1 (zh)

Families Citing this family (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113658697B (zh) * 2021-07-29 2023-01-31 北京科技大学 一种基于视频注视差异的心理测评系统
CN113784215B (zh) * 2021-09-08 2023-07-25 天津智融创新科技发展有限公司 基于智能电视的性格特征的检测方法和装置
CN113946217B (zh) * 2021-10-20 2022-04-22 北京科技大学 一种肠镜操作技能智能辅助评估系统
CN115990016B (zh) * 2022-12-02 2024-04-19 天津大学 一种基于眼动特征的孤独特质程度检测装置

Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20140364761A1 (en) * 2012-01-05 2014-12-11 University Court Pf The University Of Aberdeen An apparatus and method for psychiatric evaluation
CN107256332A (zh) * 2017-05-24 2017-10-17 上海交通大学 基于眼动数据的脑电实验评估系统及方法
CN109157231A (zh) * 2018-10-24 2019-01-08 阿呆科技（北京）有限公司 基于情绪刺激任务的便携式多通道抑郁倾向评估系统
CN109508755A (zh) * 2019-01-22 2019-03-22 中国电子科技集团公司第五十四研究所 一种基于图像认知的心理测评方法
CN109620259A (zh) * 2018-12-04 2019-04-16 北京大学 基于眼动技术与机器学习对孤独症儿童自动识别的系统
CN109820524A (zh) * 2019-03-22 2019-05-31 电子科技大学 基于fpga的自闭症眼动特征采集与分类可穿戴系统
CN211862821U (zh) * 2019-12-04 2020-11-06 中国科学院深圳先进技术研究院 一种基于深度学习的孤独症辅助评估系统

Patent Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20140364761A1 (en) * 2012-01-05 2014-12-11 University Court Pf The University Of Aberdeen An apparatus and method for psychiatric evaluation
CN107256332A (zh) * 2017-05-24 2017-10-17 上海交通大学 基于眼动数据的脑电实验评估系统及方法
CN109157231A (zh) * 2018-10-24 2019-01-08 阿呆科技（北京）有限公司 基于情绪刺激任务的便携式多通道抑郁倾向评估系统
CN109620259A (zh) * 2018-12-04 2019-04-16 北京大学 基于眼动技术与机器学习对孤独症儿童自动识别的系统
CN109508755A (zh) * 2019-01-22 2019-03-22 中国电子科技集团公司第五十四研究所 一种基于图像认知的心理测评方法
CN109820524A (zh) * 2019-03-22 2019-05-31 电子科技大学 基于fpga的自闭症眼动特征采集与分类可穿戴系统
CN211862821U (zh) * 2019-12-04 2020-11-06 中国科学院深圳先进技术研究院 一种基于深度学习的孤独症辅助评估系统

Also Published As

Publication number Publication date
CN112890815A (zh) 2021-06-04

Similar Documents

Publication Publication Date Title
Zunino et al. Video gesture analysis for autism spectrum disorder detection
WO2021109855A1 (zh) 一种基于深度学习的孤独症辅助评估系统和方法
Vargas-Cuentas et al. Developing an eye-tracking algorithm as a potential tool for early diagnosis of autism spectrum disorder in children
Chen et al. Strabismus Recognition Using Eye‐Tracking Data and Convolutional Neural Networks
US8388529B2 (en) Differential diagnosis of neuropsychiatric conditions
CN111326253A (zh) 自闭症谱系障碍患者的多模态情感认知能力的评估方法
He et al. The characteristics of intelligence profile and eye gaze in facial emotion recognition in mild and moderate preschoolers with autism spectrum disorder
Heaton et al. Reduced visual exploration when viewing photographic scenes in individuals with autism spectrum disorder.
CN211862821U (zh) 一种基于深度学习的孤独症辅助评估系统
PATEL Methods in the study of clinical reasoning
Melo et al. How doctors generate diagnostic hypotheses: a study of radiological diagnosis with functional magnetic resonance imaging
Tan et al. Virtual classroom: An ADHD assessment and diagnosis system based on virtual reality
Fabiano et al. Gaze-based classification of autism spectrum disorder
CN115517681A (zh) Md患者情绪波动监测和情感障碍状态评估的方法和系统
Zhang et al. A human-in-the-loop deep learning paradigm for synergic visual evaluation in children
Huang et al. Automatic recognition of schizophrenia from facial videos using 3D convolutional neural network
Cilia et al. Eye-tracking dataset to support the research on autism spectrum disorder
Xia et al. Dynamic viewing pattern analysis: towards large-scale screening of children with ASD in remote areas
Cheng et al. Computer-aided autism spectrum disorder diagnosis with behavior signal processing
Zuo et al. Deep Learning-based Eye-Tracking Analysis for Diagnosis of Alzheimer's Disease Using 3D Comprehensive Visual Stimuli
Fernández et al. A convolutional neural network for gaze preference detection: A potential tool for diagnostics of autism spectrum disorder in children
Chen Cognitive load measurement from eye activity: acquisition, efficacy, and real-time system design
Jeyarani et al. Eye Tracking Biomarkers for Autism Spectrum Disorder Detection using Machine Learning and Deep Learning Techniques
Guo et al. Design and application of facial expression analysis system in empathy ability of children with autism spectrum disorder
Zhou et al. Gaze Patterns in Children with Autism Spectrum Disorder to Emotional Faces: Scanpath and Similarity

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 20895924

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 20895924

Country of ref document: EP

Kind code of ref document: A1

32PN Ep: public notification in the ep bulletin as address of the adressee cannot be established

Free format text: NOTING OF LOSS OF RIGHTS PURSUANT TO RULE 112(1) EPC (EPO FORM 1205 DATED 12.01.2023)

122 Ep: pct application non-entry in european phase

Ref document number: 20895924

Country of ref document: EP

Kind code of ref document: A1