KR20230119295A - Autism diagnosis system using artificial intelligence based on multi-indicator - Google Patents

Abstract

The present invention provides a user interface for acquiring voice data obtained from a subject, image data obtained by capturing an image of the subject, and Childhood Autism Rating Scale (CARS) data of the subject, and communicating the data with the user interface. The present invention relates to an autism diagnosis system using multi-indicator-based artificial intelligence including a server configured to transmit and receive data and configured to determine the degree of autism disorder through ensemble analysis using a plurality of artificial intelligence.

Description

Autism diagnosis system using multi-indicator based artificial intelligence {AUTISM DIAGNOSIS SYSTEM USING ARTIFICIAL INTELLIGENCE BASED ON MULTI-INDICATOR}

The present invention relates to an autism diagnosis system using multi-indicator based artificial intelligence.

Autism Spectrum Disorder is a neurodevelopmental disorder characterized by persistent impairment in interactive social communication and social interaction from early childhood and limited and repetitive patterns of behavior, interests and activities. It is a category.

Children with autism spectrum disorder show a lack of interest in interpersonal relationships from an early stage of development and delayed or abnormal functions in the development of social skills, language, and communication.

Conventionally, in order to distinguish the degree of disability of such autism spectrum disorder, a method of evaluation according to an autism rating scale was used. However, this method was evaluated based on the experience of a specialist, so objectivity could be low. Recently, a technology was developed to provide an autism diagnosis and verification process in the form of a game in consideration of the behavioral characteristics of children with autism spectrum disorders. In this regard, Korean Registered Patent No. 1784429 has been disclosed.

However, this prior art has a problem of low accuracy because it diagnoses autism based on a single indicator.

Republic of Korea Patent No. 1784429

An object of the present invention is to provide a diagnostic system for accurately diagnosing the degree of autism spectrum disorders based on multiple indicators of conventional autism spectrum disorders.

As a means for solving the above problems, a user interface for acquiring voice data obtained from the subject, image data obtained by capturing an image of the subject, and Childhood Autism Rating Scale (CARS) data of the subject and communication with the user interface It is configured to send and receive data by using a plurality of artificial intelligence units, and includes a server configured to determine the degree of autism disorder through ensemble analysis using a plurality of artificial intelligence units, and the plurality of artificial intelligence units receive voice data and perform language-based control 1st artificial intelligence unit that determines the degree of autism disorder, 2nd artificial intelligence unit that determines the degree of autism based on behavior by receiving image data, and 3rd artificial intelligence unit that determines the degree of autism disorder by receiving childhood autism rating scale data An autism diagnosis system using a multi-index based artificial intelligence unit including a third artificial intelligence unit may be provided.

On the other hand, the first artificial intelligence unit uses tone data and semantic data extracted from voice data as input values. The tone data includes the response time, volume, and pitch variability of the subject's voice, and the semantic data is used. It may include the words and the frequency of each word.

Meanwhile, the second artificial intelligence unit may use motion data, facial expression data, and reaction data extracted from image data as input values.

On the other hand, the third artificial intelligence unit uses the included metadata and fraudulent autism rating scale data as input values, and the childhood autism rating scale data is data entered by the guardian observing the subject, and the metadata is It may include data on at least one of grade and occupation.

On the other hand, it is provided in the server and may include a pre-processing unit for pre-processing audio data, video data, childhood autism rating scale data and meta data of the subject, respectively.

In addition, the pre-processing unit may perform data de-identification, data noise removal, missing data supplementation using interpolation, data distribution/unit conversion, and data imaging.

Meanwhile, data imaging of the pre-processing unit may be performed by digitizing input data, converting digitized data into an nxn matrix, and converting into image data in the form of a heatmap.

Meanwhile, at least one of the first artificial intelligence unit, the second artificial intelligence unit, and the third artificial intelligence unit may predict a classification value associated with the degree of autism disorder by receiving and learning image data transmitted from the preprocessing unit.

On the other hand, at least one of the first artificial intelligence unit, the second artificial intelligence unit, and the third artificial intelligence unit receives image data and constructs a convolutional neural network (CNN) or long short-term memory (LSTM) model. The classification value is calculated using the fully-connected layer, and the degree of autism disorder can be determined through ensemble analysis based on the classification value.

Meanwhile, the server may be configured to transmit the result of determining the degree of autism disorder to the user interface.

The autism diagnosis system using multi-indicator-based artificial intelligence according to the present invention analyzes multi-indicator data using a data-customized artificial intelligence algorithm to minimize data feature loss that can occur from high-dimensional data analysis, and fully reflect data characteristics to autism disorders. The accuracy of the degree judgment can be improved.

1 is a block diagram of an autism diagnosis system using multi-index based artificial intelligence according to an embodiment of the present invention.
FIG. 2 is a detailed block diagram of an artificial intelligence unit, a pre-processing unit, and a client management unit of FIG. 1 .
3 is a flowchart illustrating a pre-processing process of input data performed by a pre-processing unit.
4 is a flowchart illustrating a process of determining the degree of autism disorder in the artificial intelligence unit.
5 is a conceptual diagram of an autism diagnosis process using multi-indicator based artificial intelligence according to an embodiment of the present invention.

Hereinafter, an autism diagnosis system using multi-indicator based artificial intelligence according to an embodiment of the present invention will be described in detail with reference to the accompanying drawings. In addition, in the description of the following embodiments, the name of each component may be called a different name in the art. However, if they have functional similarity and identity, even if a modified embodiment is employed, it can be regarded as an equivalent configuration. In addition, signs added to each component are described for convenience of description. However, the contents of the drawings in which these symbols are written do not limit each component to the scope in the drawings. Likewise, even if an embodiment in which the configuration in the drawings is partially modified is employed, it can be regarded as an equivalent configuration if there is functional similarity and identity. In addition, in light of the level of a general technician in the relevant technical field, if it is recognized as a component that should be included, the description thereof will be omitted.

1 is a block diagram of an autism diagnosis system using multi-index based artificial intelligence according to an embodiment of the present invention.

Referring to FIG. 1 , an autism diagnosis system using multi-indicator based artificial intelligence according to an embodiment of the present invention may include a user interface 100 and a server 200.

The user interface 100 is configured to receive information from a user or to generate data about the user, and to display information about the degree of autism disorder determined by the server 200 . The user interface 100 may be provided in a user terminal. The user terminal may be, for example, a device such as a tablet, a PC, a laptop (Lap top), a smart pad, or a smart device such as a smart phone. The user interface 100 may transmit/receive data by communicating with the server 200 wired/wireless.

The user interface 100 may include an input unit 110 and a display unit 120 . The input unit 110 may be configured to allow a user to input various information or to generate data. The input unit 110 may include a camera and a microphone. The camera is configured to record videos of children with autism. The microphone is configured to record the sounds and conversations of children with autism.

Meanwhile, although not shown, the user interface 100 may include an input means for a user to input information, such as a touch pad, a mouse, and a keyboard. Also, the user interface 100 may include a speaker for delivering sound to children with autism.

The server 200 is configured to determine the degree of autism disorder based on the data collected by the interface 100 .

The server 200 may include a database unit 210, a client management unit 220, an artificial intelligence unit 240, and a pre-processing unit 230.

The database unit 210 is configured to store data transmitted from the user interface 100, and may transmit data or store newly received data according to a request of the client management unit 220. The database unit 210 may include image data, audio data, Childhood Autism Rating Scale (CARS), and metadata.

Image data is data acquired by photographing a child with autism (hereinafter referred to as 'subject'). The image data may include data captured when communicating with other people as well as when being alone. The voice data is data obtained by recording the sound and conversation contents of the subject.

The Childhood Autism Rating Scale (CARS) is data that is observed, judged, and input by a guardian of a subject, for example, a parent of a child with autism disorder. In the case of children with autism, parental influence is very important, and accurate information acquired through careful and steady observation by parents can be an important factor in determining autism disorder. The childhood autism rating scale can evaluate scores according to severity according to each question, and can include scores evaluated for each question.

Metadata refers to data that can identify the target audience. Meta data may include information such as gender, age, grade, and occupation. Metadata can provide a more accurate criterion for judging the degree of autism disorder. That is, it is possible to more accurately determine whether the symptoms of a child suspected of autism are simply due to a young age and slow development or a result of autism.

The client management unit 220 manages information on each subject and is configured to transmit/receive data with a database, a pre-processing unit, and an artificial intelligence unit. The client management unit 220 determines the degree of autism disorder determined by artificial intelligence when requested by each individual from the user interface 100, and analyzes the subject's condition using the accumulated data to display the user interface 100 can be sent to

The pre-processing unit 230 is provided to process the data on the subject in a form suitable for the artificial intelligence unit to determine. The pre-processing unit may perform pre-processing on each of audio data, image data, childhood autism rating scale data, and metadata.

The artificial intelligence unit 240 determines autism using voice data, video data, childhood autism rating scale data and metadata, determines the degree of autism disorder by data, that is, by each index, and uses ensemble analysis to determine the final autism disorder. degree can be judged.

Hereinafter, referring to FIG. 2 , the artificial intelligence unit 240 , the pre-processing unit 230 and the client management unit 220 will be described in detail.

FIG. 2 is a detailed block diagram of an artificial intelligence unit, a pre-processing unit, and a client management unit 220 of FIG. 1 .

Referring to FIG. 2 , the client manager 220 distinguishes voice data as a first indicator, video data as a second indicator, and childhood autism rating scale data and meta data as a third indicator and transmits the respective data to the preprocessor. can

The pre-processing unit 230 may include a first pre-processing unit 231 , a second pre-processing unit 232 and a third pre-processing unit 233 .

The first preprocessor 231 preprocesses audio data, the second preprocessor 232 preprocesses image data, and the third preprocessor 233 performs childhood autism rating scale data and meta data. can be configured.

The first preprocessing unit 231 to the third preprocessing unit 233 may perform preprocessing by performing data de-identification, data noise removal, missing data supplementation using interpolation, data distribution/unit conversion, and data imaging, respectively. . Meanwhile, the function of the pre-processing unit will be described in detail later with reference to FIG. 3 .

The artificial intelligence unit may include a first artificial intelligence unit 241 , a second artificial intelligence unit 242 , and a third artificial intelligence unit 243 .

The first artificial intelligence unit 241 inputs voice data imaged through preprocessing by dividing it into meaning and tone, and outputs a predicted value of the tone data through spectrogram analysis, CNN, and LSTM models. Voice data may include response time, volume, and pitch variability of voice. In addition, the semantic data uses an LSTM model based on the words used and the frequency of each word, and then the degree of the first autism disorder is determined by the analysis result of the tone data and the fully connected layer.

The second artificial intelligence unit 242 receives pre-processed image data, uses CNN and LSTM models, and determines the degree of the second autism disorder through a fully connected layer. The image data may include motion data of the subject, facial expression data, and response data to external stimuli (commands and conversations).

The third artificial intelligence unit 243 receives preprocessed childhood autism rating scale data and meta data, uses CNN and LSTM models, and performs text analysis using fully connected layers to determine the degree of third autism disorder do.

On the other hand, the first artificial intelligence unit 241, the second artificial intelligence unit 242, and the third artificial intelligence unit 243 can be used in a learned state based on the results of determining the child with autism disorder and the degree of disability. can

The artificial intelligence unit may further include an ensemble analysis unit 244 . The ensemble analysis unit 244 is configured to perform final classification based on the first degree of autism disorder, the second degree of autism disorder, and the third degree of autism disorder. The ensemble analysis unit 244 is configured to more accurately determine the degree of autism disorder based on the determination result according to the degree of autism disorder determined by each of the artificial intelligence units 241 , 242 , and 243 . That is, although the result of determining the degree of autism disorder using any one of the artificial intelligence units 241, 242, and 243 can be used, the determination of the degree of autism disorder using the ensemble technique based on multiple indicators enables improved accuracy.

3 is a flowchart illustrating a pre-processing process of input data performed by a pre-processing unit.

Referring to FIG. 3 , as described above, the pre-processing unit may perform data de-identification, data noise removal, missing data supplementation using an interpolation method, data distribution/unit conversion, and data imaging.

Data de-identification corresponds to the step of removing information that can identify a specific individual due to data. Data noise removal corresponds to a step of filtering abnormally acquired data. On the other hand, supplementing missing data using interpolation is a step of securing data continuity by interpolating filtered or missing data.

Data distribution/unit conversion and data imaging are performed to generate image data for AI to judge.

Data imaging digitizes the data distribution/unit conversion data, converts the digitized data into an n x n matrix, and finally converts it into image data in the form of a heatmap.

4 is a flowchart illustrating a process of determining the degree of autism disorder in the artificial intelligence unit.

Referring to FIG. 4 , the first artificial intelligence, the second artificial intelligence, and the third artificial intelligence extract and learn image characteristics based on each input image data. In this case, CNN can be used. Then, based on the LSTM model, the time-series characteristics of the data are learned. Thereafter, the first degree of autism disorder, the second degree of autism disorder, and the third degree of autism disorder are predicted through the fully connected layer. Then, the learning results are ensemble to finally determine the degree of autism disorder. Meanwhile, in the first artificial intelligence unit, learning of time-series characteristics may be additionally performed using semantic data, and based on image data preprocessed based on voice and learning contents of semantic data, the first autism through a fully connected layer. degree of disability can be assessed.

5 is a conceptual diagram of an autism diagnosis process using multi-indicator based artificial intelligence according to an embodiment of the present invention.

Referring to FIG. 5, the first artificial intelligence unit 241, the second artificial intelligence unit 242, and the third artificial intelligence unit 243 receive data representing different indicators, respectively, and each artificial intelligence Based on the result of the determined degree of autism disorder, the ensemble analysis unit 244 determines the final degree of autism disorder. Meanwhile, the finally determined result may be transmitted to the user interface by the client management unit.

As described above, the autism diagnosis system using multi-indicator-based artificial intelligence according to the present invention uses multi-indicator data and fully reflects the characteristics of the data to improve the accuracy of determining the degree of autism disorder.

100: user interface
200: server
210: database unit
220: client management unit
231: first pre-processing unit
232: second pre-processing unit
233: third pre-processing unit
241: 1st artificial intelligence department
242: 2nd artificial intelligence department
243: 3rd Artificial Intelligence Division

Claims (10)

a user interface for acquiring voice data obtained from the subject, image data obtained by capturing an image of the subject, and Childhood Autism Rating Scale (CARS) data of the subject; and
It is configured to transmit and receive data by communicating with the user interface, and includes a server configured to determine the degree of autism disorder by ensemble analysis using a plurality of artificial intelligence units,
The plurality of artificial intelligence departments,
a first artificial intelligence unit that receives the voice data and determines a degree of a first autism disorder based on language;
a second artificial intelligence unit receiving the image data and determining a degree of a second autism disorder based on behavior; and
An autism diagnosis system using a multi-index based artificial intelligence unit including a third artificial intelligence unit that receives the childhood autism rating scale data and determines a third degree of autism disorder. According to claim 1,
The first artificial intelligence unit uses tone data and semantic data extracted from the voice data as input values;
The tone data includes a response time, volume, and pitch variability of the subject's voice,
The semantic data is an autism diagnosis system using a multi-index based artificial intelligence unit including words used and frequencies for each word. According to claim 1,
The autism diagnosis system of According to claim 1,
The third artificial intelligence unit uses the included metadata and the fraudulent autism rating scale data as input values,
The childhood autism rating scale data is data entered by the guardian observing the subject,
The meta data is an autism diagnosis system using a multi-indicator based artificial intelligence unit including data on at least one of gender, age, grade and occupation of the subject. According to claim 1,
It is provided in the server,
Autism diagnosis system using a multi-indicator-based artificial intelligence unit including a preprocessor for preprocessing the audio data, the image data, the subject's childhood autism rating scale data, and the meta data, respectively. According to claim 5,
The pre-processing unit performs data de-identification, data noise removal, missing data supplementation using interpolation, data distribution / unit conversion, and data imaging using a multi-indicator-based artificial intelligence unit. According to claim 6,
The data imaging of the pre-processing unit is performed by digitizing the input data, converting the digitized data into an nxn matrix, and converting the image data in the form of a heatmap. According to claim 7,
At least one of the first artificial intelligence unit, the second artificial intelligence unit, and the third artificial intelligence unit receives and learns the image data transmitted from the preprocessor to predict a classification value associated with the degree of autism disorder Autism diagnosis system using multi-indicator based artificial intelligence. According to claim 8,
At least one of the first artificial intelligence unit, the second artificial intelligence unit, and the third artificial intelligence unit receives the image data and uses a data convolutional neural network (CNN) or long short-term memory (LSTM) Learning using a model, calculating a classification value using a fully-connected layer,
An autism diagnosis system using a multi-indicator based artificial intelligence unit that determines the degree of the autism disorder through ensemble analysis based on the classification value. According to claim 9,
The server,
An autism diagnosis system using a multi-indicator based artificial intelligence unit configured to transmit a result of determining the degree of autism disorder to the user interface.

Images