KR20200143940A - Method and system for assessing language ability based on artificial intelligence - Google Patents

Abstract

According to an embodiment of the present invention, provided is a system for evaluating a language ability, which includes a user device and a server. The user device provides conversation data including speech data of a subject for measuring a language ability to the server. The server performs an artificial intelligence algorithm calculation using conversation data of the subject received from the user device. In accordance with the result of the calculation, the level of the language ability of the subject is evaluated. The abnormality may be determined by comparing predicted age information of the subject calculated through the artificial intelligence algorithm with the actual age of a previously registered subject.

Description

Method and system for assessing language ability based on artificial intelligence}

The present invention relates to an artificial intelligence-based language ability evaluation method and system.

Speech pathology is the field of study of related disorders, including sensory and motor disorders used in communication. Communication disorders are difficult to define in symptoms and forms, but range from communication disorders due to disabilities to disorders related to the learning of the native language of general children. This can be applied not only to children, but also to adults in similar circumstances.

According to a study on the demand forecast for speech therapists in Korea, the demand for speech and hearing clinicians in 2020 is estimated to be around 60,000, but the demand is inadequate. Accordingly, in Korea, a number of studies have been conducted on securing experts in speech therapy and to secure expertise.

In this background, the field that is most actively researched is the field of language ability for children, which is due to the fact that language ability explodes in the preschool age group. As a conventional analysis method for capturing developmental disabilities in preschool and school-age age groups, there is a method of analyzing spontaneous speech, and analysis of spontaneous speech is a statistical analysis of various measurement indicators obtained by collecting utterances from treatment tools through counseling or play. Therefore, it is an analysis method that predicts the language age of the test subject, determines whether there is a developmental disability, and is used in various studies. This conventional language ability analysis process has a problem in that a large cost is incurred in the process of recording the speech of the test subject and transcribing it to analyze it.

1) Registered Patent Publication No. 10-1684424 (Device, system and method for evaluating social language ability of children with autism spectrum disorders)

The present invention was devised to perform a method of automatically evaluating a language ability of a specific object through a convolutional neural network model.

In particular, the present invention automatically classifies the level of the spoken sentence of the subject for which the language ability is to be evaluated, calculates the language age of the subject, and determines whether there is a disability by comparing the actual age of the subject with the calculated language age. Designed for

The system for evaluating language ability according to an embodiment of the present invention includes a user device and a server, and the user device provides conversation data including speech data of an examinee to measure language ability to the server, and the server Performs an artificial intelligence algorithm operation using the conversation data of the subject received from the user device, and evaluates the level of language ability of the subject according to the result of the operation, and the predicted age of the subject calculated through the artificial intelligence algorithm The abnormality can be determined through comparison between the information and the actual age of the previously registered subject.

According to an exemplary embodiment of the present invention, not only a subject having a problem in language ability but also a subject having excellent language ability can be evaluated in an effective and simple manner.

According to an embodiment of the present invention, speech data of a subject can be easily extracted.

According to an exemplary embodiment of the present invention, the subject can automatically communicate with the matching counselor without having to face the counselor directly, thereby reducing inconvenience caused by visiting an institution.

1 is a system diagram illustrating a system for implementing a language ability evaluation method according to an embodiment of the present invention.
2 is a block diagram showing the configuration of a server according to an embodiment of the present invention.
3 is a diagram schematically illustrating a sequence of an operation of evaluating language ability based on an artificial intelligence algorithm according to an embodiment of the present invention.
4 is a diagram illustrating a form in which speech data of a subject is extracted and stored according to an embodiment of the present invention.
5 is a diagram schematically showing the configuration of an artificial intelligence algorithm according to an embodiment of the present invention.
6 is a graph showing an individual prediction probability distribution within an age group that is a basis for a statistical analysis operation according to an embodiment of the present invention.
7 is a flowchart illustrating a sequence of a language ability evaluation operation according to an embodiment of the present invention.

Advantages and features of the present invention, and a method of achieving them will become apparent with reference to the embodiments described below in detail together with the accompanying drawings. However, the present invention is not limited to the embodiments disclosed below, but may be implemented in a variety of different forms, only the present embodiments are intended to complete the disclosure of the present invention, It is provided to fully inform the technician of the scope of the present invention, and the present invention is only defined by the scope of the claims.

The terms used in the present specification are for describing exemplary embodiments and are not intended to limit the present invention. In this specification, the singular form also includes the plural form unless specifically stated in the phrase. As used in the specification, “comprises” and/or “comprising” do not exclude the presence or addition of one or more other elements other than the mentioned elements. Throughout the specification, the same reference numerals refer to the same elements, and “and/or” includes each and all combinations of one or more of the mentioned elements. Although "first", "second", and the like are used to describe various elements, it goes without saying that these elements are not limited by these terms. These terms are only used to distinguish one component from another component. Therefore, it goes without saying that the first component mentioned below may be the second component within the technical idea of the present invention.

Unless otherwise defined, all terms (including technical and scientific terms) used in the present specification may be used as meanings that can be commonly understood by those of ordinary skill in the art to which the present invention belongs. In addition, terms defined in a commonly used dictionary are not interpreted ideally or excessively unless explicitly defined specifically.

The terms "unit" or "module" used in the specification means software, hardware components such as FPGA or ASIC, and "unit" or "module" performs certain roles. However, "unit" or "module" is not meant to be limited to software or hardware. The "unit" or "module" may be configured to be in an addressable storage medium or may be configured to reproduce one or more processors. Thus, as an example, "sub" or "module" refers to components such as software components, object-oriented software components, class components and task components, processes, functions, properties, It includes procedures, subroutines, segments of program code, drivers, firmware, microcode, circuits, data, databases, data structures, tables, arrays and variables. Components and functions provided within "sub" or "module" may be combined into a smaller number of components and "sub" or "module" or into additional components and "sub" or "module" Can be further separated.

Spatially relative terms "below", "beneath", "lower", "above", "upper", etc., as shown in the figure It can be used to easily describe the correlation between a component and other components. Spatially relative terms should be understood as terms including different directions of components during use or operation in addition to the directions shown in the drawings. For example, if a component shown in a drawing is turned over, a component described as "below" or "beneath" of another component will be placed "above" the other component. I can. Accordingly, the exemplary term “below” may include both directions below and above. Components may be oriented in other directions, and thus spatially relative terms may be interpreted according to orientation.

In the present specification, a computer refers to all kinds of hardware devices including at least one processor, and may be understood as encompassing a software configuration operating in a corresponding hardware device according to embodiments. For example, the computer may be understood as including all of a smartphone, a tablet PC, a desktop, a laptop, and a user client and an application running on each device, but is not limited thereto.

Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings.

Each of the steps described herein is described as being performed by a computer, but the subject of each step is not limited thereto, and at least some of the steps may be performed by different devices according to embodiments.

1 is a system diagram illustrating a system for implementing a language ability evaluation method according to an embodiment of the present invention.

As illustrated in FIG. 1, the system may include a server 100 and a user device 200.

The user device 100 includes a smartphone, a tablet PC, a mobile phone, a video phone, an e-book reader, a desktop PC, and a laptop PC. laptop PC), a netbook computer, a workstation, a server, a personal digital assistant (PDA), and a portable multimedia player (PMP).

The user device 200 is a conversation between a subject who wishes to be evaluated for their level of language ability and the tester (e.g., it may refer to a test assistant that serves to lead a conversation, such as asking a question to the subject to speak) This is a configuration for performing an operation of collecting data (eg, a recorded voice file) and transmitting the collected conversation data to the server 100.

According to various embodiments, the user device 200 may be divided into a subject, a tester, and the like. In addition, the content of a call between the tester's device and the tester's device for the purpose of collecting chat data through the server 100 may be automatically recorded, and the recorded chat data may be transmitted to the server 100.

The server 100 may receive conversation data from the user device 200, extract speech data of the subject from the received conversation data, and analyze the language ability of the subject from the extracted speech data. have. In this case, the operation of extracting speech data performed by the server 100 may be performed after a transcription operation of converting conversation data in the form of a voice file into text.

The server 100 may further collect, from the user device 200, not only the conversation data, but also various information (eg, age, disease information, etc. of the subject) required to analyze the language ability of the subject.

Specifically, the server 100 may perform an operation of predicting an age corresponding to the speech data by inputting and analyzing the speech data of the subject into a pre-learned artificial intelligence algorithm (CNN). In addition, the server 100 may determine the language characteristic features (eg, language disorder, excellent language ability, etc.) of the subject through comparison of the actual age of the subject and the predicted age calculated as a result of analysis of the artificial intelligence algorithm.

The server 100 according to the exemplary embodiment of the present invention may be configured as a single server, as needed, may be configured as a plurality of servers, and may be a concept including a cloud server.

2 is a block diagram showing the configuration of a server according to an embodiment of the present invention.

The server 100 according to an embodiment of the present invention obtains various information required to perform the language ability analysis of the subject from the user device 200, and evaluates the language ability of the subject by processing and analyzing the obtained information. You can perform the operation

The server 100 according to an embodiment of the present invention may include a communication unit 110, a storage unit 120, and a control unit 130.

First, the communication unit 110 may use a network to transmit and receive data between a user device and a server, and the type of the network is not particularly limited. The network may be, for example, an Internet Protocol (IP) network that provides a service for transmitting and receiving large amounts of data through an Internet Protocol (IP) or an All IP network integrating different IP networks. In addition, the network includes a wired network, a wireless broadband (Wibro) network, a mobile communication network including WCDMA, a high speed downlink packet access (HSDPA) network and a mobile communication network including a long term evolution (LTE) network, and LTE advanced (LTE-A). ), a mobile communication network including 5G (Five Generation), a satellite communication network, and a Wi-Fi network, or at least one or more of them may be combined.

The communication unit 110 according to an embodiment of the present invention may receive a conversation file of users recorded by the user device 200 through communication with the user device 200. In this case, the communication unit 110 may receive a collection request operation for collecting conversation data, which is a conversation voice file between the examinee and the examinee, from the user device 200.

In this case, the user device 200 is managed by, for example, a speech therapy institution, and may be a device capable of instantly recording a conversation between a subject and an examiner. However, the present invention is not limited thereto, and the user device 200 may mean a device such as a personal smartphone of a subject or an examiner. When the user device 200 is a personal device, the user device 200 may be classified into a subject device 200a and a tester device 200b.

If the subject wants to receive a diagnosis of his/her language ability remotely without meeting the examiner directly, the examinee will be assigned a examiner who will communicate with himself by accessing the server 100 through his personal device, the examinee's device 200a. I can. Thereafter, the subject may conduct a conversation with the examiner to whom he or she has been assigned, and the conversation content may be automatically provided to the server 100, and the subject's language ability may be measured based on the conversation content.

In this case, the communication unit 110 may receive a conversation request signal from the subject device 200a and transmit a subject matching signal to a suitable tester device 200b in response thereto. According to an embodiment, the communication unit 110 of the server 100 may transmit a subject matching signal that causes the tester device 200b to ring a specific notification bell, and accordingly, the tester notifies similarly to a general call connection method. After confirming the bell, you can start talking to the subject. Conversation contents performed after that are automatically recorded, and the recorded conversation contents through the communication unit 110 may be automatically transmitted to the server 100 side.

According to various embodiments, the communication unit 110 may obtain data related to learning an artificial intelligence algorithm from a web or a linked institution server.

In addition, according to various embodiments, the communication unit 110 may perform an operation of transmitting information as a result of the analysis of the language ability of the subject performed by the server 100 and feedback information corresponding thereto to the user device 200.

The storage unit 120 may include, for example, an internal memory or an external memory. The internal memory includes, for example, volatile memory (e.g., dynamic RAM (DRAM), static RAM (SRAM), synchronous dynamic RAM (SDRAM), etc.)), non-volatile memory (e.g., OTPROM (one time programmable ROM), programmable ROM (PROM), erasable and programmable ROM (EPROM), electrically erasable and programmable ROM (EEPROM), mask ROM, flash ROM, flash memory (such as NAND flash or NOR flash), hard drive, Alternatively, it may include at least one of a solid state drive (SSD).

External memory is a flash drive, for example, compact flash (CF), secure digital (SD), micro secure digital (Micro-SD), mini secure digital (mini-SD), extreme digital (XD), It may further include a multi-media card (MMC) or a memory stick. The external memory may be functionally and/or physically connected to the electronic device through various interfaces.

The storage unit 120 according to an exemplary embodiment of the present invention may store voice chat data received from the user device 200 and text chat data obtained by converting contents of the voice chat data into text. The storage unit 120 further includes a program required to perform an operation of converting the voice conversation data into text conversation data and an operation of separately extracting the speech data of the subject and the speech data of the examinee from the text conversation data. Can be saved.

In addition, the storage unit 120 matches an examiner who will perform a conversation with the examinee in response to a request of the examinee according to an embodiment of the present invention, and matches the examinee's device 200a and the examinee's device 200a requesting the conversation. It is possible to store a program for performing a call connection between the tester devices 200b, which is the tester's device, and acquiring data recorded on the call.

In addition, the storage unit 120 may store an artificial intelligence algorithm that determines the conversation ability of the subject based on the acquired conversation data, and also stores a program that supports the overall operation of analyzing the language ability of the subject. .

The control unit 130 may also be referred to as a processor, a controller, a microcontroller, a microprocessor, a microcomputer, or the like. Meanwhile, the controller 130 may be implemented by hardware, firmware, software, or a combination thereof.

In the case of implementation by firmware or software, an embodiment of the present invention may be implemented in the form of a module, procedure, or function that performs the functions or operations described above. The software code may be stored in the storage unit 120 and driven by the control unit 130. The storage unit 120 may be located inside or outside the user terminal and the server, and may exchange data with the controller 130 through various known means.

The control unit 130 according to an embodiment of the present invention may perform the overall language ability evaluation operation described in the following drawings.

According to an embodiment, the controller 130 may perform a language ability evaluation operation based on the artificial intelligence algorithm calculation process shown in FIG. 3.

As shown in FIG. 3, the control unit 130 includes an operation of acquiring conversation data between a subject and an examiner (Data collection), processing and pre-processing the conversation data into data for analysis (Processing), based on an artificial intelligence algorithm. An operation of performing a convolution operation, a statistical analysis, and an operation of predicting language age (determining whether there is a language disorder) may be performed.

Before performing the calculation and analysis operation by the artificial intelligence algorithm, the control unit 130 according to an embodiment of the present invention is to learn the artificial intelligence algorithm based on a data set corresponding to a specific criterion set by the manager (inspector). I can.

The controller 130 may input speech data extracted from conversation data of the subject into the learned and stored artificial intelligence algorithm, and perform an operation of evaluating the language ability of the subject through this.

Specifically, the controller 130 may perform an operation of obtaining text conversation data from voice conversation data between a user (subject) and an examiner received through the communication unit 110.

Also, the controller 130 may extract only text data corresponding to the voice frequency of the subject from the text conversation data as speech data of the subject.

The control unit 130 analyzes the extracted speech data of the examinee based on an artificial intelligence algorithm, and whether the level of the speech data of the examinee corresponds to school age (SA) or preschool age (PSA; Pre School Age) ) Can be determined.

The utterance data of the subject may be composed of a plurality of sentences, and accordingly, the control unit 130 may perform analysis on all of the plurality of sentences constituting the utterance data. The control unit 130 may calculate the user's expected age group based on the proportion of sentences corresponding to the school age among sentences in the total utterance data according to a result of analyzing all sentences constituting the subject's utterance data. have. Through the above-described method, the control unit 130 may evaluate the language ability for a specific target.

Hereinafter, the operation of the control unit 130 will be described in more detail. According to an embodiment, the control unit 130 may include a speech data extraction unit 131, a learning support unit 132, and an evaluation support unit 133.

The speech data extracting unit 131 may extract speech data of a subject through three major operations. Specifically, the speech data extraction unit 131 includes an operation of acquiring voice conversation data between a subject and an examiner, an operation of converting the obtained voice conversation data into text conversation data, and converting the text conversation data into the subject's speech data and the examinee's speech data. You can perform the classification operation.

The operation of obtaining conversation data by the speech data extracting unit 131 may be performed in more detail as follows. According to an exemplary embodiment, the speech data extracting unit 131 may directly acquire conversation contents (eg, recorded conversation data) performed by the subject and the examinee face-to-face from the user device 200.

In addition, according to various embodiments, the speech data extracting unit 131 may acquire conversation contents (eg, call recording conversation data, etc.) made in a state in which the examinee and the examinee do not face each other. At this time, the speech data extracting unit 131 may perform an operation of matching and communicating with the examinee device 200a and the examiner device 200b in order to establish a conversation between the examinee and the examinee.

Specifically, the speech data extraction unit 131 may receive a conversation request signal received through the communication unit 110 from either the testee device 200a or the tester device 200b. The speech data extraction unit 131 may mediate a conversation connection in response to the received conversation request signal. For example, when receiving a conversation request signal from the subject device 200a, the speech data extracting unit 131 extracts the tester and tester device 200b to be matched with the test subject device 200a, and extracts the extracted matching signal. It can be transmitted to the examiner's device 200b. When the speech data extracting unit 131 transmits the matching signal to the tester device 200b, the tester device 200b outputs a ringtone such as an incoming call notification sound and a notification asking whether to proceed with a conversation with the subject. The screen may be displayed. In response to the operation of the tester device 200b, when the tester responds to the matching signal (e.g., selecting a button indicating conversation acceptance, etc.), the utterance data extracting unit 131 sends a conversation request signal to the testee device 200a. In response to) and the matching signal, a call connection with the tester device 200b that sent a response signal may be performed, and conversation contents may be automatically recorded and acquired during the call. This may mean that the speech data extracting unit 131 records a conversation content during a call and requests at least one of the testee device 200a and the tester device 200b to transmit the recorded data.

In this way, the principle of matching the examinee with the examinee can be similarly performed even when the examinee first performs a conversation request. When the tester sends a conversation request signal, the speech data extracting unit 131 may extract a corresponding subject and transmit a matching signal to the device 200a of the subject.

According to various embodiments, the speech data extracting unit 131 manages a list of subjects who are required to test language ability through conversation with the examinee, and a list of examinees who are ready to communicate with the examinee, and a specific target responds to a matching signal. If not, another matching object to be replaced may be extracted from the subject list or the test subject list to perform rematching.

In addition, the speech data extracting unit 131 may convert voice conversation data acquired by the above-described methods into text conversation data. The speech data extraction unit 131 may convert the obtained speech data into text using a sound to text (STT) program or the like.

In addition, the speech data extraction unit 131 may perform an operation of classifying the text conversation data into the subject speech data and the examinee speech data.

According to an embodiment, the speech data extracting unit 131 may classify a speaker's voice by using frequency characteristic information in order to distinguish between the subject speech data and the examiner speech data, and accordingly classify the conversation contents for each speaker. . Specifically, the speech data extraction unit 131 may obtain feature values (eg, waveform characteristics, frequency values, etc.) by obtaining voice frequency information of one of the examinee or one of the examinees in advance. The speech data extracting unit 131 classifies speech contents of a sentence unit constituting text conversation data into two types (or more) of talkers for each type of voice, and determines whether the talker is a subject or a tester They can be classified into subject speech data and examiner speech data. For example, the classification may include labeling a talker (eg, a subject and a tester) corresponding to each sentence.

According to an embodiment, the speech data extractor 131 divides the talkers into A and B through voice frequency analysis, and then determines the contents of the conversation (eg, determination of the side with a high proportion of questions), Among them, it is possible to determine who is the subject.

In addition, the speech data extracting unit 131 may specify a subject and an examiner in a manner that selects an object matching a previously registered voice characteristic value. To this end, the speech data extracting unit 131 may request at least one of the examiner and the subject to register a voice before the conversation request to the server 100, and the registered voice is analyzed and stored in the form of a frequency characteristic value. I can.

The speech data extracting unit 131 may classify and store the speech data of the examinee and speech data of the examinee, respectively, when it is determined whether the speech contents constituting the text conversation data are speech of the examinee or speech of the examinee.

According to various embodiments, the speech data extracting unit 131 may request the conversation data from the examinee or the examiner again when the total speech amount of speech sentences constituting the speech data of the examinee is less than a reference value. Accordingly, the speech data extracting unit 131 may secure a sufficient total amount of data required to evaluate the language ability of the subject.

A plurality of speech sentences corresponding to the subject speech data may be arranged and recorded as shown in FIG. 4.

4 is a diagram illustrating a form in which speech data of a subject is extracted and stored according to an embodiment of the present invention.

As shown in FIG. 4, the speech data extraction unit 131 according to an embodiment of the present invention includes information on the conversation order in which sentences were generated during the consultation process, the total number of sentences, personal information such as the name of the subject, and age, and the consultation was performed. Data such as the location, the extracted speech data of the examinee, and the extracted speech data of the examinee can be stored in a separate form. Furthermore, the speech data extraction unit 131 may additionally store whether or not the examinee and the examinee have performed face-to-face consultation.

The learning support unit 132 may support learning of an artificial intelligence algorithm according to an embodiment of the present invention. The artificial intelligence algorithm may evaluate language ability through an operation of predicting the language age of the subject.

The learning support unit 132 may perform pre-processing of learning data to be put into an artificial intelligence algorithm. In addition, the learning support unit 132 may input data for learning to an artificial intelligence algorithm. The artificial intelligence algorithm may refer to, for example, a convolutional neural network model (CNN), and may perform sentence classification through an RCNN in which the concept of an RNN is applied to the CNN.

First, an operation in which the learning support unit 132 performs preprocessing of learning data will be described.

The learning support unit 132 may acquire data to be used as learning data, and the data to be used as learning data may include, for example, a data set prepared in advance and speech data extracted from the speech data extraction unit 131. .

The learning support unit 132 performs a pre-processing operation of removing noise from the acquired data (a data set composed of text-type conversation contents, speech data extracted from the data collection process, etc.) in order to remove data that is not suitable for learning. Can be done. At this time, the learning support unit 132 may define a sentence structure that is not suitable for learning as noise, and free speech recognizes all utterances generated by the subject (the subject) during the counseling process, so the speech is used as an analysis unit. To do this, you need to separate the boundaries of your utterance. When the speech boundary is divided, a simple positive or negative response, or a short speech less than or equal to a predetermined reference value (eg, 3 words or less) can be recorded as the speech of the subject (the subject). These utterances can lead to incorrect learning because it is difficult to find the difference in the learning process. Therefore, the learning support unit 132 may define and remove such speech sentences as noise data. Table 1 below is a table summarizing noise data.

However, when the noise data is removed from the PSA group, two problems may occur. The first problem is that the amount of data may become too small, and the second is that it may be difficult to learn because these noises are expressed as representative qualities of the PSA group. Therefore, the learning support unit 132 sets the noise removal criterion to a value lower than the noise removal criterion of the training data of the SA group (reduces the removed data compared to the SA group) when preprocessing the training data of the PSA group. I can. For example, the learning support unit 132 may not remove noise except for a simple answer. In addition, the learning support unit 132 may set a limit ratio of the difference in the number of sentences between each data so that the ratio of the number of sentences of the SA learning data and the PSA learning data that are finally calculated after noise removal is similar. For example, the learning support unit 132 may set the difference between the number of sentences between the SA learning data and the PSA learning data to be 10% or less. At this time, each sentence constituting the learning data may be labeled with age group information corresponding thereto. Accordingly, the artificial intelligence algorithm can determine whether the subject's speech sentence corresponds to which age group later by learning the individual sentences constituting the learning data and the age group information labeled corresponding to each sentence. Can be. Table 2 below shows the number of sentences extracted from each group and examples of labeled sentences.

In this way, the learning support unit 132 may generate the learning data by dividing it into SA learning data and PSA learning data, and adjust the quantity of sentences constituting the SA learning data and PSA learning data according to respective criteria.

After the learning support unit 132 generates data for learning, it inputs it to an artificial intelligence algorithm to learn, and calculates and applies appropriate parameters (e.g., weight required for convolution operation, normalization setting value, etc.) calculated accordingly. can do.

The description of the artificial intelligence algorithm will be referred to FIG. 5.

5 is a diagram schematically showing the configuration of an artificial intelligence algorithm according to an embodiment of the present invention.

As shown in Figure 5, according to an embodiment of the present invention, the AI algorithm of the language ability of the subject slides the matrix from the Sentence Matrix consisting of N*k and filters 2, 3, and 4 having different sizes. You can perform convolutional operations. The artificial intelligence algorithm may generate a final feature vector by performing 1-max pooling based on three Feature Maps extracted therefrom and concatenating the extracted primary feature vectors. Thereafter, the artificial intelligence algorithm may perform binary classification in a Softmax layer to which a dropout is finally applied to the final feature vector.

The convolutional neural network is an artificial intelligence analysis method commonly used in image recognition and processing. In order to use the convolutional neural network for natural language processing, the learning support unit 132 performs a word embedding operation in which individual speeches are expressed as an embedding matrix. This means that the sentence should be expressed as a matrix in the same way that the input of the neural network is expressed as a matrix of the image pixel size during the operation for image processing. In order to express sentences in a matrix, a process of expressing words as vectors is first required.

와 같이 k차원의 벡터로 표현하기 위한 일련의 동작을 수행할 수 있다. In accordance with such a request, the learning support unit 132 may perform an operation of extracting words in a sentence constituting learning data (eg, speech data of an existing subject, a prepared data set, etc.) for artificial intelligence learning. . And the learning support unit 132 is a word constituting the sentence

As described above, a series of operations for expressing a k-dimensional vector can be performed.

에서

까지의 단어를 채워(padding) 동일한 문장길이로 표현할 수 있다. When the maximum number of words in sentences constituting the data set is N, the learning support unit 132 may create a sentence matrix of N *k dimension by defining this as a sequence length. In addition, the learning support unit 132 generates a Sentence Matrix, if the number of words in a sentence is n, if n ≤ N,

in

Words up to can be padded and expressed in the same sentence length.

The learning support unit 132 may use a previously learned vector expression value in implementing the sentence matrix. However, it is not limited to the above method, and according to an embodiment of the present invention, a vector value is randomly initialized and may be updated through a learning process to a value suitable for performing a classification operation. To this end, the learning support unit 132 may use a previously prepared data set as learning data, set a vector value to an arbitrary value, and perform a learning operation using speech data of the subject for updating.

In addition, the learning support unit 132 may control overall driving operation of an artificial intelligence algorithm required for artificial intelligence learning.

The convolutional layer may include a layer in which filters having different sizes perform a convolution operation, and a pooling layer that extracts a feature vector from a generated feature map. The convolutional layer according to an embodiment of the present invention may include multiple filters configured to approximate a single size value representing an optimal performance. The learning support unit 132 according to an embodiment of the present invention may control the multiple filters of the convolutional layer to extract a vector representing an important feature from a feature map of each filter through a pooling (Feature Extraction). Thereafter, the learning support unit 132 may control to output the classification result after the normalization operation in the last Softmax Layer through the Fully Connected Layer.

The learning support unit 132 may preferably perform a dropout normalization operation in a convolutional neural network. The dropout method is a typical normalization method that solves an overfitting problem occurring in a deep neural network, and refers to a method of randomly dropping units in a learning process. According to an embodiment of the present invention, the dropout may be performed with a probability of, for example, 0.4 to 0.6. In addition, according to an embodiment of the present invention, the normalization operation of the dropout method may be performed between the convolutional layer and the full connection layer.

The learning support unit 132 may first set a macro variable (Hyper Parameter) before performing an operation of learning an artificial intelligence algorithm. For examples of the macroscopic variables, refer to the contents shown in Table 3.

The learning support unit 132 may perform training, cross validation, and data set configuration operations in addition to setting macroscopic variables. The data set may be set as shown in Table 4.

In addition, each score obtained by setting and learning a model with the data set and macroscopic variables is shown in Table 5.

Unlike the positives and negatives that appear in emotion analysis, the age ranges from 20s to 100 years of age. Therefore, the distribution of prediction results is analyzed by inputting data from the age group different from the learning age group. For example, if data on a test subject in their 30s is input, the prediction result will be biased toward the result corresponding to the SA group. In contrast, in the case of children aged 4-5 years old, the predicted results will be biased toward those in the SA group. Based on this, if the predicted result of the test subject corresponding to the high school part is biased to the result corresponding to the infant part, it can be concluded that the language age is lower than the actual age.

Equations 1 and 2 below may represent a probability of predicting a distribution of an input subject's utterance as a PSA group and a probability of predicting as an SA group, respectively.

The SA group prediction probability can be calculated as in Equation 1 below.

[Equation 1]

P(SA) = (number of predictions to SA)/(Total utterance supposed SA)*100

And the PSA group prediction probability can be calculated as in Equation 2 below.

[Equation 2]

P(PSA) = (number of predictions to PSA)/(Total utternace supposed PSA)*100

First, the denominator of P(SA), which is an SA group prediction probability, indicates the total number of utterances assuming that all actual utterances are SA groups, and the numerator may indicate the number of SA-predicted input utterances. Conversely, the denominator of P(PSA), which is the predicted probability of the PSA group, may represent the total number of utterances assumed as the PSA group, and the numerator may represent the actual number predicted by the PSA group. Using the above equation, you can find the predicted probability derived from the subsequent experiment.

The learning support unit 132 may repeatedly perform a learning operation such that the SA prediction probability and the PSA group prediction probability reach a preset value.

According to an embodiment of the present invention, the learning support unit 132 may implement an artificial intelligence algorithm to analyze the language ability of a subject based on statistical data such as a prediction probability distribution for each age group and an individual probability distribution within an age group.

Tables 6 to 7 will be referred to to help understand the prediction probability for each age group among the statistical data.

Table 6 shows the distribution of age groups of subjects included in the learning data.

Table 7 shows the age group (the age group used for learning) classified as the data in Table 6, the PSA group not used for learning (eg, middle/high school) and the SA group (eg, infant; infant). ) It shows the predicted probability for preschool age (PSA) and school age (SA) according to the model.

As shown in Table 7, the predicted probability of the preschool age (PSA) group increases as the age decreases, and the predicted probability of the school age (SA) group increases as the age increases. In the case of Adult 3, it can be seen that the rate of speech predicted by school age (SA) decreases again with aging, which is consistent with previous studies that show that language skills deteriorate with age. In addition, the prediction probability of data for infant and middle/high school that are not used for learning shows an accuracy of more than 70%.

In addition, the predicted probability distribution of utterances of individual counselees constituting each age group may be calculated as follows. The probability distribution of prediction of utterances of individual counselors allows the cause of Table 7 to be identified, and the method of predicting the actual language age of individual counselors can be summarized.

6 is a graph showing an individual prediction probability distribution within an age group that is a basis for a statistical analysis operation according to an embodiment of the present invention.

6 is a graph showing the distribution of predicted probability for each individual existing in each age group. In the graph, the circle shape represents the probability predicted by school age (SA) among personal speech data, and the square represents the probability predicted by preschool age (PSA). (a) represents the individual probability distribution for elementary group, (b) represents the individual probability distribution for adult1 group, (c) represents the individual probability distribution for adult2 group, and finally (d) represents the individual probability distribution for adult3 group.

When looking at the prediction results of individual talkers, the difference between the predicted probability for preschool age (PSA) and the predicted probability value for school age (SA) is smaller in the elementary group than in other groups. In addition, it can be seen that about 40% of the cases where the PSA probability value is greater than the SA probability value occurs. This can mean that language development is the age group in which language is actively developed. For example, talker #6 (corresponding to the 6th graph from the left) shows a very high level of language development, while talker #16 (corresponding to the 16th from the left) shows a very slow development level. .

In contrast, in adult group 1 and adult group 2, the probability value of predicting school age is higher than that of predicting preschool age. Also, the difference between the predicted probability value of school age and the predicted probability value of preschool age is larger in Adult 2. This shows that language development has reached its peak. However, in adult group 3, the probability value of school age and the probability value of preschool age are reversed again, which is due to the decline in language ability due to aging.

The artificial intelligence algorithm for evaluating language ability according to an embodiment of the present invention may be configured to infer the language ability of a new subject based on the characteristics of such individual probability distributions. Specifically, the artificial intelligence algorithm according to an embodiment of the present invention first performs an operation for calculating the SA and PSA probabilities of a specific subject, and includes the result in a corresponding graph to perform an operation of comparing with values of other subjects. Can be implemented.

In addition, the artificial intelligence algorithm according to an embodiment of the present invention is configured to determine the number of vocabularies used by the examinee when speaking (i.e., length of sentences), characteristic vocabulary, and speech structure in order to determine the language ability of the examinee. Can be.

The artificial intelligence algorithm may be configured to determine that the lower the number of vocabulary words in the spoken sentence of a specific subject, the lower the language ability. At this time, the length distribution of sentences varies according to each age group, and as the age increases, the length of the sentences increases. The length characteristics of spoken sentences by age can be an index that distinguishes the elementary group and the adult group.

The following [Table 8] shows examples of utterances showing a peculiar distribution, and [Table 9] shows examples of utterances showing correct classification results. Exemplary utterances are omitted for convenience, and actual utterances may be composed of more words than exemplary utterances.

In addition, the artificial intelligence algorithm may determine whether or not a characteristic vocabulary appears according to each age group exists, and when it is determined that a specific characteristic vocabulary exists, may calculate an age group corresponding thereto. For example, since the TV programs mainly watched for each age group will be different, the artificial intelligence algorithm can calculate the age group of the subject, which is indicated by the specific TV program name.

In addition, the artificial intelligence algorithm may predict the age of the subject based on the structure of the speech sentence. Since the collected utterance data of the subject is based on dialogue held in a free atmosphere, some of the constituent elements of the sentence will be omitted in many cases. In consideration of these characteristics, the artificial intelligence algorithm may be configured to evaluate a speaker's language age lower as the sentence structure with many omissions increases.

Table 10 below shows the results of analysis by modifying the sentence and sentence structure including the characteristic vocabulary of the elementary school. As a result of the analysis, it can be seen that the probability of predicting from all age groups to the SA group increased (compared with Table 7).

In addition, the artificial intelligence algorithm may be implemented to determine whether there is a speech impairment, taking into account that the probability of predicting preschool age as preschool age may be higher in some adult (adult 3; adult 3) groups compared to the age group of other school age groups. .

The present invention can evaluate the language ability of a subject by using an artificial intelligence algorithm designed in consideration of the above characteristics.

The evaluation support unit 133 may perform an operation of evaluating the language ability of the subject by using the artificial intelligence algorithm according to an embodiment of the present invention. The evaluation support unit 133 may perform an analysis operation by inputting data for analysis (eg, conversation data or speech data) of a subject into an artificial intelligence algorithm that is learned according to the learning support unit 132 and set parameters. The learning support unit 132 may support performing an analysis operation according to the driving order of the artificial intelligence algorithm according to the embodiment of the present invention described above.

For example, the evaluation support unit 133 extracts the number of vocabulary words from the individual speech sentences of the subject from the dialogue data or speech data extracted by the speech data extraction unit 131 in the artificial intelligence algorithm and padding (Padding; preset) A word embedding (a process of expressing a sentence in a matrix) based on an operation of filling in to generate a sentence of length) may be performed. The word embedding operation may be performed similarly to a process in which the learning support unit 132 learns an artificial intelligence algorithm through a data set. Since a detailed description of the word embedding operation has been previously described, it will be omitted.

In addition, the evaluation support unit 133 may perform a convolution operation on the Sentence Matrix generated by the word embedding operation according to a parameter (eg, filter size) of the determined artificial intelligence algorithm. The evaluation support unit 133 may perform a convolution operation to predict whether to attend school (PSA or SA determination) for each of a plurality of spoken sentences included in the speech data of the subject. Accordingly, the evaluation support unit 133 may acquire result information according to the calculation of the artificial intelligence algorithm.

Furthermore, the evaluation support unit 133 may perform an additional testee's language ability evaluation operation based on a result value calculated from an artificial intelligence algorithm.

In addition, the evaluation support unit 133 can determine the school-age-age prediction probability (SA probability) and preschool-age-age prediction probability (PSA probability) for each one of the subjects by summing up the school-age prediction results for each sentence. have.

In addition, the evaluation support unit 133 may determine at least one numerical information of an SA probability and a PSA probability for one subject. In addition, the evaluation support unit 133 may compare the actual age of the target subject and the age of the subject predicted through at least one of the SA probability and PSA probability. For example, the evaluation support unit 133 may compare the statistical SA probability and PSA probability corresponding to the actual age of the subject, and the SA probability and PSA probability, which are results of analyzing the speech data of the subject.

The evaluation support unit 133 may determine that there is no abnormality when the difference between the statistical probability value corresponding to the actual age group of the subject and the predicted age probability value based on the subject's speech data is less than the reference through the comparison. On the other hand, the evaluation support unit 133 compares the predicted age probability of the subject with the statistical predicted age probability corresponding to the actual age of the subject, and the SA probability of the subject indicates a value lower than the reference value than the statistical SA probability ( Alternatively, if the subject's PSA probability is higher than the reference value compared to the statistical PSA probability), it may be determined that the subject has a speech impairment.

On the other hand, the evaluation support unit 133 compares the predicted age probability of the subject with the statistical predicted age probability corresponding to the actual age of the subject, and the SA probability of the subject shows a value higher than the reference value than the statistical SA probability (or If the test subject's PSA probability is lower than the standard value compared to the statistical PSA probability), it can be determined that the language ability is excellent. Further, the evaluation support unit 133 may determine the level of the language ability of the subject in response to the numerical difference according to the comparison.

7 is a flowchart illustrating a sequence of a language ability evaluation operation according to an embodiment of the present invention.

The control unit 130 of the server 100 according to an embodiment of the present invention may perform operation S105 of processing a preset data set (arbitrary conversation data for training) into learning data for training an artificial intelligence algorithm. Thereafter, the controller 130 may perform operation S100 of learning an artificial intelligence algorithm using the learning data. In this process, the controller 130 may extract and apply parameters for calculation of an artificial intelligence algorithm required for an operation of evaluating language ability. This artificial intelligence learning operation is not limited to one time, and additional learning may be performed by processing the previously analyzed data of the subject into learning data.

In addition, the controller 130 of the server may perform operation S115 of acquiring voice conversation data consisting of conversations between the examinee and the subject from the user device. At this time, the testee device and the tester device may make a call, and conversation contents made during the call may be obtained as conversation data. In addition, the conversation data obtained at this time may be conversation data in a voice form.

The controller 130 of the server may perform operation S120 of converting the acquired voice conversation data into text conversation data and extracting speech data of the subject from the text conversation data.

Thereafter, the controller 130 of the server may perform operation S125 of processing the speech data of a specific subject into data for analysis to be input to the artificial intelligence algorithm in which the learning has been performed. At this time, a noise removal operation may be performed.

The controller 130 of the server may perform operation S130 for calculating by inputting analysis data to an artificial intelligence algorithm, and operation S135 for evaluating the language ability of the subject based on the result of the operation of the artificial intelligence algorithm. At this time, the control unit 130 of the server may evaluate the level of language ability of the subject by comparing the predicted age probability of the subject calculated as a result of the computation of the artificial intelligence algorithm and the statistical predicted age probability corresponding to the actual age of the subject.

In short, the language ability evaluation system according to various embodiments of the present disclosure may include a user device and a server, and the user device transmits conversation data including speech data of a subject to be measured language ability to the server. Provided, and the server performs an artificial intelligence algorithm operation using the conversation data of the subject received from the user device, and evaluates the level of language ability of the subject according to the result of the operation, but calculated through the artificial intelligence algorithm The abnormality can be determined by comparing the predicted age information of the previously registered subject with the actual age of the previously registered subject.

The user device may be configured to include a subject device and a tester device, and the server supports matching and call connection between any test subject device and tester device, and provides voice-type conversation data recording conversation contents performed during a call. To generate voice-type conversation data by recording the contents of the receiving communication unit and the conversation between the examinee and the examinee during the call, and transmit the generated speech-type conversation data to at least one of the examinee's device and the examinee's device to the server. It may be configured to include a requesting control unit.

And the control unit converts the speech-type conversation data into a text form to generate text conversation data, and a speech data extracting unit that extracts speech data of the examinee from the text conversation data based on the speech frequency characteristic of the examinee, and speech of the examinee. A learning support unit that generates learning data to be put into an artificial intelligence algorithm including at least one of the data and a preset data set, and inserts the learning data into the artificial intelligence algorithm to perform learning, and from the speech data of the subject to the artificial intelligence algorithm. It may include an evaluation support unit that generates data for analysis to be injected, and determines whether or not the subject's language ability is abnormal by inputting and calculating the generated analysis data to an artificial intelligence algorithm.

When generating the learning data, the learning support unit labels predicted age information on each spoken sentence so that the analysis result of the speech data of the subject is calculated as a probability of a school age (SA) group and a probability of a pre school age (PSA) group. And, by classifying the training data of the SA group and the training data of the PSA group, noise removal is performed, but by setting the noise removal criterion for the training data of the PSA group to a lower value than the training data of the SA group, the noise ratio to be removed is set. Can be reduced. The reason for the differential setting of the noise removal criterion is that most of the learnable data is recognized as noise when the SA group noise removal criterion is applied to the training data of the PSA group.

The evaluation support unit determines whether the subject's language ability is abnormal as a result of analyzing the analysis data through an artificial intelligence algorithm, and calculates a predicted age probability for the subject's speech data through the analysis of the artificial intelligence algorithm, Comparing the predicted age probability of the subject and the statistical predicted age probability corresponding to the actual age group of the subject, when the SA probability of the subject exhibits a value lower than the reference value than the statistical SA probability, it may be determined that the subject has a speech impairment. .

On the other hand, the evaluation support unit compares the predicted age probability of the subject with the statistical predicted age probability corresponding to the actual age of the subject, and when the SA probability of the subject shows a value higher than the reference value than the statistical SA probability, the numerical difference Correspondingly, the degree of excellence (eg, excellence grade calculation) of the subject's language ability can be evaluated.

Although the present invention has been described in detail with reference to the above-described examples, those skilled in the art can make modifications, changes, and modifications to these examples without departing from the scope of the present invention. In short, in order to achieve the intended effect of the present invention, it is not necessary to separately include all functional blocks shown in the drawings or to follow all the sequences shown in the drawings as shown in the order shown. Note that it may fall within the range.

100: server
110: communication department
120: storage
130: control unit
131: speech data extraction unit
132: Learning Support Department
133: Evaluation Support Department

Claims (8)

In an artificial intelligence-based language ability evaluation system comprising a user device and a server,
The user device
Provides conversation data including speech data of the subject who wants to measure language ability to the server,
The server
Performs an artificial intelligence algorithm operation using the conversation data of the subject received from the user device, and evaluates the level of language ability of the subject according to the result of the operation, and predicted age information of the subject calculated through the artificial intelligence algorithm And, the language ability evaluation system, characterized in that the abnormality is determined through comparison with the actual age of the previously registered subject. The method of claim 1,
The user device
It may be configured to include a subject device and a tester device,
The server
A communication unit that supports matching and call connection between an arbitrary testee device and the tester device, and receives voice-type chat data obtained by recording chat contents performed during a call; And
A control unit for requesting at least one of the testee device and the tester device to generate voice-type conversation data by recording the contents of the conversation between the subject and the examinee during the call, and transmit the generated voice-type conversation data to a server; Language ability evaluation system comprising a. The method of claim 2,
The control unit
A speech data extracting unit configured to convert the speech-type conversation data into a text format to generate text conversation data, and extract speech data of the examinee from the text conversation data based on a voice frequency characteristic of the examinee;
A learning support unit for generating learning data to be injected into an artificial intelligence algorithm including at least one of speech data of the subject and a preset data set, and performing learning by inputting the learning data into an artificial intelligence algorithm; And
And an evaluation support unit that generates data for analysis to be injected into the artificial intelligence algorithm from speech data of the subject, and determines whether or not the subject's language ability is abnormal by inputting and calculating the generated analysis data to the artificial intelligence algorithm. Language ability evaluation system. The method of claim 3,
The learning support unit
When generating the learning data, the predicted age information is labeled on each of the speech sentences so that the analysis result of the subject's speech data is calculated as a probability of a school age (SA) group and a probability of a pre school age (PSA) group,
Noise removal is performed by classifying the training data of the SA group and the training data of the PSA group, but by setting the noise removal criterion for the training data of the PSA group to a lower value than the training data of the SA group, reducing the removed noise ratio. Language ability evaluation system, characterized in that. The method of claim 3,
The evaluation support unit
It is determined whether or not the subject's language ability is abnormal as a result of analyzing the analysis data through an artificial intelligence algorithm,
The predicted age probability for the subject's utterance data is calculated through the analysis of the artificial intelligence algorithm, and compared with the predicted age probability of the subject and the statistical predicted age probability corresponding to the subject's actual age, the SA probability of the subject is statistically If the value is lower than the reference value compared to the SA probability, it is judged that the subject has a speech disorder,
Compared with the predicted age probability of the subject and the statistical predicted age probability corresponding to the actual age group of the subject, when the SA probability of the subject shows a value higher than the reference value than the statistical SA probability, it is determined that the subject's language ability is excellent. , The language ability evaluation system, characterized in that to determine the superiority of the language ability of the subject in response to the numerical difference. In an artificial intelligence-based language ability evaluation system comprising a user device and a server,
The user device
Provides conversation data including speech data of the subject who wants to measure language ability to the server,
The server is
Performs an artificial intelligence algorithm operation using the conversation data of the subject received from the user device, and evaluates the level of language ability of the subject according to the result of the operation, and predicted age information of the subject calculated through the artificial intelligence algorithm Wow, it is determined whether there is an abnormality through comparison with the actual age of the previously registered subject,
The server is
A communication unit that supports matching and call connection between an arbitrary testee device and the tester device, and receives voice-type chat data obtained by recording chat contents performed during a call; And
A control unit for requesting at least one of the testee device and the tester device to generate voice-type conversation data by recording the contents of the conversation between the subject and the examinee during the call, and transmit the generated voice-type conversation data to a server; Language ability evaluation system comprising a. In an artificial intelligence-based language ability evaluation system comprising a user device and a server,
The user device
Provides conversation data including speech data of the subject who wants to measure language ability to the server,
The server
Performs an artificial intelligence algorithm operation using the conversation data of the subject received from the user device, and evaluates the level of language ability of the subject according to the result of the operation, and predicted age information of the subject calculated through the artificial intelligence algorithm Wow, it is determined whether there is an abnormality through comparison with the actual age of the previously registered subject,
The server
A speech data extracting unit configured to convert the speech-type conversation data into a text format to generate text conversation data, and extract speech data of the examinee from the text conversation data based on a voice frequency characteristic of the examinee;
A learning support unit for generating learning data to be injected into an artificial intelligence algorithm including at least one of speech data of the subject and a preset data set, and performing learning by inputting the learning data into an artificial intelligence algorithm; And
And an evaluation support unit that generates data for analysis to be injected into the artificial intelligence algorithm from speech data of the subject, and determines whether or not the subject's language ability is abnormal by inputting and calculating the generated analysis data to the artificial intelligence algorithm. Language ability evaluation system. In the language ability evaluation method performed by the language ability evaluation system comprising a user device and a server,
Processing, by the server, a preset data set into learning data for learning an artificial intelligence algorithm;
Learning, by the server, an artificial intelligence algorithm using the learning data;
Obtaining, by the server, voice conversation data consisting of conversations between an examiner and a subject from the user device;
Converting, by the server, the obtained voice conversation data into text conversation data, and extracting speech data of the subject from the text conversation data;
Processing, by the server, the speech data of a specific subject into analysis data to be input to the artificial intelligence algorithm in which the learning has been performed; And
And evaluating, by the server, the language ability of the subject based on a result of the calculation by inputting the analysis data to an artificial intelligence algorithm.

Images