KR102569339B1 - Speaking test system - Google Patents

Abstract

The present invention relates to a speaking test system that provides detailed analysis results on pronunciation and tone by measuring pronunciation accuracy through an artificial neural network.

Description

Speaking test system {Speaking test system}

The present invention relates to a speaking test system, and more particularly, to a speaking test system that measures pronunciation accuracy through an artificial neural network and provides detailed analysis results on pronunciation and tone.

In general, off-line learning has secured an educational service area through schools, general private academies, and tutoring and other home study materials markets. However, collective education such as schools based on PBT (paper based training) learning is a form of learning the same curriculum in the same space, and does not consider individual learning levels, due to the fragmentary time constraints of the instructor and unilateral learning It was an education in which the achievement of learners was not considered. In other words, some students understand the contents of the class after the end of the class, but in the case of students who do not understand, most of them pass the process without reconfirming the contents.

The biggest problem with constructivist learning is that when a superiority/inferiority situation occurs between peer groups, it is very difficult for members with a sense of inferiority to ask the instructor to retake the course for their insufficient part after the end of the class.

As a result, there are cases in which students want to reacquire it through private academies, private tutoring, or study materials, which incurs a lot of opportunity costs in the course of learning, increasing private education expenses and becoming a major burden on households.

On the other hand, such offline learning has been expanded to online learning (e-Learning) through the Internet, etc., which can be learned through a new method called online service since 2001 as the IT industry develops.

This online learning method has the advantage of being able to perform repeated learning, regardless of time or place, and being able to learn in advance by taking lectures that have already been filmed.

For this reason, online learning has become an axis of the education industry as one of the learning methods equivalent to the offline learning method as a supplement or alternative to offline learning through such continuous growth.

However, since conventional real-time education through online has no choice but to rely solely on the monitor for the learner's condition, unlike offline education, guidance for learners taking classes is extremely limited.

In particular, when the relationship between educators and learners is one-to-many, it is almost impossible to control the learners, so it is difficult for the educator to detect and induce the learner to focus on learning even when the learners doze off or do unnecessary words and actions like offline. There was a fatal problem that the same learning effect as offline could not be achieved.

On the other hand, the above-mentioned background art is technical information that the inventor possessed for derivation of the present invention or acquired in the process of derivation of the present invention, and cannot necessarily be said to be known art disclosed to the general public prior to filing the present invention. .

Korean Patent Publication No. 10-2022-0126412

One aspect of the present invention provides a speaking test system that measures pronunciation accuracy through an artificial neural network and provides detailed analysis results for pronunciation and tone.

The technical problem of the present invention is not limited to the technical problem mentioned above, and other technical problems not mentioned will be clearly understood by those skilled in the art from the following description.

A speaking test system according to an embodiment of the present invention includes a learner terminal for recognizing a learner's voice spoken in response to text output through a display module or sound output through a speaker module; and an educational content management server that analyzes voice data received from the learner terminal through a pre-learned artificial neural network and provides detailed analysis results on pronunciation and tone.

The educational content management server,

A learner analysis unit that evaluates the learner's ability according to preset evaluation criteria and generates an analysis report for each learner terminal;

The learner analysis unit,

Evaluation content for testing the learner's speaking ability is transmitted to the learner's terminal, and the response data received from the learner's terminal for the pre-evaluation content is analyzed through an artificial neural network, and detailed analysis results for pronunciation and tone accuracy are included. generate test reports,

Analyzing the response data through an artificial neural network,

For a specific test item, the voice data received from the learner's terminal is classified into words, and a first feature vector representing the meaning of the word and a second feature vector representing the tone characteristics of the word are extracted for each word,

Dividing pre-stored answer data for a specific test item into words, extracting a first feature vector representing the meaning of the word for each word and a second feature vector representing the tone characteristics of the word,

The first feature vector and the second feature vector extracted from the correct word and the first feature vector and the second feature vector extracted from the word corresponding to the correct word among the learner's voice data are input to the artificial neural network trained in advance. From this, the phonetic similarity and the tonal similarity between the correct word and the pronunciation word are calculated.

The learner analysis unit,

An evaluation score is calculated for each learner terminal using Equation 1 below.

[Equation 1]

Here, St is the final evaluation score of the learner terminal, S1 is the first evaluation score calculated through Equation 2 below, S2 is the second evaluation score calculated through Equation 3 below, and S3 is calculated through Equation 4 below. 3rd evaluation score, α is the first weight, β is the second weight, and γ is the third weight.

[Equation 2]

Here, S1 is the primary evaluation score, Nic is the number of correct words for the ith question, Nia is the number of response words received from the learner terminal for the ith question, Ti is the time required to solve the ith question, n is the number of problem data, la is the average number of response words of other learners, Na is the average number of words in problem data, wi is the weight assigned to each problem data, and wa is the average weight.

[Equation 3]

Here, S2 is the secondary evaluation score, ts is the critical response time for each item, tr is the actual response time for each item, w1 is the first constant, tsn is the average response critical time, and trn is the average response time of other learners.

[Equation 4]

Here, S3 is the secondary evaluation score, vs is the embedding vector of the correct word, vr is the embedding vector of the response word, vsn is the embedding vector of the word located next to the correct word, vt is the embedding vector of the trap word, vtn is the embedding vector of the trap word is the embedding vector of the word located at .

According to one aspect of the present invention described above, the current level is scored and calculated based on the result of the pre-test of the learner, and a learning problem based on the calculated result can be customized.

1 is a diagram showing a schematic configuration of a problem providing system based on learner test scores according to an embodiment of the present invention.
FIG. 2 is a diagram showing a specific configuration of the educational content management server shown in FIG. 1 .

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS The detailed description of the present invention which follows refers to the accompanying drawings which illustrate, by way of illustration, specific embodiments in which the present invention may be practiced. These embodiments are described in sufficient detail to enable one skilled in the art to practice the present invention. It should be understood that the various embodiments of the present invention are different from each other but are not necessarily mutually exclusive. For example, specific shapes, structures, and characteristics described herein may be implemented in another embodiment without departing from the spirit and scope of the invention in connection with one embodiment. Additionally, it should be understood that the location or arrangement of individual components within each disclosed embodiment may be changed without departing from the spirit and scope of the invention. Accordingly, the detailed description set forth below is not to be taken in a limiting sense, and the scope of the present invention, if properly described, is limited only by the appended claims, along with all equivalents as claimed by those claims. Like reference numbers in the drawings indicate the same or similar function throughout the various aspects.

Hereinafter, preferred embodiments of the present invention will be described in more detail with reference to the drawings.

1 is a diagram showing a schematic configuration of a speaking test system according to an embodiment of the present invention.

The speaking test system according to the present invention aims to provide detailed analysis results on pronunciation and tone by measuring pronunciation accuracy through an artificial neural network.

Specifically, the speaking test system according to the present invention includes a learner terminal 100 and an educational content management server 200.

The learner terminal 100 is an electronic device possessed by a learner who wishes to receive online learning services, and is capable of wired and wireless communication with external devices and capable of inputting and processing information, such as smartphones, PCs, laptops, tablet PCs, wearable devices, It may be in the form of a kiosk or the like.

The learner accesses the educational content management server 200 through the learner terminal 100 and receives the online-based multi-learning service according to the present invention from the educational content management server 200.

The educational content management server 200 is connected to a plurality of learner terminals 100 through wired or wireless communication and provides customized online education services for each learner terminal 100 .

2 is a diagram showing a specific configuration of such an educational content management server 200.

As shown, the educational content management server 200 includes a learner analysis unit 210 and a learning content management unit 220.

The learner analysis unit 210 calculates an evaluation score for each learner terminal based on the learner data stored in the educational content management server 200 .

Here, the learner data stored in the educational content management server 200 refers to information input for the learner to use the online-based multi-learning system according to the present invention, such as task performance data, test data, and learning history data, and online-based multi-learning data. It includes all information generated in the process of using the learning system.

Specifically, the learner analysis unit 210 generates pre-evaluation content consisting of a 1st test item group, a 2nd test item group, and a 3rd test item group, transmits it to the learner terminal, and receives the pre-evaluation content from the learner terminal. An evaluation score is calculated based on the received response data.

At this time, the learner analysis unit 210 configures each item group based on item response theory.

It is a test theory that focuses on each item constituting the test and estimates the potential characteristics or abilities of the test subject and the characteristics of the items according to the unique item characteristic curve for each item. The item characteristic curve, a key element of item response theory, represents the functional relationship between the subject's latent characteristics or abilities and the probability of answering the item correctly. This functional relationship develops the theory based on the basic assumption of regional independence, that item scores can be predicted with the subject's latent characteristics or abilities, and that the response to one question and the response to another question of a subject with a certain ability are mutually independent.

For example, let the first test item group consist of basic questions, the second test item consist of questions about reading a question and choosing the correct answer, and the third test item consist of listening to the word in the question and the first word in the word. It consists of problems of choosing sounds.

The learner analysis unit 210 generates pre-evaluation content composed of these questions, transmits it to the learner terminal, and calculates an evaluation score based on response data received from the learner terminal for the pre-evaluation content. A detailed process of calculating the evaluation score will be described later.

In addition, the learner analysis unit transmits evaluation contents for testing the learner's speaking ability to the learner terminal in order to measure and evaluate the learner's speaking ability, and transmits response data received from the learner terminal with respect to the pre-evaluation content to the artificial neural network. It generates a test report with detailed analysis results on pronunciation and tone accuracy.

To this end, the learner analysis unit classifies voice data received from the learner terminal for a specific test item into words, and for each word, a first feature vector representing the meaning of the word and a second feature vector representing the tone characteristics of the word. extract

At the same time, the learner analysis unit divides the pre-stored correct answer data for a specific test item by word, and extracts a first feature vector representing the meaning of the word and a second feature vector representing the tone characteristics of the word for each word. .

Thereafter, the learner analysis unit transmits the first feature vector and the second feature vector extracted from the correct word and the first feature vector and the second feature vector extracted from the word corresponding to the correct word among the learner's voice data to the pre-learned artificial neural network. and the phonetic similarity and tonal similarity between the correct word and the pronunciation word are calculated based on the output value of the artificial neural network.

Here, the artificial neural network may be in the form of a deep neural network composed of an input layer, a hidden layer, and an output layer.

Thereafter, the learner analysis unit generates a test report including information on pronunciation accuracy and tone accuracy of the learner based on the calculated similarity.

3 and 4 are diagrams showing specific examples of test reports generated by the learner analysis unit.

As shown, the learner analysis unit may include in the test report an analysis result of how the specific word pronounced by the learner sounded like, and accordingly, the learner may include information about pronunciation habits to be corrected through the test report. can be provided in detail.

In addition, the learner analysis unit accumulates and stores test reports generated for each learner, so that the learner can check the change in his or her speaking ability at a glance over time.

In addition, although not shown, the educational content management server 200 may further include components such as a learning content management unit, a live class management unit, a quiz content management unit, a gamification content management unit, a project management unit, and an online mall management unit.

The learning content management unit provides online learning content corresponding to the learner's competency evaluated by the learner analysis unit to the learner's terminal.

The learning content management department provides an optimized education course with colorful digital contents such as E-Library Channel, US School Channel, Game Channel, Toon Channel, and Media Channel that arouse interest according to the level and stage of the learner.

The live class management unit 230 provides a real-time video conference service so that live classes can be made between the instructor terminal and the learner terminal that lectures the online learning content.

The quiz content management unit 240 provides a quiz for checking achievement corresponding to the learner's competency evaluated by the learner analysis unit to the learner terminal.

The quiz content management unit 240 can maximize the efficiency of problem management by digitizing all problems created and accumulating them together with meta information in a database.

The gamification content management unit 250 provides game content related to the online learning content to a learner terminal.

The gamification content management unit 250 controls game content suitable for any one of five learning types such as Unscramble Words, Unscramble Sentence, Sentence Speaking, Sentence Typing, and Word Typing to be output from the learner terminal 100.

The project management unit 260 creates learner-directed learning content.

Specifically, the project management unit sets a learning goal through the learner terminal, sets a weekly project to achieve the set learning goal, so that the learner can acquire background knowledge for each weekly project, related to the weekly project. It is characterized in that the learner-directed learning content is created through the step of providing background knowledge information to the learner's terminal and the step of receiving presentation materials created through an application pre-installed on the learner's terminal.

The online mall management unit accumulates and manages the rewards (points) generated in the learning process for each learner, uses the accumulated rewards to purchase various products, and the learners plan and produce products themselves through the educational content management server (200). Register on the web page linked to and sell and purchase through the online mall.

On the other hand, the educational content management server 200 recommends customized learning through data mining of the learner's big data accumulated in the database, and recommends customized educational content suitable for the student by analyzing the student's interest, tendency, and learning content. .

To this end, the learner analysis unit calculates an evaluation score for each learner terminal using Equation 1 below.

[Equation 1]

Here, St is the final evaluation score of the learner terminal, S1 is the first evaluation score calculated through Equation 2 below, S2 is the second evaluation score calculated through Equation 3 below, and S3 is calculated through Equation 4 below. 3rd evaluation score, α is the first weight, β is the second weight, and γ is the third weight.

[Equation 2]

Here, S1 is the primary evaluation score, Nic is the number of correct words for the ith question, Nia is the number of response words received from the learner terminal for the ith question, Ti is the time required to solve the ith question, n is the number of problem data, la is the average number of response words of other learners, Na is the average number of words in problem data, wi is the weight assigned to each problem data, and wa is the average weight.

[Equation 3]

Here, S2 is the secondary evaluation score, ts is the critical response time for each item, tr is the actual response time for each item, w1 is the first constant, tsn is the average response critical time, and trn is the average response time of other learners.

[Equation 4]

Here, S3 is the secondary evaluation score, vs is the embedding vector of the correct word, vr is the embedding vector of the response word, vsn is the embedding vector of the word located next to the correct word, vt is the embedding vector of the trap word, vtn is the embedding vector of the trap word is the embedding vector of the word located at .

In this way, in the evaluation of the third-order responsive problem, the learner analysis unit comprehensively measures the similarity between the word set as the correct answer and the word the learner responded to, and the similarity between the trap word, which is a pronunciation word that is easily misunderstood as the correct answer, and the word the learner responded to. It is possible to improve the reliability of evaluation by evaluating the third-order responsive problem area in consideration of

Meanwhile, the first weight (α), the second weight (β), and the third weight (γ) described in Equation 1 may be calculated by the following equation.

[Equation 5]

Here, α is the first weight, n is the total number of items, and qi is the score of the ith item.

[Equation 6]

Here, β is characterized in that the second weights h, k, and q are constants, h is less than k, and q is a number greater than 0.

5 is a diagram showing an example of hx ³ +kx ² +q described in Equation 6, the y-axis represents the difficulty of the question, and the x-axis represents the question. For example, if the second test consists of the j-th item to the k-th item, the second weight corresponds to the result of integrating the difficulty function for each item.

[Equation 7]

In this way, the learner analysis unit calculates and calculates the first evaluation score for the first test result, the second evaluation score for the second test result, and the third evaluation score for the third test result using the above-described equations. The calculated evaluation scores are calculated to calculate the final evaluation score.

In addition, the educational content management server 200 pre-assigns and manages level scores for each educational content, and provides learner-customized educational content by providing online educational content having a level score corresponding to the calculated evaluation score to the learner terminal. and the reliability of the analysis process can be improved.

In some other embodiments, the educational content management server 200 may further include an instructor evaluation unit (not shown) that analyzes review data for each instructor received from the learner terminal 100 and calculates a rating for each instructor.

For example, the instructor evaluator may build an artificial neural network that extracts contextual information from input data.

Here, the input data may be review data for the instructor.

The instructor evaluation unit may classify review data received from the learner terminal 100 for each instructor, accumulate and store the review data, and extract the stored review data as learning data.

The instructor evaluation unit may build a neural network that learns the learning data using the Word2Vec algorithm and extracts contextual information from the input data.

The Word2Vec algorithm may include a Neural Network Language Model (NNLM). The neural network language model is basically a neural network consisting of an input layer, a projection layer, a hidden layer, and an output layer. A neural network language model is what is used to vectorize words. Since the neural network language model is a well-known technology, a detailed description thereof will be omitted.

The Word2vec algorithm, for text mining, is an algorithm that determines the degree of proximity by looking at the relationship between each word before and after it. Word2vec algorithm is an unsupervised learning algorithm. As the name suggests, the Word2vec algorithm may be a metric technique that expresses the meaning of a word in a vector form. The Word2vec algorithm can represent each word as a vector in a space of about 200 dimensions. Using the Word2vec algorithm, a vector corresponding to a word can be obtained for each word.

The Word2vec algorithm can dramatically improve precision in the field of natural language processing compared to other conventional algorithms. Word2vec can learn the meaning of words by using the relationship between words in the sentences of the input corpus and adjacent words. The Word2vec algorithm is based on an artificial neural network and starts from the premise that words with the same context have close meanings. The Word2vec algorithm learns through text documents, and learns other words that appear nearby (about 5 to 10 words before and after) a word as related words to the artificial neural network. Since words with related meanings are highly likely to appear close to each other in the document, the two words can gradually have close vectors in the process of repeating learning.

The learning method of the Word2vec algorithm includes the CBOW (Continuous Bag Of Words) method and the skip-gram method. The CBOW method predicts a target word using the context created by surrounding words. The skip-gram method predicts words that may come around based on one word. For large datasets, the skip-gram method is known to be more accurate.

Therefore, in the embodiment of the present invention, the Word2vec algorithm using the skip-gram method is used. For example, if learning is successfully completed through the Word2vec algorithm, similar words in a high-dimensional space can be located nearby. According to the above-described Word2vec algorithm, the calculated vector value may be similar to a word having a closer distribution of neighboring words in a learning document, and words having similar calculated vector values may be regarded as similar. Since the Word2vec algorithm is a well-known technology, a detailed description of vector value calculation will be omitted.

The instructor evaluation unit may divide the evaluation level into a plurality of evaluation levels and set evaluation standard text corresponding to each evaluation level. For example, the instructor evaluator may access an external server to which a rating has been assigned by an expert according to an evaluation result based on review data, and may obtain evaluation criterion text for each of a plurality of rating stages.

The instructor evaluator may input evaluation criterion text for each rating level into the neural network, and extract a reference vector value for each rating level representing context information about the evaluation standard text for each rating level.

The instructor evaluation unit may extract an evaluation result vector value representing context information by inputting evaluation result text for the review data received from the learner terminal 100 to the neural network.

The instructor evaluation unit may calculate a similarity between the evaluation result vector value and a plurality of reference vector values, and extract a reference vector value having the highest similarity with the evaluation result vector value among the plurality of reference vector values. In this case, as a similarity calculation method, Euclidean distance, cosine similarity, Tanimoto coeffieient, and the like may be adopted.

The instructor evaluation unit may calculate a rating level corresponding to a criterion vector value having the highest similarity with the evaluation result vector value as a rating of the corresponding instructor.

The instructor evaluation unit may match and store the calculated ratings for each instructor, and guide the ratings for each instructor through an application executed by the learner 100 .

Such technology according to the present invention may be implemented as an application or implemented in the form of program instructions that can be executed through various computer components and recorded on a computer-readable recording medium. The computer readable recording medium may include program instructions, data files, data structures, etc. alone or in combination.

Program instructions recorded on the computer-readable recording medium may be those specially designed and configured for the present invention, or those known and usable to those skilled in the art of computer software.

Examples of computer-readable recording media include magnetic media such as hard disks, floppy disks and magnetic tapes, optical recording media such as CD-ROMs and DVDs, and magneto-optical media such as floptical disks. media), and hardware devices specially configured to store and execute program instructions, such as ROM, RAM, flash memory, and the like.

Examples of program instructions include high-level language codes that can be executed by a computer using an interpreter or the like as well as machine language codes such as those produced by a compiler. The hardware device may be configured to act as one or more software modules to perform processing according to the present invention and vice versa.

Although the above has been described with reference to embodiments, those skilled in the art will understand that the present invention can be variously modified and changed without departing from the spirit and space of the present invention described in the claims below. You will be able to.

100: learner terminal
200: educational content management server

Claims (3)

a learner terminal for recognizing a learner's voice uttered in response to text output through a display module or sound output through a speaker module; and
In the speaking test system including an educational content management server that analyzes the voice data received from the learner terminal through a pre-learned artificial neural network and provides detailed analysis results for pronunciation and tone,
The educational content management server,
A learner analysis unit that evaluates the learner's ability according to preset evaluation criteria and generates an analysis report for each learner terminal;
The learner analysis unit,
The evaluation content for testing the learner's speaking ability is transmitted to the learner's terminal, and the response data received from the learner's terminal for the pre-evaluation content is analyzed through an artificial neural network, which includes detailed analysis results on the accuracy of pronunciation and tone. Generate a test report,
Analyzing the response data through an artificial neural network,
For a specific test item, the voice data received from the learner's terminal is classified into words, and a first feature vector representing the meaning of the word and a second feature vector representing the tone characteristics of the word are extracted for each word,
Dividing pre-stored answer data for a specific test item into words, extracting a first feature vector representing the meaning of the word for each word and a second feature vector representing the tone characteristics of the word,
The first feature vector and the second feature vector extracted from the correct word and the first feature vector and the second feature vector extracted from the word corresponding to the correct word among the learner's voice data are input to the artificial neural network trained in advance. Calculate the phonetic similarity and the tonal similarity between the correct word and the pronunciation word from
The learner analysis unit,
A speaking test system that calculates an evaluation score for each learner terminal using Equation 1 below.

[Equation 1]

Here, St is the final evaluation score of the learner terminal, S1 is the first evaluation score calculated through Equation 2 below, S2 is the second evaluation score calculated through Equation 3 below, and S3 is calculated through Equation 4 below. A third evaluation score, α is a first weight calculated through Equation 5 below, β is a second weight calculated through Equation 6 below, and γ is a third weight.

[Equation 2]

Here, S1 is the primary evaluation score, Nic is the number of correct words for the ith question, Nia is the number of response words received from the learner terminal for the ith question, Ti is the time required to solve the ith question, n is the number of problem data, la is the average number of response words of other learners, Na is the average number of words in problem data, wi is the weight assigned to each problem data, and wa is the average weight.

[Equation 3]

Here, S2 is the secondary evaluation score, ts is the critical response time for each item, tr is the actual response time for each item, w1 is the first constant, tsn is the average response critical time, and trn is the average response time of other learners.

[Equation 4]

Here, S3 is the secondary evaluation score, vs is the embedding vector of the correct word, vr is the embedding vector of the response word, vsn is the embedding vector of the word located next to the correct word, vt is the embedding vector of the trap word, vtn is the embedding vector of the trap word is the embedding vector of the word located at .

[Equation 5]

Here, α is the first weight, n is the total number of items, and qi is the score of the ith item.

[Equation 6]

Here, β is a second weight, h, k, and q are constants, h is less than k, and q is a number greater than 0.
delete delete