KR102509367B1 - Learning amount recommendation method and apparatus using deep learning-based artificial intelligence composed of a plurality of hidden layers that compensate for overfitting tendencies through dropout - Google Patents

Abstract

A learning amount recommendation method and device using deep learning-based artificial intelligence composed of a plurality of hidden layers that compensate for overfitting tendency through dropout are disclosed. The learning amount recommendation device using deep learning-based artificial intelligence composed of the plurality of hidden layers comprises: a learning data collection unit that collects a plurality of learning data from student terminals corresponding to a plurality of users; an academic achievement ability prediction unit that predicts an academic achievement ability type corresponding to each user of the student terminals using the plurality of learning data; a learning amount table generator that generates a learning amount table by pre-processing the learning data according to the predicted academic achievement ability type; and a recommended learning amount determining unit that determines a recommended learning amount corresponding to a target user by referring to the learning amount table, and provides the determined recommended learning amount to a student terminal of the target user. The academic achievement ability prediction unit comprises an artificial neural network engine in which supervised-learning is performed using training data composed of learning data for each of the users, corresponding training input values, and training output values labeled with academic achievement ability types through a clustering engine, and which predicts the type of academic achievement ability corresponding to the target user based on a deep learning network composed of the plurality of hidden layers.

Description

Method and apparatus for recommending learning amount using artificial intelligence based on deep learning consisting of multiple hidden layers compensating for overfitting tendency through dropout LAYERS THAT COMPENSATE FOR OVERFITTING TENDENCIES THROUGH DROPOUT}

The present invention relates to a learning amount recommendation technology, and more particularly, to a learning amount recommendation method and apparatus using deep learning-based artificial intelligence composed of a plurality of hidden layers that compensate for overfitting tendency through dropout.

In Korea, parents' interest and enthusiasm for education are very high, and because of this, many people are interested in students' school classes and college entrance exams through them.

The more students increase the amount of learning as much as possible, the more likely they are to achieve good learning outcomes. It becomes a very important issue.

In general, since the current learning result affects the next learning, the next learning amount is individually determined by increasing or decreasing the current learning result, so the results of academic achievement often appear differently.

In order to solve this problem, the conventional technology (Patent Application No. 10-2018-0135206) is based on a predetermined number of recently performed learning result data, and a pre-specified weight in a way that a higher weight is given the more recent in time. is applied to estimate the expected amount of learning.

However, in the case of the above-described prior art, there is a problem that the prediction result may vary greatly depending on the pre-specified weight, and it is difficult to expect a constant prediction result because the criteria for selecting the weight are determined very differently depending on the implementation method. there was

On the other hand, recently, research on artificial intelligence (artificial intelligence), which performs complex tasks by imitating human intelligence, has been actively conducted. ), and in the field of machine learning, deep learning implemented by imitating human neurons is known as a slightly more advanced concept.

A deep neural network (DNN) based on deep learning is an artificial neural network (ANN) that learns various nonlinear relationships, including multiple hidden layers, and provides prediction results based on the learning results, There is a lot of interest and research on this.

Depending on the algorithm, deep neural networks include a deep belief network (DBN) based on unsupervised learning, a deep autoencoder, and the like, and are synthesized for processing two-dimensional data such as images. A convolutional neural network (CNN), a recurrent neural network (RNN) useful for processing time series data, and the like are known.

An object of the present invention to solve the above problems is to provide a method and apparatus for recommending a learning amount using deep learning-based artificial intelligence composed of a plurality of hidden layers compensating for overfitting tendency through dropout.

One aspect of the present invention for achieving the above object provides a learning amount recommendation device using deep learning-based artificial intelligence composed of a plurality of hidden layers.

The device for recommending a learning amount using deep learning-based artificial intelligence composed of a plurality of hidden layers includes a learning data collection unit that collects a plurality of learning data from student terminals corresponding to a plurality of users; an academic achievement predictor predicting an academic achievement ability type corresponding to a user of each of the student terminals by using the plurality of learning data; a learning amount table generation unit generating a learning amount table by pre-processing the learning data according to the predicted academic achievement ability type; and a recommended learning amount determining unit for determining a recommended learning amount corresponding to a target user by referring to the learning amount table, and providing the determined recommended learning amount to a student terminal of the target user.

The academic achievement predictor determines a cluster corresponding to each user by clustering users corresponding to each of the learning data according to the current curriculum, and assigns the academic achievement type assigned to the determined cluster to the corresponding cluster. a clustering engine that predicts a user's academic achievement type belonging to; And supervised-learning is performed using training data composed of training input values corresponding to the learning data of each of the users and training output values labeled with academic achievement types through the clustering engine, and a plurality of hidden layers and an artificial neural network engine that predicts the type of academic achievement corresponding to the target user based on the deep learning network consisting of .

The artificial neural network engine may include an input layer configured of the same number of neurons as the number of component values of the learning feature vector, which receives a learning feature vector obtained by converting learning data according to the current curriculum of each of the users; the plurality of hidden layers for transmitting an output vector calculated using the output values received from the input layer to an output layer; and an output layer which determines a probability corresponding to the output vector by applying an activation function to the output vector, and outputs an output vector having the highest determined probability.

The artificial neural network engine, when receiving the training input value, uses an output vector obtained as an output of the hidden layer and a training output vector indicating the type of academic achievement according to the training output value to obtain a loss function according to the following equation It is calculated, and supervised so that the resultant value of the calculated loss function is minimized,

In the above equation, Ym is the m-th component (m is a natural number greater than or equal to 1) of the target output vector Ym, and Y'm is the m-th component of the output vector Y' obtained as an output of the hidden layer.

The artificial neural network engine further includes a self-attention layer performing a technique according to a self-attention mechanism between the input layer and the hidden layer.

The self-attention concentration layer determines the importance of each of the intermediate operation vectors obtained as an output of the input layer, and selectively selects some of the intermediate operation vectors by excluding intermediate operation vectors having a relatively low level of importance. passed to the hidden layer.

The hidden layers include a first hidden layer, a second hidden layer, a third hidden layer, and a fourth hidden layer sequentially connected to each other, the first to fourth hidden layers are linear layers, and the first to fourth hidden layers At least one of the hidden layers is a dense layer.

Drop-out is applied to at least one of the first to fourth hidden layers based on a probability determined according to the number of hidden layers and the number of neurons each of the hidden layers has.

The probability determined according to the number of hidden layers and the number of neurons each of the hidden layers is defined by the following equation,

In the above equation, NRn may be the average number of neurons included in the first to fourth hidden layers, and HDn is the total number of hidden layers.

In the case of using the learning amount recommendation method and apparatus using deep learning-based artificial intelligence composed of a plurality of hidden layers according to the present invention as described above, the student's academic achievement ability is predicted using an artificial neural network trained in advance, and the prediction By determining the optimal recommended learning amount based on big data and providing it to students according to the learning achievement ability, it is possible to guide students to the most efficient learning amount and maximize the learning effect.

1 is a conceptual diagram illustrating an environment in which a method for recommending a learning amount using artificial intelligence based on deep learning composed of a plurality of hidden layers according to an embodiment is performed.
FIG. 2 is a block diagram showing functional components of an apparatus for recommending a learning amount using artificial intelligence based on deep learning composed of a plurality of hidden layers according to FIG. 1 .
3 is a diagram showing the configuration of the learning amount table according to FIG. 1 by way of example.
4 is a diagram showing the result of calculating the weighted score with reference to the learning amount table according to FIG. 2 .
FIG. 5 is a diagram for explaining the operation of the clustering engine according to FIG. 2 .
6 is a diagram for explaining the operation of the artificial neural network engine according to FIG. 2;
FIG. 7 is a diagram showing the structure of the artificial neural network engine according to FIG. 6 in detail.
8 is a diagram for explaining a method of determining a K vector in a self-attention layer according to an embodiment.
9 is a diagram showing the hardware configuration of an apparatus for recommending a learning amount using deep learning-based artificial intelligence composed of a plurality of hidden layers according to an embodiment by way of example.
10 is a graph showing statistical analysis results for confirming the effect of a learning amount recommendation method and apparatus using deep learning-based artificial intelligence composed of a plurality of hidden layers according to an embodiment.

Since the present invention can make various changes and have various embodiments, specific embodiments will be illustrated in the drawings and described in detail in the detailed description. However, this is not intended to limit the present invention to specific embodiments, and should be understood to include all modifications, equivalents, and substitutes included in the spirit and scope of the present invention. Like reference numerals have been used for like elements throughout the description of each figure.

Terms such as first, second, A, and B may be used to describe various components, but the components should not be limited by the terms. These terms are only used for the purpose of distinguishing one component from another. For example, a first element may be termed a second element, and similarly, a second element may be termed a first element, without departing from the scope of the present invention. The terms and/or include any combination of a plurality of related recited items or any of a plurality of related recited items.

It is understood that when an element is referred to as being "connected" or "connected" to another element, it may be directly connected or connected to the other element, but other elements may exist in the middle. It should be. On the other hand, when an element is referred to as “directly connected” or “directly connected” to another element, it should be understood that no other element exists in the middle.

Terms used in this application are only used to describe specific embodiments, and are not intended to limit the present invention. Singular expressions include plural expressions unless the context clearly dictates otherwise. In this application, the terms "include" or "have" are intended to designate that there is a feature, number, step, operation, component, part, or combination thereof described in the specification, but one or more other features It should be understood that the presence or addition of numbers, steps, operations, components, parts, or combinations thereof is not precluded.

Unless defined otherwise, all terms used herein, including technical or scientific terms, have the same meaning as commonly understood by one of ordinary skill in the art to which the present invention belongs. Terms such as those defined in commonly used dictionaries should be interpreted as having a meaning consistent with the meaning in the context of the related art, and unless explicitly defined in the present application, they should not be interpreted in an ideal or excessively formal meaning. don't

Hereinafter, preferred embodiments according to the present invention will be described in detail with reference to the accompanying drawings.

1 is a conceptual diagram illustrating an environment in which a method for recommending a learning amount using artificial intelligence based on deep learning composed of a plurality of hidden layers according to an embodiment is performed.

Referring to FIG. 1, the learning amount recommendation method using deep learning-based artificial intelligence composed of a plurality of hidden layers is described below. It can also be referred to as a used learning amount recommendation device).

The learning amount recommendation server 100 may collect learning data in association with a plurality of student terminals 200a to 200n.

Each of the student terminals 200a to 200n may have a pre-distributed learning application installed and run, and as the learning application is executed, each of the student terminals 200a to 200n selects a plurality of learning problems according to the curriculum and It may be provided to the user of the student terminal 200 and the user's problem solving result may be stored.

The user of the student terminal 200 accesses the learning volume recommendation server 100 using the student terminal 200 in advance and registers user information in the learning volume recommendation server 100 through a user authentication procedure. User information may include the user's age, grade, school, and the like.

In addition, each of the student terminals 200a to 200n transfers learning data representing problem solving results according to the learning problems provided to the user of the student terminal 200 in real time, at predetermined intervals, or at predetermined time intervals as a learning amount recommendation server. (100) can be provided.

Specifically, the learning data may mean solution results for learning problems provided for at least one lesson according to a weekly or daily educational process (eg, an addition process). The educational process may correspond to a large classification of learning tasks provided according to a curriculum, and the lessons may correspond to a small classification of learning tasks provided within a specific education course. For example, the educational process may have a predetermined order, such as a basic addition course, an advanced addition course, a basic subtraction course, an advanced subtraction course, and the like. Specifically, a curriculum to be learned after the basic course of addition may be an advanced course of addition, and a curriculum to be learned after the advanced course of addition may be a basic course of subtraction.

For example, lessons on the basic course of addition may include addition to single-digit numbers, addition to two-digit numbers, and so on, and lessons on the advanced course of addition may include addition to three-digit numbers, addition to all three or more numbers. It may include addition to .

The learning amount recommendation server 100 collects a plurality of past learning data from the plurality of student terminals 200a to 200n and, based on the collected past learning data, sets each user of the student terminals 200a to 200n as a machine. By clustering using the learning engine 10, it is possible to determine the type of academic achievement capability corresponding to each user.

Here, the machine learning engine 10 may include a clustering engine 102a that operates based on a Gaussian Mixture Model (GMM). The clustering engine 102a may correspond to a kind of source code that implements a Gaussian Mixture Model (GMM) in software or an operation program of the source code.

Since each person has a different innate talent, memory, mathematical calculation ability, etc., users are divided into those who can solve easily and those who cannot easily solve even if they are provided with the same problem. In addition, even among users with the same academic achievement level, there is a tendency that a specific person is good at math but weak in Korean language, while another specific person is relatively poor at math than Korean language.

In an embodiment of the present invention, in consideration of the fact that each person has a different academic achievement ability, a plurality of clusters are formed by clustering users to recommend the most suitable amount of learning according to the academic achievement ability, A type of academic achievement capability corresponding to each of the configured clusters is determined. Here, the academic achievement ability type is a kind of identification symbol that is individually assigned to each cluster to which the user belongs. For example, symbols that are determined without superiority or inferiority such as A, B, C, etc., or various other types of unique identification symbols can be used. . That is, the academic achievement ability type is used as a meaning representing the academic achievement ability of users belonging to the corresponding cluster.

When the academic achievement type corresponding to each user is determined through clustering, the learning amount recommendation server 100 classifies the learning data of the users according to the academic achievement type and the past learning amount, and pre-processes the learning amount. Table 20 can be created.

The learning amount table 20 generated here is used as big data to determine the recommended amount of learning for a specific user in the future, and learning data is collected for as many users as possible, and the academic achievement ability type corresponding to each user is determined. It may be desirable to configure the learning amount table 20 based on the largest amount of big data by determining.

When the learning amount table 20 is generated through clustering, the learning amount recommendation server 100 may operate to provide a recommended learning amount for a specific person.

First, the learning amount recommendation server 100 may receive target learning data from the student terminal 200 of a target user among users, and use the machine learning engine 10 based on the received target learning data to target the target user. It is possible to predict the type of academic achievement ability corresponding to

In this case, the aforementioned clustering engine 102a may be used as the machine learning engine 10, but is not limited thereto.

For example, an artificial neural network engine 102b based on a deep neural network may be used as the machine learning engine 10 . When the artificial neural network engine 102b receives learning data of a specific user using training data consisting of a set of training input values and training output values, an output value indicating an academic achievement type corresponding to the specific user It may be supervised learning in advance to output.

Therefore, the learning amount recommendation server 100 predicts the cluster to which the target user belongs and the corresponding academic achievement type based on the target learning data using the clustering engine 102a, or uses the artificial neural network engine 102b to predict the target user 102b. Based on the learning data, it can be configured to predict the type of academic achievement ability corresponding to the target user.

The learning amount recommendation server 100 may analyze the learning amount table 20 based on the predicted academic achievement type to determine a recommended learning amount for a target user according to the predicted academic achievement type.

For example, the recommended learning amount (or learning amount) may mean the next education course (subtraction process on the drawing) to be learned after the target user's current education course according to the target learning data, the number of learning per week, and the number of lessons per day. .

The learning amount recommendation server 100 may transmit the determined recommended learning amount to the student terminal 200 of the target user.

Since the target user is provided with the student terminal 200 with the recommended amount of learning that is expected to achieve the highest level of achievement in their academic achievement ability type, the possibility of achieving optimal academic achievement results by proceeding with learning according to the recommended amount of learning this may increase

FIG. 2 is a block diagram showing functional components of an apparatus for recommending a learning amount using artificial intelligence based on deep learning composed of a plurality of hidden layers according to FIG. 1 . 3 is a diagram showing the configuration of the learning amount table according to FIG. 1 by way of example. 4 is a diagram showing the result of calculating the weighted score with reference to the learning amount table according to FIG. 2 .

Referring to FIG. 2 , the learning amount recommendation server 100 includes a learning data collection unit 101 that collects learning data from each of the student terminals 200 corresponding to a plurality of users, and collection from the student terminals 200. The academic achievement predicting unit 102 predicts the academic achievement ability type corresponding to each user of the student terminals 200 using a plurality of learning data, and preprocesses the learning data according to the predicted academic achievement ability type ( A learning amount table generation unit 103 for generating a learning amount table by pre-processing, and determining a recommended learning amount corresponding to a target user by referring to the learning amount table, and providing the determined recommended learning amount to the student terminal 200 of the target user A recommended learning amount determining unit 104 may be included.

The learning data collection unit 101 may collect learning data in association with a plurality of student terminals 200 . To this end, learning data may be obtained by periodically communicating with the student terminals 200 through a wired or wireless network. For example, when learning according to an educational process is completed in each of the student terminals 200, each of the student terminals 200 may transmit learning data according to completion of learning to the learning data collection unit 101.

The academic achievement ability prediction unit 102 may predict an academic achievement ability type corresponding to each user by using the learning data, and provide the result to the learning amount table generation unit 103 .

Specifically, the academic achievement predictor 102 determines a cluster corresponding to each user by clustering users corresponding to each of the learning data, and a clustering engine that determines the academic achievement type assigned to the determined cluster. (102a).

For example, the clustering engine 102a may perform clustering according to a Gaussian Mixture Model (GMM) or clustering according to a K-means algorithm.

Here, the K-means algorithm may be a machine learning-based clustering technique belonging to unsupervised learning, unlike the Gaussian mixture model. Specifically, the K-means algorithm sets a vector value corresponding to an initial center point for each of the K clusters, and then assigns users to a cluster having an initial center point close to a learning feature vector representing each user.

Next, after cluster assignment to all users is completed, the center point of each cluster is reset to the median or average value of the learning feature vectors of users belonging to the cluster, and cluster assignment to all users is performed again based on the reset center point. do.

Users can be assigned to clusters by repeating the above-described center point resetting and cluster reallocation until there is no change in the center point.

The learning amount table generator 103 may generate the learning amount table 20 by pre-processing the learning data according to the academic achievement type determined by the clustering engine 102a.

The learning amount table generator 103 may generate the learning amount table 20 by pre-processing the learning data according to the academic achievement type determined by the clustering engine 102a.

Specifically, the learning amount table generator 103 determines the academic achievement type determined by the clustering engine 102a based on the learning data according to the user's current education course and the amount of learning performed in the next education course after the current education course. The learning data may be pre-processed by classifying according to the classification and calculating statistical data on the learning data corresponding to the classified academic achievement type and amount of learning.

That is, the academic achievement type is predicted by the clustering engine 102a based on learning data according to the user's current educational process.

Referring to FIG. 3 , the learning amount table 20 classifies learning data according to the current educational process according to the type of academic achievement ability and the amount of learning, and is statistical data obtained through statistical calculation processing on the classified learning data. It may include the evaluation passing rate and the average evaluation score for the obtained training data.

Here, the amount of learning may mean the number of learning per week and the number of daily learning lessons performed in the next curriculum of users of the corresponding academic achievement ability type. As another example, the amount of learning may mean the number of learning days and learning time per week performed in the next curriculum. Here, the amount of learning may be set in units of hours, days, or months according to the criteria for setting the indicator, or may be set in units of a mixture thereof, and since this corresponds to a modification easily performed by a person skilled in the art, a detailed description thereof will be omitted.

Here, the evaluation passing rate may be a ratio of users whose solution result according to the amount of learning provided in the next curriculum exceeds a specific reference score, targeting users determined to have the corresponding academic achievement type.

The average evaluation score may be an average value of evaluation scores in a solution result according to an amount of learning provided in a next curriculum targeting users determined to have a corresponding academic achievement type.

Referring to FIG. 3, among users whose academic achievement ability type in the current curriculum belongs to cluster A, users who have 3 weekly learning times and 3 daily learning lessons according to the amount of learning provided to the next curriculum are 169 , and 169 users, it can be confirmed that the evaluation pass rate for the amount of learning provided in the next curriculum is 92%, and the average evaluation score for the amount of learning is 94 points.

In this way, the learning amount table 20 predicts, using the clustering engine 102a, the academic achievement ability type corresponding to each user, targeting the learning data according to the current curriculum of a very large number of users, and predicts the predicted Depending on the type of academic achievement ability, it may be configured in a way of calculating statistical data for learning data according to the next curriculum.

In this way, the learning amount table 20 is classified according to the predicted academic achievement ability type based on the learning data according to the current curriculum, and the learning amount according to the next curriculum based on the learning data according to the current curriculum of a specific target user It is easy to recommend

When the learning amount table 20 is created, a function of providing a recommended learning amount is activated.

Specifically, the learning data collection unit 101 may obtain target learning data according to the current curriculum of the target user from the student terminal 200 of the target user. Here, the current curriculum may refer to a level of a curriculum that the target user is currently progressing.

The academic achievement ability prediction unit 102 may predict the academic achievement ability type corresponding to the target user based on the target learning data according to the current curriculum.

In an embodiment, the academic achievement ability prediction unit 102 determines a cluster corresponding to the target user by using the clustering engine 102a, and sets the academic achievement capability type assigned to the determined cluster to the academic achievement corresponding to the target user. It can be predicted by ability type.

In another embodiment, the academic achievement predictor 102 may predict the academic achievement type corresponding to the target user by using the deep learning-based artificial neural network engine 102b. The artificial neural network engine 102b may be pre-supervised learning using training data.

In this case, the training input value constituting the training data may be a learning feature vector generated using learning data collected from each of a plurality of users, and the training output value corresponds to each user using the clustering engine 102a. It may be a type of academic achievement ability granted to the cluster.

That is, since the artificial neural network engine 102b receives training output values labeled with academic achievement types through the clustering engine 102a, it is possible to solve the difficulty of collecting massive learning data for supervising the artificial neural network. can

In addition, since the clustering engine 102a is implemented using a machine learning-based Gaussian mixture model, even if the training output value labeled through the clustering engine 102a is used, it can be statistically meaningfully labeled, and thus the learned artificial neural network The predictive accuracy of the engine 102b for the academic achievement type is implemented relatively accurately.

The recommended learning amount determining unit 104 calculates a weighted score according to a predetermined mathematical formula using statistical data for each learning amount corresponding to the type of academic achievement ability predicted for the target user in the learning amount table 20, and calculates the weighted Based on the score, a recommended amount of learning may be determined.

For example, as shown in FIG. 4 , the weighted score is calculated for statistical data according to the type of academic achievement ability and the amount of learning, and can be specifically calculated according to Equation 1 below.

Referring to Equation 1, N is the number of learners corresponding to the type and amount of learning ability, Nmin is the minimum number of learners individually defined according to the type and amount of learning ability, and PP is the amount of learning ability. The evaluation pass rate of the number of learners corresponding to the type and amount of learning, acSC is the average of the evaluation scores of the number of learners corresponding to the type of academic achievement ability and the amount of learning, and k is a weight constant defined individually according to the type of learning achievement ability and the amount of learning , and avgN may be an average value of the number of learners for each type of academic achievement ability.

For example, when the academic achievement type predicted for the target user corresponds to cluster A, as shown in FIG. Thus, a weighted score according to Equation 1 can be calculated for each of the learning amounts.

At this time, the recommended learning amount determining unit 104 may determine, as the recommended learning amount, a learning amount corresponding to the highest calculated weight score among the weighted scores calculated for each learning amount according to Equation 1.

That is, in the case of FIG. 4, since the weighted score is calculated as the highest value when the number of learning times per week is 3 as the learning amount and the number of daily learning lessons is 3, the recommended learning amount in the next curriculum is the number of learning times per week 3 times, and the number of learning lessons per day is determined by the learning amount of 3 individuals.

FIG. 5 is a diagram for explaining the operation of the clustering engine according to FIG. 2 .

The clustering engine 102a may generate a learning feature vector representing the user's current educational process using learning data collected for each user.

For example, referring to FIG. 5 , the clustering engine 102a uses learning data according to the current curriculum collected for each user to solve problems according to the current curriculum and lessons, the number of times of passing the evaluation, and the current curriculum. Preprocessing learning data including the average number of passes, the average score in the current curriculum, and the test score in the lesson may be obtained.

The clustering engine 102a may generate a learning feature vector corresponding to the user by using at least some of the indices included in the obtained preprocessing learning data.

For example, the clustering engine 102a takes the identifier of the lesson in the current curriculum, the number of problem solving in the preprocessing learning data, the number of passing the evaluation, the average number of passing, the average score, and each of the test scores in the lesson as component values. You can create a learning feature vector that

For example, one learning feature vector may be generated for each lesson belonging to the current curriculum.

The clustering engine 102a may classify each of the users into one of a plurality of clusters using at least one learning feature vector according to the current curriculum of each user. For example, in the case of a Gaussian Mixture Model (GMM), each of a predetermined K (K is a natural number greater than 1) Gaussian distributions corresponds to a cluster. It operates to classify users into one of K Gaussian distributions.

Specifically, in the case of the clustering engine 102a, a Gaussian distribution having the highest result value of the Gaussian distribution selection function (γ) according to Equation 2 below is selected from K Gaussian distributions using Bayes' theorem. and the selected Gaussian distribution is determined as the academic achievement ability type corresponding to the user's current curriculum.

In Equation 2, (γ) is a selection function, x _n is the learning feature vector according to the nth lesson in the corresponding curriculum, and z _nk is k in the Gaussian mixture model when the learning feature vector is given (k is 1 and K It is a binary variable with a value of 1, otherwise it is 1 if the natural number between)th Gaussian distribution is selected. Also, μ and Σ in Equation 2 are parameters according to the Gaussian mixture model, and are values determined through a learning process for the Gaussian mixture model in advance. The learning process for determining the parameters according to the Gaussian mixture model can be understood by referring to Christopher Bishop's Pattern Recognition and Machine Learning, 2008.03.18, so a detailed description is omitted.

6 is a diagram for explaining the operation of the artificial neural network engine according to FIG. 2; FIG. 7 is a diagram showing the structure of the artificial neural network engine according to FIG. 6 in detail. 8 is a diagram for explaining a method of determining a K vector in a self-attention layer according to an embodiment.

Referring to FIG. 6, when the deep learning-based artificial neural network engine 102b sequentially receives learning feature vectors for each lesson according to the current curriculum as training input values constituting training data, the artificial neural network engine 102b Supervised learning is performed by comparing the academic achievement ability type obtained as the output value of with the training output value and tuning the parameters constituting the artificial neural network engine 102b according to the comparison result.

Specifically, the deep learning-based artificial neural network engine 102b generates an output vector indicating the type of academic achievement obtained as an output value of the artificial neural network engine 102b and a training output vector indicating the type of academic achievement according to the training output value. Parameters constituting the artificial neural network engine 102b may be adjusted so that the resultant value of the calculated loss function is minimized.

In this case, the loss function may be a cross entropy function.

Referring to FIG. 7 , the artificial neural network engine 102b receives a learning feature vector (X) and inputs an input layer 11 composed of the same number (N) of input nodes as the number of component values of the received learning feature vector. ), the hidden layer 12 that transfers the output vector Y′ calculated using the output values received from the input layer 11 to the output layer 13, and the output vector Y′ by applying an activation function to the output vector It may include an output layer 13 that determines a probability p corresponding to (Y′) and outputs an output vector Y′ having the highest probability p. In the present invention, each of the nodes constituting the artificial neural network engine 102b may be interchangeably referred to as a neuron, which is an expression commonly used in the art to which the present invention belongs.

Specifically, when the artificial neural network engine 102b receives the learning feature vector (X) provided as a training input value, the output vector (Y`) obtained as an output of the hidden layer 12 and the academic achievement capability provided as the training output value A loss function is calculated using the training output vector (Y) indicating the type, and supervised learning is performed so that the resulting value of the calculated loss function is minimized.

For example, the loss function H(Y,Y′) may be a cross entropy function. The cross entropy (H(Y,Y`)) between the output vector (Y`) and the training output vector (Y) can be defined as in Equation 3 below.

In Equation 3, Ym is the mth component (m is a natural number greater than or equal to 1) of the training output vector Y, and Y`m may be the mth component of the output vector Y′.

The input layer 11 receives a learning feature vector (X) and applies one or more connection strength values corresponding to the input nodes to each of the components of the input learning feature vector (X) so that the hidden layer (12) can be conveyed

For example, one or more connection strength values corresponding to each of the input nodes may be expressed as a first connection strength matrix (W _N×M ) having a size of N×M. At this time, N may be the same number as the input nodes, and M is set sufficiently smaller than N by 1/10 times or less. The first connection strength matrix (W _N×M ) may be a parameter that is continuously updated through supervised learning after being set to a random initial value.

In summary, the input layer 11 may transfer the intermediate operation vector (X) obtained by matrix multiplication of the first connection strength matrix (W _N×M ) to the input learning feature vector (X) to the hidden layer (12). .

The hidden layer 12 applies one or more connection strengths corresponding to each of the hidden nodes to a feature vector (F) obtained from an intermediate operation vector (X) received from the input layer 11 to obtain an output vector (Y `) can be generated, and the generated output vector (Y`) can be transmitted to the output layer 13.

For example, the hidden layer 12 may obtain a feature vector F, which is a diagonal matrix that satisfies Equation 4 below, from an intermediate operation vector X.

In Equation 4, R is a matrix that reversibly satisfies the relationship according to Equation 4 between the feature vector (F), which is a diagonal matrix, and the intermediate operation vector (X), and corresponds to the coordinate transformation matrix. The operation according to Equation 4 may be referred to as one of the diagonalization operations, and since it can be easily understood by those skilled in the art, a detailed description thereof will be omitted.

In this case, one or more connection strength values corresponding to each of the hidden nodes may be expressed as a second connection strength matrix (U _M×Q ) having a size of M×Q. That is, the second connection strength matrix (UM×Q) increases the feature vector (F) mapped in M dimensions to Q dimensions again. Q is set to a value that is at least 10 times larger than M.

Meanwhile, after the initial value of the second coupling strength matrix (U _M×Q ) is set to an arbitrary value, an output generated by performing a matrix multiplication operation between the feature vector (F) and the second coupling strength matrix (U _M×Q ) The vector (Y′) can be continuously updated to become the training output vector (Y), which is a training output value. That is, the second connection strength matrix (U _M×Q ) may also be a parameter that is updated as training data is continuously supervised.

That is, since the hidden layer 12 converts the intermediate operation vector (X) received from the input layer 11 into a feature vector and applies the connection strength to the feature vector to generate an output vector (Y`), it is like a CNN The hidden layer 12 may perform a function similar to a convolutional layer responsible for extracting features of a convolutional neural network.

The output layer 13 determines the probability p corresponding to the output vector Y′ by applying an activation function to the output vector Y′ received from the hidden layer 12, and the determined probability p is the highest. An output vector (Y`) can be output. The activation function has the effect of converting values having various ranges into probabilities by expanding or reducing values between 0 and 1. For example, the activation function may be a ReLU function or a Softmax function, but is not limited thereto.

On the other hand, in one embodiment, between the input layer 11 and the hidden layer 12, the number of neurons corresponding to p multiples of 256 (p is a natural number of 2 or more, for example 3), which is a value corresponding to the 8th power of 2 A configured self-attention layer 14 may be further included. A self-attention layer (14) is a layer that performs a technique according to a self-attention mechanism, and is one of the neural network structures that integrates information from a sequence transmitted from the input layer (11). It is kind.

For self-focusing mechanisms, Vaswani, output probability to another component or to another sys-A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, tem, and the like. Since it can be understood by referring to Examples of the output terminal may 45 A. N., Kaiser, L. and Polosukhin, I., 2017., the basic description has been simplified.

On the other hand, in one embodiment of the present invention, the self-attention layer 14 determines the importance of each of the intermediate operation vectors X provided as the output of the input layer 11, and excludes information of relatively low importance. By operating to transfer to the hidden layer 12, it serves to increase the prediction result. In particular, when the learning feature vector (X) for each lesson according to one educational process is sequentially input into the input layer 11, lessons that have a large impact on the next training process and lessons that have a small impact are distinguished. Considering this, the hidden layer It is preferable to configure to increase the prediction success rate by passing to (12).

Specifically, the self-attention-focused layer 14 performs an intermediate operation received from the input layer 11 using vectors corresponding to each of three variables called a query, a key, and a value. After calculating the self-attention value (α) indicating the importance for each vector (X), among the intermediate calculation vectors (X) transmitted from the input layer 11, the middle in which the calculated self-attention value is the highest It may be configured to select an operation vector (X) and pass it to the hidden layer (12).

In order to calculate the self-attention value (α), a q vector corresponding to a query variable, a K vector corresponding to a key variable, and a V vector corresponding to a value variable must first be determined. Here, the q vector is an object to determine the importance, and in the present invention, it may be an intermediate operation vector (X) provided as an output of the input layer 11. The K vector may play a role in deriving a reference value for determining the importance by being operated with the q vector. The V vector serves to derive the self-attention value (α) of an appropriate size for the probability value obtained as a result of the operation between the K vector and the q vector. The q vector, K vector, and v vector are all vectors having the same dimension.

Referring to FIG. 8, in one embodiment, K vector is the ratio of the total number of lessons according to the current curriculum, the ratio of the number of passing evaluations according to the number of problem solving in the lesson, and the ratio of the average score according to the average number of passing in the current curriculum An initial K vector having each of the component values may be constructed.

Next, the initial K vector may be padded using a preset reference value to have a dimension corresponding to the q vector, and the K vector may be determined as a result of shift dot product operation with a mask vector. A mask vector is a vector for data smoothing on component values to be operated, and since various types of smoothing vectors exist, a person skilled in the art can selectively utilize one of well-known smoothing vectors.

The shift dot product operation may mean sequentially performing a dot operation with the padded initial K vector while the mask vector is shifted by one component value. That is, the result values sequentially obtained each time the dot product operation is performed become component values of the K vector.

The V vector may be a vector with fixed component values preset by an administrator.

Based on the determined q vector, K vector, and V vector, the self attention value (α) according to the following equation can be calculated as follows.

Referring to Equation 5 above, activate is an activation function that outputs a probability value for an input value, and may be the aforementioned ReLU function or Softmax function. q is the q vector, K is the K vector, V is the V vector, and d is a constant that lowers the operation result between the q vector and the K vector to a value of a preset magnitude or less, and may be a preset value. .

In one embodiment of the present invention, the hidden layer 12 may be composed of a plurality of hidden layers sequentially connected to each other. For example, the hidden layer 12 may include a first hidden layer, a second hidden layer, a third hidden layer, and a fourth hidden layer sequentially connected to each other. In this case, the first to fourth hidden layers may be linear layers, and at least one of the first to fourth hidden layers may be a dense layer. The designation of the linear layer or the dense layer here can be easily set through a predefined function in Tensorflow, so the description is omitted.

In addition, drop out may be applied to at least one of the first to fourth hidden layers according to a probability p determined according to the number of hidden layers and the number of neurons each of the hidden layers has.

In one embodiment of the present invention, dropout refers to setting the weights of at least some neurons randomly selected according to a probability (p) from among the neurons constituting the layer to 0 or a predefined constant so that some neurons are It can mean disabling or configuring it to interfere.

For example, if the first hidden layer is composed of 768 neurons and the probability (p) is 0.2, for 20% of neurons (i.e., 768/5 neurons) randomly selected among the 768 neurons, The weight can be set to 0 or a predefined constant.

For example, probability p may be defined as in Equation 6 below.

In Equation 6, NRn may be the average number of neurons included in the first to fourth hidden layers, and HDn is the total number of hidden layers, which may be 4 when the first to fourth hidden layers are included.

As such, when dropout is applied to at least one of the first to fourth hidden layers, it is possible to compensate or offset the overfitting tendency that may occur as the number of hidden layers and each neuron increases, resulting in a higher prediction effect. can help bring

9 is a diagram showing the hardware configuration of an apparatus for recommending a learning amount using deep learning-based artificial intelligence composed of a plurality of hidden layers according to an embodiment by way of example.

Referring to FIG. 9, the learning amount recommendation server 100 stores at least one processor 110 and instructions instructing the at least one processor 110 to perform at least one operation. It may include a memory (memory, 120) to.

Here, at least one operation is interpreted as including at least some of the operations or functions of the aforementioned learning amount recommendation server 100, and detailed descriptions are omitted to prevent redundant descriptions.

Here, the at least one processor 110 may mean a central processing unit (CPU), a graphics processing unit (GPU), or a dedicated processor for performing methods according to embodiments of the present invention. can The memory 120 may include at least one of volatile storage media and non-volatile storage media.

The learning amount recommendation server 100 may further include a storage device 160 for non-temporarily storing input data, intermediate processing data, temporary data, and output data according to performing at least one operation.

For example, the memory 120 may be one of a read only memory (ROM) and a random access memory (RAM), and the storage device 160 may be a flash-memory. , a hard disk drive (HDD), a solid state drive (SSD), or various memory cards (eg, a micro SD card).

In addition, the learning amount recommendation server 100 may further include a transceiver 130 that performs communication through a wireless network. In addition, the learning amount recommendation server 100 may further include an input interface device 140 and an output interface device 150. Each component included in the learning amount recommendation server 100 is connected by a bus 170 to communicate with each other.

For example, the learning amount recommendation server 100 may include a communicable desktop computer, a laptop computer, a notebook, a smart phone, a tablet PC, and a mobile phone. phone), smart watch, smart glass, e-book reader, portable multimedia player (PMP), portable game device, navigation device, digital camera, digital multimedia broadcasting (DMB) ) player, digital audio recorder, digital audio player, digital video recorder, digital video player, personal digital assistant (PDA), and the like.

10 is a graph showing statistical analysis results for confirming the effect of a learning amount recommendation method and apparatus using deep learning-based artificial intelligence composed of a plurality of hidden layers according to an embodiment.

Referring to FIG. 10, a learning amount recommendation method using deep learning-based artificial intelligence consisting of a plurality of hidden layers or learning amount recommendation using deep learning-based artificial intelligence consisting of a plurality of hidden layers according to an embodiment of the present invention The probability distribution (with AI) according to the evaluation score (X _i ) for the first user group who has learned in the next curriculum with the recommended amount of learning provided by the device is shown.

In addition, referring to FIG. 10, learning in the next curriculum is not performed with the recommended learning amount provided according to an embodiment of the present invention, but learning in the next curriculum is performed with the learning amount selected by each user. A probability distribution (General) according to evaluation scores (X _i ) for 2 user groups is shown.

Here, the current training course and the next training course performed by the first user group and the second user group are the same, and the number of users belonging to each group is the same. In the drawing, multiplication, division, and fraction/decimal mean the next curriculum, respectively, and have the curriculum order of addition -> multiplication -> division -> fraction/decimal.

At this time, as shown in FIG. 9, all of the following curriculum according to the academic achievement ability type, except for the case where the academic achievement ability type is D type and the next curriculum is fractional/decimal, are all according to an embodiment of the present invention. It is confirmed that it has a higher evaluation score when the recommended learning amount is implemented.

Therefore, in the case of learning with the recommended learning amount provided according to an embodiment of the present invention, the learning effect can be further enhanced, and thus the study efficiency can be greatly increased. In addition, for young students who have difficulty choosing or planning their own learning volume, a higher academic achievement increase effect may appear when they are provided with the recommended learning volume.

The methods according to the present invention may be implemented in the form of program instructions that can be executed by various computer means and recorded on a computer readable medium. Computer readable media may include program instructions, data files, data structures, etc. alone or in combination. Program instructions recorded on a computer readable medium may be specially designed and configured for the present invention or may be known and usable to those skilled in computer software.

Examples of computer readable media may include hardware devices specially configured to store and execute program instructions, such as ROM, RAM, flash memory, and the like. Examples of program instructions may include not only machine language codes generated by a compiler but also high-level language codes that can be executed by a computer using an interpreter and the like. The hardware device described above may be configured to operate with at least one software module to perform the operations of the present invention, and vice versa.

In addition, the above-described method or device may be implemented by combining all or some of its components or functions, or may be implemented separately.

Although the above has been described with reference to preferred embodiments of the present invention, those skilled in the art will variously modify and change the present invention within the scope not departing from the spirit and scope of the present invention described in the claims below. You will understand that it can be done.

10: Machine learning engine
20: learning amount table
100: learning volume recommendation server
101: learning data collection unit
102: Academic Achievement Ability Prediction Department
102a: clustering engine
102b: artificial neural network engine
103: learning amount table generation unit
104: recommended learning amount determining unit
200: student terminal

Claims (1)

A learning amount recommendation device using deep learning-based artificial intelligence consisting of a plurality of hidden layers,
a learning data collection unit that collects a plurality of learning data from student terminals corresponding to a plurality of users;
an academic achievement predictor predicting an academic achievement ability type corresponding to a user of each of the student terminals by using the plurality of learning data;
a learning amount table generation unit generating a learning amount table by pre-processing the learning data according to the predicted academic achievement ability type; and
A recommended learning amount determining unit for determining a recommended learning amount corresponding to a target user by referring to the learning amount table and providing the determined recommended learning amount to a student terminal of the target user;
The academic achievement ability prediction unit,
By clustering users corresponding to each of the learning data according to the current curriculum, a cluster corresponding to each user is determined, and the academic achievement ability type assigned to the determined cluster is the academic achievement ability of the user belonging to the cluster. A clustering engine that predicts by type; and
Supervised-learning is performed using training data composed of training input values corresponding to the learning data of each of the users and training output values labeled with an academic achievement type through the clustering engine, and a plurality of hidden layers. An artificial neural network engine that predicts the type of academic achievement corresponding to the target user based on the configured deep learning network;
The artificial neural network engine,
an input layer that receives a learning feature vector obtained by converting learning data according to the current curriculum of each of the users and is composed of the same number of neurons as the number of component values of the learning feature vector;
a plurality of hidden layers for transmitting an output vector calculated using the output values received from the input layer to an output layer;
an output layer which determines a probability corresponding to the output vector by applying an activation function to the output vector, and outputs an output vector having the highest determined probability; and
A self-attention layer performing a technique according to a self-attention mechanism between the input layer and the plurality of hidden layers,
The self-attention concentration layer calculates a self-attention concentration value α for each of the intermediate calculation vectors provided as output values of the input layer 11, and selects an intermediate calculation vector having the highest calculated self-attention concentration value. Selected and delivered to the hidden layer,
The self-attention value (α) is calculated according to the following equation,

In the above equation, activate is a ReLU function or Softmax function as an activation function that outputs a probability value for an input value, q is a q vector representing each of intermediate operation vectors provided as output values of the input layer, and V is a previously set by a manager. A V vector having fixed component values to be set, K is a K vector derived as a result of a shift dot product operation with a mask vector after padding on the initial K vector,
The initial K vector is a vector having as component values each of the total number of lessons according to the current curriculum, the ratio of the number of passing evaluations according to the number of problem solving in the lesson, and the ratio of the average score according to the average number of passages in the current curriculum. ego,
The hidden layers are
Including a first hidden layer, a second hidden layer, a third hidden layer, and a fourth hidden layer sequentially connected to each other,
The first to fourth hidden layers are linear layers, and at least one of the first to fourth hidden layers is a dense layer,
Among the neurons constituting the first hidden layer, the weights of neurons randomly selected according to a probability determined according to the total number of the hidden layers and the number of neurons each of the hidden layers are set to 0 or a predefined constant, A learning amount recommendation device using deep learning-based artificial intelligence composed of multiple hidden layers.

Images