WO2020080826A1 - 학습 효율을 기반으로 개인 맞춤형 교육 컨텐츠를 제공하기 위한 기계학습 방법, 장치 및 컴퓨터 프로그램 - Google Patents

학습 효율을 기반으로 개인 맞춤형 교육 컨텐츠를 제공하기 위한 기계학습 방법, 장치 및 컴퓨터 프로그램 Download PDF

Info

Publication number
WO2020080826A1
WO2020080826A1 PCT/KR2019/013590 KR2019013590W WO2020080826A1 WO 2020080826 A1 WO2020080826 A1 WO 2020080826A1 KR 2019013590 W KR2019013590 W KR 2019013590W WO 2020080826 A1 WO2020080826 A1 WO 2020080826A1
Authority
WO
WIPO (PCT)
Prior art keywords
user
score
test
users
correct answer
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/KR2019/013590
Other languages
English (en)
French (fr)
Korean (ko)
Inventor
차영민
신동민
이재민
이용구
허재위
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Riiid Inc
Original Assignee
Riiid Inc
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Riiid Inc filed Critical Riiid Inc
Priority to US16/626,835 priority Critical patent/US11704578B2/en
Priority to JP2019570009A priority patent/JP6960688B2/ja
Priority to CN201980003209.4A priority patent/CN111328407A/zh
Priority to BR112021007166-0A priority patent/BR112021007166B1/pt
Publication of WO2020080826A1 publication Critical patent/WO2020080826A1/ko
Anticipated expiration legal-status Critical
Priority to JP2021164287A priority patent/JP7474517B2/ja
Ceased legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N5/00—Computing arrangements using knowledge-based models
    • G06N5/04—Inference or reasoning models
    • G—PHYSICS
    • G09—EDUCATION; CRYPTOGRAPHY; DISPLAY; ADVERTISING; SEALS
    • G09B—EDUCATIONAL OR DEMONSTRATION APPLIANCES; APPLIANCES FOR TEACHING, OR COMMUNICATING WITH, THE BLIND, DEAF OR MUTE; MODELS; PLANETARIA; GLOBES; MAPS; DIAGRAMS
    • G09B7/00—Electrically-operated teaching apparatus or devices working with questions and answers
    • G09B7/06—Electrically-operated teaching apparatus or devices working with questions and answers of the multiple-choice answer-type, i.e. where a given question is provided with a series of answers and a choice has to be made from the answers
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
    • G06Q50/00—Information and communication technology [ICT] specially adapted for implementation of business processes of specific business sectors, e.g. utilities or tourism
    • G06Q50/10—Services
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N20/00—Machine learning
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
    • G06Q50/00—Information and communication technology [ICT] specially adapted for implementation of business processes of specific business sectors, e.g. utilities or tourism
    • G06Q50/10—Services
    • G06Q50/20—Education
    • G06Q50/205—Education administration or guidance
    • G06Q50/2057—Career enhancement or continuing education service
    • G—PHYSICS
    • G09—EDUCATION; CRYPTOGRAPHY; DISPLAY; ADVERTISING; SEALS
    • G09B—EDUCATIONAL OR DEMONSTRATION APPLIANCES; APPLIANCES FOR TEACHING, OR COMMUNICATING WITH, THE BLIND, DEAF OR MUTE; MODELS; PLANETARIA; GLOBES; MAPS; DIAGRAMS
    • G09B5/00—Electrically-operated educational appliances
    • G09B5/04—Electrically-operated educational appliances with audible presentation of the material to be studied

Definitions

  • the present invention relates to a method for providing customized content based on data. More specifically, the present invention relates to a method of estimating learning efficiency for a user's problem by applying the collected problem solving result data to a machine learning framework and recommending educational content based on the problem.
  • Educational content has been generally provided in packages. For example, a collection of problems written on paper contains at least 700 questions per volume, and online or offline lectures are also sold at a time in groups of at least one month to study for at least one month.
  • the present invention aims to solve the above problems. More specifically, the present invention analyzes a user and / or a problem based on a large amount of problem solving result data, and calculates a learning efficiency for each user based on the problem, and recommends learning content according to the learning efficiency How to do it.
  • a method for providing user-specific learning content configures a problem database including at least one multiple-choice question including at least one view for a specific subject, and the user Providing to a device and collecting user's view selection data for the problem from the user device; Estimating a probability of a correct answer to the problem for each of the users using the view selection data of each of the users; Assuming that a user selects each view for a certain problem, the rate of change of the probability of correct answers to the user for all problems included in the problem database is calculated for each problem and included in the problem database It characterized in that it comprises a step of recommending the problem to the user by arranging the problems in the order in which the change rate is high.
  • FIG. 1 is a flow chart for explaining a process for recommending user-customized content according to an embodiment of the present invention
  • Figure 2 is a flow chart for explaining the process of calculating the learning efficiency of each user problem according to an embodiment of the present invention
  • Figure 3 is a flow chart for explaining the process of calculating the learning efficiency of each user problem according to another embodiment of the present invention
  • the concepts of the subject are manually defined by an expert, and the expert individually determines what tags are included in each problem for the subject and tagging them. After that, each user analyzes the skills of the learner based on the result of solving problems tagged for a specific concept.
  • the intention of the issuer may be one or two, but there may be dozens of reasons why the user is wrong with the problem. For example, we can assume that the type of problem is "the past of the family law" and the intention of the presenter is "do you know how to distinguish between the past form of the verb and the family law?"
  • the reason that the user is wrong with the problem may be "because he can't distinguish between the past form of the verb and the family law" in the same way as the questionnaire, but it may also be because he did not know a specific word in the fingerprint.
  • the most efficient problem for the user will be a problem for learning the word.
  • the method of analyzing the user based on the tag information predefined by the expert has a problem in that the tag information depends on the subjectivity of the person, and the number of various cases of human behavior is not comprehensive. There was a problem in that the reliability of the result data could not be high because the subject information of the human being was not involved and the mathematically generated tag information was not given to the problem mathematically.
  • a data analysis server is intended to exclude human intervention in a data processing process by applying a machine learning framework to learning data analysis.
  • the data analysis server collects a result log of a user's problem solving, constructs a multidimensional space composed of a user and a problem, and assigns a value to the multidimensional space based on whether a user is hit or wrong, and each user And modeling the user and / or problem in a way that computes the vector for the problem.
  • the user model may include information about each characteristic of the user for the entire problem
  • the problem model may include information about each characteristic of the problem for the entire user.
  • the problem model may be expressed as an inclusion of concepts constituting the subject of a specific problem
  • the user model may be expressed as an understanding of the concept constituting the subject of a specific user.
  • the problem model and the user model by using the problem model and the user model, it is possible to calculate a probability that a certain user selects a correct answer for a certain problem, that is, a user's correct answer probability for each problem.
  • the user vector and the problem vectors cannot be interpreted by limiting which attributes or features are included.
  • the user vector may include a degree to which the user understands a certain concept, that is, a degree of understanding of the concept.
  • the problem vector may include what concepts the problem is composed of, that is, a conceptual diagram.
  • problems since the learning efficiency for each of the problems included in the problem database for a specific user can be calculated, problems may be recommended to the user in the order of high learning efficiency.
  • the problem to be corrected is excluded, and only the wrong problems are selected and recommended.
  • the user can learn from a problem that is certain to be wrong.
  • the second embodiment of the present invention when a user encounters a certain problem, a problem in which an increase rate of a correct answer probability for the user in the whole problem predicted by a newly trained model including the problem is high, learning efficiency is high.
  • a problem that can solve the entire problem database better to the user will be recommended. According to this, the user will learn from the problem that can increase the probability of correct answer of the whole problem.
  • a problem that can increase a user's actual test score is high in learning efficiency. According to this, an actual test score for the same subject, which is provided from outside the problem database, is estimated for each user, and a problem that can increase the actual test score will be recommended. According to this, the user will learn from questions that can increase the actual test score.
  • FIG. 1 is a flowchart illustrating a process of recommending user-customized content according to an embodiment of the present invention.
  • Steps 110 and 130 are steps of collecting learning data and modeling problems and users in a data analysis system according to an embodiment of the present invention.
  • the data analysis server may generate a problem database and a user database.
  • the data analysis server may collect various problems on the market and create a problem database.
  • the problem includes a listening evaluation problem, and may be in the form of text, image, audio, and / or video.
  • the data analysis server may generate a user database including user identification information.
  • the data analysis server may provide the problem database to a user device, and collect data as a result of the user solving the problem.
  • the data analysis server may organize the collected problem solving result data in the form of a list of users, problems, and results.
  • Y (u, i) means the result of the user u solving the problem i, and a value of 1 for a correct answer or 0 for a wrong answer may be assigned.
  • the data analysis server configures a multi-dimensional space composed of a user and a problem, and assigns a value to the multi-dimensional space based on whether a user is hit or wrong, and vector for each user and problem. Can be calculated. At this time, the features included in the user vector and the problem vector should be interpreted as not specified. (Step 130)
  • the data analysis server if Y is sufficiently large, does not separately define the concept, and can estimate the user's conceptual understanding L and the conceptual configuration R of the problem from Y.
  • each element of L has a value between 0 and 1, and the sum of the elements of each row of R can be 1.
  • the first problem includes about 20% of the second concept, about 50% of the third concept, and the fourth.
  • the concept will be interpreted as including about 30%.
  • the service server assumes the number of unknown concepts constituting the subject as r, and the user's understanding matrix for each concept is n. It can be defined as a by r matrix, and a matrix R for a degree of inclusion by concept of a problem can be defined as a m by r matrix.
  • L is connected to the transpose matrix R T of R, the relationship between the user and the problem can be analyzed without separately defining the concept or the number of concepts.
  • the first row of L is [0, 0, 1, 0.5, 1]
  • the value of the first row of R is [0, 0.2, 0.5, 0.3, 0]
  • the present invention introduces a methodology of Item Response Theory used in psychology, cognitive science, pedagogy, and the like.
  • the item response theory is a test theory for measuring the characteristics of a subject, for example, cognitive ability, physical ability, skill, knowledge, attitude, personality characteristics, or difficulty of the evaluation item, by using a response to the evaluation items.
  • the probability that the first user will hit the first problem is It is calculated as follows and corresponds to 86%. That is, the first user does not understand the second and fourth concepts at all, fully understands the third concept, and in the first problem, the second concept is 20%, the third concept is 50%, and the fourth concept is 30%. This is a configured problem. According to the above formula, if the first user solves the first problem 1, it can be estimated that there is a 86% probability that the answer is correct.
  • the data analysis system may assign an initial value of the conceptual understanding of user i L i and the conceptual configuration of problem j R j as arbitrary values. For example, the data analysis system may assign the initial value of the user i to any concept comprehension L i and the initial value of the arbitrary concept diagram R j of the problem j to any value between 0 and 1.
  • a list in which val is set as the problem solving result data is set to 1 for each user i and 1 for each problem j and 0 for incorrect answers.
  • the result data set for each user can be converted into a list in the form of (i, j, 1) if the user i solves the problem j and (i, j, 0) if the user answers the problem j. .
  • the data analysis system may randomly load the data of the problem solving result list one by one. More specifically, the data analysis system applies the conceptual understanding of the user i at that time, the conceptual understanding L i , and the conceptual configuration diagram R j of the problem j, so that the probability P ( i, j ) of a specific user i will fit the specific problem j
  • the temporary value of tmp can be calculated.
  • the real problem solution for the problem j is to update L and R by using the difference between the result value val (1 for correct answer and 0 for incorrect answer).
  • the data analysis system uses the user's L i (conceptual understanding of user i) and R j (conceptual schematic diagram of problem j) at that time to allow user i to solve problem j. Temporary value of probability Can be calculated.
  • the data analysis system uses the difference between the temporary value tmp of the probability that user i will fit problem j and the difference between val, which is the data that user i actually solved problem j (i, j, We can calculate the approximate gradient grad for val). This can be calculated using Equation 2 below.
  • mu is a parameter that determines the performance of the formula and is selected by experiment according to a conventional technique
  • the step is a parameter for the speed at which the formula accepts new data and is selected by experiment according to a conventional technique.
  • the list is randomly shuffled again, and if the above process is sufficiently repeated, L and R can be reliably estimated.
  • the reason for expressing the user and the problem as a modeling vector is ultimately for accurately predicting whether a specific user will hit or wrong a specific problem.
  • the correct answer rate for the user's problem can be estimated by collecting and analyzing the user's correct answer result, but it can also be estimated using the selection probability of the view.
  • the probability that the first user selects a view for a particular problem is (0.1, 0.2, 0, 0.7)
  • the user will select the example 4 with a high probability
  • the correct answer to the problem is 4, the first user Users can expect to be more likely to encounter the problem.
  • the difficulty of the problem can be estimated by collecting and analyzing the results of correct answers from all users, but it can also be estimated using the selection probability of the view.
  • a user's view selection probability is (0.5, 0.1, 0.3, 0.6) for a specific problem composed of 4 multiple choices
  • the data analysis server can collect the solution data for the entire problem and all users, including the view elements selected by the user.
  • the data analysis server can collect a log of the results of the solution by collecting the view selection results and which view the user has selected from the problem for all problems belonging to the problem database. You can also collect incorrect results.
  • the data analysis server may construct a list of users, problems, and views selected by the user in the problems.
  • the data analysis server may perform a data analysis process by expanding one problem into a view unit.
  • problem i is a quadrilateral problem
  • problem i expands to 4 variables of (i, 1) (i, 2) (i, 3) (i, 4), each user's choice in each problem-view Whether it can be given as a value.
  • E (i, j) means example j of problem i
  • Y '(u, E) means that user u selected example j of problem i, and according to an embodiment of the invention In case 1, if not selected, a value of 0 may be assigned.
  • the data analysis server constructs a multi-dimensional space using a user and a problem-view as variables, and assigns values to the multi-dimensional space based on whether a user selects a corresponding problem-view, so that a vector for each user and the problem-view is obtained. Can be calculated.
  • the data analysis server may estimate the probability, that is, the selectivity, of any user selecting any problem-view using the user vector and the problem-view vector.
  • the selection rate can be estimated by applying various algorithms to the user vector and the problem-view vector, and an algorithm for calculating the selection rate is not limited in interpreting the present invention.
  • the user's problem-view selection rate can be estimated. (x is the problem-view vector, the user vector)
  • the data analysis server may estimate the correct answer rate of the problem using the view selection rate of the user.
  • a specific user's view selection probability is (0.5, 0.1, 0.3, 0.6)
  • the correct answer view is 1, what is the probability that the user will fit the problem? It becomes a problem. That is, a method of estimating the correct answer rate of the corresponding problem may be considered using a plurality of view selection rates for the corresponding problem.
  • a simple method of reducing the view selection rate to the problem correct answer rate may consider a method of comparing the selection rate of the correct answer view to the selection rate of the entire view.
  • the correct answer rate for the corresponding question of the user will be calculated as 0.5 / (0.5 + 0.1 + 0.3 + 0.6).
  • the user does not understand the problem by dividing it into view units, but by understanding the problem in units including the composition of the whole view and the intention of the question.
  • the correct answer rate of the problem from the view selection rate by averaging the overall view selection rate of the problem and applying the averaged selection rate of the correct answer view to the selection rate of the entire view.
  • the scale of each view selection rate can be changed to (0.33, 0.07, 0.20, 0.41). If the correct answer example is 1, the average selection rate of example 1 may be estimated to be 0.33, and the correct answer rate for the corresponding problem of the user may be estimated to be 33%.
  • the service server may estimate a correct answer rate of a problem using a user's problem-view selection probability, and through this, a user's understanding of a specific concept.
  • the data analysis server can calculate the learning efficiency for a specific problem of the corresponding user based on the problem model and the user model. (Step 140)
  • the data analysis server uses the model of the view unit of the problem and the user model to determine the probability of selection of the view unit of the problem and the probability of correct answer of the user's problem unit for each user. Can be calculated.
  • the service server considers that the problem with a high probability of correct answer is already understood by the user, excludes the problem that the user will guess, and selects and recommends only the wrong problems.
  • the service server may create a list of recommendation questions for the user by arranging the remaining problems in the order of the lowest answer rate for a specific user, excluding problems already solved by the user in the problem database set. . (Step 150)
  • the service server may provide a user with a recommendation list of problems, which is given priority in the order of problems 3, 1, 2, 5, and 4. At this time, if user 1 has already solved problem 5, the service server may create a problem recommendation list except for problem 5.
  • the service server may sort the problems included in the problem database in the order in which the user's correct answer probability is low and recommend them to the user. The user will learn from the obvious problem to be wrong.
  • the data analysis server may recommend user-customized content using at least one of a user vector, a problem-view vector, a view selection rate, a correct answer rate of a problem, a user's conceptual understanding, and a conceptual configuration diagram of a problem.
  • the service server may recommend an advanced course for a concept to a user who does not understand a specific concept.
  • the service server may recommend a problem configured to include the multiple concepts to a user having a low correct answer rate for a problem including all of the multiple concepts.
  • the second embodiment of the present invention when a user encounters a certain problem, it can be assumed that a problem with a high rate of change in the probability of correct answer to the user of the whole problem is high in learning efficiency. According to this, a problem that can better solve the entire problem database will be recommended to the user, and the user will learn from a problem that can increase the probability of correct answering of the whole problem.
  • FIG. 2 is a flow chart for explaining a process of calculating the learning efficiency of each user according to an embodiment of the present invention.
  • Step 210 to Step 230 The probability of a user's correct answer by question at any time or the probability of selection by example of a random question will be updated to reflect this when additional data of the corresponding user's solution is collected. Considering this, assuming that the user selects the correct answer for any problem included in the problem database for a specific problem, the probability of the user's correct answer for other problems will change little by little. (Step 210 to Step 230)
  • the service server assumes that the user selects the correct answer for each of the problems included in the entire problem database, calculates an increase in the probability of the correct answer for each of the other problems, and sums or corrects the total change of the correct answer probability
  • the average of the probability change values can be calculated as the learning efficiency of the problem.
  • the corresponding problem when the user selects a corresponding view to a selection probability of a specific user's random question, the corresponding problem is applied when an average value of the correct answer rate change value of another problem is applied. Let's calculate the learning efficiency.
  • the selectivity for the first view of user A's specific problem a is a1
  • the selectivity for the second view is a2
  • the selectivity for the third view is a3
  • the selectivity for the fourth view is a4.
  • the model of User A and the model of all problems will be slightly changed by applying the first view selection event of User A, and each of the problems may be changed according to the changed model.
  • the correct answer rate will change somewhat.
  • the data analysis server may calculate the average value of the correct answer rate change AVC_a1 of all questions by applying a virtual event for the first view selection of the user A.
  • the data analysis server may calculate AVC_a2 by assuming that the user A has selected the second view, AVC_a3 by assuming that the third view is selected, and AVC_a4 by assuming that the fourth view is selected.
  • the data analysis server may calculate the learning efficiency E (A, a) through the problem a of the user A through the following formula.
  • the data analysis server can calculate E (A) for the entire problem and utilize E (A) as the learning efficiency of the problem.
  • the service server can sort the entire problems in the order of high learning efficiency. That is, when the user learns and corrects the corresponding problem, the probability of correct answer of all other problems is changed. If the change value of the correct answer probability is high, the user can better solve the entire problem database, thereby improving learning efficiency. It is highly recommended for users. (Step 250)
  • the third embodiment of the present invention it can be assumed that a problem that can increase a user's actual test score is high in learning efficiency. According to this, an actual test score for the same subject, which is provided from outside the problem database, is estimated for each user, and a problem that can increase the actual test score will be recommended. According to this, the user will learn from questions that can increase the actual test score.
  • FIG. 3 is a flow chart for explaining a process of calculating the learning efficiency for each problem that can increase the actual test score according to the third embodiment of the present invention.
  • the process of estimating the actual test score using the problem model and the user model is essential. How do you estimate actual test scores from outside the problem database?
  • the mock exam is based on whether the actual test and the type of questions are similar, the questions are given regardless of the candidate's ability.
  • the mock exams are only aimed at verifying their position among all students by estimating the test scores, and it is not a problem constructed for the candidate's learning, and the way to increase the learning efficiency will not be even more.
  • the average of the correct answer probability and the actual test score for all questions included in the problem database of the user group and Modeling function representing the relationship of can be calculated for each user. That is, using the actual test score data, a modeling function that converts the average of the probability of correct answers of all questions into a real score can be calculated. (More detailed method of calculating the modeling function will be described later)
  • the user model is calculated with sufficient reliability in step 310, the user model is used to predict the probability of correct answers to problems included in the problem database.
  • the predicted correct answer probability is applied to the modeling function, the correct answer probability can be converted into an external test score.
  • Step 340 the user model and the problem model will be slightly changed by applying the user's view selection event.
  • the correct answer rate is slightly changed, and the actual test score prediction value converted by applying the modeling function will also be changed.
  • the service server assumes that the user selects each view for each of the problems included in the entire problem database, calculates the change value of the actual test score prediction, and the total sum of the change values of the actual test score prediction, or The average of the changed values can be calculated as the learning efficiency of the problem.
  • the estimated score for the user group is the actual A set of mock questions can be constructed from the question database to be similar to test score data. (More detailed methods of constructing the above mock test set will be described later)
  • the simulated test problem set has a feature that is configured based on whether the estimated score is similar to the actual test score, unlike the conventional simulated test, which consists of the same type of questions as the actual test by an expert.
  • an estimated score of the mock test set can be calculated using a user model of a user who does not belong to the user group. (Step 330) Since the simulated test set is configured such that the estimated score is similar to the actual test score, users who have no actual test score data can replace the estimated score for the simulated test set with the predicted value of the actual test score. have.
  • Step 340 the user model and the problem model will be slightly changed by applying the user's view selection event.
  • the correct answer rate will change somewhat, and the estimated score for the set of mock questions will also change.
  • the service server assumes that the user selects each view for each of the problems included in the entire problem database, calculates the change value of the actual test score prediction (ie, the estimated score for the simulated test set), The total sum of the change values of the actual test score prediction or the average of the change values can be calculated as the learning efficiency of the problem.
  • the service server can sort the problem database in the order of the highest increase rate of the actual test score prediction value and recommend it to the user. (Step 360)
  • the data analysis server may identify a user group in which external test score data exists among all users. This is for constructing a set of problems having high external test score prediction efficiency by using the external test score data of the user group and the correct answer probability obtained from each user model and question-view model.
  • the data analysis server is a modeling function that checks the performance of a problem set, that is, whether the problem set has high predictive efficiency of an external test score, or converts the score of the problem set to an external test score.
  • a part of the user group may be divided into test groups.
  • the data analysis server is ⁇ A, B, C, D, E, F, G, H ⁇
  • the data analysis server is ⁇ A, B, C, D , E ⁇ can be classified as a training set, and ⁇ F, G, H ⁇ can be classified as a test set.
  • the data analysis server divides a user group having external test score data into two groups arbitrarily, and one group is trained to provide data analysis framework with data for constructing the problem set and test score transformation modeling function. You can group into sets, and the other group into test sets that provide data to check the performance of the problem set and modeling functions.
  • data of users belonging to the training set are used for constructing a problem set and test score transformation modeling function
  • data of users belonging to the test set are used for the problem set and tests It can be used to check the performance of the score transformation modeling function.
  • the data analysis server may configure a set of problems having high external test score prediction efficiency in the problem database by using the external test score data and the user modeling vector of the user group.
  • the problem set according to the embodiment of the present invention is not configured based on whether it is similar to an external test like a conventional mock test, but is configured based on whether prediction efficiency for an external test score is high, so that the number of problems and / or the type of problems, etc. There are features that do not have to be subject to external testing.
  • the data analysis server calculates the relationship between the probability of the correct answer calculated using the modeling vector of the user group in which the external test score data exists and the external score data of the user group. It can be expressed in a multidimensional plane.
  • the data analysis server may generate 100 as many problems as the x-axis and configure the y-axis as an external test score to generate a multidimensional plane.
  • the data analysis server uses the modeling vector calculated in advance for user A.
  • the probability of a correct answer to one problem can be calculated, and this is set to the x value of the first x-axis (x1), and the external test score of the user A is set to the y value to determine the coordinates of the first plane of the user A.
  • the data analysis server may determine the coordinates of the first planes of the user B to the user E in the same manner and express the relationship with the external test score of the first problem (x1) on the x1 plane in the multidimensional space.
  • the data analysis server forms a plane for the second to the 100th problems in the same way and determines the (x, y) coordinates of the users A to E for each problem plane to the outside of the second to 100th problems.
  • the relationship with test scores can be expressed in a multidimensional space.
  • the estimated correct answer probability and the external test score for the individual problem of the user group will be substantially expressed in a two-dimensional plane, and the data analysis server according to an embodiment of the present invention can determine the relationship between values expressed in the two-dimensional plane. It can be changed to a modeling function expressed mathematically.
  • the data analysis server includes external test scores of users A, B, C, D, and E, and user modeling vectors of users A, B, C, D, E belonging to the training set.
  • the modeling function f1 (x1) which mathematically expresses the relationship between probability of correct answer to the first problem calculated by using, may be calculated.
  • modeling functions f1 (x2) to f1 (x100) may be obtained by processing data for the second to 100th problems in the same manner. At this time, f1 (x1) to f1 (x100) should be determined according to the same criteria.
  • the form of the modeling function calculated to construct the problem set is determined in the same form for all problems belonging to the database.
  • the modeling function f1 (x1) of the first problem is expressed as a one-dimensional function of the form ax + b
  • the modeling functions f1 (x2) to f1 (x100) for the second to 100th problems are also It would be appropriate to be represented in the form of a one-dimensional function.
  • comparison set should be the same because the problem set is composed of problems that belong to the problem database and have high external test score prediction efficiency.
  • modeling function is not limited if the same criteria are applied to the entire problem. That is, the modeling function is sufficient if it is written in an optimized form for representing the data of the training set, and can be calculated according to various algorithms applied in the related art.
  • the data analysis server uses the modeling function of the problem and the deviation of the user's external test scores belonging to the test set to perform external tests A set of problems with high score prediction efficiency can be constructed.
  • the data analysis server calculates the probability of correct answer of the first problem (x1) calculated using the user modeling vectors of users F, G, and H belonging to the test set, and the value applied to the modeling function f (x1) of the first problem.
  • the average of the difference values between the external test scores of the users F, G, and H is calculated, and if the average value (e) is small, it can be determined that the external test score prediction efficiency is high.
  • e may mean the efficiency of estimating the external test score of the problem set. More specifically, it can be interpreted that the smaller the e value, the higher the external test score estimation efficiency of the corresponding problem set.
  • the data analysis server may calculate e2 to e100 according to the same criterion, and the problem with the smallest e may be determined to have the highest external test score prediction efficiency.
  • the data analysis server issues the fourth problem because the fourth problem can be determined to have the highest external test prediction efficiency. You can include it for the first time in a set.
  • the data analysis server may extract a problem in which external test prediction efficiency is high among the first, second, third, and fifth to 100th problems on the premise that it is analyzed together with the fourth problem.
  • the data analysis server fixes a plane for the fourth problem, and forms planes for the remaining first, second, third, and fifth to 100 problems, to the training set.
  • the modeling functions f (x4, x1), f (x4, x2), f (x4, x3), f (x4, x5) to f (x4, x100) that mathematically express the relationship between probability of correct answer to the problem You can calculate once.
  • the second modeling function value may be interpreted as an external test score estimate for the fourth problem and the corresponding problem.
  • the above two-dimensional modeling functions f (x4, x1), f (x4, x2), f (x4, x3), f (x4, x5) to f (x4, x100) are modeling functions with only the fourth problem It should be decided according to the same criteria as when constructing.
  • the modeling function f (x4, x1) of the first problem provided with the fourth problem is in the form of cx4 + dx1 + e, which is based on the criteria of the linear model as ax4 + b of the existing f (x4). As it is, it should be expressed as a 2D function with only the dimension added by adding the variable of x1.
  • Modeling functions f2 (x4, x2), f2 (x4, x3), f2 (x4, x5) to f2 (x4) for the 2nd, 3rd, 5th to 100th problems provided with the 4th problem , x100) is also the same linear model as the existing f (x4), and it would be appropriate to express each problem x as a two-dimensional function added as a variable.
  • the data analysis server is external to the user belonging to the two-dimensional function and test set of the problems. Using a variation in test scores, a problem set with high external test score prediction efficiency can be constructed.
  • the data analysis server uses the user modeling vectors of users F, G, and H belonging to the test set to calculate the probability of the correct answer of the first problem (x1) and the probability of the correct answer of the fourth problem (x4). Calculate the average of the difference between the values applied to f (x4, x1) (which will be interpreted as the external test score estimates of the 4th and 1st questions) and the external test scores of users F, G, and H, and the average value If it is small, it can be judged that the external test score prediction efficiency is high. This can be expressed in the following equation.
  • the data analysis server can calculate e4, 2 to e4, 100 according to the same criteria, and it can be determined that the problem with the smallest e has the highest efficiency of predicting the external test score.
  • the data analysis server can include the first problem in the problem set.
  • the data analysis server may extract the problem of high external test prediction efficiency among the second, third, and fifth to 100 problems according to the same logic as the above-described method on the premise that it is analyzed together with the first and fourth problems. .
  • the completion time of the problem set construction may be a problem.
  • the precision of estimation may increase, but to configure the problem set, modeling functions are calculated for each individual problem, and resource usage required to calculate priority may increase. Therefore, it is efficient to construct the problem set with the number optimized for the estimation of the external test score.
  • the difference between the value applied to the modeling function of the corresponding problem set and the external test score of the users belonging to the test set is calculated by using the modeling vector of users belonging to the test set. Calculate the average of, i.e., and determine whether to end the configuration of the problem set according to the change trend of e according to the update of the problem set.
  • the data analysis server may determine whether to end the update of the problem set by checking the update trend of e of the problem set. More specifically, when ending the problem set update, two things can be considered.
  • the first is when the external test scores of users belonging to the test group can be sufficiently estimated with a set of questions at the time. That is, the external test score estimation can be sufficiently performed only with a specific set of problems. This can be determined by checking whether e of the problem set at the time is below a threshold.
  • the second case is that the efficiency of estimating the external test score is no longer high even if you add another question to the problem set. In other words, there is no effect of adding a problem, even if a problem is added, it is expected that there will be no change in the efficiency of estimating the score of an external test. This can be determined by checking whether the estimated efficiency of the update problem set, i.e., e remains unchanged and remains within an arbitrary range despite the update of the problem set.
  • the data analysis server performs an update of the problem set, checks the external test score estimation efficiency of the problem set, that is, changes in the e, and estimates the efficiency of the problem set at a specific time.
  • the preset threshold is lowered or the estimation efficiency is maintained within an arbitrary range, updating of the problem set may be terminated and the problem set may be configured.
  • the modeling function according to an embodiment of the present invention may be determined according to various algorithms on the assumption that the problems included in the problem set are of the same type. However, among a number of modeling functions generated by applying various algorithms, the problem of selecting the most efficient function for estimating external test scores may remain a problem.
  • the efficiency of the modeling function can be calculated using e, that is, the efficiency of predicting the external test score of the problem set.
  • a first set of modeling functions f1 are extracted according to a first criterion using data of a user belonging to a training set ⁇ A, B, C, D, E ⁇ , and a problem set constructed by applying the first modeling function
  • the ⁇ third problem (x3), fourth problem (x4), fifth problem (x5) ⁇ cases can be considered.
  • the second modeling function f2 is extracted according to the second criterion using data belonging to the same training set ⁇ A, B, C, D, E ⁇ , and the problem set constructed by applying the second modeling function is ⁇ first 1
  • the case of the problem (x1), the third problem (x3), and the fifth problem (x5) ⁇ can be considered.
  • the data analysis server calculates e of each problem set and each modeling function, that is, the external test score prediction efficiency of the problem set using the data of the test set ⁇ F, G, H ⁇ And select a modeling function with a smaller e value.
  • the data analysis server uses the user modeling vectors of users F, G, and H belonging to the test set, which is the first problem set ⁇ third problem (x3), fourth problem (x4), fifth problem ( x5) ⁇ , the average of the difference between the value of applying the correct answer probability of the first modeling function f1 and the external test scores of the users F, G, and H, that is, e1 for the first modeling function can be calculated.
  • the data analysis server uses the user modeling vectors of F, G, and H belonging to the test set to determine the probability of correct answer of the second problem ⁇ first problem (x1), third problem (x3), fifth problem (x5) ⁇ .
  • the average of the difference between the values applied to the second modeling function f2 and the external test scores of the users F, G, and H, that is, e2 for the second modeling function may be calculated.
  • the data analysis server may select a modeling function and a problem set having a smaller e value, and then proceed with a subsequent procedure for estimating the user's external test score.
  • the mock test can be configured in the following way.
  • the first is to construct a set of questions so that the average score of the simulated test for all users falls within a certain range using the average correct answer rate for all users of each database problem.
  • the data analysis server constructs a set of problems so that the average score of the simulated test falls within the range of 67 to 69 can do.
  • a set of questions can be constructed by considering the distribution of the types of questions in the test. For example, referring to the statistics of the language proficiency test, if the actual test is about 20 percent of the first type, 30 percent of the second type, 40 percent of the third type, and 10 percent of the fourth type, the simulated test The problem type distribution can be configured to resemble an actual test.
  • a label for a problem type can be generated in advance to add index information to the problem database.
  • the data analytics server pre-defines labels for problems that can be classified into any type, clusters the problems by learning the characteristics of the problem model that follows the problem type, and labels the problem types in a clustered problem group. Index information can be generated in a manner that gives.
  • the data analysis server does not pre-define the label for the problem type, but clusters the problems using the modeling vector of the problems, interprets the meaning of the clustered problem group, and labels the problem information by labeling the problem type. You can also create the label for the problem type, but clusters the problems using the modeling vector of the problems, interprets the meaning of the clustered problem group, and labels the problem information by labeling the problem type. You can also create the label for the problem type, but clusters the problems using the modeling vector of the problems, interprets the meaning of the clustered problem group, and labels the problem information by labeling the problem type. You can also create
  • the second method of constructing a mock test according to an embodiment of the present invention is to use the actual score information of any users for the test.
  • the correct answer rate of users A, B, C A set of simulated test problems can be constructed such that the estimated scores of the simulated test are 60, 70, and 80 points, respectively.
  • the similarity between the simulated test and the actual test can be mathematically calculated using the score information of the user who took the actual test. Therefore, it is possible to increase the reliability of the mock test, that is, the simulation score is close to the actual test score.
  • a set of simulated test problems may be constructed by applying distribution information of the problem type of the corresponding test, and other information that is statistically analyzed may be applied.
  • the data analysis server can adjust the distribution of problems in the process of constructing a set of simulated exam problems. This is because separate points information is not assigned to the problems belonging to the problem database, but actual tests are assigned different points to each of the problems.
  • a real test is given a high score for difficult problems and a low score for easy problems. If this is interpreted, an actual problem score is given in consideration of the average correct answer rate of the problem, the number of concepts constituting the problem, and the length of the problem fingerprint, and a specific score may be given in advance according to the type of the problem.
  • the data analysis server reflects at least one of the average correct answer rate of the corresponding problem, the number of concepts constituting the problem, the length of the problem fingerprint, and the problem type information, thereby forming a set of mock test problems. You can give them a score.
  • the data analysis server generates a metadata set for a minimum learning element by listing the learning elements and / or topics of the subject in a tree structure to generate a label for the concept of the problem, and the minimum learning element is suitable for analysis It is also possible to generate index information for concepts constituting problems by classifying them into groups.
  • the data analysis server can estimate the predicted score for each simulated test of the user.
  • the score of the simulated test is estimated by the score of the actual test.
  • the user can estimate the score of the mock test with high reliability without having to solve the mock test directly.
  • the mock test according to an embodiment of the present invention is composed of a problem included in the problem database, and the correct answer rate of the user for each problem belonging to the database is calculated in advance as described above. Therefore, it is possible to estimate the predicted score of the user's mock test using the correct answer rate of each user for all the questions that make up the mock test.
  • a plurality of simulated test sets for estimating random test scores may be configured, and the estimated scores for a plurality of simulated test scores of a specific user may be averaged to estimate the predicted score for the actual test of the user. have.

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Business, Economics & Management (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Educational Technology (AREA)
  • Educational Administration (AREA)
  • Software Systems (AREA)
  • Tourism & Hospitality (AREA)
  • General Engineering & Computer Science (AREA)
  • Mathematical Physics (AREA)
  • Computing Systems (AREA)
  • Evolutionary Computation (AREA)
  • Data Mining & Analysis (AREA)
  • Artificial Intelligence (AREA)
  • Human Resources & Organizations (AREA)
  • Strategic Management (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Medical Informatics (AREA)
  • General Health & Medical Sciences (AREA)
  • Primary Health Care (AREA)
  • Marketing (AREA)
  • General Business, Economics & Management (AREA)
  • Economics (AREA)
  • Health & Medical Sciences (AREA)
  • Computational Linguistics (AREA)
  • Electrically Operated Instructional Devices (AREA)
  • Management, Administration, Business Operations System, And Electronic Commerce (AREA)
  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
PCT/KR2019/013590 2018-10-16 2019-10-16 학습 효율을 기반으로 개인 맞춤형 교육 컨텐츠를 제공하기 위한 기계학습 방법, 장치 및 컴퓨터 프로그램 Ceased WO2020080826A1 (ko)

Priority Applications (5)

Application Number Priority Date Filing Date Title
US16/626,835 US11704578B2 (en) 2018-10-16 2019-10-16 Machine learning method, apparatus, and computer program for providing personalized educational content based on learning efficiency
JP2019570009A JP6960688B2 (ja) 2018-10-16 2019-10-16 学習効率に基づいて個人カスタマイズ型教育コンテンツを提供するための機械学習方法、装置及びコンピュータプログラム
CN201980003209.4A CN111328407A (zh) 2018-10-16 2019-10-16 用于基于学习效率提供私人定制教育内容的机械学习方法、装置及计算机程序
BR112021007166-0A BR112021007166B1 (pt) 2018-10-16 2019-10-16 Método de análise de um usuário em um servidor de serviço
JP2021164287A JP7474517B2 (ja) 2018-10-16 2021-10-05 学習効率に基づいて個人カスタマイズ型教育コンテンツを提供するための機械学習方法、装置及びコンピュータプログラム

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
KR1020180123240A KR102015075B1 (ko) 2018-10-16 2018-10-16 학습 효율을 기반으로 개인 맞춤형 교육 컨텐츠를 제공하기 위한 기계학습 방법, 장치 및 컴퓨터 프로그램
KR10-2018-0123240 2018-10-16

Publications (1)

Publication Number Publication Date
WO2020080826A1 true WO2020080826A1 (ko) 2020-04-23

Family

ID=67807818

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/KR2019/013590 Ceased WO2020080826A1 (ko) 2018-10-16 2019-10-16 학습 효율을 기반으로 개인 맞춤형 교육 컨텐츠를 제공하기 위한 기계학습 방법, 장치 및 컴퓨터 프로그램

Country Status (5)

Country Link
US (1) US11704578B2 (https=)
JP (2) JP6960688B2 (https=)
KR (1) KR102015075B1 (https=)
CN (1) CN111328407A (https=)
WO (1) WO2020080826A1 (https=)

Families Citing this family (22)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
KR102015075B1 (ko) * 2018-10-16 2019-08-27 (주)뤼이드 학습 효율을 기반으로 개인 맞춤형 교육 컨텐츠를 제공하기 위한 기계학습 방법, 장치 및 컴퓨터 프로그램
KR20210125250A (ko) 2020-04-08 2021-10-18 주식회사 셀바스에이아이 문항 평가 방법 및 이를 이용한 디바이스
US12100315B2 (en) * 2020-06-18 2024-09-24 The Hong Kong University Of Science And Technology Peer-inspired student performance prediction in interactive online question pools with graph neural network
CN111914176B (zh) * 2020-08-07 2023-10-27 腾讯科技（深圳）有限公司 题目的推荐方法及装置
KR102283711B1 (ko) * 2020-09-16 2021-07-30 (주)뤼이드 사용자의 학습효과를 반영하여 추천문제를 결정하는 학습 컨텐츠 추천 장치, 시스템 및 그것의 동작 방법
KR102645628B1 (ko) * 2020-11-13 2024-03-11 주식회사 포티투마루 선다형 질문에 대한 정답 선택 방법 및 장치
KR102385073B1 (ko) * 2020-11-13 2022-04-11 (주)뤼이드 점수 확률분포 형태 통일화를 통해 평가 가능한 문제를 추천하는 학습 문제 추천 시스템 및 이것의 동작방법
KR102316980B1 (ko) * 2020-11-20 2021-10-27 태그하이브 주식회사 사용자 맞춤형 문제 제공 장치 및 이의 실행 방법
KR102406414B1 (ko) * 2021-01-04 2022-06-08 (주)뤼이드 잠재인자에 기반한 협업 필터링을 사용하여 사용자의 정답확률을 예측하는 학습 컨텐츠 추천 시스템 및 그것의 동작방법
KR102406458B1 (ko) * 2021-04-01 2022-06-08 (주)뤼이드 복수의 시험 도메인에 적용되는 전이요소로 학습된 인공지능 모델을 통해 사용자의 실력을 평가하는 장치, 시스템 및 그것의 동작방법
KR20220164186A (ko) * 2021-06-04 2022-12-13 주식회사 천재교과서 인공지능 취약 진단 학습 추천 서비스 시스템 및 방법
KR102412378B1 (ko) * 2021-07-01 2022-06-23 (주)뤼이드 사용자에게 교육 효과가 최대화되는 웹페이지를 추천하는 방법, 장치, 및 시스템
KR102396078B1 (ko) * 2021-10-26 2022-05-10 (주)뤼이드 사용자 평가를 위한 진단고사 추천방법
JP7803124B2 (ja) * 2021-12-27 2026-01-21 富士フイルムビジネスイノベーション株式会社 情報処理装置、及び情報処理プログラム
KR102465634B1 (ko) * 2022-07-20 2022-11-11 주식회사 튜링 사용자들의 문제 풀이 실력 향상을 위한 맞춤형 문제를 제공하는 기법
KR102518474B1 (ko) * 2022-07-21 2023-04-05 주식회사 튜링 사용자의 실력을 업데이트하는 기법
JP7777555B2 (ja) * 2023-03-06 2025-11-28 株式会社ベネッセコーポレーション 学習支援システム
WO2024247017A1 (ja) * 2023-05-26 2024-12-05 日本電信電話株式会社 教材提供装置、教材提供方法、及びプログラム
KR102942120B1 (ko) * 2023-09-11 2026-03-20 주식회사 일릭서 학습결과의 전 과정을 자동 진단 및 평가하는 빅데이터 분석 알고리즘이 적용된 2시그마 학습 성취도 평가 시스템
KR102627731B1 (ko) * 2023-09-18 2024-01-24 이동근 학생의 학업성취를 위한 맞춤형 학습방향 설정 서비스 제공 장치 및 방법
KR20250104337A (ko) 2023-12-29 2025-07-08 이주열 사용자의 경험 데이터 수집에 기반하는 교육 정보 제공 시스템
KR20250121734A (ko) 2024-02-05 2025-08-12 이주열 수강생들의 강의 노트 거래를 위한 매매 플랫폼 시스템

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
KR20100022427A (ko) * 2009-04-20 2010-03-02 강남희 이동 기기를 이용하는 교육 컨텐트 기반의 테스트, 랭킹, 평가, 분석 및 ｂ2ｃ/ｂ2ｂ연계를 위한 시스템 및 방법
KR101285217B1 (ko) * 2011-10-06 2013-07-11 주식회사 퀀트랩 큐벡터를 이용한 문항출제 시스템 및 방법
KR20170034106A (ko) * 2015-09-18 2017-03-28 아주대학교산학협력단 사용자 적정 난이도 문제 추천 장치 및 그 방법
KR101853091B1 (ko) * 2017-05-19 2018-04-27 (주)뤼이드 기계학습이 적용된 사용자 답변 예측 프레임워크를 통한 개인 맞춤형 교육 컨텐츠 제공 방법, 장치 및 컴퓨터 프로그램
KR101895963B1 (ko) * 2017-10-31 2018-10-04 (주)뤼이드 신규 사용자를 분석하는 방법
KR102015075B1 (ko) * 2018-10-16 2019-08-27 (주)뤼이드 학습 효율을 기반으로 개인 맞춤형 교육 컨텐츠를 제공하기 위한 기계학습 방법, 장치 및 컴퓨터 프로그램

Family Cites Families (25)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US6921268B2 (en) * 2002-04-03 2005-07-26 Knowledge Factor, Inc. Method and system for knowledge assessment and learning incorporating feedbacks
KR100747141B1 (ko) * 2002-09-25 2007-08-07 가부시키가이샤 베네세 코포레이션 테스트시스템, 테스트시스템 제어방법 및 그 제어방법을 실행하는 컴퓨터 프로그램을 기억하는 기억매체
JP4447411B2 (ja) * 2004-09-03 2010-04-07 株式会社エヌ・ティ・ティ・データ 学習者習得特性分析システム及びその方法並びにプログラム
US20070269788A1 (en) * 2006-05-04 2007-11-22 James Flowers E learning platform for preparation for standardized achievement tests
JP4923246B2 (ja) * 2006-07-21 2012-04-25 国立大学法人山口大学 適応型テストシステムとその方法
JP5481295B2 (ja) * 2010-07-15 2014-04-23 エヌ・ティ・ティ・コムウェア株式会社 オブジェクト推薦装置、オブジェクト推薦方法、オブジェクト推薦プログラムおよびオブジェクト推薦システム
JP2013178707A (ja) * 2012-02-29 2013-09-09 Kddi Corp レコメンド装置、レコメンド方法およびプログラム
JP6015065B2 (ja) * 2012-03-29 2016-10-26 富士通株式会社 学習支援サーバ、学習支援方法、学習支援プログラム及び学習支援システム
US20140272914A1 (en) * 2013-03-15 2014-09-18 William Marsh Rice University Sparse Factor Analysis for Learning Analytics and Content Analytics
KR101285127B1 (ko) 2013-06-07 2013-07-19 양경명 차량의 적재물 감시 장치
US10558924B2 (en) * 2014-05-23 2020-02-11 DataRobot, Inc. Systems for second-order predictive data analytics, and related methods and apparatus
US10366332B2 (en) * 2014-08-14 2019-07-30 International Business Machines Corporation Tailoring question answering system output based on user expertise
US10691698B2 (en) * 2014-11-06 2020-06-23 International Business Machines Corporation Automatic near-real-time prediction, classification, and notification of events in natural language systems
US9818306B2 (en) * 2014-12-17 2017-11-14 Afficient Academy, Inc. System and method for assessing learning or training progress
US9715621B2 (en) * 2014-12-22 2017-07-25 Mcafee, Inc. Systems and methods for real-time user verification in online education
US10885803B2 (en) * 2015-01-23 2021-01-05 Massachusetts Institute Of Technology System and method for real-time analysis and guidance of learning
US10475043B2 (en) * 2015-01-28 2019-11-12 Intuit Inc. Method and system for pro-active detection and correction of low quality questions in a question and answer based customer support system
KR20160117097A (ko) 2015-03-31 2016-10-10 이소열 추정필터를 이용한 교육 성과 예측 방법 및 시스템
CN104835087B (zh) * 2015-04-30 2019-04-05 泸州市金点教育科技有限公司 教育测试系统的数据处理方法及装置
JP2017068189A (ja) * 2015-10-02 2017-04-06 アノネ株式会社 学習支援装置、学習支援方法、学習支援装置用プログラム
JP2017207691A (ja) * 2016-05-20 2017-11-24 阪本システムズ株式会社 択一問題の正答率予測方法
CN106682768B (zh) * 2016-12-08 2018-05-08 北京粉笔蓝天科技有限公司 一种答题分数的预测方法、系统、终端及服务器
US20200193317A1 (en) * 2017-05-19 2020-06-18 Riiid Inc. Method, device and computer program for estimating test score
CN108345697A (zh) * 2018-03-22 2018-07-31 山东财经大学 面向大学生群体的智慧课程推荐方法、系统及存储介质
KR102104660B1 (ko) * 2018-04-23 2020-04-24 주식회사 에스티유니타스 사용자 맞춤형 학습 컨텐츠 제공 시스템 및 방법

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
KR20100022427A (ko) * 2009-04-20 2010-03-02 강남희 이동 기기를 이용하는 교육 컨텐트 기반의 테스트, 랭킹, 평가, 분석 및 ｂ2ｃ/ｂ2ｂ연계를 위한 시스템 및 방법
KR101285217B1 (ko) * 2011-10-06 2013-07-11 주식회사 퀀트랩 큐벡터를 이용한 문항출제 시스템 및 방법
KR20170034106A (ko) * 2015-09-18 2017-03-28 아주대학교산학협력단 사용자 적정 난이도 문제 추천 장치 및 그 방법
KR101853091B1 (ko) * 2017-05-19 2018-04-27 (주)뤼이드 기계학습이 적용된 사용자 답변 예측 프레임워크를 통한 개인 맞춤형 교육 컨텐츠 제공 방법, 장치 및 컴퓨터 프로그램
KR101895963B1 (ko) * 2017-10-31 2018-10-04 (주)뤼이드 신규 사용자를 분석하는 방법
KR102015075B1 (ko) * 2018-10-16 2019-08-27 (주)뤼이드 학습 효율을 기반으로 개인 맞춤형 교육 컨텐츠를 제공하기 위한 기계학습 방법, 장치 및 컴퓨터 프로그램

Also Published As

Publication number Publication date
BR112021007166A2 (pt) 2021-07-20
JP2022008867A (ja) 2022-01-14
JP6960688B2 (ja) 2021-11-05
US11704578B2 (en) 2023-07-18
JP7474517B2 (ja) 2024-04-25
JP2021508065A (ja) 2021-02-25
KR102015075B1 (ko) 2019-08-27
CN111328407A (zh) 2020-06-23
US20210133598A1 (en) 2021-05-06

Similar Documents

Publication Publication Date Title
WO2020080826A1 (ko) 학습 효율을 기반으로 개인 맞춤형 교육 컨텐츠를 제공하기 위한 기계학습 방법, 장치 및 컴퓨터 프로그램
WO2020204468A1 (ko) 액티브 러닝 기법을 적용한 머신 러닝 프레임워크 운용 방법, 장치 및 컴퓨터 프로그램
WO2022045425A1 (ko) 역강화학습 기반 배달 수단 탐지 장치 및 방법
WO2022211326A1 (ko) 복수의 시험 도메인에 적용되는 전이요소로 학습된 인공지능 모델을 통해 사용자의 실력을 평가하는 장치, 시스템 및 그것의 동작방법
Vajjala et al. What do we Really Know about State of the Art NER?
WO2022235073A1 (ko) 독해와 작문 능력의 향상을 가이드하는 방법 및 그 장치
WO2021182921A1 (ko) 신경망을 이용한 기술 유사도 판단 방법
WO2021010744A1 (ko) 음성 인식 기반의 세일즈 대화 분석 방법 및 장치
WO2022191513A1 (ko) 데이터 증강 기반 지식 추적 모델 학습 장치, 시스템 및 그것의 동작방법
JP6913706B2 (ja) 試験問題予測システム及び試験問題予測方法
EP3523932A1 (en) Method and apparatus for filtering a plurality of messages
WO2022265127A1 (ko) 인공 지능 학습 기반의 사용자 이탈율 예측 및 사용자 지식 추적 시스템 및 그것의 동작 방법
WO2024162581A1 (ko) 개선된 적대적 어텐션 네트워크 시스템 및 이를 이용한 이미지 생성 방법
WO2022191368A1 (ko) 자연어 의도를 분류하는 뉴럴 네트워크 훈련을 위한 데이터 처리 방법 및 장치
WO2022149758A1 (ko) 풀이 경험이 없는 추가된 문제 컨텐츠에 대한 예측된 정답 확률을 기초로, 문제를 평가하는 학습 컨텐츠 평가 장치, 시스템 및 그것의 동작 방법
KR101836206B1 (ko) 개인 맞춤형 교육 컨텐츠를 제공하는 방법, 장치 및 컴퓨터 프로그램
WO2023182713A1 (ko) 인공지능 기반으로 텍스트 및 비-텍스트 이미지를 포함하는 화면 정보를 인지하여 화면 상의 오브젝트에 이벤트를 발생시키는 방법 및 시스템
Hasibuan et al. Detecting learning style based on level of knowledge
WO2016190636A1 (ko) 라이브웨어의 상태변환 예측 및 상태 개선 방법, 그리고 상기 방법을 구현하는 장치
WO2026049166A1 (ko) 브랜드 아이덴티티를 반영한 기업 영상 생성 방법 및 장치
KR20180127188A (ko) 개인 맞춤형 교육 컨텐츠를 제공하기 위한 기계학습 프레임워크 운용 방법, 장치 및 컴퓨터 프로그램
WO2016195196A1 (ko) 평가 대상에 대한 평가 결과를 정량화하는 방법
WO2020122339A1 (ko) 전자 장치 및 그의 제어 방법
WO2022158696A1 (ko) 복수의 역량에 대한 역량평가를 수행하는 기계학습모델을 학습시키는 방법, 장치, 및 컴퓨터-판독가능매체
WO2014025173A1 (ko) 전자적 서적의 통합 제공방법

Legal Events

Date Code Title Description
ENP Entry into the national phase

Ref document number: 2019570009

Country of ref document: JP

Kind code of ref document: A

121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 19874706

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

REG Reference to national code

Ref country code: BR

Ref legal event code: B01A

Ref document number: 112021007166

Country of ref document: BR

ENP Entry into the national phase

Ref document number: 112021007166

Country of ref document: BR

Kind code of ref document: A2

Effective date: 20210415

122 Ep: pct application non-entry in european phase

Ref document number: 19874706

Country of ref document: EP

Kind code of ref document: A1