KR20210071681A - How to automatically classify the unit and difficulty of math problems - Google Patents

Abstract

When a user photographs a mathematical problem in a knowledge sharing server platform connected to a network environment or a wireless environment with a smart device to register the mathematical problem in a platform server, a formula is converted into text by a Mathpix API in the optical character recognition (OCR) technology, and the language except for the formula is converted into text by using a Google API. The reason for this is because Mathpix has very high accuracy whereas Google or Naver OCR has low accuracy in formula recognition. Artificial intelligence is trained based on data obtained by the OCT technology to match a previously constructed unit and difficulty classification database to automatically classify the unit and difficulty of the corresponding problem to automatically store the corresponding classification and difficulty in the previously constructed database.

Description

How to automatically classify the unit and difficulty of math problems

In the present invention, when a user takes a picture of a mathematical problem and registers it in a knowledge sharing server platform, using optical character recognition (OCR) (1) technology and artificial intelligence technology, the unit and difficulty of the corresponding question are automatically classified and stored in the database. it's about how to

The future talents required in the 21st century are critical thinking, communication, collaboration, and creativity, and the improvement of these competencies is achieved through the active participation of students. Education methods delivered vertically by teachers are no longer competitive. Therefore, these days, the reverse class (flip learning) in which students take the initiative is spreading rapidly. In the existing teaching method, the teacher conducts a concept class first, and after solving the examples, the students review it, but in flip learning, the students first learn by watching the lecture video made by the teacher at home, and then discuss with the students in the classroom. Collaborative learning to present. This teaching method is preferred by students because it induces students' voluntary participation and increases the efficiency of learning.

However, such learning cannot always proceed due to time and space constraints, and it is difficult for students to solve the problems themselves in difficult learning.

In addition, the video lectures provided by online educational institutions are one-way, lack the level of diversity, and the learning efficiency is low because the time is relatively long.

In addition, private hagwon classes are limited in individualized learning because several students conduct classes together, and time and space are limited and cost is high.

Although private tutoring has the great advantage of 1:1 personalized learning, it is expensive and lacks a systematic online learning system, so online learning linked to offline learning and analysis of learning results is insufficient, and there are limitations in time and space.

Now, with the development of information and communication technology, almost all students own smartphones and access the Internet across time and space to play various SNS and games to obtain necessary information immediately. In addition, the Ministry of Education has decided to introduce digital textbooks from 2018, which means a drastic change in the educational method, and more and more various smart devices (phones, tablets, laptops, etc.)

In terms of software, it is possible to support efficient learning of students by using artificial intelligence and big data using machine learning (17) and deep learning (5) technologies. In other words, it not only supports feedback learning by analyzing all the data students have learned online, but also supports feedback learning by analyzing individual weaknesses. can provide

However, using this ICT environment as much as possible, anyone can easily create and share content for subjects, and users can take pictures of math problems on the knowledge sharing server platform so that they can economically solve the questions or needs of students across time and space. After registration, using optical character recognition (OCR) (1) technology and artificial intelligence technology, the unit and difficulty of the question are automatically classified and stored in the database, and efficient feedback learning is possible using the learning data stored in this method. A learning system and a method for performing this have not been developed in the prior art.

An object of the present invention is to satisfy the technical needs required from the background of the above invention.

Specifically, it is an object of the present invention to classify the units and difficulty levels of the problems for each problem by a math expert and store them in a database, which takes a lot of time and money and is inefficient. It not only saves time and money by classifying them into categories, but also allows anyone to easily know the unit and difficulty of a math problem, so that math problems can be produced and shared.

Another object of the present invention is to automatically store information on the unit and difficulty of the mathematical problem taken as above in the database to enable feedback learning such as weak unit analysis, learning plan, similar problem, and deep problem.

Until now, in order to store a mathematical problem in a database, an expert had to designate a unit and a difficulty level for each problem, but with the present invention, the unit and difficulty are automatically classified and stored in the database, so time and cost can be dramatically reduced.

There are four major techniques according to the present invention for achieving this object.

A. Optical Character Recognition (OCR) (1) that converts characters, symbols, and formulas of mathematical problems in photos into text

B. An artificial intelligence system that automatically classifies units based on data obtained by OCR(1)

C. An artificial intelligence system that automatically classifies difficulty based on data obtained by OCR(1)

D. A system that automatically stores questions including information on units and difficulty levels obtained through the above three steps in a database

As described above, the present invention automatically classifies the units and difficulty levels of the mathematical problems taken by the user on the knowledge sharing server platform with artificial intelligence and automatically stores them in the database, so that the experts classify and store units and difficulties for each problem. Not only can it save time and cost, but also feedback learning such as weak unit analysis, learning plan, similar and deep problems using the information provided by the database classified by unit and difficulty to all users of the knowledge sharing platform With the support of the system, users can utilize many math problems stored in the platform to provide users with optimal, personalized learning anytime, anywhere, and analyze the learner's learning pattern, problem type, and problem content in detail through artificial intelligence. Thus, it has the effect of providing predicted content to each user.

And the present invention has the effect of minimizing private education expenses and returning time to students through efficient individualized learning anytime, anywhere from a social point of view so that they can further their dreams of the future.

The technical effects of the present invention are not limited to the technical effects mentioned above, and other technical effects not mentioned can be clearly understood by those of ordinary skill in the art to which the present invention belongs from the description of the claims. There will be.

1 is a representative diagram of a system for delivering knowledge content desired by a user in a knowledge sharing server platform according to an embodiment of the present invention.
When students take a picture of a math problem they do not know with a smart device and register it on the platform server, the characters and formulas in the picture are converted into text by the ocr system, and an artificial intelligence system that classifies the unit and difficulty of the question based on the data And it is a representative diagram of the overall platform service, such as content matching with mentors optimized for student needs, and feedback learning after learning.
[Fig. 2] relates to an embodiment of a bounding box.
In order to increase the recognition rate of the formula part and the natural language part using Mathpix OCR, it is an embodiment in which a test is carried out by processing a bounding box on the formula part.
[Figure 3] relates to the mathematical unit and difficulty classification of the database.
The units and difficulties of mathematics were classified by experts, and the units were classified into 4 levels as “Grade-Great Unit-Middle Unit-Middle Unit”, and the difficulty of each item was classified into 4 stages as “Best-High-Middle-Low”. A database was built according to the classified units and difficulty classification, and when the unit and difficulty classification of the registered questions were automatically linked and saved by taking pictures with artificial intelligence.

Prior to the detailed description of each configuration step of the present invention, the terms or words used in the present specification and claims should not be construed as being limited to their ordinary or dictionary meanings, and the inventors should develop their own inventions in the best way. Based on the principle that the concept of a term can be appropriately defined for explanation, it should be interpreted as a meaning and concept consistent with the technical idea of the present invention. Accordingly, the configuration shown in the drawings described in the present specification is only the most preferred embodiment of the present invention and does not represent all the technical spirit of the present invention, and thus various equivalents and It should be understood that there may be variations. In order to complete the present invention, the following four steps are largely required.

Step 1: How to convert a math problem taken as a picture into text using OCR(1)

* For math problems in English, both letters and formulas are recognized with Mathpix OCR(2).

*For math problems in languages other than English, the letter part is recognized by Google OCR(3), the formula part is displayed first with the formula area model, and the problem with the formula area is expressed through Mathpix OCR(2) recognize the part. (For math problems in languages other than English, Mathpix OCR(2) recognizes the formula only when the formula area is marked with a bounding box(4).)

The formula area model is a deep learning (5) model that receives a mathematical problem and displays the formula part as a bounding box (4). To make this model, data with the formula area displayed as a bounding box (4) is required, and data collection It is then made by learning with deep learning (5)

* Recognized characters are saved as text, and recognized formulas are saved as text in latex format(6).

Step 2: A method of automatically classifying the units of a question by using the data extracted by OCR(1)

* After collecting data composed of a set of units of the problem converted into text, it is learned by deep learning (5) to make a unit classification model (7).

* When the unit classification model (7) is created, the mathematical problem picture taken as a picture is converted into text using OCR(1). After recognizing/converting the mathematical problem picture into text, the converted text is As described in [Definition and composition of the classification model (7)] If you put it as the input of the unit classification model, the unit of the problem is automatically entered as the output of the model.

comes out

[Definition and composition of the unit classification model (7)]

The unit classification model 7 is a deep learning (5)-based model whose purpose is to receive a text-converted problem (including natural language and formulas) as input and predict the unit to which the problem belongs.

Here, unit refers to a classification system of concepts commonly used in all countries including the subject in the curriculum. This classification system is hierarchically structured. That is, a specific unit in the classification system includes several other units. Accordingly, the unit classification model 7 performs hierarchical classification.

The unit classification model 7 is mainly composed of two components.

The first element is the Feature Extractor(9), which extracts the most suitable features(8) for judging a unit from the input data of the model (problem descriptions and formulas in text form). This feature extractor (9) is composed of a deep neural network (10), and the gradient (11) is back-propagated to reduce the error in unit classification, and its parameters are updated.

The second element is a classification network (12) that determines which unit the problem belongs to based on the extracted feature (8). This classification network (12) is trained by gradient (11) to reduce the error by generating a large error when the unit of each problem given to the training data does not fit properly. The above two elements are learned in the end-to-end method (13).

After learning, when a new problem that did not exist in the training data is given as an input, the unit classification model extracts a feature (8) from the problem and predicts and returns the unit of the given problem based on this.

Step 3: A method of automatically classifying the difficulty level of a question with artificial intelligence using the data extracted by OCR(1)

* After collecting the data composed of the problem converted into text and the set of difficulty of the problem, it is learned by deep learning (5) to make the difficulty classification model (14).

* When the difficulty classification model 14 is created, after recognizing/converting the mathematical problem photo into text through step 1, if the converted text is input to the difficulty classification model 14, the difficulty of the problem is displayed as the output of the model. comes out automatically

[Definition and composition of the difficulty classification model (14)]

The difficulty classification model 14 is a deep learning (5)-based model that aims to predict the difficulty of a problem by receiving a text-converted problem (including natural language and formulas) as an input.

Here, the difficulty may be defined by the background knowledge required to solve the problem. In general, problems with ‘low difficulty’ can be solved with a small amount of background knowledge, and problems with ‘high difficulty’ can be solved only by comprehensively using background knowledge from various fields.

Difficulty is fundamentally a relatively defined concept. Certain problems are “more” or “less” difficult to solve than others. To reflect these characteristics, the difficulty classification model 14 is trained to learn the relative difficulty level (pair-wise ranking) between problems.

The difficulty classification model 14 is largely composed of two components.

The first element is the Feature Extractor (9), which extracts the most suitable features (8) for judging the difficulty from the input data of the model (problem descriptions and formulas in text form). This feature extractor 9 is composed of a deep neural network 10, and the gradient 11 is back-propagated to reduce the error of difficulty prediction, and the parameter 15 is updated.

The second element is a Decision Network (16) that judges the relative difficulty level between problems based on the extracted Feature (8). This decision network (16) is trained by gradient (11) to reduce the error by generating a large error when the difficulty level of the problems given to the training data is not properly matched. The above two elements are learned in the end-to-end method (13).

After learning, when a new problem that did not exist in the training data is given as an input, the difficulty classification model extracts a feature (8) from the problem and predicts and returns the difficulty of the given problem based on this.

Step 4: Saving the unit and difficulty information automatically classified by artificial intelligence in the database

Way

* After converting the math problem photo file taken with the camera or captured on the screen into text using OCR(1) to convert the math problem taken as a photo into text, the data extracted with OCR(1) is converted to text. Using the two-step process of automatically classifying the units of the item with artificial intelligence, and using the data extracted with OCR(1), the difficulty of the item is determined by artificial intelligence.

Through the three-step process of automatically classifying by skill, the unit and difficulty of the problem are extracted.

The data for one math problem configured in this way, that is, the picture of the math problem, the converted text, the unit of the math problem, and the difficulty of the math problem are stored as one record in the database.

Although described above with reference to the drawings of the present invention, those skilled in the art to which the present invention pertains will be able to make various applications, modifications and adaptations within the scope of the present invention based on the above contents. Accordingly, the true protection scope of the present invention should be defined only by the appended claims.

Claims (2)

It is a method of automatically classifying the units of a question with artificial intelligence using the data extracted by OCR(1).
* After collecting data composed of a set of units of the problem converted into text, it is learned by deep learning (5) to make a unit classification model (7) *When the unit classification model (7) is created, the mathematical problem picture is recognized/converted into text through the process of converting the mathematical problem taken as a picture into text using OCR(1), and then the converted text is As described in [Definition and composition of the classification model (7)] If you put it as the input of the unit classification model, the unit of the problem is automatically displayed as the output of the model.
The unit classification model 7 is a deep learning (5)-based model whose purpose is to receive a text-converted problem (including natural language and formulas) as input and predict the unit to which the problem belongs.
Here, unit refers to a classification system of concepts commonly used in all countries including the subject in the curriculum. This classification system is hierarchically structured. That is, a specific unit in the classification system includes several other units. Accordingly, the unit classification model 7 performs hierarchical classification.
The unit classification model 7 is mainly composed of two components.
The first element is the Feature Extractor(9), which extracts the most suitable features(8) for judging a unit from the input data of the model (problem descriptions and formulas in text form). This feature extractor (9) is composed of a deep neural network (10), and the gradient (11) is back-propagated to reduce the error in unit classification, and its parameters are updated.
The second element is a classification network (12) that determines which unit the problem belongs to based on the extracted feature (8). This classification network (12) is trained by gradient (11) to reduce the error by generating a large error when the unit of each problem given to the training data does not fit properly. The above two elements are learned in the end-to-end method (13).
After learning, when a new problem that did not exist in the training data is given as an input, the unit classification model extracts a feature (8) from the problem and predicts and returns the unit of the given problem based on this. It is a method of automatically classifying the difficulty of questions by artificial intelligence using the data extracted by OCR(1). After collecting data consisting of a set of problems and problems converted into text, it is learned by deep learning (5). A difficulty classification model (14) is created.
* When the difficulty classification model 14 is created, after recognizing/converting the mathematical problem photo into text through step 1, if the converted text is input to the difficulty classification model 14, the difficulty of the problem is displayed as the output of the model. comes out automatically
[Definition and composition of the difficulty classification model (14)]
The difficulty classification model 14 is a deep learning (5)-based model that aims to predict the difficulty of a problem by receiving a text-converted problem (including natural language and formulas) as an input.
Here, the difficulty may be defined by the background knowledge required to solve the problem. In general, problems with 'low difficulty' can be solved with a small amount of background knowledge, and problems with 'high difficulty' can be solved only by comprehensively using background knowledge from various fields.
Difficulty is fundamentally a relatively defined concept. Certain problems are 'more' or 'less' difficult to solve than others. To reflect these characteristics, the difficulty classification model 14 is trained to learn the relative difficulty level (pair-wise ranking) between problems.
The difficulty classification model 14 is largely composed of two components.
The first element is the Feature Extractor (9) that extracts the most suitable features (8) for judging the difficulty from the input data of the model (problem descriptions and formulas in text form). This feature extractor 9 is composed of a deep neural network 10, and the gradient 11 is back-propagated to reduce the error of difficulty prediction, and the parameter 15 is updated.
The second element is a Decision Network (16) that judges the relative difficulty level between problems based on the extracted Feature (8). This decision network (16) is trained by gradient (11) to reduce the error by generating a large error when the difficulty level of the problems given to the training data is not properly matched. The above two elements are learned in the end-to-end method (13).
After learning, when a new problem that did not exist in the training data is given as an input, the difficulty classification model extracts a feature (8) from the problem and predicts and returns the difficulty of the given problem based on this.

Images