WO2023095991A1

WO2023095991A1 - System for automatically extracting question area and type within content for learning included in electronic document and method therefor

Info

Publication number: WO2023095991A1
Application number: PCT/KR2021/018277
Authority: WO
Inventors: 신은영
Original assignee: 신은영
Priority date: 2021-11-29
Filing date: 2021-12-03
Publication date: 2023-06-01
Also published as: KR20230080113A

Abstract

The present invention relates to a system for automatically extracting an area and type of questions within content for learning included in an electronic document, and a method therefor. The system for automatically extracting an area and type of questions within content for learning included in an electronic document, according to the present invention, comprises: an input unit that registers an electronic document; a memory that stores a program for automatically extracting an area and type of questions from content for learning included in the electronic document; and a processor that executes the program, wherein the processor extracts a page on which questions are present, extracts an area and type of the questions from the extracted page, and constructs a question information database.

Description

System and method for automatically extracting question areas and types in contents for learning electronic documents

The present invention relates to a system and method for automatically extracting an area and type of an item from contents for learning an electronic document.

Moving away from the traditional print publishing method, the provision of learning contents using electronic documents and e-books is becoming more common.

According to the prior art, according to the provision of learning content in the form of an electronic document, online learning can be performed using a PC, laptop, smart phone, tablet PC, etc. without time and space limitations.

However, according to the prior art, only the method of containing content has changed from paper books to electronic documents, and it is a platform that provides various services using the characteristics of electronic documents so that learners can learn more conveniently. There are limitations to which provision has not been made.

According to the prior art, there has been a proposal to extract data through a method in which a user manually designates an item area in learning content, but this still has a problem in that it is difficult to reflect the various needs of learners in terms of interaction.

In addition, the method of collecting item text information in electronic documents and generating problem-solving data has a problem in that the data capacity is large and it is impossible to collect information about special symbols, mathematical symbols, and images.

The present invention has been proposed to solve the above problems, and an object of the present invention is to provide a system and method capable of automatically extracting and storing question areas and types within electronic document learning contents using machine learning.

A system for automatically extracting item areas and types in contents for learning electronic documents according to the present invention executes a memory and a program in which an input unit for registering electronic documents and a program for automatically extracting item areas and types from learning contents included in electronic documents are stored. and a processor that executes the task, and the processor extracts a page on which the question exists, extracts the area and type of the question from the extracted page, and builds a question information database.

The processor performs automatic extraction using an object detection model that detects information on questions, answers, explanations, and correct answers.

The processor extracts the type of the question based on a criterion including at least one of a true/false type, a selective type, a connection type, a short answer type, a complete type, and a descriptive type.

The processor extracts information of the question, correct answer information, and explanation information, checks whether they match, checks the accuracy of the object detection model, adjusts a threshold value when matching criteria are not met, and matches matching criteria. If it is, the detailed area for the above question is extracted.

The processor determines whether an example number area for each type of question, whether or not an example text area exists, whether an answer area for writing short answers, whether a view area for selecting authenticity or falseness exists, and whether a paired answer group area exists. Check whether it exists and whether the view area and answer input area are included.

According to the present invention, by predicting the area and type of an item together from the learning content, detecting a detailed area according to the type within the item area, and storing the detected information, the speed and reliability of object detection in the learning content are increased. There are possible effects.

According to the present invention, since the area of item data is stored as coordinates, data capacity is reduced, and item information can be stored regardless of mathematical symbols or images.

The effects of the present invention are not limited to those mentioned above, and other effects not mentioned will be clearly understood by those skilled in the art from the description below.

1 illustrates a system for automatically extracting item areas and types in contents for learning electronic documents according to an embodiment of the present invention.

2 shows an object area of a question sheet according to an embodiment of the present invention.

3 illustrates a method for automatically extracting a question area and type from contents for learning electronic documents according to an embodiment of the present invention.

The foregoing and other objects, advantages and characteristics of the present invention, and a method of achieving them will become clear with reference to the detailed embodiments described below in conjunction with the accompanying drawings.

However, the present invention is not limited to the embodiments disclosed below and may be implemented in various different forms, and only the following embodiments provide the purpose of the invention, As only provided to easily inform the configuration and effect, the scope of the present invention is defined by the description of the claims.

Meanwhile, terms used in this specification are for describing the embodiments and are not intended to limit the present invention. In this specification, singular forms also include plural forms unless specifically stated otherwise in a phrase. As used herein, “comprises” and/or “comprising” means the presence of one or more other components, steps, operations, and/or elements in which a stated component, step, operation, and/or element is present. or added.

A system for automatically extracting item areas and types in content for learning electronic documents according to an embodiment of the present invention includes an input unit 110 for registering electronic documents, and areas and types (labels) of items in learning contents included in the electronic documents. It includes a memory 120 in which a program to be extracted is stored and a processor 130 that executes the program, and the processor 130 extracts a page on which the question exists, extracts the area and type of the question from the extracted page, and Build an information database.

The processor 130 uses a pre-learned object detection model, and this object detection model is a model for detecting item, answer, commentary, and correct answer information. Machine learning algorithms, artificial neural network algorithms, etc. may be used.

With respect to the item type, the processor 130 classifies the type of item existing within the domain of the item using a criterion including at least one of true/false type, optional type, connection type, short answer type, complete type, and descriptive type.

The processor predicts an item having the highest class score (probability value) by using an object detection model, finds a region of interest (ROI), and predicts coordinate values.

The processor 130 estimates an area in which items are estimated to exist. At this time, the processor quickly finds a region in which the object is expected to exist in the input image, makes the size the same for the estimated region in the form of an image in a rectangular bounding box, and then CNN (Convolutional Classification is performed through a Neural Network), and an object is predicted using a confidence score value.

As another example, the processor 130 classifies each region of the image into an object region and a non-object region, and classifies a region matching the detection result among predicted object regions as a normal detection region.

As another example, the processor 130 classifies each region of the image into an object region and a non-object region, and predicts the type of object by ensembling results obtained through a plurality of classifiers.

The processor 130 extracts location information corresponding to the question number, question number, answer number, example, and answer input within the area of the question.

Processor 130 creates the necessary additional locations within the item area.

The processor 130 classifies the area and type of the object within the learning content, but predicts whether it is an item or a correct answer and a commentary in relation to the type of object.

The processor 130 extracts item information, extracts correct answers and explanation information, and compares the extracted information to check the accuracy of values predicted by the object detection model.

Through this, the processor 130 adjusts the threshold value when the matching criterion is not satisfied.

When it is confirmed that the matching criterion is satisfied, the processor 130 extracts detailed areas for each item type.

The processor 130 detects a detailed area from the detected question area. At this time, it is possible to select the existence or non-existence of an answer number area and an example text area for each type of question, the existence or non-existence of an answer area for composing a brief short answer, and whether or not it is true. It checks whether there is a viewing area, whether there is a paired answer group area, and whether the viewing area and answer input area are included.

In detecting the detailed region, the processor 130 may use a second object detection model different from the object detection model that finds the region and type of the object or extract object information through text analysis.

The processor 130 provides information of the electronic document along with location information through the electronic document viewer.

The processor 130 stores detected object information (item information) for each predicted object type.

According to another embodiment of the present invention, the processor 130, in the process of simultaneously extracting item areas and types, i) characteristic information of item areas and types acquired in the learning process or ii) information included in the database for each publisher It is possible to improve the speed of extraction by considering the characteristic information of the item domain and type.

For example, in the case of the workbook published by 'AA Education' publisher (or published as an e-book), questions are provided for each concept test, descriptive test, thinking test, creativity/convergence/coding test, and midterm/final basic test. The displayed form is the same or similar.

The processor 130 simultaneously extracts the item area and type for each test item (conceptual test, descriptive test, thinking test, creativity/convergence/coding test, midterm/final basic test), object detection model and question It is possible to perform learning while updating the information database, and to increase the extraction speed for item areas and types to be subsequently classified based on the learned results.

Alternatively, the processor 130 displays characteristic information included in the database for each publisher (eg, AA education 2nd grade math problem book, RR education 1st grade math problem book, etc., each test item and each test item for each publisher) It is possible to extract the item area and type at the same time by using the form information stored as feature information. Through this, it is possible to more quickly extract the area and type of an item based on feature information without scanning and classifying the entire area within the extracted page.

The processor 130 determines whether the item number area and the item text area are included in the detected item area, and if the item number area (QN) and the item text area (QT) are included, the corresponding object is an item unit object. to decide

When the question unit object is determined, whether there is an example number area (EN1 to EN5), an example text area (ET1 to ET5), an answer area (ab) for writing a short answer, and whether or not it is true (O, X) The type of question is determined through the existence or non-existence of the view area for selecting, the existence of the answer sheet group area, which is a paired view number area, and the existence of the view area and answer input area.

According to the determination of the item type, page information and coordinate information for the detected object are stored.

3 illustrates a method for automatically extracting the position and type of a question in contents for learning electronic documents according to an embodiment of the present invention.

The method for automatically extracting the location and type of items in contents for learning electronic documents according to an embodiment of the present invention includes the steps of receiving contents for learning (S310), simultaneously predicting the area and type of an object based on an object detection model (S320), , Extracting item information and extracting correct answer and commentary information (S330), checking whether the item, correct answer, and commentary information match (S340), and extracting detailed areas for each item type (S350) and storing object information extracted for each object type (S360).

In step S320, in simultaneously predicting the area and type of an object within the learning content, whether it is an item or a correct answer and a commentary is predicted in relation to the object type.

In step S320, after making the estimated region, which is an image form in a rectangular bounding box, the same size according to the region estimation method, classification is performed through a Convolutional Neural Network (CNN), and a confidence score (confidence score) is performed. score) value to estimate the area of the object.

In step S320, it is possible to classify each region of the image into an object region and a non-object region, and classify a region matching the detection result among the predicted object regions as a normal detection region.

In step S320, classification is performed for each region of the image into an object region and a non-object region, and it is possible to predict the type of object by ensembling the results obtained through a plurality of classifiers.

Step S320 classifies the type of question existing in the domain of the question into at least one of a true/false type, a selective type, a connection type, a short answer type, a complete type, and a narrative type in relation to the item type.

Step S320 extracts location information corresponding to the question number, question number, answer number, answer, and answer input within the area of the question.

In step S330, unit information, number of problems, problem type, problem number, problem type, and page information are extracted in extracting information.

Step S340 checks the accuracy of the value predicted by the object detection model using the item information extracted in step S330, the correct answer, and the commentary information.

In step S340, when it is determined that the matching is less than or equal to the standard, a step of adjusting the threshold value is performed.

In step S350, as a result of checking in step S340, if it is confirmed that the matching criterion is satisfied, detailed areas are extracted for each item type.

In step S350, whether there is an example number area for each type of question, whether there is an example text area, whether or not an answer area for writing a short answer, whether a view area for selecting authenticity or falseness exists, and existence of a paired answer group area Check whether and whether the view area and answer input area are included.

At this time, in step S350, it is possible to use a second object detection model different from the object detection model in step S320 or to extract object information through text analysis.

In operation S360 , object information detected for each type of object is stored, and electronic document information is displayed along with object location information through an electronic document viewer.

Meanwhile, the method for automatically extracting the position and type of a question in contents for learning electronic documents according to an embodiment of the present invention may be implemented in a computer system or recorded in a recording medium. A computer system may include at least one processor, a memory, a user input device, a data communication bus, a user output device, and a storage. Each of the aforementioned components communicates data through a data communication bus.

The computer system may further include a network interface coupled to the network. The processor may be a central processing unit (CPU) or a semiconductor device that processes instructions stored in memory and/or storage.

The memory and storage may include various types of volatile or non-volatile storage media. For example, memory may include ROM and RAM.

Accordingly, the method for automatically extracting the location and type of questions in the content for learning electronic documents according to an embodiment of the present invention can be implemented in a computer-executable manner. When the method of automatically extracting the location and type of items in the content for learning electronic documents according to an embodiment of the present invention is performed in a computer device, computer-readable instructions perform the method of automatically extracting the location and type of items in the contents for learning electronic documents according to the present invention. can be done

Meanwhile, the above-described method for automatically extracting the location and type of questions in contents for learning electronic documents according to the present invention can be implemented as computer-readable codes on a computer-readable recording medium. Computer-readable recording media includes all types of recording media in which data that can be decoded by a computer system is stored. For example, there may be read only memory (ROM), random access memory (RAM), a magnetic tape, a magnetic disk, a flash memory, an optical data storage device, and the like. In addition, the computer-readable recording medium may be distributed in computer systems connected through a computer communication network, and stored and executed as readable codes in a distributed manner.

Claims

an input unit for registering an electronic document;

a memory storing a program for automatically extracting an area and type of an item from learning contents included in the electronic document; and

A processor for executing the program;

The processor extracts the page where the question exists, extracts the area and type of the question from the extracted page, and builds the question information database.

A system for automatically extracting question areas and types within content for learning electronic documents.
According to claim 1,

The processor performs automatic extraction using an object detection model that detects item, answer, commentary, and correct answer information.

A system for automatically extracting question areas and types within content for learning electronic documents.
According to claim 1,

The processor extracts the type of the question based on a criterion including at least one of a true/false type, a selection type, a connection type, a short answer type, a complete type, and a descriptive type.

A system for automatically extracting question areas and types within content for learning electronic documents.
According to claim 1,

The processor extracts information of the question, correct answer information, and explanation information, checks whether they match, checks the accuracy of the object detection model, adjusts a threshold value when matching criteria are not met, and matches matching criteria. If it is, extracting the detailed area for the above question

A system for automatically extracting question areas and types within content for learning electronic documents.
According to claim 1,

The processor determines whether an example number area for each type of question, whether or not an example text area exists, whether an answer area for writing short answers, whether a view area for selecting authenticity or falseness exists, and whether a paired answer group area exists. To check whether it exists and whether the view area and answer input area are included

A system for automatically extracting question areas and types within content for learning electronic documents.