KR102467096B1 - Method and apparatus for checking dataset to learn extraction model for metadata of thesis - Google Patents

Abstract

A method for reviewing datasets for learning thesis metadata area classification model is provided. A method for reviewing a dataset for learning a thesis metadata domain classification model according to an embodiment of the present invention extracts thesis data for representing a thesis containing academic information and the machine-learned thesis metadata domain classification model. Receiving metadata of the thesis, displaying the metadata area text of the metadata and the thesis at the same time, modifying the metadata area text according to a user input for modifying the metadata area text , and processing the metadata correction data into learning data for learning the metadata area classification model.

Description

Dissertation Method and apparatus for checking dataset to learn extraction model for metadata of thesis}

The present invention relates to a method and apparatus for inspecting a dataset for learning a paper metadata domain classification model. More specifically, by using a web-based review tool, the metadata area text extracted from thesis is inspected, the modified contents are built into a dataset, and the built dataset can be used to learn thesis metadata area classification model. It is to provide a method and apparatus for inspecting a dataset having a data set.

Existing dissertation services have been made into a database of standardized literature metadata and shared mainly in PDF file format. On the other hand, services that provide full-text XML to readers are increasing, especially in influential journals and open access journals. attempts are increasing.

Creating a database of literature metadata was a costly and time-consuming process that was mostly done manually for a large number of papers. However, according to technical and social needs, a technology for quickly collecting full-text XML from existing PDF files and literature databases is required.

However, in the case of papers, the technical fields are diverse, and the contents are complex and new in most cases. Since vast literature is sporadically scattered, it is necessary to standardize and build a vast amount of thesis DB in order to efficiently utilize thesis. Most of the conventional thesis is shared in PDF file format of ADOBE, and thesis in PDF format Or, it is provided in other formats, but in many cases, thesis files in different formats are difficult to extract text from. For example, in the case of PDF-format thesis files, since there is no standardized format, the position where the text is placed is different for each paper, and even papers in the same technical field are all written in different formats. . In the case of thesis included in the PDF file, it is common for the title and author to be listed at the top of the first page, but in the case of abstract, keywords, and author information, which are other classifications, each paper has a different structure. Also, in the case of the main text, some papers consist of one column, and some papers consist of two columns.

Conventionally, in order to extract text from papers with different structures, text was extracted using image processing techniques such as OCR, but the text was sequentially extracted from left to right regardless of the classification of the metadata of the paper. did In this case, it was not possible to extract accurate metadata reflecting the document structure because it was mixed with the text of other table of contents.

In order to solve this problem, various models capable of extracting the metadata of papers are being developed, but even with the metadata extracted by these models, it has not been practically easy to accurately extract the metadata of PDF papers with different structures. Therefore, it was not possible to accurately recognize the text structure within the thesis, the text could not be extracted with high quality, and some kind of post-processing work was required to partially modify the extracted metadata.

Since this work takes time and money due to comparing thesis and metadata one by one, consumption to the extent of actually reproducing metadata occurred. In addition, since the revision history recorded through these tasks is the history of supplementing and correcting the metadata that the existing metadata domain classification model failed to properly extract, it can be used as excellent data for training the metadata domain classification model. However, these useful resources were not utilized and wasted.

Therefore, it was necessary to modify the extracted metadata. When metadata such as XML files are compared with papers, tags and other information are included in the nature of the XML format, resulting in inefficient visual confirmation. In the past, a review or correction tool existed, but when reviewing 'thesis' using it, the structure of the thesis was different and most of them contained new content, so it was not easy to work on the thesis.

Accordingly, a technology is needed to assist comparison work so that thesis and metadata can be recognized at a glance, and to provide a platform that can be reviewed and corrected. There is a need for a technology capable of constructing a learning dataset that can be stored in a format and used for learning a metadata domain classification model.

Registered Patent Publication No. 10-1500598 "XML Generation System and Method" (registered on March 3, 2020)

The technical problem to be solved by the present invention is to provide a dataset review method and apparatus for learning a thesis metadata area classification model capable of inspecting metadata area text extracted from a paper using a web-based review tool. is to do

Another technical problem to be solved by the present invention is a dataset for learning a thesis metadata region classification model in which the modified contents are built as a dataset and the constructed dataset can be used for learning thesis metadata region classification model. To provide an inspection method and device for

The technical problems of the present invention are not limited to the technical problems mentioned above, and other technical problems not mentioned will be clearly understood by those skilled in the art from the description below.

In order to solve the above technical problem, the method for reviewing a dataset for learning a thesis metadata domain classification model according to an embodiment of the present invention is a method performed by a computing device, in which a thesis containing academic information is expressed. Receiving thesis data and the metadata of the thesis extracted by the machine learning thesis metadata area classification model, displaying the metadata area text of the metadata and the thesis at the same time, the metadata area text It may include modifying the metadata area text according to a user input for correction, and processing data about the modification of the metadata into learning data for learning the metadata area classification model. .

In another embodiment, an apparatus for inspecting a dataset for learning a paper metadata domain classification model includes a processor, a network interface, a memory executed by the processor to load a computer program, and a storage for storing the computer program. Including, but the computer program, an instruction to receive the metadata of the thesis data extracted by thesis data for representing thesis including academic information and the machine-learned thesis metadata area classification model, the metadata of the metadata An instruction to display the data area text and the thesis together;

An instruction for modifying the metadata region text according to a user input for modifying the metadata region text, and an instruction for processing the metadata modification data into learning data for learning the metadata region classification model. can include

1 is an exemplary view of a method for verifying a dataset for learning a classification model for a thesis metadata area according to an embodiment of the present invention.
2 is a flowchart of a dataset verification method for learning a thesis metadata domain classification model according to an embodiment of the present invention.
3 is a diagram for explaining an example in which text and papers in the metadata area are displayed and learning data for learning a metadata area classification model is processed.
FIG. 4 is a diagram for explaining step S300 of FIG. 2 in detail.
5 is a diagram showing an example of a part of a thesis PDF.
FIG. 6 is a diagram showing the metadata area text output according to the classified metadata items on the data verification screen. FIG. 7 shows an example of classification and correction of text elements for the metadata area classification in which an error occurred in step S310. It is a drawing that represents FIG. 8 is a diagram showing an example of text element order correction in step S320. FIG. 9 is a diagram for explaining data input to the paper metadata domain classification model and results output by the paper metadata domain classification model.
10 is a diagram schematically illustrating a process of a paper metadata area classification model for extracting metadata of a paper from a PDF paper.
11 is a hardware configuration diagram of an apparatus for classifying a meta data area of a paper according to another embodiment of the present invention.

Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings. Advantages and features of the present invention, and methods for achieving them, will become clear with reference to the embodiments described below in detail in conjunction with the accompanying drawings. However, the present invention is not limited to the embodiments disclosed below and may be implemented in various different forms, only the present embodiments make the disclosure of the present invention complete, and the common knowledge in the art to which the present invention belongs It is provided to fully inform the holder of the scope of the invention, and the present invention is only defined by the scope of the claims. Like reference numbers designate like elements throughout the specification.

Unless otherwise defined, all terms (including technical and scientific terms) used in this specification may be used in a meaning commonly understood by those of ordinary skill in the art to which the present invention belongs. In addition, terms defined in commonly used dictionaries are not interpreted ideally or excessively unless explicitly specifically defined. Terminology used herein is for describing the embodiments and is not intended to limit the present invention. In this specification, singular forms also include plural forms unless specifically stated otherwise in a phrase.

Hereinafter, some embodiments of the present invention will be described with reference to the drawings.

1 is an exemplary view of a method for verifying a dataset for learning a classification model for a thesis metadata area according to an embodiment of the present invention.

Referring to FIG. 1 , when the thesis data and metadata field text are input to the thesis metadata checking device, a tool for checking the metadata of the thesis may be provided by the thesis metadata checking device 100 .

The thesis 10 may be composed of thesis data for expressing thesis including academic information, and in one embodiment, it may be a PDF file format thesis. The thesis 10 may be a document prepared by being classified into thesis title, English title, author information, English abstract, Korean abstract, text, annotation, etc. according to a typical structure. Metadata is data extracted by a machine-learned thesis metadata domain classification model, and may consist of XML, TXT, or JATS-XML, but the format is not limited thereto.

In some embodiments, the metadata input to the dissertation metadata review device 100 and the dissertation 10 may be displayed together through the web, and the page of the dissertation 10 and the meta corresponding to the page of the dissertation 10 A data area text 20 may be placed. The metadata area text 20 displayed on the web can be modified by user input, and the modified contents can be built into learning data. Reference is made to FIG. 2 for a detailed description of the method for verifying a dataset according to the present invention.

2 is a flowchart of a dataset review method for learning a thesis metadata area classification model according to an embodiment of the present invention, and FIG. 3 displays thesis and metadata area text, and learning of the metadata area classification model It is a diagram for explaining an example in which learning data for is processed.

In step S100 of FIG. 2 , thesis data and metadata of the thesis may be input. Specifically, thesis data for representing thesis including academic information and the metadata of the thesis extracted by the machine-learned thesis metadata domain classification model may be input. At this time, the thesis may be input in a PDF file format, and the thesis file may be data input by the user (local file) or data managed in the integrated thesis service database (file managed on the server). The text of the metadata field of the thesis may be composed of a metadata table classified according to the table of contents of the thesis. For example, the metadata field of a thesis may be data classified into title, English title, author information, English abstract, Korean abstract, Korean keyword, English keyword, and others (text, etc.).

In step S200, the metadata area text and the thesis may be displayed together on the web, and a page of the thesis and a metadata area corresponding to the page of the thesis may be arranged. In one embodiment, articles may be displayed on the left and metadata may be displayed on the right. The first page of the thesis may be displayed, and a table including contents corresponding to the first page of the thesis may be displayed as a metadata area table.

Referring to FIG. 3 , the thesis 10 and the metadata area text 20 may be displayed on the web. For example, input for text correction in the metadata area can be received on a thesis review platform based on the web of python or javascript. When the thesis 10 and the metadata area text 20 are input, the thesis 10 may be displayed on the left side and the metadata area text 20 may be displayed on the right side on the web.

The metadata area text (20) is title (2-1), English title (2-2), author information (2-3), annotation (2-4), and English according to the classification of the table of contents described in the thesis (10). It can be classified into abstract (2-5), Korean abstract (2-6), English keyword (2-7), Korean keyword (2-8), etc.

Metadata can be modified when an edit button displayed on the web is clicked, and modified metadata can be saved by clicking a save button. The stored data may be processed into learning data 30 for learning a metadata area classification model.

In step S300, the metadata area text 20 may be modified according to a user input. Here, the user's input may indicate keyboard input, mouse input, etc. on the web, and means correcting misclassification of metadata field text (or text element).

The text 20 in the metadata area that has been checked or modified by the user may be changed into training data of the thesis metadata area classification model in step S400. In one embodiment, a metadata area text classification divided into several text elements is refined by a user, and a rectangle enclosing the metadata area is formed based on data properties of coordinates located at the outermost part of the metadata area texts. Coordinates, text, font properties of text, etc. are stored as learning data. In this way, the metadata area text 20 can be processed into learning data.

The learning data secured through the inspection method according to the present invention can be used as a dataset for learning a thesis metadata area classification model. That is, the data on correction of metadata area text is input to the thesis metadata area classification model together with the thesis in order to learn the thesis metadata area classification model that outputs the refined metadata area for the thesis when the thesis is input. is the training dataset.

The inspection method according to the present invention supplements the metadata extracted through the thesis metadata area classification model to augment the dataset that can be learned so that the thesis metadata area classification model can extract metadata more accurately.

So far, the method for reviewing the dataset for learning the thesis metadata area classification model according to this embodiment has been schematically described. This embodiment may be performed by a computing device. For example, the computing device may be a device for classifying a metadata field of a paper described with reference to FIG. 1 . In describing the present embodiment, description of a performer of some operations may be omitted. At this time, the performing subject is the computing device.

Hereinafter, with reference to FIGS. 4 to 9 , an example of modifying metadata region text according to a user input for modifying metadata will be described.

When step S300 is performed in FIG. 4 , step S310 or step S320 may be selectively performed. In step S310, when it is determined that the classification of text elements in the PDF file is an error, the classification of the text element corresponding to the error may be corrected using the thesis metadata region classification model. In step S320, the classification of text elements disposed in a method different from a preset syntax analysis method among text elements may be modified.

Figure 5 is an example of a portion of the thesis PDF. Each text element is represented by a box represented by a dotted line, and the metadata area to be found is represented by a gray background. The indicator 'Hangeul abstract' at the top is a metadata area composed of one text element. The Korean abstract metadata area below consists of 6 text elements. Even if there is a single line in a picture, it can be composed of several text elements, and even a single sentence can be divided into text elements. Since the calculation of the metadata area composed of several text elements is based on the coordinates of the outermost elements, after properly extracting the order of the elements, the corresponding keywords can be entered in the Korean keyword area located below it.

The metadata area text extracted in FIG. 6 is output according to the metadata items classified on the data verification screen (table). In the figure, the texts in the Korean abstract area are arranged from top to bottom in order for people to read them, and printed in the 'Korean abstract' column. In addition, the Korean keyword field text is output in the 'Hangul keyword' column. Properly finding the order of text elements is of high importance as a required operation to classify the metadata area based on the coordinates of ○a outermost elements and to increase the accuracy of ○b metadata extraction. As a result, as shown in FIG. 6, text in the metadata area can be extracted in a normal order.

FIG. 7 shows an example of correcting the classification of a text element in which the metadata area classification in step S310 is incorrect. As shown in the figure, the 'Hangul Keyword' field is empty, and the text elements of the Korean keyword area are displayed in the 'Hangul Abstract'. In this case, since the classification of the text element in the PDF file is determined to be an error, the classification of the text element corresponding to the error can be corrected using the thesis metadata area classification model.

8 shows an example of correcting the order of text elements in step S320. According to the PDF file, the classification of text elements disposed in a method different from a preset syntax analysis method among text elements may be modified. Specifically, the classification of the text element may be modified to conform to a preset syntax analysis method. Through this, it is possible to increase the accuracy of determining the outermost element for calculating the metadata area and extracting the metadata text.

Using the corrected data, coordinates of the metadata area are calculated and refined metadata text information is obtained. This data can be used as training data for a metadata area classification model. In addition, this data can be used for database construction by separating author detailed information and keywords through artificial intelligence or metadata item extraction through hard coding.

Referring to FIG. 9 , when a paper 10 is input to a paper metadata area classification model, metadata area text 20 according to each classification of the paper may be output. Specifically, the thesis may include contents such as title, English title, author information, English abstract, Korean abstract, body, etc., and the paper metadata area classification model 200 classifies the contents included in the paper 10 It can be classified according to the table of contents. The thesis metadata area classification model 200 can classify the text element located at the top as the thesis title, classify the text element located at the bottom as the English title, and classify the text element located at the bottom as author information, , the rest of the elements can also be classified into English abstract, Korean abstract, text or notes according to the corresponding table of contents.

The metadata area text may be extracted using the result of automatically classifying the metadata area in this way. Metadata (20) includes thesis title (2-1), English title (2-2), author information (2-3), annotations (2-4), English abstract (2-5), Korean abstract (2- 6), English keywords (2-7), and Korean keywords (2-8).

The format of the thesis differs by journal, and even within the same journal, it may change according to the time period. The order of relative position has a rule, but the exact position of the metadata item varies for each paper, and the existence of a specific metadata item depends on the paper. may vary. The output of metadata field classifiers executed under these conditions may contain inaccurate classifications or text alignment sequences. In the case of incorrectly classified metadata area texts or texts arranged in the wrong order, the spatial distance between the correct objects, positional relationships, and word reconstruction by post-processing ('metadata' in the middle of the abstract of Fig. 5 are 'metade' + divided into 'data') can cause errors, so these errors must be accurately corrected to improve the learning performance of the metadata domain classifier and the accuracy of database construction.

Conventionally, since such a process has to be performed by directly modifying a text (or XML) file output by a metadata field classifier, there is room for deterioration in readability and accuracy due to occurrence of typos. Using the metadata area classification data method and apparatus according to the present invention, it is possible to modify the metadata area classification result under conditions and environments that can minimize human error, thereby increasing the accuracy of data inspection results.

The dataset inspection method for learning the thesis metadata area classification model 200 according to an embodiment of the present invention is provided based on a web implemented with python or javascript and is not dependent on any platform or OS, so various users can be implemented to review papers on various devices. In addition, when the present embodiment is provided in the server, it may be implemented so that a plurality of users access the server and utilize the present embodiment.

So far, with reference to FIGS. 10 and 11, the thesis metadata domain classification model of the present invention has been examined in detail.

12 is a diagram for explaining an example of learning a paper metadata area classification model with a training dataset processed by modifying PDF papers and metadata.

The above-described thesis PDF file 10 and the learning data 30 processed by the inspection method of this embodiment can be used for learning the thesis metadata area classification model. For example, the thesis metadata area classification model calculates an error with the metadata extracted from the existing model through the learning data 30 when the thesis PDF file 10 and the learning data 30 are input, and calculates the error. It can be learned by backpropagating and updating weights. The method by which the thesis metadata area classification model is learned is not limited thereto.

Hereinafter, referring to FIG. 11 , an exemplary computing device 500 capable of implementing the devices described in various embodiments of the present invention will be described.

11 is an exemplary hardware configuration diagram illustrating a computing device 500 .

As shown in FIG. 11, the computing device 500 loads one or more processors 510, a bus 550, a communication interface 570, and a computer program 591 executed by the processor 510. It may include a memory 530 and a storage 590 for storing the computer program 591. However, only components related to the embodiment of the present invention are shown in FIG. 11 . Accordingly, those skilled in the art to which the present invention pertains can know that other general-purpose components may be further included in addition to the components shown in FIG. 11 .

The processor 510 controls the overall operation of each component of the computing device 500 . The processor 510 may include at least one of a Central Processing Unit (CPU), a Micro Processor Unit (MPU), a Micro Controller Unit (MCU), a Graphic Processing Unit (GPU), or any type of processor well known in the art. can be configured to include Also, the processor 510 may perform an operation for at least one application or program for executing a method/operation according to various embodiments of the present disclosure. Computing device 500 may include one or more processors.

Memory 530 stores various data, commands and/or information. Memory 530 may load one or more programs 591 from storage 590 to execute methods/operations according to various embodiments of the present invention. For example, when the computer program 591 is loaded into the memory 530, the logic (or module) shown in FIG. 4 may be implemented on the memory 530. An example of the memory 530 may be RAM, but is not limited thereto.

The bus 550 provides a communication function between components of the computing device 500 . The bus 550 may be implemented as various types of buses such as an address bus, a data bus, and a control bus.

The communication interface 570 supports wired and wireless Internet communication of the computing device 500 . The communication interface 570 may support various communication methods other than Internet communication. To this end, the communication interface 570 may include a communication module well known in the art.

Storage 590 may non-temporarily store one or more computer programs 591 . The storage 590 may be a non-volatile memory such as read only memory (ROM), erasable programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), flash memory, or the like, a hard disk, a removable disk, or a device well known in the art. It may be configured to include any known type of computer-readable recording medium.

Computer program 591 may include one or more instructions in which methods/operations according to various embodiments of the invention are implemented. When computer program 591 is loaded into memory 530, processor 510 may execute the one or more instructions to perform methods/acts according to various embodiments of the present invention.

In one embodiment, a processor, a network interface, a memory that is executed by the processor to load a computer program, and a storage for storing the computer program, wherein the computer program includes academic information. An instruction to receive thesis data for expression and the metadata of the thesis extracted by the machine-learned thesis metadata area classification model, an instruction to display the metadata and the thesis together, and a modification of the metadata area text It may include an instruction for modifying the metadata according to a user input, and an instruction for processing data about modification of the metadata area text into learning data for learning the metadata classification model.

In one embodiment, the instruction for displaying the metadata and the thesis together may include an instruction for arranging a page of the thesis and metadata corresponding to the page of the thesis.

In one embodiment, the instruction for modifying the metadata according to a user input for modifying the metadata may include an instruction for receiving an input for modifying the metadata in a web-based thesis review platform. .

The methods according to the embodiments of the present invention described so far can be performed by executing a computer program implemented as a computer readable code. The computer program may be transmitted from the first computing device to the second computing device through a network such as the Internet, installed in the second computing device, and thus used in the second computing device. The first computing device and the second computing device include both a server device, a physical server belonging to a server pool for a cloud service, and a fixed computing device such as a desktop PC.

The computer program may be stored in a recording medium such as a DVD-ROM or a flash memory device.

Although the embodiments of the present invention have been described with reference to the accompanying drawings, those skilled in the art to which the present invention pertains can be implemented in other specific forms without changing the technical spirit or essential features of the present invention. can understand that Therefore, the embodiments described above should be understood as illustrative in all respects and not limiting.

Claims (13)

In a method performed by a computing device,
Receiving thesis data for representing thesis including academic information and the metadata of the thesis extracted by the machine-learned thesis metadata domain classification model;
displaying a classification result of the metadata by the thesis metadata area classification model and the thesis together on one screen;
correcting misclassification of the metadata according to a user input for modifying the metadata; and
Processing the metadata correction data into learning data for re-learning of the machine-learned thesis metadata area classification model,
A method for reviewing datasets for learning thesis metadata area classification model. According to claim 1,
The thesis metadata domain classification model,
Identify the metadata area text included in the thesis using the coordinates according to the arrangement of the text elements of the thesis and the properties of the text elements, and determine the area where the metadata area text included in the thesis is arranged as the metadata area and extract text from the metadata area,
The thesis metadata area classification model is machine-learned to classify the metadata area of the metadata area text based on the coordinates according to the arrangement of the text elements of the thesis and the properties of the text elements,
A method for reviewing datasets for learning thesis metadata area classification model. According to claim 2,
The thesis metadata domain classification model,
A model for classifying the metadata area based on coordinates of outermost elements of a plurality of text elements,
A method for reviewing datasets for learning thesis metadata area classification model. According to claim 2,
The thesis metadata domain classification model,
A model for classifying the table of contents to which the metadata area text belongs in the thesis using the arrangement of the text elements and the properties of the text,
A method for reviewing datasets for learning thesis metadata area classification model. According to claim 1,
The step of displaying the classification result by the thesis metadata area classification model and the thesis together on one screen,
Arranging the page of the paper and the classification result corresponding to the page of the paper,
A method for reviewing datasets for learning thesis metadata area classification model. According to claim 1,
In accordance with the user input for modifying the metadata, the step of correcting the misclassification of the metadata,
When the classification of the text element of the paper is determined to be an error, correcting the classification of the text element corresponding to the error using the paper metadata region classification model,
A method for reviewing datasets for learning thesis metadata area classification model. According to claim 1,
In accordance with the user input for modifying the metadata, the step of correcting the misclassification of the metadata,
Modifying the classification of text elements arranged in a method different from a preset syntax analysis method among the text elements of the thesis to conform to the preset syntax analysis method,
A method for reviewing datasets for learning thesis metadata area classification model. According to claim 1,
In accordance with the user input for modifying the metadata, the step of correcting the misclassification of the metadata,
Receiving an input for correction of misclassification of the metadata in a web-based thesis review platform,
A method for reviewing datasets for learning thesis metadata area classification model. According to claim 1,
The step of processing the metadata correction data into learning data for re-learning of the machine-learned thesis metadata area classification model,
Converting and storing data on the modification of the metadata into a format that can be input to the thesis metadata area classification model,
A method for reviewing datasets for learning thesis metadata area classification model. processor;
network interface;
a memory that is executed by the processor and loads a computer program; and
Including a storage for storing the computer program,
The computer program,
An instruction for receiving thesis data for representing a thesis including academic information and the metadata of the thesis extracted by the machine-learned thesis metadata domain classification model;
an instruction for displaying a classification result of the metadata by the thesis metadata domain classification model and the thesis together on one screen;
an instruction for correcting misclassification of the metadata according to a user input for modifying the metadata; and
Including instructions for processing the metadata correction data into learning data for re-learning of the machine-learned thesis metadata area classification model,
A dataset review device for learning a thesis metadata area classification model. According to claim 10,
The thesis metadata domain classification model,
Identify the metadata area text included in the thesis using the coordinates according to the arrangement of the text elements of the thesis and the properties of the text elements, and determine the area where the metadata area text included in the thesis is arranged as the metadata area and extract text from the metadata area,
The thesis metadata area classification model is machine-learned to classify the metadata area of the metadata area text based on the coordinates according to the arrangement of the text elements of the thesis and the properties of the text elements,
A dataset review device for learning a thesis metadata area classification model. According to claim 10,
The thesis metadata domain classification model,
A model that classifies the metadata area based on the coordinates of the outermost elements of a plurality of text elements,
A dataset review device for learning a thesis metadata area classification model. A computer program that operates in combination with a computing device,
Receiving thesis data for representing thesis including academic information and the metadata of the thesis extracted by the machine-learned thesis metadata domain classification model;
displaying a classification result of the metadata by the thesis metadata area classification model and the thesis together on one screen;
correcting misclassification of the metadata according to a user input for modifying the metadata; and
Stored in a computer-readable recording medium to execute the step of processing the data for modification of the metadata into learning data for re-learning of the machine-learned thesis metadata area classification model,
computer program.

Images