WO2012005414A1

WO2012005414A1 - System and method for evaluating suitability of a reference document

Info

Publication number: WO2012005414A1
Application number: PCT/KR2010/008114
Authority: WO
Inventors: 정한민; 김평; 이승우; 이미경; 서동민; 성원경
Original assignee: 한국과학기술정보연구원
Priority date: 2010-07-09
Filing date: 2010-11-16
Publication date: 2012-01-12
Also published as: KR101033611B1

Abstract

The present invention relates to a system and method for evaluating the suitability of a reference document and comprises: a subject-word comparison module comparing one or more subject words assigned in an input document with one or more subject words assigned in a reference document in a matching method; and a correlation evaluation module evaluating the correlation between the input document and reference document using the compared result in the subject-word comparison module. According to the present invention, a reference document can be analyzed to investigate the correlation of the input document with the reference document, thus providing support in determining that "a suitable reference document is cited" and further determining objects on which to perform inspections of academic papers for plagiarism.

Description

Relevance Determination System and Method

The present invention relates to a system and method for determining the suitability of a bibliography, and more particularly, to comparing one or more main words assigned to an input document with one or more main words assigned to one or more references in the input document in a matching manner, and comparing A reference system and method for determining the suitability of a reference for determining an association between the input document and the reference using the results.

References are related documents attached to documents such as articles and reports and are very important. In the case of a manuscript, it is possible to evaluate the manuscript only by looking at the table of contents, the introduction, and the bibliography, because it is important to refer to the literature appropriate for the contents of the topic and introduction.

Citation indexes for these references have been studied, including the Science Citation Index (SCI) developed by ISI in the United States.

However, these indicators are the result of manual analysis of references, and their role is limited to the citation of documents.

In addition, this information only provides a limited service of how much a particular document is cited and how influential it is, and there is no way to determine whether an appropriate reference is cited.

SUMMARY OF THE INVENTION The present invention has been made to solve the above problems, and an object of the present invention is to provide a system and method for determining the suitability of references, which can identify the association with the input literature by analyzing the references.

According to an aspect of the present invention to achieve the above object, a main word comparison module for comparing the at least one main word assigned to the input document and at least one main word assigned to the at least one reference document in the input document, the main word comparison When the matching result obtained from the comparison result in the module is compared with the threshold value, the entire reference is referred to a suitable citation when the reference value is higher than the threshold value, and when the reference value is less than the threshold value, the reference is referred to as an inappropriate citation. A reference conformity determination system is provided that includes a determining association module.

The reference suitability determination system parses the input literature to determine a reference region, extracts each reference in the determined reference region, and separates the components in the reference. A bibliographic linkage module that stores or updates in a bibliographic database and extracts one or more main words from at least one of bibliographic information, summaries, and original texts of the bibliography and the bibliography. The module may further include.

The main word extracting module may include: an index word extracting unit extracting terms except for stop words from at least one of bibliographic information, a summary, and an original text of the input document and the reference document; Refined) includes a subject filtering unit for filtering the subject compared with the subject dictionary, and a subject determining unit for selecting a subject ranked above by using the frequency condition of the filtered subject.

The frequency condition may be at least one of a term frequency condition, a document frequency condition, and a field classification code frequency condition.

The main keyword comparison module is a statistical calculation unit for collecting and sorting the main words of the reference, and calculates in order of frequency, matching the main word calculated by the statistical calculation unit and the main word of the input document, the matching ratio, frequency-based matching weight It may include a subject comparison unit for obtaining at least one.

In addition, the association determination module, if the matching result obtained through the comparison of the main word is a threshold value or more and the main word not present in the reference is present in the input document, the input document is a progressive research topic for the entire reference I think it covered.

In addition, when the matching result obtained through the comparison of the main words is less than a threshold value, the association determination module searches whether there is another document including the main word of the input document in a previously registered literature database, and based on the search result Determine at least one of omission, omission, and fit.

The main word comparison module compares the main word of the input document with the main word of each reference to obtain at least one of a matching ratio for each reference and a frequency-based matching weight, and the correlation determination module determines that the obtained matching result is a threshold value. The above reference is considered to be an appropriate citation, and the reference below the threshold is judged to be an inappropriate citation.

According to another aspect of the present invention, (a) comparing at least one subject word assigned to the input document and at least one subject word assigned to at least one reference document in the input document in a matching manner, (b) through the comparison result Comparing the obtained reference result with a threshold value and determining the association between the input document and the reference document as a proper citation when the reference value is greater than or equal to a threshold value, and when the reference value is less than the reference value as an entire citation value when the reference value is less than the threshold value. A document conformity determination method is provided.

Parsing the input literature prior to step (a) to determine a bibliography region, extracting each bibliography in the determined bibliography region, and separating the components in the bibliography into a bibliographic database. The method may further include storing or updating the data, and associating with the input document, extracting one or more subject words from at least one of bibliographic information, a summary, and an original text of the input document and the reference document, and assigning them to the document.

In the step (a), the main words of all the references are sorted and arranged in order of frequency, the main words calculated in the order of frequency and the main words of the input document are matched, and at least one of a matching ratio and a frequency-based matching weight. Obtaining a step.

In the step (b), if at least one of a matching ratio and a frequency-based matching weight obtained through the main word comparison result is greater than or equal to a threshold, the entire reference is determined to be an appropriate citation, and if less than the threshold The entirety of this reference is considered to be inadequate citation.

If the matching result obtained from the comparison of the main words is greater than or equal to a threshold value and there is a main word in the input document that is not present in the entire reference, it is determined that the input document deals with a progressive research topic for the entire reference.

When the matching result obtained through the comparison of the main words is less than a threshold value, a search is made in the previously registered literature database to see if there is another document including the main word of the input document, and at least one of plagiarism, omission, and goodness based on the search result. The abnormality is determined.

In the step (a), the main word of the input reference is compared with the main word of each reference to obtain at least one of a matching ratio for each reference and a frequency-based matching weight.

The reference that the obtained matching result is greater than or equal to the threshold is determined to be a suitable citation, and the reference that is less than the threshold is determined to be an inappropriate citation.

As described above, according to the present invention, by analyzing the reference literature and identifying the association with the input literature, it is possible to support the determination of 'relevant reference citation', which is one of the criteria for reviewing the thesis, and to determine the subject of the duplicate article inspection.

1 is a diagram illustrating a reference conformity determination system according to an embodiment of the present invention.

FIG. 2 is a block diagram schematically showing the main word extraction module shown in FIG.

FIG. 3 is a block diagram schematically illustrating a configuration of the main keyword comparison module shown in FIG. 1. FIG.

4 and 5 are flowcharts illustrating a method of determining conformance of references according to an embodiment of the present invention.

6 is a view for explaining a method for determining a reference region according to an embodiment of the present invention.

7 is a view for explaining a method of associating an input document with a reference document according to an embodiment of the present invention.

8 is a view for explaining a result of matching keywords between an input document and a reference document according to an embodiment of the present invention.

9 is a flowchart illustrating a correlation determination method using a matching result and a threshold according to the present invention.

Details of the above-described objects and technical configurations of the present invention and the effects thereof according to the present invention will be more clearly understood by the following detailed description based on the accompanying drawings.

1 is a view showing a reference conformity determination system according to an embodiment of the present invention.

Referring to FIG. 1, the reference conformity determination system 100 compares, in a matching manner, one or more main words assigned to an input document and one or more main words assigned to one or more references in the input document. And an association determining module 150 that determines an association between the input document and the reference document using the comparison result in the main keyword comparison module 140.

The main word comparison module 140 sorts all the main words of all references and calculates them in frequency order, and obtains at least one of a matching ratio and a frequency-based matching weight by matching the calculated main word and the main word of the input document.

The association determination module 150 compares the matching result obtained through the main keyword comparison result in the main keyword comparison module 140 with a predetermined threshold value and determines that the entire reference is a proper citation when the threshold value is greater than or equal to the threshold value. In this case, the entirety of the reference is determined to be an inappropriate citation. Here, the matching result may be at least one of a matching ratio and a frequency-based matching weight.

In addition, the association determination module 150 may determine that the input document is progressive with respect to the entire reference when the matching result obtained through the comparison of the main word is greater than or equal to a threshold value and a main word not present in the reference is present. Judging from the topic of research

In addition, the association determination module 150, if the matching result obtained through the comparison of the main word in the main word comparison module 140 is less than a threshold value, there is another document including the main word of the input document in a pre-registered literature database. Search for

The association determination module 150 then determines at least one of plagiarism, omission, and suitability based on the search results.

According to another embodiment of the present invention, the main word comparison module 140 may obtain at least one of a matching ratio for each reference and a frequency-based matching weight by comparing the main word of the input document with the main word of each reference. In this case, the correlation determination module compares at least one of the obtained matching ratio and frequency-based matching weight with a predetermined threshold.

The association determination module 150 then determines that the reference in the main comparison module 140 is a reference that is greater than or equal to a threshold value as a suitable citation, and determines that the reference that is less than the threshold value is an invalid citation.

The reference suitability determination system 100 configured as described above may further include a reference recognition module 110, a reference linkage module 120, and a main word extraction module 130.

The bibliography recognition module 110 parses the input literature to determine a bibliography region, and extracts each bibliography in the determined bibliography region.

The bibliography linkage module 120 separates the elements in the bibliography and stores or updates the bibliography in a bibliographic database. Here, the components in the reference refers to the author, title, source, page number, year, and the like.

In addition, the reference linkage module 120 performs component normalization if necessary, merges and assigns documents having similarities over a certain threshold compared to other documents stored in a previously registered reference database. It performs the function.

The main word extracting module 130 extracts one or more main words from at least one of bibliographic information, a summary, and an original text of the input document and the reference document and assigns them to the document.

A detailed description of the main control module 140 performing the above-described role will be given with reference to FIG. 3.

Detailed description of the main word extraction module 130 will be described with reference to FIG. 2.

FIG. 2 is a block diagram schematically illustrating a main word extraction module shown in FIG. 1.

Referring to FIG. 2, the main word extracting module 130 extracts terms except for stop words from at least one of bibliographic information, a summary, and an original text of an input document and a reference document as an index word extracting unit 132. A main word filtering unit 134 for filtering a main word by comparing the extracted index words with a previously refined main word dictionary, and a main word determination unit 136 for selecting a main word ranked higher using the frequency condition of the filtered main word. It includes.

The main keyword filtering unit 134 compares the extracted index terms with the main dictionary to exclude those having no main keyword value.

The main word determining unit 136 applies the main words filtered by the main word filtering unit 134 to at least one frequency condition among a term frequency condition, a document frequency condition, and a field classification code frequency condition. Rank it.

Then, the main word determining unit 136 determines a predetermined number of main words as the main words of the document for each higher rank.

FIG. 3 is a block diagram schematically illustrating a configuration of the main keyword comparison module shown in FIG. 1.

Referring to FIG. 3, the main word comparison module 140 collects and sorts main words of a reference in an input document, and calculates them in order of frequency, and the main word calculated by the statistical calculation part 142 and the main word. And a main word comparator 144 that matches a main word of the input document to obtain at least one of a matching ratio and a frequency-based matching weight.

The statistical calculation unit 142 collects and sorts the main words of all the references in the input document, and calculates a frequency for each of the sorted main words.

The main word comparison unit 144 obtains at least one of a matching ratio or a frequency-based matching weight in which the main word of the input document matches the main word of the reference document based on the frequency of the main word calculated by the statistical calculation unit 142. do.

4 is a flowchart illustrating a reference conformity determination method according to an embodiment of the present invention.

Referring to FIG. 4, the reference conformity determination system compares one or more main words assigned to the input document with one or more main words assigned to the one or more reference documents in the input document (S400).

That is, the reference conformity determination system collects the main words of all the references and sorts them in frequency order. The reference suitability determination system then matches at least one of a main word arranged in the frequency order and a main word of the input document to obtain at least one of a matching ratio and a frequency-based matching weight.

After performing the S400, the reference suitability determination system determines an association between the input document and the reference document using the comparison result (S402).

A detailed description of how the reference conformity determination system determines the association between the references will be described with reference to FIG. 9.

5 is a flowchart illustrating a method of determining reference conformity according to an embodiment of the present invention, FIG. 6 is a view for explaining a method for determining a reference region according to an embodiment of the present invention, and FIG. 7 is according to an embodiment of the present invention. 8 is a diagram for describing a method of linking an input document and a reference document, and FIG. 8 is a view for explaining a result of matching keywords between an input document and a reference document according to an embodiment of the present invention.

Referring to FIG. 5, the reference conformity determination system parses an input document to determine a reference area (S500), and extracts each reference in the determined reference area (S502).

That is, the reference suitability determination system parses the input literature, and if an indicator indicating an area such as 'Reference / s' or 'Reference literature' is identified in the text document, the reference is mainly focused on the identified region. Determine the literature area.

For a method of determining a reference region in the input literature, referring to FIG. 6, the reference conformity determination system refers to the region 600 indicated by the 'Reference', since 'Reference' exists in the input literature. Determined by the literature area.

The reference suitability determination system then extracts a list of references in the determined reference area. In this case, the reference list includes all the components in the reference, and the components refer to, for example, author, title, source, page, year, and the like.

After the operation of S502, the reference conformity determination system separates the components in the reference, stores or updates them in the literature database, and associates them with the input literature (S504). That is, the reference suitability determination system divides the components in the reference into titles, authors, subjects, sources, etc., and stores or updates them in the reference database as shown in FIG. 7 and associates them with the input documents.

Referring to FIG. 7, since references corresponding to 1, 3, and 4 correspond to references of the input literature among the references stored in the literature database, reference literature corresponding to the

references

1, 3, and 4 may be referred to. In conjunction with

Then, the reference suitability determination system extracts one or more subject words from at least one of the bibliographic information, summary, and original text of the input document and the reference document and assigns them to the document (S506).

That is, the reference suitability determination system extracts terms except for stop words from at least one of bibliographic information, a summary, and an original text of the input reference and the reference.

The reference conformity determination system then filters the subject words by comparing the extracted index terms with a previously refined subject dictionary. Thereafter, the reference conformity determination system determines the uppermost main word as the final main word using the filtered condition of the frequency of the main word and assigns each to the corresponding document.

After performing S506, the reference suitability determination system compares the main word assigned to the input document with the main word assigned to the reference method in a matching manner (S508). That is, the reference suitability determination system collects and sorts the main words of all the references and calculates them in order of frequency.

For example, when the main word of the entire reference document is calculated in order of frequency, it is as shown in FIG. 8. That is, throughout the references, A comes from three references, B and D come from two references, and C, E, G, K, P, Q, R, and S come from one reference each. Able to know.

Then, the reference suitability determination system obtains at least one of a matching ratio and a frequency-based matching weight by matching the main word calculated in the frequency order with the main word of the input document. At least one of the obtained matching ratio and frequency-based matching weight is called a matching result.

Then, the reference suitability determination system compares the matching result obtained through the step S508 with a threshold value and determines the association between the input document and the reference (S510).

Referring to FIG. 9, the reference suitability determination system compares a matching result, which is at least one of a matching ratio and a frequency-based matching weight, obtained through a main control comparison result with a threshold value (S900) to determine whether the matching result is greater than or equal to a threshold value. It is determined whether or not (S902).

When the matching result is greater than or equal to the threshold result of the determination in S902, the reference suitability determination system determines that the entire reference is a proper citation (S904).

In this case, the reference suitability determination system determines that the input literature has dealt with a progressive research topic with respect to the entire reference when the matching result is greater than or equal to a threshold value and a main word not present in the reference is present. do.

If, as a result of the determination in S902, the matching result is not more than a threshold value, the reference suitability determination system determines that the entire reference is an inappropriate citation (S906).

Then, the reference conformity determination system searches for the existence of another document including the main word of the input document in the previously registered literature database (S908), and determines at least one or more of plagiarism, omission, and suitability based on the search result. (S910).

Through the above process, by identifying the relationship between input literature and reference literature, it is possible to support the determination of 'citation of suitable references', which is one of the criteria for reviewing thesis, and to determine the subjects of the duplicate article inspection.

As such, those skilled in the art will appreciate that the present invention can be implemented in other specific forms without changing the technical spirit or essential features thereof. Therefore, the above-described embodiments are to be understood as illustrative in all respects and not as restrictive. The scope of the present invention is shown by the following claims rather than the above description, and all changes or modifications derived from the meaning and scope of the claims and their equivalents should be construed as being included in the scope of the present invention. do.

As described above, the system and method for determining the suitability of a bibliography according to the present invention supports the determination of 'citation of a suitable bibliography' which is one of the criteria for reviewing a paper by analyzing the bibliography and identifying the association with the input literature. It is suitable for the high necessity to determine.

Claims

A subject comparison module that compares one or more subject words assigned to the input document with one or more subject words assigned to the one or more references in the input document in a matching manner; And

When the matching result obtained from the comparison result in the main keyword comparing module is greater than or equal to a threshold, the entire reference is referred to an appropriate citation, and when it is less than a threshold, the reference is referred to as an inappropriate citation. An association determination module that determines an association between the associations;

Reference suitability determination system comprising a.
The method of claim 1,

A bibliography recognition module for parsing the input literature to determine a bibliography region, and extracting each bibliography in the determined bibliography region;

A bibliography association module that separates components in the bibliography and stores or updates them in a bibliographic database and associates them with the input bibliography; and

And a main word extracting module for extracting one or more main words from at least one of the bibliographic information, the summary, and the original text of the input document and the reference document and assigning them to the corresponding document.
The method of claim 2,

The main word extraction module,

An index word extraction unit that extracts terms except for stop words from at least one of bibliographic information, a summary, and an original text of the input document and the reference document;

A subject filtering unit configured to filter the subject by comparing the extracted index terms with a refined subject dictionary; and

And a main word determiner which selects a main word ranked above using the filtered frequency condition of the main word.
The method of claim 3,

And the frequency condition is at least one of a term frequency condition, a document frequency condition, and a field classification code frequency condition.
The method of claim 1,

The main keyword comparison module,

A statistical calculation unit for collecting and sorting the main words of the reference, and calculating the frequency in order of frequency;

And a main word comparator for matching at least one of a main word calculated by the statistical calculator and a main word of the input document to obtain at least one of a matching ratio and a frequency-based matching weight.
The method of claim 1,

If the matching result obtained from the comparison of the main words is greater than or equal to a threshold value and there is a main word in the input document that is not present in the entire reference, the association determination module may allow the input document to deal with the progressive research topics of the entire reference. A reference conformity determination system, characterized in that it judges that it is.
The method of claim 1,

If the matching result obtained from the comparison of the main words is less than the threshold value, the correlation determination module searches whether there is another document including the main word of the input document in the previously registered literature database, and based on the search result, plagiarism and omission And determining at least one or more of the conformance.
The method of claim 1,

The main word comparison module compares a main word of the input document with a main word of each reference to obtain at least one of a matching ratio for each reference and a frequency-based matching weight,

And the association determination module determines that the reference that the obtained matching result is equal to or greater than a threshold value is a proper citation, and determines that the reference that is less than the threshold value is an invalid citation.
In the method of determining the suitability of a bibliographic reference system,

(a) comparing, in a matching manner, one or more subject words assigned to the input document with one or more subject words assigned to the one or more references in the input document; and

(b) the correlation between the input document and the reference document, when the matching result obtained through the comparison result is compared with a threshold value, and if the reference value is greater than or equal to the threshold value, the entire reference is an appropriate citation; Determining;

Reference suitability determination method comprising a.
The method of claim 9,

Before step (a) above,

Parsing the input literature to determine a bibliographic region and extracting each bibliography in the determined bibliographic region;

Separating and storing components in the bibliography in a bibliographic database and associating with the input bibliography; and

And extracting at least one main word from at least one of the bibliographic information, the summary, and the original text of the input document and the reference document and assigning it to the document.
The method of claim 9,

In step (a),

Collecting and sorting key words of all the references;

Obtaining at least one of a matching ratio and a frequency-based matching weight by matching the main word calculated in the frequency order with the main word of the input document.
The method according to claim 9 or 11,

In the step (b), if at least one of a matching ratio and a frequency-based matching weight obtained through the main word comparison result is greater than or equal to a threshold, the entire reference is determined to be an appropriate citation, and if less than the threshold A method for determining the suitability of a reference, characterized in that the entire reference is determined to be an inappropriate citation.
The method of claim 12,

If the matching result obtained from the comparison of the main words is greater than or equal to a threshold value and a main word not present in the reference is present in the input document, it is determined that the input document deals with a progressive research topic with respect to the reference. Reference method suitability determination method.
The method of claim 12,

When the matching result obtained through the comparison of the main words is less than the threshold value, a search is made in the existing registered database to see if there is another document including the main word of the input document, and at least one of plagiarism, omission, and goodness based on the search result. Reference method suitability determination method characterized by determining the above.
The method of claim 9,

In the step (a), comparing the main word of the reference with the main word of each reference to obtain at least one of the reference ratio matching reference, frequency-based matching weights.
The method of claim 15,

And determining that the reference that the obtained matching result is greater than or equal to a threshold is a good citation, and that the reference that is less than or equal to a threshold is an inappropriate citation.