KR20020009077A - Method of searching for piracy and steal on a piece of writing - Google Patents

Abstract

PURPOSE: A sentence plagiarism search method is provided to extract sentence patterns or code patterns included in a document, and to automatically search for plagiarized sentences by comparing the sentence or code patterns included in the documents. CONSTITUTION: The method comprises steps of searching for new web documents by using a robot agent(201), extracting morphologic characteristic vectors from new searched web documents(202), making a database from the extracted morphologic characteristic vectors(203), enabling a remote user to send a query of requesting a plagiarism on a specific sentence by using a query input interface connected to an application server(204), the application server extracting characteristic vectors from the input sentence and checking a similarity by comparing with the characteristic vectors stored in the database(205), and the server transmitting the compared result to the user over a network.

Description

Method of searching for piracy and steal on a piece of writing}

The present invention relates to a method of searching for plagiarism (or theft) of documents and various source codes on a web, and to a computer readable recording medium recording a program for realizing the method. Extracting the features of a pattern or code pattern to determine plagiarism between documents on the web and automatically searching for plagiarism documents.

In recent years, as the use of data on the Internet is rapidly increased and electronic publishing is active, the need for copyright protection of various documents and program codes such as technology and industry is gradually increasing.

However, in the case of the conventional Internet search engine, there is no method of checking the uniqueness or plagiarism of documents by constructing a database based on a simple feature word or a classification of a specific subject. In order to identify illegal theft or plagiarism of an individual's creation, an individual had to find relevant sites and documents by using specific words or topics, and read the contents to determine plagiarism. In practice, this is almost impossible when the number of related documents increases, and it takes a considerable time and effort.

Therefore, in order to protect copyrights and prevent plagiarism and theft of documents and codes on the Internet, there is a way for users to identify the correspondence of contents by using the features of the two documents without having to read the contents of the documents. It is required.

The present invention devised to solve the problems described above, by using the feature elements between the two documents without the user to read the contents of the document by identifying the correspondence of the content, the document and code on the Internet It is an object of the present invention to provide a method of searching for plagiarism and theft to protect copyrights, preventing plagiarism and theft, and a computer-readable recording medium recording a program for realizing the method.

1 is an exemplary configuration of a plagiarism and theft search system to which the present invention is applied.

2 is a flowchart illustrating a sentence plagiarism and theft search method according to the present invention;

3A is an exemplary database configuration used in the present invention.

Figure 3b is a diagram illustrating a query configuration used in the present invention.

Figure 4 illustrates a user interface for sentence plagiarism and theft search service used in the present invention.

5 is an exemplary diagram of feature vector extraction of sentences used in the present invention.

* Explanation of symbols for the main parts of the drawings

11: ViRobot 12: User Interface

13: application server 14: database

According to an aspect of the present invention, there is provided a method for checking plagiarism / theft of various documents and codes on a network including the Internet, the method comprising: a first step of extracting a first feature vector of a sentence of the document; A second step of constructing a database using the extracted first feature vectors; A third step of extracting a second feature vector of the query-input document or sentence upon inputting a plagiarism / theft query from a remote user via a network; And a fourth step of determining whether plagiarism / theft is performed by checking similarity with the first feature vector previously stored in the database by using the second feature vector.

The present invention may further comprise a fifth step of informing the user of the plagiarism / theft determination result through the network according to the determination result of the fourth step.

The present invention may further include a sixth step of informing a user through a network of the plagiarism / theft search result along with additional information stored in the database according to the fourth step search result.

In order to check the plagiarism / theft of various documents and codes on the network for achieving the above object, a document (sentence) plagiarism / theft search system having a processor extracts a first feature vector for a sentence of the document. 1 function; A second function of constructing a database using the extracted first feature vectors; A third function of extracting a second feature vector of the query-input document or sentence upon inputting a plagiarism / theft query from a remote user via a network; And a computer readable recording medium having recorded thereon a program for realizing a fourth function of judging plagiarism / theft by checking similarity with the first feature vector previously stored in the database using the second feature vector. to provide.

The present invention also provides a computer-readable recording medium having recorded thereon a program for further realizing a fifth function of informing a user over a network of plagiarism / theft determination results in accordance with the determination result of the fourth function.

In addition, the present invention can be read by a computer recording a program for further realizing a fifth function of notifying a user via a network along with additional information stored in the database according to the search result of the fourth function. Provide a recording medium.

The present invention retrieves the contents of various documents used on the web, extracts the feature elements of each sentence in the document, constructs a feature database of the document, and enters a feature database when a query comes in about the plagiarism of the user's original text. To search for documents and plagiarism documents with the same feature elements.

The present invention discloses a method for searching for plagiarism and theft of documents and various source codes on the web. This is applicable to the contents of HyperText Mark-up Language (HTML) documents and various documents registered on the web.

To this end, the present invention searches the documents including the HTML of the existing Internet using the Internet search robot, detects the feature elements of the original text for each document, and create and store it as a database (DB). If a user is in doubt about the plagiarism of a document he or she has written, if a user makes a query using his or her document on the Internet, the existing DB is searched for the feature elements of the queried document, and the result of plagiarism is checked. Shows. At this time, extraction of feature elements for plagiarism search extracts feature elements for each document by using a character extraction algorithm using meanings and syntactic relations between words in a sentence. In addition, as a retrieval method, a full comparison and a partial comparison are possible using existing approximation matching algorithm.

According to the present invention, plagiarism between documents on a web can be determined by extracting a feature of a character pattern or a code pattern included in a document, and a plagiarism document can be automatically searched.

The above objects, features and advantages will become more apparent from the following detailed description taken in conjunction with the accompanying drawings. Hereinafter, exemplary embodiments of the present invention will be described in detail with reference to the accompanying drawings.

1 is an exemplary configuration of a plagiarism and theft search system to which the present invention is applied.

As shown in FIG. 1, the plagiarism and theft search system for searching for the plagiarism and theft for documents and various source codes on the web to which the present invention is applied uses the ViRobot 11 which is a web search robot. Continue searching to find new documents, and the application server 13 extracts a feature vector using the morphological or morphological features of the text from the contents of the found new document and stores it in the database 14 together with the additional information of the document. . If a user is suspicious of plagiarism for a document he or she has written, if the user queries the application server 13 using his or her document through the user interface 12, the application server 13 sends a query to the queried document. The existing database 14 is searched for the features of the search results, and the results of plagiarism are shown. At this time, the feature element for plagiarism extraction is extracted by using a text watermarking algorithm using the interrelationship between letters. In addition, as a retrieval method, a full comparison and a partial comparison are possible using a conventional character matching algorithm.

The plagiarism and theft search system for searching for plagiarism and theft of documents and various source codes on the web to which the present invention is applied is obtained by using the ViRobot 11 to retrieve the documents on the web, and the application server 13 makes a statement in each document. The feature vector of the word shape is extracted and stored in the database 14 for each document. Based on the database 14 stored in this way, when a user inquires directly about a document of his creation which is suspected of being stolen or plagiarized by using a document, the application server 13 extracts a feature vector from the document of the creation and generates the database. The plagiarism and theft are searched against the feature vectors previously stored in (14).

As such, in order to extract the feature vector for each sentence in the document, in the present invention, as shown in FIG. 3A, a spelling or character that can form a morphological feature for each key word of the sentence for each sentence in the document Extract and construct the feature vector for the sentence. This feature vector is composed of one table, such as document additional information (document position, date, subject, etc.), and this table is composed of a database 14. That is, by using a robot having the same function as the ViRobot 11 of the current Internet search system, the document on the Internet is searched and the feature vector is extracted for each document and stored in the database 14.

Then, the user query interface and query service for retrieving data in the database 14 by extracting feature vectors from the user's data when the user remotely queries the desired data over a general network including the Internet. (See FIG. 3B). At this time, an algorithm for quickly comparing the feature vector of the sentence or document entered into the query with the feature vector of the existing database is required, and the feature vectors can be compared with each other on the vector and each part. Then, the comparison result of the feature vector is transmitted to the remote user together with additional information stored in the database 14 together with the plagiarism of the data.

Now, the plagiarism and theft search method using the form features of the document sentences according to the present invention will be described in more detail.

2 is a flowchart illustrating a plagiarism and theft search method according to the present invention.

As shown in Fig. 2, the method for searching for plagiarism and theft according to the present invention uses the ViRobot 11 to continuously search for documents on the web to search for plagiarism or theft of documents on a network including the Internet. In step 201, a morphological or morphological feature vector of a sentence is extracted from the content of a new document found at this time (202), and the database 14 is constructed using the extracted feature vector (203).

Subsequently, when a remote user queries plagiarism or theft for a specific document or sentence through a query input interface (see FIG. 4) connected to the application server 13 through a network (204), the application server 13 sends a query to the query. A feature vector is extracted from the input document or sentence, and similarity with the feature vector stored in the database 14 is checked using the extracted feature vector (205). The search result is then transmitted to the user through the network.

In the construction process 203 of the database 14 which is performed offline or online, the viRobot 11 searching the Internet continues to search the document on the web to find new documents (201), and the sentence in the content of the found new document. A feature vector is extracted using the morphological or morphological features of 202 and stored in the database 14 together with additional information of the document (203).

The feature vector of the document can be constructed using the morphological features of the key words of each sentence in the document. For example, feature vectors are constructed using elements of various word forms such as consonants, vowel or consonant overlap, and vowel overlap of key words except articles, prepositions, etc. As shown in FIG. 5, an example of configuring a feature vector using the first consonants of a key word is shown.

The feature vector extracted from the document is combined with various additional information as shown in FIG. 5. At this time, one record is formed together with information such as a location on the web, the author, the date and time of production, a keyword, and the like, and the record is stored in the database 14.

In the interface for a remote user, a user can directly query a text document of his own document to query his creation, and can query a specific sentence element of the document. You can also substitute the query text by inputting the internet address of the web document you created. At this time, the feature vector is constructed with the same algorithm as the algorithm used when constructing the database 14 for the document or sentence entered into the query text.

The feature vector generated at the remote user interface is compared with the feature vector of the database 14. At this time, the similarity between the query feature vector and the feature vectors stored in the database 14 is calculated for all documents registered in the database 14 to determine whether each feature vector matches.

In order to extract the feature vector, it is necessary to grasp the morphological and syntactic features of the words constituting the sentence. Identify the type of language from the document and select a stemmer for the language to process the sentence. This breaks up a sentence into several unit morphemes. From these, feature vector strings composed of meaningful words are generated. A feature vector string consisting of only the first letters of a word is generated. The most significant in a sentence is a noun, so a feature vector consisting of the first letter of the noun is created. If there are 100 sentences in the document, 100 feature vector strings are created, and each individual string has a different length.

In Japanese and Chinese, there is no spacing in sentences, so you can't extract meaningful nouns from sentences by simple mechanical methods, but you must use morphemes. You do not have to do. In the case of Japanese and Chinese morphological analysis, only the first letter is extracted from the extracted nouns to create a feature vector, but if the morphological analysis is not performed, only the first letter of the meaningful word is extracted. At this time, it is difficult to extract meaningful words because the syntax information of the sentence is not known because the morphological analysis is not performed. However, the smaller the number of letters is, the more likely it is to be meaningless letters or stopwords such as surveys and prepositions. Extract only the first letter from a word. However, in the case of English, a word of 4 letters or more is used. In the case of Hangul, unlike English, several morphemes combine to form a single word, but there is no problem because a meaningless word, such as survey or ending, follows a word.

Now, a feature vector extracted from a query sentence or a query document is Q and S is a feature vector extracted from a document to be compared to check plagiarism.

DB: An improved technique for 3D head tracking under varying illumination conditions is proposed. The head is modeled as a texture mapped cylinder. ...

Query: The head was modeled as a contour mapped cylinder.

S: ithtuvicp hmtmc ...

Q: hmcmc

A string is constructed for a statement from documents in the DB. The query string is similar to, but not identical to, the string in the DB. In this case, the second string 'hmtmc' in S and the string 'hmcmc' in Q differ only in the middle character. In this case, the matching method is only exact matching, not exact matching or partial matching. In determining whether a match is made through the similarity between the feature vectors, it is determined whether plagiarism is performed using an approximation matching algorithm.

The reason for using approximate matching, not exact matching or partial matching, is that while plagiarizing the sentences without altering them, they often change some words or tone. As an approximation matching algorithm, there is an approximation matching method based on n-gram, which is applied to the comparison of two feature vectors.

The similarity measure between two strings uses the similarity coefficient formula.The important formulas are Dice's coefficient, Jaccard's coefficient, Cosine coefficient, Overlap coefficient, and Tanimoto coefficient. There is a (Tanimoto coefficient), and there is another Angel's coefficient.

Equation 1 below is a relatively widely used formula compared to other coefficients and was used for n-gram based approximation matching of Equation 2.

where, g _c = common grams of Q and S,

g _Q , g _S = Q, grams of S

Here, the gram is a unit obtained by dividing the feature vector string into certain size units. There may be several methods of segmentation, but it is usually divided into 1, 2, 3 letter units and may or may not overlap.

Plagiarism and theft is determined by Equation 3 below (algorithm for plagiarism and theft).

Sim (Q, S)> T

In Equation (3), "Sim (Q, S)" represents the similarity between the query vector and one record of the database 14, and "T" represents a reference value, respectively.

As described above, the present invention can extract the features of the pattern of the sentence or the code pattern included in the document to determine the plagiarism between the documents on the web, and automatically retrieve the plagiarism document. At this time, if the user selects the search result with the mouse, the user interface unit 12 may directly connect to the web document suspected of plagiarism or theft. In addition, the user may add or search a specific topic by limiting or expanding the range of the search result.

The present invention described above is capable of various substitutions, modifications, and changes without departing from the spirit of the present invention for those skilled in the art to which the present invention pertains, and the above-described embodiments and accompanying It is not limited to the drawing.

As described above, the present invention enables a more accurate and quick search for unauthorized theft on the Internet, thereby ensuring the protection of copyrights on electronic publications on the web, and directly searching for theft and plagiarism of documents on the Web. It is easy to determine whether plagiarism with the document on the web through simple comparison of document feature vectors without having to read the contents, and also the effect of pre-registration on the copyright of existing electronic publications through database of feature vectors. It can also act as a precaution against plagiarism attempts and illegal theft, preventing unauthorized theft of academic, commercial and technical press documents.

Claims (12)

In a method for checking plagiarism / theft of various documents and codes on a network including the Internet, Extracting a first feature vector of the sentence of the document; A second step of constructing a database using the extracted first feature vectors; A third step of extracting a second feature vector of the query-input document or sentence upon inputting a plagiarism / theft query from a remote user via a network; And A fourth step of determining whether plagiarism / theft is performed by examining similarity with the first feature vector previously stored in the database by using the second feature vector. Sentence plagiarism and theft search method comprising a. The method of claim 1, A fifth step of informing the user of the plagiarism / theft determination result through the network according to the determination result of the fourth step Sentence plagiarism and theft search method further comprising a. The method of claim 1, A fifth step of informing a user via a network of the plagiarism / theft search result along with additional information stored in the database according to the search result of the fourth step Sentence plagiarism and theft search method further comprising a. The method according to any one of claims 1 to 3, The first and second feature vectors, Sentence plagiarism and theft retrieval method characterized by using the morphological or morphological features of the sentence. The method of claim 4, wherein The fourth step, Determining whether plagiarism / theft is performed by checking the correspondence with the first feature vector previously stored in the database using the second feature vector. Sentence plagiarism and theft search method characterized in that each part can be compared. The method of claim 4, wherein Upon inputting a plagiarism / theft query from a remote user via the network in the third step, Substantially, the sentence plagiarism and theft search method characterized in that the Internet address of the document or sentence, the web document is input. The method of claim 4, wherein The process of checking the consistency (similarity) of the fourth step, Sentence plagiarism and theft retrieval method characterized in that for measuring the similarity (Sim (Q, S)) of the first and second feature vectors, as shown in the following equation. Sim (Q, S) = (However, g _c = common grams of Q and S, g _Q , g _S = Q, grams of S, and grams are units of feature vector strings divided into units of a certain size.) The method of claim 4, wherein The process of determining whether the fourth step is plagiarism / theft, Plagiarism and theft retrieval method characterized in that it is determined whether the plagiarism / theft by the following equation. Sim (Q, S)> T (where, similarity between the second feature vector and a record in the database, T is a reference value) The method of claim 3, wherein The additional information, Substantially, sentence plagiarism and theft retrieval method comprising the location of the document, author, date and time of production, subject (keyword) information. In order to check plagiarism / theft of various documents and codes on a network, a document (text) retrieval system equipped with a processor, A first function of extracting a first feature vector for a sentence of the document; A second function of constructing a database using the extracted first feature vectors; A third function of extracting a second feature vector of the query-input document or sentence upon inputting a plagiarism / theft query from a remote user via a network; And A fourth function of determining whether plagiarism / theft is performed by checking similarity with the first feature vector previously stored in the database using the second feature vector A computer-readable recording medium having recorded thereon a program for realizing this. The method of claim 10, A fifth function of informing the user of the plagiarism / theft determination result through the network according to the determination result of the fourth function A computer-readable recording medium that records a program for further realization. The method of claim 10, A fifth function of notifying a user through a network of plagiarism / theft search results with additional information stored in the database according to the search result of the fourth function A computer-readable recording medium that records a program for further realization.