KR102337543B1 - Method and apparatus for normalization of modified post for copyright protection - Google Patents

Abstract

The present invention relates to a method and apparatus for normalizing modified postings for copyright protection, and more particularly, to a method and apparatus for performing restoration and normalization on all modified content postings to block illegal contents for copyright protection will be.
According to the present invention, in order to block illegal content, restoration and normalization of all modified content postings are performed, thereby not only blocking search terms for illegal content search, but also between copyrighted content and non-copyrighted content A method and apparatus for protecting copyright by comparing the similarity of

Description

Method and apparatus for normalization of modified post for copyright protection

The present invention relates to a method and apparatus for normalizing modified postings for copyright protection, and more particularly, to a method and apparatus for performing restoration and normalization on all modified content postings to block illegal contents for copyright protection will be.

Recently, with the development of Internet technology and the active use of personal broadcasting services, illegal sharing of copyrighted works through web hard or P2P sites has become a problem. The above services provide convenience to users, but they must be provided legally while protecting copyrighted information.

Although technologies for copyright protection have developed a lot, there are still limitations to protecting all copyrights. none.

In the past, technology trends for copyright protection used Digital Right Management (DRM) or Digital Watermarking. DRM is a technology that can control the number of installations, form, period of use, transfer right, etc. depending on the user's content purchase method, while at the same time encrypting the content and providing the user with an encryption key to decrypt the content Alternatively, there is a disadvantage that the copyright cannot be protected if a slight modification is applied. Digital watermarking refers to a technology that inserts and manages confidential information such as copyright information in various digital information such as photos and videos. Although this can protect digital data from being illegally distributed, there is a limitation in that it is impossible to know who, how, where, and how illegally created and distributed digital data.

Recently, useful technologies for copyright protection include title filtering, string comparison method, specific type file filtering, hash value comparison filtering, audio/video recognition filtering, and the like. Most of these are methods that block when a user enters a set search keyword, and block access to copyrighted information in advance for copyright protection. In this method, the search word entered by the user and the original content keyword must match, and in order to circumvent this, illegal content distributors tend to use modified keywords.

As described above, as a method used to circumvent the techniques for copyright protection, special characters are inserted, or in the case of Korean, consonants and vowels are transformed into similar English characters (h, I, o, l, etc.). . In addition, the initial, middle, and final consonants are separated (e, o, etc.) It is possible, but there are various methods that make it difficult for the computer to recognize it.

Accordingly, if the bypass method applied to copyrighted content information as described above can be restored and normalized to block it, it is possible to minimize illegally shared content and protect the copyright of the content.

KR 10-0889728 B1

"Normalization Technique of Modified Korean SMS Text for Filtering Spam Text (Journal of Information Processing Society) / Seungsik Kang / Vol.3, No.7 pp.271~pp276"

The present invention was created to solve such a problem, and in order to block illegal content, restoration and normalization of all modified content postings are performed, and through this, not only blocking of search terms for illegal content search, but also copyright An object of the present invention is to provide a method and apparatus for protecting copyright by comparing the similarity between content with and without copyright.

In order to achieve the above object, the method for normalizing a modified posting by the apparatus for normalizing a modified posting for copyright protection according to the present invention is a method for normalizing a modified posting for copyright protection, (a) content data posted from a file sharing site on the Internet collecting and extracting posts for each content therefrom; (b) removing special characters from the extracted posts; (c) converting the number transformed into English characters from the posting into the corresponding number; (d2) converting one English character, number, or special character into one Hangul phoneme using a Korean phoneme conversion pattern database; (d4) determining whether the Korean phoneme converted in the step (d2) is a combination of the first letter or the rear letter and the Korean phoneme; and (d41) when the Korean phoneme converted in step (d2) cannot form a Hangul syllable because the phoneme combination is not made with any of the preceding and following characters, the conversion in step (d2) is determined as an error and inversely converting the Hangul phoneme converted in step (d2) into the original character before conversion.
The removal of the special character may include leaving only one of the consecutively repeatedly inserted 'space' characters and removing the rest.
In the step (c), in the posting, an uppercase English letter 'O' or a lowercase English letter 'o' may be converted into the number '0', and an English capital letter 'I' may be converted into the number '1'.
In the conversion to the number, when the letter immediately preceding the uppercase letter 'O' or lowercase English letter 'o' or uppercase English letter 'I' is a number, conversion to the number '0' or the number '1' may be performed. .
When the letter immediately preceding the uppercase English letter 'O' or the lowercase English letter 'o' or the uppercase English letter 'I' is not a number, conversion to a number may be performed in word units.
Between the steps (c) and (d2), (c1) may further include the step of removing a space due to 'space' in the post separated by word units.
Before the step (d2), (d1) may further include the step of converting a character surrounded by a circle or other symbols so that only the corresponding character is displayed.
After the step (d2), (d3) using the Hangul phoneme combination pattern database, it may further include the step of converting the separated and successively described Hangul consonant phoneme and Hangul vowel phoneme by combining them into one Hangul character.
After the step (d41), (d5) the conversion pattern stored in the conversion pattern database (hereinafter referred to as the 'composite pattern database') for the part that has not been converted through the process from the post to step (d41) is converted using the conversion pattern It may further include the step of performing.
According to another aspect of the present invention, for copyright protection, a computer program for normalizing a modified posting is stored in a non-transitory storage medium, and by the processor, (a) content data posted from a file sharing site on the Internet collecting and extracting posts for each content therefrom; (b) removing special characters from the extracted posts; (c) converting the number transformed into English characters from the posting into the corresponding number; (d2) converting one English character, number, or special character into one Hangul phoneme using a Korean phoneme conversion pattern database; (d4) determining whether the Korean phoneme converted in the step (d2) is a combination of the first letter or the rear letter and the Korean phoneme; and (d41) when the Korean phoneme converted in step (d2) cannot form a Hangul syllable because the phoneme combination is not made with any of the preceding and following characters, the conversion in step (d2) is determined as an error and a command for inversely converting the Hangul phoneme converted in step (d2) into the original character before conversion is executed.
According to another aspect of the present invention, for copyright protection, an apparatus for normalizing a modified posting includes: at least one processor; and at least one memory storing computer-executable instructions, wherein the computer-executable instructions stored in the at least one memory are executed by the at least one processor (a) from a file sharing site on the Internet; collecting published content data and extracting posts for each content therefrom; (b) removing special characters from the extracted posts; (c) converting the number transformed into English characters from the posting into the corresponding number; (d2) converting one English character, number, or special character into one Hangul phoneme using a Korean phoneme conversion pattern database; (d4) determining whether the Korean phoneme converted in the step (d2) is a combination of the first letter or the rear letter and the Korean phoneme; and (d41) when the Korean phoneme converted in step (d2) cannot form a Hangul syllable because the phoneme combination is not made with any of the preceding and following characters, the conversion in step (d2) is determined as an error and inversely converting the Hangul phoneme converted in step (d2) into the original character before conversion is executed.
According to another aspect of the present invention, for copyright protection, the device for normalizing modified postings collects posted content data from a file sharing site on the Internet and crawls the site to extract the postings for each content therefrom. crawling) part; A basic preprocessor that removes special characters from the extracted posts and converts numbers transformed into English characters into corresponding numbers; Hangul phoneme conversion pattern database for storing a pattern for converting one English letter, number, or special character into one Hangul phoneme; By using the conversion pattern stored in the Hangul phoneme conversion pattern database, one English letter, number, or special character is converted into one Hangul phoneme, and the converted Hangul phoneme is a combination of the preceding character or the rear character and the Korean phoneme. If the converted Hangul phoneme cannot form a Hangul syllable because a phoneme combination is not made with any of the preceding and following characters, it is determined that the conversion to the Hangul phoneme is an error, and the converted Hangul phoneme is converted It includes a pattern application transformation unit that performs a function of inverse transformation to the original character.
The modified posting normalization device for copyright protection may further include a original character pattern database storing a pattern for converting a character surrounded by a circle or other symbol so that only the corresponding character is displayed, and the pattern application conversion unit includes: Using the original character pattern database, it may further include a function of performing conversion to only characters by removing the circle, parentheses, or other symbols from the original characters, parentheses, or characters implied by other symbols.
The modified post normalization device for copyright protection may further include a Hangul phoneme combination pattern database that stores a pattern for converting a Hangul consonant phoneme and a Hangul vowel phoneme to be converted into a single Hangul character by combining the Hangul consonant phonemes and Hangul vowel phonemes separately and consecutively. The pattern application conversion unit may further include a function of converting a Hangul consonant phoneme and a Hangul vowel phoneme, which are separately and continuously described, into one Hangul character using the Hangul phoneme combination pattern database.
The modified post normalization device for copyright protection may further include a complex pattern database, wherein the complex pattern database is a conversion that is not stored in the original character pattern database, the Hangul phoneme conversion pattern database, and the Hangul phoneme combination pattern database. Only the patterns are stored, or some of the stored conversion patterns may be the same as conversion patterns stored in the original character pattern database, the Hangul phoneme conversion pattern database, or the Hangul phoneme combination pattern database, and the pattern application conversion unit, the complex pattern Using the database, it may further include a function of detecting a modified posting and performing normalization transformation.

delete

delete

delete

delete

delete

delete

delete

delete

delete

delete

delete

delete

delete

delete

delete

delete

delete

delete

delete

According to the present invention, in order to block illegal content, restoration and normalization of all modified content postings are performed, thereby not only blocking search terms for illegal content search, but also between copyrighted content and non-copyrighted content It has the effect of providing a method and apparatus for protecting copyright by comparing the similarity of

1 is a diagram showing a network configuration for performing normalization of a modified posting for copyright protection according to the present invention.
2 is a diagram showing an embodiment of P2P (web hard) crawling posting data including modified Korean sentences on a website.
3 is a view showing an embodiment of a state in which special characters and duplicate spaces are removed from the posting.
4 is a view showing an embodiment of a state in which alphanumeric conversion is performed in a post.
5 is a view showing an embodiment of changing the original text to the corresponding text when there is an original text in the posting.
6 is a view showing an embodiment of converting English letters, numbers, or special characters into Korean phonemes in a post.
7 is a view showing an embodiment of generating syllables by combining Hangul phonemes in a post.
8 is a view showing an embodiment of a state in which inverse conversion is performed on the converted Hangul phoneme in order to solve a problem that may appear in the Hangul phoneme conversion.
9 is a view showing an embodiment of a portion that is not normalized even through the normalization process through FIGS. 2 to 7 .
10 is a diagram showing an embodiment of a normalization process applying a complex pattern in a post.
11 is a view showing an embodiment of a posting on which normalization is performed by applying FIGS. 2 to 10;
12 is a flowchart illustrating a modified posting normalization method for copyright protection of FIGS. 2 to 10 performed.
13 is a diagram showing the configuration of an apparatus for performing a modified posting normalization method for copyright protection according to the present invention.

Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings. Prior to this, the terms or words used in the present specification and claims should not be construed as being limited to conventional or dictionary meanings, and the inventor should properly understand the concept of the term in order to best describe his invention. Based on the principle that it can be defined, it should be interpreted as meaning and concept consistent with the technical idea of the present invention. Accordingly, the embodiments described in this specification and the configurations shown in the drawings are only the most preferred embodiment of the present invention and do not represent all the technical spirit of the present invention, so at the time of the present application, various It should be understood that there may be equivalents and variations.

1 is a diagram showing a network configuration for performing normalization of a modified posting for copyright protection according to the present invention.

The P2P server 200 is a generic term for servers that provide various Internet file sharing sites such as P2P and web hard, and is referred to as 'P2P server 200'. Hereinafter, the terms used as 'P2P server' or 'P2P site' shall collectively refer to all servers or sites that provide file sharing services through the Internet.

Modified posting normalization apparatus 100 for copyright protection is a device for normalizing modified postings of the present invention, and its configuration will be described with reference to FIG. 13, and modified postings normalization performed will be described in FIGS. 2 to It will be described later in detail with reference to 12. Here, 'normalization' means converting a modified post into characters according to the original meaning of the post.

A large amount of illegal works that are illegally downloaded and distributed exist on such P2P sites. In the case of detecting such illegal works, detection without any pre-processing on the title of the illegal work file has the advantage that it can be carried out quickly without cost, but it can be bypassed through various methods such as title change and spacing Possible, there is a problem that the detection rate for illegal works is low. Accordingly, the modified posting normalization method provided by the modified posting normalization apparatus 100 for copyright protection of the present invention performs effective pre-processing on these postings, so that even postings with modified sentences can be detected as much as possible.

2 is a diagram illustrating an embodiment of P2P (web hard) crawling posting data including modified Korean sentences on a website.

The modified posting normalization apparatus 100 for copyright protection according to the present invention first performs crawling of fetching data shared in the P2P site 200 . 2 is an embodiment of data crawled and retrieved from various P2P sites on the network as described above. It is an object of the present invention to generate posting data capable of detecting illegal works by converting and normalizing the deformed parts into normal sentences in order to prevent detection in each title of such data.

3 is a view showing an embodiment of a state in which special characters and duplicate spaces are removed from postings.

In illegal postings, there are cases where symbols such as unnecessary special characters are inserted in the title or 'space' is used repeatedly in succession, so take measures to remove these symbols and duplicate spaces. Examples of special characters to be removed are as follows.

- Examples of symbols to be removed

[-=+_#}/\?;:{^$,.@*\"※~&%ㆍ!＋：]\\'|\(\)\[\]\<\>`\'… 》「◁◀▷▶▲□■」○◆⇒√☆♥★！→◐「」™●]

FIG. 3 shows a state in which the sentence on the right is generated in which symbols and unnecessary spaces are removed from the title of the illegal post on the left.

4 is a diagram showing an embodiment of a state in which alphanumeric conversion is performed in a post.

One of the examples frequently used in the titles of illegal posts is to avoid detection by replacing the number '0' with an English capital letter 'O' and the number '1' with an English capital letter 'I'. FIG. 4 detects this and converts the uppercase English letter 'O' into the number '0' and the English capital letter 'I' into the number '1' again. That is, it means converting alphabetic characters into numbers, and it will be referred to as alphanumeric conversion. In some cases, the uppercase English 'O' above is used as an English lowercase letter 'o', and the conversion is the same as above.

First, it is divided into word units based on a single space.

Then, for example, when an English capital letter 'O' is detected in one word, the preceding character is checked, and if the letter is a number, the corresponding English capital letter 'O' is converted to the number '0'. will be. The same applies to the method of converting the English capital letter 'I' to the number '1'. The title indicated by the upper arrow on the left of FIG. 4 shows an example in which the title has been converted as such. That is, '2O18' is converted to '2018', and the preceding number includes an English capital letter 'O', and the converted number includes the number '0'.

In the case of '2OI8', it cannot be detected by the above method. Because, after 2, an English capital letter 'O' and an English capital letter 'I' are arranged consecutively, 'I' is not converted to a number because the preceding character is an English capital letter. In this case, '2OI8' is viewed as a single word, and when this word is found, it is converted into '2018' consisting of only numbers. However, in this case, all words that are likely to be transformed as described above may be stored in the database, and transformation by a 'pattern' to be described later may be performed.

After such alphanumeric transformation, spaces are removed from sentences separated by word units to form sentences in which all syllables are connected.

5 is a diagram illustrating an embodiment of changing the original text to a corresponding text when there is an original text in the posting.

In other words, it is converted into a character by removing the surrounding circle from the original character. Here, only the 'original characters' have been described as examples, but parentheses characters (eg (1), 1), etc., or various types of characters containing characters as specific symbols are also included in the 'original characters' processed here , and can remove such specific symbols and convert only the nested characters. For such conversion of original characters, as shown in FIG. 5 , the original text pattern database 151 (refer to FIG. 13 ) in which patterns before and after conversion are matched may be used.

6 is a diagram illustrating an embodiment of converting English letters, numbers, or special characters into Korean phonemes in a post.

6 to 10 show an embodiment in which a specific character is converted into another specific character by using a pattern stored in the pattern database 152 to 154 (refer to FIG. 13).

In the case of FIG. 6, one alphabetic character, number, or special character similar to a Korean phoneme is converted into a Korean phoneme, and such a conversion pattern is stored in the Korean phoneme conversion pattern database 152 (refer to FIG. 13). 'Hangul phoneme' refers to individual vowels or consonants of Hangul.

That is, in the title of an illegal work, another method to avoid detection is to use an English letter or number in a similar form instead of a Korean phoneme. Referring to FIG. 6(a), for example, an 'r' ' or the number '1' in place of 'ㅣ'. In the present invention, using the Hangul phoneme conversion pattern database 152, it is restored to 'A' and 'ㅣ'. Fig. 6(b) shows an embodiment restored in this way.

7 is a diagram illustrating an embodiment of generating syllables by combining Hangul phonemes in a post.

In the case of FIG. 7, separated consonants and vowels are combined and converted into one Hangul character (syllable), and such a conversion pattern is stored in the Hangul phoneme combination pattern database 153 (refer to FIG. 13).

That is, as another method to avoid detection in the title of illegal works, a method of separating consonants and vowels, that is, phonemes, is used in one syllable. Referring to FIG. 7(a), for example, 'pa' is expressed as 'P'. The present invention restores it back to 'pa' using the Hangul phoneme combination pattern database 153 . The sentence in the rightmost column of FIG. 7(b) shows the example restored in this way.

8 is a diagram illustrating an embodiment of a state in which inverse transformation is performed on a converted Hangul phoneme in order to solve a problem that may appear in Hangul phoneme conversion.

In the above-described Hangul phoneme conversion process, a case may occur in which normal English characters or numbers are replaced with Hangul phonemes. For example, referring to FIG. 8 , there may be a case where 'Lost' in the post on the upper left is changed to 'Nost' on the upper right. That is, the English capital letter 'L' is replaced with the Korean phoneme 'ㄴ'. In this case, it is determined whether the letter following 'ㄴ' and the Korean phoneme are combined. In the case of FIG. 3, since a lowercase English letter 'o' appears after 'ㄴ', it is impossible to combine Korean phonemes. In this case, it is judged that the conversion to 'ㄴ' is an error, and 'ㄴ' is inversely converted back to an English capital letter 'L'. Such inverse conversion does not require a separate database, and may be converted in the opposite direction using the Hangul phoneme conversion pattern database 152 (refer to FIG. 13 ) described with reference to FIG. 6 .

As another embodiment, there may be a case in which 'r' in a post is converted to 'a' through a Hangul phoneme conversion process. Even in this case, if the preceding character and Hangul phoneme are combined, it is set as a combined Hangul syllable, and if it is not a Hangul syllable because the Hangul phoneme combination is not made, inverse conversion is performed again to 'r'.

9 is a diagram showing an embodiment of a portion that is not normalized even through the normalization process through FIGS. 2 to 7 , and FIG. 10 is a diagram showing an embodiment of a normalization process to which a complex pattern is applied in a post.

The arrows in FIG. 9 indicate such non-normalized portions. For example, in FIG. 9 , in the portion indicated by the upper arrow, 'or' composed of two lowercase English letters and 'oa' composed of lowercase English letters 'o' and a Korean phoneme 'A' are the same. 'oㅣ' in the middle arrow and 'ㄹI' in the bottom arrow indicate such a case.

As such, the pattern that cannot be detected by conversion using the Hangul phoneme conversion pattern database (152, see FIG. 13) and the Hangul phoneme combination pattern database (153, see FIG. 13) described in FIGS. 6 to 8 is a complex pattern database (154, 13) to perform transformation and normalization. 10 (a) is a diagram showing an embodiment of transformation using such a complex pattern database 154, and FIG. 10 (b) is before (left) and after (left) and after ( It is a diagram showing an embodiment of the right).

Examples of such a conversion include, if two English syllables are used to express a Korean syllable, separation of the last consonant as in expressing 'land' as 'ta', but is not limited thereto and various patterns are provided in the complex pattern database (154). ) may be stored in

The advantage of such a conversion using the complex pattern database 154 is, the above-mentioned original character pattern database 151, Hangul phoneme conversion pattern database 152 and Hangul phoneme combination pattern database 153 that is not filtered in the normalization using various Conversion patterns can be used, and furthermore, it is excellent in scalability because it continuously detects new transformation patterns in a non-existing method appearing on the P2P site, and can be easily updated and used in the complex pattern database 154. It has the advantage of being able to detect and normalize almost all of the modified postings.

11 is a diagram illustrating an embodiment of a post on which normalization is performed by applying FIGS. 2 to 10 .

The left side is the post data crawled on the P2P site, and the right side is the post normalized by applying FIGS. 2 to 10, respectively.

12 is a flowchart in which the modified posting normalization method for copyright protection of FIGS. 2 to 10 is performed, and FIG. 13 is a configuration of an apparatus 100 for performing the modified posting normalization method for copyright protection according to the present invention. It is a drawing showing

Since the modified posting normalization method for copyright protection of the present invention has already been described in detail with reference to FIGS. 2 to 10, the process will be briefly described with reference to FIG. 12, and with reference to FIG. 13, The apparatus 100 for performing the same modified posting normalization method will be briefly described focusing on its configuration.

The control unit 110 performs a series of processes for normalizing the modified posting by controlling each of the following components of the apparatus 100 for performing the modified posting normalization method for copyright protection.

The modified post normalization device 100 for copyright protection, the site crawling unit 120, first collects published content data from the file sharing site 200 on the Internet, and from this, the published posts for each content Crawling to extract is performed (S1201). With respect to the collected posting data, first, the basic preprocessor 130 removes various symbols such as special characters and redundant spaces (S1202), as described with reference to FIGS. 3 and 4, and alphanumeric conversion (S1203), blank removal (S1204), and the like are performed.

Thereafter, the pattern application conversion unit 140 converts and normalizes postings using various pattern databases 150 . That is, the pattern application conversion unit 140, first, using the original character pattern database 151, removes circles, parentheses, or other symbols from original characters, parentheses, or characters implied by various symbols, or converts them into characters only. is performed (S1205).

Subsequently, the pattern application conversion unit 140 performs Hangul phoneme conversion as described with reference to FIG. 6 using the Hangul phoneme conversion pattern database 152 (S1206), and uses the Hangul phoneme combination pattern database 153 The Hangul phoneme combination as described with reference to FIG. 7 is performed (S1207). In addition, the inverse Hangul phoneme conversion as described with reference to FIG. 8 is performed on the erroneously converted characters during Hangul phoneme conversion (S1208). In this case, the Hangul phoneme conversion pattern database 152 is used.

For various patterns that cannot be detected with the pattern as described above, the pattern application transformation unit 140 performs detection and normalization transformation of a modified posting using the complex pattern database 154 as described with reference to FIG. 10 . will perform In this complex pattern database 154, only the conversion patterns that are not included in the original character pattern database 151, the Hangul phoneme conversion pattern database 152, and the Hangul phoneme combination pattern database 153 described above may be stored, In some cases, some of the patterns stored in these pattern databases may be redundantly stored in the complex pattern database 154 .

Transformation using such a complex pattern database 154 continuously detects a new deformation pattern of an unprecedented method appearing on the P2P site 200, and can be easily updated and used in the complex pattern database 154, so that scalability is possible. It is excellent, and it has the characteristic that it is possible to detect and normalize almost all of the latest and modified postings.

100: Modified post normalization device for copyright protection
200: P2P server

Claims (20)

A modified posting normalization device for copyright protection, for copyright protection, as a method of normalizing a modified posting,
(a) collecting published content data from a file sharing site on the Internet and extracting a post for each content therefrom;
(b) removing special characters from the extracted posts;
(c) converting the number transformed into English characters from the posting into the corresponding number;
(d2) converting one English character, number, or special character into one Hangul phoneme using a Korean phoneme conversion pattern database;
(d4) determining whether the Korean phoneme converted in the step (d2) is a combination of the first letter or the rear letter and the Korean phoneme; and,
(d41) If the Korean phoneme converted in step (d2) cannot form a Hangul syllable because the phoneme combination is not made with any of the preceding and following characters, the conversion in step (d2) is determined as an error, Inversely converting the Hangul phoneme converted in step (d2) into the original character before conversion
A modified post normalization method for copyright protection, including The method according to claim 1,
To remove the special character,
Retaining only one of the consecutively inserted 'space' characters and removing the rest.
A modified post normalization method for copyright protection, characterized by The method according to claim 1,
The step (c) is, in the post,
Converts uppercase English 'O' or lowercase English 'o' to number '0',
Converting the uppercase letter 'I' to the number '1'
A modified post normalization method for copyright protection, characterized by 4. The method according to claim 3,
The conversion to this number is,
When the letter immediately preceding the uppercase English letter 'O' or the lowercase English letter 'o' or the uppercase English letter 'I' is a number, converting to the number '0' or the number '1'
A modified post normalization method for copyright protection, characterized by 5. The method according to claim 4,
When the letter immediately preceding the uppercase English letter 'O' or the lowercase English letter 'o' or the uppercase English letter 'I' is not a number, converting to a number in word units
A modified post normalization method for copyright protection, characterized by The method according to claim 1,
Between steps (c) and (d2),
(c1) Step of removing the space due to 'space' in the post separated by word unit
A modified post normalization method for copyright protection, characterized in that it further comprises. delete The method according to claim 1,
Before step (d2),
(d1) converting a character surrounded by a circle or other symbol so that only the character is displayed;
A modified post normalization method for copyright protection, characterized in that it further comprises.
delete The method according to claim 1,
After the step (d2),
(d3) using the Hangul phoneme combination pattern database to convert the separated and consecutively written Hangul consonant phonemes and Hangul vowel phonemes into one Hangul character
A modified post normalization method for copyright protection, characterized in that it further comprises.
delete delete The method according to claim 1,
After the step (d41),
(d5) performing conversion using the conversion pattern stored in the conversion pattern database (hereinafter referred to as 'composite pattern database') for the part that has not been converted even through the process from step (d41) in the above post
A modified post normalization method for copyright protection, characterized in that it further comprises. A computer program for normalizing a modified post for copyright protection,
It is stored in a non-transitory storage medium, and by the processor,
(a) collecting published content data from a file sharing site on the Internet and extracting a post for each content therefrom;
(b) removing special characters from the extracted posts;
(c) converting the number transformed into English characters from the posting into the corresponding number;
(d2) converting one English character, number, or special character into one Hangul phoneme using a Korean phoneme conversion pattern database;
(d4) determining whether the Korean phoneme converted in the step (d2) is a combination of the first letter or the rear letter and the Korean phoneme; and,
(d41) If the Korean phoneme converted in step (d2) cannot form a Hangul syllable because the phoneme combination is not made with any of the preceding and following characters, the conversion in step (d2) is determined as an error, Inversely converting the Hangul phoneme converted in step (d2) into the original character before conversion
A computer program stored in a non-transitory storage medium for normalizing a modified posting for copyright protection, including instructions to be executed. As a device for normalizing modified postings for copyright protection,
at least one processor; and
at least one memory for storing computer-executable instructions;
The computer-executable instructions stored in the at least one memory are executed by the at least one processor,
(a) collecting published content data from a file sharing site on the Internet and extracting a post for each content therefrom;
(b) removing special characters from the extracted posts;
(c) converting the number transformed into English characters from the posting into the corresponding number;
(d2) converting one English character, number, or special character into one Hangul phoneme using a Korean phoneme conversion pattern database;
(d4) determining whether the Korean phoneme converted in the step (d2) is a combination of the first letter or the rear letter and the Korean phoneme; and,
(d41) If the Korean phoneme converted in step (d2) cannot form a Hangul syllable because the phoneme combination is not made with any of the preceding and following characters, the conversion in step (d2) is determined as an error, Inversely converting the Hangul phoneme converted in step (d2) into the original character before conversion
A modified post normalization device for copyright protection that allows . As a device for normalizing modified postings for copyright protection,
a site crawling unit that collects published content data from a file sharing site on the Internet and extracts a post for each content therefrom;
A basic preprocessor that removes special characters from the extracted posts and converts numbers transformed into English characters into corresponding numbers;
Hangul phoneme conversion pattern database for storing a pattern for converting one English letter, number, or special character into one Hangul phoneme;
By using the conversion pattern stored in the Hangul phoneme conversion pattern database, one English letter, number, or special character is converted into one Hangul phoneme, and the converted Hangul phoneme is a combination of the preceding character or the rear character and the Korean phoneme. If the converted Hangul phoneme cannot form a Hangul syllable because a phoneme combination is not made with any of the preceding and following characters, it is determined that the conversion to the Hangul phoneme is an error, and the converted Hangul phoneme is converted A pattern application conversion unit that performs the function of inverse conversion to the original character
Modified post normalization device for copyright protection, including. 17. The method of claim 16,
a character pattern database for storing a pattern for converting characters surrounded by circles or other symbols to display only the characters;
The pattern application conversion unit,
Using the original character pattern database, further comprising a function of performing conversion to only characters by removing the circle, parentheses, or other symbols from the original characters, parentheses, or characters implied by other symbols
A modified post normalization device for copyright protection, characterized by
delete 17. The method of claim 16,
It further comprises a Hangul phoneme combination pattern database that stores a pattern for converting into one Hangul character by combining the Hangul consonant phoneme and Hangul vowel phoneme separately and consecutively,
The pattern application conversion unit,
Using the Hangul phoneme combination pattern database, it further comprises a function of converting into a single Hangul character by combining the Hangul consonant phoneme and Hangul vowel phoneme separately and consecutively
A modified post normalization device for copyright protection, characterized by 17. The method of claim 16,
Further comprising a complex pattern database,
The complex pattern database,
Only the conversion patterns not stored in the original character pattern database, the Hangul phoneme conversion pattern database and the Hangul phoneme combination pattern database are stored,
Or some of the stored conversion patterns are the same as conversion patterns stored in the original character pattern database, the Hangul phoneme conversion pattern database, or the Hangul phoneme combination pattern database,
The pattern application conversion unit,
Using the complex pattern database, further comprising a function of performing detection and normalization transformation of modified postings
A modified post normalization device for copyright protection, characterized by

Images