KR102244982B1 - Text filtering method and device using the image learning - Google Patents

Abstract

Disclosed are a method and device for generating an image for text, learning the image and classifying prohibited words included in the text, and converting into special characters. The text filtering method and device using image learning according to the present invention include an artificial intelligence module, a robot, an augmented reality (AR) device, a virtual reality (VR) device, and a device related to a 5G service and the like can be linked.

Description

Text filtering method and device using image learning {TEXT FILTERING METHOD AND DEVICE USING THE IMAGE LEARNING}

The present invention relates to a method and apparatus for generating an image for a text, learning the image, classifying the prohibited words included in the text, and converting them into special characters.

Recently, a large number of users can register texts including various contents online. These texts sometimes contain prohibited words such as profanity or profanity.

Conventionally, in order to detect prohibited words such as profanity contained in the text, a method of classifying the prohibited words from the text using a machine learning-based algorithm was mainly used.

However, there is a problem in that it is difficult to establish a database for all the modified prohibitions because the prohibitions are constantly transformed. In addition, existing databases for machine learning do not store a lot of data on banned words and modified banned words, so there was a limit to filtering the banned words included in the text through machine learning.

Moreover, when the banned words are composed of Hangul, there is a problem that the banned words separated into initial, neutral, and ending are not filtered by the conventional filtering method due to the characteristics of Hangul that can be classified and expressed as syllables or phonemes.

It is an object of the present invention to solve the aforementioned necessities and/or problems.

A text filtering method and apparatus using image learning according to the present invention provides a method of generating an image for text, classifying prohibited words by reading the corresponding image, and converting them into special characters.

In addition, the text filtering method and apparatus using image learning according to the present invention can filter forbidden words and modified forbidden words using image analysis and image deep learning, even for variants of prohibitions and prohibitions that are not stored or registered in an existing database. We propose a text filtering method.

In the text filtering method using image learning according to an embodiment of the present invention, the receiving unit receives the text input from the user terminal, the first identification unit first identifies whether the text contains a first banned word, Determining whether a second identification unit includes a first word or a first character that is not included in the first prohibition word among the text and is not included in the list of normal words stored in the database at the same time, the second identification Additional steps of generating a first image for the text, the second identification unit deep learning the first image to secondly identify whether there is a second banned word, and the first and second modification units included in the text 2 Including the step of replacing a banned word with a special character, wherein the first banned word includes banned words stored in the database and similar banned words, and the second banned word is banned words that are not stored in the database and their modified banned words. .

The step of secondly identifying whether there is a second prohibition word by deep learning by the second identification unit deep-learning the first image may include how similar the first word or the first character is to the first prohibition word or the second prohibition word. The method may further include determining a degree of similarity indicating a degree of similarity, and determining whether to regard the first character or the first character as the second prohibition word according to the degree of similarity.

The step of replacing the second prohibition word included in the text by the transforming unit with a special character may further include determining in which language the text is composed.

Determining which language the text is composed of may further include confirming that the text is composed of Hangul and applying a Hangul transformation algorithm to the second prohibited words included in the text.

The step of applying the Hangul transformation algorithm to the second banned word included in the text includes the steps of separating individual characters constituting the text into an initial, neutral, and final, respectively, and an initial, neutral, and final sentence constituting the second banned word. The method may further include randomly extracting special characters to be substituted for each, and replacing the initial, neutral, and final characters constituting the second banned word with the extracted special characters.

After the step of replacing the initial, neutral and ending characters constituting the second prohibition word with the special characters, assigning an order to individual characters constituting the text in order from 1 to n, from 1 to n The method may further include randomly selecting a plurality of specific orders among the times, randomly selecting special characters to be added in each specific order, and adding the selected special characters to the selected specific order.

The determining in which language the text is composed of may further include confirming that the text is composed of English, and applying an English transformation algorithm to the second prohibited words included in the text.

The step of applying the English transformation algorithm to the second prohibition included in the text includes: randomly extracting special characters to replace individual characters constituting the text, and special characters from which individual characters constituting the text are extracted It may further include the step of replacing with.

After the step of replacing the individual characters constituting the text with the extracted special characters, assigning an order to individual characters constituting the text in order from 1 to n times, a plurality of the first to n times The method may further include randomly selecting specific orders, randomly selecting special characters to be added in each specific order, and adding the selected special characters to the selected specific order.

After the step of replacing the second banned word included in the text by the transforming unit with a special character, generating a transformed word formed by being replaced with the special character or adding the special character, and storing the generated transformed word as a text file And storing the modified word as a second image file in the database.

A text filtering apparatus using image learning according to an embodiment of the present invention includes a receiving unit for receiving text from a user terminal, a database storing first prohibited words including profanity and profanity, and the first prohibited words from the database. A first identification unit that reads a list and identifies whether the text contains the first prohibition, prohibitions not stored in the database, and whether the second prohibition, which is a modified prohibition, is included in the text. A second identification unit to identify and a transformation unit for replacing the first and second prohibition words included in the text with special characters, and the second identification unit generates a first image by photographing the text, and The first image is deep-learned to determine whether the second prohibition word is included in the text.

The second identification unit further includes an image generation unit generating the first image and a learning unit deep learning the first image.

A text filtering method and apparatus using image learning according to the present invention filters a modified prohibition word based on a newly created prohibition list by deep learning an image of the text, thereby detecting a prohibition word using conventional machine learning. It can save time and effort spent compared to.

In addition, the text filtering method and apparatus using image learning according to the present invention can solve the shortage of prohibition data stored or registered in an existing database.

In addition, the text filtering method and apparatus using image learning according to the present invention reduce the time, effort, and cost required to collect the banned words newly created or modified from the existing banned words, so that it is difficult to obtain data. -start) problem can be solved.

1 is a block diagram of a text filtering apparatus using image learning according to an embodiment of the present invention.
2 is a flowchart illustrating a text filtering method using image learning according to an embodiment of the present invention.
3 is a flow chart illustrating a process of identifying whether or not a prohibited word is included in a text by a second identification unit according to an embodiment of the present invention.
4 is a flowchart illustrating a process of converting a banned word included in a text into a special character by a transforming unit according to an embodiment of the present invention.
5 is a flowchart illustrating a process of generating a modified word by the transforming unit 130 according to an embodiment of the present invention.
6 is a flowchart illustrating a condition in which the transforming unit 130 ends generation of a transformed word.

Hereinafter, exemplary embodiments disclosed in the present specification will be described in detail with reference to the accompanying drawings, but identical or similar elements are denoted by the same reference numerals regardless of reference numerals, and redundant descriptions thereof will be omitted. The suffixes "module" and "unit" for constituent elements used in the following description are given or used interchangeably in consideration of only the ease of preparation of the specification, and do not have meanings or roles that are distinguished from each other by themselves. In addition, in describing the embodiments disclosed in the present specification, when it is determined that a detailed description of related known technologies may obscure the subject matter of the embodiments disclosed in the present specification, the detailed description thereof will be omitted. In addition, the accompanying drawings are for easy understanding of the embodiments disclosed in the present specification, and the technical idea disclosed in the present specification is not limited by the accompanying drawings, and all changes included in the spirit and scope of the present invention It should be understood to include equivalents or substitutes.

Terms including ordinal numbers such as first and second may be used to describe various elements, but the elements are not limited by the terms. The terms are used only for the purpose of distinguishing one component from another component.

When a component is referred to as being "connected" or "connected" to another component, it is understood that it may be directly connected or connected to the other component, but other components may exist in the middle. It should be. On the other hand, when a component is referred to as being "directly connected" or "directly connected" to another component, it should be understood that there is no other component in the middle.

Singular expressions include plural expressions unless the context clearly indicates otherwise.

In this application, terms such as "comprises" or "have" are intended to designate the presence of features, numbers, steps, actions, components, parts, or combinations thereof described in the specification, but one or more other features. It is to be understood that the presence or addition of elements or numbers, steps, actions, components, parts, or combinations thereof does not preclude in advance.

Hereinafter, a text filtering apparatus using image learning according to an embodiment of the present invention will be described with reference to FIG. 1.

1 is a block diagram of a text filtering apparatus 10 using image learning according to an embodiment of the present invention.

The text filtering apparatus 10 using image learning according to an embodiment of the present invention includes a receiving unit 100, a first identification unit 110, a second identification unit 120, a transformation unit 130, and a database 140. ) Can be included.

The receiving unit 100 receives predetermined data from a user terminal. The data received here means text data that the user inputs using his/her user terminal.

The user terminal may include a communication terminal capable of performing a function of a computing device (not shown), and the user terminal illustrated in the present embodiment includes a desktop computer, a smartphone, a laptop computer, a tablet PC, and Smart TV, mobile phone, PDA (Personal Digital Assistant), laptop, media player, micro server, GPS (Global Positioning System) device, e-book terminal, digital broadcasting terminal, navigation, kiosk, MP3 player, digital camera, home appliance and others It may be a mobile or non-mobile computing device, but is not limited thereto.

In addition, the user terminal may be a wearable terminal such as a watch, glasses, hair band, and ring having a communication function and a data processing function. The user terminal is not limited to the above, and any terminal capable of web browsing may be included in the user terminal.

The first identification unit 110 receives the text input by the user from the reception unit 100 and identifies whether the text contains a first prohibition word.

Meanwhile, in the present embodiment, the forbidden words include words, characters, and profanity similar to swear words and swear words, and the first forbidden words mean forbidden words stored in the database 140 or a list of them. In addition, the first prohibited words may include prohibited words similar to those stored in the database 140 and a list thereof. In addition, the second prohibited words mean prohibited words that are not stored in the database 140. That is, the second forbidden word means abusive language, words or letters similar to swearwords, and swearwords in which profanity is modified, words or letters and profanity similar to swearwords previously stored in the database 140. In addition, the second forbidden word may mean a newly created profanity, a word similar to the profanity, or a letter, and a profanity. Accordingly, the second prohibited words are not stored in the database 140, but are words, characters, and profanities similar to swear words and swear words, and thus correspond to prohibited words.

The first identification unit 110 according to the present embodiment loads the first prohibition word and/or the first prohibition word list from the database 140, and the character or word corresponding to the first prohibition word is displayed in the text input by the user. Whether or not it is included can be identified.

Meanwhile, the first identification unit 110 may expand the range of the first prohibition word by uploading a word or character similar to the profanity included in the first prohibition word to the database 140.

The database 140 may store in advance a commonly known swearword, words, letters, and slang words similar to such swearwords. It has been described above that profanity stored in the database 140 in advance, words or characters similar to such profanity, and profanity may be classified as the first prohibition words. In addition, the database 140 may store normal words or characters that do not correspond to profanity. In this way, the database 140 may classify non-profanity, normal words or normal characters into a normal word list or a normal word list, and store the corresponding list and list in advance.

In addition, the database 140 according to the present embodiment may be configured as a database server that provides data related to text recognition and big data required to apply various artificial intelligence algorithms. In addition, when the database 140 is configured as a server, such a server may include an application installed in the user terminal or a web server or an application server that enables remote control of the user terminal using a web browser.

On the other hand, the second identification unit 120 identifies words or characters that are not included in the first banned word and are not included in the normal word list stored in the database 140, among words or characters constituting the text, It is possible to determine whether these words or letters correspond to words or letters similar to profanity, swear words, and swear words. For convenience of explanation, among words or characters constituting the text, words or characters that are not included in the first banned word and are not included in the list of normal words stored in the database 140 are referred to as first words or first characters. Can be referred to.

The second identification unit 120 is configured to capture or photograph text in order to determine whether the first word or the first character included in the text corresponds to the second prohibition. To this end, the image generation unit 121 ) May be further included.

The image generator 121 generates an image for the text by capturing or photographing the text input by the user through the user terminal. Here, an image generated by capturing or photographing the text input by the user may be classified as a first image.

In addition, when capturing or photographing text input by a user, the image generator 121 may generate an image in a predetermined unit. That is, the image generator 121 may generate the first image in units of characters included in the text, and may also generate the first image in units of words. In addition, the image generator 121 may generate the first image in units of syllables, phonemes, or spellings of words included in the text.

Accordingly, the first image may be a captured image of the entire text, but may include a captured image of a character or word included in the text.

Meanwhile, the second identification unit 120 deep-learns the text so that the image generating unit 121 can capture or capture the entire text and words or characters that are constituting units of the text. It may further include a learning unit 122 capable of recognizing and distinguishing characters.

Accordingly, the learning unit 122 according to the present embodiment allows the image generation unit 121 to capture or capture the entire text and words or characters that are constitutional units of the text, and capture or photograph the text. It is configured to deep-learn 1 image. In addition, the learning unit 122 may store the result of deep learning the first image in the database 140, and later apply the result stored in the database 140 to other deep learning.

The learning unit 122 according to the present embodiment may itself include an AI processor (not shown), a memory (not shown), and/or a communication unit (not shown) to perform deep learning.

AI processors can learn neural networks using programs stored in memory. In particular, the AI processor can learn a neural network to recognize device-related data. Here, the neural network for recognizing device-related data may be designed to simulate a human brain structure on a computer, and may include a plurality of network nodes having weights that simulate neurons of the human neural network. The plurality of network modes can exchange data according to their respective connection relationships so that neurons can simulate synaptic activity of neurons that send and receive signals through synapses. Here, the neural network may include a deep learning model developed from a neural network model. In a deep learning model, a plurality of network nodes are located in different layers and may exchange data according to a convolutional connection relationship. Examples of neural network models include deep neural networks (DNN), convolutional deep neural networks (CNN), Recurrent Boltzmann Machine (RNN), Restricted Boltzmann Machine (RBM), and deep trust. It includes various deep learning techniques such as deep belief networks (DBN) and deep Q-network, and can be applied to fields such as computer vision, speech recognition, natural language processing, and speech/signal processing.

Meanwhile, the AI processor performing the functions as described above may be a general-purpose processor (eg, a CPU), but may be an AI-only processor (eg, a GPU) for artificial intelligence learning.

The memory may store various programs and data required for the operation of the learning unit 122. The memory may be implemented as a non-volatile memory, a volatile memory, a flash memory, a hard disk drive (HDD), or a solid state drive (SDD). The memory is accessed by the AI processor, and data read/write/edit/delete/update by the AI processor may be performed. In addition, the memory may store a learning algorithm used by the learning unit 122 for recognition and data classification of the first image photographed text or a neural network model (for example, a deep learning model) generated by such an algorithm. have.

Meanwhile, the AI processor according to the present embodiment may further include a data learning unit for learning a neural network for data classification/recognition. The data learning unit may learn a criterion for how to classify and recognize data using which training data to use to determine data classification/recognition. The data learning unit may learn the deep learning model by acquiring training data to be used for training and applying the acquired training data to the deep learning model.

The data learning unit may be manufactured in the form of at least one hardware chip and mounted on the learning unit 122. For example, the data learning unit may be manufactured in the form of a dedicated hardware chip for artificial intelligence (AI), or may be manufactured as a part of a general-purpose processor (CPU) or a graphics dedicated processor (GPU) and mounted on the learning unit 122. . In addition, the data learning unit may be implemented as a software module. When implemented as a software module (or a program module including an instruction), the software module may be stored in a computer-readable non-transitory computer readable media. In this case, at least one software module may be provided by an operating system (OS) or an application.

In addition, the data learning unit may include a learning data acquisition unit and a model learning unit.

The training data acquisition unit may acquire training data necessary for a neural network model for classifying and recognizing data. For example, the training data acquisition unit may acquire data and/or sample data for a first image to be input into the neural network model as training data.

The model learning unit may learn to have a criterion for determining how the neural network model classifies predetermined data by using the acquired training data. In this case, the model learning unit may train the neural network model through supervised learning using at least a portion of the training data as a criterion for determination. Alternatively, the model learning unit may train the neural network model through unsupervised learning that discovers a criterion by learning by itself using the training data without guidance. In addition, the model learning unit may train the neural network model through reinforcement learning by using feedback on whether the result of situation determination according to the learning is correct. In addition, the model learning unit may train the neural network model using a learning algorithm including error back-propagation or gradient decent.

When the neural network model is trained, the model learning unit may store the trained neural network model in a memory. The model learning unit may store the learned neural network model in the memory of the database 140 and/or the server connected to the learning unit 122 through a wired or wireless network.

The data learning unit may further include a training data preprocessor (not shown) and a training data selection unit (not shown) to improve the analysis result of the recognition model or to save resources or time required for generating the recognition model.

The learning data preprocessor may preprocess the acquired data so that the acquired data can be used for learning to determine a situation. For example, the training data preprocessor may process the acquired data into a preset format so that the model training unit can use the training data acquired for learning for first image recognition.

In addition, the learning data selection unit may select data necessary for learning from the learning data acquired by the learning data acquisition unit or the training data preprocessed by the preprocessor. The selected training data may be provided to the model training unit. For example, the learning data selection unit selects only data for text for a specific area, that is, an object included in a specific area, as the training data by detecting only a specific area of the first image acquired through the image generating unit 121. I can.

In addition, the data learning unit may further include a model evaluation unit (not shown) to improve the analysis result of the neural network model.

The model evaluation unit may input evaluation data to the neural network model and, when an analysis result output from the evaluation data does not satisfy a predetermined criterion, may cause the model learning unit to learn again. In this case, the evaluation data may be predefined data for evaluating the recognition model. As an example, the model evaluation unit may evaluate as not satisfying a predetermined criterion if the number or ratio of evaluation data whose analysis result is not accurate among the analysis results of the learned recognition model for evaluation data exceeds a threshold value. have.

Meanwhile, the communication unit may transmit the result of AI processing by the AI processor to the outside of the device 10 according to the present embodiment.

Here, the external term of the device 10 according to the present embodiment includes an external electronic device, and the external electronic device may be defined as a server, a user terminal, or a user terminal or server connected to them through a 5G network.

On the other hand, the transforming unit 130 according to the present embodiment replaces the first prohibition word and the second prohibition word included in the text input by the user with special characters. For example, words, characters, and slang words similar to swear words and swear words included in the text may be replaced with "*", or special characters such as "*" may be placed between corresponding prohibitions.

In addition, the transforming unit 130 may directly remove the first prohibited words and the second prohibited words included in the text by commands of the first identification unit 110 and the second identification unit 120. In addition, the modification unit 130 receives a first prohibition list from the database 140 or a second prohibition list recognized by the second identification unit 120 from the second identification unit, You can also remove the second ban on yourself.

Accordingly, in the apparatus 10 according to the present embodiment, in order to filter the banned words separated or transformed in the form of phonemes or characters from text, the first identification unit 110 firstly stores the banned words data stored in the database 140 in the text. Is applied, and the second identification unit 120 secondly identifies unknown words or out-of-vocabulary (OOV) words included in the text, and is registered or registered through image detection for them. No, but check for variations on the banned words that are very similar to the banned words.

Hereinafter, a method of filtering text using the text filtering apparatus 10 using image learning will be described according to an embodiment of the present invention with reference to FIGS. 2 to 6.

Referring to FIG. 2, the text filtering apparatus 10 using image learning is configured to perform various steps (S100 to S160) to filter text in the present embodiment. 2 is a flowchart illustrating a text filtering method using image learning according to an embodiment of the present invention.

Although the steps are sequentially represented in FIG. 2, some steps may be performed simultaneously by the apparatus 10 according to the present embodiment.

Referring to FIG. 2, the text filtering apparatus 10 using image learning starts to start when the power is turned on.

Thereafter, when the user inputs text through the user terminal, the receiving unit 100 receives the text (S100).

The first identification unit 110 first identifies whether or not there is a first banned word in the received text (S110). The first identification unit 110 loads the first prohibition word list stored in the database 140 and identifies whether the first prohibition word is included in the text input by the user.

For example, the online conversation below is a conversation between two different users, A and B.

A: ahnwa, stupid motherfucker.

B: I'm breaking it. Fold the game ㅅㅅㅣㅂㅏㄹㄴㅓㅁㅇㅏ.

Referring to the above dialogue, A first inputs the general profanity word ‘bird’ as text, and B then enters the profanity ‘ㅅㅅㅣㅂㅏㄹㄴㅓㅁㅇㅏ' by separating them into letters or phonemes.

In the above dialogue,'Bit' mentioned by A is a general profanity word included in the first banned word list stored in the database 140, and'I am close' and'ㅅㅅㅣㅂㅏㄹㄴㅓㅁㅇㅏ' are database 140 It is assumed that the word is not included in the first prohibition list stored in ).

In this case, the first identification unit 110 may identify that the'suckle' corresponds to the banned word in the text, since'Bit' is stored in the first banned word list. However, since the first identification unit 110 is not stored in the first prohibition list,'You are close' and'ㅅㅅㅣㅂㅏㄹㄴㅓㅁㅇㅏ' and'ㅅㅅㅣㅂㅏㄹㄴㅓㅁㅇㅏ' correspond to prohibited words. Cannot be identified.

Thereafter, the second identification unit 120 determines whether a first word or a first character not included in the list of normal words stored in the database 140 at the same time is included in the text that is not included in the first banned word. (S120).

The first word or the first character refers to a word or character that is not included in the first prohibited word among texts and is not included in the list of normal words stored in the database 140 at the same time.

According to the above-described example, the second identification unit 120 does not include'You are close' and'ㅅㅅㅣㅂㅏㄹㄴㅓㅁㅇㅏ' in the first banned word, and is not included in the list of normal words stored in the database 140 at the same time. It can be recognized as a first word or a first letter.

Then, the second identification unit 120 generates a first image for the text (S130). Here, as described above, the first image may include an image for the entire text and an image generated for each character or word constituting the text.

For example,'I'm close. Fold the game ㅅㅅㅣㅂㅏㄹㄴㅓㅁㅇㅏ.' can be created as a first image as a whole. In addition, each image of'You are close' and'ㅅㅅㅣㅂㅏㄹㄴㅓㅁㅇㅏ' may be generated as the first image.

Then, the second identification unit 120 deep-learns the generated first image to secondly identify whether or not the second prohibition is included in the first image (S140). That is, the second identification unit 120 analyzes the first image and identifies whether the first word or the first character is a character similar to an abusive language, slang, or abusive language equivalent to the second prohibition.

Here, as described above, the second forbidden word is a character or word similar to swearword, slang, and swearword that is not included in the first prohibited word and is not included in the list of normal words stored in the database 140 at the same time.

When the second identification unit 120 analyzes the first image and determines that the first word or the first character corresponds to the second prohibition (S150), the transforming unit 130 2 Replace all prohibited words with special characters (S160).

If, in step S120 described above, if the text does not contain the first word or the first character, the prohibition words determined by the first identification unit 110 as the first prohibition words are transformed into special characters by the transforming unit 130 Is output.

In addition, in step S150 described above, if the second identification unit 120 determines that the first word or the first character does not correspond to the second prohibition word, the prohibited words determined by the first identification unit 110 are modified. The unit 130 may be transformed into a special character and output, and text filtering may be terminated.

Meanwhile, step S150 described above will be described in more detail with reference to FIG. 3. 3 is a flow chart illustrating a process of identifying whether or not a banned word is included in a text by the second identification unit 120 according to an embodiment of the present invention.

Referring to FIG. 3, step S150 further includes a step of determining, by the second identification unit 120, how similar the first word or the first character included in the text is to the first prohibition word or the second prohibition word (S151). Includes. In this case, the second identification unit 120 includes a first word or a first character included in the text and/or a first prohibition word previously stored in the database 140 and/or a product identified through deep learning or an external server search. 2 It is determined whether it is similar to the prohibition word (S152). The second identification unit 120 learns features of the syllables, phonemes, and meanings of the first word or the first letter through deep learning. In addition, the second identification unit 120 determines whether the characteristics of syllables, phonemes, and meanings of the first and/or second prohibited words and the learning result are similar in a certain ratio (S152). For example, when the similarity between the two elements is 70% or more, it may be set to be determined to be substantially the same.

In this way, if the second identification unit 120 determines that the first word or the first character is similar to the first prohibition word and/or the second prohibition word is more than a certain standard (S152), the second identification unit 120 The word or the first letter may be regarded as a second prohibition (S153).

In the above-described example, the first prohibition word'don't get caught' is registered, and'ㅅㅅㅣㅂㅏㄹㄴㅓㅁㅇㅏ' means the same as'fuck' and'fucker' through deep learning by the second identification unit 120 Assuming that they have learned that they have, the second identification unit 120 can determine that the semantic characteristics are very similar to the first word or the first character,'I am close', which is the first prohibition word,'Don't get close'. There is (S152), and'ㅅㅅㅣㅂㅏㄹㄴㅓㅁㅇㅏ' can be judged to have the same meaning as'fuck' and'fucker'.

Thereafter, the second identification unit 120 may replace the first word or the first character, which is regarded as the second prohibition word, with a special character through the deformation unit 130 (S160).

On the other hand, in the above-described example, when the first word or the first letter is'don't laugh' instead of'you're close', and'ㅅㅅㅣㅂㅏㄹㄴㅓㅁㅇㅏ' is'ㄴㄴ ㅈㅎㅅㅇ', the second identification unit 120 It may be determined that the first word or the first character is not similar to the first prohibition word and the second prohibition word (S152).

In other words,'Don't be funny' is not registered as the first banned word, but as a result of deep learning by the second identification unit 120, it is difficult to see abusive language, a word or character similar to abusive language, or as a profanity, and for'ㄴㄴ ㅈㅎㅅㅇ' 2 As a result of the deep learning of the identification unit 120, it may be determined that it has the meaning of'You are good', so'ㄴㄴ ㅈㅎㅅㅇ' will not be seen as a word or character or profanity similar to swearwords and swear words.

However, in this case, the second identification unit 120 generates a list of'Don't be funny' and'ㄴㄴ ㅈㅎㅅㅇ' (S154), or'Do not be funny' and'ㄴㄴ ㅈㅎㅅㅇ' to the existing first prohibition list. Although not similar to profanity or profanity, it can be added as reference word or text data (S154).

In addition, the second identification unit 120 may generate third images for'don't be laughable' and'ㄴㄴ ㅈㅎㅅㅇ', deep learning, and use them as learning data for the second prohibition inference (S155).

In addition, referring to FIG. 4, step S160 will be described in detail. 4 is a flowchart illustrating a process of converting a banned word included in a text into a special character by the transforming unit 130 according to an embodiment of the present invention.

Referring to FIG. 4, the transforming unit 130 checks in which language the second identification unit 120 is composed of a first word or a text including a first character considered as a second prohibition (S161).

The second identification unit 120 may preferentially determine whether the text is in Korean (S162).

In step S162, when the second identification unit 120 determines that the text including the first word or the first character, which is regarded as the second prohibition word, is composed of Korean (S162), the transforming unit 130 comprises the first and second characters. The'Hangul transformation algorithm' may be applied to the banned word (S163). If, in step S162, the second identification unit 120 determines that the text including the first word or the first character, which is regarded as the second prohibition word, is not composed of Korean (S162), the transforming unit 130 is And it is possible to apply the'English transformation algorithm' to the second prohibition word (S164).

The above-described'Hangul transform algorithm' and'English transform algorithm' may be developed with, for example, Windows-based Visual Studio 2015 C++/MFC. The Hangul transforming algorithm' and the'English transforming algorithm' read a text file including the second banned word, recognize a language, and then apply a'transformation algorithm' for the language to create a word list for the modified banned word.

In particular, the above-described'Hangul Transformation Algorithm' and'English Transformation Algorithm' can output, for example, a list of words of the transformed prohibition words generated in this way to an edit control, capture the corresponding area, and save it as an image file. have.

In addition, the above-described'Hangul Transformation Algorithm' and'English Transformation Algorithm' can create a word list of modified prohibitions according to the name of the text file to be loaded, or capture the transformed prohibitions as words as an image file and save them as image files. have.

In addition, the above-described'Hangul transforming algorithm' and'English transforming algorithm' may add a task of writing a modified character included in a modified banned word in a text file when creating a word list of modified banned words.

For example, when the first word or the first character is'ㅅㅅㅣㅂㅏㄹㄴㅓㅁㅇㅏ', the transformed character may be'ㅅㅅ'. That is,'ㅅㅅ' is a modified character of'ㅆ', and'ㅅㅅ' is a modified character that is likely to be included in the second prohibition and may be recognized by the second identification unit 120.

Accordingly, the'Hangul modification algorithm' of the transforming unit 130 may make a list as shown in Table 1 below or may modify the prohibited words by referring to Table 1 when creating a word list of modified banned words.

[Table 1]

In addition, the'English transformation algorithm' of the transformation unit 130 may create a list as shown in Table 2 below, or may modify the prohibition word by referring to Table 2 below when creating a word list of modified prohibitions.

[Table 2]

In addition, the transforming unit 130 may use the special characters shown in Table 3 below when converting the modified banned words into special characters or adding special characters to the modified banned words so that the banned words are not properly displayed.

[Table 3]

On the other hand, the step (S163) of applying the'Hangul transforming algorithm' to the first and second prohibited words by the transforming unit 130 (S163) includes more detailed steps as shown in FIG. 4.

Referring to FIG. 4, the transforming unit 130 separates individual words or characters constituting the text into an initial, neutral, and final (S1631), and a special to replace the initial, neutral and final respectively constituting the second prohibition. The characters are randomly extracted (S1632), and the initial, neutral, and final constituents constituting the second banned word are replaced with the extracted special characters (S1633).

In addition, after step S1633, the transforming unit 130 assigns an order to individual characters constituting the text in order from 1 to n times (S1634), and randomly selects a plurality of specific sequences from 1 to n times Then, the special characters to be added in a specific order are randomly selected (S1635), and the selected special characters are added to the selected specific order (S1636) so that the prohibited words included in the text are not displayed, or the prohibited words included in the text are selected. It prints out so that it doesn't recognize it.

As described above, in the text filtering method using image learning according to the present embodiment, when the text is Hangul, a Hangul transformation algorithm is applied. In addition, the Hangul transformation algorithm separates the input Hangul into the initial, neutral, and ending, and performs a transformation operation of replacing or adding special characters.

Because banned words in Korean are often composed of two or three letters, separate one or two letters of the word and write the neutral or finality on the lower line. In addition, the Hangul transformation algorithm performs a task of replacing the created words with special characters and/or adding a special character.

In addition, the step (S164) of applying the'English transforming algorithm' to the first and second prohibited words by the transforming unit 130 (S164) includes more detailed steps as shown in FIG. 4.

Referring to FIG. 4, the transforming unit 130 randomly extracts special characters to replace individual words or characters constituting the text (S1641), and extracts the words or characters constituting the second prohibition into the extracted special characters. Replace (S1642).

In addition, after step S1642, the transforming unit 130 assigns an order to individual characters constituting the text in order from 1 to n times (S1643), and randomly selects a plurality of specific sequences from 1 to n times. In this way, special characters to be added in a specific order are randomly selected (S1644), and the selected special characters are added to the selected specific order (S1645), so that the prohibited words contained in the text are not displayed, or the prohibited words contained in the text It prints out so that it doesn't recognize it.

When the text filtering method using image learning according to the present embodiment applies the'English transformation algorithm', since English is not a combination character like Hangul, the work of separating the initial, neutral, and final characters or the spelling is not performed. . Since English banned words have different lengths of words compared to Korean banned words, the'English transformation algorithm' according to the present embodiment randomly selects some of the letters constituting the banned words and replaces them with special characters and/or special characters between spellings or words. Add text.

On the other hand, referring to FIGS. 5 and 6, the text filtering method using image learning according to the present embodiment shows a process of generating a modified word formed by replacing a banned word with a special character or adding a special character to the banned word after step S150. Explain.

FIG. 5 is a flowchart illustrating a process in which the transforming unit 130 generates a transformed word according to an embodiment of the present invention, and FIG. 6 is a flowchart illustrating a condition in which the transforming unit 130 terminates generating a transformed word.

Meanwhile, the modified word here refers to a word formed by replacing a banned word with a special character, and a word formed by adding a special character to the banned word. In addition, the second recognition unit 120 according to an embodiment of the present invention learns the modified word through the learning unit 122, and through the first and second prohibited words and special characters used in the process of generating the modified word, You can learn various examples of variations of banned words.

Referring to FIG. 5, after step S150, the transforming unit 130 generates a transformed word formed by being replaced with a special character or by adding a special character (S170).

In addition, the transforming unit 130 may convert the generated transformed word into a text file and store it in the database 140 (S180).

In addition, the transforming unit 130 may send the generated transformed word to the second recognition unit 120 so that a second image for the transformed word can be generated, and store the corresponding second image in the database 140. have.

In particular, as shown in FIG. 6, in generating the transformed word, the transforming unit 130 may continuously generate transformed words by repeating steps S163 and/or S164 described above (S171).

The transforming unit 130 may generate a transformed word by repeating steps S163 and/or S164 N times until the number of transformed words generated reaches a predetermined number (N) (S172). Thereafter, when the transforming unit 130 repeats steps S163 and/or S164 as many as N of the generated words in step S172, the transforming unit 130 may finish generating the transformed word (S173).

This is, in order to enable the learning unit 122 of the second recognition unit 120 to learn various modified embodiments of the second prohibition word and the second prohibition word that are not registered or stored as the first prohibition word, the transformation unit 130 Is a process of generating learning data of the learning unit 122.

For example, if the condition is set so that the transforming unit 130 generates 10,000 transformed words, the transforming unit 130 repeats steps S163 and/or S164 tens of thousands of times, and generates tens of thousands of transformed words.

In this process, the first and second prohibited words modified by the transforming unit 130, special characters used for replacement and addition, and prohibited words replaced with special characters or added with special characters are all generated as learning data of the learning unit 122 do. Then, the learning unit 122 learns the learning data and expands the range of the second prohibition by itself.

In this way, if the learning unit 122 learns and expands the range of the second prohibition words, for example, when 140 Hangul prohibitions are used to replace the prohibitions with special characters or add special characters to the prohibitions, 140 Hangul prohibitions are each Modified Hangul prohibited words image files may be stored in the database 140.

Meanwhile, an experiment was conducted to classify a Hangul banned word class from a normal word class using the apparatus 10 according to the present invention. The result is as follows.

The banned words class to be classified includes a total of 140 banned words in Hangul, and each class has a Hangul banned words image file created with the Hangul transformation algorithm. The number of images for each class used in the experiment is more than 10,000. A normal word class consisting of unmodified non-banning words is composed of about 10,000 words that are not modified and are not duplicated. The total number of images used in the experiment was 1,473,378.

In the transformed Hangeul, characters are separated into initial, neutral, and final, and neutral or final characters are written in the next line, special characters are added in the middle, and special characters resembling consonants and vowels are sometimes replaced. In order not to create rules for character generation, special characters to be used and the positions of characters to be added or changed are chosen at random each time.

Meanwhile, the structure of a neural network model included in the learning unit 120, that is, a CNN model, is a form that is often seen in image learning, and a convolution layer and a pooling layer are stacked up. The number of outputs of the model is the number of classes, and the Softmax function is used to output the class with the highest probability.

In addition, an experiment was conducted to classify an English banned word class from a normal word class using the apparatus 10 according to the present invention. The result is as follows.

The prohibition classes to be classified are 69 English prohibitions, and each class has an English prohibition image file created with an English transformation algorithm. The number of images for each class used for learning is more than 10,000 each. The normal word class consists of about 10,000 non-overlapping word images using the corpus corpus of the NLTK package. The total number of images used in the experiment was 726,114. Transformed English words do not separate characters like Hangul, but add or replace special characters to create transformed words. Changed English words are also randomly selected for special characters and positions to prevent rules from occurring.

Through these experiments, it was proved that the apparatus 10 and the text filtering method using the same according to the present invention can transform words and create transformed words according to language characteristics constituting text. In addition, image files were created for transformed words, and edited distance algorithm experiments and OCR experiments were conducted with the transformed words. In both experiments, it was difficult to predict the original word, but modified word recognition was possible through CNN learning.

That is, the device 10 according to the present invention can recognize the second prohibition words that are previously stored in the database 140, the second prohibition words that are not registered or classified as the first prohibition word, and similar prohibition words modified from the second prohibition word are also viewed. The device 10 according to the invention is recognizable.

The present invention can be implemented as a computer-readable code on a medium on which a program is recorded. The computer-readable medium includes all types of recording devices that store data that can be read by a computer system. Examples of computer-readable media include hard disk drives (HDDs), solid state disks (SSDs), silicon disk drives (SDDs), ROMs, RAM, CD-ROMs, magnetic tapes, floppy disks, optical data storage devices, etc. There is also a carrier wave (for example, transmission over the Internet) also includes the implementation in the form of. Therefore, the detailed description above should not be construed as restrictive in all respects and should be considered as illustrative. The scope of the present invention should be determined by reasonable interpretation of the appended claims, and all changes within the equivalent scope of the present invention are included in the scope of the present invention.

100: receiver
110: first identification unit
120: second identification unit
130: deformation portion
140: database

Claims (12)

Receiving, by a receiving unit, the text input from the user terminal;
First identifying, by a first identification unit, whether or not a first prohibition word is present in the text;
Determining whether a first word or a first character that is not included in the first prohibition word among the text and is not included in the normal word list stored in the database at the same time is included by a second identification unit;
Generating, by the second identification unit, a first image for the text;
Secondly identifying whether there is a second prohibition word by deep learning the first image by the second identification unit; And
A step of replacing the first and second prohibited words included in the text with special characters,
The first banned words include banned words and similar banned words stored in a database,
The second banned words are banned words not stored in the database and modified banned words, a text filtering method using image learning.
The method of claim 1,
The step of secondly identifying whether there is a second prohibition word by deep learning the first image by the second identification unit,
Determining a degree of similarity indicating how similar the first word or the first letter is to the first prohibition word or the second prohibition word; And
Further comprising determining whether to regard the first word or the first character as the second kinsoku word according to the degree of similarity.
The method of claim 2,
The step of replacing the second prohibition word included in the text by the transforming unit with a special character,
The text filtering method using image learning, further comprising the step of determining in which language the text is composed.
The method of claim 3,
The step of determining in which language the text is composed of,
Confirming that the text is composed of Korean; And
The text filtering method using image learning, further comprising the step of applying a Hangul transformation algorithm to the second banned word included in the text.
The method of claim 4,
The step of applying a Hangul transforming algorithm to the second banned word included in the text,
Separating individual characters constituting the text into an initial, neutral, and final sound, respectively;
Randomly extracting special characters to replace the initial, neutral, and final characters constituting the second banned word; And
The text filtering method using image learning further comprising the step of replacing the initial, neutral, and final constituents constituting the second banned word with extracted special characters.
The method of claim 5,
After the step of replacing the initial, neutral and ending constituting the second prohibition word with the special characters,
Assigning an order to individual characters constituting the text in order from 1 to n;
Randomly selecting a plurality of specific sequences among the 1 to n times, and randomly selecting special characters to be added in each specific order; And
Text filtering method using image learning, further comprising the step of adding the selected special character to the selected specific order.
The method of claim 3,
The step of determining in which language the text is composed of,
Confirming that the text is in English; And
The text filtering method using image learning, further comprising the step of applying an English transformation algorithm to the second prohibited words included in the text.
The method of claim 7,
The step of applying an English transformation algorithm to the second prohibited words included in the text,
Randomly extracting special characters to replace individual characters constituting the text;
Further comprising the step of replacing the individual characters constituting the text with the extracted special characters, text filtering method using image learning.
The method of claim 8,
After the step of replacing individual characters constituting the text with the extracted special characters,
Assigning an order to individual characters constituting the text in order from 1 to n
Randomly selecting a plurality of specific sequences among the 1 to n times, and randomly selecting special characters to be added in each specific order; And
Text filtering method using image learning, further comprising the step of adding the selected special character to the selected specific order.
The method of claim 1,
After the step of replacing the second prohibition word included in the text by the transforming unit with a special character,
Generating a modified word formed by being replaced with the special character or by adding the special character;
Storing the generated modified word as a text file; And
The text filtering method using image learning further comprising the step of storing the modified word as a second image file in the database.
A receiver for receiving text from a user terminal;
A database storing first prohibited words including profanity and profanity;
A first identification unit for reading a list of the first prohibited words from the database and identifying whether the first prohibited words are included in the text;
A second identification unit that identifies whether or not the text contains prohibited words not stored in the database and a second prohibited words that are modified prohibited words; And
And a transforming unit for replacing the first and second prohibited words included in the text with special characters,
The second identification unit,
A text filtering apparatus using image learning, which generates a first image by photographing the text, and determines whether the second prohibition is included in the text by deep learning the first image.
The method of claim 11,
The second identification unit,
An image generator for generating the first image; And
A text filtering apparatus using image learning, further comprising a learning unit for deep learning the first image.

Images