KR102046640B1 - Automatic terminology recommendation device and method for big data standardization - Google Patents

Abstract

The present invention relates to an apparatus and a method for automatically recommending standard terms for big data standardization and a method thereof. The apparatus for automatically recommending standard terms for big data standardization can comprise: a database containing standardized term data; a standard dictionary part including and managing a standard word dictionary, a standard term dictionary, and a standard domain dictionary; a word substitution part separating morphemes of a first term inputted to perform standardization, and substituting a plurality of words included in the first term with a standard word included in the standard word dictionary; a term recommendation part combining a plurality of words included in the substituted first term and recommending the standard term included in the standard term dictionary as the first standard term of the first term; and a term determination part selecting one of the standard term recommended in the term recommendation part as the first standard term of the first term and determining a standard domain included in the standard domain dictionary as a first standard domain of the first term.

Description

AUTOMATIC TERMINOLOGY RECOMMENDATION DEVICE AND METHOD FOR BIG DATA STANDARDIZATION}

The present invention relates to a standard term automatic recommendation apparatus and method for big data standardization.

Data standardization is one of the essential tasks to ensure data quality. Simultaneous development of systems, lack of data management mindset, lack of data management personnel and tools can lead to data redundancy and inconsistency, as well as difficulties in data integration. This phenomenon increases the likelihood of problems when data grows exponentially.

As the interest and demand for big data increases, the necessity of data standardization increases. Data standardization can have the expected effects of clear communication and data grasping, and reduce the costs and resources of data transformation and remediation that result from the most important data quality improvements in the areas of data analysis and machine learning.

Data standardization refers to establishing and applying principles throughout the system on the names, definitions, rules, and formats of data information elements scattered throughout the system. Standardization also has the effect of adjusting conflicting views of each piece of data.

There are business terms and technical terms to define standard terminology and define standards for terms used in the business so that there is no misunderstanding or confusion in communication. By defining this, Korean name and English name can be defined consistently. Managing these standard terms is the standard dictionary.

Morphological analyzer is the core technology in natural language processing. It separates words or sentences into morphemes, the smallest unit of speech, and determines the parts of speech of the separated morphemes. The morphological analysis within the data set separates the words in the standard word dictionary as well as the morphological units, thus providing users with various recommended terms.

If there is no separate word, it calculates the similarity with the word of the suggested word dictionary, recommends the highest word, corrects the error derived from human error such as typo, and can be registered in the standard word dictionary. do.

The most commonly used algorithm for calculating the similarity between data in text data is Fuzzy Matching algorithm. This algorithm calculates the similarity between data using the calculated result based on the editing distance (Levenshtein Distance). By applying this algorithm, the algorithm finds the closest word and term in the dictionary and recommends it to the user.

In addition, Fuzzy Matching algorithm is developed based on English, so when applied to Korean, it has a problem of calculating similarity based on syllables, not phonemes. Calculate

The background technology of the present application is disclosed in Korean Patent Publication No. 10-1975419.

The present application is to solve the above-mentioned problems of the prior art, to overcome the difficulties encountered when registering a standard term, by using a standard term dictionary and a standard word dictionary that exist, recommending the standard terminology to the user data An object of the present invention is to provide a standard term automatic recommendation device and method for standardizing big data, which shortens the time for selecting or selecting terms.

The present application is to solve the above-described problems of the prior art, including a terminology collector applying the morphological analyzer to the apparatus and method, and automatically divides the input terms into words in the morpheme unit to replace the words in the standard word It is an object of the present invention to provide a standard term automatic recommendation apparatus and method that reduces the time required for standardization by recommending a recommendation.

The present application is to solve the above-mentioned problems of the prior art, and by using a method of calculating Korean data separately by phonological units rather than syllable units, a standard term that recommends terms by correcting human errors such as typos of users. It is an object of the present invention to provide an automatic recommendation apparatus and method.

However, the technical problem to be achieved by the embodiments of the present application is not limited to the technical problems as described above, and other technical problems may exist.

As a technical means for achieving the above technical problem, the standard term automatic recommendation apparatus for standardizing big data according to an embodiment of the present application, a database containing standardized term data, standard word dictionary, standard term dictionary and standard A standard dictionary section including and managing a domain dictionary, a word for separating a morpheme of a first term input for performing standardization, and replacing a plurality of words included in the first term with a standard word included in the standard word dictionary Using a combination of a plurality of standard words substituted with a plurality of substituted standard words, and recommending at least one of a plurality of standard terms included in the standard term dictionary as a standard term of the first term using the combined plurality of standard words Selecting one of a term recommender and a standard term recommended in the term recommender as the first standard term of the first term, Group may include a term determination unit determining a first standard domain of the first terminology at least one of a plurality of standard domain included in the standard domain dictionary.

According to an embodiment of the present application, the word substitution unit, using a morpheme analysis technique, a word breaker for separating the first term into a plurality of words and a set of a plurality of words separated from the word breaker to the standard word dictionary It may include a word matcher to match any one of the included words.

According to one embodiment of the present application, the word breaker, if any one of the plurality of words included in the first term is a word that does not exist in the standard word dictionary, the word does not exist in the standard word dictionary It can be separated into units, perform similarity calculations using AI-based algorithms, and recommend similar words.

According to an embodiment of the present disclosure, the term recommendation unit performs a similarity operation using an AI based algorithm using a plurality of standard words combined with a standard term included in the standard term dictionary part, Standard terminology of 1 term can be recommended.

According to an embodiment of the present disclosure, the term automatic recommending apparatus may include a data providing unit providing selection items related to input information for term standardization to a user terminal and a user input receiving unit for receiving input information for term standardization from the user terminal. It may include.

According to an embodiment of the present disclosure, the standard term automatic recommendation method for big data standardization includes receiving a first term to perform standardization, a database including standardized term data, and a standard word dictionary and a standard term dictionary And separating a morpheme of the first term input to perform standardization by using a standard dictionary unit including and managing a standard domain dictionary, and converting a plurality of words included in the separated first term into the standard word dictionary. Substituting a standard word included in a standard word, combining a plurality of words included in the substituted first term, and using a plurality of combined standard words, at least one of a plurality of standard terms included in the standard term dictionary Recommending any one as a standard term of the first term, any one of the recommended standard terms as a first standard of the first term Steps and can include determining a first standard domain of said plurality of standard domain of at least one of said first standard terms like a standard included in the domain dictionary to select language.

The above-mentioned means for solving the problems are merely exemplary and should not be construed as limiting the present application. In addition to the above-described exemplary embodiments, additional embodiments may exist in the drawings and detailed description of the invention.

According to the aforementioned problem solving means of the present application, for standardizing the database, terms input by the user, such as logical names of columns to be standardized, are separated by word units through morphological analysis, replaced with standard words, and combined with each other. It can be suggested as a term that exists in the standard term dictionary of or combines standard words.

According to the aforementioned problem solving means of the present application, the user can manually perform the standard word substitution after separating the input terms into word units using both the morpheme separation and the existing standard word dictionary. It is possible to replace the existing method of separating.

According to the above-described problem solving means of the present invention, if there is no standard word corresponding to the separated word, it calculates the similarity with the word in the standard word dictionary, recommends the most similar words, even typographical errors such as typos It can be used by substituting the suggested word. If it does not exist, it can be registered in the standard word dictionary.

According to the above-described problem solving means of the present application, through the calculation of the similarity between the terms entered in the standard term dictionary, it automatically recommends based on terms standardized by other institutions or companies to provide information for the user to determine the standardized term can do.

However, the effects obtainable herein are not limited to the effects as described above, and other effects may exist.

1 is a block diagram schematically illustrating a configuration of a standard term recommendation apparatus according to an exemplary embodiment of the present application.
FIG. 2 is a diagram illustrating a process of generating a standard word dictionary during standardization of a standard term recommendation apparatus according to an exemplary embodiment of the present disclosure.
3 is a view for explaining a process of generating a standard domain dictionary of the standardization process of the standard term recommendation apparatus according to an embodiment of the present application.
4 is a view for explaining a process of generating a standard term in the standardization process of the standard term recommendation apparatus according to an embodiment of the present application.
FIG. 5 is a diagram illustrating a screen made of a standard word dictionary of a standard term recommendation apparatus according to an embodiment of the present application stored in a database so as to be confirmed on a web.
FIG. 6 is a diagram illustrating a screen of a standard domain dictionary of a standard term recommendation apparatus according to an embodiment of the present disclosure stored in a database so that the web can be viewed on a web.
FIG. 7 is a diagram illustrating a screen of a standard term recommendation apparatus of a standard term recommendation apparatus according to an embodiment of the present disclosure stored in a database for checking on a web.
8 is a diagram schematically illustrating a morphological analysis of a standard term recommendation apparatus according to an embodiment of the present application.
9 is a schematic diagram of an example when a word breaker of a standard term recommendation apparatus according to an embodiment of the present disclosure is performed.
FIG. 10 is a diagram schematically illustrating an example when a word matcher of the standard term recommendation apparatus according to the exemplary embodiment of the present application is performed.
11 is a view schematically showing an example of a task performed by the term recommendation unit of the standard term recommendation apparatus according to an embodiment of the present application.
12 is a diagram schematically showing an example of a task performed by the term decision unit of the standard term recommendation apparatus according to an exemplary embodiment of the present application.
13 is a flowchart illustrating a standard term automatic recommendation method for big data standardization according to an embodiment of the present application.

DETAILED DESCRIPTION Hereinafter, exemplary embodiments of the present disclosure will be described in detail with reference to the accompanying drawings so that those skilled in the art may easily implement the present disclosure. As those skilled in the art would realize, the described embodiments may be modified in various different ways, all without departing from the spirit or scope of the present invention. In the drawings, parts irrelevant to the description are omitted for simplicity of explanation, and like reference numerals designate like parts throughout the specification.

Throughout this specification, when a part is "connected" with another part, it is not only "directly connected" but also "electrically connected" or "indirectly connected" with another element in between. Also includes the case where

Throughout this specification, when a member is said to be located on another member "on", "upper", "top", "bottom", "bottom", "bottom", this means that any member This includes not only the contact but also the presence of another member between the two members.

Throughout this specification, when a part is said to "include" a certain component, it means that it can further include other components, without excluding the other components unless specifically stated otherwise.

Data standardization is ideally ideal for all objects that can be defined in the data model and database, but it is more efficient to standardize data only for objects that need to be managed. In general, subjects managed by data standards include terms, words, domains, and codes.

The standard word defines the standard for the words that make up the standard term so that the Korean and English names for the term can be defined consistently. By managing standard words, it is possible to prevent the creation of terms that mean the same concept and to be a standard for writing English names.

The standard domain is a grouping property of columns, which can be classified into character, numeric, and time, and can be classified into name, address, ID, amount, rate, and quantity. By defining the standard for the domain, the data type and data length of the column with the same property can be managed consistently, and common data validation rules can be applied to the column values. It is also possible to consistently define the data type and length of a column by applying a standard domain to the column.

Data standard definitions start with standard dictionary definitions. Standard word dictionary definitions extract all words used in the database through data models and glossaries. The extracted words classify word types and types and define standard words in consideration of their purpose. When defining the standard word dictionary, care should be taken in the processing of the synonym and the homonym. For the defined standard word, the English name and the English abbreviation name are defined referring to the standardization principle.

1 is a block diagram schematically illustrating a configuration of a standard term recommendation apparatus according to an exemplary embodiment of the present application.

Referring to FIG. 1, the standard term recommendation apparatus 100 includes a database 110, a standard dictionary unit 121, a word substitution unit 130, a term recommendation unit 140, a term determination unit 150, and a data system. Study 160, may include a user input receiving unit 170.

According to an embodiment of the present disclosure, the standard term recommendation apparatus 100 may recommend a standard term when standardizing a standardized target database or a new term based on a standard dictionary generated from a database that has already been standardized. Can be. In addition, the standard term recommendation apparatus 100 may automatically separate terms to be standardized into nouns of word units using both morphological analysis and standard word dictionaries, and replace the words with standard words. have. In addition, the standard term recommendation apparatus 100 may variously combine the substituted standard words and calculate the similarity with terms existing in the standard term dictionary to recommend the standard term in ascending order.

According to the exemplary embodiment of the present application, since the standard term recommendation apparatus 100 calculates the similarity in phonological units, even if there is a notation error for a typo, the standard term recommendation apparatus 100 may correct a typo through a similar term recommendation.

According to an embodiment of the present disclosure, since the standard term recommendation apparatus 100 automatically separates word units using morphological analysis, it provides convenience to a user, and similarity when the separated word does not exist in the standard dictionary. It can recommend words registered in the standard word dictionary by calculating the, and can help you register easily in the standard word dictionary.

According to an embodiment of the present disclosure, the database 110 may include standardized term data. The database 110 may be a standardization target and include a space in which standardized terms are stored. The database 110 is an object for data standardization and a storage space for storing standard term recommendation results and converted standard terms. In other words, the database 110 includes a storage space to be standardized and a space where standardized terms are stored. In addition, the database 110 may include a space in which standard dictionaries used in the standard dictionary unit 120 are loaded.

According to an embodiment of the present disclosure, the standard dictionary unit 120 may include and manage a standard word dictionary 121, a standard term dictionary 122, and a standard domain dictionary 123. The standard dictionary unit 120 may search, register, and correct words, terms, domains, and the like of each dictionary. For example, the standard dictionary unit 120 may include a standard dictionary generated while being performed by another institution or company. The standard dictionary unit 120 may include a standard word dictionary 121, a standard term dictionary 122, and a standard domain dictionary 123 previously generated. In addition, the standard dictionary unit 120 may add a standard word, a standard term, and a standard domain generated during the standardization process performed by the term automatic recommendation apparatus 100.

FIG. 2 is a diagram illustrating a process of generating a standard word dictionary during standardization of a standard term recommendation apparatus according to an exemplary embodiment of the present disclosure.

According to an embodiment of the present disclosure, the standard word dictionary 121 may be a dictionary including a word (morpheme) that is a minimum semantic unit that can be used independently. Standard words are the smallest units of grammar that have certain meanings. They are the minimum units that make up entity names, attribute names, table names, column names, and so on. Standard words may include single words, compound words, and business abbreviations. The standard word dictionary 121 may store the standard word in association with the standard abbreviation. For example, if the standard word is a customer and the standard abbreviation is CLNT, the standard word dictionary 121 may store the word and the abbreviation in association with the customer-CLNT. In addition, the standard word dictionary 121 may store the kinsoku word and the standard word in association. For example, if the banned word is a guest and the standard word is a customer, the standard word dictionary 121 may store the banned word and the standard word as a guest-customer.

Generally, a word refers to a minimum unit of words, and a standard word refers to a minimum unit word having a certain meaning used in a company's business. In order to create a dictionary in which the standard words are defined, terms that are mixed and used are divided by word units. Split the word by collecting current terms. You can define a dictionary by refining words by arranging words such as synonyms and homonyms.

According to an embodiment of the present disclosure, the standard term dictionary 122 is defined by combining words as names for data elements managed by the logical / physical data model, that is, entity name / property name / table name / column name. It can be a dictionary. The standard term dictionary may include a plurality of standard terms defined by combining words.

3 is a view for explaining a process of generating a standard domain dictionary of the standardization process of the standard term recommendation apparatus according to an embodiment of the present application.

According to an embodiment of the present disclosure, the standard domain dictionary 123 may include a standard domain in association with standard terms. In addition, the standard domain dictionary 123 may include a standard domain associated with a standard term, data type, and domain name. For example, the standard domain dictionary 123 may include a standard domain linked by an ID when the standard term is a contract ID.

The standard domain dictionary stores information such as the type and length of data belonging to the group by grouping them in domain units representing a range of values satisfying an attribute. Domains are classified and typed in consideration of their business purpose, frequency of use, and physical characteristics of the data. Therefore, the domain can be identified only by checking the domain.

Referring to FIG. 3, in order to define a standard domain, an analysis of current terms is performed, and then, names and data types and lengths of respective domains are defined.

4 is a view for explaining a process of generating a standard term in the standardization process of the standard term recommendation apparatus according to an embodiment of the present application.

The standard term dictionary may constitute a standard term dictionary after all standards for words, domains, etc. are determined. The combination of standard words and domain classification determine the data type and length of the column corresponding to each term.

For example, referring to FIG. 4, in order to define a standard term dictionary, words included in the current term should be derived as standard words and terms defined therein. Once the terms are defined, the domain is finally defined and standardized.

According to an embodiment of the present disclosure, the standard dictionary unit 120 may store the standard terms generated by matching the current terms with the standard words included in the standard word dictionary in the standard term dictionary 122.

FIG. 5 is a diagram illustrating a screen made of a standard word dictionary of a standard term recommendation apparatus according to an embodiment of the present application stored in a database so as to be confirmed on a web.

Referring to FIG. 5, an example of a screen for checking a standard word dictionary dictionary is shown. Its main role is to search, register and modify standard words and physical names and meanings of English. When a word in the standard word dictionary is changed, the term consisting of the word is also changed. However, in order to configure the standard term automatic recommendation device, the lexical word used in the process of forming the standard word or the word before the change is also included in the dictionary to replace the standard word.

FIG. 6 is a diagram illustrating a screen of a standard domain dictionary of a standard term recommendation apparatus according to an embodiment of the present disclosure stored in a database so that the web can be viewed on a web.

Referring to FIG. 6, an example of a screen for checking a standard domain dictionary is shown. The main role is to define and verify the column's domain group and data type. In addition, you can check the valid values and the change history of the domain, and if you need to change the domain or register it, you can register and request the modification. If there is a word that includes the domain meaning, the domain can be recommended as well.

FIG. 7 is a diagram illustrating a screen of a standard term recommendation apparatus of a standard term recommendation apparatus according to an embodiment of the present disclosure stored in a database for checking on a web.

Referring to FIG. 7, an example of a screen for confirming a standard term dictionary is shown. The main role can mainly confirm the inquiry and registration of standard terms. In addition, it is possible to check the words and domains constituting the standard terminology, and to change the history applied to the standard term when the standard word is changed.

According to an embodiment of the present disclosure, the word substitute unit 130 may include a word separator 131 and a word matcher 132.

According to the exemplary embodiment of the present application, the word substitution unit 130 separates the morphemes of the first term 1 input to perform standardization, and converts a plurality of words included in the first term 1 into a standard word dictionary. It may be substituted with a standard word included in (121). The word substitution unit 13 may calculate similarity only for words that do not exist in the dictionary and recommend similar words, and if not, register the corresponding word in a standard word dictionary.

In addition, the word replacement unit 130 is intended to replace the words included in the terms to be standardized with the words registered in the standard word dictionary. The term to be standardized may be divided into word units through the word separator 131, and then replaced with a standard word registered in the standard word dictionary 121 in the word matcher 132. When there are a plurality of similar words, a plurality of (multiple) may be substituted. When the word to be standardized does not exist in the standard word dictionary 121, a new word may be added to the standard word dictionary 121.

In addition, the word substitution unit 130 may replace a set of separated words by matching standard words, suffixes, and similar words included in the standard word dictionary. The word substitution unit 130 may calculate similarity only for words that do not exist in the dictionary and recommend similar words, and if not, the word substitution unit 130 may register the corresponding word in the standard word dictionary 121. In this case, the word substitution unit 130 provides a word which is not included in the standard word dictionary 121 to the user terminal (not shown) through the data providing unit 160, and receives the user input received by the user input receiving unit 170. Based on the information, the user may request to register the word in the standard word dictionary 120.

If there is no word that matches the word included in the standard word dictionary 121, the word substitute unit 130 may divide the word into phonological units and calculate the similarity to recommend the most similar word. The word substitution unit 13 may perform a similarity operation between each separated word and a standard word using a Fuzzy Data Matching algorithm, but is not limited thereto. Fuzzy Data Matching Algorithm is an algorithm that performs matching between data using the result value calculated based on the editing distance (Levenshtein Distance).

The word substitution unit 130 may separate terms consisting of syllables into phonemes according to the Hangul alphabet by initial, neutral, and finality. For example, the Fuzzy Data Matching algorithm, which is one of AI-based algorithms, uses a method of calculating similarity based on shapes between words. Unlike English, which is the language on which the algorithm is based, in the case of Korean, since the alphabets of the Korean alphabet corresponding to the English alphabet are combined to form letters, the word substitution unit 130 solves the Hangul syllables like the alphabets to form the alphabet. By separating, the similarity can be calculated. For example, when calculating the similarity without phonological separation, 'gang' and 'ball' are completely different letters, but after phonological separation, 'ㄱㅏㅇ' and 'ㄱ ㅗㅇ' are similar letters with only the middle neutral. If the original term is Korean, it may be divided into phonemes according to the Hangul alphabet.

The word substitution unit 130 replaces a set or arrangement of separated words with standard words in the standard word dictionary 121, and if there is a case corresponding to a domain listed in the domain dictionary, the word replacement unit 130 may recommend the corresponding domain together. have. The word corresponding to the domain may be recommended after being combined last in the term combination.

According to the exemplary embodiment of the present application, even if the word substitution unit 130 is separated into phonological units and performs a similarity operation with the standard word, when there is no standard word matching the first term, the data providing unit 160 Through the corresponding word may be provided to the user terminal (not shown). The user may determine whether to add a non-matching first term shown in the user terminal (not shown) to the standard word dictionary. When the user input receiver 170 receives input information for adding a first term that does not match from the user terminal (not shown) to the standard word dictionary, the user input receiver 170 may provide the input information to the word substitute unit 130. . The word substitute unit 130 may provide additional request information to the standard word dictionary 121 as a first term that does not match the standard dictionary unit 120 based on the user input information. The standard dictionary unit 120 may store the first term that does not match, based on the user input information, in addition to the standard word dictionary 121.

That is, when there is a word that does not match, the word substitute unit 130 provides a mismatched word to the user terminal (not shown), and based on the input information received from the user terminal (not shown), You can ask to add it to a standard word dictionary.

8 is a diagram schematically illustrating a morpheme analysis of a standard term recommendation apparatus according to an embodiment of the present application.

A morpheme is the smallest unit that has a meaning. A morpheme analyzer separates a sentence or text into the smallest units and automatically determines the parts of speech of the morpheme. Because they are used to replace standard words, they are separated by morphemes in noun form only. Morphological analysis is the most basic technology of natural language processing and has been applied in various ways.

Referring to FIG. 8, the sentence in the box above, 'Ordered by the Korean Criminal Policy Institute,' is a morphological analysis target, and the results are divided by words and parts of speech. If you enter the sentence 'Order a project ordered by the Korean Criminal Policy Institute' into the code, the morpheme analyzer can analyze the sentence automatically and separate the morpheme and the part of speech. For example, the word breaker 131 may provide a separated word in association with a morpheme part-of-speech tag table to each separated word using a morpheme analysis technique. NNG can be a common noun, JKM can be a side shot, XSV can be a verb-derived suffix, ETD is a tubular malleable, JKO is a target investigation, and NNP can be a proper noun.

9 is a schematic diagram of an example when a word breaker of a standard term recommendation apparatus according to an embodiment of the present disclosure is performed.

According to an embodiment of the present disclosure, the word breaker 131 may separate the first term 1 into a plurality of words using a morpheme analysis technique. In addition, the word breaker 131 may be divided into a noun-type word using both a matching method using a word included in the standard word dictionary 121 and a method using a morphological analysis technique. The word breaker 131 may transmit the separated words to the word matcher 132 in the form of a set or an arrangement instead of each word. For example, the word breaker 131 uses the stemming technique of the mail order forwarding delivery address, which is the first term 1, to provide a mail / resident / relocation / delivery / address, mail / water / resident / relocation / delivery / It may be separated into words of each of the first terms 1, such as an address.

According to an embodiment of the present disclosure, the word breaker 131 is a standard word dictionary 121 when any one of the plurality of words included in the first term 1 is a word that does not exist in the standard word dictionary 121. Words that do not exist in can be separated into phonological units, similarity calculations are performed using AI-based algorithms, and similar words can be recommended. The AI based algorithm may be a Fuzzy Data Matching algorithm.

For example, the word breaker 131 converts' post 'to' ㅇ, TT,, ㅕ, when the word zip among the plurality of separated words does not exist in the standard word dictionary 121. Can be separated. In addition, the word breaker 131 may perform a similarity operation on the word separated by 'ㅇ, ㅜ, ㄷ, ㅕ, ㄴ' using an AI-based algorithm, and recommend the result as a similar word.

According to an embodiment of the present application, FIG. 9 may be a result of substituting a column name of 'mail residential transfer address' to a word separator configured by applying a morpheme analyzer. This is only an example and may be different from the actual column name. The word separator 131 divides the column names of the 'mail residential transfer address' based on the morpheme analyzer and the standard term dictionary, and separates the words by the number of various cases. The word breaker 131 divides the word set into the most appropriate set of the separated word sets. If there is no standard word matching in the word matcher 132, the word replacer 130 selects another word set. Match and replace with standard words.

FIG. 10 is a diagram schematically illustrating an example when a word matcher of the standard term recommendation apparatus according to the exemplary embodiment of the present application is performed.

According to an exemplary embodiment of the present disclosure, the word matcher 132 may match a set of a plurality of words separated by the word separator 131 with any one of the words included in the standard word dictionary 121. The word matcher 132 may match the set of words separated by the word separator 131 with at least one of standard words, banned words, and similar words included in the standard word dictionary 120.

For example, referring to FIG. 10, the mail / resident / previous / delivery / address 2 may be a set of words separated by the word separator 131. The word matcher 132 may match each classified word with at least one of standard words, banned words, and similar words included in the standard word dictionary 120. The first matching word that has passed through the word matcher 132 is a mail / resident / previous / delivery / address of the words separated from the word breaker 131, the delivery is matched to the delivery, and the address can be confirmed that the address is matched to the address. have. On the other hand, the second matching word that has passed through the word matcher 132 may be matched before the change to the mail / residential / before the change / delivery / address, it can be confirmed that the address is matched by the address. If the word is not substituted (changed), it may be a word included in the existing standard word dictionary 121. The plurality of word sets passing through the word matcher 132 may be matched to a plurality of words when there are similar words, instead of matching only one word.

According to one embodiment of the present application, referring to FIG. 10, a matching result for 'mail / resident / previous / delivery / address' inserted into the word matcher 132. The drawings are only examples, and the results may differ depending on the words registered in the standard word dictionary and the banned words. Substituted words are substituted for the word based on the standard word dictionary. In the case of the word without substitution, the word is already a standard word or is not listed in the standard word dictionary. For words that are not listed in the standard word dictionary, the similarity is calculated and the highest word is suggested to the user to correct the typo, and in the case of a new word other than the typo, the word may be added to the standard word dictionary 121.

11 is a view schematically showing an example of a task performed by the term recommendation unit of the standard term recommendation apparatus according to an embodiment of the present application.

According to an embodiment of the present disclosure, the term recommendation unit 140 combines a plurality of substituted standard words and uses the combined plurality of standard words, and includes at least one of a plurality of standard terms included in the standard term dictionary 122. Any one can be recommended as a standard term for the first term (1). In addition, the term recommendation unit 140 may recommend a standard term or a combination of standard words included in the standard term dictionary 122. For example, the term recommender 140 may generate a term by combining standard words substituted by the word substitute unit 130 and provide the term to a user terminal (not shown) through the data provider 160.

For example, the term recommendation unit 140 is a mail / residence / transfer / delivery / address, post / residential / transfer / delivery / address, post / residential / change / delivery / address, etc. substituted in the word substitution unit 130 A plurality of substituted standard words can be combined. The term recommender 140 may recommend standard terms generated by combining two or more words. For example, when there are five standard words substituted, a standard term generated from a combination of at least two words and at most five words may be recommended.

Referring to FIG. 11, the term recommendation unit 140 combines a plurality of substituted words, such as postal_delivery_address, postal_resident_previous_delivery_address, postal_resident_change_delivery_address, and the like. I can recommend them. The combination words shown in FIG. 11 are only an example, and there may be more combinations of words.

When a word is substituted in the word replacement unit 130, a problem may occur with respect to a standard word about a homonym, so that the word is replaced with not a single word but the term recommender 140 replaces the word. Each of the terms used may be generated and provided to the user terminal (not shown) through the data provider 160. In addition, the term recommendation unit 140 may recommend similar standard terms existing in the standard term dictionary among words provided.

According to an embodiment of the present disclosure, the term recommender 140 performs a similarity operation using an AI-based algorithm, using a plurality of standard words combined with standard terms included in the standard term dictionary unit 122. The standard term of the first term (1) can be recommended. The term recommender 140 may perform a similarity operation of a plurality of standard words combined with a standard term using a Fuzzy Data Matching algorithm, but is not limited thereto. Fuzzy Data Matching Algorithm is an algorithm that performs matching between data using the result value calculated based on the editing distance (Levenshtein Distance). The term recommendation unit 140 may recommend the standard term of the first term 1 in the order of high similarity based on the result of the similarity calculation performed using an AI based algorithm. The term recommendation unit 140 may recommend only standard terms, which are equal to or greater than a preset result, among similarity calculation results performed using an AI-based algorithm, as standard terms of the first term 1.

According to an embodiment of the present application, Figure 11 is a representation of the term recommendation. This is only an example, and may not be an actual standard terminology and different results may be obtained in the standard terminology recommendation process. The set of words passing through the word substitution unit 130 is converted into a standard term form through the term recommender 140. In the process, a term having high similarity is recommended among terms in the standard term dictionary. It may also recommend terms that are a concatenation of words, or a change of order.

12 is a diagram schematically showing an example of a task performed by the term decision unit of the standard term recommendation apparatus according to an exemplary embodiment of the present application.

According to one embodiment of the present application, the term determining unit 150 selects any one of the standard terms recommended in the term recommending unit as the first standard term 3 of the first term 1, and the standard domain dictionary 123 At least one of the plurality of standard domains included in) can be determined as the first standard domain of the first standard term (3). For example, the term determining unit 150 may select any one of the plurality of standard terms recommended by the term recommending unit 140 as the first standard term of the first term 1. For example, the term determination unit 150 may change the housing_ among the mail_delivery_address, mail_resident_previous_delivery_address, residence_change_mail_delivery_address recommended by the term recommendation unit 140. Mail_delivery_address may be selected as the first standard term (3).

In addition, the term determination unit 150 may determine the standard domain associated with the first standard term 3 as the standard domain of the first standard term 3. The term determining unit 150 may receive a standard domain associated with the first standard term 3 from the standard domain dictionary 123 to determine the first standard domain of the first standard term 3. The standard domain dictionary 123 may include a first standard domain associated with the first standard term. The standard domain dictionary 123 may include standard domains associated with standard terms.

For example, the term determining unit selects the residence_change_mail_delivery_address as the first standard term (3), the standard domain ID address associated with the first standard term (3), and the data type VARCHAR (150). The phosphor domain may be determined as the first standard domain.

The term determiner 150 may select a term recommended by the user based on the term. The user may select a standard term which is determined to be the most appropriate based on the term recommended by the term recommender 140 and store it in the database 110.

In addition, the term determination unit 150 may directly modify or input a standard term. If all of the recommended standard terms are not the term desired by the user, or if the user wants to save only part of the standard term, the user may directly input or modify the standard term and save the term.

According to one embodiment of the present application, Figure 12 will illustrate the term determination. This is an example and may not be the standard terminology and standard domain in use. If you want to change the type of domain that comes after the term 'Residual_Change_Mail_Delivery_Address' among the terms recommended by the term recommender, you can select the desired domain after selecting the recommended term. The domain is limited to domains registered in the standard domain dictionary.

According to an embodiment of the present disclosure, the data provider 160 may provide a selection item related to input information for term standardization to a user terminal (not shown). The user may check an item related to the term standardization displayed on a user terminal (not shown) and select a correction item. For example, the selection item related to the term standardization may include a selection related to identification and correction according to a request provided by the standard dictionary unit 120, the word substitution unit 130, the term recommendation unit 140, and the term determination unit 150. It can contain an item.

According to an embodiment of the present disclosure, the user input receiving unit 170 may receive input information for term standardization from a user terminal (not shown). In other words, the user input receiver 170 may receive input information for standardizing terms selected by a user from a user terminal (not shown). The user input receiver (not shown) may provide corresponding information to each unit (unit) that has requested a selection item related to input information for term standardization. For example, the input information for term standardization may be user input information on an item provided from a term input menu, a standard word selection menu, a standard term selection menu, a standard domain selection menu, and the like.

According to an embodiment of the present disclosure, the term automatic recommendation apparatus 100 may provide a selection menu required for term standardization to a user terminal (not shown). For example, a user terminal (not shown) may download and install an application program provided by the term standardization apparatus 100, and a term standardization menu may be provided through the installed application. It may include, but is not limited to, a term input menu, a standard word selection menu, a standard term selection menu, a standard domain selection menu, etc. required for term standardization.

The term automatic recommendation apparatus 100 may include any type of server, terminal, or device that transmits and receives data, content, and various communication signals with a user terminal (not shown) through a network, and has a function of data storage and processing. have.

A user terminal (not shown) is a device interworking with the term automatic recommendation apparatus 100 through a network, for example, a smartphone, a smart pad, a tablet PC, a wearable device, and the like, and personal communication (PCS). System, Global System for Mobile Communication (GSM), Personal Digital Cellular (PDC), Personal Handyphone System (PHS), Personal Digital Assistant (PDA), International Mobile Telecommunication (IMT) -2000, Code Division Multiple Access (CDMA)- 2000, W-Code Division Multiple Access (W-CDMA), wireless broadband devices such as Wibro (Wireless Broadband Internet) terminals, and fixed terminals such as desktop computers and smart TVs.

Examples of a network for sharing information between the automatic recommendation device 100 and a user terminal (not shown) include a 3rd Generation Partnership Project (3GPP) network, a Long Term Evolution (LTE) network, a 5G network, and a World Interoperability for Microwave (WIMAX). Access (Network), Wired / Wireless Internet, Local Area Network (LAN), Wireless Local Area Network (WLAN), Wide Area Network (WAN), Personal Area Network (PAN), Bluetooth Network, Wifi Network, NFC (Near Field Communication) network, satellite broadcasting network, analog broadcasting network, DMB (Digital Multimedia Broadcasting) network, etc. may be included, but is not limited thereto.

Hereinafter, based on the details described above, the operation flow of the present application will be briefly described.

13 is a flowchart illustrating a standard term automatic recommendation method for big data standardization according to an embodiment of the present application.

The standard term automatic recommendation method for big data standardization illustrated in FIG. 13 may be performed by the term automatic recommendation apparatus 100 described above. Therefore, even if omitted below, the description of the term automatic recommendation apparatus 100 may be equally applicable to the description of the standard term automatic recommendation method for big data standardization.

In operation S131, the term automatic recommendation apparatus 100 may receive the first term 1 to perform standardization. For example, the term automatic recommendation apparatus 100 may input a mail residence transfer delivery address (first term).

In operation S132, the term automatic recommendation apparatus 100 uses the database 110 including the standardized term data and the standard dictionary unit 120 including and managing a standard word dictionary, a standard English dictionary, and a standard domain dictionary. In order to perform standardization, the morpheme of the first term 1 received may be separated. For example, the term automatic recommendation apparatus 100 may separate the received mail residence transfer address (first term) into morpheme units. For example, the term automatic recommendation apparatus 100 may divide the first term into morpheme units by mail / resident / previous / delivery / address.

In operation S133, the term automatic recommendation apparatus 100 may replace a plurality of words included in the separated first term 1 with standard words included in the standard word dictionary 121. For example, the term automatic recommendation apparatus 100 may replace a plurality of words included in the separated first term with standard words included in the standard word dictionary. For example, the term automatic recommendation apparatus 100 may replace 'address' with 'address' among a plurality of words of mails / residential / previous / delivery / address.

In operation S134, the term automatic recommendation apparatus 100 may combine a plurality of words included in the substituted first term 1. For example, the term automatic recommendation apparatus 100 may variously combine mails / residents / previous / delivery / addresses which are a plurality of words included in the substituted first term. For example, the term automatic recommendation apparatus 100 may combine a plurality of words such as postal / residential, residential / modified / postal / delivery / address.

In operation S135, the term automatic recommendation apparatus 100 may recommend at least one of the plurality of standard terms included in the standard term dictionary 122 as the standard term of the first term using the combined plurality of standard words. have. For example, the term automatic recommendation apparatus 100 may recommend a mail residence transfer address (first term).

In operation S136, the term automatic recommendation apparatus 100 may select any one of the recommended standard terms as the first standard term of the first term. For example, the term automatic recommendation apparatus 100 may select the first standard term of the mail dwelling delivery address (first term) as a residence_change_mail_delivery_address.

In operation S137, the term automatic recommendation apparatus 100 may determine at least one of the plurality of standard domains included in the standard domain dictionary 123 as the first standard domain of the recommended first standard term. For example, the term automatic recommendation apparatus 100 may determine a first standard term, which is a residence_change_mail_delivery_address, as a domain name address and a first standard domain of data type VARCHAR 150.

In the above description, steps S131 to S137 may be further divided into additional steps or combined into fewer steps, according to an embodiment of the present disclosure. In addition, some steps may be omitted as necessary, and the order between the steps may be changed.

The standard term automatic recommendation method for standardizing big data according to an exemplary embodiment of the present disclosure may be implemented in the form of program instructions that may be executed by various computer means and recorded in a computer readable medium. The computer readable medium may include program instructions, data files, data structures, etc. alone or in combination. Program instructions recorded on the media may be those specially designed and constructed for the purposes of the present invention, or they may be of the kind well-known and available to those having skill in the computer software arts. Examples of computer-readable recording media include magnetic media such as hard disks, floppy disks, and magnetic tape, optical media such as CD-ROMs, DVDs, and magnetic disks, such as floppy disks. Magneto-optical media, and hardware devices specifically configured to store and execute program instructions, such as ROM, RAM, flash memory, and the like. Examples of program instructions include not only machine code generated by a compiler, but also high-level language code that can be executed by a computer using an interpreter or the like. The hardware device described above may be configured to operate as one or more software modules to perform the operations of the present invention, and vice versa.

In addition, the standard term automatic recommendation method for standardizing big data may be implemented in the form of a computer program or an application executed by a computer stored in a recording medium.

The above description of the present application is intended for illustration, and it will be understood by those skilled in the art that the present invention may be easily modified in other specific forms without changing the technical spirit or essential features of the present application. Therefore, it should be understood that the embodiments described above are exemplary in all respects and not restrictive. For example, each component described as a single type may be implemented in a distributed manner, and similarly, components described as distributed may be implemented in a combined form.

The scope of the present application is indicated by the following claims rather than the above description, and it should be construed that all changes or modifications derived from the meaning and scope of the claims and their equivalents are included in the scope of the present application.

100: term automatic recommendation device
110: database
120: standard dictionary
130: word substitution
140: term recommendation
150: term determination unit

Claims (6)

In the standard term automatic recommendation device for big data standardization,
A database comprising normalized terminology data;
A standard dictionary unit including and managing a standard word dictionary, a standard term dictionary, and a standard domain dictionary;
A word substitution unit that separates the morphemes of the first term received to perform standardization, and replaces a plurality of words included in the first term with standard words included in the standard word dictionary;
A term recommender for recommending at least one of a plurality of standard terms included in the standard term dictionary as a standard term of the first term by combining the plurality of substituted standard words and using the combined plurality of standard words ; And
Selecting one of the standard terms recommended in the term recommendation unit as the first standard term of the first term, and selecting at least one of a plurality of standard domains included in the standard domain dictionary as the first standard term of the first standard term A term decision unit that determines the standard domain,
Including,
The word substitution is
A word separator that separates the first term into a plurality of words using a morpheme analysis technique; And
A word matcher for matching a set of words separated by the word breaker with any one of words included in the standard word dictionary,
The word breaker,
When any one of the plurality of words included in the first term is a word that does not exist in the standard word dictionary, words not existing in the standard word dictionary are separated into phonological units, and an artificial intelligence based algorithm is used. Suggest a similar word based on a result of performing a similarity operation between the separated word and the standard word,
Words that do not exist in the standard word dictionary are registered in the standard word dictionary,
The term recommendation unit,
Using a plurality of standard words combined with standard terms included in the standard term dictionary, a similarity operation is performed by using an AI-based algorithm, and standard terms of a predetermined similarity or more are converted into standard terms of the first term. Recommended,
The standard domain dictionary includes a standard domain associated with a standard term, data type, domain name,
The word substitution portion,
Substituting the plurality of separated words with standard words in the standard word dictionary, and if the domain is listed in the domain dictionary, the recommended domain together,
The term determination unit,
In consideration of the recommended domain, determining a standard domain associated with the first standard term as a standard domain of the first standard term. delete delete delete The method of claim 1,
A data providing unit providing selection items related to input information for term standardization to a user terminal; And
And a user input receiver for receiving input information for term standardization from the user terminal. In the standard term automatic recommendation method for big data standardization,
Receiving a first term to perform standardization;
Separating the morphemes of the inputted first terms for performing standardization, using a standard dictionary part including and managing a database including standardized term data and a standard word dictionary, a standard term dictionary and a standard domain dictionary;
Replacing a plurality of words included in the separated first term with standard words included in the standard word dictionary;
Combining a plurality of words included in the substituted first term;
Recommending at least one of the plurality of standard terms included in the standard term dictionary as the standard term of the first term using the combined plurality of standard words;
Selecting any one of the recommended standard terms as a first standard term of the first term; And
Determining at least one of a plurality of standard domains included in the standard domain dictionary as the first standard domain of the recommended first standard term,
Including,
Substituting the standard word,
Separating the first term into a plurality of words using a morphological analysis technique; And
Matching a set of words separated in a word breaker to any one of the words contained in the standard word dictionary,
Separating into the plurality of words,
When any one of the plurality of words included in the first term is a word that does not exist in the standard word dictionary, words not existing in the standard word dictionary are separated into phonological units, and an artificial intelligence based algorithm is used. Suggest a similar word based on a result of performing a similarity operation between the separated word and the standard word,
Words that do not exist in the standard word dictionary are registered in the standard word dictionary,
Recommend in the standard terminology,
Using a plurality of standard words combined with standard terms included in the standard term dictionary, a similarity operation is performed using an artificial intelligence-based algorithm, and the standard term of the first term is recommended for a standard term equal to or greater than a preset similarity. But
The standard domain dictionary includes a standard domain associated with a standard term, data type, domain name,
Substituting the standard word,
Substituting the plurality of separated words with standard words in the standard word dictionary, and if the domain is listed in the domain dictionary, the recommended domain together,
Determining the first standard domain,
Considering the recommended domain, determining a standard domain associated with the first standard term as the standard domain of the first standard term.

Images