KR101806452B1 - Method and system for managing total financial information - Google Patents

Abstract

An article automatic mapping method based on text mining includes: an article automatic mapping apparatus receiving technical document data for a plurality of articles from a database; a step for extracting a plurality of words from the article automatic mapping apparatus on the technical document data; A step of generating a keyword dictionary of a company on the basis of a plurality of words from which the mapping device is extracted, a step of calculating a TF-IDF weight reflecting an association analysis between the order product and the existing product based on the keyword dictionary , The step of calculating the cosine similarity degree of the ordered product and the existing product based on the TF-IDF weight, and the automatic product mapping device may generate the search result based on the cosine similarity.

Description

Technical Field [0001] The present invention relates to a method and apparatus for automatically mapping an article based on text mining,

The present invention relates to a method and apparatus for providing product information, and more particularly, to a method and apparatus for automatic product mapping based on text mining.

The text mining technique is a technique to convert structured data after analyzing unstructured data in natural language to obtain useful information. Text mining can be applied to Internet search engines or search systems in the reading room, and documents can be classified according to existing categories. If there is no information about the dictionary document, it may be possible to find meaning or find the author by clustering similar documents among the document clusters. Document clusters apply to intellectual structure studies in the discipline. The intellectual structure of the secretarial can be grasped through document clustering using text mining, and the documents can be expressed in two dimensions by using Multidimensional Scaling.

Enterprises can use text mining techniques to obtain more detailed information about their customers, so they can use more information in making decisions. In addition, researchers can easily retrieve or analyze documents of interest to get new research results. Text mining can be used for preliminary investigation and analysis for questionnaire preparation. By using text mining results, we can create more realistic and faithful questionnaires, and the results of these questionnaires can be more objective data.

KR 10-2009-0124469

One aspect of the present invention provides a method for automatic product mapping based on text mining.

Another aspect of the present invention provides an apparatus for performing an automatic product mapping method based on text mining.

상기 N은 기존 상품의 수이고, 상기 DF_t는 단어 t가 포함된 기존 상품의 수이고,
상기 AW_t,d는 아래의 수학식 2를 기반으로 결정되고,
<수학식 2>

상기 ATF_t,d는 연관 분석 가중치와 TF 가중치를 합산한 수학식 3을 기반으로 결정되고,
<수학식 3>

상기 A는 연관 분석 가중치이고, 상기 TF_t,d는 TF 가중치이고, 상기 TF는 복수의 상품들 각각에서 특정 단어가 포함된 빈도수이다.According to an aspect of the present invention, there is provided a method for automatic product mapping based on text mining, the method comprising: receiving technical document data for a plurality of products from a database; The automatic product mapping apparatus extracting a plurality of words from the technical document data; The product automatic mapping device generating a keyword dictionary of a business based on the extracted plurality of words; The automatic product mapping apparatus calculating a TF (inverse document frequency) -IDF (inverse document frequency) weight based on the linkage analysis of the order product and the existing product based on the keyword dictionary; The product automatic mapping apparatus calculating a cosine similarity degree of an ordered product and an existing product based on the TF-IDF weight; And generating a search result based on the cosine similarity by the automatic goods mapping apparatus, wherein the keyword dictionary is generated through morphological analysis of technical document data on the plurality of products related to a specific company, The association analysis may be performed such that when the descriptive document data is a transaction unit, the lift value indicates a probability that the first word and the second word are simultaneously present in the descriptive document data, the probability that the first word exists in the descriptive document data, Wherein the TF-IDF weight is determined based on a probability that a second word is present in the descriptive document data, and wherein the TF-IDF weight reflecting the association analysis is a weighted sum of the first word, Generates a document-keyword matrix, and adds itemets in the first document-keyword matrix Generating a second document-keyword matrix in which the association analysis is reflected by using a weight and a frequency of the item set, as follows: < EMI ID = 2.0 > And generating a third document-keyword matrix including the TF-IDF weight by using Equation (1), which reflects the association analysis based on the matrix.
&Quot; (1) &quot;

Where N is the number of existing products, DF _t is the number of existing products that contain the word t,
The AW _{t, d} is determined based on the following equation (2)
&Quot; (2) &quot;

The ATF _{t, d} is determined based on Equation (3) which is the sum of the association analysis weight and the TF weight,
&Quot; (3) &quot;

A is the association analysis weight, TF _{t, d} is the TF weight, and TF is the frequency at which a specific word is included in each of the plurality of products.

delete

Further, the lift value is determined based on the following equation,

&Lt; Equation &

Supp (XUY) is a probability that the first word and the second word are simultaneously present in the descriptive document data at the same time, Supp (X) is a probability that the first word is present in the descriptive document data, May be the probability that the second word is present in the descriptive document data.

delete

delete

delete

delete

delete

delete

delete

delete

delete

delete

delete

상기 N은 기존 상품의 수이고, 상기 DF_t는 단어 t가 포함된 기존 상품의 수이고,
상기 AW_t,d는 아래의 수학식 2를 기반으로 결정되고,
<수학식 2>

상기 ATF_t,d는 연관 분석 가중치와 TF 가중치를 합산한 수학식 3을 기반으로 결정되고,
<수학식 3>

상기 A는 연관 분석 가중치이고, 상기 TF_t,d는 TF 가중치이고, 상기 TF는 복수의 상품들 각각에서 특정 단어가 포함된 빈도수이다.According to another aspect of the present invention, there is provided an article automatic mapping apparatus based on text mining, the article automatic mapping apparatus based on text mining includes a document collection module configured to receive technical document data on a plurality of articles from a database; A word extraction module configured to extract a plurality of words from the technical document data; A data mining module that is implemented to generate a business keyword dictionary based on the extracted plurality of words; A text mining module that is configured to calculate a TF (inverse document frequency) weighting factor reflecting association analysis for each order product and existing product based on the keyword dictionary; And a similarity measure module that is configured to calculate a cosine similarity of an ordered product and an existing product based on the TF-IDF weight to generate a search result, wherein the keyword dictionary includes a description of the plurality of products And the association analysis is generated by the morphological analysis on the document data. When the descriptive document data is a transaction unit, the lift value is a probability that the first word and the second word are simultaneously present in the descriptive document data at the same time, One word is determined to be present in the descriptive document data and the probability that the second word is present in the descriptive document data, and the TF-IDF weight reflecting the association analysis is determined based on the keyword dictionary, Generates a first document-keyword matrix for each ordered product and existing product filtered, The method includes extracting an itemet from a first document-keyword matrix, and using the weighted value and the frequency of the item set for the keywords constituting the item set, Generating a document-keyword matrix and generating a third document-keyword matrix including the TF-IDF weight using the following Equation 1 reflecting the association analysis based on the second document-keyword matrix .
&Quot; (1) &quot;

Where N is the number of existing products, DF _t is the number of existing products that contain the word t,
The AW _{t, d} is determined based on the following equation (2)
&Quot; (2) &quot;

The ATF _{t, d} is determined based on Equation (3) which is the sum of the association analysis weight and the TF weight,
&Quot; (3) &quot;

A is the association analysis weight, TF _{t, d} is the TF weight, and TF is the frequency at which a specific word is included in each of the plurality of products.

delete

Further, the lift value is determined based on the following equation,

&Lt; Equation &

Supp (XUY) is a probability that the first word and the second word are simultaneously present in the descriptive document data at the same time, Supp (X) is a probability that the first word is present in the descriptive document data, May be the probability that the second word is present in the descriptive document data.

delete

delete

delete

delete

delete

delete

delete

delete

delete

delete

delete

The method and apparatus for automatically mapping an article based on text mining according to an embodiment of the present invention can provide convenience by providing a user with a more intuitive experience on the search result and promptly confirm the search result, . In addition, by expanding to automatic product mapping, the user can perform order product mapping for various products automatically by simple command. Instead of searching through the words separated by spaces like the existing method, the whole product name is analyzed automatically, and the product is analyzed and the user shows the goods that he / she wants. .

1 is a conceptual diagram illustrating an automatic product mapping apparatus according to an embodiment of the present invention.
FIG. 2 is a table showing values of support, confidence, and lift indicating the inter-word association measure included in a document according to an embodiment of the present invention. FIG.
FIGS. 3 and 4 are conceptual diagrams illustrating results of similarity measurement according to an embodiment of the present invention.
5 is a flowchart illustrating a method of automatic product mapping based on text mining according to an embodiment of the present invention.

The following detailed description of the invention refers to the accompanying drawings, which illustrate, by way of illustration, specific embodiments in which the invention may be practiced. These embodiments are described in sufficient detail to enable those skilled in the art to practice the invention. It should be understood that the various embodiments of the present invention are different, but need not be mutually exclusive. For example, certain features, structures, and characteristics described herein may be implemented in other embodiments without departing from the spirit and scope of the invention in connection with an embodiment. It is also to be understood that the position or arrangement of the individual components within each disclosed embodiment may be varied without departing from the spirit and scope of the invention. The following detailed description is, therefore, not to be taken in a limiting sense, and the scope of the present invention is to be limited only by the appended claims, along with the full scope of equivalents to which such claims are entitled, if properly explained. In the drawings, like reference numerals refer to the same or similar functions throughout the several views.

Hereinafter, preferred embodiments of the present invention will be described in more detail with reference to the drawings.

Integrated online / offline inventory management system (hereinafter referred to as an inventory management system) can perform mapping work with existing products managed by the inventory management system to manage products ordered online, such as coupons and auction sites. This mapping function with existing products is a function for automating inventory management and is often referred to as custom product mapping. However, most of these methods of order product mapping are performed by simply cutting out words using sentence spacing or special character distinction and simply comparing words. In this case, words with similar meanings are similar but words with different composition can not be found. Of course, if mistakes are made in typing or omission of words, the performance of the system may be drastically reduced. In addition, each vendor does not reflect the characteristics of various product names, and the performance of inventory management functions will gradually decrease over time. For example, since the product name of an IT company is managed by a product code, English may be a keyword that distinguishes a product. Other companies, however, may not be as important in English as they are in product names. In the big data environment system that manages many types of product data, the conventional method rather than the semantic-based approach through the data analysis has a limitation in quickly processing the work.

Hereinafter, in the embodiment of the present invention, it is possible to automate custom product mapping by text mining products out of existing custom product word mapping system of simple word search. A company-specific keyword dictionary can be constructed using a machine learning model that accumulates existing product data and learns data patterns and flows. In addition, association rule analysis, which is a data mining technique, is applied to document type data to analyze the relationship and pattern of data, thereby extracting keywords and assigning weights. Then, by using the technique of text mining, it is possible to structure the products including the unstructured text and to measure the similarity between the products. By exposing products with high similarity in order, they can be used for a recommendation system, and then automatic mapping can be performed. Thus, in the embodiment of the present invention, a semantic-based product mapping system is proposed instead of a simple character comparison through a probability / statistical approach.

1 is a conceptual diagram illustrating an automatic product mapping apparatus according to an embodiment of the present invention.

In Fig. 1, the operation of each module (or constituent unit) constituting the automatic goods mapping apparatus is specifically disclosed.

1, the automatic product mapping apparatus may include a document collection module 100, a word extraction module 110, a data mining module 120, a text mining module 130, and a similarity measurement module 140 .

The document collection module 100 may be implemented to receive or download data for technical documents from a database. Technical documents may include information on existing products and order products registered by the companies (product name, product specification, etc.).

The word extraction module 110 may first perform a standardization operation on each of the documents. Standardization may include at least one of the following: removing special characters, removing abbreviations, and converting uppercase to lowercase. That is, the converted document can be subjected to morphological analysis as a document in which the standardization work has been performed. After standardization of the document, morphological analysis can proceed to extract words from the document. Through morphological analysis, the words extracted from each document and the frequency of the words can be calculated. Through this process, a document-word matrix including the frequency TF (Term frequency) included in each of a plurality of documents of each of a plurality of words can be generated. For example, the document-word matrix can be expressed as Equation 1 below.

&Quot; (1) &quot;

In Equation (1), m (m is a natural number of 1 or more) may be a sum of an existing product and an ordered product. That is, the word extraction module can perform morphological analysis on m products. Also, n (n is a natural number greater than or equal to 1) may mean the number of morphemes that are the smallest semantic units in sentences derived as a result of morpheme analysis. At this time, it can be determined that the same part of the morpheme is different for the same word. Thus, the document-word matrix X can be represented by an mxn matrix.

The data mining module 120 can perform data mining on existing products registered by the vendor. First, you can build a company's keyword dictionary based on a large number of product data accumulated in the business. In other words, the word extraction module 110 can accumulate word lists of company products analyzed and store different types of data for each company. Accordingly, in the business keyword dictionary construction, it is possible to know the values of the words that can appear in the company's products and the number of the products (documents) in which the respective words appear (DF: Document frequency). Through this process, you can see which words are used by the vendor. In other words, the flow of data can be known by analyzing the existing data such as what words are common in products or what are the key words of the company.

A keyword dictionary of different types can be constructed for each company by accumulating word lists extracted by standardized and morphological analysis of company products analyzed by word extraction module. Accordingly, in the business keyword dictionary construction, the values for the words that can appear in the company's products and the number of products (documents) in which the respective words appear (DF: Document frequency) can be known.

The data mining module 120 may then perform association rule analysis. In data mining, an association relationship can refer to a similarity or pattern existing between items. Association rules in data mining The basic concept is that if there are n transaction sets in the database and all itemsets contained in this transaction are I, then X⊂I, Y ⊂ I, and X ∩ Y = "means that when an event of condition X occurs, event Y occurs, and it can be expressed as" X → Y ". When applying this to the literature, it can be seen that {a} and {b} are related terms if both terms {a} and {b} appear simultaneously in various documents. A transaction is a unit that stores generated data. It is a concept that can have several items. When applied to a document group, a document can be a transaction. The Apriori algorithm can be performed to analyze this association rule. As a result, support, confidence and lift can be used as a measure of relevance between words.

support means the number of occurrences of a particular item in the number of specific transactions and can be calculated based on Equation 2 below.

&Quot; (2) &quot;

In Equation (2), X is an itemset and T is the number of transactions. Confidence can also refer to the percentage of transactions involving Y for transactions involving X. This confidence is an indicator of the accuracy of the rule X → Y, and high confidence can make accurate predictions possible. confidence can satisfy Equation (3) below.

&Quot; (3) &quot;

Finally, Lift is a scale for expressing the correlation between X and Y, and the formula is expressed by Equation (4) below.

&Quot; (4) &quot;

The numerator means the probability that X and Y exist simultaneously in the transaction, and the denominator can mean the probability that each X, Y exists independently in the transaction. When Lift is 1, X and Y are independent of each other. If X is greater than 1, X and Y are positive. If X is less than 1, X and Y are negative. .

Specifically, lift (lift) can mean the ratio of lift (x-> y) to the case where the word y is associated with the word x rather than when it appears randomly. That is, it may be a measure expressing how much it has been improved to predict the right item of the association rule.

Therefore, if support is low, confidence is likely to be a useful association rule. In conclusion, confidence can indicate how often the results can be applied to the conditional, while support indicates how reliable the rule itself is. Using these measures, keywords with high relevance in products can be extracted and weights can be given.

According to the embodiment, the document-word matrix generated above can be converted into a Boolean model to generate transaction matrix data to be used in the algorithm. Association rules can be extracted by performing the Apriori algorithm with the minimum support defined by the user.

FIG. 2 is a table showing values of support, confidence, and lift indicating the inter-word association measure included in a document according to an embodiment of the present invention. FIG.

Referring to FIG. 2, for example, support values for a word set of '500', 'container' and 'ml' may be 0.097142857, a confidence value of 1, and a lift value of 2.611940299.

Specifically, an itemset is a combination of the leading and trailing of a rule (meaning a set of items with no common elements), and may refer to the entire preceding '500', 'container', and the trailing 'ml' . Here, the association rule can mean that the word 'ml' appears together when the words '500' and 'container' appear. A measure that numerically represents the degree of the word set or itemset at that time may be support, confidence, or lift.

The text mining module 130 may first perform a keyword selection operation. Keyword selection is to select only the core keywords in the product (document) separated by the minimum semantic unit through morphological analysis. That is, the document-keyword matrix can be generated from the document-word matrix by referring to the keyword dictionary of the company constructed by the data mining module 120. The keyword dictionary already accumulated the existing data of the company and learned it according to the company. Therefore, a keyword that is not in the keyword dictionary is determined to be meaningless, and the weight of the word in the document-word matrix can be set to 0 and excluded from the target. As mentioned above, keyword selection is a process of removing obstructed elements (words) when performing product mapping. Thus, a document-keyword matrix X1 filtered by keywords from the document-word matrix X can be generated.

The text mining module 130 may then weight the associated words in the document using association rule analysis. First, the user can define a specific support value and execute the association analysis algorithm. As a result, a plurality of itemsets satisfying the minimum support can be obtained. Then weights can be given to the keywords that make up each itemset. By doing so, weights are given to the selected keywords, and the analysis of the products can be performed more realistically. In the embodiment of the present invention, the association analysis weight (A) is defined as the itemset frequency of the keyword.

The text mining module 130 may then reflect the association analysis weights in the TF weights. Here, the term frequency (TF) may mean the frequency of a particular word in each of the products. The larger the TF value, the more important the word is. In addition, the association analysis weight is also a number that is considered to be an important word among many words, so it can be said that it is proportional to the importance of the word as TF. In fact, the TF value in product data is easy to be in Boolean form, so the influence of TF can be reduced. In this data environment, the correlation is strengthened by the numerical value of the TF, so that weights can be given more finely. Therefore, the TF weight value that reflects the association analysis can be calculated as shown in Equation (4) below.

Equation (5)

In Equation (5), A may mean the association analysis weight.

&Quot; (6) &quot;

In Equation (6), TF reflecting the association analysis result can take a log value for scale adjustment. Therefore, a document-keyword matrix X2 in which the association analysis result is reflected from the document-keyword matrix X1 can be generated.

Finally, the TF-IDF weights reflecting the association analysis can be calculated. In this case, DF (document frequency) means the number of documents in which a specific word appears, and inverse DF can be defined as IDF (inverse document frequency). The larger the DF value, the more likely the word is a common word and the less important the word. That is, as the IDF value increases, the corresponding word can be considered as an important word. Equation (7) below is the IDF equation.

&Quot; (7) &quot;

In Equation (7), N means the number of documents of the company to be analyzed, i.e., the number of existing products, and may mean the number of documents including the word t. IDF can also take log values for scaling. As a result, the TF-IDF weights that are reflected in the association analysis can be assigned high values to words that appear frequently in a particular product and not frequently appear in the overall product, as well as the results of association analysis, High TF-IDF weights can also be reflected in these higher words. Therefore, the association rule TF-IDF, which is the weight of the TF-IDF reflecting association analysis, can satisfy the following equation.

&Quot; (8) &quot;

A document-keyword matrix X3 to which ATI is assigned from the document-keyword matrix X2 can be generated using Equation (8). Through this process, the keyword of the corresponding product can be extracted and a weight for the keyword can be given to each keyword. This is the process by which unstructured data is structured, and the similarity can be measured between products using this figure.

The similarity measurement module 140 can measure similarity between an ordered product and an existing product. For similarity measure, cosine similarity which is one of the similarity measure of vector space model can be used. This is because it is possible to measure the distance in a plurality of many dimensional spaces if the condition that the cosine similarity satisfies the condition of the positive space. When the vector values of the attributes q and d are respectively given, the cosine similarity can be expressed as a scalar product and magnitude of a vector as shown in Equation (9) below.

&Quot; (9) &quot;

In Equation (9), q and d vectors denote an order product and an existing product, respectively, and keyword weights in the document can be used. That is, q _i can be regarded as a weight of the product corresponding to the i-th column in the document-keyword matrix X3. Also, cosine similarity can be seen as one way of normalizing the document length when comparing documents. Therefore, the calculated similarity is expressed as a value from 0 to 1 because the weight of the keyword can not be negative, and the angle between the two keyword weights can not exceed 90 degrees. When 0 is independent from each other, 1 can mean the case where they are completely equal to each other.

Table 1 below is an example of a value obtained by normalizing the weight of the keyword and the weight of the keyword.

<Table 1>

In Table 1, it is possible to normalize the weight of a keyword for an ordered product and an existing product, respectively, and then calculate an inner product of the vectors, as shown in Equation (10).

&Quot; (10) &quot;

A result of 0.8 in Equation 10 can mean the similarity of the two products. Thus, it is possible to structure the ordered product and the existing product, measure the similarity, and recommend the products that the user desires in order of high similarity.

FIGS. 3 and 4 are conceptual diagrams illustrating results of similarity measurement according to an embodiment of the present invention.

In Fig. 3, the keywords of the ordered product and the existing product are all the same. The keyword of the order product is '500 sprayer 500 ml sprayer', and the keyword of existing product may be '500 sprayer 500 ml sprayer'. That is, the common keyword frequency of the ordered product and the existing product is 6, and the degree of similarity can be 1.

In Fig. 4, the keywords of the ordered product and the existing product are different. The keyword of the order product is "Atosafe cleaning supplies collection, 5) Coconut versatile cleaner 420g * 1" can be. The keyword of the existing product may be 'Coconut Versatile Cleaner 420'. In this case, the common keyword frequency of the ordered product and the existing product is 3, and the similarity degree may be 0.4617 ..

Since the numerical value is calculated for each product based on the process of the device performing the automatic product mapping method based on text mining, the products can be sorted and exposed to the user. In other words, since it is possible to use the system as a recommendation system by ranking the products that appear in the search result, it provides convenience to the users by providing a more intuitive experience on the search results, You can make it. Furthermore, by expanding to automatic product mapping, the user can perform order product mapping for various products automatically by simple command. Instead of searching through the words separated by spaces like the existing method, you can search the whole product name automatically by analyzing the product and show the products that you want. .

5 is a flowchart illustrating a method of automatic product mapping based on text mining according to an embodiment of the present invention.

Referring to FIG. 5, the automatic goods-mapping apparatus can receive technical document data for a plurality of products from a database (step S500).

As described above, the document collection module can receive technical document data for a plurality of products.

The article automatic mapping apparatus can extract a plurality of words on the technical document data (step S510).

As described above, the word extraction module performs a standardization operation on the technical document data to perform morphological analysis in a sentence to extract words.

The product automatic mapping apparatus can generate a keyword dictionary of a company based on a plurality of extracted words (step S520).

The keyword dictionary is generated based on the association between the plurality of words, and the association between the plurality of words may be determined based on a lift value between the first word and the second word among the plurality of words. If the descriptive document data is a transaction unit, the lift value indicates a probability that the first word and the second word are simultaneously present in the descriptive document data, the probability that the first word exists in the descriptive document data, Can be determined based on the probability of being present in the technical document data.

The product automatic mapping apparatus can calculate a term frequency (TF) -IDF (inverse document frequency) weight reflecting the association analysis of the ordered product and the existing product based on the keyword dictionary (step S530).

The TF-IDF weights reflecting the association analysis are generated by generating a first document-keyword matrix for each ordered product and existing product filtered out unnecessary keywords based on the keyword dictionary, and generating a first document- And generates a second document-keyword matrix reflecting the association analysis by reflecting the weight and the frequency of the item set with respect to the keywords constituting the item set, and based on the second document-keyword matrix, -IDF &lt; / RTI &gt; weight. &Lt; RTI ID = 0.0 &gt;

The product automatic mapping apparatus can calculate the cosine similarity of the ordered product and the existing product based on the TF-IDF weight (step S540).

The cosine similarity can be calculated based on Equation (9).

The goods automatic mapping apparatus can generate the search result based on the cosine similarity (step S550).

Existing product information whose cosine similarity is equal to or greater than a predetermined value can be provided as a search result.

Such an automatic product mapping method based on text mining may be implemented in an application or in the form of program instructions that can be executed through various computer components and recorded in a computer-readable recording medium. The computer-readable recording medium may include program commands, data files, data structures, and the like, alone or in combination.

The program instructions recorded on the computer-readable recording medium may be those specially designed and configured for the present invention and may be those known and used by those skilled in the computer software arts.

Examples of computer-readable recording media include magnetic media such as hard disks, floppy disks and magnetic tape, optical recording media such as CD-ROMs and DVDs, magneto-optical media such as floptical disks, media, and hardware devices specifically configured to store and execute program instructions such as ROM, RAM, flash memory, and the like.

Examples of program instructions include machine language code such as those generated by a compiler, as well as high-level language code that can be executed by a computer using an interpreter or the like. The hardware device may be configured to operate as one or more software modules for performing the processing according to the present invention, and vice versa.

While the present invention has been particularly shown and described with reference to exemplary embodiments thereof, it will be understood by those skilled in the art that various changes and modifications may be made therein without departing from the spirit and scope of the invention as defined in the appended claims. It will be possible.

Claims (10)

An automatic product mapping method based on text mining,
The article automatic mapping apparatus receiving technical document data for a plurality of articles from a database;
The automatic product mapping apparatus extracting a plurality of words from the technical document data;
The product automatic mapping device generating a keyword dictionary of a business based on the extracted plurality of words;
The automatic product mapping apparatus calculating a TF (inverse document frequency) -IDF (inverse document frequency) weight based on the linkage analysis of the order product and the existing product based on the keyword dictionary;
The product automatic mapping apparatus calculating a cosine similarity degree of an ordered product and an existing product based on the TF-IDF weight; And
Wherein the automatic goods mapping apparatus generates search results based on the cosine similarity,
Wherein the keyword dictionary is generated through morphological analysis of technical document data of the plurality of products related to a specific company, and the association analysis is performed when the technical document data is one transaction unit, A probability that two words are simultaneously present in the descriptive document data at the same time, a probability that the first word exists in the descriptive document data, and a probability that the second word exists in the descriptive document data,
The TF-IDF weight, which reflects the association analysis,
Generating a first document-keyword matrix for each of an ordered product and an existing product filtered out unnecessary keywords based on the keyword dictionary,
The method includes extracting itemets from the first document-keyword matrix, and calculating a weighted sum of the weighted items and the frequency of the item set by using Equation (2) Generate a document-keyword matrix,
And generating a third document-keyword matrix including the TF-IDF weight using the following Equation 1 reflecting the association analysis based on the second document-keyword matrix.
&Quot; (1) &quot;

Where N is the number of existing products, DF _t is the number of existing products that contain the word t,
The AW _{t, d} is determined based on the following equation (2)
&Quot; (2) &quot;

The ATF _{t, d} is determined based on Equation (3) which is the sum of the association analysis weight and the TF weight,
&Quot; (3) &quot;

A is the association analysis weight, TF _{t, d} is the TF weight, and TF is the frequency at which a specific word is included in each of the plurality of products. delete The method according to claim 1,
The lift value is determined based on the following equation,
&Lt; Equation &

Supp (XUY) is a probability that the first word and the second word are simultaneously present in the descriptive document data at the same time, Supp (X) is a probability that the first word is present in the descriptive document data, Is a probability that the second word is present in the descriptive document data. delete delete An article automatic mapping apparatus based on text mining,
A document collection module that is implemented to receive technical document data for a plurality of products from a database;
A word extraction module configured to extract a plurality of words from the technical document data;
A data mining module that is implemented to generate a business keyword dictionary based on the extracted plurality of words;
A text mining module that is configured to calculate a TF (inverse document frequency) weighting factor reflecting association analysis for each order product and existing product based on the keyword dictionary; And
And a similarity measure module that is configured to calculate a cosine similarity of an ordered product and an existing product based on the TF-IDF weight to generate a search result,
Wherein the keyword dictionary is generated through morphological analysis of technical document data of the plurality of products related to a specific company, and the association analysis is performed when the technical document data is one transaction unit, The probability that the two words are simultaneously present in the descriptive document data at the same time, the probability that the first word is present in the descriptive document data, and the probability that the second word is present in the descriptive document data,
The TF-IDF weight, which reflects the association analysis,
Generating a first document-keyword matrix for each of an ordered product and an existing product filtered out unnecessary keywords based on the keyword dictionary,
The method includes extracting itemets from the first document-keyword matrix, and calculating a weighted sum of the weighted items and the frequency of the item set by using Equation (2) Generate a document-keyword matrix,
And a third document-keyword matrix including the TF-IDF weight is determined by using the following Equation 1 reflecting the association analysis based on the second document-keyword matrix. .
&Quot; (1) &quot;

Where N is the number of existing products, DF _t is the number of existing products that contain the word t,
The AW _{t, d} is determined based on the following equation (2)
&Quot; (2) &quot;

The ATF _{t, d} is determined based on Equation (3) which is the sum of the association analysis weight and the TF weight,
&Quot; (3) &quot;

A is the association analysis weight, TF _{t, d} is the TF weight, and TF is the frequency at which a specific word is included in each of the plurality of products. delete The method according to claim 6,
The lift value is determined based on the following equation,
&Lt; Equation &

Supp (XUY) is a probability that the first word and the second word are simultaneously present in the descriptive document data at the same time, Supp (X) is a probability that the first word is present in the descriptive document data, Is a probability that the second word is present in the descriptive document data. delete delete

Images