KR20210003540A - Apparatus and method for embedding multi-vector document using semantic decomposition of complex documents - Google Patents

Abstract

The present invention relates to an apparatus and a method for embedding a multi-vector document through semantic decomposition of complex documents which can correct vector distortion and accurately embed a document. The apparatus comprises: a target document parsing unit to separate all terms into tokens by parsing target documents included in a document set; a word embedding unit to convert each token into a word vector by word embedding; a keyword vector extraction unit to extract a word vector of a token designated as a keyword for each target document among the word vectors to generate a keyword vector set for each document; a keyword clustering unit to perform clustering analysis on the keyword vector set for each document to generate a plurality of keyword clusters; and a multi-vector generation unit to generate vectors for each keyword cluster based on keyword vectors included in the plurality of keyword clusters to determine the vectors for each keyword cluster as multiple vectors of a target document associated with the plurality of keyword clusters.

Description

Multi-vector document embedding device and method through semantic decomposition of compound documents {APPARATUS AND METHOD FOR EMBEDDING MULTI-VECTOR DOCUMENT USING SEMANTIC DECOMPOSITION OF COMPLEX DOCUMENTS}

The present invention relates to a technology for embedding a multi-vector document through semantic decomposition of a compound document, and more particularly, embedding a multi-vector document through semantic decomposition of a compound document capable of correcting vector distortion and embedding documents more accurately. It relates to an apparatus and method.

To the extent that the term big data is used more frequently than data, the amount of data created, distributed, and used in everyday life is increasing rapidly. In addition, the number of interactions that users perform during the day through the Internet, social network services, and IoT is expected to increase rapidly in the future. Since such interactions are mainly made through text, the amount of text data distributed in the future can be expected to increase sharply compared to the present. As the proportion of text data in the data ecosystem increases, attempts to create new knowledge through systematic management and various analysis of text data are also being made very actively.

Unlike structured data that can be directly applied to various operations and traditional analysis techniques, all unstructured texts must be structured to convert the original document into a form that can be understood by a computer before analysis. The mapping of an arbitrary object into a space of a specific dimension while maintaining algebraic properties for structuring text data is called embedding. Specifically, word embedding, which represents words as vectors, and documents are represented as vectors. It is realized by document embedding.

The most classical method for structuring words is the One-hot Encoding method. This is a method of creating a word vector space having a dimension corresponding to the size of the word set, assigning an index to each word, and then setting the dimension value of the index corresponding to the word to 1 and other dimension values to 0. This method is easy to understand and easy to implement, but has limitations in that the computational cost increases as the number of words increases, and the word vector does not sufficiently reflect the meaning of the word.

Recently, attempts to embed not only individual words but also sentences, paragraphs, and entire documents have been made in various aspects. In particular, as the demand for analysis on document embedding increases rapidly, a number of methods to support this have been proposed.

Korean Patent Registration No. 10-0490442 (2005.05.11)

An embodiment of the present invention is to provide a multi-vector document embedding apparatus and method through semantic decomposition of a compound document capable of correcting a vector distortion phenomenon and embedding a document more accurately.

An embodiment of the present invention is a semantic decomposition of a compound document in which a document vector is generated by focusing on the core terms of the document by actively reflecting prior knowledge about the document, that is, keyword information directly selected by the document author, in the process of embedding a document. An apparatus and method for embedding a multi-vector document through

An embodiment of the present invention provides an apparatus and method for embedding a multi-vector document through semantic decomposition of a compound document that can perform more sophisticated text structuring and improve the quality of various text analysis results such as classification, clustering, and topic modeling. I want to.

Among embodiments, a multi-vector document embedding apparatus through semantic decomposition of a compound document is a target document parsing unit that separates all terms into tokens through parsing a target document included in a document set, words A word embedding unit that converts each token into a word vector through embedding, a keyword vector extraction unit that extracts a word vector of a token designated as a keyword for each target document from the word vectors to generate a keyword vector set for each document, the document A keyword clustering unit that generates a plurality of keyword clusters by performing clustering analysis on each keyword vector set, and a vector for each keyword cluster based on the keyword vectors included in each of the plurality of keyword clusters And a multiple vector generator that determines as multiple vectors of the target document associated with the keyword clusters of.

The target document parsing unit may perform the parsing after adding the keyword set of the target document to a word dictionary used for the parsing.

When there is no keyword set, the target document parsing unit may determine a key word extracted through analysis of the target document as a keyword.

The word embedding unit may perform the word embedding using Word2Vec and generate a vector having an n-dimensional real value for each token as the word vector.

The keyword vector extractor may generate a keyword vector set including each keyword of the keyword set and a word vector pair for the corresponding keyword for a specific target document as one element.

The keyword clustering unit may perform the cluster analysis by applying a hierarchical cluster analysis or a non-hierarchical cluster analysis based on an n-dimensional real value of a keyword vector included in the keyword vector set.

The multiple vector generator may generate a vector for each keyword cluster through an average of keyword vectors included in each keyword cluster.

The multiple vector generator may generate a member vector set including the vector for each keyword cluster as a member vector, and may determine the multiple vector.

Among the embodiments, the multi-vector document embedding method through semantic decomposition of a compound document is a step of separating all terms into tokens through parsing a target document included in a document set, and word embedding. ), converting each token into a word vector, generating a keyword vector set for each document by extracting a word vector of a token designated as a keyword for each target document from the word vectors, cluster analysis for the keyword vector set for each document Generating a plurality of keyword clusters by performing (Clustering Analysis), and generating a vector for each keyword cluster based on the keyword vectors included in each of the plurality of keyword clusters to multiplex target documents related to the plurality of keyword clusters. Determining as a vector.

The disclosed technology can have the following effects. However, since it does not mean that a specific embodiment should include all of the following effects or only the following effects, it should not be understood that the scope of the rights of the disclosed technology is limited thereby.

The apparatus and method for embedding a multi-vector document through semantic decomposition of a compound document according to an embodiment of the present invention actively reflects prior knowledge about the document, that is, keyword information directly selected by the document author, in the process of performing document embedding. You can create document vectors by focusing on the key terms of.

The apparatus and method for embedding a multi-vector document through semantic decomposition of a compound document according to an embodiment of the present invention can perform more sophisticated text structuring and improve quality of various text analysis results such as classification, clustering, and topic modeling. .

1 is a diagram illustrating a multi-vector document embedding system through semantic decomposition of a compound document according to the present invention.
FIG. 2 is a block diagram illustrating a logical configuration of the apparatus for embedding a multi-vector document in FIG. 1.
FIG. 3 is a flowchart illustrating a multi-vector document embedding process performed by the multi-vector document embedding apparatus of FIG. 1.
4 is a diagram illustrating a multi-vector document embedding process through semantic decomposition of a compound document according to the present invention.
5 is an exemplary diagram illustrating a parsing process performed by the target document parsing unit of FIG. 2.
6 is an exemplary view illustrating a word embedding process performed by the word embedding unit of FIG. 2.
FIG. 7 is an exemplary diagram illustrating a process of generating a keyword vector set performed by the keyword vector extraction unit of FIG. 2.
8 and 9 are exemplary diagrams illustrating a multi-vector generation process performed by the multi-vector generator of FIG. 2.
10 is a diagram illustrating a visualization of the vector of FIG. 9 in a three-dimensional space.
11 to 16 are diagrams illustrating an embodiment of a performance test procedure according to the present invention.

Since the description of the present invention is merely an embodiment for structural or functional description, the scope of the present invention should not be construed as being limited by the embodiments described in the text. That is, since the embodiments can be variously changed and have various forms, the scope of the present invention should be understood to include equivalents capable of realizing the technical idea. In addition, since the object or effect presented in the present invention does not mean that a specific embodiment should include all of them or only those effects, the scope of the present invention should not be understood as being limited thereto.

Meanwhile, the meaning of terms described in the present application should be understood as follows.

Terms such as "first" and "second" are used to distinguish one component from other components, and the scope of rights is not limited by these terms. For example, a first component may be referred to as a second component, and similarly, a second component may be referred to as a first component.

When a component is referred to as being "connected" to another component, it should be understood that although it may be directly connected to the other component, another component may exist in the middle. On the other hand, when it is mentioned that a certain component is "directly connected" to another component, it should be understood that no other component exists in the middle. On the other hand, other expressions describing the relationship between the constituent elements, that is, "between" and "just between" or "neighboring to" and "directly neighboring to" should be interpreted as well.

Singular expressions are to be understood as including plural expressions unless the context clearly indicates otherwise, and terms such as “comprise” or “have” refer to implemented features, numbers, steps, actions, components, parts, or It is to be understood that it is intended to designate that a combination exists and does not preclude the presence or addition of one or more other features or numbers, steps, actions, components, parts, or combinations thereof.

In each step, the identification code (for example, a, b, c, etc.) is used for convenience of explanation, and the identification code does not describe the order of each step, and each step has a specific sequence clearly in context. Unless otherwise stated, it may occur differently from the stated order. That is, each of the steps may occur in the same order as specified, may be performed substantially simultaneously, or may be performed in the reverse order.

The present invention can be embodied as computer-readable codes on a computer-readable recording medium, and the computer-readable recording medium includes all types of recording devices storing data that can be read by a computer system. . Examples of computer-readable recording media include ROM, RAM, CD-ROM, magnetic tape, floppy disk, and optical data storage devices. Further, the computer-readable recording medium is distributed over a computer system connected by a network, so that the computer-readable code can be stored and executed in a distributed manner.

All terms used herein have the same meaning as commonly understood by one of ordinary skill in the field to which the present invention belongs, unless otherwise defined. Terms defined in commonly used dictionaries should be construed as having meanings in the context of related technologies, and cannot be construed as having an ideal or excessive formal meaning unless explicitly defined in the present application.

1 is a diagram illustrating a multi-vector document embedding system through semantic decomposition of a compound document according to the present invention.

Referring to FIG. 1, a multi-vector document embedding system 100 through semantic decomposition of a compound document may include a user terminal 110, a multi-vector document embedding device 130, and a database 150.

The user terminal 110 may correspond to a computing device capable of providing a specific document and confirming a multi-vector representation of the document, and may be implemented as a smartphone, laptop, or computer, but is not necessarily limited thereto, and a tablet PC It can be implemented in various devices such as. The user terminal 110 may be connected to the multi-vector document embedding apparatus 130 through a network, and the plurality of user terminals 110 may be connected to the multi-vector document embedding apparatus 130 at the same time.

The multi-vector document embedding apparatus 130 may be implemented as a computer or a server corresponding to a program capable of expressing multiple vectors through document embedding for a document. The multi-vector document embedding device 130 may be wirelessly connected to the user terminal 110 through Bluetooth, WiFi, a communication network, and the like, and may exchange data with the user terminal 110 through a network.

In an embodiment, the multi-vector document embedding apparatus 130 may store information necessary in the document embedding process in conjunction with the database 150. Meanwhile, unlike FIG. 1, the multi-vector document embedding apparatus 130 may be implemented by including the database 150 therein. In addition, the multi-vector document embedding apparatus 130 may be implemented as a basic physical configuration including a processor, a memory, a user input/output unit, and a network input/output unit.

The database 150 may correspond to a storage device that stores various pieces of information necessary in a process of embedding a document. The database 150 may store a set of documents of various subjects subject to document embedding, and may store words extracted from the document and word vectors derived through word embedding, but is not necessarily limited thereto, and a multi-vector document While the embedding device 130 performs multi-vector document embedding through semantic decomposition on the compound document and generates the result, information collected or processed in various forms may be stored.

FIG. 2 is a block diagram illustrating a logical configuration of the apparatus for embedding a multi-vector document in FIG. 1.

2, the multi-vector document embedding device 130 includes a target document parsing unit 210, a word embedding unit 220, a keyword vector extracting unit 230, a keyword clustering unit 240, and a multi-vector generating unit ( 250) and a control unit 260 may be included.

The target document parsing unit 210 may separate all terms into tokens by parsing the target document included in the document set. The target document parsing unit 210 may separate all terms constituting the target document into token units through parsing. Here, a token may mean a morpheme that is the smallest unit with meaning.

In an embodiment, the target document parsing unit 210 may perform parsing after adding a keyword set of the target document to a word dictionary used for parsing. The target document parsing unit 210 may perform structuring by using the token derived from the result of morpheme analysis as it is, as in general text analysis, and may perform structuring by referring to keyword sets for each document in the morpheme analysis process.

More specifically, one of the key operations of the multi-vector document embedding apparatus 130 is to express a word corresponding to a keyword among words in a document as a vector, and such keywords may include a number of complex nouns as well as simple nouns. The commonly used Korean morpheme analyzer has a limitation that it is not possible to separate compound nouns with tokens except for some idioms or proper nouns. For example, a keyword in a document contains the vocabulary of'document embedding'. Only'document' and'embedding' exist in the token set obtained through morphological analysis of the text of the figure, but the vocabulary of'document embedding' may not exist.

Accordingly, the target document parsing unit 210 may add a keyword set to the word dictionary used for parsing in a preprocessing step of parsing the target document, and through this process, a more accurate parsing result may be derived.

In an embodiment, when there is no keyword set, the target document parsing unit 210 may determine, as a keyword, a key word extracted through analysis of the target document. The target document parsing unit 210 may apply the keyword set in the process of parsing the target document, and if the keyword set for the target document is not defined, it may generate the keyword set through analysis of the target document. The target document parsing unit 210 may use various previously known analysis methods to generate a keyword set.

The word embedding unit 220 may convert each token into a word vector through word embedding. In one embodiment, the word embedding unit 220 may perform word embedding using Word2Vec and generate a vector having an n-dimensional real value for each token as a word vector. The word embedding unit 220 may apply various word embedding methods, but may convert the token into a vector through the Word2Vec algorithm, which is known to best reflect the features of words among the currently used word embedding methods. .

Here, the Word2Vec algorithm may correspond to a method of representing a word as a vector, and may be implemented in a Continuous Bag of Words (CBOW) method and a Skip-Gram method. A word vector output through Word2Vec may have an n-dimensional real value, where n may correspond to the number of features each word can have. Accordingly, the word vector converted by the word embedding unit 220 may be projected onto an n-dimensional space.

The keyword vector extractor 230 may generate a keyword vector set for each document by extracting a word vector of a token designated as a keyword for each target document from among the word vectors. That is, the keyword vector extracting unit 230 may extract only a keyword of each document from among all words included in the document and generate a keyword vector set consisting of only word vectors for the keyword. Since a keyword set can be independently defined for each document, a keyword vector set can be independently defined for each document.

In an embodiment, the keyword vector extraction unit 230 may generate a keyword vector set including each keyword of the keyword set and a word vector pair for the corresponding keyword as one element for a specific target document. The keyword vector set may be defined for each document, and each keyword belonging to the keyword set of a corresponding document and a word vector pair of the corresponding keyword may be included as elements.

The keyword clustering unit 240 may generate a plurality of keyword clusters by performing clustering analysis on a keyword vector set for each document. Here, clustering analysis may correspond to a statistical analysis method that classifies a high target group by measuring the similarity of each entity, and identifies the similarity of entities in the cluster and differences between entities in different clusters. have. That is, the keyword clustering unit 240 may group keywords having similar subjects through cluster analysis.

In an embodiment, the keyword clustering unit 240 may perform cluster analysis by applying a hierarchical cluster analysis or a non-hierarchical cluster analysis based on an n-dimensional real value of a keyword vector included in the keyword vector set. Cluster analysis can be classified according to the method of analyzing the target, and more specifically, hierarchical clustering method and non-hierarchical clustering based on the size of data and whether or not the cluster targets overlap. Method).

The multiple vector generation unit 250 may generate a vector for each keyword cluster based on the keyword vectors included in each of the plurality of keyword clusters, and determine the multiple vectors of the target document related to the plurality of keyword clusters. That is, a keyword set may be defined corresponding to one target document, and the keyword set may be divided into a plurality of keyword clusters by cluster analysis. A vector may be defined for each divided keyword cluster, and as a result, a plurality of vectors may be defined for one target document.

In an embodiment, the multiple vector generator 250 may generate a vector for each keyword cluster through an average of keyword vectors included in each keyword cluster. The multiple vector generation unit 250 may generate a vector for a keyword cluster by using the n-dimensional real value of a word vector corresponding to the keyword, and an arithmetic operation on real values of the same dimension extracted from each word vector A vector for the keyword cluster can be created by applying.

In an embodiment, the multiple vector generation unit 250 may generate a member vector set including a vector for each keyword cluster as a member vector and determine the multiple vector. Here, the member vector may correspond to a vector constituting multiple vectors for one target document, and may be generated based on keywords clustered among keywords of the target document. As a result, the multi-vector generator 250 may generate a multi-vector representing a specific target document, and the multi-vector may be composed of at least one member vector.

The control unit 260 controls the overall operation of the multi-vector document embedding device 130, and the target document parsing unit 210, the word embedding unit 220, the keyword vector extracting unit 230, the keyword clustering unit 240, and A control flow or data flow between the multi-vector generator 250 may be managed.

3 is a flowchart illustrating a multi-vector document embedding process performed by the multi-vector document embedding apparatus in FIG. 1.

Referring to FIG. 3, the multi-vector document embedding device 130 may separate all terms into tokens through parsing for a target document included in a document set through the target document parsing unit 210. Yes (step S310). The multi-vector document embedding apparatus 130 may convert each token into a word vector through word embedding through the word embedding unit 220 (step S330). The multi-vector document embedding apparatus 130 may generate a keyword vector set for each document by extracting a word vector of a token designated as a keyword for each target document from among the word vectors through the keyword vector extractor 230 (step S350).

Further, the multi-vector document embedding apparatus 130 may generate a plurality of keyword clusters by performing clustering analysis on a keyword vector set for each document through the keyword clustering unit 240 (step S370). The multi-vector document embedding apparatus 130 generates a vector for each keyword group based on the keyword vectors included in each of the plurality of keyword clusters through the multi-vector generator 250 to multiplex target documents for the plurality of keyword clusters. It can be determined as a vector (step S390).

4 is a diagram illustrating a multi-vector document embedding process through semantic decomposition of a compound document according to the present invention.

Referring to FIG. 4, the multi-vector document embedding apparatus 130 may operate on a document in which body content and keywords are explicitly separated. Of course, in the case of documents for which keywords are not explicitly provided, the pre-processing step of deriving key words in the body through various analysis methods and defining them as keywords can be performed.

The multi-vector document embedding device 130 may be composed of main modules of (1) parsing, (2) word embedding, (3) keyword vector derivation, (4) keyword clustering, and (5) multiple vector generation. have. In the parsing step, all terms can be separated into tokens through morphological analysis of the target document. Next, after converting each token into a vector having an N-dimensional real value through word embedding, only the word vector of the token designated as the keyword for each document is extracted from among these vectors to construct a keyword vector set for each document. Next, cluster analysis is performed on a set of keywords for each document to identify complex topics included in the document, and finally, a vector for each cluster can be derived from the vector of keywords constituting each cluster.

5 is an exemplary diagram illustrating a parsing process performed by the target document parsing unit of FIG. 2.

Referring to FIG. 5, the multi-vector document embedding apparatus 130 may separate all terms into tokens by parsing a target document included in a document set through the target document parsing unit 210.

Figure (a) shows the keyword set for each document, and Figure (b) shows the result of general morpheme analysis without considering the keyword set. On the other hand, figure (c) shows the result of morpheme analysis considering compound nouns included in the keyword set. Vocabularies such as “joint stabilization”, “long distance running”, and “interval training” shown in Fig. (c) are compound nouns that are not included in general-purpose dictionaries generally used for morpheme analysis. Therefore, in order to manage these vocabularies as tokens, it is necessary to add these vocabularies to the dictionary in the morpheme analysis stage.

6 is an exemplary diagram illustrating a word embedding process performed in the word embedding unit of FIG. 2.

Referring to FIG. 6, the multi-vector document embedding device 130 may structure each token into a vector using a real value through the word embedding unit 220. In particular, the word embedding unit 220 can convert the token into a vector through the word2Vec algorithm, which is known to best reflect the features of the word among the currently used word embedding methods. You can see an example of converting to an n-dimensional vector.

FIG. 7 is an exemplary diagram illustrating a process of generating a keyword vector set performed by the keyword vector extraction unit in FIG. 2.

Referring to FIG. 7, the multi-vector document embedding device 130 extracts a word vector of a token designated as a keyword for each target document from word vectors generated by the word embedding unit 220 through the keyword vector extraction unit 230 You can create a set of keyword vectors for each document.

For example, the word vector shown in Fig. 6 (b) includes not only keywords but also vectors of surrounding words. The keyword vector derivation process performed by the keyword vector extraction unit 230 is a process of extracting only a vector of a vocabulary specified as a keyword for each document from among these word vectors, an example of which is shown in FIG. 7. In Figure 6 (b), tokens such as'use','exercise','development', and'verify' are not included in the keyword set in Figure 7 (a). It can be seen that it does not appear in the keyword vector set of b). The multi-vector document embedding apparatus 130 may utilize only the keyword vector shown in Fig. 7 (b) for document embedding.

8 and 9 are exemplary diagrams illustrating a multi-vector generation process performed by the multi-vector generator of FIG. 2.

The multi-vector document embedding apparatus 130 may generate a plurality of keyword clusters by performing clustering analysis on a keyword vector set for each document through the keyword clustering unit 240, and the multi-vector generating unit 250 A vector for each keyword cluster may be generated based on a keyword vector included in each of the plurality of keyword clusters.

In the case of a document containing a variety of topics in a complex manner, it is very difficult to expect that the vector resulting from embedding accurately represents each of these topics. For example, document Doc ₁ shown in FIG. 7 includes'application'and'gyrosensor', which are keywords of'IT', and at the same time,'joint stabilization'and'sense of balance' are keywords of'medical'. ,'Reflex nerve', and'biomechanics'. However, when a document encompassing all of these keywords is expressed as a single vector, for example, when the average of all keyword vectors is used as a document vector, this vector is far from the'IT' topic and is far from the'Medical' topic. It can also be mapped to a long distance. These phenomena appear similarly in other documents, and as a result, most documents tend to map in similar spaces despite the differences in the detailed topics they contain.

The multi-vector document embedding apparatus 130 may represent a plurality of vectors according to the number of subjects constituting each document. For example, in the case of Doc ₁ , if the keyword set is divided into two groups, the multi-vector document embedding apparatus 130 may express the corresponding document as two member vectors.

The multi-vector document embedding apparatus 130 can divide the keyword sets constituting each document into two groups by performing clustering for each document on the keyword vector of FIG. 7 (b), and each document member An example of deriving a vector is shown in FIG. 8. However, in FIG. 8, subject names such as'IT','Medical', and'Sports' are arbitrarily inserted for convenience of explanation. For example, if Doc1 is expressed as a single vector, it can be expressed as Doc ₁ = (-0.0058, -0.0084, …, 0.0497) by using the overall average of keywords constituting Doc1.

However, the multi-vector document embedding device 130 uses two member vectors for the document to be Doc ₁ ¹ = (0.0179,-0.0702, …, 0.0177) and Doc ₁ ² = (-0.0177, 0.0225, …, 0.0657). Expressed as FIG. 9 compares the case of expressing the three documents of FIG. 8 as a single vector and the case of expressing a multi-vector. In FIG. 9, it can be seen that the relationship between three documents that are not explicitly identified with a single vector is confirmed in detail with a multi-vector representation.

10 is a diagram illustrating the vector of FIG. 9 visualized in a three-dimensional space in order to intuitively confirm this.

In FIG. 10, a vector is visualized in a three-dimensional space of (D ₁ , D ₂ , D _N ), and each document is represented by a rectangle, and keywords included in each document are represented by a triangle. Figure 10 (a) is an example in which each document is expressed as a single vector, and the three documents are mapped in a space adjacent to each other despite the different subject matter. Meanwhile, figure (b) of FIG. 10 is an example in which member vectors constituting each document are represented together with a circle. For example, Doc ₁ appears as one vector in the middle area of the entire space in a single vector representation, but in a multi-vector representation, it can be seen that it appears as two vectors, Doc ₁ ¹ at the top left and Doc ₁ ² at the bottom. . Through this multi-vector expression, the similarity between Doc ₁ ¹ and Doc ₃ ^1, the relationship between three documents that could not be clearly identified in the single vector expression, the similarity between Doc ₁ ² and Doc ₂ ¹ , and the similarity between Doc ₂ ² and Doc ₃ ² I can confirm.

4. Experiment

4.1. Experiment outline

This chapter summarizes the process and results of the experiment to evaluate the usefulness of the present invention. In the present invention, a new embedding scheme for representing documents as multiple vectors is proposed. However, there is no direct criterion for evaluating whether vectors derived by which of the various embedding methods represent the original document more accurately. Therefore, this chapter indirectly evaluates the quality of document vectors derived through various embedding methodologies in terms of using embedding results.

Specifically, document embedding is performed in various ways for documents whose categories are identified, and as a result, documents judged to be similar to each document are identified. If the similar documents identified in this way belong to the same category as the reference document, it is determined that embedding has been accurately performed, and when the document belonging to a different category from the reference document is determined as a similar document, the embedding is determined to be incorrect.

Therefore, in the experimental data of the present invention, the category to which each document belongs must be specified together with a list of keywords for each document. In this experiment, 3,147 papers were collected on the topic in a total of 7 articles on the Korean Academic Information (KISS) site as data that satisfies these conditions. The overall experiment was conducted using Python 3.6, and Komoran was mainly used for token separation, Gensim for word2Vec modeling, and Numpy package for vector operation. This section introduces the overall outline of the experiment, and the subsequent sections of this chapter introduce the main process and results of the experiment. First, an overview of the entire experiment is shown in FIG. 11.

The entire experimental process of FIG. 11 is divided into three steps. First of all, Phase 1 is the step of creating a complex document for experimentation, and the necessity and process of creating a compound document is dealt with in the next section, Section 4.2. Next, Phase 2 is a process of generating multiple vectors according to the present invention, and this part is as introduced in detail above. In the final stage, Phase 3, the same set of documents is embedded by several methodologies. Document embedding is performed in a total of five methods including the present invention, and details of this are covered in Section 4.3. In addition, the results of the performance evaluation for document embedding performed in these five ways are introduced in Section 4.4, the last section of this chapter.

4.2. Create compound document

The present invention improves the limitations of the existing document embedding method in which a compound document including various subjects is represented as a single vector. Most of the documents circulated in real life can be regarded as a compound document in which one document includes several subjects, and the excellence of the present invention is expected to appear more clearly in the embedding of documents including various subjects. In order to more clearly compare the performance difference of the document embedding methodology in the process of expressing the compound document as described above, in this experiment, a compound document was artificially created by using the category information of each document, and the detailed process is as follows.

Documents used in the experiment are abstracts of 3,147 papers in 7 categories:'Humanities','Social Science','Natural Science','Engineering','Agriculture','Medicine', and'Art and Physical Education'. For a total of 7 categories, the number of cases where 2 category combinations (eg'Natural Science + Engineering') can be created is 21, and 50 compound documents are created for each of these combinations. For example, it is possible to create a composite document in the field of'Natural Science + Engineering' by merging one abstract in the'Natural Science' and one abstract in the'Engineering' (FIG. 12).

The performance of each was tested by applying two methods of selecting documents for each category to be used for creating compound documents. The first criterion was a random selection method, and 50 documents to participate in each combination were randomly selected for each category. The second method is to select a representative document based on the center of each category. Specifically, a keyword vector average of documents belonging to each category is calculated and identified as the center of the category, and then the top 50 documents of the category that are close to the center of the category are selected as the representative documents of each category, and a compound document Used for generation.

According to the above-described method, 1,050 compound documents were generated by a random selection method, and 1,050 compound documents were selected by the center-based method. 350 original documents used in the composition of compound documents were excluded from the single document set. As a result, word vectors were learned by learning a total of 3,847 documents (2,797 single documents + 1,050 complex documents). Word embedding was performed using word2Vec modeling. Specifically, the dimension of the vector was set to 300, the number of learning was 50 times, and the window size was set to 5. As a result, a total of 36,798 word vectors were derived.

4.3. Create document vector

In this section, there are five document embedding methods used to compare the performance of the present invention, namely (1) Gen_doc2Vec, (2) doc2Vec_Key, (3) Proposed SV, (4) Multi-Vector (Ideal), and (5). A Multi-Vector (Clustering) method is introduced (FIG. 13).

In FIG. 13, (1) Gen_doc2Vec represents a document embedding result through general doc2Vec modeling, and (2) doc2Vec_Key represents a result of using a keyword list as a term dictionary while performing general doc2Vec modeling. (3) Proposed SV is the result of deriving the document vector by the word vector average of the keywords specified in the document. All of (1) to (3) have a common point in that each document is expressed as a single vector. On the other hand, (4) and (5) are methods of expressing each document as multiple vectors. Among them, (4) Multi-Vector (Ideal) utilizes the original category information of the two original documents constituting the composite document.

For example, in the case of a compound document in the field of'natural science + engineering', the two original documents belong to the category of'natural science' and'engineering', respectively. At this time, the keyword set constituting the composite document is divided into a keyword subset specified in the document of the'Natural Science' category and the keyword subset specified in the document of the'Engineering' category. After that, the average vector of word vectors belonging to each subset is obtained, and these two average vectors are used as member vectors of the compound document.

However, this method can be performed only as an experiment for performance comparison, and the keyword set constituting the real-world composite document does not have prior information that can be divided in this manner. Therefore, in the present invention, the keyword set is divided into subsets through clustering of keywords, and the result of deriving the member vector of the document in this manner is (5) Multi-Vector (Clustering). That is, the core methodology proposed by the present invention is (5), and (4) is an ideal model introduced for a relative comparison regarding the performance of the present invention, although it is not practically applicable.

4.4. Performance evaluation

4.4.1. Performance rating scale

This subsection introduces a method to evaluate the performance of the five document embedding methodologies introduced earlier. Specifically, five document embeddings are repeatedly performed, and the document with the highest similarity is identified for each of the compound documents based on each method. If the identified document belongs to the same category as the reference document, it is determined that embedding has been correctly performed, and when the document belonging to a different category from the reference document is determined to be a similar document, it is determined that the embedding is incorrect.

Since the composite document was composed by merging documents of two categories, this experiment attempts to evaluate the accuracy by recommending two similar documents for each document. In the case of multiple vectors (5) and (6), the nearest document is recommended for each of the two member vectors constituting the document.

Therefore, in the case of (1) to (4), which is a single vector, in order to maintain fairness with the multi-vector methodology, which recommends two similar documents, each document vector and two adjacent documents are recommended. The category matching of similar documents is divided into three categories: when both categories are matched (Totally Correct), only one of them (Partially Correct), and when both categories are not (Completely Incorrect).

In this case, if the category of the original document is'Engineering' and'Humanities', one of the two documents recommended as similar documents is'Engineering' or'Humanities', and both are'Engineering' or both In the case of'humanities', it is judged as Partially Correct. If two documents recommended as similar documents do not belong to either'engineering' or'humanities and society', they are judged as Completely Incorrect. Under this scale, it can be determined that the more Totally Correct and less Completely Incorrect, the more accurate embedding has been.

4.4.2. Performance analysis result

This subsection presents the results of comparing the accuracy of the five document embedding methodologies according to the performance evaluation scale introduced earlier (Fig. 14).

Fig. 14 (a) is an experiment result of selecting 50 representative documents for each category based on the center of each category, and Fig. 14 (b) is an experiment result of randomly selecting 50 representative documents for each category. FIG. 15 shows the results of FIG. 14 as a ratio of the number of documents corresponding to each decision relative to the total documents, and FIG. 16 is a graph showing this.

In FIG. 16, it was confirmed that in both the center-based selection method and the random selection method, the Multi-Vector (Ideal) method showed the best performance, that is, the result in which Totally Correct was the highest and Completely Incorrect was the lowest. In addition, in the case of Multi-Vector (Clustering), which is the method of the present invention, it has been found that it has almost the same accuracy as Multi-Vector (Ideal) even though the category dictionary information of the original document is not used. In other words, it was found that overall, the multi-vector scheme showed superior performance compared to the single vector scheme.

In particular, Totally Correct, that is, the ratio of documents matching both categories is 8% to 10.67% in the case of three single vector methods including the traditional doc2Vec, whereas in the case of Multi-Vector (Clustering) proposed in the present invention, this ratio It can be seen that this is high as 23.71% ~ 24.86%. In the above results, it is worth noting more about the ratio of Completely Incorrect located at the top of the bar graph. This represents the percentage of cases where two documents recommended as similar documents do not belong to either of the two original categories.

For example, two similar documents were recommended for a composite document that merged two documents in the'Humanities' and'Engineering' categories, and these two documents belong to'medicine' or'arts and sports'. In the case of the three single vector methods, this ratio was 24.19% ~ 28.10%, whereas in the case of Multi-Vector (Clustering), this ratio was low, 16.38% ~ 16.67%. It is believed that the reason why the present invention exhibits excellent performance is that the features constituting the composite document are semantically decomposed and expressed.

Existing technologies dealing with text analysis have mainly been performed with focus on the stages after text structuring, that is, analysis and utilization stages such as classification, clustering, and topic modeling. However, with the recent discovery that text structuring work substantially influences the quality of analysis results, the importance of this process is being emphasized, and accordingly, studies on document embedding are being actively conducted. Accordingly, in the present invention, a limitation of the existing document structuring method represented by doc2Vec was pointed out, and a method for overcoming this was suggested.

Although the above has been described with reference to preferred embodiments of the present invention, those skilled in the art will variously modify and change the present invention within the scope not departing from the spirit and scope of the present invention described in the following claims. You will understand that you can do it.

100: Multi-vector document embedding system through semantic decomposition of compound documents
110: user terminal 130: multi-vector document embedding device
150: database
210: target document parsing unit 220: word embedding unit
230: keyword vector extraction unit 240: keyword clustering unit
250: multiple vector generation unit 260: control unit

Claims (9)

A target document parsing unit that separates all terms into tokens through parsing a target document included in the document set;
A word embedding unit that converts each token into a word vector through word embedding;
A keyword vector extraction unit for generating a keyword vector set for each document by extracting a word vector of a token designated as a keyword for each target document from among the word vectors;
A keyword clustering unit for generating a plurality of keyword clusters by performing clustering analysis on the keyword vector set for each document; And
Semantic decomposition of a composite document including a multiple vector generator that generates a vector for each keyword cluster based on the keyword vector included in each of the plurality of keyword clusters and determines it as a multiple vector of the target document associated with the plurality of keyword clusters Device for embedding multi-vector documents through.
The method of claim 1, wherein the target document parsing unit
A multi-vector document embedding apparatus through semantic decomposition of a compound document, characterized in that the parsing is performed after adding a keyword set of the target document to a word dictionary used for the parsing.
The method of claim 2, wherein the target document parsing unit
When there is no keyword set, a key word extracted through analysis of the target document is determined as a keyword. A multi-vector document embedding apparatus through semantic decomposition of a compound document, characterized in that.
The method of claim 1, wherein the word embedding unit
A multi-vector document embedding apparatus through semantic decomposition of a compound document, characterized in that the word embedding is performed using Word2Vec and a vector having an n-dimensional real value for each token is generated as the word vector.
The method of claim 1, wherein the keyword vector extraction unit
A multi-vector document embedding apparatus through semantic decomposition of a compound document, characterized in that for generating a keyword vector set including each keyword of the keyword set and a word vector pair for the keyword for a specific target document as one element.
The method of claim 4, wherein the keyword clustering unit
A multi-vector document through semantic decomposition of a compound document, characterized in that the cluster analysis is performed by applying hierarchical cluster analysis or non-hierarchical cluster analysis based on the n-dimensional real value of the keyword vector included in the keyword vector set. Embedding device.
The method of claim 1, wherein the multiple vector generator
A multi-vector document embedding apparatus through semantic decomposition of a compound document, characterized in that generating a vector for each keyword cluster through an average of keyword vectors included in each keyword cluster.
The method of claim 7, wherein the multiple vector generator
A multi-vector document embedding apparatus through semantic decomposition of a compound document, characterized in that a member vector set including the vector for each keyword group is generated as a member vector and is determined as the multiple vector.
In the method performed in a multi-vector document embedding apparatus,
Separating all terms into tokens through parsing a target document included in the document set;
Converting each token into a word vector through word embedding;
Generating a keyword vector set for each document by extracting a word vector of a token designated as a keyword for each target document from the word vectors;
Generating a plurality of keyword clusters by performing clustering analysis on the keyword vector set for each document; And
Through semantic decomposition of a compound document including the step of generating a vector for each keyword cluster based on the keyword vector included in each of the plurality of keyword clusters and determining as a multiple vector of a target document for the plurality of keyword clusters How to embed multiple vector documents.

Images