KR102268570B1 - Apparatus and method for generating document cluster - Google Patents

Abstract

Disclosed are a device for generating a document cluster, and a method thereof. The device for generating a document cluster according to one embodiment of the present invention comprises: a document information acquisition unit which acquires bibliographic information and text information for each of the plurality of documents; a citation-based similarity matrix generation unit which generates a citation-based similarity matrix having, as an element, the citation-based similarity between the plurality of documents based on the bibliographic information; a text-based similarity matrix generation unit which generates a text-based similarity matrix having a text-based similarity between the plurality of documents as an element based on the text information; a combination similarity matrix generation unit which generates a combination similarity matrix having a combination similarity between the plurality of documents as an element based on the citation-based similarity matrix and the text-based similarity matrix; and a cluster generation unit which generates one or more clusters by clustering the plurality of documents based on the combination similarity matrix. According to the present invention, it is possible to derive the optimal clustering result based on a similarity between documents.

Description

Document cluster generating apparatus and method {APPARATUS AND METHOD FOR GENERATING DOCUMENT CLUSTER}

Disclosed embodiments relate to techniques for generating document clusters.

In order to efficiently manage a large amount of documents, there has been an attempt to calculate the similarity between documents using various methods and to cluster the documents based on the similarity.

However, most of these attempts are based on only the bibliographic information of the document or the text information in the document, so there is a problem that a significant difference occurs in the clustering result depending on the type of the document.

Republic of Korea Patent Publication No. 10-1769035 (registered on Aug. 10, 2017)

The disclosed embodiments are for effectively clustering various kinds of documents.

The apparatus for generating a document group according to an embodiment includes a document information acquisition unit configured to acquire bibliographic information and text information for each of a plurality of documents, and a citation-based similarity between the plurality of documents based on the bibliographic information as an element. A citation-based similarity matrix generator for generating a similarity matrix, a text-based similarity matrix generator for generating a text-based similarity matrix having as an element text-based similarity between the plurality of documents based on the text information, the citation-based similarity matrix, and A joint similarity matrix generator generating a joint similarity matrix having a joint similarity between the plurality of documents as an element based on the text-based similarity matrix, and clustering the plurality of documents based on the joint similarity matrix to generate one or more clusters It includes a cluster generating unit that

The citation-based similarity matrix generator may calculate the citation-based similarity based on any one of a co-citation strength and a bibliographic coupling strength, and the text-based similarity matrix generator includes a vector space The text-based similarity may be calculated using a vector space-based similarity calculation method.

The apparatus for generating a document cluster according to another embodiment may calculate a first information entropy based on the citation-based similarity and a second information entropy based on the text-based similarity, and the citation-based similarity, the text-based similarity, and the first information The method may further include a weight calculator configured to calculate a citation weight and a text weight based on the entropy and the second information entropy.

The joint similarity matrix generator may calculate the joint similarity based on the citation-based similarity, the text-based similarity, the citation weight, and the text weight.

The cluster generating unit, k-means algorithm (k-means algorithm), density-based clustering (Density-Based Clustering), probability distribution-based clustering (Probability distribution-Based Clustering), hierarchical clustering (Hierarchical Clustering) and Govern-Newman algorithm ( The one or more clusters may be generated using any one of the Girvan-Newman algorithm.

The cluster generator may generate a plurality of cluster scenarios by varying the number of clusters to be generated.

The apparatus for generating a document cluster according to another embodiment may further include a cluster evaluation unit configured to select an optimal cluster scenario from among the plurality of cluster scenarios.

The cluster evaluator may select any one of a plurality of preset indices as a reference index, and select a cluster scenario when the reference index has a maximum value as the optimal cluster scenario.

The cluster evaluation unit is configured to calculate a rank for each of the plurality of cluster scenarios for each of a plurality of preset indicators, calculate an average value for each of the plurality of cluster scenarios of the calculated rank, and when the calculated average value is the minimum may be selected as the optimal clustering scenario.

A method for generating a document cluster according to an embodiment includes obtaining bibliographic information and text information for each of a plurality of documents, and a citation-based similarity matrix having citation-based similarity between the plurality of documents as an element based on the bibliographic information. generating, based on the text information, generating a text-based similarity matrix having text-based similarity between the plurality of documents as an element, between the plurality of documents based on the citation-based similarity matrix and the text-based similarity matrix generating a joint similarity matrix having joint similarity as an element, and generating one or more clusters by clustering the plurality of documents based on the joint similarity matrix.

The generating of the citation-based similarity matrix may include calculating the citation-based similarity based on any one of a co-citation strength and a bibliographic coupling strength, and generating the text-based similarity matrix In the doing, the text-based similarity may be calculated using a vector space-based similarity calculation method.

According to another embodiment, a method for generating a document cluster includes calculating a first information entropy based on the citation-based similarity, calculating a second information entropy based on the text-based similarity, and the citation-based similarity and the text The method may further include calculating a citation weight and a text weight based on the base similarity, the first information entropy, and the second information entropy.

The generating of the joint similarity matrix may include calculating the joint similarity based on the citation-based similarity, the text-based similarity, the citation weight, and the text weight.

The step of generating the cluster includes k-means algorithm, density-based clustering, probability distribution-based clustering, hierarchical clustering, and Govern- The one or more clusters may be generated using any one of a Girvan-Newman algorithm.

The generating of the cluster may include generating a plurality of cluster scenarios by varying the number of generated clusters.

The method for generating a document cluster according to another embodiment may further include selecting an optimal cluster scenario from among the plurality of cluster scenarios.

The selecting may include selecting any one of a plurality of preset indices as a reference index, and selecting a cluster scenario when the reference index has a maximum value as the optimal clustering scenario.

The selecting may include calculating a rank for each of the plurality of clustering scenarios for each of a plurality of preset indicators, calculating an average value for each of the plurality of clustering scenarios of the calculated rank, and the calculated average value is The method may include selecting a minimum cluster scenario as the optimal cluster scenario.

According to the disclosed embodiments, by grouping documents in consideration of both the citation-based similarity and the text-based similarity, it is possible to more easily use information for group classification among bibliographic information and text information included in the document.

Also, according to the disclosed embodiments, by selecting an optimal clustering scenario from among the generated clustering scenarios according to the number of clusters, it is possible to derive an optimal clustering result based on the similarity between documents.

1 is a block diagram illustrating an apparatus for generating a group of documents according to an embodiment;
2 is a block diagram illustrating an apparatus for generating a group of documents according to an additional embodiment;
3 is a diagram for explaining a process of calculating a citation weight and a text weight according to an embodiment;
4 is a flowchart illustrating a method for generating a document cluster according to an exemplary embodiment;
5 is a flowchart illustrating a method for generating a document cluster according to an additional exemplary embodiment;
6 is a block diagram illustrating and describing a computing environment including a computing device suitable for use in example embodiments;

Hereinafter, specific embodiments will be described with reference to the drawings. DETAILED DESCRIPTION The following detailed description is provided to provide a comprehensive understanding of the methods, devices, and/or systems described herein. However, this is merely an example and the disclosed embodiments are not limited thereto.

In describing the embodiments, if it is determined that a detailed description of a related known technology may unnecessarily obscure the gist of the disclosed embodiments, the detailed description thereof will be omitted. And, the terms to be described later are terms defined in consideration of functions in the disclosed embodiments, which may vary according to intentions or customs of users and operators. Therefore, the definition should be made based on the content throughout this specification. The terminology used in the detailed description is for the purpose of describing the embodiments only, and should in no way be limiting. Unless explicitly used otherwise, expressions in the singular include the meaning of the plural. In this description, expressions such as “comprising” or “comprising” are intended to indicate certain features, numbers, steps, acts, elements, some or a combination thereof, one or more other than those described. It should not be construed to exclude the presence or possibility of other features, numbers, steps, acts, elements, or any part or combination thereof.

Hereinafter, the term 'document cluster' means classifying a plurality of documents into similar documents according to characteristics of data in the document.

1 is a block diagram illustrating an apparatus 100 for generating a document group according to an exemplary embodiment.

Referring to FIG. 1 , an apparatus 100 for generating a group of documents according to an embodiment includes a document information acquisition unit 110 , a citation-based similarity matrix generation unit 120 , a text-based similarity matrix generation unit 130 , and a combined similarity matrix It includes a generator 140 and a cluster generator 150 .

The document information acquisition unit 110 acquires bibliographic information and text information for each of a plurality of documents.

Hereinafter, 'bibliographic information' refers to a list in which bibliographic items of documents are collected and arranged and organized according to a predetermined method. For example, 'bibliographic information' refers to personal information of the person who created the document, information about the country where the document was created, information about the date the document was created, information about the number assigned to the document, classification information, and reference information based on citation or citation of the document It may include, but is not necessarily limited thereto, and the above-described examples may further include various types of bibliographic information according to the type or form of the document.

Hereinafter, 'text information' means text including sentences or words constituting the document itself. For example, 'text information' may include, but is not limited to, the title of a document, a summary of the document, the content of the document, and the title of a reference document included in the document.

According to an embodiment, the document information acquisition unit 110 may acquire bibliographic information and text information from a separate database (not shown) separated from the document cluster generating apparatus 100, but is not necessarily limited thereto. According to an embodiment, a database (not shown) may be provided in the document group generating apparatus 100 .

The citation-based similarity matrix generator 120 generates a citation-based similarity matrix having, as an element, a citation-based similarity between a plurality of documents based on bibliographic information.

According to an embodiment, the citation-based similarity matrix generator 120 may calculate the citation-based similarity based on any one of a co-citation strength and a bibliographic coupling strength.

Specifically, the 'simultaneous citation intensity' is a number indicating the number of times two documents are commonly cited by one or more other documents, and may be changed as a document citing two documents in common is newly created.

In addition, the 'bibliographic coupling strength' is a number indicating the number of commonly cited references among the references cited by the document, and may be normalized by, for example, Equation 1 below.

[Equation 1]

은 정규화된 서지 결합 강도,

는 문서 A와 B가 공통으로 인용한 참고 문헌의 수,

는 문서 A가 인용한 참고 문헌의 수,

는 문서 B가 인용한 참고 문헌의 수를 나타낸다.At this time,

is the normalized surge bond strength,

is the number of references commonly cited by documents A and B;

is the number of references cited by document A,

indicates the number of references cited by document B.

According to an embodiment, the 'citation-based similarity' may be a numerical value expressed as a value within a range of 0 or more or 1 or less by normalizing the strength of simultaneous citation or bibliographic binding.

For example, the citation-based similarity between identical documents is 1, and the citation-based similarity between two documents that are not at all similar is 0.

According to an embodiment, the 'matrix having citation-based similarity as an element' may be a square matrix in which serial numbers sequentially assigned to each of a plurality of documents are a row index and a column index.

In this case, the values of elements located on the main diagonal of the matrix are 1.

The text-based similarity matrix generator 130 generates a text-based similarity matrix having text-based similarity between a plurality of documents as an element based on text information.

According to an embodiment, the text-based similarity matrix generator 130 may calculate the text-based similarity using a vector space-based similarity calculation method.

Specifically, the vector space-based similarity calculation method refers to a method of calculating a vector corresponding to each document based on text information in the document, and calculating the similarity between documents by using the calculated vector.

For example, the text-based similarity may be calculated using Euclidean distance, cosine similarity, Manhattan distance, and Jaccard similarity.

According to an embodiment, the text-based similarity matrix generator 130 may calculate the text-based similarity using a similarity calculation model including a deep learning-based artificial neural network (ANN) structure. have.

Meanwhile, according to an embodiment, the 'text-based similarity' may be a numerical value expressed as a value within a range of 0 or more and 1 or less by normalizing a result value by a vector space-based similarity calculation method or a result value by a similarity calculation model.

For example, the text-based similarity between identical documents is 1, and the text-based similarity between two documents that are not at all similar is 0.

According to an embodiment, the 'matrix having text-based similarity as an element' may be a square matrix in which serial numbers sequentially assigned to each of a plurality of documents are a row index and a column index.

In this case, the values of elements located on the main diagonal of the matrix are 1.

The joint similarity matrix generator 140 generates a joint similarity matrix having the joint similarity between a plurality of documents as an element based on the citation-based similarity matrix and the text-based similarity matrix.

Specifically, the combined similarity matrix generator 140 may generate a matrix having, as elements, a value obtained by summing elements in the citation-based similarity matrix and elements in the text-based similarity matrix positioned at the same index as the corresponding element at a preset ratio. .

The cluster generator 150 generates one or more clusters by clustering a plurality of documents based on the joint similarity matrix.

According to an embodiment, the cluster generating unit 150 includes a k-means algorithm, a density-based clustering, a probability distribution-based clustering, and a hierarchical clustering. Clustering) and one or more clusters may be generated using any one of the Girvan-Newman algorithm.

2 is a block diagram illustrating an apparatus 200 for generating a document group according to an additional exemplary embodiment.

Referring to FIG. 2 , the apparatus 200 for generating a group of documents according to an additional embodiment may further include a weight calculation unit 210 and a group evaluation unit 220 . In the example shown in FIG. 2 , the document information obtaining unit 110 , the citation-based similarity matrix generating unit 120 , the text-based similarity matrix generating unit 130 , the combined similarity matrix generating unit 140 , and the cluster generating unit 150 . ) is the same configuration as the configuration shown in FIG. 1, and thus a redundant description thereof will be omitted.

The weight calculator 210 calculates the first information entropy based on the citation-based similarity in the citation-based similarity matrix generated by the citation-based similarity matrix generator 120 .

Specifically, 'information entropy' is a value indicating the uncertainty of specific information. When there are a plurality of states that specific information can take, 'information entropy' is the probability that specific information will take a specific state and the nature of the probability. It is calculated by summing values multiplied by log values for all states, and 'first information entropy' means uncertainty of citation-based similarity.

According to an embodiment, the weight calculator 210 may calculate the first information entropy by Equation 2 below.

[Equation 2]

는 제1 정보 엔트로피,

은 두 문서 간의 인용 기반 유사도에 대응되는 확률,

은 1-

, log는 밑을 e로 하는 자연로그를 나타낸다.At this time,

is the first information entropy,

is the probability corresponding to the citation-based similarity between two documents,

Silver 1-

, log represents the natural logarithm with base e.

는 0.8,

는 0.2이고, 위 수학식 2에 의하여 제1 정보 엔트로피

는 0.5가 된다.For example, if the citation-based similarity of two papers A and B is 80%,

is 0.8,

is 0.2, and the first information entropy according to Equation 2 above

becomes 0.5.

Also, the weight calculator 210 calculates the second information entropy based on the text-based similarity in the text-based similarity matrix generated by the text-based similarity matrix generator 130 .

Specifically, 'second information entropy' refers to the uncertainty of text-based similarity.

According to an embodiment, the weight calculator 210 may calculate the second information entropy by Equation 3 below.

[Equation 3]

는 제2 정보 엔트로피,

은 두 문서 간의 텍스트 기반 유사도에 대응되는 확률,

은 1-

, log는 밑을 e로 하는 자연로그를 나타낸다.At this time,

is the second information entropy,

is the probability corresponding to the text-based similarity between two documents,

Silver 1-

, log represents the natural logarithm with base e.

Next, the weight calculator 210 calculates a citation weight and a text weight based on the citation-based similarity, the text-based similarity, the first information entropy, and the second information entropy.

In this case, the sum of the citation weight and the text weight between two specific documents is 1.

3 is a diagram for explaining a process of calculating a citation weight and a text weight according to an embodiment.

에 기초하여 산출된 제1 정보 엔트로피

와 텍스트 기반 유사도 행렬 내 i행 j열의 텍스트 기반 유사도

에 기초하여 산출된 제2 정보 엔트로피

를 비교한다.Referring to FIG. 3 , the weight calculator 210 calculates the citation-based similarity of row i and column j in the citation-based similarity matrix.

First information entropy calculated based on

and text-based similarity of row i and column j in text-based similarity matrix

The second information entropy calculated based on

compare

가

보다 큰 경우,

가 0.5 이상이면 인용 가중치

를

와 동일한 값으로 산출하고, 텍스트 가중치

를 1-

와 동일한 값으로 산출하며,

가 0.5 미만이면 인용 가중치

를 1-

와 동일한 값으로 산출하고, 텍스트 가중치

를

와 동일한 값으로 산출한다.Then, the weight calculation unit 210

end

greater than,

citation weight if is greater than or equal to 0.5

to

Calculated with the same value as , and text weight

1-

is calculated as the same value as

citation weight if is less than 0.5

1-

Calculated with the same value as , and text weight

to

is calculated with the same value as

가

보다 크지 않은 경우,

가 0.5 이상이면 인용 가중치

를 1-

와 동일한 값으로 산출하고, 텍스트 가중치

를

와 동일한 값으로 산출하며,

가 0.5 미만이면 인용 가중치

를

와 동일한 값으로 산출하고, 텍스트 가중치

를 1-

와 동일한 값으로 산출한다.In addition, the weight calculation unit 210

end

If not greater than

citation weight if is greater than or equal to 0.5

1-

Calculated with the same value as , and text weight

to

is calculated as the same value as

citation weight if is less than 0.5

to

Calculated with the same value as , and text weight

1-

is calculated with the same value as

Referring back to FIG. 2 , the joint similarity matrix generator 140 may calculate the joint similarity based on the citation-based similarity, the text-based similarity, the citation weight, and the text weight.

Specifically, the joint similarity matrix generator 140 may calculate the joint similarity by adding a value obtained by multiplying the citation weight and the citation-based similarity by the text weight and the text-based similarity.

The cluster generator 150 may generate a plurality of cluster scenarios by varying the number of clusters to be generated.

For example, the cluster generating unit 150 sets the number of generated clusters to 10, 15, and 20, respectively, and as a result of generating document clusters according to each case, the first cluster scenario, the second cluster scenario, and the third cluster You can create scenarios.

The cluster evaluator 220 selects an optimal cluster scenario from among the plurality of cluster scenarios generated by the cluster generator 150 .

According to an embodiment, the cluster evaluator 220 may select any one of a plurality of preset indices as a reference index, and select a cluster scenario when the reference index has a maximum value as an optimal cluster scenario.

For example, the cluster evaluation unit 220 may select a silhouette index as a reference index, and select a cluster scenario when the silhouette index has a maximum value as the optimal cluster scenario according to Equation 4 below. .

[Equation 4]

는 실루엣 지표,

는 문서 i와 같은 군집에 속한 다른 모든 문서 각각과 문서 i와의 거리의 평균(average dissimilarity),

는 문서 i가 속하지 않은 모든 군집들 내 문서 각각과 문서 i 간의 거리를 구하여 군집 별로 산출한 거리의 평균 중 최소값을 나타낸다.At this time,

is the silhouette indicator,

is the average dissimilarity of the distance from document i to each of all other documents belonging to the same cluster as document i,

denotes the minimum value among the averages of distances calculated for each cluster by finding the distance between each document and document i in all clusters to which document i does not belong.

Specifically, the cluster evaluator 220 may calculate the distance between documents based on the Euclidean distance, the cosine distance, the Manhattan distance, and the like, but is not limited thereto.

As another example, the cluster evaluation unit 220 selects a blurring mean-shift (BMS) index as a reference index, and selects a cluster scenario when the BMS index has a maximum value according to Equation 5 below as an optimal cluster scenario can do.

[Equation 5]

는 문서 i의 BMS 지표,

는 특정 군집 h,

는 특정 군집 h', i와 i'는

에 속하는 문서, j는

에 속하는 문서,

는 문서 i와 문서 i' 간의 코사인 거리(cosine distance),

는 문서 i와 문서 j 간의 코사인 거리를 나타낸다.At this time,

is the BMS indicator of document i,

is a specific cluster h,

is a specific cluster h', i and i' are

A document belonging to, j is

documents belonging to

is the cosine distance between document i and document i',

denotes the cosine distance between document i and document j.

As another example, when the cluster generating unit 150 generates one or more clusters using the Govern-Newman algorithm for elements of the joint similarity matrix, the cluster evaluation unit 220 determines the network between the plurality of documents based on the joint similarity. and calculates edge betweenness centrality for each edge in the created network. Thereafter, the cluster evaluation unit 220 removes the edge with the highest edge-mediated centrality and repeats the process of calculating modularity, and selects the cluster scenario when the calculated modularity is maximum as the optimal cluster scenario. .

Specifically, the cluster evaluator 220 may calculate edge-mediated centrality according to Equation 6 below, and may calculate modularity according to Equation 7 below.

[Equation 6]

는 노드

와 노드

를 잇는 엣지,

는 엣지

의 엣지 매개 중심성, V는 네트워크,

와

는 V에 속한 노드,

는 노드

와

사이의 최단 경로의 수,

는 노드

와

사이의 최단 경로 중 엣지

를 통과하는 경로의 수를 나타낸다.At this time,

is the node

and node

edge connecting

is the edge

Edge-mediated centrality of, V is the network,

Wow

is the node belonging to V,

is the node

Wow

number of shortest paths between

is the node

Wow

edge of the shortest path between

It represents the number of paths passing through.

[Equation 7]

One)

2)

는 전체 엣지 중 동일한 커뮤니티 i 내의 노드를 연결하는 엣지의 비율,

는 전체 엣지 중 커뮤니티 i 내의 노드와 커뮤니티 j 내의 노드를 연결하는 엣지의 비율을 나타낸다.In this case, Q is modularity,

is the proportion of edges connecting nodes within the same community i among all edges,

represents the ratio of edges connecting nodes in community i to nodes in community j among all edges.

According to an embodiment, the cluster evaluation unit 220 calculates a rank for each of a plurality of cluster scenarios for each of a plurality of preset indicators, calculates an average value for a plurality of cluster scenarios of the calculated rank, and calculates the average value A cluster scenario in the case of this minimum may be selected as an optimal cluster scenario.

For example, the first to fourth clustering scenarios calculated using the silhouette index as the reference index are ranked 1st, 2nd, 3rd, and 4th in order, and the first clustering scenario calculated using the BMS index as the reference index. When the ranks of the first to fourth cluster scenarios are 4, 1, 3, and 2 in the order, the average ranks of the first to fourth cluster scenarios are 2.5, 1.5, 3, and 3 in the order, so the cluster evaluator 220 ) may select the second clustering scenario having the minimum average rank as the optimal clustering scenario.

4 is a flowchart illustrating a method for generating a document cluster according to an exemplary embodiment.

The method illustrated in FIG. 4 may be performed, for example, by the above-described document group generating apparatus 100 .

Referring to FIG. 4 , first, the document group generating apparatus 100 obtains bibliographic information and text information for each of a plurality of documents ( 410 ).

Thereafter, the document cluster generating apparatus 100 generates a citation-based similarity matrix having a citation-based similarity between a plurality of documents as an element based on the obtained bibliographic information ( 420 ).

According to an embodiment, the document cluster generating apparatus 100 may calculate the citation-based similarity based on any one of the simultaneous citation strength and the bibliographic coupling strength.

Thereafter, the document cluster generating apparatus 100 generates a text-based similarity matrix having text-based similarity between a plurality of documents as an element based on the acquired text information ( 430 ).

According to an embodiment, the document cluster generating apparatus 100 may calculate the text-based similarity using a vector space-based similarity calculation method.

Thereafter, the document cluster generating apparatus 100 generates a joint similarity matrix having a joint similarity between a plurality of documents as an element based on the citation-based similarity matrix and the text-based similarity matrix ( 440 ).

Thereafter, the document cluster generating apparatus 100 generates one or more clusters by clustering a plurality of documents based on the joint similarity matrix ( S450 ).

In the illustrated flowchart, the method is described by dividing the method into a plurality of steps, but at least some of the steps are performed in a reversed order, are performed together in combination with other steps, are omitted, are performed separately, or are not shown. One or more steps may be added and performed.

5 is a flowchart illustrating a method for generating a document group according to an additional exemplary embodiment.

The method illustrated in FIG. 5 may be performed, for example, by the above-described document group generating apparatus 200 .

Referring to FIG. 5 , first, the document group generating apparatus 200 obtains bibliographic information and text information for each of a plurality of documents ( S510 ).

Thereafter, the document cluster generating apparatus 200 generates a citation-based similarity matrix having a citation-based similarity between a plurality of documents as an element based on the obtained bibliographic information ( 520 ).

Thereafter, the document cluster generating apparatus 200 generates a text-based similarity matrix having text-based similarity between a plurality of documents as an element based on the acquired text information ( 530 ).

Thereafter, the document cluster generating apparatus 200 calculates the first information entropy based on the citation-based similarity ( S540 ).

Thereafter, the document cluster generating apparatus 200 calculates a second information entropy based on the text-based similarity ( S550 ).

Thereafter, the document cluster generating apparatus 200 calculates a citation weight and a text weight based on the citation-based similarity, the text-based similarity, the first information entropy, and the second information entropy ( 560 ).

Thereafter, the document cluster generating apparatus 200 generates a joint similarity matrix having the joint similarity between a plurality of documents as an element based on the citation-based similarity, the text-based similarity, the citation weight, and the text weight ( 570 ).

Thereafter, the document cluster generating apparatus 200 generates one or more clusters by clustering the plurality of documents based on the joint similarity matrix ( 580 ).

Thereafter, the document cluster generating apparatus 200 selects an optimal cluster scenario from among the generated plurality of cluster scenarios ( 590 ).

According to an embodiment, the document cluster generating apparatus 200 may select any one of a plurality of preset indices as a reference index, and select a cluster scenario when the reference index has a maximum value as an optimal cluster scenario.

According to an embodiment, the document cluster generating apparatus 200 calculates a rank for each cluster scenario with respect to each of a plurality of preset indicators, calculates an average value for each cluster scenario with respect to the calculated rank, and then calculates A clustering scenario in which the averaged value is the minimum may be selected as the optimal clustering scenario.

In the illustrated flowchart, the method is described by dividing the method into a plurality of steps, but at least some of the steps are performed in a different order, are performed together in combination with other steps, are omitted, are performed separately, or are not shown. One or more steps may be added and performed.

6 is a block diagram illustrating and describing a computing environment 10 including a computing device suitable for use in example embodiments. In the illustrated embodiment, each component may have different functions and capabilities other than those described below, and may include additional components in addition to those described below.

The illustrated computing environment 10 includes a computing device 12 . In an embodiment, the computing device 12 may be the document cluster generating device 100 or 200 illustrated in FIG. 1 or FIG. 2 .

Computing device 12 includes at least one processor 14 , computer readable storage medium 16 , and communication bus 18 . The processor 14 may cause the computing device 12 to operate in accordance with the exemplary embodiments discussed above. For example, the processor 14 may execute one or more programs stored in the computer-readable storage medium 16 . The one or more programs may include one or more computer-executable instructions that, when executed by the processor 14, configure the computing device 12 to perform operations in accordance with the exemplary embodiment. can be

Computer-readable storage medium 16 is configured to store computer-executable instructions or program code, program data, and/or other suitable form of information. The program 20 stored in the computer readable storage medium 16 includes a set of instructions executable by the processor 14 . In one embodiment, computer-readable storage medium 16 includes memory (volatile memory, such as random access memory, non-volatile memory, or a suitable combination thereof), one or more magnetic disk storage devices, optical disk storage devices, flash It may be memory devices, other forms of storage medium accessed by computing device 12 and capable of storing desired information, or a suitable combination thereof.

Communication bus 18 interconnects various other components of computing device 12 , including processor 14 and computer readable storage medium 16 .

Computing device 12 may also include one or more input/output interfaces 22 and one or more network communication interfaces 26 that provide interfaces for one or more input/output devices 24 . The input/output interface 22 and the network communication interface 26 are coupled to the communication bus 18 . Input/output device 24 may be coupled to other components of computing device 12 via input/output interface 22 . Exemplary input/output device 24 may include a pointing device (such as a mouse or trackpad), a keyboard, a touch input device (such as a touchpad or touchscreen), a voice or sound input device, various types of sensor devices, and/or imaging devices. input devices, and/or output devices such as display devices, printers, speakers and/or network cards. The exemplary input/output device 24 may be included in the computing device 12 as a component constituting the computing device 12 , and may be connected to the computing device 12 as a separate device distinct from the computing device 12 . may be

Meanwhile, an embodiment of the present invention may include a program for performing the methods described in this specification on a computer, and a computer-readable recording medium including the program. The computer-readable recording medium may include program instructions, local data files, local data structures, etc. alone or in combination. The medium may be specially designed and configured for the present invention, or may be commonly used in the field of computer software. Examples of the computer-readable recording medium include hard disks, magnetic media such as floppy disks and magnetic tapes, optical recording media such as CD-ROMs and DVDs, and program instructions specially configured to store and execute program instructions such as ROMs, RAMs, flash memories, and the like. Hardware devices are included. Examples of the program may include high-level language codes that can be executed by a computer using an interpreter or the like as well as machine language codes such as those generated by a compiler.

Although representative embodiments of the present invention have been described in detail above, those of ordinary skill in the art will understand that various modifications are possible without departing from the scope of the present invention with respect to the above-described embodiments. . Therefore, the scope of the present invention should not be limited to the described embodiments, but should be defined by the following claims as well as the claims and equivalents.

10: Computing Environment
12: computing device
14: Processor
16: computer readable storage medium
18: communication bus
20: Program
22: input/output interface
24: input/output device
26: network communication interface
100, 200: document cluster generator
110: document information acquisition unit
120: Citation-based similarity matrix generator
130: text-based similarity matrix generator
140: joint similarity matrix generator
150: cluster generator
210: weight calculation unit
220: cluster evaluation unit

Claims (18)

a document information obtaining unit which obtains bibliographic information and text information for each of the plurality of documents;
a citation-based similarity matrix generator for generating a citation-based similarity matrix having, as an element, a citation-based similarity between the plurality of documents based on the bibliographic information;
a text-based similarity matrix generator configured to generate a text-based similarity matrix having text-based similarity between the plurality of documents as an element based on the text information;
a joint similarity matrix generator configured to generate a joint similarity matrix having the joint similarity between the plurality of documents as an element based on the citation-based similarity matrix and the text-based similarity matrix; and
and a cluster generator configured to generate one or more clusters by clustering the plurality of documents based on the joint similarity matrix.
The method according to claim 1,
The citation-based similarity matrix generator,
Calculate the citation-based similarity based on any one of the co-citation strength and the bibliographic coupling strength,
The text-based similarity matrix generator,
and calculating the text-based similarity by using a vector space-based similarity calculation method.
The method according to claim 1,
Calculate a first information entropy based on the citation-based similarity and a second information entropy based on the text-based similarity, and cite based on the citation-based similarity, the text-based similarity, the first information entropy and the second information entropy The apparatus for generating a group of documents further comprising a weight calculation unit for calculating a weight and a text weight.
4. The method according to claim 3,
The joint similarity matrix generator,
and calculating the combined similarity based on the citation-based similarity, the text-based similarity, the citation weight, and the text weight.
The method according to claim 1,
The cluster generating unit,
k-means algorithm, density-based clustering, probability distribution-based clustering, hierarchical clustering, and Girvan-Newman algorithm An apparatus for generating the one or more clusters by using any one of them.
The method according to claim 1,
The cluster generating unit,
A document cluster generating apparatus for generating a plurality of cluster scenarios by varying the number of clusters to be generated.
7. The method of claim 6,
and a cluster evaluation unit for selecting an optimal cluster scenario from among the plurality of cluster scenarios.
8. The method of claim 7,
The cluster evaluation unit,
An apparatus for generating a document cluster, selecting any one of a plurality of preset indices as a reference index, and selecting a clustering scenario when the reference index has a maximum value as the optimal clustering scenario.
8. The method of claim 7,
The cluster evaluation unit,
Calculating the rank for each of the plurality of clustering scenarios with respect to each of a plurality of preset indices, calculating an average value for each of the plurality of clustering scenarios of the calculated rank, and calculating the clustering scenario when the calculated average value is the minimum. A document cluster generating device that selects the optimal cluster scenario.
obtaining bibliographic information and text information for each of a plurality of documents;
generating a citation-based similarity matrix having, as an element, a citation-based similarity between the plurality of documents based on the bibliographic information;
generating a text-based similarity matrix having text-based similarity between the plurality of documents as an element based on the text information;
generating a joint similarity matrix having the joint similarity between the plurality of documents as an element based on the citation-based similarity matrix and the text-based similarity matrix; and
and generating one or more clusters by clustering the plurality of documents based on the joint similarity matrix.
11. The method of claim 10,
The generating of the citation-based similarity matrix includes calculating the citation-based similarity based on any one of a co-citation strength and a bibliographic coupling strength,
The generating of the text-based similarity matrix includes calculating the text-based similarity using a vector space-based similarity calculation method.
11. The method of claim 10,
calculating a first information entropy based on the citation-based similarity;
calculating second information entropy based on the text-based similarity; and
The method further comprising the step of calculating a citation weight and a text weight based on the citation-based similarity, the text-based similarity, the first information entropy, and the second information entropy.
13. The method of claim 12,
The generating of the joint similarity matrix includes calculating the joint similarity based on the citation-based similarity, the text-based similarity, the citation weight, and the text weight.
11. The method of claim 10,
The step of generating the cluster includes k-means algorithm, density-based clustering, probability distribution-based clustering, hierarchical clustering, and Govern- A method for generating a cluster of documents, wherein the one or more clusters are generated using any one of a Girvan-Newman algorithm.
11. The method of claim 10,
The generating of the cluster includes generating a plurality of cluster scenarios by varying the number of generated clusters.
16. The method of claim 15,
and selecting an optimal cluster scenario from among the plurality of cluster scenarios.
17. The method of claim 16,
The selecting may include selecting any one of a plurality of preset indices as a reference index, and selecting a cluster scenario when the reference index has a maximum value as the optimal cluster scenario.
17. The method of claim 16,
The selecting step is
calculating a ranking for each of the plurality of cluster scenarios for each of a plurality of preset indicators;
calculating an average value for each of the plurality of cluster scenarios of the calculated rank; and
and selecting a cluster scenario in which the calculated average value is minimum as the optimal cluster scenario.

Images