JP2006338157A - Document group processor, document group processing method, document group processing program and recording medium with the same program stored therein - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a method for extracting a topic group adaptive for a document group by using an existing topic group attached to an existing document group and a method for classifying document groups for every topic suitable for the document group. <P>SOLUTION: This document group processing method comprises an existing topic group probability model construction step for constructing the probability model of an existing topic group, an adaptive topic group extraction step for extracting a topic group matched with a document group by using the probability model constructed by the existing topic group probability model construction step and a document group classification step for classifying the document groups for every adaptive topic group extracted by the suitable topic group extraction step. <P>COPYRIGHT: (C)2007,JPO&INPIT

Description

  The present invention relates to a document group processing apparatus, a document group processing method, a document group processing program, a document group processing program, and a recording medium storing a document group processing program.

Clustering a document group for each topic enables a user to easily find a target document. Here, clustering is a technique for grouping similar objects by grouping similar objects together. Most of the clustering is done manually, but in recent years, a large number of electronic documents are being accumulated, and the technology of mechanical clustering has become extremely important, and many clustering methods have been proposed so far. (For example, refer nonpatent literature 1).
R.Duda, P.Hart, and D.Stork, translated by Morio Oshiage, “Pattern Identification”, 2nd Edition, (USA), New Technology Communications, 2001 519-601

  However, for example, the technique disclosed in Non-Patent Document 1 cannot appropriately cluster a document group. Moreover, an appropriate topic cannot be assigned to each class generated as a result of clustering.

  For example, in the case of a Web page, in a directory type search engine such as a category search of a search site goo (registered trademark) (http://www.goo.ne.jp), a large number of pages are labeled for each topic. . However, 240 or more topics exist even in the second level of the category search of goo (registered trademark), and even if a document group is clustered using all topics, a useful clustering result cannot be obtained. In order to enable a user to easily find a target document, it is considered that the number of topics is preferably about 5 to 20. Therefore, since it is necessary to obtain an appropriate number of topic groups, it is necessary to select a topic that matches the document group from among these many topics.

  Therefore, the present invention has been made in view of the above problems, and a document group capable of extracting a topic group suitable for the document group using the existing topic group attached to the existing document group. It is an object of the present invention to provide a processing device, a document group processing method, a document group processing program, and a recording medium storing the document group processing program. Another object of the present invention is to provide a document group processing method, a document group processing program, and a recording medium storing the document group processing program, which further allow clustering of document groups for each extracted relevant topic group.

  The present invention was created to solve the above-described problems, and the document group processing apparatus according to the present invention includes an existing topic group probability model construction unit that constructs a probability model of an existing topic group, and an existing topic group probability model. The configuration includes a matching topic group extraction unit that extracts a topic group that matches a document group by using the probability model constructed in the construction unit.

  According to such a configuration, the document group processing apparatus can extract a topic group suitable for the document group by using the existing topic group attached to the existing document group. Here, the topic group suitable for the document group is a group of topics preferable as a topic group for explaining the document group, and means an appropriate number of groups.

  Further, the document group processing method of the present invention is adapted to a document group using an existing topic group probability model construction step for constructing a probability model of an existing topic group and a probability model constructed by the existing topic group probability model construction step. And a matching topic group extraction step for extracting a topic group to be performed.

  According to such a method, the document group processing apparatus can extract a topic group that matches the document group by using the existing topic group attached to the existing document group.

  The document group processing method of the present invention is adapted to a document group by using an existing topic group probability model construction step for constructing a probability model of an existing topic group and a probability model constructed by the existing topic group probability model construction step. The method includes a matching topic group extraction step for extracting a topic group to be performed, and a document group classification step for classifying the document group for each matching topic group extracted by the matching topic group extraction step.

  According to such a configuration, the document group processing apparatus can extract a topic group suitable for the document group by using the existing topic group attached to the existing document group. Furthermore, it is possible to cluster document groups for each extracted matching topic group.

  Further, the existing topic group probability model construction step in the document group processing method of the present invention is a method including a word occurrence probability estimation step for estimating a word occurrence probability using the word appearance frequency of the existing document group. The word appearance frequency and the word occurrence probability will be described in detail in the best mode for carrying out the invention.

  According to such a method, the document group processing apparatus can estimate the word occurrence probability using the word appearance frequency of the existing document group. Further, it is possible to extract a topic group that matches the document group using the estimated word occurrence probability.

  In the document group processing method of the present invention, the matching topic group extraction step uses a word occurrence probability to calculate the likelihood of the topic group, and the topic group likelihood calculated by the likelihood calculation step. And a topic group extraction step of extracting topic groups using the degree.

  According to such a method, the document group processing apparatus can calculate the likelihood of the topic group using the word occurrence probability. Further, it is possible to extract a topic group that matches the document group using the likelihood of the topic group obtained by the calculation.

  The document group classification step in the document group processing method of the present invention is a method including a classification step of classifying a document group based on a calculation using likelihood.

  According to such a method, the document group processing apparatus can classify the document group based on the calculation using the likelihood.

  Further, according to the document group processing program for causing a computer to execute such a document group processing method, it is possible to cause the computer to execute the same function as that of the document group processing apparatus. Furthermore, according to a recording medium storing a program for causing a computer to execute such a document group processing method, it is possible to store in the recording medium a program for causing the computer to execute the same function as the document group processing apparatus. .

  According to the present invention, it is possible to extract a topic group that matches a document group using the existing topic group attached to the existing document group. In addition, it is possible to cluster document groups for each extracted matching topic group.

  Next, the best mode for carrying out the present invention (hereinafter referred to as “embodiment”) will be described with reference to the drawings.

  FIG. 1 is a block diagram of a document group processing apparatus according to an embodiment of the present invention. As shown in FIG. 1, the document group processing apparatus 1 includes a calculation unit 2, an input unit 3, a storage unit 4, and an output unit 5. Each means is connected to the bus line 11.

  The computing means 2 includes an existing topic group probability model construction unit 21, a compatible topic group extraction unit 22, and a memory 23. The computing means 2 reads the existing topic group probability model construction program 41 and the compatible topic group extraction program 42 from the storage means 4, stores them in the memory 23, and executes them, whereby the existing topic group probability model construction unit 21 and the compatible topic groups are executed. The extraction unit 22 is realized. Details of the configurations of the existing topic group probability model construction unit 21 and the compatible topic group extraction unit 22 will be described later. The computing means 2 includes, for example, a CPU (Central Processing Unit) that performs arithmetic processing and a RAM (Random Access Memory) that stores information.

  The input means 3 includes a keyboard and a disk drive device. The existing document group and the document group to be clustered can be input via the input unit 3 and stored in the storage unit 4.

  The storage means 4 is composed of a hard disk device or the like. The storage means 4 can store an existing topic group probability model construction program 41 and a compatible topic group extraction program 42 that are the basis of the existing topic group probability model construction unit 21 and the matching topic group extraction unit 22. . The storage unit 4 includes an existing document group table 43, a word occurrence probability table 44, and a document group table 45.

  Here, the existing document group table 43 is a table for storing a page set with a topic label, and is a table for storing a page set registered in a directory type search engine or the like.

  The word occurrence probability table 44 is a table for storing the word occurrence probabilities calculated by the existing topic group probability model construction unit 21.

  Further, the document group table 45 is a table for storing a set of pages for clustering, and is a table for storing a set of pages obtained by searching with a directory search engine or the like.

  The output means 5 is, for example, a graphic board and a monitor connected thereto, and displays the result of clustering the document group.

  Hereinafter, the configurations of the existing topic group probability model construction unit 21 and the matching topic group extraction unit 22 will be described with reference to FIGS. 2 and 3. Here, the existing topic group probability model construction unit 21 is called by the computing means 2, and when the processing of the existing topic group probability model construction unit 21 ends, the suitable topic group extraction unit 22 is called by the computing means 2.

(Description of the existing topic group probability model construction unit 21)
FIG. 2 is a block diagram of the existing topic group probability model construction unit according to the embodiment of the present invention. As shown in FIG. 2, the existing topic group probability model construction unit 21 includes an existing document group reading unit 211, a word occurrence probability estimation unit 212, and a word occurrence probability writing unit 213.

(Description of Existing Document Group Reading Unit 211)
The existing document group reading unit 211 reads the existing document group X ′ from the existing document group table 43 (see FIG. 1) and stores it in the memory 23. The existing document group table 43 (see FIG. 1) is a set of pages with labels as described above. X ′ is defined by X ′ _k (k = 1,..., K ′). Here, K ′ is the number of topics of the entire existing document group. X ′ _k is a document set belonging to the topic k, and is defined by the following equation (1).

Here, N ′ _k represents the number of pages belonging to the topic k. Each element x ′ _kn is a V-dimensional word appearance frequency vector in the page x ′ _n belonging to the topic k, and V represents the total number of words. The total number of words means the total number of words in the entire existing document group X ′. The word appearance frequency vector x ′ _kn is defined by the following equation (2).

Here, x ′ _knj represents the appearance frequency of the word w _j in the page x ′ _n belonging to the topic k. The appearance frequency represents the number of times a specific word appears within a certain page range.

(Description of word occurrence probability estimation unit 212)
The word occurrence probability estimation unit 212 calculates a word occurrence probability θ _kj based on the existing document group X ′ _k (k = 1,..., K ′) stored in the memory 23 by the existing document group reading unit 211, It is stored in the memory 23. Here, the word occurrence probability θ _kj means the appearance probability of the word w _{j on} the page belonging to the topic k. The word occurrence probability θ _kj can be estimated by the following equation (3), for example, by adopting an NB (Naive Bayes) model as a probability model of each topic.

The NB model is described in, for example, “McCallum, A., Nigam, K. (1998) A comparison of event models for naive Bayes text classification. In: AAAI-98 Workshop on Learning for Text Categorization”. . λ _k is a smoothing parameter and can be calculated using a cross-validation method. About the cross-validation method, it describes in above-mentioned nonpatent literature 1, for example.

(Description of word occurrence probability writing unit 213)
The word occurrence probability writing unit 213 stores the word occurrence probability θ _kj estimated by the word occurrence probability estimation unit 212 and stored in the memory 23 in the word occurrence probability table 44 (see FIG. 1). The word occurrence probability θ _kj stored in the word occurrence probability table 44 (see FIG. 1) is used by the matching topic group extraction unit 22.

(Description of relevant topic group extraction unit 22)
FIG. 3 is a block diagram of an adapted topic group extraction unit according to the embodiment of the present invention. As shown in FIG. 3, the matching topic group extraction unit 22 includes a word occurrence probability reading unit 221, a document group reading unit 222, an existing topic group acquisition unit 223, a log likelihood calculation unit 224, and a topic group sorting unit. 225 and a topic deletion unit 226.

(Description of word occurrence probability reading unit 221)
The word occurrence probability reading unit 221 reads the word occurrence probability θ _kj from the word occurrence probability table 44 (see FIG. 1) and stores it in the memory 23.

(Description of document group reading unit 222)
The document group reading unit 222 reads the document group X from the document group table 45 (see FIG. 1) and stores it in the memory 23. As described above, the document group table 45 (see FIG. 1) is a set of pages for clustering. X is defined by x _n (n = 1,..., N). Here, N is the number of pages of the entire document group. In addition, x _n is the word appearance frequency vector of V dimension in the page X _n, V represents the total number of words. As described above, the total number of words means the total number of words in the entire existing document group X ′. The word appearance frequency vector x _n is defined by the following equation (4) using the word appearance frequency x _nj .

ここで、単語出現頻度ｘ_njは、ページＸ_nにおける単語ｗ_jの出現頻度を意味する。

Here, the word appearance frequency x _nj means the appearance frequency of the word w _j in the page X _n .

(Description of the existing topic group acquisition unit 223)
The existing topic group acquisition unit 223 acquires the existing topic group G ′ from the word occurrence probability θ _kj stored in the memory 23 by the word occurrence probability reading unit 221 and stores it in the memory 23. Here, the existing topic group G ′ acquired by the existing topic group acquisition unit 223 is defined by G ′ = {1,..., K ′}. Since the word occurrence probability θ _kj is a value obtained by designating the topic k and the word w _j , the word occurrence probability θ _kj has information on the existing topic group G ′ {1,..., K ′}. Further, by substituting the existing topic group G ′ for the matching topic group G, an initial value is set for the matching topic group G. The matching topic group G is used to extract topics that match the document group X, and satisfies G を 満 た す {1,..., K ′}.

(Description of Log Likelihood Calculation Unit 224)
The log likelihood calculation unit 224 uses the word occurrence probability θ _kj read into the memory 23 by the word occurrence probability reading unit 221 and the document group X read into the memory 23 by the document group reading unit 222 to calculate the logarithm. The likelihood logp (x _n | k) is calculated and stored in the memory 23. The log likelihood logp (x _n | k) is calculated between the page X _n (n = 1,..., N) of the document group X and the topic k (k = 1,..., K ′) of the existing document group X ′. Calculated for all combinations. The log likelihood logp (x _n | k) is calculated by the following equation (5).

(Description of Topic Group Sorting Unit 225)
The topic group sorting unit 225 calculates the estimated topic c _n (G) using the log likelihood logp (x _n | k) calculated by the log likelihood calculation unit 224 and stored in the memory 23, and the memory 23 is stored. The estimated topic c _n (G) is calculated by the following equation (6).

The topic group sorting unit 225 calculates the number of pages N _k (k = 1,..., K ′) corresponding to the estimated topic k using the estimated topic c _n (G) stored in the memory 23. Are stored in the memory 23.

Further, the topic group sorting unit 225 selects the estimated topic k as the applicable topic group in the memory 23 when there is no page corresponding to the estimated topic k, that is, when the number of pages N _k corresponding to the estimated topic k is not positive. The process of deleting from G is performed for the estimated topic k (k = 1,..., K ′). Here, K ′ is the number of elements of the existing topic group G ′. By this processing, the topic group sorting unit 225 can extract the matching topic group G = {1,..., K} where the corresponding page exists.

Further, the topic group sorting unit 225 creates a list L = {L ₁ ,..., L _K } in which the matching topic group G is sorted in ascending order of the number of pages N _k of the estimated topic k in the memory 23. To store.

(Description of Topic Deletion Unit 226)
The topic deletion unit 226 deletes a topic that does not correspond to the conforming topic in order to extract the optimum conforming topic group G by selecting the conforming topic group G that minimizes the value of AIC (Akaike's Information Criterion). It is. The AIC is described in, for example, “Akaike, H. (1973). Information theory and extension of the maximum likelihood principle”.

As a method of selecting a model that minimizes the value of AIC, for example, when AIC (G) is compared with AIC (G- _m ) and AIC (G)> AIC (G- _m ) is satisfied, it is suitable. It can be considered that the process of deleting L _m from the topic group G is performed in order for m = 1,..., K, that is, in order from the topic having the smallest page number N _k . Here, G− _m is obtained by removing L _m from the matching topic group G. The value of AIC (G) is calculated using the following formula (7) using the matching topic group G, the log likelihood logp (x _n | k), and the estimated topic c _n (G) in the memory 23. Is done.

ここで、｜Ｇ｜は適合トピック群Ｇの要素数を表すものである。ＡＩＣ（Ｇ_−m）の値も同様の方法で計算することが可能である。

Here, | G | represents the number of elements of the matching topic group G. The value of AIC (G _−m ) can be calculated by the same method.

Note that when the matching topic group G is changed, the values of the estimated topics c _n (G) and AIC (G) are also changed. Therefore, after deleting L _m from the matching topic group G, the estimated topics c _n When the values of (G) and AIC (G) are used again, it is necessary to update the values of the estimated topics c _n (G) and AIC (G) in the memory 23 to correct values. . The same applies to the topic group G- _m .

  In addition, the topic deletion unit 226 deletes the topic k having no corresponding page from the matching topic group G in advance, thereby sorting the matching topic group G including the topic k having no corresponding page, It is possible to omit the AIC (G) calculation process.

  Furthermore, if the matching topic group G and the document group X extracted by the matching topic group extraction unit 22 are applied to Expression (7), the document group X can be clustered for each matching topic group G. .

  Next, processing performed by the computing means 2 (see FIG. 1) will be described with reference to FIG. 4 (see FIG. 1 as appropriate). FIG. 4 is a flowchart showing the processing of the calculation means according to the embodiment of the present invention.

  As shown in FIG. 4, the computing means 2 first performs an existing topic group probability model construction process by the existing topic group probability model construction unit 21 (S10). Subsequently, a matching topic group extraction process by the matching topic group extraction unit 22 is performed (S20), and the process ends. Hereinafter, details of the process of S10 will be described with reference to FIG. Details of the process of S20 will be described with reference to FIGS.

  Processing performed by the existing topic group probability model construction unit 21 (see FIG. 1) will be described with reference to FIG. 5 (see FIGS. 1 and 2 as appropriate). FIG. 5 is a flowchart showing processing of the existing topic group probability model construction unit according to the embodiment of the present invention.

As shown in FIG. 5, the existing document group reading unit 211 reads the existing document group X ′ from the existing document group table 43 and stores it in the memory 23 (S11). Subsequently, the word occurrence probability estimation unit 212 estimates the word occurrence probability θ _kj based on the existing document group stored in the memory 23 and stores it in the memory 23 (S12). The word occurrence probability θ _kj can be calculated by the above equation (3). Then, the word occurrence probability writing unit 213 writes the word occurrence probability θ _kj stored in the memory 23 in the word occurrence probability table 44 (S13), and ends the process.

  The processing performed by the adapted topic group extraction unit 22 (see FIG. 1) will be described with reference to FIGS. 6, 7 and 8 (see FIGS. 1 and 3 as appropriate). 6, 7 and 8 are flowcharts showing the processing of the adapted topic group extraction unit according to the embodiment of the present invention.

As shown in FIG. 6, the word occurrence probability reading unit 221 reads the word occurrence probability θ _kj from the word occurrence probability table 44 and stores it in the memory 23 (S201). Subsequently, the document group reading unit 222 reads the document group X from the document group table 45 and stores it in the memory 23 (S202).

The existing topic group acquisition unit 223 acquires the existing topic group G ′ from the word occurrence probability θ _kj in the memory 23 and stores it in the memory 23 (S203). Also, a suitable topic group G is secured in the memory 23, and an initial value is set by substituting the existing topic group G ′ (S204).

The log likelihood calculation unit 224 calculates the log likelihood logp (x _n | k) using the word occurrence probability θ _kj and the document group X in the memory 23 and stores them in the memory 23 (S205). The log likelihood logp (x _n | k) can be calculated by the above-described equation (5).

Next, as shown in FIG. 7, the topic group sorting unit 225 calculates the estimated topic c _n (G) using the log likelihood logp (x _n | k) in the memory 23 and stores it in the memory 23. (S206). The estimated topic c _n (G) can be calculated by the equation (6) described above. Subsequently, the estimated number of pages N _k of the estimated topic _k is calculated using the estimated topic c _n (G) in the memory 23 and stored in the memory 23 (S207).

The topic group sorting unit 225 secures the counter t in the memory 23 and sets 1 to t (S208). Subsequently, it is determined whether the page number N _t is positive (S209). If the page number N _t is not positive (No in S209), the topic t is deleted from the matching topic group G in the memory 23 (S210), and the process proceeds to S211. If the page number _Nt is positive (Yes in S209), nothing is done and the process proceeds to S211. When the process proceeds to S211, 1 is added to the counter t (S211), and it is determined whether or not the counter t exceeds the number of elements K ′ of the existing topic group G ′ (S212). When t does not exceed the number of elements K ′ of the existing topic group G ′ (No in S212), the process returns to S209. When t exceeds the number of elements K ′ of the existing topic group G ′ (Yes in S212), the process proceeds to S213.

When the process proceeds to S213, the topic group sorting unit 225 creates a list L = [L ₁ ,..., L _K ] in which the matching topic group G is sorted in ascending order of the number of pages N _k of the estimated topic k in the memory 23. And stored in the memory 23 (S213).

Next, as illustrated in FIG. 8, when the process proceeds to S <b _> 214, the topic deletion unit 226 displays the matching topic group G, the log likelihood logp (x _n | k), and the estimated topic c _n (G) in the memory 23. Using this, AIC (G) is calculated and stored in the memory 23 (S214). AIC (G) can be calculated by the above-described equation (7). Subsequently, a counter m is secured in the memory 23, and 1 is set to m (S215).

The topic deletion unit 226 uses L _m as the topic group G _−m deleted from the matching topic group G, and uses the log likelihood logp (x _n | k) in the memory 23 to estimate the topic c _n (G _−m ). Is calculated and stored in the memory 23 (S216). Subsequently, AIC (G _−m ) is calculated using G _−m , log likelihood logp (x _n | k), and estimated topic c _n (G _−m ), and is stored in the memory 23 (S217). Then, the values of AIC (G) and AIC (G _−m ) are compared. If AIC (G _−m ) is not smaller than AIC (G) (No in S218), the process proceeds to S222. AIC (Yes in S218) _{(G -m)} is AIC (G) is smaller than the case, remove the L _m from the fitted topic group G (S219), the value of _{c n (G) c n (} G -m ) (S220), the value of AIC (G) is updated with AIC (G- _m ) (S221), and the process proceeds to S222.

When the process proceeds to S222, 1 is added to the counter m (S222), and it is determined whether or not the counter m exceeds the number K of elements in the list L (S223). If m does not exceed the number K of elements in the list L (No in S223), the process returns to S216. If m exceeds the number K of elements in the list L (Yes in S223), the estimated topic c _n is calculated using the log likelihood logp (x _n | k) in the memory 23 and the extracted matching topic group G. (G) is calculated and stored in the memory 23 (S224), and the process is terminated.

  Through the above steps, the document group processing apparatus 1 can extract the relevant topic group G. Further, by applying the extracted matching topic group G and the document group X to Expression (7), the document group X can be clustered for each matching topic group G.

  Note that the document group processing apparatus 1 according to this embodiment applies the extracted matching topic group G and document group X to Expression (7) to cluster the document group X for each matching topic group G. Although a function is also provided, it is also possible to realize a document group processing apparatus simply as an apparatus having a function of extracting a suitable topic group G that matches the document group X.

  In this embodiment, AIC is used as a method for selecting the matching topic group G. However, the method for selecting the matching topic group G is not limited to this. For example, model selection criteria such as MDL (Minimum Description Length) can be used instead of AIC. For MDL, see, for example, “Rissanen, J., (1983) .A universal prior for integers and estimation by minimum description length, The annals of Statistics, Vol.11, NO.2, pp.416-431”. Are listed.

  In this embodiment, the number of conforming topics is not limited. However, the number of conforming topics is limited, and when the number of conforming topics falls below the limited value, the process of selecting conforming topics is terminated. Is also possible. By limiting the number of applicable topics, it is possible to extract appropriate topics with an appropriate number of topics.

(Evaluation of document group processing device)
In order to evaluate the effectiveness of the document group processing apparatus in the embodiment of the present invention, clustering of page groups obtained from Web search results was performed. The existing topic group used is 242 topics in the second level of category search of goo (registered trademark), and a probability model of each topic is constructed using 74233 pages included therein. The total number of words V here was 50129.

  As a result of clustering about 1000 pages of search results when “hub” is used as a search term by the document group processing apparatus in the embodiment of the present invention, “biology”, “pet”, “cooking, gourmet”, “auction”, A total of 12 topics were extracted: “specialized store”, “computer shop”, “hardware”, “network related”, “business news”, “humanities”, “dictionary, dictionary”, “broadband knowledge”. .

  Although it is difficult to attach each topic label (biology, pet, etc.) when using the conventional clustering method, easy-to-understand labels manually attached are attached to the clustering in the embodiment of the present invention. .

  As pages corresponding to each topic label, “biology”, “pets” are pages about snake hubs, “cooking and gourmet” are pages about hub teas, “auctions”, “special stores”, “computer shops” In "Hardware" and "Network-related", a page on network hubs was found, and in "Humanities", a page on research projects called the historical material hub regional culture research center was found.

  “Hub” has various meanings, and it can be expected that search efficiency is improved by classifying each topic in this way. In addition to improving search efficiency, it is also possible to know what meaning a search term is used on the Web. In this case, for example, a page related to a research project called “Historical Resource Hub Regional Culture Research Center” can be found, and it can be found that it is also used in “humanities”.

It is a block diagram of a document group processing apparatus according to an embodiment of the present invention. It is a block diagram of the existing topic group probability model construction part which concerns on embodiment of this invention. It is a block diagram of the suitable topic group extraction part which concerns on embodiment of this invention. It is a flowchart showing the process of the calculating means which concerns on embodiment of this invention. It is a flowchart showing the process of the existing topic group probability model construction part which concerns on embodiment of this invention. It is a flowchart showing the process of the suitable topic group extraction part which concerns on embodiment of this invention. It is a flowchart showing the process of the suitable topic group extraction part which concerns on embodiment of this invention. It is a flowchart showing the process of the suitable topic group extraction part which concerns on embodiment of this invention.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 Document group processing apparatus 2 Calculation means 3 Input means 4 Storage means 5 Output means 11 Bus line 21 Existing topic group probability model construction part 22 Conforming topic group extraction part 23 Memory 41 Existing topic group probability model construction program 42 Conforming topic group extraction program 43 existing document group table 44 word occurrence probability table 45 document group table 211 existing document group reading unit 212 word occurrence probability estimating unit 213 word occurrence probability writing unit 221 word occurrence probability reading unit 222 document group reading unit 223 existing topic group acquiring unit 224 Log likelihood calculation unit 225 Topic group sorting unit 226 Topic deletion unit

Claims (8)

A document group processing apparatus that extracts a topic group that matches a document group using the existing topic group attached to the existing document group,
The document group processing device includes:
An existing topic group probability model building unit for building a probability model of the existing topic group;
A document group processing apparatus comprising: an adapted topic group extraction unit that extracts a topic group that matches the document group using the probability model constructed in the existing topic group probability model construction unit. A document group processing method by a document group processing apparatus for extracting a topic group suitable for a document group using an existing topic group attached to an existing document group,
The document group processing method includes:
An existing topic group probability model construction step of constructing a probability model of the existing topic group;
A document group processing method comprising: a matching topic group extraction step of extracting a topic group that matches the document group using the probability model constructed by the existing topic group probability model construction step. A document group processing method by a document group processing apparatus that extracts a topic group that matches a document group using an existing topic group attached to an existing document group and classifies the document group for each extracted conforming topic group. ,
The document group processing method includes:
An existing topic group probability model construction step of constructing a probability model of the existing topic group;
A matching topic group extraction step of extracting a topic group that matches the document group using the probability model constructed by the existing topic group probability model construction step;
A document group processing method comprising: a document group classification step for classifying the document group for each of the matching topic groups extracted by the matching topic group extraction step. The existing topic group probability model construction step includes:
The document group processing method according to claim 2, further comprising a word occurrence probability estimating step of estimating a word occurrence probability using a word appearance frequency of the existing document group. The matching topic group extraction step includes:
A likelihood calculating step of calculating the likelihood of the topic group using the word occurrence probability;
5. The document group processing method according to claim 2, further comprising: a topic group extracting step of extracting a topic group using the likelihood of the topic group calculated by the likelihood calculating step. The document group classification step includes:
The document group processing method according to claim 3, further comprising: a classification step of classifying the document group based on calculation using likelihood.   A document group processing program for causing a computer to execute the document group processing method according to claim 2.   A recording medium storing the document group processing program according to claim 7.

Images