JP6194760B2 - Keyword generation method, program, and information processing apparatus - Google Patents

Description

  The present invention relates to a technique for generating a keyword related to text search.

  For example, even if a document in a text database is searched in accordance with a keyword specified by the user, what matches the user's intention is not always extracted.

  It is troublesome for the user to come up with another keyword when trying to extract a target document that has been missed. For example, the next keyword may be selected from the words included in the previously extracted document, but the candidates are enormous.

  If there are many keyword candidates, the processing time becomes longer. On the other hand, if the words that are keyword candidates are limited, an effective keyword may not be generated, and the target document may be missed again.

Japanese Patent Laid-Open No. 10-240743

  An object of the present invention is, in one aspect, to efficiently extract keywords from a set of documents.

  A keyword generation method according to one aspect includes a detection process for detecting one or a plurality of partial character strings whose appearance frequency is equal to or higher than a reference value in a set of documents, and a relationship between the detected partial character strings and the set. A score indicating strength is calculated, a selection process for selecting a keyword from the partial character string based on the score, a number of newly selected keywords is calculated, and based on the number of the keywords, a detection process and A determination process for determining whether to end the selection process, and an iteration for repeating the detection process, the selection process, and the determination process by updating the reference value when it is determined not to end the detection process and the selection process Processing.

  As one aspect, keywords can be efficiently extracted from a set of documents.

FIG. 1 is a diagram showing an outline of a network. FIG. 2 is a diagram illustrating an example of text data. FIG. 3 is a diagram illustrating a module configuration example of the extraction device. FIG. 4 is a diagram illustrating an example of the first extraction result. FIG. 5 is a diagram showing a main processing flow. FIG. 6 is a diagram illustrating an internal module configuration example of the first extraction unit. FIG. 7 is a diagram showing a first extraction process flow. FIG. 8 is a diagram illustrating an internal module configuration example of the second extraction unit. FIG. 9 is a diagram showing a second extraction process flow. FIG. 10 is a diagram illustrating the characteristics of the number of partial character strings with respect to the frequency. FIG. 11 is a diagram illustrating characteristics of the number of keywords with respect to frequency. FIG. 12 is a diagram illustrating an internal module configuration example of the keyword generation unit. FIG. 13 is a diagram showing a keyword generation processing flow. FIG. 14 is a diagram showing a partial character string detection processing flow. FIG. 15 is a diagram illustrating an example of a detection result. FIG. 16 is a diagram showing a keyword selection processing flow. FIG. 17 is a diagram illustrating a score calculation processing flow. FIG. 18 is a diagram illustrating a first appearance probability calculation process flow. FIG. 19 is a diagram illustrating a second appearance probability calculation process flow. FIG. 20 is a diagram illustrating an example of a score. FIG. 21 is a diagram showing the end determination processing flow. FIG. 22 is a diagram illustrating an example of a keyword frequency distribution table. FIG. 23 is an example of a keyword frequency distribution diagram. FIG. 24 is a diagram illustrating an example of a keyword frequency distribution diagram. FIG. 25 is a diagram showing a flow of the lowest frequency update process (A). FIG. 26 is a diagram illustrating an example of a detection result. FIG. 27 is a diagram illustrating an example of a score. FIG. 28 is a diagram illustrating an example of a keyword table. FIG. 29 is a diagram illustrating an example of a detection result. FIG. 30 is a diagram illustrating an example of a score. FIG. 31 is a diagram illustrating an example of a keyword table. FIG. 32 is a diagram showing a flow of the lowest frequency update process (B). FIG. 33 is a diagram illustrating a minimum frequency update process (C) flow. FIG. 34 is a functional block diagram of a computer.

[Embodiment 1]
FIG. 1 shows an outline of the network. The extraction apparatus 101 and the database management system 103 are connected via a network. The database management system 103 has a text database 105. The text database 105 stores text data. The text data includes a plurality of text units. The text unit is a unit of data managed by the text database 105 and includes character string data. The text unit may include additional data other than text, such as image data, photo data, or audio data. The text unit in this example is a simple blog article. The text unit may be a document other than a blog article.

  An example of text data stored in the text database 105 will be described. FIG. 2 shows an example of text data. The text data in this example is in the form of a table. The text data has a record for each article.

  This example shows an article of a simple blog in a time zone in which a broadcast program of a sports tournament where a player named “Murata Kanako” participates was broadcast. The article of the first record (excluding omitted records, the same applies below) includes the character string data of “Kanako Murata, good! #Xyztv”. In this article, viewers of the relay program posted their impressions of the sports tournament. “#Xyztv” included in this article is an example of a hashtag. The contributor writes a hash tag for identifying the theme in the article to clearly indicate that the article to be posted relates to a specific theme. “#Xyztv” in this example indicates that the article is about an XYZ broadcast station program.

  The hash tag is used as a search key when browsing. For example, when a viewer searches for an article by specifying a hash tag, an article including the hash tag is displayed on the list screen. That is, by using hash tags, for example, articles related to specific events and matters are collected. Hashtags are also used for research and analysis on specific themes. The hash tag is composed of a “#” symbol and alphanumeric characters.

  The article of the second record includes character string data of “Kanako-chan is wonderful!”. In this article, viewers of the relay program posted their impressions of the sporting event. However, this article does not include hashtags.

  The article of the third record includes character string data of “Smoothly cute #xyztv”. In this article, viewers of the relay program posted their impressions of the sporting event. This article includes the hash tag “#xyztv” as in the first record. “Murakana” in the article is a nickname for the player who abbreviated the name “Kanako Murata”.

  The article of the fourth record includes character string data of “Make a decision smoothly”! In this article, viewers of the relay program posted their impressions of the sporting event. However, this article does not include hashtags. Also in this article, the nickname “Muraka” is used.

  The article of the fifth record includes the character string data of “Murata, is it really good #xyztv”. In this article, viewers of the relay program posted their impressions of the sporting event. This article includes the hash tag “#xyztv” as in the first record and the third record.

  In addition, the text data includes an article posted by the viewer of the relay program regarding the sporting event. The text data also includes articles not related to the sporting event.

  Returning to the description of FIG. The network shown in FIG. 1 is, for example, the Internet or a LAN (Local Area Network). The extraction apparatus 101 extracts a text unit (in this example, a simple blog article) related to a specific theme from the text database 105 via the network. In this example, the extraction apparatus 101 extracts articles about the sports tournament posted by the viewer of the relay program from the text data shown in FIG. Although FIG. 1 illustrates an example in which the extraction apparatus 101 and the database management system 103 are connected via a network, the database management system 103 and the extraction apparatus 101 may be an integrated apparatus.

  A user terminal 107 is connected to the extraction apparatus 101. The user terminal 107 is a terminal used by a user who uses the service provided by the extraction apparatus 101. In FIG. 1, the user terminal 107 and the extraction device 101 are connected via the same network as described above, but the user terminal 107 and the extraction device 101 are connected via a network different from the above-described network. You may do it.

  Next, the module configuration of the extraction apparatus 101 will be described. FIG. 3 shows a module configuration example of the extraction apparatus 101. The extraction apparatus 101 includes a parameter storage unit 301, a reception unit 303, a first extraction unit 305, a first extraction result storage unit 307, a sampling unit 309, a sampling result storage unit 311, a second extraction unit 313, and a second extraction result storage unit. 315 and an output unit 317.

  The parameter storage unit 301 stores parameters used in the processing of the extraction device 101. The accepting unit 303 accepts data input from the user terminal 107. The accepting unit 303 accepts, for example, the above parameters and a first extraction condition described later. The parameters include, for example, an initial value of the lowest frequency described later.

  The extraction apparatus 101 performs extraction twice, for example. The first extraction unit 305 performs earlier extraction. Specifically, the first extraction unit 305 extracts a text unit (in this example, a simple blog article) that meets the first extraction condition from the text database 105. In addition, the first extraction unit 305 transmits a first query including a hash tag to the database management system 103. The partial match for the hash tag corresponds to the first extraction condition. However, the first extraction condition may be a condition relating to text data other than the hash tag.

  The first extraction unit 305 acquires the extracted result (referred to as a first extraction result) from the database management system 103. In this example, the first extraction result is a set of text units (in this example, simple blog articles) that match the first extraction condition. The first extraction result storage unit 307 stores the first extraction result.

  FIG. 4 shows an example of the first extraction result. The first extraction result is a set of articles searched by the hash tag “#xyztv”. The first record article “Kanako Murata, good! #Xyztv” shown in FIG. 4 is obtained from the first record shown in FIG. Similarly, the article “Murky Cute #xyztv” in the second record shown in FIG. 4 is obtained from the third record shown in FIG. 2. Similarly, the third record article “Murata, Kana #xyztv”, shown in FIG. 4, was obtained from the fifth record shown in FIG.

  The articles of the second record and the fourth record shown in FIG. 2 are not extracted because they do not include the hash tag “#xyztv”.

  Returning to the description of FIG. 3, the sampling unit 309 executes a sampling process. Specifically, the sampling unit 309 requests the database management system 103 to randomly extract a text unit (in this example, a simple blog article) from the text database 105, and the extracted result (referred to as a sampling result). .) Is acquired from the database management system 103. The sampling result is a set of randomly selected text units. The sampling result storage unit 311 stores the sampling result.

  The second extraction unit 313 extracts a text unit (in this example, a simple blog article) related to the first extraction result from the text database 105. Specifically, the second extraction unit 313 generates a keyword based on the first extraction result, and transmits a second query including the keyword to the database management system 103.

  The second extraction unit 313 acquires the extracted result (referred to as “second extraction result”) from the database management system 103. In this example, the second extraction result is a set of text units (in this example, simple blog articles) including the keyword. The second extraction result storage unit 315 stores the extracted result (referred to as a second extraction result).

  The output unit 317 outputs the second extraction result. In this example, the output unit 317 transmits the second extraction result to the user terminal 107. However, the form of output by the output unit 317 is not limited to transmission. The output unit 317 may display the second extraction result. The output unit 317 may write the second extraction result in the storage medium. Alternatively, the output unit 317 may print the second extraction result.

  Parameter storage unit 301, reception unit 303, first extraction unit 305, first extraction result storage unit 307, sampling unit 309, sampling result storage unit 311, second extraction unit 313, second extraction result storage unit 315, and output unit 317 Is realized by, for example, hardware resources shown in FIG. In addition, the accepting unit 303, the first extracting unit 305, the sampling unit 309, the second extracting unit 313, and the output unit 317 can execute a program loaded in the memory 2501 (FIG. 34), with some or all of the processing of the module. It may be realized by executing sequentially by the CPU 2503 (FIG. 34). This is the end of the description of the module configuration of the extraction apparatus 101.

  Subsequently, processing of the extraction apparatus 101 will be described. FIG. 5 shows a main processing flow. The accepting unit 303 accepts parameters from the user terminal 107 (S501). The accepting unit 303 stores the accepted parameters in the parameter storage unit 301. The accepted parameters are, for example, various reference values (including a minimum frequency) described later, an initial value, an additional value, and the like. When using parameters that have already been set, the processing in S501 may be omitted.

  The accepting unit 303 accepts the first extraction condition from the user terminal 107 (S503). The first extraction condition is a condition for extracting a text unit (in this example, a simple blog article) related to a specific theme, for example. For example, the first extraction condition is specified by a hash tag as described above. The first extraction condition may be another search condition. The first extraction condition may be a character string for searching the article.

  The first extraction unit 305 executes a first extraction process (S505). The internal module configuration of the first extraction unit 305 that performs the first extraction process will be described. FIG. 6 shows an internal module configuration example of the first extraction unit 305. The first extraction unit 305 includes a first query generation unit 601, a first request unit 603, and a first acquisition unit 605. The first query generation unit 601 generates a first query based on the first extraction condition. The first query is a processing request for causing the database management system 103 to extract a text unit that satisfies the first extraction condition (in this example, a simple blog article). The first request unit 603 transmits the first query to the database management system 103. The first acquisition unit 605 acquires the first extraction result from the database management system 103.

  The first query generation unit 601, the first request unit 603, and the first acquisition unit 605 are realized by, for example, hardware resources illustrated in FIG. The first query generation unit 601, the first request unit 603, and the first acquisition unit 605 sequentially execute a program loaded in the memory 2501 (FIG. 34) on the CPU 2503 (FIG. 34) in part or all of the processing of the module. It may be realized by executing.

  FIG. 7 shows a first extraction process flow. The first query generation unit 601 of the first extraction unit 305 generates a first query based on the first extraction condition (S701).

  The first request unit 603 of the first extraction unit 305 transmits the first query to the database management system 103 (S703). The first acquisition unit 605 of the first extraction unit 305 acquires the first extraction result from the database management system 103, and writes the acquired first extraction result in the first extraction result storage unit 307 (S705). This is the end of the description of the first extraction process.

  In this way, for example, the article shown in FIG. 4 is extracted. The articles included in the first extraction result are assumed to be related to a specific theme intended by the user. However, not all articles related to the subject are collected at this stage. For example, the article of the second record and the article of the fourth record in FIG. 2 are posted with the same impression as the article of the first record, the article of the third record, or the article of the fifth record, but are extracted. Absent.

  Returning to the description of FIG. 5, the sampling unit 309 performs a sampling process (S507). Specifically, the sampling unit 309 requests the database management system 103 to randomly extract a text unit (in this example, a simple blog article) from the text database 105, and the extracted result (sampling result) Is acquired from the database management system 103. The sampling result storage unit 311 stores the sampling result obtained by the sampling unit 309. The sampling result includes a plurality of text units.

  The second extraction unit 313 executes a second extraction process (S509). The internal module configuration of the second extraction unit 313 that performs the second extraction process will be described. FIG. 8 shows an internal module configuration example of the second extraction unit 313. The second extraction unit 313 includes a keyword generation unit 801, a keyword storage unit 803, a second query generation unit 805, a second request unit 807, and a second acquisition unit 809. The keyword generation unit 801 generates a keyword based on the first extraction result. The keyword is a condition for extracting a text unit (in this example, a simple blog article) in the second extraction process. The keyword is selected from partial character strings included in text units included in the first extraction result. For example, a partial character string “Muraka” included in the article of the second record shown in FIG. 2 becomes a keyword.

  The keyword storage unit 803 stores the generated keyword. The second query generation unit 805 generates a second query based on the keyword. The second request unit 807 transmits the second query to the database management system 103. The second acquisition unit 809 acquires the second extraction result from the database management system 103.

  The keyword generation unit 801, the keyword storage unit 803, the second query generation unit 805, the second request unit 807, and the second acquisition unit 809 are realized by, for example, hardware resources illustrated in FIG. The second extraction unit 313 includes a keyword generation unit 801, a second query generation unit 805, a second request unit 807, and a second acquisition unit 809 that store part or all of the processing of the module in the memory 2501 (FIG. 34). You may make it implement | achieve by executing the loaded program by CPU2503 (FIG. 34) sequentially.

  FIG. 9 shows a second extraction process flow. The keyword generation unit 801 of the second extraction unit 313 executes keyword generation processing (S901). The keyword generation unit 801 detects partial character strings included in a text unit (in this example, a simple blog article) included in the first extraction result, and relates to the first extraction result from the partial character strings. Select a strong substring. The selected partial character string becomes a keyword.

  For example, there is a method of detecting a word from a text unit (in this example, a simple blog article) by morphological analysis. A keyword may be selected from words detected by morphological analysis. However, words detected by morphological analysis are limited to those registered in advance as candidates. Therefore, words that are not yet registered as candidates are not detected by morphological analysis. For example, new words, abbreviations, technical terms or dialect words included in text units may not be detected.

  In the present embodiment, a partial character string is detected by frequent pattern mining so that new words, abbreviations, technical terms, dialect words, etc. included in the text unit are also adopted as keywords.

  The partial character string is a series of two or more consecutive characters. For example, the substrings in the sentence “Mura Kawaii” are “Mura”, “Muraka”, “Muraka”, (omitted), “Mura Kawaii”, “Raka”, “Raka”, “ "Kanaka", (Omitted), "Easy Cute", (Omitted), "Wai", "Wai", "Good". On the other hand, in the morphological analysis for the sentence “Smoothly cute”, for example, the word “cute” is detected. Thus, the theoretical number of partial character strings is much larger than the number of words detected by morphological analysis.

  However, if there are no theoretical partial character strings and keywords are candidates, the process of selecting the keywords becomes complicated. In the present embodiment, keyword candidates are limited to partial character strings having a predetermined frequency or higher. In this way, since the number of keyword candidates is limited, the burden of processing for selecting a keyword is reduced. In addition, the frequency here is the number of the text unit containing the said partial character string among the text units (this example article of a simple blog) contained in a 1st extraction result. In other words, the frequency is the number of times the partial character string appears in the text unit included in the first extraction result. However, when a partial character string is included in a plurality of locations in a text unit, it is counted once. The above-mentioned predetermined frequency is a kind of reference value, and is hereinafter referred to as a minimum frequency.

  Generally, in frequent pattern mining, a pattern that appears frequently from a set of element sequences is detected. In the present embodiment, a text unit corresponds to an element string, a character corresponds to an element, and a partial character string corresponds to a pattern. That is, partial character strings that frequently appear from a set of text units are detected by frequent pattern mining. If a partial character string that appears frequently by frequent pattern mining is detected, the processing load is reduced compared to a method of calculating the frequency for each theoretical partial character string.

  In the present embodiment, Modified Prefixspan which is a kind of frequent pattern mining is used. Modified Prefixspan detects an element sequence on the condition of the distance between elements. In the present embodiment, a character string is detected under the condition that the distance between characters is zero.

  Modified Prefixspan is detailed in H. Kitakami, T. Kanbara, Y. Mori, S. Kuroki, Y. Yamazaki: "Modified PrefixSpan Method for Motif Discovery in Sequence Databases". PRICAI 2002: pp.482-491, 2002 Has been.

  Also, Modified Prefixspan detects an element sequence related to a frequency that is equal to or higher than the lowest frequency. However, if the minimum frequency is too high, a partial character string to be a keyword is not detected. In that case, there is a possibility of failing to extract a text unit to be extracted (in this example, a simple blog article).

  On the other hand, if the lowest frequency is too low, an extra partial character string is detected. As a result, the processing load increases and the processing time becomes longer.

  In the present embodiment, the minimum frequency is determined by paying attention to the frequency of partial character strings that become keywords.

  FIG. 10 shows the characteristics of the number of partial character strings with respect to the frequency. The horizontal axis indicates the frequency of the partial character string. The vertical axis indicates the number of partial character strings. The characteristic of the number of partial character strings with respect to the frequency is assumed to approximate an empirical rule for the appearance frequency of words in English (Zip's law). Assuming that, the number of partial character strings with low frequency as shown in the figure is extremely large. On the other hand, the number of partial character strings having a high frequency is extremely small. The partial character string in this figure is a theoretical partial character string, and includes both a partial character string that becomes a keyword and a partial character string that does not become a keyword.

  FIG. 11 shows the characteristics of the number of keywords with respect to the frequency. The horizontal axis indicates the keyword frequency. The vertical axis indicates the number of keywords. This figure shows the distribution of the partial character strings in FIG. 10 that become keywords. In the present embodiment, it is assumed that the keyword distribution with respect to the frequency indicates a mountain shape. The example of FIG. 11 approximates a normal distribution.

  In this figure, for example, if Modified Prefixspan is performed by setting a value smaller than the frequency with the largest number of keywords (corresponding to the average frequency) to the lowest frequency, more than half of the keywords are detected. In the present embodiment, the minimum frequency is set based on the number of keywords with respect to the frequency.

  The keyword generation unit 801 performs processing so as to search for a low frequency when appropriate while shifting the minimum frequency from the larger one to the smaller one. Specifically, the keyword generation unit 801 performs Modified Prefixspan while reducing the minimum frequency, and determines an appropriate minimum frequency based on a change in the number of keywords associated therewith. Then, the keyword detected when it is determined that the frequency is an appropriate minimum frequency becomes a generation result by the keyword generation unit 801.

  Note that the keyword generation unit 801 determines whether or not the partial character string is suitable for the keyword based on a score indicating the strength of association with the first extraction result.

  An internal module configuration of the keyword generation unit 801 that performs keyword generation processing will be described. FIG. 12 shows an internal module configuration example of the keyword generation unit 801. The keyword generation unit 801 includes a partial character string detection unit 1201, a detection result storage unit 1203, a keyword selection unit 1205, a selection result storage unit 1207, an end determination unit 1209, a frequency distribution storage unit 1211, a repetition unit 1213, and a keyword identification unit 1215. Have.

  The partial character string detection unit 1201 detects a partial character string from the text unit (in this example, a simple blog article) included in the first extraction result storage unit 307 by using the above-described Modified Prefixspan. The detection result storage unit 1203 stores the detection result by the partial character string detection unit 1201. The detection result includes one or more detected partial character strings. The keyword selection unit 1205 selects a keyword from the partial character string included in the detection result. The selection result storage unit 1207 stores the selection result by the keyword selection unit 1205. The selection result includes a keyword table at each time. The selected keyword is registered in the keyword table. The end determination unit 1209 generates a keyword frequency distribution table and determines the end of the second extraction process. The frequency distribution storage unit 1211 stores a frequency distribution table of keywords. The repetition unit 1213 updates the minimum frequency used for Modified Prefixspan, and repeats the processing by the partial character string detection unit 1201, the processing by the keyword selection unit 1205, and the processing by the end determination unit 1209. The keyword specifying unit 1215 specifies a keyword that is a result of the second extraction process.

  The partial character string detection unit 1201, the detection result storage unit 1203, the keyword selection unit 1205, the selection result storage unit 1207, the end determination unit 1209, the frequency distribution storage unit 1211, the repetition unit 1213, and the keyword specification unit 1215 are illustrated in FIG. 34, for example. Realized by hardware resources. The partial character string detection unit 1201, the keyword selection unit 1205, the end determination unit 1209, the repetition unit 1213, and the keyword identification unit 1215 execute a program loaded in the memory 2501 (FIG. 34), with some or all of the processing of the module. It may be realized by executing sequentially by the CPU 2503 (FIG. 34).

  FIG. 13 shows a keyword generation processing flow. The partial character string detection unit 1201 of the keyword generation unit 801 performs a partial character string detection process (S1301).

  FIG. 14 shows a partial character string detection processing flow. The partial character string detection unit 1201 executes frequent pattern mining processing (S1401).

  A frequent pattern mining process according to the present embodiment will be described. As described above, characters are regarded as elements, text units (in this example, simple blog articles) included in the first extraction result are regarded as element strings, and partial character strings are regarded as patterns to perform frequent pattern mining. In the present embodiment, Modified Prefixspan, which is a type of frequent pattern mining, is executed. At this time, the partial character string detection unit 1201 applies the lowest frequency stored in the parameter storage unit 301 to Modified Prefixspan. Also, the partial character string detection unit 1201 sets the minimum number of elements applied to Modified Prefixspan to 2. Furthermore, the partial character string detection unit 1201 sets the inter-element distance applied to Modified Prefixspan to 0.

  The partial character string detection unit 1201 of the keyword generation unit 801 writes the detected result (detection result) in the detection result storage unit 1203 (S1403). The detection result is a set of partial character strings detected by Modified Prefixspan.

  FIG. 15 shows an example of the detection result. The detection result has a record for each detected partial character string. The record has a field for storing the partial character string and a field for storing the frequency.

  This example shows an example of a detection result by frequent pattern mining performed first. The minimum frequency at this time is set to 60. Therefore, a partial character string having a frequency of 60 or more is detected.

  The first record detects the partial character string “good” and indicates that the frequency of the partial character string is “70”. The second record detects the partial character string “Kana” and indicates that the frequency of the partial character string is “90”. The partial character string “Good” and the partial character string “Kana” mean that they frequently appear in articles included in the first extraction result.

  Returning to the description of FIG. 13, the keyword selection unit 1205 of the keyword generation unit 801 executes keyword selection processing (S1303). FIG. 16 shows a keyword selection processing flow. The keyword selection unit 1205 identifies one unprocessed partial character string that is not the target of the score calculation process among the partial character strings included in the detection result stored in the detection result storage unit 1203 (S1601). . The keyword selection unit 1205 executes score calculation processing for the identified partial character string (S1603). In the score calculation process, a score for the specified partial character string is calculated.

  The score in this example is a lift value obtained by dividing the first appearance probability by the second appearance probability. The first appearance probability is a probability that the partial character string appears in the first extraction result. The second appearance probability is a probability that the partial character string appears in the sampling result. When the score is large, it means that the partial character string is strongly related to the first extraction result. When the score is small, it means that the partial character string is weakly related to the first extraction result.

  FIG. 17 shows a score calculation processing flow. The keyword selection unit 1205 executes a first appearance probability calculation process (S1701). In the first appearance probability calculation process, the first appearance probability described above is calculated.

  FIG. 18 shows a first appearance probability calculation process flow. The keyword selection unit 1205 identifies the number of text units (in this example, simple blog articles) included in the first extraction result (S1801). Specifically, the keyword selection unit 1205 counts text units included in the first extraction result stored in the first extraction result storage unit 307. Hereinafter, this number of text units is referred to as a first text unit number.

  Next, the keyword selection unit 1205 counts the number of text units (in this example, simple blog articles) including the partial character string in the first extraction result (S1803). Hereinafter, this number of text units is referred to as a first appearance frequency.

  Then, the keyword selection unit 1205 calculates the probability (first appearance probability) that the partial character string appears in the first extraction result (S1805). Specifically, in the keyword selection unit 1205, the quotient obtained by dividing the first appearance frequency by the first text unit number is the first appearance probability.

  Returning to the description of FIG. 17, the keyword selection unit 1205 executes a second appearance probability calculation process (S <b> 1703). In the second appearance probability calculation process, the above-described second appearance probability is calculated.

  FIG. 19 shows a second appearance probability calculation process flow. The keyword selection unit 1205 identifies the number of text units (in this example, simple blog articles) included in the sampling result (S1901). Specifically, the keyword selection unit 1205 counts text units included in the sampling results stored in the sampling result storage unit 311. Hereinafter, this number of text units is referred to as a second text unit number.

  Next, the keyword selection unit 1205 counts the number of text units (in this example, simple blog articles) including the partial character string in the sampling result (S1903). Hereinafter, this number of text units is referred to as a second appearance frequency.

  Then, the keyword selection unit 1205 calculates the probability (second appearance probability) that the partial character string appears in the sampling result (S1905). Specifically, in the keyword selection unit 1205, the quotient obtained by dividing the second appearance frequency by the second text unit number is the second appearance probability.

  Returning to the description of FIG. 17, the keyword selection unit 1205 calculates a ratio (lift value) of appearance probabilities (S1705). Specifically, the ratio obtained by dividing the first appearance probability by the second appearance probability in the keyword selection unit 1205 is the appearance probability ratio (lift value).

  An example of the score for the partial character string included in the detection result shown in FIG. 15 is shown in FIG. The first record indicates that the score of the partial character string “good” is “4”. The second record indicates that the score of the partial character string “Kana” is “3”. Both the partial character string “good” and the partial character string “kana” have small scores. That is, these partial character strings are weakly related to the first extraction result. Since these character strings are character strings that frequently appear in a general sentence, it cannot be said that the relation with the first extraction result is particularly strong.

  Returning to the description of FIG. 16, the keyword selection unit 1205 determines whether or not the calculated score exceeds the reference value (S1605). The reference value is a parameter for determining whether or not the relationship with the first extraction result is strong. The reference value is stored in the parameter storage unit 301. The reference value is one of the parameters accepted by the parameter storage unit 301. The reference value is sometimes referred to as a minimum score.

  If it is determined that the calculated score exceeds the reference value, the keyword selection unit 1205 registers the partial character string in the current keyword table (S1607). The keyword table for each time is generated in the selection result storage unit 1207. Specifically, the keyword selection unit 1205 sets the partial character string in the current keyword table stored in the selection result storage unit 1207. At this time, the keyword selection unit 1205 may add a score corresponding to the partial character string in the current keyword table.

  If it is determined that the calculated score does not exceed the reference value, the keyword selection unit 1205 does not register the partial character string in the current keyword table. For example, the partial character strings included in the detection results shown in FIGS. 15 and 20 are not registered in the current keyword table because their scores do not exceed the reference value.

  The keyword selection unit 1205 determines whether there is an unprocessed partial character string that is not a target of the score calculation process among the partial character strings included in the detection result stored in the detection result storage unit 1203. (S1609). If it is determined that there is an unprocessed partial character string, the process returns to S1601, and the keyword selection unit 1205 repeats the above-described process. If it is determined that there is no unprocessed partial character string, the keyword selection unit 1205 ends the keyword selection process. Then, the process proceeds to S1305 in FIG.

  Returning to the description of FIG. 13, the end determination unit 1209 of the keyword generation unit 801 executes an end determination process (S1305).

  FIG. 21 shows an end determination process flow. The end determination unit 1209 determines whether or not the current minimum frequency is equal to or less than a reference value (S2101). For example, when the frequency is 2 or 3 as shown in FIG. 10, the number of partial character strings is extremely large, and the processing load such as the second appearance probability calculation process described above becomes enormous. Therefore, in the present embodiment, a lower limit is set for the lowest frequency. The reference value means the lower limit of the lowest frequency.

  If it is determined that the current minimum frequency is equal to or less than the reference value, the end determination unit 1209 sets the determination result to “end” (S2103). In this way, when it is determined that the current lowest frequency is below the reference value, the subsequent search is not performed.

  If it is determined that the current minimum frequency is not less than or equal to the reference value, the end determination unit 1209 identifies a new range (S2105). The end determination process is performed based on the keyword distribution with respect to the frequency as described above. Therefore, the end determination unit 1209 generates a keyword frequency distribution table. The new range is a frequency range and corresponds to a new class in the frequency distribution table.

  FIG. 22 shows an example of a keyword frequency distribution table. The frequency distribution table has a record for each class. The class is specified by the frequency range. The record has a field for storing a frequency range and a field for storing the number of keywords. A new record is added each time the end determination process is performed by repetition.

  The 1st record has shown that the number of keywords in the range whose frequency is 60 or more is 0. The number of keywords in the first record indicates the number of keywords generated for the first time.

  The second record indicates that the number of keywords is 3 in a frequency range of 57 or more and less than 60. The number of keywords in the second record indicates the difference between the number of keywords generated for the second time and the number of keywords generated for the first time, that is, the number of keywords newly generated for the second time.

  The third record indicates that the number of keywords in the range where the frequency is 54 or more and less than 57 is 6. The number of keywords in the third record indicates the difference between the number of keywords generated for the third time and the number of keywords generated for the second time, that is, the number of keywords newly generated for the third time.

  The fourth record indicates that the number of keywords in the range where the frequency is 51 or more and less than 54 is 12. The number of keywords in the fourth record indicates the difference between the number of keywords generated for the fourth time and the number of keywords generated for the third time, that is, the number of keywords newly generated for the fourth time.

  The fifth record indicates that the number of keywords is 18 in a frequency range of 48 or more and less than 51. The number of keywords in the fifth record indicates the difference between the number of keywords generated at the fifth time and the number of keywords generated at the fourth time, that is, the number of keywords newly generated at the fifth time.

  In this example, the size of the range is made uniform, but the size of the range may be different every time.

  FIG. 23 shows an example of a keyword frequency distribution diagram. This frequency distribution diagram is a graph (histogram) of the frequency distribution table shown in FIG.

  Returning to the description of FIG. 21, the end determination unit 1209 calculates the number of keywords in the new range (S2107). Specifically, the end determination unit 1209 counts the keywords included in the current keyword table and subtracts the previous number of keywords from the current number of keywords to obtain the number of keywords in the new range.

  The end determination unit 1209 adds a new record to the frequency distribution table (S2109). A new range and the number of keywords are set in the new record.

  The end determination unit 1209 determines a change tendency of the number of keywords (S2111). The change tendency is, for example, either an increasing tendency or a decreasing tendency. For example, when the number of keywords in the current range is greater than the number of keywords in the previous range, the end determination unit 1209 determines that the trend is increasing, and the number of keywords in the current range is greater than the number of keywords in the previous range. If it is decreasing, it is judged as a decreasing trend. Alternatively, the end determination unit 1209 may compare the number of keywords in the current range with the number of keywords in the previous time range. The end determination unit 1209 may compare the number of keywords in the current range with the number of keywords in a plurality of ranges, respectively, and determine a change tendency based on the comparison result of each time.

  The end determination unit 1209 determines whether or not the change tendency of the number of keywords is an increasing tendency (S2113). When it is determined that the change tendency of the number of keywords is an increasing tendency, the end determination unit 1209 sets “not end” as the determination result (S2119). For example, as shown in FIG. 23, when the number of keywords in the fourth range tends to increase, it is presumed that most of the partial character strings to be used as keywords have not yet been detected. Let it continue.

  When it is determined that the change tendency of the number of keywords is not an increasing tendency, the end determination unit 1209 determines whether or not the number of keywords in the current range is equal to or less than the first reference value (S2115). The first reference value is a reference for determining that the number of keywords is converging with a decrease in frequency. The first reference value is stored in the parameter storage unit 301. The first reference value may be received by the receiving unit 303. The first reference value may be set according to the number of first text units (in this example, the number of articles included in the first extraction result).

  FIG. 24 shows an example of a keyword frequency distribution diagram. The broken line in the figure indicates the first reference value. Since the number of keywords in the range where the minimum frequency is set to 15 and the partial character string is detected exceeds the first reference value, it is assumed that the number of keywords has not yet reached the stage of convergence.

  Next, since the number of keywords in the range where the minimum frequency is set to 12 and the partial character string is detected is below the first reference value, it is assumed that the number of keywords is converging.

  Furthermore, the number of keywords in the range in which the partial character string is detected with the minimum frequency set to 9 is again below the first reference value. At this time, it is below the first reference value twice in succession. In this embodiment, the keyword generation is terminated when the number of keywords in each range is continuously lower than the first reference value. The continuous number may be 3 or more. In addition, the keyword generation may be terminated when the value falls below the first reference value even once.

  Returning to the description of FIG. 21, if it is determined in S2115 that the number of keywords in the current range is not less than or equal to the first reference value, the end determination unit 1209 sets the determination result to “not end” (S2119). ).

  When it is determined that the number of keywords is equal to or less than the first reference, the end determination unit 1209 determines whether or not the number of consecutive determination results has reached a predetermined number (2 in this example). (S2117). When it is determined that the number of consecutive determination results has reached a predetermined number, the end determination unit 1209 sets the determination result to “end” (S2103).

  If it is determined that the number of consecutive determination results has not reached the predetermined number, the end determination unit 1209 sets the determination result to “not end” (S2119). When the end determination process ends, the process returns to S1307 in FIG.

  Returning to the description of FIG. 13, the repetitive unit 1213 determines whether or not the determination result is set to “end” (S1307). When it is determined that the determination result is not set to “end”, that is, when the determination result is set to “do not end”, the iterative unit 1213 of the keyword generation unit 801 executes the minimum frequency update process. (S1309). The lowest frequency in the next round is set by the lowest frequency update process.

  In this embodiment, the minimum frequency update process (A) is performed. FIG. 25 shows the flow of the lowest frequency update process (A). The iterative unit 1213 subtracts a predetermined number (3 in this example) from the lowest frequency (S2501). The predetermined number corresponds to the size of a range that specifies the class of the frequency distribution. Therefore, in this embodiment, the size of the range for specifying the class of the frequency distribution is uniform.

  When the minimum frequency update process is finished, the repetitive unit 1213 returns the process to S1301, and repeats the partial character string detection process of S1301, the keyword selection process of S1303, and the end determination process of S1305.

  In this way, the partial character string detection process (S1301), the keyword selection process (S1303), and the end determination process (S1305) are repeated until it is determined that the keyword generation is to be ended while decreasing the minimum frequency.

  An example of the detection result at the time when the minimum frequency is set to 12 is shown in FIG. At this time, a partial character string having a frequency of 12 or more is detected.

  The first record detects the partial character string “Murata” and indicates that the frequency of the partial character string is “32”. The second record detects the partial character string “Kanako” and indicates that the frequency of the partial character string is “29”. The third record detects the partial character string “Kanako-chan” and indicates that the frequency of the partial character string is “13”. The fourth record detects the partial character string “good” and indicates that the frequency of the partial character string is “70”. The fifth record detects the partial character string “Kana” and indicates that the frequency of the partial character string is “90”. The sixth record detects the partial character string “cute” and indicates that the frequency of the partial character string is “38”. The frequency of the substring “Kanako-chan” detected for the first time this time is the lowest among these substrings. That is, the probability that the partial character string “Kanako-chan” appears in the article included in the first extraction result is relatively low.

  An example of the score for the partial character string included in the detection result shown in FIG. 26 is shown in FIG. The first record indicates that the score of the partial character string “Murata” is “10”. The second record indicates that the score of the partial character string “Kanako” is “100”. The third record indicates that the score of the partial character string “Kanako-chan” is “150”. The fourth record indicates that the score of the partial character string “good” is “4”. The fifth record indicates that the score of the partial character string “Kana” is “3”. The sixth record indicates that the score of the partial character string “cute” is “5”. The score of the substring “Kanako-chan” detected for the first time this time is the highest among these substrings. That is, the partial character string “Kanako-chan” has the strongest relationship with the article included in the first extraction result.

  FIG. 28 shows an example of a keyword table based on the score shown in FIG. In this example, the reference value of the score is 50. Therefore, the partial character string “Kanako” and the partial character string “Kanako-chan” with a score exceeding 50 are selected and set in the keyword table of this time. The partial character string “Murata”, the partial character string “good”, the partial character string “Kana”, and the partial character string “cute” whose scores do not exceed 50 are not selected.

  FIG. 29 shows an example of the detection result in the next round. At this time, the minimum frequency is set to 12. Therefore, a partial character string having a frequency of 12 or more is detected. The first to fifth records are the same as in the case of FIG. The sixth record detects the partial character string “Muraka” and indicates that the frequency of the partial character string is “10”. The seventh record is the same as the sixth record in FIG. The frequency of the partial character string “Muraka” detected for the first time this time is even lower than the frequency of the partial character string “Kanako-chan” detected for the first time last time. The probability that the partial character string “Muraka” appears in the article included in the first extraction result is even lower than the probability that the partial character string “Kanako-chan” appears in the article included in the first extraction result.

  An example of the score for the partial character string included in the detection result shown in FIG. 29 is shown in FIG. The first to fifth records are the same as in the case of FIG. The sixth record indicates that the score of the partial character string “Muraka” is “60”. The seventh record is the same as the sixth record in FIG. The score of the partial character string “Muraka” detected for the first time this time is relatively large. That is, the partial character string “Muraka” is relatively strongly related to the article included in the first extraction result.

  FIG. 31 shows an example of a keyword table based on the score shown in FIG. In this example, the reference value of the score is 50 as described above. Therefore, in addition to the partial character string “Kanako” and the partial character string “Kanako-chan” whose score exceeds 50, the partial character string “Muraka” is also selected and set in the keyword table of this time. The partial character string “Murata”, the partial character string “good”, the partial character string “Kana”, and the partial character string “cute” whose scores do not exceed 50 are not selected as in the previous case.

  Returning to the description of FIG. 13, if it is determined in S1307 that the determination result is set to “end”, the keyword specifying unit 1215 of the keyword generating unit 801 specifies the keyword (S1311). Specifically, the keyword specifying unit 1215 specifies a keyword set in the keyword table in the last round. When the keyword generation process is completed, the process returns to S903 shown in FIG.

  Returning to the description of FIG. 9, the second query generation unit 805 of the second extraction unit 313 generates a second query including the keyword generated by the keyword generation processing (S903). For example, the second query generation unit 805 generates a query for searching with the OR condition without leaving the keywords generated in the keyword generation process. Or you may make it the 2nd query production | generation part 805 produce | generate the query for searching by AND conditions not leaving the keyword produced | generated by the keyword production | generation process. Or the 2nd query production | generation part 805 may produce | generate the query for searching a part of keyword produced | generated by the keyword production | generation process by OR condition. Alternatively, the second query generation unit 805 may generate a query for searching a part of the keywords generated by the keyword generation process using an AND condition.

  The second request unit 807 of the second extraction unit 313 transmits the second query to the database management system 103 (S905). The second acquisition unit 809 of the second extraction unit 313 acquires the second extraction result from the database management system 103, and writes the acquired second extraction result in the second extraction result storage unit 315 (S907). This is the end of the description of the second extraction process. When the second extraction process is completed, the process returns to S511 in FIG.

  Returning to the description of FIG. 5, the output unit 317 outputs the second extraction result (S511). For example, the output unit 317 transmits the second extraction result to the user terminal 107.

  According to the present embodiment, it is possible to estimate that a keyword latent in a set of text units (for example, a document such as a simple blog article) has been extracted to some extent, and to end the processing. . Therefore, useless processing can be omitted and the effectiveness of the keyword can be secured. An appropriate minimum frequency is specified at the end of processing.

  In addition, since a series of processing is generally omitted in a range where the number of partial character strings is large (range where the frequency is low), the processing load is reduced. In particular, the processing burden related to score calculation is reduced.

  Furthermore, it can be estimated that many of the latent keywords are extracted according to the assumed change tendency of the number of keywords.

[Embodiment 2]
In the above-described embodiment, an example in which the size of the frequency range specifying the frequency distribution class is made equal is shown. However, in this embodiment, the frequency specifying the frequency distribution class as the frequency decreases. An example of narrowing the range will be described.

  In the present embodiment, a divisor parameter for obtaining the minimum frequency is provided. The initial value of the divisor parameter is stored in the parameter storage unit 301. The initial value of the divisor parameter may be received by the receiving unit 303.

  In addition, a predetermined additional value to be added to the divisor parameter in the lowest frequency update process is provided. The predetermined additional value is stored in the parameter storage unit 301. The predetermined additional value may be received by the receiving unit 303.

  In the present embodiment, in S1309 shown in FIG. 13, the repetitive unit 1213 of the keyword generating unit 801 executes the minimum frequency update process (B). FIG. 32 shows the flow of the lowest frequency update process (B). The iterative unit 1213 adds a predetermined additional value to the divisor parameter (S3201).

  The repeater 1213 divides the first text unit number by the divisor (S3203) and sets the quotient to the lowest frequency (S3205).

  In this way, the change amount of the lowest frequency becomes large at the stage where the frequency is high. Therefore, the number of iterations can be reduced.

[Embodiment 3]
In the second embodiment, the example in which the predetermined additional value is constant has been described. However, in the present embodiment, the first additional value until the number of keywords starts to converge and the number of keywords starts to converge. An example of switching the second additional value will be described.

  In the present embodiment, two types of additional values to be added to the divisor parameter in the minimum frequency update process are provided. The first additional value is larger than the second additional value. The first additional value and the second additional value are stored in the parameter storage unit 301. The first additional value and the second additional value may be received by the receiving unit 303.

  In addition, a second reference value for determining whether or not the number of keywords is approaching a stage is provided. The second reference value is a value larger than the first reference value used in S2115. The second reference value is stored in the parameter storage unit 301. The second reference value may be received by the receiving unit 303. The second reference value may be set according to the first text unit number (in this example, the number of articles included in the first extraction result).

  In the present embodiment, in S1309 shown in FIG. 13, the repetitive unit 1213 of the keyword generating unit 801 executes the minimum frequency update process (C). FIG. 33 shows a minimum frequency update process (C) flow. The iterative unit 1213 determines whether or not the change tendency determined in S2111 is an increasing tendency (S3301). If it is determined that the change tendency determined in S2111 is an increasing tendency, the iterating unit 1213 adds the first additional value to the divisor parameter described above (S3303).

  For example, in the distribution shown in FIG. 11, in the region where the frequency is 30 to 60, the change tendency tends to increase, so the first additional value is added to the divisor parameter.

  On the other hand, if it is determined that the change trend determined in S2111 is not an increasing trend, the repetitive unit 1213 determines whether the number of keywords is equal to or greater than the second reference value. (S3305). If it is determined that the number of keywords is greater than or equal to the second reference value, the iterative unit 1213 adds the first additional value to the divisor parameter (S3303).

  For example, in the distribution shown in FIG. 11, if it is assumed that the second reference value corresponds to about 10 keywords, the number of keywords is greater than or equal to the second reference value in the frequency range of 20 to 30. An additional value of 1 is added.

  On the other hand, when it is determined that the number of keywords is less than the second reference value, the repetitive unit 1213 adds the second additional value to the divisor parameter (S3307).

  For example, in the distribution shown in FIG. 11, in the left region where the frequency is less than 20, the number of keywords is less than the second reference value, so the second additional value is added to the divisor parameter.

  The repeater 1213 divides the first text unit number by the divisor (S3309), and the repeater 1213 sets the quotient to the lowest frequency (S3311).

  In this way, the amount of change with the lowest frequency is large until the number of keywords converges. Therefore, the number of iterations can be reduced.

  Although the embodiment of the present invention has been described above, the present invention is not limited to this. For example, the functional block configuration described above may not match the actual program module configuration.

  Further, the configuration of each storage area described above is an example, and the above configuration is not necessarily required. Further, in the processing flow, the processing order can be changed if the processing result does not change. Further, it may be executed in parallel.

  The extraction device 101 described above is a computer device, and as shown in FIG. 34, a memory 2501, a CPU (Central Processing Unit) 2503, a hard disk drive (HDD: Hard Disk Drive) 2505, and a display device 2509. A display control unit 2507 connected to the computer, a drive device 2513 for a removable disk 2511, an input device 2515, and a communication control unit 2517 for connecting to a network are connected by a bus 2519. An operating system (OS) and an application program for executing the processing in this embodiment are stored in the HDD 2505, and are read from the HDD 2505 to the memory 2501 when executed by the CPU 2503. The CPU 2503 controls the display control unit 2507, the communication control unit 2517, and the drive device 2513 according to the processing content of the application program, and performs a predetermined operation. Further, data in the middle of processing is mainly stored in the memory 2501, but may be stored in the HDD 2505. In the embodiment of the present invention, an application program for performing the above-described processing is stored in a computer-readable removable disk 2511 and distributed, and installed in the HDD 2505 from the drive device 2513. In some cases, the HDD 2505 may be installed via a network such as the Internet and the communication control unit 2517. Such a computer apparatus realizes various functions as described above by organically cooperating hardware such as the CPU 2503 and the memory 2501 described above and programs such as the OS and application programs. .

  The embodiment of the present invention described above is summarized as follows.

  The keyword generation method according to the present embodiment includes a detection process for detecting one or a plurality of partial character strings whose frequency of occurrence in a document set is equal to or higher than a reference value, and the above set for each of the detected partial character strings. Calculates a score indicating the strength of the relationship, selects a keyword from the partial character string based on the score, calculates the number of newly selected keywords, and detects based on the number of keywords The determination process for determining whether to end the process and the selection process, and when it is determined not to end the detection process and the selection process, the reference value is updated, and the detection process, the selection process, and the determination process are repeated. Repetitive processing.

  In this way, it is possible to finish the processing by guessing that the keywords that are latent in the document set have been extracted to some extent. Therefore, useless processing can be omitted and the effectiveness of the keyword can be secured.

  Further, in the above iterative process, the reference value may be updated to a smaller value sequentially at each iteration.

  In this way, a series of processing in a range where the number of partial character strings is generally increased (range where the frequency is low) is omitted, so that the processing load is reduced. In particular, the processing burden related to score calculation is reduced.

  In the determination process, when the number of keywords shows a tendency to decrease and it is determined that the number of keywords has fallen below a threshold value, the detection process and the selection process may be determined to end.

  In this way, it is possible to estimate that many of the latent keywords have been extracted according to the assumed change tendency of the number of keywords.

  Further, in the above iterative process, the update amount of the reference value may be made smaller for each iteration.

  In this way, the number of iterations can be reduced while ensuring the effectiveness of the keyword.

  Further, in the above iterative process, it is determined whether the number of keywords is approaching the convergence level, and when it is determined that the keyword is approaching, the reference value update amount may be reduced. .

  In this way, the number of iterations can be reduced while ensuring the effectiveness of the keyword.

  A program for causing a computer to perform the processing according to the above method can be created. The program can be a computer-readable storage medium such as a flexible disk, a CD-ROM, a magneto-optical disk, a semiconductor memory, a hard disk, or the like. It may be stored in a storage device. Note that intermediate processing results are generally temporarily stored in a storage device such as a main memory.

  The following supplementary notes are further disclosed with respect to the embodiments including the above examples.

(Appendix 1)
A detection process for detecting one or a plurality of partial character strings whose frequency of occurrence in a set of documents is equal to or higher than a reference value;
Calculating a score indicating the strength of association with the set for each of the detected partial character strings, and a selection process for selecting a keyword from the partial character strings based on the score;
A determination process for calculating the number of newly selected keywords and determining whether to end the detection process and the selection process based on the number of keywords;
When it is determined not to end the detection process and the selection process, the reference value is updated, and the detection process, the selection process, and an iterative process that repeats the determination process are performed, and are executed by a computer Keyword generation method.

(Appendix 2)
The keyword generation method according to claim 1, wherein, in the iterative process, the reference value is updated to a smaller value sequentially at each iteration.

(Appendix 3)
The keyword generation method according to claim 2, wherein, in the determination process, the detection process and the selection process are determined to end when it is determined that the number of keywords is decreasing and the number of keywords is less than a threshold value. .

(Appendix 4)
The keyword generation method according to appendix 2 or 3, wherein, in the iterative process, the update amount of the reference value is made smaller at each iteration.

(Appendix 5)
In the iterative process, it is determined whether or not the number of keywords is approaching a stage, and when it is determined that the keyword is approaching, the update amount of the reference value is decreased. A keyword generation method according to claim 1.

(Appendix 6)
A detection process for detecting one or a plurality of partial character strings whose frequency of occurrence in a set of documents is equal to or higher than a reference value;
Calculating a score indicating the strength of association with the set for each of the detected partial character strings, and a selection process for selecting a keyword from the partial character strings based on the score;
A determination process for calculating the number of newly selected keywords and determining whether to end the detection process and the selection process based on the number of keywords;
A program for causing a computer to execute the iterative process of updating the reference value and repeating the detection process, the selection process, and the determination process when it is determined that the detection process and the selection process are not terminated .

(Appendix 7)
A detection unit that detects one or a plurality of partial character strings whose frequency of occurrence in a set of documents is equal to or higher than a reference value;
For each of the detected partial character strings, a score indicating the strength of association with the set is calculated, and a selection unit that selects a keyword from the partial character strings based on the score;
A determination unit that calculates the number of newly selected keywords and determines whether to end the detection process and the selection process based on the number of keywords;
When it is determined that the detection process and the selection process are not terminated, the reference value is updated, and a repetition unit that repeats the process by the detection unit, the process by the selection unit, and the process by the determination unit Information processing device.

DESCRIPTION OF SYMBOLS 101 Extraction apparatus 103 Database management system 105 Text database 107 User terminal 301 Parameter storage part 303 Reception part 305 1st extraction part 307 1st extraction result storage part 309 Sampling part 311 Sampling result storage part 313 2nd extraction part 315 2nd extraction result Storage unit 317 output unit 601 first query generation unit 603 first request unit 605 first acquisition unit 801 keyword generation unit 803 keyword storage unit 805 second query generation unit 807 second request unit 809 second acquisition unit 1201 partial character string detection Unit 1203 detection result storage unit 1205 keyword selection unit 1207 selection result storage unit 1209 end determination unit 1211 frequency distribution storage unit 1213 iteration unit 1215 keyword identification unit

Claims (5)

A detection process for detecting one or a plurality of partial character strings whose frequency of occurrence in a set of documents is equal to or higher than a first reference value;
For each of the detected one or more partial character strings, a score indicating the strength of association with the set is calculated, and among the one or more partial character strings, the partial character whose score exceeds a second reference value A selection process to select columns ,
The number of the newly selected partial character strings is calculated, and when the condition that the number shows a tendency to decrease and the number falls below the third reference value is satisfied , the detection process and the selection process are performed. determined to end, if the condition is not satisfied, a determination process not to end the detection process and the selection process,
When it is determined that the detection process and the selection process are not terminated, the first reference value is updated to a smaller value, and an iterative process for repeating the detection process, the selection process, and the determination process A keyword generation method including and executed by a computer. In the repetitive processing, keyword generation method according to claim 1, wherein the smaller the update quantity of the first reference value at every iteration. In the iterative process, before numeration is determined whether the approaching step of converging, if it is determined to be approaching to the stage, according to claim 1, wherein to reduce the amount of update of the first reference value Keyword generation method. A detection process for detecting one or a plurality of partial character strings whose frequency of occurrence in a set of documents is equal to or higher than a first reference value;
For each of the detected one or more partial character strings, a score indicating the strength of association with the set is calculated, and among the one or more partial character strings, the partial character whose score exceeds a second reference value A selection process to select columns ,
The number of the newly selected partial character strings is calculated, and when the condition that the number shows a tendency to decrease and the number falls below the third reference value is satisfied , the detection process and the selection process are performed. determined to end, if the condition is not satisfied, a determination process not to end the detection process and the selection process,
When it is determined that the detection process and the selection process are not terminated, the first reference value is updated to a smaller value, and an iterative process for repeating the detection process, the selection process, and the determination process A program that causes a computer to execute. A detection unit for detecting one or a plurality of partial character strings whose frequency of appearance in a set of documents is equal to or higher than a first reference value;
For each of the detected one or more partial character strings, a score indicating the strength of association with the set is calculated, and among the one or more partial character strings, the partial character whose score exceeds a second reference value A selection section for selecting a column ;
The number of the newly selected partial character strings is calculated, and when the condition that the number shows a tendency to decrease and the number falls below a third reference value is satisfied , the processing by the detection unit and the selection A determination unit that determines that the process by the unit is to be terminated, and determines that the process by the detection unit and the process by the selection unit are not terminated when the condition is not satisfied ,
If it is determined not to end the processing by the processing and the selection portion by the detecting unit, and updates the first reference value to a smaller value, the determination processing by the detecting unit and the processing by the selection unit An information processing apparatus comprising: a repeating unit that repeats processing by the unit.

Images