JP5193669B2 - Search system - Google Patents

Description

  The present invention relates to a search system, and more particularly to a search system that extracts terms closely related to an input search term and visually displays the extraction result.

A search system is used to extract necessary information from a vast amount of information. In the case of a general search system, there is a mechanism for extracting information that contains the same or similar concept as the input search term. ing. For example, if a search term “Fuji” is given to a database that stores information on a large number of companies, the search system can accurately output a list of companies that include the character string “Fuji” in the name. If you enter "environmental problem" at a search site on the Internet, a list of Web pages that contain the text "environmental problem" is displayed on the display.
As a result, the user can reach the target information, but the search results there are only in the expected range, and even if you look at the search result list, you can expect unexpected discoveries. There wasn't. Of course, new knowledge can be obtained in the process of examining details of individual data in the search result list, but information including other terms closely related to the search term cannot be extracted directly.

In this regard, in the case of the “associative search system” disclosed in Patent Document 1, the related term storage means that stores the related terms of each term and the co-occurrence with each term (appear in the same document) It has a co-occurrence company name storage means that stores company names (high probability). When a search term is entered, it extracts terms related to it and extracts company names with high co-occurrence for each term. It has a mechanism to do.
JP 2004-110386 A

As a result, when users enter "environmental problems" as a search term, it becomes possible to directly list the names of companies that often appear in documents related to environmental problems. Recognize and connect with investment behavior.
However, in the conventional search system including Patent Document 1, the search results are generally displayed in a list format arranged in a predetermined order (for example, new arrival order), and the relationship between the data as the search results is displayed. It could not be expressed visually.
For this reason, it is necessary to classify and analyze the search result on the user side, and it cannot be used for the purpose of easily and intuitively grasping the overall concept between related terms including the search term.

  The present invention has been devised to solve the above problem, and it is possible to extract information closely related to a search word based on the co-occurrence between keywords, and the relationship that is a search word and an extraction result. The purpose is to provide a search system that can visually express the relationship between words.

In order to achieve the above object, the search system according to claim 1 includes a keyword co-occurrence frequency storage means for storing a result of totaling the appearance frequencies of a plurality of keywords for each document data, and each document for each keyword. A keyword relevance storage unit that stores a relevance level based on co-occurrence between keywords, calculated using appearance frequency data in the data, and a relevance level for a specific keyword with reference to the keyword relevance storage unit Means for extracting a predetermined number of other keywords as related words in descending order, and the keyword co-occurrence frequency storage means, the ID of the document data in which the specific keyword and the related word appear, and the specification in each document data Means for acquiring the frequency of appearance of keywords and related words, and generating a cross tabulation table filled with the frequency of appearance for each document data; A data compression unit that compresses the cross tabulation table and stores it in the total data storage unit in association with the specific keyword, and, when a search word is input, the specific keyword corresponding to the search word is totaled from the total data storage unit A means for extracting data, a means for restoring a cross tabulation table based on this tabulation data, and a multivariate data with reference to this cross tabulation table and the occurrence frequency of specific keywords and related words for each document data as variables. Means for performing principal component analysis on each of them, calculating respective first principal component values and second principal component values, and based on the first principal component values and second principal component values, a specific keyword on a predetermined plane, A means for calculating the coordinate value of the related word and a relevance map in which a tag representing the specific keyword and the related word is arranged on the plane based on the respective coordinate values are generated. And a means for outputting the relevance map , wherein the data compression means compresses the cross tabulation table relating to the specific keyword into one-line tabulation data according to the following conversion rule: It is characterized by doing.
(1) A specific keyword and its related words are arranged in order, and a predetermined delimiter is placed between each keyword.
(2) Describe the numerical value representing the appearance frequency of each keyword in each document in association with the number representing the order of the keywords.
(3) When there are a plurality of numerical values representing the appearance frequency of each keyword in the same document, a predetermined delimiter is placed between them.
(4) When the appearance frequency is 0, the description in (2) is omitted.
(5) A predetermined delimiter is placed at the turn of the document.

In the search system according to claim 1, a keyword having a high degree of relevance with respect to the search word is extracted as a related word, and a principal component analysis is performed on the co-occurrence frequency of the search word and each related word for each document data. And calculating the coordinates of each keyword on a predetermined plane based on the first principal component value and the second principal component value derived as a result, and placing a tag indicating the keyword at each coordinate It has a mechanism to generate a map.
Therefore, the relationship between the keywords is reflected in the arrangement of each tag on the relevance map, and therefore the user can determine the similarity between the related words and the search word according to the positional relationship and accumulation degree between the tags. Can be visually recognized.
In addition, since the cross tabulation table for each keyword is converted into one-line tabulation data excluding redundancy, the cross tabulation table can be stored in the system very efficiently.

FIG. 1 is a block diagram showing a functional configuration of the first search system 10 according to the present invention. A document DB 12, a keyword extraction unit 14, a keyword DB 16, a relevance calculation unit 18, a keyword co-occurrence frequency table. 20, a keyword combination frequency total table 22, a keyword frequency total table 24, a keyword relevance table 26, and a map generation unit 28.
A web server 32 is connected to the system 10, and the web server 32 is connected to a plurality of clients 36 via a network 34 such as the Internet or an intranet.

  The keyword extraction unit 14, the relevance calculation unit 18, and the map generation unit 28 are realized by the CPU of the server computer executing necessary processes according to the OS and a dedicated application program.

The document DB 12, the keyword DB 16, the keyword co-occurrence frequency table 20, the keyword combination frequency sum table 22, the keyword frequency sum table 24, and the keyword association degree table 26 are stored in the hard disk of the server computer.
The document DB 12 stores a large number of document data (text data) such as newspaper articles, academic journals, and papers in advance. Each document data is associated with time information such as creation date / time, release date / time, and data storage date / time as one of metadata.

  As shown in FIG. 2, the keyword extraction unit 14 includes a dependency expression extraction filter 14a, a delimiter extraction filter 14b, a character string frequency statistical filter 14c, a TermExtract filter 14d, and a keyword recognition filter 14e.

Next, the keyword extraction process by the keyword extraction unit 14 will be described with reference to the flowchart of FIG.
First, the keyword extraction unit 14 applies a dependency expression extraction filter 14a to each document data stored in the document DB 12, and extracts a character string having a predetermined dependency expression from each document data (S10).
That is, the dependency expression extraction filter 14a has a large number of dependency expression patterns “XX manufacturer”, “XX is the main force”, and “XX is produced” in advance. After the expression pattern that applies to is detected, a character string corresponding to “XX” is extracted as a keyword candidate.

  Next, the keyword extraction unit 14 applies a delimiter extraction filter 14b to each document data, and commas such as “XX”, “XX”, (XX), [XX], XX, etc. The part of XX surrounded by delimiters such as parentheses, spaces, tabs, etc. is extracted as a keyword candidate (S12).

Next, the keyword extraction unit 14 applies a character string frequency statistical filter 14c to each document data, and counts how many times each character string included in each document data appears, including other documents. A character string having an appearance frequency is extracted as a keyword candidate (S14).
First, as shown in FIG. 4, the character string frequency statistical filter 14c pays attention to a noun (here, “DVD”) in the document, and the attention word “DVD” appears in each document data stored in the document DB 12. Add up the number you want. Next, the character string frequency statistical filter 14c expands the range to the morpheme before and after this attention word, and totals the frequency that appears in all the documents, and the appearance frequency becomes less than a certain value (for example, 20 or less). Stop character range expansion at this point.

  For example, since the appearance frequency of “done DVD” including the previous morpheme of the DVD is as low as “2”, the range is not expanded to the previous morpheme. On the other hand, since the appearance frequency of “DVD recorder” including the next morpheme of DVD is as many as “862”, the appearance frequencies of “DVD recorder” including the next morpheme are tabulated. Since the appearance frequency is as low as “5”, the expansion of the range to subsequent morphemes is stopped.

  The above “morpheme” refers to the smallest linguistic unit having meaning. For example, when “my name is Suzuki” is broken down into morphemes, “I (pronoun)” “no (particle)” “name (general noun)” “ha (counselor)” “Suzuki (proprietary noun)” “ Is (auxiliary verb) ".

Next, the character string frequency statistical filter 14c extracts “DVD” and “DVD recorder” as keyword candidates because they have an appearance frequency within a predetermined range (for example, 20 to 5,000). On the other hand, “done DVD” and “in the DVD recorder” are out of the above range, and are excluded from keyword candidates.
This is because, if the frequency of occurrence is less than 20 in all documents, it is not an important word in the first place, and if it exceeds 5,000, it is considered a general word or general word with no features. The amount is adjusted as appropriate according to the amount of use and the purpose of use of the search system.

  By the way, since it takes an enormous amount of time to count the appearance frequency of each character string included in a large amount of document data stored in the document DB 12, as shown in FIG. An index (so-called transposed index) is generated that lists whether or not each morpheme appearing in all document data is present in each document data. Therefore, the keyword extracting unit 14 can acquire the appearance frequency in a relatively short time by referring to the index.

Next, the keyword extraction unit 14 applies the TermExtract filter 14d to the document data stored in the document DB 12, and extracts a character string having a score equal to or higher than a predetermined value from each document data as a keyword candidate (S16).
This TermExtract filter 14d is a string extraction algorithm devised to automatically extract technical terms from specialized corpora (huge text data consisting mainly of natural language sentences collected mainly for research purposes). It has a function of extracting single nouns and compound nouns from the document data as candidate words, and calculating respective importance levels based on the appearance frequency and connection frequency of each candidate word. Since this TermExtract filter 14d itself is a known technique, further explanation is omitted.

Next, the keyword extraction unit 14 inputs each keyword candidate extracted by the dependency expression extraction filter 14a, the delimiter extraction filter 14b, the character string frequency statistical filter 14c, and the TermExtract filter 14d to the keyword certification filter 14e, and narrows down the keywords. .
In the keyword certification filter 14e, keyword candidates listed by each filter are matched, and those listed as keyword candidates by two or more filters are certified as final keywords and stored in the keyword DB 16 (S18). .

  As described above, by using the four filters of the dependency expression extraction filter 14a, the delimiter extraction filter 14b, the character string frequency statistical filter 14c, and the TermExtract filter 14d, important words are leaked when keywords are extracted from document data. In addition to this, it is possible to prevent unnecessary keywords (noise) from being mixed by narrowing down using the keyword certification filter 14e.

As described above, the keyword candidate selected by two or more of the four filters is recognized as an official keyword, and selection by three or more filters may be a requirement for keyword recognition. it can.
Further, the number of filters is not limited to the above, and other effective keyword candidate extraction filters may be provided in the keyword extraction unit 14.

Next, according to the flowchart of FIG. 6, the relevance calculation process between the keywords by the relevance calculation unit 18 is described.
First, the relevance calculation unit 18 aggregates the co-occurrence frequencies of each keyword in each document data, and generates a keyword co-occurrence frequency table 20 (S20).
FIG. 7 shows a specific example of the keyword co-occurrence frequency table 20. The appearance frequency of each keyword KW-1 to n is described for each document D1 to Dn stored in the document DB 12.

Here, the degree of association between a keyword X and Y can be theoretically calculated by substituting the appearance frequency of X and Y described in the keyword co-occurrence frequency table 20 into i of Equation 1. is there.

  Since the numerator of Equation 1 means the sum of the products of the appearance frequencies of the keywords X and Y for all documents, the value increases as the frequency of occurrence of X and Y in the same document increases. However, if the absolute number of occurrence frequencies of X and Y in a specific document is large, the value of the numerator increases accordingly, and it does not necessarily indicate the high co-occurrence of X and Y. . On the other hand, the denominator is obtained by adding the square roots of the sums of all the squares of the appearance frequencies of the keywords X and Y for each document, and the value increases as the appearance frequency in the specific document of X and Y increases. Become. For this reason, by dividing the numerator value by the denominator value, the influence of the large number of occurrence frequencies of X and Y in a specific document is eliminated, and the co-occurrence between X and Y is increased. It is possible to derive the degree of relevance based on it.

However, when the amount of document data and the total number of keywords are large, an enormous amount of calculation occurs, and a lot of processing time is required.
Therefore, in this embodiment, the calculation process is simplified by generating the keyword combination frequency summation table 22 and the keyword frequency summation table 24 based on the keyword co-occurrence frequency table 20.

FIG. 8 illustrates the procedure. In this case, the keyword co-occurrence frequency table 20 describes the appearance frequencies of the keywords KW-1 to KW-5 in the document D1, and among them, the appearance frequencies of KW-3 and KW-4 are 0. The combination of keywords for which the relevance is to be actually calculated is the following three patterns.
(KW-1, KW-2), (KW-1, KW-5), (KW-2, KW-5)
Next, the degree-of-relevance calculation unit 18 generates a keyword combination frequency sum table 22 describing values multiplied by the appearance frequency for each combination, and a keyword frequency sum table 24 describing values obtained by squaring the appearance frequency of each keyword. (S22, S24).

In the keyword combination frequency summation table of FIG. 8, only the value for the document D1 is described. However, the same processing is executed for each document, and the values are added based on the result. Is equivalent to the numerator of Equation 1.
Similarly, in the keyword frequency total table of FIG. 8, only the value for the document D1 is described, but the value obtained by squaring the appearance frequency of each keyword in each document is added, and the final value of each keyword is calculated. By obtaining the square root, a value corresponding to the denominator of Equation 1 is obtained.

As a result, as shown in FIG. 9, the relevance between the keywords can be calculated relatively easily, and the value is described in the keyword relevance table 26 (S26).
As described above, a combination pattern between keywords is extracted for each document, the product and the square value of each keyword are obtained, and the value of each document is added to obtain a keyword having a value of 0. It is possible to omit the calculation processing related to.
For this reason, it is realistic to calculate the degree of association between all keywords without being limited to the company name as in the search system of Patent Document 1.

Further, when new document data is added to the document DB 12, data related to each keyword in the new document data is added to the keyword combination frequency sum table 22 and the keyword frequency sum table 24, and the existing total value is added. By adding the additional values, the degree of association between keywords can be easily recalculated.
Even when the influence of obsolete document data is excluded, the data related to each keyword in the document data is deleted from the keyword combination frequency summation table 22 and the keyword frequency summation table 24, and the deleted value from the existing total value By subtracting, it is possible to easily maintain the degree of association between keywords in the latest state.

Next, the search process procedure in the system 10 will be described with reference to the flowchart of FIG.
First, when the user inputs a character string to be searched from the client 36, the map generating unit 28 that has received the character string (S30) refers to the keyword relevance table 26, and is similar to the character string or within a certain range. Are identified as search terms, and a predetermined number of keywords having high relevance to the keywords are extracted as related words (S32).

For example, as shown in FIG. 11, when the hiragana “himan” is given as a search target from the client 36, the map generation unit 28 recognizes the kanji “obesity” registered as a keyword as a search word. The keywords “Fat”, “Diabetes”, “Diabetes”,... “Child” are selected as related terms having a high degree of association with this.
There is no particular limitation on the number of keywords selected as related words, but here, keywords with the highest degree of relevance are selected.

Next, the map generating unit 28 refers to the keyword co-occurrence frequency table 20 and acquires the appearance word ID of the search word, the extracted 50 related words, and the appearance frequency for each document (S34).
FIG. 12 is a table (cross tabulation table) showing the extraction results, in which a document ID is set as a line item, and a search word and each related word are set as a column item. In each cell, the appearance frequency of each keyword in the corresponding document is described.
For example, the document ID: 13691 indicates that obesity: 3 times, fat: 13 times, diabetes: 0 times, diabetes: 0 times, habit: 0 times,.

  Next, the map generation unit 28 performs principal component analysis on the multivariate data having the appearance frequency of each keyword for each document shown in the table of FIG. 12 as a variable (S36), and as shown in FIG. As the analysis result, the value of the first principal component and the value of the second principal component of each keyword are calculated (S38).

Next, the map generation unit 28 calculates coordinate values for arranging each keyword on a two-dimensional plane having a predetermined area based on the first principal component value and the second principal component value of each keyword (S40). ).
A specific example of coordinate calculation is shown below.

-Coordinate plane size: set to horizontal 740 dots x vertical 500 dots-X coordinate conversion ratio = horizontal width of coordinate plane / (maximum value of first principal component-minimum value of first principal component)
Y coordinate conversion ratio = vertical width of coordinate plane ÷ (maximum value of second principal component−minimum value of second principal component)
X coordinate of keyword A = (first principal component value of keyword A−minimum value of first principal component) * X coordinate conversion ratio Y coordinate of keyword A = (second principal component value of keyword A−second principal Minimum value of component) * Y coordinate conversion ratio

With the above calculation, the first principal component value and the second principal component value of each keyword can be converted into numerical values between the X axis between 0 and 740 and the Y axis between 0 and 500. It becomes possible to arrange each tag to be described later.
FIG. 14 illustrates the coordinate values of each keyword calculated according to the above conversion law.

Next, the map generator 28 generates a tag indicating the presence of each keyword (S42).
That is, as shown in FIG. 15, the tag 40 has a rectangular shape, and includes a character string of each keyword (search word and related word) and a blank portion surrounding the character string.
In order to distinguish from related words, a character color and a background color different from each related word are assigned to the search word.

The area of the tag 40 is automatically determined according to the keyword font size and the number of characters. As the keyword font size, a larger font size is assigned as the degree of association with the search word increases.
Below, the font size allocation method of each keyword is demonstrated.

-Maximum font size = 36 points-Minimum font size = 12 points-Font size conversion ratio = (maximum font size-minimum font size) ÷ (maximum relevance-minimum relevance)
-Keyword A font size = {(Keyword A relevance-Relevance minimum) * Font size conversion ratio * Rounded down

Next, the map generation unit 28 arranges each of the tags 40 on the two-dimensional plane, and generates a first relevance map (S44).
At this time, the center points of the respective tags 40 are arranged so as to overlap the coordinate points on the two-dimensional plane obtained as described above.

This first relevance map is processed into a Web file by the Web server 32 and transmitted to the client 36 (S46).
As a result, the first relevance map 42 is displayed on the Web browser of the client 36 as shown in FIG.

  On the first relevance map 42, since the font size of each tag 40 represents the strength of the relevance with the search word as described above, the keyword most relevant to the search word “obesity” is “ It can be understood that it is “fat”. It can also be understood that keywords such as “diabetes”, “diabetes”, and “custom” have a relatively strong relationship with “obesity”.

In addition, the position (coordinates) of each tag 40 reflects the strength of the association based on the co-occurrence between the search term and each related word, and the closer the distance between the tags 40, the stronger the mutual relationship. Is shown.
For example, tags 40 such as “adiponectin (a hormone secreted from adipocytes)”, “built-in fat”, “myocardial infarction”, “metabolic syndrome” are gathered in the vicinity of the keyword “fat”. It can be seen that these keywords are strongly related.
Thus, the user can recognize the similarity and category between the keywords by observing the accumulation of the tags 40 on which the keywords are written.

FIG. 17 shows a relevance map 42 when the user inputs “environmental technology” as a search term, and is generated by the map generator 28 through the above steps S30 to S46. .
In this case, the keyword “global environment” has the largest font size, and thus indicates that the keyword “environment technology” is strongly related to the search term. It can also be understood that “global warming” and “hybrid” have a relatively strong relationship with “environmental technology”.

When the user puts the mouse pointer on the “fuel cell” tag 40 and clicks on it, the control program incorporated in the Web page by JavaScript or the like functions and the blowing menu 44 is expanded.
When the user selects “Go to“ fuel cell ”” from the menu 44, a character string “fuel cell” is transmitted from the client 36 to the Web server 32 as a new search term.

Upon receiving this, the map generation unit 28 generates the new relevance map 42 in which the fuel cell is set as the search word by executing the processes of S30 to S46, and transmits it to the client 36.
As a result, as shown in FIG. 18, the first relevance map 42 in which the fuel cell is set as the search word is displayed on the Web browser of the client 36.
On the first relevance map 42, a tag 42 describing a keyword that is highly relevant to the search term “fuel cell” is arranged.

  In this way, the user selects the tag 40 displayed on the first relevance map 42 from the next to the next, and sets the keyword as a search word so that the user can continue the associative journey. You can broaden your horizons and experience new discoveries and unexpected awareness in the course.

  Next, when the user clicks on a specific tag 40 on the relevance map 42, for example, “city gas”, and selects “relevance confirmation 1 (in the site)” from the expanded balloon menu 44, the client 36. The web server 32 transmits a request for presenting the basis for associating the fuel cell with the city gas.

Upon receiving this, the map generation unit 28 searches the keyword co-occurrence frequency table 20 based on the search term “fuel cell” and the selected “city gas” as shown in FIG. Generate a list of document numbers that have occurred.
Next, the map generation unit 28 searches the document DB 12 based on the document number list, generates a list of document texts, and transmits the list to the client 36 via the Web server 32.
As a result, the display of the client 36 displays a list of document numbers, titles, abstracts, dates, etc. in which fuel cells and city gas appear simultaneously.

When the user selects one of these, the map generation unit 28 extracts the corresponding document data from the document DB 12 and transmits it to the client 36.
As a result, the user can browse the contents of the document data and individually confirm the relevance between “fuel cell” and “city gas”.

On the other hand, when the user selects “Relevance Confirmation 2 (on the Web)” in the balloon menu 44 of FIG. 18, a new window for connecting to a preset search site by the control program embedded in the Web file. Alternatively, the tag is activated on the Web browser, and a search query in which “fuel cell” and “city gas” are connected under an AND condition is transmitted to the search site.
As a result, website information including both fuel cells and city gas will be listed on the web browser, and users will be able to find out the relationship between “fuel cells” and “city gas” through the internet website. It becomes possible to confirm.

On the other hand, when the user selects “Search on the Web” in the balloon menu 44 of FIG. 18, a new Web browser for connecting to a preset search site is started, Only “gas” is sent as a search term.
As a result, it is possible to check information on the website related to “city gas”.

Further, when the user selects “reject association” in the balloon menu 44 of FIG. 18, information that denies the association between “fuel cell” and “city gas” is stored in a predetermined storage means on the server. Is done.
And when this information exceeds a certain number (for example, 10 or more), the data in the keyword relevance table 20 is corrected, and the relevance between “fuel cell” and “city gas” is subtracted by a predetermined point, Or it is reset to 0.

By the way, when each tag 40 is arranged on a two-dimensional plane having a predetermined area based on the respective coordinates, the tags may overlap each other as shown in FIG.
In such a case, it is desirable to make adjustment so that the user's visibility is emphasized and the duplication between the tags is resolved so that the characters can be easily recognized. It is important to keep it to a minimum.
Therefore, this system 10 employs the following algorithm in order to eliminate the overlapping relationship while minimizing the moving distance of each tag.
The procedure will be described below with reference to the flowchart of FIG.

First, the map generation unit 28 compares the areas of the tags and fixes the positions in order of increasing area (S50). As described above, the size of each tag is determined depending on the font size and the number of characters of the keyword.
Further, when an overlap between tags is detected in this process (S52: Yes), the areas of the overlapping tags are compared (S54), and the position of the tag with the largest area is fixed (S56).

Next, the map generation unit 28 moves the tag having the second largest area among the overlapping tags in either the top, bottom, left, or right direction to eliminate the overlapping state with the tag having the largest area (S58).
At this time, the map generator 28 is bound by the following rules.
(1) In principle, select the direction that requires the shortest travel distance.
(2) In principle, you cannot select a direction that overlaps with a tag that has already been fixed.
(3) If the tag overlaps with a fixed tag in any direction, select the direction that requires the smallest overlapping area.
(4) The direction beyond the overall frame α cannot be selected.
(5) Movement in the return direction cannot be selected.

When duplication with another tag occurs at the destination (S60: Yes), the map generation unit 28 repeats steps S54 to S58 to avoid duplication.
The map generation unit 28 repeats the processing of S54 to S60 until adjustment processing for avoiding duplication is completed for all tags (S62).

FIG. 22 shows a specific example of the duplication elimination processing between tags, and between the “ABC” tag 40a, “DEF” tag 40b, “GHI” tag 40c, and “JKLMNO” tag 40d in the vicinity of the overall frame α. Shows a state where duplication occurs.
In this case, the map generation unit 28 first compares the areas between the overlapping tags (S54), and fixes the “ABC” tag 40a having the largest area at the current position (S56).

Next, the map generation unit 28 moves the position of the “DEF” tag 40b, which has the next largest area after the “ABC” tag 40a, in either the top, bottom, left, or right direction, and eliminates the overlap with the “ABC” tag 40a ( S58).
In this case, the shortest moving distance is in the upward direction, and it does not overlap with other fixed tags or conflict with the entire frame α. Therefore, as shown in FIG. Moves the “DEF” tag 40b upward.

Next, the map generation unit 28 moves the position of the “GHI” tag 40c, which has the next largest area after the “DEF” tag 40b, in either the top, bottom, left, or right direction, and eliminates the overlap with the “ABC” tag 40a.
In this case, since there is a conflict with the whole frame α, the left direction is excluded as a destination according to the rule (4). In addition, the upward movement overlaps with the fixed “DEF” tag 40b, the rightward direction also overlaps with the fixed “STUV” tag 40f, and the downward direction with the fixed “PQR” tag 40e. Since duplication occurs, the rule (2) cannot be selected. Therefore, the map generation unit 28 applies the rule (3) described above, and selects the downward direction with the smallest overlap area as the movement destination of the “GHI” tag 40c as shown in FIG.

As a result, since a new overlapping relationship between the “GHI” tag 40c and the “PQR” tag 40e occurs, the map generation unit 28 continues to set the “GHI” tag 40c as a movement target.
In this case, the movement in the left and down directions causes a conflict with the whole frame α (the rule violation of (4) above), and the upward direction is the return direction (the rule violation of (5) above). As shown in FIG. 25, the map generation unit 28 moves the “GHI” tag 40c to the right to eliminate the duplication with the “PQR” tag 40e.

As a result, since the overlapping relationship between the “GHI” tag 40c and the “STUV” tag 40f occurs, the map generation unit 28 sets the “GHI” tag 40c as the movement target again.
In this case, duplication with the “STUV” tag 40f can be avoided with the shortest moving distance in the downward direction, and it does not overlap with the fixed tag nor conflict with the entire frame α. The map generation unit 28 moves the “GHI” tag 40c downward.

Next, the map generation unit 28 moves the position of the remaining “JKLMNO” tag 40d in any of the upper, lower, left, and right directions, and eliminates the overlap with the “ABC” tag 40a.
In this case, since there is a conflict with the whole frame α, the left direction is excluded as a destination according to the rule (4). In addition, since the vertical movement and the rightward movement both overlap with the fixed tag, the map generation unit 28 follows the rule (3) above, and has the smallest overlapping area as shown in FIG. The downward movement is selected to eliminate the overlapping relationship with the “ABC” tag 40a.

As a result, since the overlapping relationship between the “JKLMNO” tag 40d and the “PQR” tag 40e is newly generated, the map generation unit 28 sets the “JKLMNO” tag 40d as the movement target again.
In this case, if it moves downward, there will be no overlap with other fixed tags, and there will be no conflict with the whole frame α. Therefore, as shown in FIG. 28, the map generator 28 uses the “JKLMNO” tag 40d. Move down.
Thereby, all the duplication states between tags are eliminated.

In the above, since it is based on the premise that the entire frame α is fixed, the rule (2) is set, and the movement in the direction that conflicts with the entire frame α cannot be selected. It is not limited.
For example, it is possible to allow movement beyond the entire frame α by making the arrangement plane of the tag scrollable up and down, left and right, or by making the entire arrangement screen zoom in / zoom out.

In the above description, the method for eliminating any duplication of tags has been described. However, the adjustment can be made flexibly so as to allow slight duplication.
For example, if the setting is made so as to allow duplication within 5% of the area of each tag, the moving distance can be kept short while maintaining the visibility of the tag relatively well.

The first relevance map 42 described above extracts keywords in the top 50 having high relevance to the search word as related words, and determines the co-occurrence frequency for each document data of each keyword including the search word. Then, the principal component analysis is performed, and the tag 40 of each keyword is arranged on the coordinate plane based on the first principal component value and the second principal component value derived as a result.
Therefore, the relationship between the keywords is reflected in the arrangement of each tag 40 on the first relevance map 42, and therefore, the user selects the category between keywords according to the positional relationship and accumulation degree between the tags 40. There is an advantage that it is possible to grasp.

On the other hand, the relationship with the search term could not be read directly from the positional relationship of the tag 40. Of course, although the size of the font written in the tag 40 expresses the strength of the relevance to the search term, the search term is placed at the center of the two-dimensional plane and It is also possible to generate a relevance map in which the strength is directly expressed by the positional relationship of the tags 40.
Hereinafter, according to the flowchart of FIG. 29, a procedure for generating such a relevance map will be described.

  First, when a user inputs a character string to be searched from the client 36, the map generation unit 28 that has received the character string (S70) refers to the keyword relevance table 26, and is similar to the character string or within a certain range. Are identified as search terms, and a predetermined number of keywords having high relevance to the keywords are extracted as related words (S72).

  For example, “environmental technology” is given as a search term, and related terms within the top 50 are extracted. FIG. 30 shows an extraction example.

ここで、「0.05」は螺旋の広がり度合を決定する固定値の係数を意味している。
「ｎ」は関連度の順位を意味し、螺旋の中心が１、すなわち検索語に該当するため、関連語の第１位についてはｎ＝２が代入されることとなる。
また、「ｅ」は自然対数を意味している。 Next, the map generation unit 28 substitutes the rank data of each related word into the spiral equation shown in Equation 2, and calculates the respective X and Y values (S74).

Here, “0.05” means a fixed value coefficient that determines the extent of the spiral.
“N” means the rank of relevance, and since the center of the spiral corresponds to 1, that is, the search word, n = 2 is substituted for the first rank of the related word.
“E” means a natural logarithm.

As a result, as shown in FIG. 31, the X value and Y value of each related word are derived by the map generation unit 28.
Next, the map generation unit 28 calculates coordinate values on a predetermined plane based on the X value and Y value of each related word (S76).
A specific example of coordinate calculation is shown below.

-Coordinate plane size: Set to horizontal 740 dots x vertical 500 dots-X coordinate conversion ratio = horizontal width of coordinate plane / (maximum value of X value-minimum value of X value)
Y coordinate conversion ratio = vertical width of coordinate plane ÷ (maximum Y value−minimum Y value)
X coordinate of keyword A = (X value of keyword A−minimum value of X value) * X coordinate conversion ratio Y coordinate of keyword A = (Y value of keyword A−minimum value of Y value) * Y coordinate conversion ratio

By the above calculation, the X value and Y value of each keyword can be converted into a numerical value between X axis 0-740 and Y axis 0-500, and each tag 40 is arranged on the above two-dimensional plane. It becomes possible.
FIG. 32 illustrates the coordinate values of each keyword calculated according to the above conversion law.

Next, the map generator 28 generates a tag indicating the presence of each keyword (S78).
As shown in FIG. 15, the tag 40 has a rectangular shape, and includes a character string of each keyword and a blank portion surrounding the character string.
In order to distinguish from related words, a character color and a background color different from each related word are assigned to the search word.
The area of the tag 40 is also automatically determined according to the keyword font size and the number of characters, as described above. The keyword font size is assigned in the same manner as described above, and a larger font size is assigned as the degree of association with the search word increases.

Next, the map generation unit 28 arranges each of the tags 40 on the two-dimensional plane, and generates a second relevance map (S80).
At this time, the center point of each tag 40 is arranged so as to overlap the coordinate point on the plane obtained above. Further, the tag 40 corresponding to the search word is arranged at the center of the two-dimensional plane.

The second relevance map is processed into a Web file by the Web server 32 and transmitted to the client 36 (S82).
As a result, the second relevance map 46 is displayed on the Web browser of the client 36 as shown in FIG.

On the second relevance map 46, since the font size of each tag 40 represents the strength of the relevance with the search term as described above, the keyword most relevant to the search term “environmental technology” is It can be understood that it is the “global environment”. It can also be understood that keywords such as “hybrid”, “global warming”, “warming” have a relatively strong relationship with “environmental technology”.
Further, in the second relevance map 46, the search term tag 40 is arranged at the center of the two-dimensional plane, and the other tags 40 are arranged so as to spirally surround the tags according to the respective relevance degrees. Therefore, it is possible to read the degree of association with the search word based on the arrangement of each tag 40 (distance from the tag of the search word).

  When the second relevance map 46 is generated, the coordinates of the tag 40 are obtained based on the ranking of each keyword. Therefore, once the correspondence between the ranking and the coordinate value is calculated, the size of the two-dimensional plane is calculated. As long as there is no change in the number of related words to be extracted, there is an advantage that the tag 40 can be immediately placed on a two-dimensional plane without recalculation, and the amount of calculation of the system 10 can be reduced.

When the user clicks in a state where an arbitrary tag 40 on the second relevance map 46 is selected, the balloon menu 44 is expanded as in the case of the first relevance map 42 described above.
When the user selects “Go to“ Toyota ”” from the menu 44, a character string “Toyota” is transmitted from the client 36 to the Web server 32 as a new search term.

Receiving this, the map generating unit 28 generates the new second relevance map 46 in which Toyota is set as the search word by executing the processing of S70 to S80, and transmits it to the client 36 (S82). ).
As a result, although not shown, a second relevance map 46 in which Toyota is set as a search word is displayed on the Web browser of the client 36.

  In this way, the user selects the tag 40 displayed on the second relevance map 46 from the next to the next, and sets the keyword as a search word so that the user can continue the associative journey. You can broaden your horizons and experience new discoveries and unexpected awareness in the course.

  Further, when the user selects “relevance confirmation 1 (in site)” from the blowing menu 44, the client 36 transmits a request for presenting the basis for associating environmental technology with Toyota to the Web server 32.

Upon receiving this, the map generation unit 28 searches the keyword co-occurrence frequency table 20 based on the search term “environmental technology” and the selected “Toyota” as shown in FIG. Generate a list of document numbers that have occurred.
Next, the map generation unit 28 searches the document DB 12 based on the document number list, generates a list of document texts, and transmits the list to the client 36 via the Web server 32.
As a result, the display of the client 36 displays a list of document numbers, titles, abstracts, dates, etc. in which environmental technology and Toyota appear simultaneously.

When the user selects one of these, the map generation unit 28 extracts the corresponding document data from the document DB 12 and transmits it to the client 36.
As a result, the user can browse the contents of the document data and individually confirm the relationship between “environmental technology” and “Toyota”.

On the other hand, when the user selects “Relevance Confirmation 2 (on the Web)” in the above-mentioned balloon menu 44, a new window for connecting to a preset search site or by a control program incorporated in the Web file The tag is activated on the web browser, and a search term that connects "environmental technology" and "Toyota" with the "and" condition is sent to the search site.
As a result, website information including both environmental technology and Toyota will be listed on the web browser, and the user will check the relationship between "environmental technology" and "Toyota" through the Internet website. It becomes possible.

On the other hand, when the user selects “Search on the Web” in the blowing menu 44, a new window or tag for connecting to the preset search site is started on the Web browser, and the search site is displayed. Only "Toyota" is sent as a search term.
As a result, it is possible to confirm information on the website related to “Toyota”.

Further, when the user selects “deny relevance” in the balloon menu 44, information denying the relevance between “environmental technology” and “Toyota” is stored in a predetermined storage means on the server.
Then, when this information exceeds a certain number (for example, 10 or more), the data in the keyword relevance table 20 is corrected, and the relevance between “environmental technology” and “Toyota” is subtracted by a predetermined point, or Reset to zero.

  When duplication between the tags 40 occurs when the second relevance map 46 is generated, the map generation unit 28 eliminates the duplication between the tags 40 according to the same rule as described above.

By the way, in the search result display method described above, only the level of relevance based on the co-occurrence between the search word and each related word is reflected on the relevance map. By reflecting the temporal elements of the above and making it possible to distinguish the related words that have recently attracted particular attention from other related words, the utility value of the relevance map can be further increased.
In the following, a method of reflecting temporal elements of each related word in the first relevance map 42 will be described according to the flowchart of FIG.

  First, search keyword reception (S101), related word extraction processing (S102) that is closely related to the search word, appearance document ID of the search word and related word, and appearance frequency in each document (S103), Implementation of principal component analysis on appearance frequency in document (S104), extraction of first principal component value and second principal component value of each keyword (S105), calculation of coordinate data of each keyword (S106), search term and each The process up to the generation of tags including related words (S107) is substantially the same as S32 to S42, which is the processing procedure for generating the first relevance map shown in FIG.

Next, the map generation unit 28 refers to the metadata of the appearance document of each related word stored in the document DB 12, and acquires date information (creation date, etc.) of each document (S108).
Next, the map generation unit 28 rearranges the appearance frequency in the order of the date of the document data in the related word unit, and gives a weight according to the newness of the date (S109).
FIG. 35 is a table showing an example of this, in which the appearance frequency for each date is described for the keyword “fat”, and a weight value is associated with each date.

  This weight is given the highest “365” on the latest date, May 01, 2008, and thereafter, the weight value decreases by one every day. All dates prior to one year are given a weight of “0”.

  Next, the map generating unit 28 calculates the freshness point of the keyword by multiplying the appearance frequency for each date by the corresponding weight and adding the product for the past one year (S110).

Next, the map generation unit 28 calculates an average value of appearance frequencies per day within a predetermined period before a preset burst period (S111). Here, the “burst period” is the latest period that serves as a reference for determining whether a certain keyword is in season or topical.
For example, if the burst period is set as “the past month based on the latest date” and the averaging period for calculating the average value is 11 months, the “fat” in the 11 months before the burst period The average is the quotient obtained by summing up the appearance frequencies and dividing this by the number of days included in 11 months.

  Next, the map generation unit 28 calculates the rapidity of each related word by dividing the freshness point of the keyword by this average value (S112).

  Next, the map generation unit 28 performs a differentiation process based on the level of rapid progress on a specific tag (S113). For example, for a related word having a rapid advance degree of 5th or lower from the top, a process of applying a color or pattern different from those of other related words to the background is applicable.

Next, the map generation unit 28 generates a first relevance map in which tags of all related words including tags subjected to differentiation processing and tags of search words are arranged on a setting plane (S114).
This first relevance map is processed into a Web file by the Web server 32 and transmitted to the client 36 (S115).

As a result, the first relevance map 42 is displayed on the Web browser of the client 36 as shown in FIG. On the first relevance map, the nutritional teacher, BMI, adiponectin, metabolic syndrome, and gene tags are provided with a different shading pattern from the other tags. This is due to the processing of S113 described above in which the tags with the first to fifth ranks are differentiated from other keyword tags.
As a result, the user can recognize at a glance that the keyword “adiponectin” is a so-called “seasonal keyword” in which the document publication frequency has rapidly increased in the past month.

  In addition, it can be said that the above-described freshness point calculated by the appearance frequency × weight for each date embodies the fact that a temporal element is added to each related word. However, when the “newness” of the keyword is determined only by the number of freshness points, a high value may be given to a general frequent word. For example, as for the keyword “obesity”, “fat”, “calorie”, “diabetes”, etc. are frequent words that always appear as a set at any time, and such “season” can never be said. As a result, a high freshness point is given to such words.

  Therefore, the map generation unit 28 suppresses the rapidity of frequently occurring frequently occurring words by adding the process of dividing the freshness point by the average value of the appearance frequency of each keyword before the burst period as described above. As a result, as described above, it is possible to effectively narrow down the keywords whose appearance frequency has rapidly increased within the burst period.

The burst period (the past one month) and the averaging target period (11 months before the burst period) are examples, and can be freely changed.
Instead of calculating by date, it can also be calculated by week, 10 days, or month.
The weighting method is not limited to the above, and it can be set so that a certain number of weights decrease in units of one week or in units of 10 days.

As described above, the minimum value of the weight given to the appearance frequency for each date is not limited to “0”, and a negative weight is given to the date before the predetermined period according to its age. Even so, it is possible to keep the freshness point after frequent frequent occurrences low.
However, in this case, for related terms that have been noticed once in the past, the degree of attention has decreased after the appearance frequency in the document data has increased, and the frequency of appearance has increased again after a predetermined sluggish period, The value of “weight × appearance frequency” is greatly reduced by the past “minus weight × appearance frequency” value, resulting in a problem that the freshness point is unduly lowered.
On the other hand, as described above, weights of “0” are uniformly given to the dates past the predetermined period regardless of the age, and the negative weights are excluded, thereby attracting attention once in the past. It is possible to reflect a legitimate degree of rapid progress for related words that have recently attracted attention.

Of course, it is possible to reflect this rapid degree of progress in the second relevance map 46.
In the following, a method for reflecting the temporal element of each related word on the second relevance map 46 will be described according to the flowchart of FIG.

First, search keyword reception (S121), extraction processing of related words within the top 50 most closely related to the search word (S122), the ranking of each related word is substituted into the spiral equation, and the X value and Y value are calculated. Processing to calculate (S123), processing to convert X value and Y value into coordinate data on setting plane (S124), generation processing of tag including search word and each related word (S125)
Up to this point, it is substantially the same as S70 to S78, which is the procedure for generating the second relevance map shown in FIG.

Next, the map generation unit 28 acquires the appearance document ID of the search word and the related word and the appearance frequency in each document (S126).
Next, the map generation unit 28 refers to the metadata of the appearance document of each related word, and acquires date information (creation date, etc.) of each document (S127).
Next, the map generation unit 28 rearranges the appearance frequencies in the order of the date of the document data in units of related words, and gives a weight according to the newness of the date (S128).
Next, the map generation unit 28 calculates the freshness point of the keyword by multiplying the appearance frequency for each date by the corresponding weight and sums the product for the past one year (S129).
Next, the map generation unit 28 calculates the average value of the appearance frequency per day for 11 months before the burst period of the most recent one month (S130).
Next, the map generation unit 28 calculates the rapidity of each related word by dividing the freshness point of the keyword by this average value (S131).
Next, the map generation unit 28 performs a differentiation process based on this rapid progress on a specific tag (S132). For example, for a related word having a rapid advance degree of 5th or higher from the top, a process of applying a color or pattern different from that of other related words to the background, or a process of changing the tag shape from a rectangle to an ellipse or the like is applicable.

Next, the map generation unit 28 generates a second relevance map in which tags of all related words including tags subjected to differentiation processing and tags of search words are arranged on a setting plane (S133).
The second relevance map is processed into a Web file by the Web server 32 and transmitted to the client 36 (S134).

  As a result, the second relevance map 46 is displayed on the Web browser of the client 36 as shown in FIG. On the second relevance map 46, the hybrid, hybrid vehicle, Toyota, global economists summit, and earth symbiosis project tags have different shading patterns than other tags. This is due to the processing of S132 described above, in which the tags with the first to fifth rapid degrees are differentiated from other keyword tags.

  As described above, according to this embodiment, by referring to the date of the document data in which each related word appears, weighting can be included in the related word search result. In the conventional search, the document data, which is the base for extracting related terms, contains various time information such as creation time, release time, accumulation time in the database, etc. Information was discarded. On the other hand, in this embodiment, it becomes possible to reflect the time information in the search result to some extent.

  In the above, a weight according to the creation date of the document data is given, but in addition, a weight according to the date when the document data is stored in the database or the date published on the website is given. Also good.

Further, in the above description, an example is given in which display differentiation processing is performed in which only a related word having a rapid advance degree within the top five is attached to the tag with a shaded pattern. However, the present invention is limited to this. Instead, it is possible to perform differentiation processing on the related words within the top 10 or perform differentiation processing only on the top 3 related words.
Alternatively, the rapidity of each related word can be divided into a plurality of bands within a predetermined range, and a display mode different from other bands can be assigned to each band.

FIG. 39 is a block diagram showing a functional configuration of the second search system 50 according to the present invention. The second search system 50 includes a document DB 12, a keyword extraction unit 14, a keyword DB 16, a relevance calculation unit 18, a keyword co-occurrence frequency table 20, a keyword combination frequency sum table 22, and a keyword frequency sum table. 24, a keyword relevance table 26, a total data generation unit 52, a total data DB 54, and a map generation unit 28.
A web server 32 is connected to the system 50, and the web server 32 is connected to a plurality of clients 36 via a network 34 such as the Internet or an intranet.

  The keyword extraction unit 14, the relevance calculation unit 18, the total data generation unit 52, and the map generation unit 28 are realized by the CPU of the server computer executing necessary processing according to the OS and a dedicated application program. .

The document DB 12, the keyword DB 16, the keyword co-occurrence frequency table 20, the keyword combination frequency sum table 22, the keyword frequency sum table 24, the keyword relevance table 26, and the total data DB 54 are stored in the hard disk of the server computer.
A large number of electronic data (text data) such as newspaper articles, academic journals, and papers is stored in the document DB 12 in advance.

  As is apparent from the figure, the second search system 50 includes a total data generation unit 52 and a total data DB 54 between the map generation unit 28, the keyword co-occurrence frequency table 20, and the keyword relevance level table 26. The other features are substantially the same as those of the first search system 10, and therefore, the same functional components are denoted by the same reference numerals, and redundant description is omitted.

  The total data generation unit 52 in the second search system 50 refers to the keyword co-occurrence frequency table 20 and the keyword relevance level table 26 in advance to generate the cross tabulation table illustrated in FIG. It has a function of storing aggregated data compressed and converted into a format in the aggregated data DB 54.

  That is, in the case of the first search system 10, after receiving a search request from the user, the map generation unit 28 refers to the keyword relevance table 26 and extracts 50 related words, and the keyword co-occurrence frequency The search document and the appearance document ID of each related word and the appearance frequency in each document are acquired with reference to Table 20, and the cross tabulation table of FIG. 12 is generated each time the search is performed. It takes a considerable amount of time to output the relevance map 42, and there is a concern that the load on the system temporarily increases.

  Of course, if the cross tabulation table of FIG. 12 is created in advance and stored in the hard disk, it is natural that the processing time at the time of the search request can be shortened. However, the actual number of keywords and the number of document data are enormous. Therefore, the number of cross tabulation tables that record the frequency of occurrence of related words up to the top 50 related words in all documents and the amount of each data are enormous, and it is necessary to secure a large disk capacity for storage. It was. In addition, in each cross tabulation table, cells with frequency = 0 occupy the majority, and it is necessary that resources be wasted. Therefore, in the case of the first search system 10, when the search request is received, The method of creating only the cross tabulation table related to was adopted.

  On the other hand, in the case of the second search system 50, the amount of data is greatly reduced by devising the data format of the cross tabulation table, and the above problem is solved. Hereinafter, the processing procedure in the total data generation unit 52 will be described with reference to the flowchart of FIG.

First, the total data generation unit 52 refers to the keyword relevance level table 26 and extracts related words having a relevance level within the top 50 for each keyword (S141).
Next, the total data generating unit 52 refers to the keyword co-occurrence frequency table 20, and acquires the appearance document ID of the specific keyword and its related word and the appearance frequency in each document (S142).
Next, the total data generation unit 52 sets the document ID in the row item, sets the specific keyword and its related word in the column item, and generates a cross tabulation table in which the appearance frequency is filled in each cell (S143). .
Next, the total data generation unit 52 compresses the redundant cross tabulation table having vast areas in the vertical and horizontal directions and occupying most of the cells with value = 0 in accordance with a certain conversion method (S144).

FIG. 41 is an explanatory diagram showing the point of this compression processing. A cross tabulation table 56 similar to that shown in FIG. 12 is converted into tabulated data 58 of only one row.
This conversion method will be explained in detail. First, keywords such as obesity and fat (specific keywords and related words) corresponding to the column names in the cross tabulation table 56 are arranged in a line separated from each other by “,” (comma). Let
For each cell, if there is a value other than 0 (appearance frequency), the position information and value are expressed as “1> 2”. Among these, “1” indicates the order of keywords, and specifically indicates “obesity” positioned first. The part “2” indicates that the appearance frequency is “2”. “>” Is a symbol that associates the order of each keyword with the appearance frequency.
The description of a cell having a value of 0 is omitted.
A line break (change of document) is expressed by “/”, which means that a value in the next document starts.
When there are a plurality of values in the same line, a delimiter of “,” (comma) is placed between them.

Hereinafter, each part of the total data 58 will be specifically described.
First, "obesity, fat, diabetes, diabetes, habits /" at the beginning are keywords corresponding to column names: obesity (first), fat (second), diabetes (third), diabetes (fourth), Expresses that it is a habit (fifth).
“1> 2 /” in (1) represents that the frequency of “obesity” in the first document (6102) is “2” and the frequency of other keywords is “0”.
In (2) “1>2,5> 3 /”, the frequency of “obesity” in the second document (8594) is “2”, the frequency of “custom” is “3”, and other keywords The frequency of “0” is expressed as “0”.
“1> 4 /” in (3) indicates that the frequency of “obesity” in the third document (10104) is “4” and the frequency of other keywords is “0”. .
“1> 2 /” in (4) indicates that the frequency of “obesity” in the fourth document (11671) is “2” and the frequency of other keywords is “0”. .
In (5), “1>3,2> 2 /” indicates that the frequency of “obesity” in the fifth document (13690) is “3”, the frequency of “fat” is “2”, and other keywords The frequency of “0” is expressed as “0”.
In (6), “1>3,2> 13 /” indicates that the frequency of “obesity” in the sixth document (13691) is “3”, the frequency of “fat” is “13”, and other keywords The frequency of “0” is expressed as “0”.
“1> 4 /” in (7) indicates that the frequency of “obesity” in the seventh document (18026) is “4” and the frequency of other keywords is “0”. .
“1> 5 /” in (8) represents that the frequency of “obesity” in the eighth document (21642) is “5” and the frequency of other keywords is “0”. .
“1> 2 /” in (9) indicates that the frequency of “obesity” in the ninth document (29478) is “2” and the frequency of other keywords is “0”. .

As is clear from the above, the recording of the cell with the appearance frequency = 0 is excluded, and only the data with the appearance frequency = 1 or more is recorded, so that the data amount is greatly reduced.
Of course, for convenience of illustration, in the example of FIG. 41, the number of documents is limited to nine and the number of keywords is limited to five. However, even if the number of each case increases, the basic structure does not change and one line of data The appearance frequency can be expressed as In addition, since the cell of frequency = 0 is excluded from the recording, the redundancy can be effectively excluded.

  The total data generation unit 52 stores the total data 58 compressed in the one-line format as described above in the total data DB 54 in association with each keyword (S145).

Next, a procedure for generating the first relevance map 42 will be described with reference to the flowchart of FIG.
First, after receiving a search keyword from the user's client 36 (S151), the map generation unit 28 reads the aggregate data related to the search term from the aggregate data DB 54 (S152), and restores the cross tabulation table (S153). Although the document ID is missing in the restored cross tabulation table, there is no particular problem because the document ID is not necessary for subsequent processing.

Next, the map generation unit 28 performs principal component analysis on the search word and related word appearance frequency data for each document (S154), and extracts the first principal component value and the second principal component value as analysis results. (S155).
Next, the map generation unit 28 converts the first principal component value and the second principal component value into coordinate data on a predetermined plane (S156).
Next, the map generation unit 28 generates a tag including the search word and the related word (S157), and then generates a first relevance map in which each tag is arranged on the setting plane (S158).
The first relevance map 42 is distributed to the client 36 via the Web server 32 (S159).

As described above, since the total data is stored in the total data DB 54 in advance, it is possible to greatly reduce the processing time during the search.
In addition, as described above, since the appearance frequency data for each document related to the related words having the top 50 related degrees for each keyword is accumulated as total data, the relations to be displayed on the first relevance map 42 are as follows. Even when it is necessary to reduce the number of words, there is an advantage of being able to respond flexibly.

  For example, if the client 36 requests that the number of related words displayed be reduced to 10, the map generation unit 28 has a relevance level within the top 10 for the search term from the cross tabulation table restored based on the aggregated data. It is possible to quickly generate the first relevance map 42 by extracting only the appearance frequency data of the related words and executing the processes of S154 to S159 based on them.

It is a block diagram which shows the function structure of the 1st search system which concerns on this invention. It is a block diagram which shows the function structure of a keyword extraction part. It is a flowchart which shows a keyword extraction process. It is explanatory drawing which shows operation | movement of a character string frequency statistical filter. It is explanatory drawing which shows a mode that the morpheme index is formed in document DB. It is a flowchart which shows the related degree calculation process between keywords. It is explanatory drawing which shows an example of a keyword co-occurrence frequency table. It is explanatory drawing which shows the method of simplifying a relevance calculation process. It is explanatory drawing which shows a mode that a keyword relevance table is produced | generated based on a keyword combination frequency total table and a keyword frequency total table. It is a flowchart which shows the procedure of a 1st relevance map production | generation. It is a schematic diagram which shows the result of having extracted the related term based on the given search term. It is a graph which shows the result of having extracted the appearance frequency of the search word and each related word for every document. It is a graph which shows the result of having calculated the 1st principal component and the 2nd principal component of a search term and each related term. It is a graph which shows the result of having calculated the coordinate on a predetermined plane based on the 1st principal component value and 2nd principal component value of a search word and each related word. It is explanatory drawing which shows the tag of a search term and each related term. It is a figure which shows a 1st relevance map. It is a figure which shows a 1st relevance map. It is a figure which shows a 1st relevance map. It is explanatory drawing which shows a mode that the basis of the relevance degree between a search word and a specific related word is shown. It is a figure which shows a mode that duplication has arisen between tags on the 1st relevance map. It is a flowchart which shows the algorithm for eliminating the duplication between tags. It is explanatory drawing which shows the specific example of the duplication elimination between tags. It is explanatory drawing which shows the specific example of the duplication elimination between tags. It is explanatory drawing which shows the specific example of the duplication elimination between tags. It is explanatory drawing which shows the specific example of the duplication elimination between tags. It is explanatory drawing which shows the specific example of the duplication elimination between tags. It is explanatory drawing which shows the specific example of the duplication elimination between tags. It is explanatory drawing which shows the specific example of the duplication elimination between tags. It is a flowchart which shows the procedure of a 2nd relevance map production | generation. It is a graph which shows a mode that several keywords were extracted in order with the high degree of relevance with a search term. It is a graph which shows the result of having calculated X value and Y value of each keyword. It is a graph which shows the result of having calculated the coordinate on a predetermined plane based on X value and Y value of each keyword. It is a figure which shows a 2nd relevance map. It is a flowchart which shows the process sequence at the time of reflecting the temporal element of a related word on a 1st relevance map. It is a table which shows a mode that the appearance frequency of each related word is rearranged in order of the date of document data, and the weight according to the newness of a date is provided. It is a figure which shows the specific example which reflected the temporal element of the related word on the 1st relevance map. It is a flowchart which shows the process sequence at the time of reflecting the time element of a related word on a 2nd relevance degree map. It is a figure which shows the specific example which reflected the time element of the related word on the 2nd relevance map. It is a block diagram which shows the function structure of the 2nd search system which concerns on this invention. It is a flowchart which shows the process sequence of a total data generation part. It is explanatory drawing which shows the point of the compression process of the cross tabulation data by a tabulation data production | generation part. It is a flowchart which shows the production | generation procedure of the 1st relevance map by a 2nd search system.

Explanation of symbols

10 First search system
12 Document DB
14 Keyword extractor
14a Dependency Expression Extraction Filter
14b Delimiter extraction filter
14c String frequency statistics filter
14d TermExtract filter
14e Keyword recognition filter
16 Keyword DB
18 Relevance calculator
20 Keyword co-occurrence frequency table
22 Keyword combination frequency summation table
24 Keyword Frequency Summation Table
26 Keyword Relevance Table
28 Map generator
32 Web server
34 network
36 clients
40 tags
42 First relevance map
44 Blowout menu
46 Second relevance map
50 Second search system
52 Total data generator
54 Total data DB
56 Crosstabulation
58 Aggregated data

Claims (1)

A keyword co-occurrence frequency storage means for storing the results of counting the appearance frequencies of a plurality of keywords for each document data;
Keyword relevance storage means for storing relevance based on co-occurrence between keywords calculated using appearance frequency data in each document data of each keyword;
Means for referring to the keyword relevance degree storage means and extracting a predetermined number of other keywords as related words in descending order of relevance to the specific keyword;
Refer to the keyword co-occurrence frequency storage means, obtain the ID of the document data in which the specific keyword and its related word appear, and the appearance frequency of the specific keyword and related word in each document data, and appear for each document data A means of generating a crosstabulation table filled with frequencies;
Data compression means for compressing the cross tabulation table and storing it in the total data storage means in association with the specific keyword;
Means for extracting aggregate data of a specific keyword corresponding to the search word from the aggregate data storage means when a search word is input;
A means of restoring the crosstabulation table based on this aggregated data;
With reference to this cross tabulation table, the principal component analysis is performed on the multivariate data with the appearance frequency of the specific keyword and the related word for each document data as a variable, and the first principal component value and the second principal component value are obtained. Means for calculating;
Means for calculating the coordinate values of the specific keyword and the related word on a predetermined plane based on the first principal component value and the second principal component value;
Means for generating a relevance map in which tags expressing specific keywords and related words are arranged on the plane based on respective coordinate values;
A search system comprising means for outputting the relevance map ,
A search system, wherein the data compression means compresses a cross tabulation table related to a specific keyword into tabulation data in a one-line format according to the following conversion rule.
(1) A specific keyword and its related words are arranged in order, and a predetermined delimiter is placed between each keyword.
(2) Describe the numerical value representing the appearance frequency of each keyword in each document in association with the number representing the order of the keywords.
(3) When there are a plurality of numerical values representing the appearance frequency of each keyword in the same document, a predetermined delimiter is placed between them.
(4) When the appearance frequency is 0, the description in (2) is omitted.
(5) A predetermined delimiter is placed at the turn of the document.

Images