JPH06348757A - Device and method for retrieving document - Google Patents

Abstract

PURPOSE:To utilize frequency information in the main body file of a keyword designated by a retrieving person for high-speed retrieval concerning document retrieval with the device and method for retrieving document so as to equivalently retrieve all the sentences at high speed by preparing a compressed file from the main body file. CONSTITUTION:This device is provided with a word division part 1, appearance frequency detection part 2 and frequency header preparation part 3, at the time of data base registration, the frequency information of a document is provided, a data managing data file (e) and a compressed data file (g) with frequency information are registered on a data base, and a document information acquisition part 11 is provided. Since only a part provided with the compressed file matched with frequency designated by a user is defined as a retrieval target out of the data base, all the sentences can be retrieved at high speed, the total frequency of the provided document between the documents of words excepting for the keyword or the number of documents to appear can be provided as the retrieved result, and retrieval noise can be reduced.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a document search method and apparatus for extracting a document containing a keyword designated by a user from a document database.

[0002]

2. Description of the Related Art With the increase in computer processing speed, a full-text search system using free words has come to be used as a document search method from index search. As a typical system, Japanese Patent Laid-Open No. H03-058311 full text search system and device (Juken reception number 31900148)
4), Proc. Of the 45th IPSJ National Convention (3)
Full-text search system by the hierarchical pre-search method described in 3-239-244 (Hitachi, Bib
liotheca / TS) and the full text database system (Matsushita, Kenzo-kun) described in Technical Research Report DE 90-34 of the Institute of Electrical, Information and Communication Engineers.

In the above-described hierarchical pre-search method, one document is created by eliminating the duplication of repeatedly appearing words from a character component table showing the existence of each character appearing in the document by 1-bit information, and a text file. It is composed of three files of compressed data and document body data. When a keyword is given, the character component table is searched first, and documents that do not include the character designated as the keyword are excluded from the search target.

Next, a word-level search is performed on the compressed text, further narrowing down is performed, and a search in the text that is the main data is performed only when necessary. For example, if only a character string is specified, only the compressed text is referenced, and the search ends without referring to the body data.
When the number of characters between two keywords is specified (neighborhood search), the text containing the two keywords is narrowed down by the compressed text search, and then the text is searched to check the character spacing between the two keywords.

In the full-text search system and apparatus, the character string of the registered document is divided into character types such as hiragana and kanji, and the character strings are mutually included in the divided partial character strings. And the set of partial character strings excluding other character strings is used as compressed data.

[0006]

In the conventional hierarchical pre-search method, (1) a document containing a large number of designated keywords and a document containing only one keyword are treated equally. (2) A document containing a word not intended by the user is also hit when a part of the word matches the specified keyword.
For example, if you specify "rice" as a keyword,
Documents including "comments" are also included in the search results. (3) When the number of hits becomes enormous, the user wants to narrow down the search results, but information other than the number of search results can be obtained. Regarding (2), the above-mentioned Kenzo-kun analyzes the text obtained as a search result as post-processing, but has the problem of significantly increasing the search time.

The object of the present invention is to include a word in a document by registering the document in a database by dividing the word using a word dictionary, calculating the word frequency, and reflecting the word frequency in the compressed text. By allowing the number of keywords to be included in the search condition, (2) reducing search noise without degrading the response time at the time of search, and further adding document information acquisition means in addition to the above means, (3) It is to provide a system that gives users a clue for narrowing down search results.

[0008]

In order to achieve the above object, the document retrieval apparatus of the present invention performs word division (morphological analysis) of text data by word division means and then repeats words by frequency detection means. Creates a compressed text with words excluding.

Next, the header creating means creates a frequency header indicating where in the compressed text the words of each frequency are located together with the compressed text, and the registration means registers the body data, the compressed data and the frequency header in the database. . Furthermore, the document search device of the present invention comprises a document information acquisition unit for acquiring information that constitutes a hit document,
The frequency information of the words included in the obtained search result set is obtained by the appearance frequency detecting means and presented to the user.

[0010]

The document retrieval apparatus according to the present invention first performs word division on the document to be registered, calculates the frequency of appearance of words in the document, and sorts the words in the order of appearance frequency. To create. Next, a frequency header indicating where in the word appearance frequency table the words having the respective frequencies or more are located is created (for example, an index file having information such that words having a frequency of 10 or more are up to the sixth compressed text). For each document, the frequency header and the entry word of the word appearance frequency table are registered as compressed text with frequency information, and the text data is registered as a full-text text file in the document database.

When searching the document database created by the above method, if a keyword appearance frequency is specified (for example, search for a document containing "rice" five times or more), the compressed text frequency file is referred to. Then, the position of the corresponding frequency is acquired as the reference end position of the compressed file, and then the search execution unit refers to the compressed text up to the end position of reading the compressed text to determine whether or not the keyword is included.

When the frequency condition is designated as described above, the search can be performed faster than the conventional method. In particular, when the frequency of the designated keyword is high, the reference amount of the compressed text is reduced, so that the high-speed search can be performed, so that the user can finely perform the search according to the request. It is also possible to sort the search results at high speed by reflecting the keywords specified at the time of display.

Further, the document search apparatus according to the present invention is arranged such that the words included in the search result set 1 are 30 in ascending order of the number of appearing documents.
It accepts user requests such as "show me personally".
At this time, in response to the user's request, each entry word, frequency, and number assigned to the document in the database from the compressed file corresponding to the hit document are written as one record in the document number-appearing frequency table. Next, with respect to the appearance frequency table with document numbers obtained from all the hit documents, the total appearance frequency between documents of each word and the number of appearance documents are obtained, and displayed according to the instruction of the user.

As described above, the user can grasp the outline of the entire hit document without referring to the text of the hit document individually, and can obtain an appropriate clue when performing a narrowed search.

[0015]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS A first embodiment of the present invention will be described below with reference to FIGS.
Will be described in more detail. FIG. 1 is a diagram showing an outline of a first embodiment of the present invention. The compressed data creation unit with frequency information is composed of a word division unit 1, an appearance frequency detection unit 2, and a header creation unit 3. First, the word division unit 1 divides the input document data a into words, and the word division result is registered in the word division table b. The appearance frequency detection unit 2 refers to the word division table b, calculates the appearance frequencies of the words, rearranges the words in the order of appearance frequencies, and registers the words in the word appearance frequency table c. The header creating unit 3 creates a cumulative frequency distribution table by referring to the word appearance frequency table c and registers it in the frequency distribution table d. The registration unit refers to the document data a, the word appearance frequency table c, and the frequency distribution table d to register the document data in the full text file f, and the frequency distribution table d and the word appearance frequency table c in the compressed data g with frequency information. To do. Further, each document number, the full text text start address, the start address of the word appearance frequency table e, and the start address of the frequency distribution table are registered in the management data file e.

The search command creating section 5 acquires the search instruction character string h input by the user, converts it into a search command,
Register in the search command table i. The search execution unit 6 refers to the search command table i and searches the document database including the management data file e, the full-text text data file f, and the compressed data file g with frequency information. The search result is registered in the search result table j. The result organizing unit 7 sorts the search result table j using the appearance frequency of the keyword as a key, and the sorted search result table k
Register with.

As is apparent from the figure, the word division unit 1, the appearance frequency detection unit 2, the frequency header creation unit 3, the registration unit 4, the search command creation unit 5, the search execution unit 6, and the search result arrangement unit 7 perform processing. Document data a, word division table b, word appearance frequency table c, frequency distribution table d, document database (management file e, full-text text file f, compressed file with frequency information g), search result table j, sorted The search result table k is a file (also called a table). As described above, according to this embodiment, each functional block is configured by the program logic. Therefore, it is possible to make LSI for each functional block unit,
It is possible to increase the speed of the document processing device.

FIG. 2 is a block diagram showing an overall hardware configuration diagram of the document retrieval apparatus in FIG. The input / output device 8 inputs data and displays various information. The processor 9 executes the processing in FIG. 1 based on the program. The storage device 10 stores various data and programs shown in FIG. The storage device 10 includes working areas a, b, which are memories for each execution process of the processor 9.
c, d, h, i, j, k, word division storage area 10
0, appearance frequency detection section storage area 200, header creation section storage area 300, registration section storage area 400, document database storage areas e, f, g, search command creation area 500, search execution section storage area 600, search result organization section It has a storage unit for the storage area 700. Storage device 10
Each program stored in is executed in the processor. When executing the operation, the input / output device 8 is used as necessary.

FIG. 3 is a PAD diagram (Problem Analysis Diagram) showing the processing procedure of the word division unit 1 in FIG. 1, and shows the processing until the document data a is acquired and stored in the word division table b. . Below, this process
It will be described with reference to the drawing. With reference to the document data a, the following processing is performed from the first document data to the last document data (step 101). First, data for one document is acquired (step 102), and the document data is divided into words (step 1).
03), the headline character string and the relative document number are stored in the word division table (step 104), and then the processing target is moved to the next document data (step 105). Through the above steps, the document data a shown in FIG. 4 is stored in the word division table b shown in FIG.

FIG. 4 shows an example of the document data a. FIG. 5 shows an example of the word division table b, which is composed of items of a document number b1 and a headword b2. FIG. 6 is a PAD diagram showing a processing procedure of the appearance frequency detection unit 2. It shows a process of acquiring data from the word division table b and storing the data in the word appearance frequency table c. From the first document data to the last document data of the document data of the word division table b (step 201), the following processing is performed.
First, data for one record is read from the heading to the end of the word division table (step 202) (step 203). Next, in step 204, records having the same heading are searched for in the word division file, and the frequency is calculated. In step 205, sorting is performed in descending order according to the appearance frequency of each entry word, and step 206
The word appearance frequency records sorted by are stored in the word appearance frequency table c. In step 207, the processing target is moved to the next record.

FIG. 7 shows an example of the appearance frequency table c created by the appearance frequency detection unit 2, which is composed of items of a headline character string c1, a frequency c2, and a document number c3. FIG. 8 is a PAD diagram showing the processing procedure of the frequency header creation unit 3, showing the processing from acquiring data from the word appearance frequency table c to storing the data in the frequency distribution table d.
The following processing is performed from the first document to the last document of the document data (step 301). In step 302, the variable count used to acquire the cumulative frequency is initialized. Next, from the first record to the last record of the word frequency table of each document (step 303), while records of the same frequency continue (step 304), the records are read (step 305) and 1 is added to the variable count ( In step 306, the processing target is moved to the next record (step 307). If the frequency of the record has changed in step 304 (if the frequency decreases because the data is arranged in descending order of word appearance frequency), step 30
In step 8, the frequency and the value of the variable count are written in the frequency distribution table. The value of the variable count indicates the value of cumulative frequency. Next, in step 309, the processing target is moved to the next record. In step 310, the processing target is moved to the next document.

FIG. 9 shows an example of the frequency distribution table d created by the header creating section, with a frequency index d1 and a cumulative frequency d.
2. Document number d3. This example is shown in Figure 7.
In the frequency distribution table created for the above example, in the document 1, one word with an appearance frequency of 7, one word with an appearance frequency of 0, one word with an appearance frequency of 5 and two words with a frequency of 4 It indicates that there is.

FIG. 10 is a PA showing the processing procedure of the registration unit 4.
FIG. 6D shows the process of acquiring the document data a, the word appearance frequency table c, and the frequency distribution table d and storing them in the files e, f, and g of the document database.
Hereinafter, they will be described in order. In step 401, registration data is acquired. The following processing is performed from the first data to the last data of the registration data (step 402). First, in step 403, the document number is acquired, then in step 404, the type of data is determined, and if the document data is a, the document data is stored in the full-text text data file (step 4
05), if it is the word appearance frequency table c, the header character string c1 and the frequency c2 are stored in the compressed data file g (step 406), and if it is the frequency distribution table d, the frequency header file as the frequency information of the compressed data file. The frequency d1 and the cumulative frequency d2 are registered in g (step 40).
7). The head address of the file stored in step 408 is written in the record column of the management data file e with the matching document number. In step 409, the processing target is moved to the next document.

FIG. 11 shows an example of a management data file registered by the registration unit 4. One record is composed of items of a document number e1, a compression start address e2, a frequency header start address e3, and a full text text data start address e4. There is. There are records for the number of documents registered in the database.

FIG. 12 is a diagram showing an example of the compressed data g with frequency information registered by the registration unit 4. For each document,
It is composed of a document number g1, a frequency header g2, and a compressed text g3. There are records for the number of documents registered in the database. FIG. 13 shows an example of a search character string input by the user on the input screen displayed via the input / output device 8.

FIG. 14 shows an example of a search character string input by the user on the input screen displayed via the input / output device 8.
In this example, the user specifies the frequency information. The keywords "network" and "computer" are each 2
This means searching for documents that contain more than one document.

FIG. 15 shows an example of a search character string input by the user on the input screen displayed via the input / output device 8.
In this example, the user performs a neighborhood search in units of words.

FIG. 16 is a PAD diagram showing the processing procedure of the search command creating section 5, which carries out the processing until the search command character string h is input and the search command is stored in the search command table i. In step 501, it is determined whether the input mode is the command mode or the natural language mode. If the input mode is the command mode, the grammar is checked in step 502, and if there is an error, the ending process is performed in step 503. When the input mode is determined to be natural language in step 501, semantic analysis is performed in step 504, and command generation is performed in step 505.

Next, the command type 506 is determined in a step, and if it is a command related to retrieval, it is stored in the retrieval command table (step 507), and if it is a command relating to document information acquisition, it is stored in the document information acquisition table (step 508). The commands created from the search instruction character strings shown in FIGS. 13 to 15 are all commands related to the search.

FIG. 17 shows an example of the search command table. The three commands are shown in FIGS. 13, 14 and 15, respectively.
It corresponds to the search instruction input of. FIG. 18 shows the search execution unit 6
It is a PAD figure which shows the process in. The search execution unit 6 searches the document databases e, f, and g according to the search command stored in the search command table i, and stores the result in the search result table j. The processing will be described below in order. First, in step 601, a search command is acquired from the search command table, the number of hits is initialized in step 602, and the hit document number storage array is initialized in step 603. Next, step 60
In step 4, a variable for counting hit documents is initialized, and in step 605, an array for storing the document numbers of hit candidates is initialized. The following processing is performed from the first document to the last document in the document database (step 606). First, from the first keyword to the last keyword specified by the user included in the search command (step 60).
7) The compressed data file with frequency information is searched (step 608). In step 609, the number of hit candidates is 0.
If it is larger, the process proceeds to step 610, and if the designated number of keywords is 1, the hit document confirmation process 611 is performed in step 611. If the number of keywords specified in the step is two or more, the process proceeds to a multi-keyword process (logical operation between keywords) in step 612, and if the value of the hit candidate count variable is larger than 0 in step 613, the hit document is found in step 611. Perform confirmation processing.

FIG. 19 is a PAD showing the procedure of the compressed data search processing 608 with frequency information. In the compressed data with frequency information search process, the compressed data with frequency information is searched for the presence or absence of a keyword specified by the user, and the document containing the keyword in the data is used as a candidate for a hit document, and the document number is used as the hit candidate document number. Stored in an array and use the hit candidate count variable to determine the number of hit candidate documents. At this time, if the frequency is specified by the user, only the portion corresponding to the frequency of the compressed file is searched. First, in step 60801, compressed data for one document is acquired. If the frequency is specified in step 60802, the flow advances to step 60803 to read the frequency header,
The reference start position of the compressed data is read in step 60804, and the reference end position is acquired in step 60805.
For example, if the number of keywords specified is greater than 3,
If the number is smaller than the number of documents, the search start position of the compressed file is the first and the search end position is the fourth in the document of document number 1 shown in FIG. 12, and the search start position of the compressed file is 3 in the document of document number 2. The search end position is the 10th item. If the frequency is not specified, the first headword of the compressed data is set to the reference start position of the compressed data in step 60806, and the last headword is set to the final reference position in step 60807. Next, from the reference start designated position of the compressed file to the reference end designated position (step 6080).
8) In step 60809, one keyword is read from the compressed file. In step 60810, the search keyword and the keyword are compared. If they match completely, the process proceeds to step 60811. The hit candidate count variable is incremented by 1 in step 60811, the document number is stored in the first column of the row indicated by the value of the hit candidate count variable in the hit candidate array in step 60812, and in step 60813 the frequency header is referenced to store the keyword The frequency is calculated, and this value is stored in the second column of the row indicated by the value of the hit candidate count variable of the hit candidate sequence. For example, when the keyword is “user”, and when there is a compressed file shown in FIG. 12, 1 indicating the document number 1 is stored in the first row and first column of the hit candidate sequence, and the first row and second column is stored. 3 indicates the frequency, and 2 indicates the document number in the 2nd row and 1st column of the hit candidate sequence.
5 is stored in the second row and the second column. In step 60814, the flow advances to step 6098 to interrupt the reference of the compressed text. If the search keyword and the entry word of the compressed file do not match in step 60810,
Proceeding to step 60818, the processing target is moved to the next heading of the compressed file.

Next, in step 60816, it is determined whether or not the number of hit candidates is one or more. If there are 0 hit candidates, the flow advances to step 60817 to end the search. Finally, in step 60818, the processing target is moved to the next keyword.

In the condensed file search method as described above, a hit candidate is obtained by having an exact match between the keyword and the heading. Therefore, as shown in FIG. 13, when the user specifies "rice" as the keyword, "comment" is given. Does not hit the containing document. Further, since the compressed files are stored according to the frequency and the frequency distribution table is prepared, the search can be performed at high speed based on the frequency.

FIG. 20 is a PAD showing the procedure of the hit document confirmation processing 611. The hit document confirmation process 611 is a process of confirming the hit candidate document as a hit document, and writes the search result in the search result table. In step 6111, the value of the hit candidate number array is set as the hit document number array, and in step 6112, the value of the hit candidate count variable is set as the number of hits. In step 6113, the hit document number, the start address of the full text, the frequency, and the number of bytes determined in advance from the position where the keyword in the body data first appears are stored as one record in the search result table.

FIG. 21 is a PAD showing the procedure of the multiple keyword processing 613. The multi-keyword process 613 is a process when the user specifies a plurality of keywords and a logical operation process is required between the hit candidate documents obtained for each keyword. In step 61301, a logical operation designated by the searcher is executed using the document number in the first column between the hit candidate sequences. In step 61302, the number of hit candidates is rewritten based on the logical operation result, and in step 61303, the hit candidate sequence is rewritten based on the logical operation result. In step 61304, the presence / absence of the neighborhood condition is determined. If not, the process proceeds to the hit document confirmation process 611. If a neighborhood condition is specified as shown in FIG. 15,
From the first line to the last line of the hit candidate sequence (step 61
305) Perform the following processing. First, step 61305
Refers to the document number stored in the first column of the hit candidate sequence. In step 61307, full-text text data of the document number is acquired. In step 61308, word division is performed by the word division unit. In step 61309, the word division table b in which the word division processing result is stored is referred to, and it is determined whether or not the positional relationship of the keywords matches the specified one. If they do not match, in step 61310, the document number is a hit candidate. Remove from array. Finally step 61311
Moves the processing target to the next array line with.

FIG. 22 is a diagram showing an example of the search result table, which is composed of items of a document number j1, a start address j2, a frequency j3, and a text portion j4.

FIG. 23 is a PAD diagram showing the processing procedure of the search result organizing unit 7. The search result organizing unit 7 reads the search result table j, sorts it based on the frequency information, and puts it in the sorted search result table j. Store. First, from the first record to the last record of the search result table (step 701), records are acquired in step 702,
In step 703, the sort key is acquired. If the input is a search result table, the frequency is used as the sort key. In step 704, sorting is performed according to the key, and step 705
The results are stored as a sorted search result table j.

FIG. 24 is shown in FIG.
An example in which the result obtained by sorting the inputs shown in is output to the input / output device 8 is shown. The second embodiment of the present invention will be described in detail below with reference to FIGS. 25 to 37. 1 to 35, the same reference numerals refer to the same things. FIG. 25 shows a second embodiment of the present invention. In the second embodiment, the frequency information of the words included in the document obtained as the search result from the compressed data file created in the first embodiment is presented according to the user's designation.

The second embodiment of the present invention will be described in detail below with reference to FIGS. 25 to 37. 1 to 35, the same reference numerals refer to the same things. FIG. 25 shows a second embodiment of the present invention. In the second embodiment, the frequency information of the words included in the document obtained as the search result from the compressed data file created in the first embodiment is presented according to the user's designation.

FIG. 25 is a functional diagram showing the outline of the second embodiment. The command creation unit 5 refers to the document information acquisition instruction input p from the user via the interface control unit 20 and stores the document information acquisition command creation table q.
The document information acquisition unit 11 uses the document information acquisition command table q
Information is acquired from each file of the document database by referring to the search result table j. When the user has designated the sentence in which the keyword appears as the target of information acquisition, the full text is acquired from the full text file f, the document in which the keyword appears is cut out, and the result is the text data. Store in table r. When the user hits and the entire document is designated as the information acquisition target, the compressed data file g is converted into the word appearance frequency table format and stored in the word appearance frequency table c. The data stored in the text data table r is word-divided by the word division unit 1, and the result is stored in the word division table b.
The appearance frequency detection unit 2 refers to the word division table b, finds the frequency of the word appearing in each sentence in the document, and stores it in the appearance frequency table s with the number of documents. The document number detection unit 12 refers to the word appearance frequency table c to obtain the total frequency between documents of each word and the number of appearing documents. The document output control unit 13 refers to the document information acquisition command table q, extracts information specified by the user from the document number-appearance frequency table s, stores the information in the document information table t, and the result is the interface control unit 20. Will be presented via. When a record of the presented document information table is selected by an input device such as a mouse or a touch panel, it is sent as a narrowing-down instruction u to the command creating unit via the interface control unit 20, and the command creating unit refers to the document information table t. A command is created and stored in the search command table g. In the case of a narrowing down instruction, a narrowing down flag is set. The search execution unit 6 refers to the search command table g and performs a search. When the narrow-down flag is set, the search result table j is referred to and the hit document is searched.

FIG. 26 is a block diagram showing an overall hardware configuration diagram of the document retrieval apparatus in FIG. The input / output device 8 inputs data and displays various information. The processor 9 executes the processing in FIG. 24 based on the program. The storage device 10 stores various data and programs shown in FIG. Furthermore, the storage device 1
0 is a working area b, c, h, j, p, q, r which is a memory for each execution process of the processor 9. s, t, word division part storage area 100, appearance frequency detection part storage area 2
00, document database storage areas e, f, g, search command creation area 500, search execution unit storage area 60
0, document information acquisition unit storage area 1100, document number detection unit storage area 1200, document information output control unit storage area 1
300, has a storage unit of the interface control unit storage area. Each program stored in the storage device 100 is executed by the processor. When executing the operation, the input / output device 8 is used as necessary.

FIG. 27 shows a document information acquisition instruction input example p. FIG. 28 is a document information acquisition instruction input example p. FIG. 29
Is a document information acquisition instruction input example p shown in FIGS. 27 and 28.
Is input to the command creation information acquisition konamm component 5,
It is an example output to the document information acquisition command table q.

FIG. 30 is a PAD diagram showing an outline of processing in the document information acquisition unit 11. Document information acquisition 11 is shown in FIG.
The document table q in 5 is referred to, information is obtained from the search result table j and the files e, f, and g of the document database, and the process of storing the word frequency data in the word frequency table c is performed.

First, from the first document to the last document of the search result (step 1101), the document number is acquired from the search result table c (step 1102). Next, in step 1103, it is determined whether the target specified in the document information acquisition command table q is a frequency or a text, and if it is a frequency, the process proceeds to step 1104, and whether the frequency is designated as an absolute value. , It is determined whether the keyword is used in the search. If the frequency is designated by a keyword, the frequency-compressed data e is referenced to refer to the frequency of the keyword used in the search in each document (step 11).
05). In step 1106, the compressed data read end position is acquired from the management data file e. Next, from the beginning of the compressed data to the read end position (step 110
In 7), the word is read (step 1108), the word frequency is acquired (step 1109), and in 1110, the document number, the headword, and the frequency are written as one record in the appearance frequency table. If the information acquisition target is text in step 1103, the text is read from the full-text file in step 1111 and the first keyword to the last keyword appearing in the text (step 1112), and the sentence containing the keyword in step 1113 ( The phrase) is acquired from the phrase, and the document number and the sentence are stored in the text data table r in step 1114. This text data is word-divided by the word division unit 1,
It is stored in the word division table b, and next, the appearance frequency detection unit 2
The appearance frequency is calculated by, and the result is stored in the word appearance frequency table c.

FIG. 31 shows an example of the text data table r, which is composed of a document number r1 and a text r2. Figure 3
2 is a PAD showing an outline of processing in the document number detection unit 12
It is a figure. The document number detection unit 12 refers to the word appearance frequency table c to obtain the total appearance frequency of the entry word and the number of appearance documents, and stores the result in the appearance frequency table with document number s. From the first record to the last record of the word frequency table c (step 1201), step 120
Search for records that have the same headword in step 2, then step 1
In 203, the number of documents having the same headword is counted, in step 1204, the total number of frequencies between documents of the headline is counted, and in step 1205, the headline, the total frequency, and the number of documents are set as one record in the appearance frequency table with the number of documents. Store. The record retrieved in step 1206 is excluded from the subsequent processing targets, and the processing target is moved to the next record in step 1207.

FIG. 33 shows an example of an appearance frequency table with the number of documents, which is composed of three items: headword s1, total frequency s2, and number of documents s3. FIG. 34 is a PAD diagram showing the outline of the processing in the document output control unit 13, and the appearance frequency table s with the number of documents.
Is stored in the document information table t according to the document information acquisition command table q. Step 1
In step 301, the appearance frequency table with the number of documents is read, in step 1302 a sort key is acquired from the document information acquisition command table, and in step 1303 sorting is performed.
If the document information acquisition instruction p instructs to display the words in the document number order, the document information is sorted in the document number order, and if the document information acquisition instruction p instructs to display the words in the total frequency order, the total frequency is used as a key for sorting.
In step 1304, the number of words to be displayed is calculated from the document information acquisition command table, and in step 1305, step 1
The number of headwords obtained in 304, the total frequency, and the number of documents are stored in the document information storage table t.

FIG. 35 shows an example in which the contents of the document information table t are displayed on the input device 8. In this example, the user can specify a keyword displayed via a pointing device such as a touch panel or a mouse as a keyword used for narrowing down. The keyword specified by the user is sent to the interface control unit 20 as a narrowing-down instruction.

FIG. 36 is a PAD diagram showing the outline of the processing of the command creating section of the second embodiment. The command is created by referring to the search instruction character string h or the narrowing down instruction u via the interface control section 20. The process until the command is stored in the search command table 6 is shown. Step 5
Steps 10 to 511 are different from the processing of the search command creation unit of the first embodiment shown in FIG. The other same steps represent the same process. That is, step 510
The input mode is determined with, and if it is a command, the process proceeds to step 502 to check the grammar. If it is a natural language, the process proceeds to step 504 to perform semantic analysis. Step 510
If the input mode is a narrow-down instruction in step, step 511
Then, the narrowing-down flag is set (this processing is not in the first embodiment), and a command is created in step 505. From step 506 onward, the same processing as in the first embodiment shown in FIG. 16 is performed.

FIG. 37 shows the outline of the processing of the search execution unit P.
19 is a modified example of the processing of the search execution unit of the first embodiment shown in FIG. 18 in the AD diagram. The narrowing-down processing shown in steps 620 to 622 is different from the first embodiment, and the processing having the same step shows the same processing.

First, in step 601, a search command is obtained by referring to the search command table g, and in step 60
From step 2 to step 605, variables are initialized as in the first embodiment. On / o of the narrowing flag in step 620
ff is determined, and if it is off, the processes from step 607 to step 613 are performed for the first document to the last document in the document database. When the narrowing-down flag is on in step 620, the hit document is referenced from the search result table j, and the processing from step 613 to step 620 is performed for the first to last hits. In step 622, the narrowing-down flag is set to off. .

[0051]

According to the document search apparatus of the present invention, the appearance frequency information in the document of the keyword designated by the user can be reflected at high speed, and the higher the frequency of appearance of the keyword designated by the searcher, the longer the retrieval time. Can be shortened. Furthermore, since the total appearance frequency information of words included in the search result set and the number-of-appearing-documents information are calculated at high speed, the user obtains keyword information for narrowing down the search results,
There is an effect that it is possible to easily perform a narrowed search.

[Brief description of drawings]

FIG. 1 is a functional block diagram showing a first embodiment of a document search device according to the present invention.

FIG. 2 is a block diagram showing a hardware configuration showing an embodiment of hardware of the document search device in FIG.

FIG. 3 is a PAD diagram of the word division program in FIG.

FIG. 4 is an example of document data in FIG.

5 is an example of a word division table in FIG.

6 is a PAD of the appearance frequency detection program in FIG.
Fig.

7 is an example of a word appearance frequency table in FIG.

FIG. 8 is a PA of the frequency header creation program in FIG.
Figure D.

9 is an example of a frequency-minute table in FIG.

10 is a PAD diagram of the registration program in FIG.

11 is an example of a management data file in FIG.

12 is an example of a compressed data file with frequency information in FIG.

13 is an example of the search instruction character string in FIG. 1 input via the input / output device in FIG.

FIG. 14 is an example of the search instruction character string in FIG. 1 input via the input / output device in FIG.

15 is an example of the search instruction character string in FIG. 1 input via the input / output device in FIG.

16 is a PAD diagram of the search command creation program in FIG.

17 is an example of a search command table in FIG.

18 is a PAD of the search execution program in FIG.
Fig.

FIG. 19 is a PAD diagram of the compressed data search program in FIG.

FIG. 20 is a PAD diagram of the hit document confirmation program in FIG.

FIG. 21 is a PAD diagram of the multi-keyword program in FIG. 18.

22 is an example of a search result table in FIG.

FIG. 23 is a PA of the search result arrangement program in FIG.
Figure D.

FIG. 24 is an example of outputting the sorted search results in FIG. 1 to an input / output device.

FIG. 25 is a functional block diagram showing a second embodiment of the document search device according to the present invention.

FIG. 26 is a block diagram showing a hardware configuration showing an embodiment of hardware of the document search device in FIG. 25.

27 is an example of inputting a document information output instruction in FIG. 25 input through the input / output device in FIG.

28 is an example of inputting a document information output instruction in FIG. 25 input through the input / output device in FIG.

FIG. 29 is an example of a document information acquisition command table in FIG. 25.

[FIG. 30] P of the document information acquisition program in FIG.
AD diagram.

FIG. 31 is an example of text data in FIG. 25.

FIG. 32 is a PA of the document number detection program in FIG.
Figure D.

FIG. 33 is an example of an appearance frequency table with the number of documents in FIG. 25.

34 is a PAD diagram of the document information output control program in FIG. 25.

FIG. 35 is an example of displaying the document information table in FIG. 25 on an input / output device.

FIG. 36 is a PAD diagram of the search command creation program in FIG. 25.

FIG. 37 is a PAD of the search execution program in FIG.
Fig.

[Explanation of symbols]

1 ... Word division unit, 2 ... Appearance frequency detection unit, 3 ... Frequency header creation unit, 4 ... Registration unit, 5 ... Command creation unit, 6 ... Search execution unit, 7 ... Search result arrangement unit, 11 ... Document information acquisition unit 1
2 ... Document number detection unit, 13 ... Document information output control unit, 20 ...
Interface control unit, a ... Document data, b ... Word division table, c ... Word appearance frequency table, d ... Frequency distribution table, e ... Management data file, f ... Full text file, g ... Compressed data file with frequency information, p ... document information acquisition instruction, q ... document information acquisition command table,
s ... Appearance frequency table with document information, t ... Document information table, u ... Instruction to narrow down.

 ─────────────────────────────────────────────────── ─── Continuation of the front page (72) Inventor Satoshi Asagawa 5030 Totsuka-cho, Totsuka-ku, Yokohama-shi, Kanagawa Prefecture Hitachi Ltd. Software Development Division

Claims (5)

[Claims] 1. A device for accumulating a large number of documents for the purpose of retrieval, comprising means for dividing each document data into words, means for calculating the appearance frequency of the divided words, and the frequency information in descending order. And a means for accumulating the cumulative frequency distribution information in the document database, and a means for executing a search with reference to the accumulated cumulative frequency distribution information. Document retrieval device. 2. A document retrieval apparatus according to claim 1, wherein the document appearance information includes the total appearance frequency of words included in all document data matching the search condition and the total number of appearance documents for each word. A document characterized by comprising document information acquisition instruction means for obtaining a user's instruction regarding the number of appearing documents, and a document information output control section for selecting a word that matches the instruction of the user by referring to the document information. Search device. 3. A document retrieval apparatus, wherein the document information acquisition means according to claim 2 acquires information only in a sentence containing a keyword or a paragraph containing a keyword in document data. 4. A document search characterized in that the search execution means according to claim 1 sends the contents of a full-text text file to be searched to a word division unit and refers to the contents of a word division table which is the result of word division. apparatus. 5. A document retrieval method using a computer divides each document data into words, calculates the appearance frequency of the divided words, rearranges the frequency information in descending order, and obtains the cumulative frequency distribution information of the words. A document search method characterized by calculating, accumulating the cumulative frequency distribution information in a document database, and executing a search by referring to the accumulated cumulative frequency distribution information.