JP2002251412A - Document retrieving device, method, and storage medium - Google Patents

Abstract

PROBLEM TO BE SOLVED: To reduce processing time in document retrieving processing, where multi-dimensional vector matching is used, even when a large amount of document data is retrieved. SOLUTION: Document data retrieving is conducted using a document database in which pairs, each composed of document data and a semantic vector representing its contents with a multi-dimensional vector, are stored. Document data stored in the document database is classified into groups according to similar semantic vectors, and the groups are stored in a group table. When a query described in a natural language statement is entered, a language analysis is made for the query to generate the semantic vector of the query (S21-S23). Based on the language analysis result, a group is selected from the group table (S24, S25). For document data belonging to the selected group, matching between the semantic vector stored in the selected group and the semantic vector of the query is performed to retrieve for a document and then the result is output (S26-S28).

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an apparatus and a method for retrieving documents by linguistic analysis and a medium storing the same.

[0002]

2. Description of the Related Art With the explosive spread of the Internet,
The number of personal computers (PCs) and personal digital assistants (PDAs) serving as entrances thereof has also increased, and communication using digitized documents using e-mail and the like is permeating our daily lives. Under such circumstances, the number of digitized documents is still increasing. For this reason, a technology for quickly searching for a necessary document from a huge amount of document data is increasingly regarded as important, and a document search system having an easier-to-use interface has appeared.

As this type of document search system, for example, there is a document search system using a semantic concept. This is because, in a document database, each document data is stored in a one-to-one correspondence with a multidimensional vector indicating a semantic concept of the content, and a simple question sentence using a natural language that a user uses everyday is stored. When input, the question sentence is linguistically analyzed, converted into a multidimensional vector representing the semantic concept, and searched by matching with the multidimensional vector in the document database. According to this method, a document that a user is searching for can be searched for using an interface using a natural language.

[0004]

However, multi-dimensional vector matching requires a large amount of calculation as compared with keyword matching or the like. Therefore, if the number of document data to be searched becomes enormous, a large amount of time is required for search processing. There was a problem that it would take.

[0005] In addition, in a search by matching of multidimensional vectors, there is a problem that a document having no relation at all is mixed in the search result in some cases, and the accuracy of the search result is not improved.

SUMMARY OF THE INVENTION The present invention has been made in view of the above-mentioned problems, and it is an object of the present invention to keep processing time of a large amount of document data short in a document search process using multidimensional vector matching. With the goal.

[0007]

A document search apparatus according to the present invention for achieving the above object has the following arrangement.
That is, storage means for storing document data and a semantic vector representing the contents thereof in a multidimensional vector in pairs, and classifying the document data stored in the storage means into groups based on the semantic vector. A managing unit for managing, a linguistic analysis of a natural language sentence input as a search condition, and a generating unit for generating a meaning vector of the search condition sentence; and a managing unit based on a result of the linguistic analysis. Selecting means for selecting a group to be searched, and for the document data belonging to the group selected by the selecting means, searching the document by matching the semantic vector stored in the storage means with the semantic vector of the search condition sentence. Search means.

Further, a document search method according to the present invention for achieving the above object provides a document search method using storage means for storing a pair of document data and a semantic vector expressing the contents of the document data in a multidimensional vector. A management step of classifying and managing the document data stored in the storage means into groups based on a semantic vector thereof, and a language analysis of a natural language sentence input as a search condition, and A generating step of generating a sentence semantic vector, a selecting step of selecting a group managed in the managing step based on the result of the linguistic analysis, and a document data belonging to the group selected in the selecting step, A search step of matching the meaning vector stored in the storage unit with the meaning vector of the search condition sentence to search for a document.

[0009]

Preferred embodiments of the present invention will be described below with reference to the accompanying drawings.

In the present embodiment, when registering document data, a multidimensional vector indicating the meaning of the document is created, and the document is automatically classified and managed in advance using the multidimensional vector according to the semantic concept. Then, when a word that matches this category appears in the user query at the time of search, first, the search target is narrowed down to this category, and multidimensional vector matching is performed. By narrowing down the search target, the accuracy of the search result is increased, and at the same time, the amount of calculation at the time of document search is reduced, and accurate and quick document search is executed. Hereinafter, the present embodiment will be described in detail.

FIG. 1 is a block diagram showing the configuration of a computer device 100 that executes a document search process according to this embodiment. In the configuration shown in FIG. 1, a CPU 101 is a microprocessor, which performs calculations, logical judgments, and the like for document search processing, and controls each component connected via a PCI bus 102. The PCI bus 102 transfers an address signal indicating a component to be controlled by the CPU 101, transfers a control signal of each component to be controlled by the CPU 101, and performs data transfer between components.

The ROM 103 is a fixed read-only memory. The ROM 103 stores a basic I / O program in the configuration of the present embodiment. Also, the RAM 104
Is a writable random access memory, which temporarily stores various data from each component and stores a program in which various processes in the present embodiment are described. The CPU 101 performs various processes based on this program. .

The DVDD 105 is a DVD drive.
Programs and data recorded on a DVD medium (DVD-MEDIA) 106 are stored in the DVD drive 10.
5 is loaded into the system. Also, DISK
Various data stored in the DVD drive 105
Through the DVD-MEDIA 106. The DVD-MEDIA 106 is specifically a DVD-ROM, DVD-RAM, DVD-R, DVD
-RW, DVD-VIDEO, DVD-AUDIO, etc., are a general term for DVD standard media. In this embodiment, the DVD-MEDIA 106 is used for reading and writing large amounts of data such as document data and related data, or programs.

INPUT C 108 is an input controller. Input signals sent from the keyboard (KB) 109 and the pointing device (PD) 110 are appropriately converted into appropriate signals by this controller,
The data is transmitted to the CPU 101 via the PCI bus 102.

KB109 is a character symbol input key such as an alphabet key, a hiragana key, a katakana key, and the like.
Various function keys such as a cursor movement key for instructing cursor movement are provided. PD 110 is
It is a pointing device such as a mouse or a trackball, and is used to indicate a cursor or a button on a display screen.

The DISK 107 is an external memory for storing data, programs, and the like. Data and programs are stored as needed, and the stored data and programs are called up when necessary according to keyboard instructions. The document database in the present embodiment is mainly mounted on the DIKS 107.

VIDEO 111 is a video controller. Data for display is stored here via the PCI bus 102, converted into a signal for display, and output to the display device (DISP) 112. DISP112
Displays a result of various processes, a state of the apparatus, a message for the user, and the like.

The DEVC 113 is a device controller. CPU 10 transmitted via PCI bus 102
In accordance with the instruction 1, the device connected to the controller is controlled, and signals and data output from the connected device are transmitted to the CPU 101 and the DISK 107 via the PCI bus 102 as appropriate. SCAN 114 is a scanner. In accordance with an instruction from the DEVC 113, the original set here is scanned by an optical method, an original image is read, and this is output to the DEVC 113.

An NI 115 is a network interface, and is a device for connecting the document search system of the present embodiment to an external system via a LAN, the Internet 116, or the like. The document search system according to the present embodiment can transmit and receive signals and data to and from an external system via this connection.

MIX 117 is a mixer. When data for audio output is sent here via the PCI bus 102, the MIX 117 synthesizes these signals, converts them into audio output signals, and outputs them to the speaker (SPK) 118. The SPK 118 outputs a processing result, a state of the apparatus, a message for the user, music, and the like by voice.

In the computer apparatus 100 according to the present embodiment having the above components, various processes are executed in response to various inputs from the keyboard 109 and the pointing device 110. That is, the keyboard 109
When an input signal is supplied from the CPU or the pointing device 110, an interrupt signal is sent to the CPU 101 via the INPUT C 108, and the CPU 101
Various control signals stored in memory 03 are read out, and various controls are performed according to the control signals.

In this embodiment, the computer is a basic I / O
When the CPU 101 executes the O program, the OS, and the document search processing program, it operates as a document search device. The basic I / O program is written in the ROM 103, and the OS is written in the DISK 107. Then, when the power supply of the present system is turned on, the disk 107 is turned on by the IPL (initial program loading) function in the basic I / O program.
S is read into the RAM 104, and O
The operation of S is started. Note that the document search processing program is a program code for causing the CPU 101 to implement the document search processing procedure shown in the flowcharts of FIGS.

FIG. 2 is a configuration diagram of the contents of the DVD-MEDIA 106 when the document search processing program and related data are recorded on the DVD-MEDIA 106. In the present embodiment, the document search processing program and related data are recorded on the DVD-MEDIA 106. As shown in the figure, in the head area of the DVD-MEDIA 106, volume information 201 and directory information 202 of the DVD-MEDIA are recorded.
A document search processing program (executable file) 203 of the present embodiment, which is the content of the D-MEDIA 106, and document search processing related data 204 are recorded.

FIG. 3 shows a computer 100 and a DVD-MEDIA 10 storing the document search processing program.
FIG. 6 is a schematic view of FIG. Document search processing program 203 and related data 2 recorded on DVD-MEIDA 106
04 can be loaded into the system through a DVD-MEDIA drive (DVDD) 105 as shown in FIG. That is, this DVD-MEDIA10
6 in the DVDD 105, the OS and the basic I / O
Under the control of the O program, the document search processing program and related data are read from the DVD-MEDIA 106 and loaded into the RAM 104 to be operable.

FIG. 4 shows that the present document search processing program is RA
8 shows a memory map in a state where the memory map is loaded into the M104 and becomes executable. When the computer device 100 starts up, R
Basic I / O loaded from OM103 or DISK107
An O program 411 and an OS 412 are stored. Further, the document search processing program 203 and its related data 204 loaded from the DVD-MEDIA 106 are stored in the document search processing program 413 and the related data 414, respectively.
Is stored as The work area 415 includes a semantic concept dictionary MDIC421, a search result buffer RBUF.
422, a result output number COUNT424, and a group table GTBL423.

FIG. 5 is a diagram for explaining an example of the configuration of the semantic concept dictionary MDIC in the present embodiment.

The semantic concept dictionary (MDI) in this embodiment
C) 421 includes a word ID 501 as shown in FIG.
It is composed of a list of a word notation 502 and a multidimensional vector 503 as a semantic expression. In FIG. 5, the rows of the table correspond to data for one word. Of these, word ID5
01 is a number that is uniquely assigned to each word in the document search system of the present embodiment in order to identify and manage each word. The word notation 502 is a character string representing the notation of each word. Also, the semantic vector 50
Numeral 3 represents a semantic concept of each word as a multidimensional vector, which is predetermined.

Each dimension of the semantic vector in the present embodiment is set by appropriately extracting words corresponding to the superordinate concept with reference to a noun thesaurus or the like. In addition, the semantic vector of each word is analyzed by analyzing a semantic sentence such as a Japanese language dictionary.
It is created by dropping into the synthesis of the concept selected as each dimension. However, such a method is general as a semantic process and is well known, and thus will not be described in detail here.

The semantic concept dictionary (MDIC) 421 has the word ID 501 or the word notation 502 according to the above configuration.
Is used as a key, and each stored information can be referred to. In the present embodiment, the semantic concept dictionary (MDIC) 421 is appropriately searched and referred to in each process described later.

FIG. 6 is a view for explaining a configuration example of the search result buffer (RBUF) in the present embodiment. In the search result buffer (RBUF) 422 in the present embodiment, among the data searched in the document search processing, the number of result outputs COUNT 424 is stored in ascending order of data having a high degree of semantic matching. Figure 5 shows COU
Search result buffer (RBUF) 42 when NT = 10
2 is a configuration example.

As shown in FIG. 6, one piece of search result data is composed of a rank 601, a semantic similarity 602, a semantic vector 603, a document ID 604, and a group ID 605. Among them, the order 601 is RBUF
The order of the degree of semantic similarity in the data stored in 422 is large. The semantic approximation degree 602 is a degree when matching is performed between a semantic vector 603 associated with each document data and a semantic vector created by linguistic analysis of a query, and in this embodiment, expressed as a percentage. Is stored. Note that the degree of matching between the two semantic vectors is determined mainly by calculating the angle between these two vectors.

The semantic vector 603 is obtained by language-analyzing text in each document data, extracting semantic information, and expressing it as a multidimensional vector. Each registered document is stored in a database in advance. This semantic vector 603 is usually obtained by decomposing a text using morphological analysis, extracting common nouns and proper nouns, retrieving these semantic vectors from a semantic concept dictionary (MDIC) 502, acquiring them, and then using syntax information. It is created by weighting and combining. Such a method is known in the field of natural language processing, and will not be described in detail here.

The document ID 604 is the ID of a document stored in the document database of the present embodiment, and actual document data can be read using this ID. The group ID 605 is the ID of a semantic approximation group to which each document belongs, and is used to refer to the group table (GTBL) 423,
The group information of this document can be obtained. The meaning approximation group will be described below with reference to FIG.

FIG. 7 is a view for explaining a configuration example of the group table (GTBL) in the present embodiment. As shown in FIG. 7, the group table (GTBL) 423 in the present embodiment includes a group ID 701, a group name 702, a data number 703, and a document ID list 70.
4 and a group semantic vector 705. Among them, the group ID 701 is a number uniquely assigned to a group created by semantically classifying document data registered in the document database in the present embodiment by a method described later. One group specified by this group ID constitutes one semantic approximation group. The group ID 701 corresponds to the group ID 605 of the RBUF 422. The group name 702 is a unique name given to each semantic approximate document group in the present embodiment. The process of determining the group name will be described later with reference to FIG. The number of data 703 is the number of document data belonging to this group.

The document ID list 704 is a list of IDs of document data belonging to this group. The group semantic vector 705 is a multidimensional vector in which the angle formed by all the semantic vectors of the document data belonging to the group is minimized. Group table (GTBL)
Reference numeral 423 is created by a semantic approximation group creation process described later, and is mainly used to narrow down search target data at the time of document search. FIG. 7 shows an example of the group table GTBL when news articles related to PCs are automatically classified.

FIG. 8 is a view for explaining an example of the configuration of one data structure stored in the document database in the present embodiment. As shown in FIG. 8, one data in the document database in the present embodiment is a document ID 80
1, a document data pointer 802, a keyword 803, and a meaning vector 804.

Here, the document ID 801 is the index number of the document data itself in the database as well as the index number of the data structure itself. The document ID 801 is the document ID 60 in the RBUF 422.
4. Document ID used in the document ID list in GTBL702. Also, the document data pointer 802
Is a pointer pointing to actual document data, and the actual document data can be accessed by using this pointer. The keyword 803 is a list of typical words appearing in the document data. The meaning vector 804 expresses the meaning of the document data by a multidimensional vector, and is the same as the meaning vector 603 stored in the RBUF 422. By adopting such a data structure, it is possible to perform a search at high speed without accessing actual document data.

The operation of the document search apparatus according to the present embodiment having the above configuration will be described in detail below.

FIG. 9 is a flowchart for explaining the entire document search process in this embodiment. Step S1
Is a user command input process for accepting a user's operation on the KB 109 or PD 110 and interpreting the operation as a command. Such a process is a general user interface process in a system using a PC or the like, and is well known and will not be described in detail here. Upon completion of the process, the process proceeds to a step S2.

Step S2 is a user command determination process for determining the command interpreted in step S1 and branching to various processes. When the user command indicates document registration, the process proceeds to step S3. If the user command indicates a document search, the process proceeds to step S4. If the user command indicates another process, the process proceeds to step S5.

In step S3, a document registration process for newly registering document data in a document database to be searched is performed. This processing will be described later using the flowchart of FIG. Upon completion of the document registration process, the process returns to the step S1. In step S4, a user query is received, and a document search process for searching document data in the database that matches the query is executed. This document search process will be described later using the flowchart of FIG. Upon completion of the process, the process returns to the step S1. In step S5, other processes such as customizing the display, changing various system settings, and registering user information are performed. These processes are common in such systems and are well known and will not be described in detail. Upon completion of the process, the process returns to the step S1.

In the system according to the present embodiment, when the process proceeds to step S1, various processing conditions such as the number of newly registered documents exceeding a certain value, an instruction to update group information, and the like are satisfied. In this case, the system is interrupted, and the process proceeds to step S6. Step S6
Performs a semantic approximation document group creation process of newly grouping documents registered in the document database of the present embodiment according to the meaning at that time and creating a group table GTBL as shown in FIG. . In addition,
Details of this processing will be described later using the flowchart of FIG. Upon completion of the semantic approximate document group creation process, the process returns to step S1 and returns from the interrupt.

FIG. 10 is a detailed flowchart of the document registration process in step S3 of FIG. By this processing, the document is registered in the document database in the form shown in FIG.

In step S11, a registered document specifying process for specifying a document to be registered by the user is executed. This processing may actually be a drag and drop of a document file using a GUI, or a designation of a text file storing a list of registered file names. This type of processing is very common as a user interface of the system as in the present embodiment, and will not be described in detail. Upon completion of the process, the process proceeds to a step S12.

In step S12, a main sentence analyzing process for extracting the main part of the text from the document data designated in step S11 and analyzing the language is performed. here,
The main part of the text is a part which has a high possibility that the meaning of the text is described, and is empirically extracted using the logic and layout structure of the document. Upon completion of the main sentence analysis process, the process proceeds to step S13.

In step S13, using the result of the language analysis (main sentence analysis processing) obtained in step S12,
A keyword extraction process for weighting and extracting keywords characteristically appearing in this document is performed. Such keyword extraction processing is generally performed in the field of language processing, and is well known and will not be described in detail. Upon completion of the keyword extraction process, the process proceeds to step S14.

In step S14, based on the keyword information obtained in step S13, a document meaning vector creation process for creating a multidimensional vector representing the meaning concept of the document data is performed. In the present embodiment, the creation of the document meaning vector is performed as follows. First, each keyword obtained in step S13 is converted into a semantic concept dictionary MDIC.
421 (FIG. 5) to obtain the meaning vector of each keyword. Next, step S13 is applied to each semantic vector.
After applying the weights obtained in the above, all the semantic vectors are synthesized. Upon completion of the process, the process proceeds to a step S15.

In step S15, the document data shown in FIG. 8 is created from the keyword obtained in step S13 and the meaning vector information obtained in step S14, and registered in the document database. The document ID is automatically assigned by the system so that duplication does not occur. At the same time, the actual document data itself is also registered in another area on the database. At this time, a document data pointer is obtained and registered in the database. Upon completion of the data registration process, the process advances to step S16.

In step S16, a semantic approximate document group to which the document to be registered belongs is determined using the semantic vector created in step S14. The determination of the semantic approximation document group is performed by matching the semantic vector of the document to be registered with the semantic vector 705 of each group in the group table (GTBL), and determining the most similar group. When the group to which the document belongs is determined, the document I of the determined group in the group table (GTBL) shown in FIG.
The document ID assigned in step S15 is stored in the D list. Upon completion of the process, the process proceeds to a step S17.

Step S17 is a registration result notifying process for notifying the user of the result of the registration process so far. For example, this is the display of a dialog box describing the result or the output of a warning sound, but this type of processing is common as a user interface and is well known and will not be described in detail. When the processing ends, the document registration processing ends.

FIG. 11 is a flowchart showing details of the document search process in step S4 shown in FIG.

In step S21, a query input process is executed to have the user input a query (question sentence) for searching for a document. The query is a simple sentence expressing the contents of the document that the user is looking for, and is input in a natural language such as “about a price drop of a PC”. Such processing is known in general search processing, and will not be described in detail here. Upon completion of the query input process, the process proceeds to a step S22.

In step S22, a query analysis process for language analysis of the query input in step S21 is executed. In this process, the query input in step S21 is subjected to linguistic processing such as morphological analysis and syntax analysis, and is converted into linguistic information in a format that can be easily used in subsequent processing. Such processing is known in general language processing, and will not be described in detail here. Upon completion of the query analysis process, the process proceeds to a step S23.

In step S23, a semantic vector creation process for creating a semantic vector of the query is performed using the linguistic information obtained as a result of step S22. In this semantic vector creation process, similar to the case of creating a semantic vector of a document (step S14), the semantic vector of a word appearing in a query is searched for and acquired from a semantic concept dictionary (MDIC) 421, and the syntax information is obtained. It is created by weighting and combining. Upon completion of the meaning vector creation processing, the process proceeds to step S24.

Step S24 is a process for checking whether or not the group name of the semantic approximation group stored in the group table (GTBL) 423 exists in the query using the language information obtained in step S22. . As a result of this processing, when one or more group names exist in the query, the process proceeds to step S25. If there is no group name, the process proceeds to step S26.

In step S25, as a result of step S24, documents to be searched are narrowed down to only documents belonging to a semantic approximation group having a group name existing in the query. By narrowing down the search targets at this point, the processing amount (computation amount) in the semantic matching processing to be performed later can be reduced, and the search can be speeded up. When the search document narrowing process is completed, the process proceeds to step S26.

In step S26, the meaning vector of the query obtained in step S23 is matched with the meaning vector in the document database to execute a meaning matching process for searching for the document that the user is looking for. Matching of the semantic vector is 2 as described above.
This is performed by calculating an angle between two multidimensional vectors. When documents to be searched are narrowed down in step S25, matching is performed only with the semantic vector of the documents to be searched. As a result of this processing, a matching result buffer (RBUF) 4
22 shows the number of result outputs COUNT in order from the data with the highest matching degree in the data in the database.
The search result data is stored by the number of. Such processing is known in general database search processing, and will not be described in detail here. Upon completion of the above semantic matching process, the process proceeds to step S27.

In step S27, the ID of the group to which each document data in the search result buffer (RBUF) output in step S26 belongs is obtained by searching the group table (GTBL) 423.
Execute a group ID acquisition process. The group ID obtained by this processing is stored in the search result buffer (RBUF).
422 is stored as a group ID 605. Upon completion of the process, the process advances to step S28.

Then, in step S28, step S
27 is a search result display process for outputting document data to a display based on the search result data in the search result buffer RBUF created by 27. This type of processing is generally performed in a display system, and is well known and will not be described in detail. When the processing ends, the document search processing ends.

FIG. 12 is a flowchart showing details of the semantic approximate document group creation processing shown in step S6 of FIG. In this process, the documents are automatically classified and managed by the semantic concept using the multidimensional vector of each document in advance, and as a result, the group table (GTBL) 423 shown in FIG. 7 is automatically generated.

In step S31, self-organizing map learning is performed using the semantic vectors stored in the document database in the present embodiment. This self-organizing map learning process is generally performed as a method of converting a set of multidimensional vector expressions into a two-dimensional map and classifying the set.
Although publicly known, the outline thereof will be briefly described with reference to FIG.

FIG. 13 is a diagram showing an example of learning of the self-organizing map. In FIG. 13, 1 is a unit U and 2 is a U
Is a neighboring unit Uc. As shown in FIG. 13, the self-organizing map is a square, and is composed of N × N (N is a positive integer) units. Each unit has a pattern vector of the same dimension as the semantic vector.

Learning of the self-organizing map is performed in the following procedure. First, all the pattern vectors of the units in the map are initialized to zero vectors. Next, for each semantic vector in the document database, a unit U having a pattern vector with the smallest angle is determined. Next, the pattern vector of U and its neighboring unit Uc is made closer to the semantic vector. At this time, as the distance between the unit U and the pattern vector of each unit is shorter, the degree of bringing the pattern vector closer to the meaning vector is increased. When the above operations other than the initialization are repeated a predetermined number of times, the vector pattern of each unit on the two-dimensional map converges to a certain constant value, and the self-organizing map learning ends. Upon completion of the self-organizing map learning process, the process proceeds to step S32.

In step S32, a process of arranging the semantic vector of each document registered in the document database in the unit on the self-organizing map learned in step S31 is performed. The meaning vector of each document is arranged in a unit having the smallest angle with the pattern vector, as in the case of map learning. When the processing is completed, step S
Go to 33.

In step S33, a process is performed in which the document data arranged on the two-dimensional map in step S32 is grouped by an 8-connection method, and a group is determined. Here, the 8-connection method is a method in which units adjacent diagonally vertically and horizontally on a two-dimensional map are grouped into the same group. FIG. 14 is a diagram illustrating an example in which a group is determined on a two-dimensional map by an 8-connection method. In FIG. 14, 3 is a group with group ID = 1, 4 is a group with group ID = 2, and 5 is a group I
D = 3 group, 6 is a group with group ID = 4. The numbers on the map indicate the number of arranged document data. Upon completion of the process, the process advances to step S34. In step S34, a multidimensional vector that minimizes the angle between all the semantic vectors of the document data belonging to the group determined in step S43 is determined, and this is set as the group semantic vector.

In step S35, a group name determining process for determining the group name of the semantically approximated document group determined in step S34 is performed. This processing is shown in FIG.
5 will be described later. Upon completion of the group name determination processing, the flow advances to step S36. Step S36 is equivalent to step S3
This is a process of creating a group table (GTBL) 423 with reference to the information of the group created in steps 3 to S35. When the processing ends, the semantic approximate document group creation processing ends. In this example, in the group table (GTBL) 423, group IDs are assigned from 1 in ascending order of the total number of document data. For example,
In FIG. 14, the number of matches for group 3 is 15 + 2
8 + 9 = 52, which is the highest, and as shown in FIG.
Is assigned. It is assumed that the search result in FIG. 6 has been discarded during the group creation processing.

FIG. 15 is a flowchart for explaining the details of the group name determination processing in step S34.
In FIG. 15, in step S41, matching is performed between the meaning vector 705 of the group for determining the group name and the meaning vector in the meaning concept dictionary (MDIC).
A plurality of words having a similarity greater than or equal to a certain threshold are searched as group name candidates of the group. The threshold used here is assumed to be determined in advance in the system of the present embodiment.

Next, in step S42, the position of the group name candidate word selected in step S41 on the thesaurus tree is determined, and the word located in the middle part of the thesaurus tree is selected as the group name. . This is because if a word located at a higher position (root part) in the thesaurus tree is selected as a group name, the semantic concept becomes too large and the concept of the group is blurred, which is avoided. Also,
Conversely, if a word located in the lower (cotyledon) part of the thesaurus tree is selected as a group name, there is a high possibility that document data that is not suitable for the group name will be mixed into the group because the semantic concept of the word is too narrow. Therefore, the word located in the cotyledon portion is excluded from the candidates.

In this processing, a word having a meaning vector having the highest similarity with the meaning vector 705 of the group among words located in the middle part of the thesaurus tree is determined as a group name. When the processing ends, the group name determination processing ends.

As described above, according to the present embodiment, when registering document data, a multidimensional vector indicating the semantic content of the document is created, and the document is automatically classified based on the semantic concept using this vector. Then, a word suitable for expressing a document group having a similar semantic concept is added as a group name. Then, when this word appears in a user query at the time of a search, the search result candidates are narrowed down to a semantic approximation group having this word as a group name, thereby reducing the amount of semantic vector matching processing and executing a quick search. It becomes possible. In particular, by selecting a word located in the middle of the thesaurus tree as a group name, it is possible to improve search accuracy.

The present invention is not limited to the above embodiment. For example, in the above-described embodiment, the PCI bus is used as the bus of the document search device.
It is possible to configure a completely similar document search device using an SA bus or a VL bus. In the above embodiment, the OS is stored in the DISK, but the OS is
Similar processing can be performed even if the data is stored on M.

In the above embodiment, the DVD-ME
Although the example in which the document search processing program and the related data are directly loaded from the DIA to the RAM and executed is described above, the document search processing program and the related data are temporarily stored (installed) in the DISK from the document from the DCD-MEDIA. It is also possible to load the document search processing program from the DISK into the RAM when operating this document search processing program.

In the above-described embodiment, the medium for storing the document search processing program is a DVD-MEDIA.
But CD-MEDIA, MO,
A DF, IC memory card, magneto-optical card, or the like may be used. Further, the document search processing program is stored in the ROM, and is configured to be a part of the memory map,
It is also possible to execute directly with CPU.

Further, an object of the present invention is to supply a storage medium (or a recording medium) recording software program codes for realizing the functions of the above-described embodiments to a system or an apparatus, and to provide a computer (a computer) of the system or the apparatus. Or a CPU or MPU) reads out and executes the program code stored in the storage medium,
It goes without saying that this is achieved. In this case, the program code itself read from the storage medium implements the functions of the above-described embodiment, and the storage medium storing the program code constitutes the present invention.
In addition, by executing the program code read by the computer, not only the functions of the above-described embodiments are realized, but also based on the instructions of the program code,
The operating system (OS) running on the computer performs part or all of the actual processing,
It goes without saying that a case where the function of the above-described embodiment is realized by the processing is also included.

Further, after the program code read from the storage medium is written into the memory provided in the function expansion card inserted into the computer or the function expansion unit connected to the computer, the program code is read based on the instruction of the program code. , The CPU provided in the function expansion card or the function expansion unit performs part or all of the actual processing,
It goes without saying that a case where the function of the above-described embodiment is realized by the processing is also included.

In addition, the present invention can be variously modified and implemented without departing from the gist thereof.

[0077]

As described above, according to the present invention,
In a document search process using multidimensional vector matching, even when a huge amount of document data is to be searched, the processing time can be kept short.

[Brief description of the drawings]

FIG. 1 is a block diagram illustrating a configuration of a computer device 100 that executes a document search process according to an embodiment.

FIG. 2 shows the document search processing program and related data in D
DVD-ME recorded on VD-MEDIA 106
FIG. 3 is a configuration diagram of the contents of a DIA.

FIG. 3 is a schematic diagram of a computer apparatus 100 and a DVD-MEDIA 106 in which a document search processing program is recorded.

FIG. 4 is a diagram showing a memory map in a state where the document search processing program is loaded into a RAM 104 and is executable.

FIG. 5 is a diagram illustrating a configuration example of a search result buffer (RBUF) 422 when COUNT = 10.

FIG. 6 shows a search result buffer (RBU) according to the embodiment;
It is a figure explaining the example of composition of F).

FIG. 7 illustrates a group table (GTB) according to the present embodiment.
It is a figure explaining the example of composition of L).

FIG. 8 is a diagram illustrating a configuration example of one data structure stored in a document database according to the present embodiment.

FIG. 9 is a flowchart illustrating an entire document search process according to the embodiment.

FIG. 10 is a detailed flowchart of a document registration process in step S3 of FIG. 9;

FIG. 11 is a flowchart showing details of a document search process in step S4 shown in FIG. 9;

FIG. 12 is a flowchart showing details of a semantic approximate document group creation process shown in step S6 of FIG. 9;

FIG. 13 is a diagram illustrating an example of learning of a self-organizing map.

FIG. 14 is a diagram showing an example in which a group is determined on a two-dimensional map by an 8-connection method in learning of a self-organizing map.

FIG. 15 is a flowchart illustrating details of a group name determination process in step S34.

Claims (16)

[Claims] A storage unit configured to store a pair of document data and a semantic vector expressing the contents of the document data in a multidimensional vector; A management means for classifying and managing, and a language analysis of a natural language sentence inputted as a search condition,
Generating means for generating a semantic vector of a search condition sentence; selecting means for selecting a group managed by the managing means based on a result of the linguistic analysis; and document data belonging to the group selected by the selecting means And a search unit for searching for a document by matching the meaning vector stored in the storage unit with the meaning vector of the search condition sentence. 2. The management unit generates a self-organizing map using a semantic vector of each document data stored by the storage unit, and generates a self-organizing map based on the semantic vector of each document data stored in the storage unit. 2. The document search device according to claim 1, wherein each document data is arranged on the self-organizing map, and one group is formed by the document data arranged nearby on the self-organizing map. . 3. The management means assigns a group name based on a word representing the group to each group, and the selection means matches a word included in the search sentence obtained as a result of the linguistic analysis processing. 3. The document search apparatus according to claim 1, wherein a group having a group name is selected. 4. The management means sets a semantic vector of the group based on a semantic vector of the document data belonging to the group, and sets a word acquired based on the semantic vector of the group as the group name. 4. The document search device according to claim 3, wherein: 5. The management unit acquires a plurality of words based on a semantic vector of the group, and selects a word located at an intermediate part on a thesaurus tree from among the acquired words as the group name. The document search apparatus according to claim 4, wherein the search is performed. 6. The document search apparatus according to claim 1, further comprising output means for outputting information indicating a predetermined number of pieces of document data in descending order of the degree of matching as a search result by the search means. . 7. The document search device according to claim 6, wherein the output unit adds information specifying a group to which the document data belongs to each of the document data and outputs the added document data. 8. A document search method using storage means for storing a pair of document data and a semantic vector representing the contents of the document data in a multidimensional vector, wherein the document data stored in the storage means is A management process of classifying and managing the groups based on the semantic vector, a language analysis of a natural language sentence input as a search condition,
A generating step of generating a semantic vector of a search condition sentence; a selecting step of selecting a group managed in the managing step based on a result of the linguistic analysis; a document data belonging to the group selected in the selecting step A search step of matching the meaning vector stored in the storage means with the meaning vector of the search condition sentence to search for a document. 9. The managing step includes: generating a self-organizing map using a semantic vector of each document data stored by the storage unit; and generating a self-organizing map based on the semantic vector of each document data stored in the storage unit. The document search method according to claim 8, wherein each document data is arranged on the self-organizing map, and one group is formed by the document data arranged in the vicinity on the self-organizing map. . 10. The management step assigns a group name by a word representing the group to each group, and the selection step matches a word included in the search sentence obtained as a result of the linguistic analysis processing. 10. The document search method according to claim 8, wherein a group having a group name is selected. 11. The managing step sets a semantic vector of the group based on the semantic vector of the document data belonging to the group, and sets a word acquired based on the semantic vector of the group as the group name. The document search method according to claim 10, wherein: 12. The management step includes acquiring a plurality of words based on a semantic vector of the group, and selecting a word located at an intermediate part on a thesaurus tree from among the acquired plurality of words as the group name. 12. The document search method according to claim 11, wherein the search is performed. 13. A search result in the search step,
9. The document search method according to claim 8, further comprising an output step of outputting information indicating a predetermined number of pieces of document data in descending order of the degree of matching. 14. The document search method according to claim 13, wherein, in the output step, information specifying a group to which the document data belongs is added to each of the document data and output. 15. A control program for causing a computer to implement the document search method according to claim 8. Description: 16. A storage medium for storing a control program for causing a computer to implement the document search method according to claim 8. Description: