JPH07105226A - Fuzzy thesaurus refining device - Google Patents

Abstract

PURPOSE:To automatically optimize a fuzzy thesaurus, generated by using the frequency of simultaneous appearance of two key words in the same document, into a fuzzy thesaurus having proper weight. CONSTITUTION:A fuzzy matrix F0 having respective weights between key words arrayed in a matrix is generated, and an OR matrix R0 of the fuzzy matrix F0 and unit matrixes U generated between key words is generated. Then an AND matrix Rj+1 of the AND matrix R0 and respective fuzzy matrixes Rj (j=0, 1, 2, 3...) is repeated until the obtained AND matrix matches an AND matrix obtained one arithmetic cycle before, and the optimized fuzzy thesaurus is generated from the AND matrixes at the point of time when a reachable matrix is obtained.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a fuzzy thesaurus refining device for improving a fuzzy thesaurus set in a fuzzy thesaurus used in a fuzzy information retrieval system to a more appropriate one.

[0002]

2. Description of the Related Art For example, a fuzzy information retrieval system is constructed as shown in FIG. A search database 2 for storing information to be searched for in the system bus 1, a search unit 3 having a fuzzy search program 4, a fuzzy thesaurus dictionary 5, a CRT display device 6, a keyboard 7,
The printer 8 and the like are connected.

As shown in FIG. 12, the search database 2 includes, for each document, a document ID (number), a document name (title), and a plurality of keywords for identifying the document in the search database 2. And document information is stored. The literature information includes an abstract, an author, a publisher, a date of publication, the number of pages, an issue and a volume in the case of a magazine.

The keyword is a word for searching what kind of information is described in the relevant document. Depending on the description content and length of the document, the keyword is 1 from the text (text).
0 to several tens are extracted and described.

In the fuzzy thesaurus dictionary 5, FIG.
As shown in FIG. 5, for each of the above-mentioned keywords, a keyword KY _j that is synonymous with its own keyword KY _i and a weight W _ij that indicates to what extent the keyword is synonymous are stored.

For example, there are a plurality of keywords KY ₂ , KY ₃ , KY ₇ , ... Synonymous with the keyword KY ₁ . And keyword KY ₁ and keyword KY ₂
The weight between and is W ₁₂ , and the weight between the keywords KY ₁ and KY ₃ is W ₁₃ . Similarly, a plurality of synonymous against Keywords KY ₂ Keyword KY
_{There are 1} , KY ₅ ... The weight between the keyword KY ₂ and the keyword KY ₁ is W ₂₁ .

The synonymous keywords are, for example, “computer” and “computer” or “computer” or “personal computer” as synonyms, and “processor”, “memory”, or “program” as subordinate terms. is there.

When the user searches for a document in which the information he or she wants is actually described using this fuzzy information search system, one or more keywords are keyed in from the keyboard 7. The search unit 3 searches the search database 2 with the input keyword KY _i and displays a list of document IDs, document names, etc. in which the corresponding keyword is set on the CRT display device 6.

Further, of the keywords KY _{j that} are similar to the corresponding keyword KY _i in the fuzzy thesaurus dictionary 5, the keyword KY _j having a higher weight W _ij is used for the search database 2
To search. The search result is displayed on the CRT display device 6.
The user can print out the search result displayed on the CRT display device 6 to the printer 8 as needed.

When a synonym registered in the fuzzy thesaurus dictionary 5 is used for retrieval, only synonyms whose weight W _ij is a certain value or more are extracted in consideration of the retrieval efficiency, but this value is used. The user can select and set it arbitrarily.

As described above, by using the fuzzy thesaurus dictionary 5, if a document in which desired information is described cannot be found by searching with the corresponding keyword, or if a wider range is searched, a synonym is used. By searching, more appropriate documents can be found.

Next, a method of registering the fuzzy thesaurus composed of synonyms and weights shown in FIG. 11 in the fuzzy thesaurus dictionary 5 will be described. Usually, one keyword KY
What kind of synonym KY _j exists for _i and what degree the weight W _ij indicating the degree of synonym have been determined by scholars and experts in the relevant field based on their experience and intuition.

However, since this method requires a lot of time and labor, Salton has developed a method for automatically generating synonyms and weights. This method is 2
It uses the idea that two terms (keywords) are close in meaning if they often appear together in a document.

The two keywords KY _i and KY _j
The frequencies V _ki and V _kj that appear independently in each document d _k and the co-occurrence frequency min (V _ki and V _kj ) that simultaneously appear are calculated.
Then, relevance for the keyword KY _j keyword KY _i (weight W _ij) is represented by equation (1).

W _ij = [Σ _k min (V _ki , V _kj )] / Σ _k V _ki (1) Therefore, the weight W _ij takes a value of 0 to 1, and the greater the value, the greater the association. , The smaller the value, the smaller the relevance. Then, when W _ij = 1 indicates that the keyword KY _i and the keyword KY _j are exactly the same, that is, the same word. When W _ij = 0, the keyword K
It indicates that Y _i and the keyword KY _j are terms (words) that have no relation to each other.

[0016] FIGS. 13 and 14 are flowcharts showing the actual calculation procedure of the weight W _ij for the keyword KY _j keywords KY _i described above. In order to calculate the weight W _ij , of course, in addition to the information of the search database 2 shown in FIG. 12, all documents (texts) of each document are required.

First, all the data (text) of the document designated by the document ID (= d _k ) of the database storing all the documents (text) of each document are read, and each keyword C appears in the same document. The number of times N _ki is calculated (P1).
Next, a keyword pair [AB] _ij composed of two keywords KY _i and KY _j is created from the plurality of keywords KY registered in this document (ID = d _K ). Then, the number of appearances of the smaller number of appearances N _ki and N _kj obtained for each of the keywords A and B is defined as the number of co-occurrence N _kij (P
2).

Next, the number of appearances N _i of each keyword C and the number of co-occurrence N _ij of the keyword pair [AB] are obtained over all the documents (k = 1.2.3 ...) (P3). In the loop of P4 to P5, the number of appearances N _i of the keyword A alone in the keyword pair [AB] _ij over all documents is calculated, and in the loop of P6 to P7, the keyword in the keyword pair [AB] _ij is searched. The number of occurrences N _j of B alone in all documents is calculated. Then, three types of appearance numbers, the co-occurrence number N _ij and the individual appearance numbers N _i and N _j , are added to the keyword pair [AB] _ij .

Next, in P8 to P9, the numbers of appearances are normalized and the corresponding appearance frequencies V _ij , V _i , V
Convert to _j . Then, the weight W _ij indicating the degree of similarity of the keyword A to the keyword B is obtained by the equation (2).

W _ij = V _ij / V _i (2) The weight W _ji indicating the degree of _synonym of the keyword B with respect to the keyword A is given by the equation (3).

W ji = V _ij / V _j (3) Thus, the weight W _ij .for one keyword pair [AB]. When W _ji is obtained, similarly, a keyword pair is created for all registered keywords, and a weight W _ij for the corresponding keyword pair is calculated. Then, as shown in FIG. 11, a fuzzy thesaurus in a format in which synonyms are added for each keyword is obtained and registered in the aforementioned fuzzy thesaurus dictionary 5.

[0022]

However, the fuzzy thesaurus obtained by the method shown in the equation (1) and the flowcharts of FIGS. 13 and 14 still has the following problems to be improved. In other words, the type of documents may inevitably be biased in a medium-scale system with at most tens of thousands of documents, rather than a large-scale search system with millions of documents used to create a fuzzy thesaurus. . In such a case, the keyword pair [AB]
weight W _ij of _ij from being calculated only with local relationship absolutely keyword pairs. As a result, the weight W _ij indicating the synonym of each keyword pair may vary.

For example, the keyword K of the keyword KY _i
The degree of synonym to Y _j should basically not be largely different from the degree of synonym of keyword KY _j to keyword KY _i . However, due to the above, the weight W _ij and the weight W _ij
It may be different from _ji . Also, if there are three or more related keywords, the similarities between the keywords should not differ greatly from each other.
Depending on the calculation results, there may be large differences.

As described above, when the weight W _ij indicating the synonym of each keyword pair varies, the optimum document in which the desired information is described is found even if the fuzzy information search system is searched. There is a high possibility that there will be problems such as being unable to do this or searching for documents that are unintended.

In order to eliminate such an inconvenience, each weight W _ij of the fuzzy thesaurus created by an experienced expert may be sequentially verified, but it requires a great deal of time and labor, When the number of registered keywords is large, it is almost impossible as a practical problem.

The present invention has been made in view of such circumstances, and a fuzzy thesaurus once created is represented by a fuzzy matrix, and the transitivity of the matrix is used to obtain variation in weight between each keyword pair. It is an object of the present invention to provide a fuzzy thesaurus scouring device that can improve the fuzzy thesaurus once created to a more optimal fuzzy thesaurus.

[0027]

In order to solve the above problems, in the fuzzy thesaurus refining device of the present invention, each keyword used in the fuzzy information retrieval system is formed by a corresponding keyword and a keyword similar to this keyword. A fuzzy thesaurus that stores a normalized weight having a value from 0 to 1 that indicates the synonym between the pair of keywords and the keywords, and the weight of each keyword pair stored in the fuzzy thesaurus is A fuzzy matrix creating means for creating a fuzzy matrix F0 in which each weight is arranged in a matrix between keywords using each keyword pair other than 0, and the same keyword for each keyword stored in the fuzzy thesaurus dictionary. The same keyword pair consisting of the combination of A unit matrix creating means for creating the unit matrix U forcibly set to 1, and a logical sum of the fuzzy matrix F0 created by the fuzzy matrix creating means and the unit matrix U created by the unit matrix creating means. a logical sum matrix creating means for creating a matrix R _0, each fuzzy matrix and logical sum matrix R ₀ created by the logical sum matrix creating means the logical sum matrix R ₀ as the initial fuzzy matrix R _{j (j} = 0 , 1,2, ...) and the logical product matrix operation means the logical product matrix R _{j + 1} are successively created while increasing the index j of the logical product matrix R _j at each operation cycle in the logical product matrix operation means ₊₁
And a fuzzy matrix R _j, and a fuzzy thesaurus generating means for converting the logical product matrix R _{j + 1} at the time when the fuzzy matrix R _j is detected by the fuzzy thesaurus. It is a thing.

[0028]

In the fuzzy thesaurus scouring device thus constructed, each fuzzy thesaurus created by the above-mentioned automatic creation method is stored in the fuzzy thesaurus using each keyword pair having a weight W other than 0. A fuzzy matrix F0 in which the weights are arranged in a matrix between the keywords is created.

On the other hand, as the special keyword pair of the keyword pair, the same keyword pair composed of a combination of the same keywords is defined. Since the keywords constituting the same keyword pair correspond to each other on a one-to-one basis, the weight W of the same keyword pair should be one. Therefore, a unit matrix U in which each diagonal weight W _ij (i = j) is set to 1 is created.

The unit matrix U and the logical sum matrix R ₀ of the fuzzy matrix F ₀ are set to 1 which is the maximum value of each weight W _ij (i = j) of each same keyword pair located on the diagonal line in the fuzzy matrix F 0. Will be the matrix

As described above, by creating the logical sum matrix R ₀ , the weight W consisting of a combination of the same keywords can be set to 1. Next, a logical product matrix R _{j + of} the logical sum matrix R ₀ and each fuzzy matrix R _j (j = 0,1,2, ...) Having this logical sum matrix R ₀ as an initial fuzzy matrix (j = 0) ₁ is sequentially created while increasing the index j, and when the created logical product matrix R _{j + 1} matches the logical product matrix R _j created one operation cycle before, that is, the logical product matrix R _{j +1} has reached the reachable matrix. Each element forming each column and each row in this logical product matrix R _{j + 1} becomes an optimized weight W _ij indicating the synonym of the keyword KY _i and the keyword KY _j .

[0032]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS An embodiment of the present invention will be described below with reference to the drawings. FIG. 1 is a block diagram showing a schematic configuration of a fuzzy thesaurus scouring device of the embodiment. This fuzzy thesaurus scouring device is incorporated in the fuzzy information retrieval system as described with reference to FIG. 10, and more specifically, a plurality of fuzzy thesaurus dictionaries 5 and a plurality of application programs built in the control unit are installed. It is composed of a task (processing unit).

In the fuzzy thesaurus dictionary 5, FIG.
As shown in FIG. 11, the fuzzy thesaurus automatically generated by the procedure shown in the flow charts of FIGS. 13 and 14 is stored. Specifically, for each keyword KY _i , a weight W _ij indicating the degree of synonym between keywords in a keyword pair composed of the corresponding keyword KY _i and a keyword KY _j that is similar to this keyword is stored. There is.

On the application program,
As shown in FIG. 1, a fuzzy matrix creation unit 11 that creates a fuzzy matrix F ₀ , an identity matrix creation unit 12 that creates a unit matrix U, and a logical sum matrix creation unit 1 that creates a logical sum matrix F _0.
3, the logical product matrix R _{j + 1} logical product matrix creation unit 14 sequentially creates a matrix comparing unit 15 for comparing the respective matrix R _{j + 1,} R _j,
A fuzzy thesaurus creation unit 16 that creates a finally optimized fuzzy thesaurus is formed.

Next, the specific operation of each section 11 to 16 will be described with reference to FIG.
This will be described with reference to the flowchart of FIG. First, in Q1,
A fuzzy thesaurus composed of keyword pairs KY _i , KY _j and weights W _ij stored in the fuzzy thesaurus dictionary 5 is read. Next, a keyword set S in which duplicate keywords are removed from the set of read fuzzy thesaurus
Is extracted (Q2). Then, the fuzzy matrix creation unit 11
Is activated, the weights W _ij between keywords excluding the fuzzy thesaurus with a weight W _ij of 0 out of each keyword pair included in the read keyword set S are arranged in a matrix as shown in FIG. 5, for example. Fuzzy matrix F
Create 0 (Q3).

Next, for each keyword in the keyword set S, the same keyword pair KY _i , KY _i consisting of a combination of the same keywords is created, and the weight W of the keyword pair is created.
Create an identity matrix U with _ii set to 1 forcibly (Q
4). In the created unit matrix U, for example, as shown in FIG. 6, among the elements forming the matrix, each weight W _ii located on the diagonal line is 1, and each weight W _ij located in other parts is 0. It is set.

At Q5, the logical sum matrix creation unit 13 is activated, and the fuzzy matrix F0 created previously by the fuzzy matrix creation unit 11 and the unit matrix U created by the identity matrix creation unit 12 are activated.
And a logical sum matrix R ₀ of Specifically, the elements at the same position forming each matrix are compared with each other, and the element having the larger value is adopted as the element at the corresponding position of the logical sum matrix R ₀ . Therefore, the created logical sum matrix R ₀ is a matrix in which each weight W on the diagonal in the fuzzy matrix F 0 is set to 1, which is the maximum value, as shown in FIG. 7, for example.

Next, in the loop of Q6 to Q10,
The logical product matrix calculation unit 14 is activated to repeatedly execute the logical product matrix calculation shown in Q7. That is, the above logical sum matrix R ₀
Is calculated as an initial fuzzy matrix, and a logical product matrix R _{j + 1} of the fuzzy matrix R _j and the logical sum matrix R ₀ is calculated.

R _{j + 1} = R _j × R ₀ ×: logical product (4) However, j is an index, the initial value is j = 0, and thereafter,
It increases in the order of j = 1,2,3 ...

In general, one element C _ij constituting the logical product matrix C obtained by performing the logical product operation of the matrix A and the matrix B is the respective elements A _i1 , A _i2 of the same row i at the corresponding position of the matrix A, A _i3 ,
, A _ij , ... _Are individually compared with the respective elements B _1j , B _2j , B _3j , ..., B _ij , ... Each element of the product) D ₁ ,
D ₂ , D ₃ , ... D _n . Select ...

D _n = min [A _in , B _nj ] Then, the selected elements D ₁ , D ₂ , D ₃ , ...,
The largest (logical sum) element D _{max of} D _n , ...
_Is an element C _ij at the corresponding position.

C _ij = max [D ₁ , D ₂ , D ₃ , ..., D _n , ...] The detailed operation of the logical product matrix operation described above will be described with reference to FIGS. 3 and 4.
Is shown in the flowchart. In the loop from T2 to T4 in the flowchart of FIG. 3, each value of the designated row X of the fuzzy matrix R _j corresponding to one element W _xy of the logical product matrix R _{j + 1} to be obtained is read. Next, in the loop from T5 to T7, one element W of the logical product matrix R _{j + 1 to be obtained}
Each value in the designated column Y of the logical sum matrix R ₀ corresponding to _xy is read.

In the loop from T8 to T9, the read values are compared with each other, and the smaller value (logical product) is selected and placed in the set B. And. The maximum value (logical sum) of the values forming the set B is adopted as the corresponding element W _xy of the logical product matrix R _{j + 1} .

Since one element W _xy forming the logical product matrix R _{j + 1} is obtained by the above procedure, T10, T11 and T
With a loop of ₁ , all the elements forming the logical sum matrix R _{j + 1} are obtained.

When the logical product matrix R _{j + 1} is calculated, the process returns to Q8 in FIG. In Q8, the matrix comparison unit 15 is activated to compare the calculated AND matrix R _{j + 1} with the fuzzy matrix R _j one operation cycle before. If they do not match, 1 is added to the index j, and the logical product matrix operation of Expression (4) is executed again. That is, obtaining the logical product matrix obtained in the previous operation cycle (R j _{+ 1)} and again the logical product matrix of the logical sum matrix _{R 0 (R j + 1 +} 1).

Then, in Q8, if the matrix R _{j + 1} obtained in the present operation cycle becomes equal to the matrix R _j obtained in the immediately previous operation cycle, the reachable arrival matrix shown in equation (5) is obtained. It can be determined that R * is generated.

[0047]

[Equation 1]

When the reachable arrival matrix R is generated, the fuzzy thesaurus generator 16 is activated in Q11 to remove the unit matrix U from the logical product matrix R _{j + 1} obtained in this operation cycle. , The finalized fuzzy matrix F
Get one. Specifically, each element set to 1 on the diagonal of the AND matrix R _{j + 1} this time is removed.

Then, in Q12, a fuzzy thesaurus in the format shown in FIG. 11 in which the synonymous keywords KY _j and the weights W _ij are arranged is created from the final fuzzy matrix F 1 for each keyword KY _i , and the fuzzy thesaurus is created. Store in dictionary 5.

Therefore, the fuzzy thesaurus stored in the fuzzy thesaurus dictionary 5 which is automatically generated by the conventional method is changed into the fuzzy thesaurus refined by the matrix operation utilizing the transitivity of the matrix shown in FIGS. To do.

According to the fuzzy thesaurus refining apparatus configured as described above, the fuzzy reachability matrix utilizing the transitivity described above is generated to thereby determine the degree of synonym between the keywords forming each keyword pair. The extreme variation of the weight W shown can be automatically corrected.

Therefore, in the fuzzy information retrieval system in which the fuzzy thesaurus dictionary 5 in which the optimized facilitation thesaurus is stored is incorporated, the intended information is described when the document is retrieved by designating one keyword. Probability that found documents are found (hit rate)
Can be greatly improved.

As a result, even if the fuzzy search / retrieval system incorporates the fuzzy thesaurus dictionary 5 automatically created with a medium-scale number of documents, by using the fuzzy thesaurus refining apparatus of the present invention, there is no bias. The reliability can be improved to the same level as that of a fuzzy search / retrieval system that employs a fuzzy thesaurus dictionary automatically created with a large number of documents.

FIG. 8 and FIG. 9 are diagrams showing the results of calculations carried out to confirm the effects of the present invention. FIG. 8 is a diagram showing a fuzzy matrix F0 created from a fuzzy thesaurus automatically created by the conventional method. This fuzzy example F0
"Compile""OS / V""Command" included in
The “software Kaihatsu” are related to each other, and the respective weights W _ij between the respective keywords are proper values, but the weights W _ij and W _ji between the “OS / V” and the “software Kaihatsu” are respectively. Only are [0] and [0]. this is,
It is considered that the documents used to create the fuzzy thesaurus were biased.

FIG. 9 shows a fuzzy matrix R _{J + 1} obtained by optimizing the fuzzy matrix F 0 of FIG. 8 by the fuzzy thesaurus refiner of the embodiment. As shown in the figure, it can be understood that the variation in each weight W of each keyword pair is reduced and optimized. For example, the above-mentioned "O
Each weight W between "S / V" and "Software Kaihatsu"
_It can be understood that _ij and W _ji are optimized to almost proper values from [0], [0] to [0.92], [0.6].

[0056]

As described above, according to the fuzzy thesaurus refining apparatus of the present invention, the fuzzy thesaurus once created is represented by a fuzzy matrix, and the transitivity of the matrix is used to weight the keyword pairs. Variation is reduced. Therefore, the fuzzy thesaurus once created can be improved to a more optimal fuzzy thesaurus, and the discovery probability (hit rate) of the intended document in the fuzzy thesaurus dictionary storing this fuzzy thesaurus or the built-in fuzzy information retrieval system can be improved. The reliability of the entire information retrieval system can be improved.

[Brief description of drawings]

FIG. 1 is a block diagram showing a schematic configuration of a fuzzy thesaurus scouring device according to an embodiment of the present invention.

FIG. 2 is a flowchart showing the operation of the apparatus of the embodiment.

FIG. 3 is a flow chart showing the operation of the apparatus of the same embodiment.

FIG. 4 is a flow chart showing the operation of the apparatus of the same embodiment.

FIG. 5 is a diagram showing a fuzzy matrix for explaining the operation of the apparatus of the embodiment.

FIG. 6 is a diagram showing a unit matrix for explaining the operation of the apparatus of the same embodiment.

FIG. 7 is a diagram showing a logical sum matrix for explaining the operation of the apparatus of the same embodiment.

FIG. 8 is a diagram showing an example of a fuzzy matrix created from a conventional fuzzy thesaurus.

9 is a diagram showing a fuzzy matrix obtained by optimizing the fuzzy matrix of FIG. 8 by an apparatus according to an embodiment.

FIG. 10 is a schematic configuration diagram of a general fuzzy information search system.

FIG. 11 is a diagram showing stored contents of a fuzzy thesaurus dictionary incorporated in the system.

FIG. 12 is a diagram showing storage contents of a search database incorporated in the system.

FIG. 13 is a flowchart showing a procedure for creating a general fuzzy thesaurus.

FIG. 14 is a flowchart showing a procedure for creating a general fuzzy thesaurus.

[Explanation of symbols]

5 ... Fuzzy thesaurus dictionary, 11 ... Fuzzy matrix creating unit, 12 ... Unit matrix creating unit, 13 ... Logical sum matrix creating unit,
14 ... Logical product matrix operation unit, 15 ... Matrix comparison unit, 16 ... Fuzzy thesaurus creation unit.

Claims (1)

[Claims] 1. A value between 0 and 1 indicating a synonym between a keyword pair formed by the corresponding keyword and a keyword similar to this keyword and the synonyms between the keywords for each keyword used in the fuzzy information retrieval system. Using a fuzzy thesaurus dictionary that stores the normalized weights that it has, and each keyword pair whose weight is other than 0 among each keyword pair stored in this fuzzy thesaurus dictionary,
A fuzzy matrix creating means for creating a fuzzy matrix F0 in which the respective weights are arranged in a matrix between the respective keywords, and the same keyword pair consisting of a combination of the same keywords is created for each of the keywords stored in the fuzzy thesaurus dictionary. And the unit matrix creating means for creating the unit matrix U in which the weight of the keyword pair is forcibly set to 1, and the fuzzy matrix F0 created by the fuzzy matrix creating means and the unit matrix creating means. The identified unit matrix U
Logical sum matrix R ₀ created ORed matrix creating means, at the logical sum matrix creating means for creating a logical sum matrix R ₀ between the
The logical sum matrix R ₀ initial fuzzy matrix to each fuzzy matrix _{R j (j = 0,1,2, ...} ) a logical product matrix R _{j + 1} and by creating sequentially while increasing the index j and A logical product matrix calculating means, a matrix comparing means for comparing whether or not the logical product matrix R _{j + 1} and the fuzzy matrix R _j in each calculation cycle in the logical product matrix calculating means, and the matrix comparing means A fuzzy thesaurus scouring device comprising: a fuzzy thesaurus creating means for converting a logical product matrix R _{j + 1} at the time when a match is detected by the fuzzy thesaurus.