JPH08287081A - Retrieving device for data with similarity - Google Patents

Abstract

PURPOSE: To easily and exactly determine data high in similarity to all the plural data in a device for retrieving data with similarity. CONSTITUTION: The device is provided with a storing means (a) for data with a similarity which holds the set of data with data on a similarity between the respective two kinds of data, a display and input means (b) for a similarity retrieving expression which displays and inputs the similarity retrieving expression including a condition data set to be the condition of similarity retrieval, a similarity retrieving means (c) determining a retrieving data set by similarity retrieval by means of the condition data set, and a retrieving result holding means (d) which holds the retrieving data set obtained by the similarity retrieving means (c). Then, the retrieving result holding means (d) determines the retrieving data set, where a value obtained based on the similarity between two kinds of data which is held in the storing means (a) for data with a similarity, the condition data set inputted by the display and input means (b) for a similarity retrieving expression, the retrieving data set held by the retrieving result holding means (d) and a prescribed calculating expression for the similarity between the condition data set and data with a similarity, is over a prescribed threshold value regulating the limit of the similarity.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a device for retrieving data with similarities.

(Prior Art 1) When searching data desired by a user from a large amount of data, a search formula is created to narrow down the data. Each data has a search key like the following example.・ Attributes such as creation date and time, creator, etc. ・ Keywords ・ Words included in the text part of data (full-text search) ・ Category name including data The search expression specifies the search key as a logical operator such as and, or, not Created by combining with. For example, the search formula in the case where “light” and “fiber” are specified as keywords and the range of creation date and time is specified from January 1993 to August 1994 is as follows. FK = (optical OR fiber) AND (PD = 199
30101: 19940831) A logical operation according to a logical operator is performed on a set of data obtained by designating a search key to realize data narrowing.

(Prior Art 2) When a similarity is given between two terms of data, "a set of data having a high (or low) similarity to a certain data" can be obtained. Some “sets of data having high (low) similarity to certain data” are processed as “and, or, not” as in the case of the related art 1.
Data can be narrowed down by combining with logical operators such as. For example, the data can be narrowed down by (set of data having high similarity to dataA) AND (set of data having high similarity to dataB).

[0004]

[Problems to be Solved by the Invention] Five data (dat
Consider a case where the similarity between the two terms (a1-data5) is set as shown in FIG. The unit of similarity value is (%)
, The higher the value, the higher the similarity. Here, according to the prior art 2, “data1 and data2
“A set of data having a high degree of similarity with both” is obtained.
When a “set of data having a high degree of similarity with data1” is calculated with the threshold value set to 70 (%), {data3, data4}
Similarly, the “set of data having a high degree of similarity with data2” is {data3, data4}. Therefore, the “set of data having a high degree of similarity with both data1 and data2” is the logical product of the above two sets and is also {data3, data4}. However, in fact, it can be said that data4 is much higher in similarity to both data1 and data2 than data3. Also, the similarity between data4 and data5 is high. Therefore, it can be said that data5 is "data having a high degree of similarity to both data1 and data2" rather than data3. From this example, it is understood that the following problems remain in the related art 2. When the “set of data similar to a certain data” is obtained in advance as in the case of the conventional technique 2 and the narrowing is performed using the logical operation,
"Data having a high degree of similarity to all of the plurality of data" cannot be accurately obtained.

This problem arises because a "set of data similar to certain data" is determined in advance. In the above example, it is decided that “a set of data having a high degree of similarity with both data1 and data2” is determined, and then “a set of data with a high degree of similarity with both data1 and data2” includes data4, And data
The information that 5 has a strong similarity with data 4 can be known. It is difficult to know these pieces of information before it is decided to obtain “a set of data having a high degree of similarity with both data1 and data2”. That is, for all combinations of data, it is practically difficult to obtain a set of data having a high degree of similarity with the data in advance because the amount of calculation becomes large.

The present invention aims to solve the above problems. That is, an object of the present invention is to easily and accurately obtain data having a high degree of similarity with all of a plurality of data in a data retrieval apparatus having a degree of similarity.

[0007]

As shown in FIG. 1, a data retrieval apparatus with similarity according to the present invention stores data with similarity, which holds a set of data having data of similarity between two data. A means a, a similarity degree search expression display / input means b for displaying and inputting a similarity degree search expression including a condition data set which is a condition for the similarity degree search, and a similarity degree for performing a similarity degree search on the condition data set to obtain a search data set Degree search means c
And a search result holding unit d that holds the search data set obtained by the similarity search unit c. And
The similarity search means d is a data storage means a with similarity degree a.
The similarity between the two data held in, and the condition data set input by the similarity search expression display input means b,
A value obtained based on the search data set held in the search result holding means d and a predetermined calculation formula for obtaining the similarity between the condition data set and the data with similarity defines the similarity limit. A search data set that exceeds a predetermined threshold is obtained.

Further, according to one aspect of the present invention, in the above configuration, the condition data set input by the similarity search expression display / input means is a first condition data set for searching for data having a high similarity. And a second condition data set for searching data having a low degree of similarity.

[0009]

According to the present invention, a logical operator is not used when narrowing down based on the degree of similarity, and the degree of similarity is calculated after considering the degree of similarity in a wide range. And a set of data with a high degree of similarity (low) can be obtained.

[0010]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS As shown in FIG. 2, the structure of the embodiment is such that search expression display input means 1, similarity degree search expression display input means 2, data storage means 3 with similarity degree, logical operation search means 4, similarity degree search. A means 5, a logical operation search result display means 6, a similarity degree search result display means 7, and a search result identification storage means 8 are provided.
Hereinafter, each component will be described.

The search expression display input means 1 has a user interface for receiving a search expression using a logical operator from an input means (not shown) such as a keyboard and a mouse and displaying the received result on an output means such as a display. It is a program module that has.

The similarity search formula display / input unit 2 receives a search formula for performing a similarity search from an input unit such as a keyboard and a mouse, and outputs the received result to an output unit such as a display program having a user interface. It is a module.

The similarity-added data storage means 3 is a program module that has a key for retrieval and holds a data group in which similarity between two terms of data is held.

The logical operation search means 4 receives the search expression from the search expression display / input means 1, and, if necessary, the already retrieved data set from the search result identification storage means 8,
Search the data storage means 3 with similarity according to the search formula,
It is a program module that obtains a data set as a search result. Notify the search result identification storage means 8 of the search result,
Receives the search result identifier.

The similarity search means 5 receives the similarity search expression from the similarity search expression display / input means 2, receives the already retrieved data set from the search result identification storage means 8, and adds the similarity according to the similarity search expression. It is a program module that calculates the similarity from the similarity data in the data storage means 3 and obtains a data set as a search (calculation) result. The search result identification storage means 8 is notified of the search result, and the search result identifier is received.

The logical operation search result display means 6 is a program module having a user interface for receiving a data set as a search result and its identifier from the logical operation search means 4 and displaying the contents on an output means such as a display.

The similarity search result display means 7 is a program module having a user interface for receiving a data set as a search result and its identifier from the similarity search means 5 and displaying the contents on an output means such as a display.

The search result identification storage means 8 receives a data set as a search result from the logical operation search means 4 and the similarity search means 5, and assigns an identifier to the data set.
It is a program module that returns the identifier to the logical operation search means 4 or the similarity search means 5 that is the sender of the search result. The received search result (data set) is stored together with the identifier, and when the identifier is received from the logical operation search means 4 and the similarity degree search means 5, the corresponding search result (data set) is returned.

FIG. 4 shows the processing flow of the logical search and the similarity search in the embodiment having the above configuration.
At the time of logical search, the search formula display input means 1 first receives the search formula from the input means. At the time of the similarity search, the similarity search formula display / input unit 2 receives the similarity search formula.
(Step S1)

In the received search expression or similarity search expression,
It is checked whether or not the identifier corresponding to the already retrieved data set is included (step S2).

If the result of determination in step S2 is that an identifier is included, the logical operation retrieval means 4 or the similarity degree retrieval means 5 receives the data set corresponding to the identifier from the retrieval result identification storage means 8 and Is expanded into elements (step S3).

It is checked whether the received retrieval formula is a similarity retrieval formula (step S4).

When the received search formula is the similarity search formula, the similarity search means 5 calculates the similarity according to the search formula and obtains a set of data as a search result (step S5). The details will be described later with reference to FIG.

When it is determined in step S4 that the received search expression is not the similarity search expression, that is, the search expression is a logical search expression, the logical operation search means 4 performs a logical operation according to the search expression. Then, a data set is obtained as a search result (step S6).

The logical operation searching means 4 or the similarity searching means 5 sends a set of data obtained as a search result to the search result identification storage means 8 and receives an identifier corresponding to the data set (step S7).

The logical operation search result display means 6 or the similarity search result display means 7 displays the identifier corresponding to the set of data obtained as the search result (step S8).

Next, the similarity search of this embodiment will be described in more detail. [Storage Information of Similarity-Added Data Storage Unit 3] n pieces of data D = {d in the similarity-added data storage unit 3
, D2, ..., dn} are stored. Furthermore, arbitrary data dx
, And all the data d1, d2, ..., dn representing the respective similarities, that is, S (dx, d
1), S (dx, d2), ..., S (dx, dn)
It is assumed that (x = 1, 2, ..., N) is stored. However, 0 ≦ S (dx, di) ≦ 100 (x, i = 1, 2, ...
., N) S (dx, dx) = 100 (x = 1,2, ..., n) S (di, dj) = S (dj, di) (i, j = 1,
2, ..., N).

[Description of Similarity Search Formula] The description of the similarity search formula is H (A) .L (B). Here, A and B are concatenations of the identifiers already assigned by the search result identification storage means 8. Therefore, the similarity search means 5 can be developed as follows. A → {a1, a2, ..., ai}, B → {b1, b
2, ..., bj} axεD (x = 1, 2, ..., i), byεD (y =
1, 2, ..., j) H (A) and L (B) are data a1, a2 ,.
i has a high degree of similarity, and data b1, b2, ..., bj
Indicates that data having a low degree of similarity is searched.

[Retrieval Algorithm of Logical Operation Retrieval Means 4] The retrieval algorithm by the logical operation retrieval means 4 is the same as a general data retrieval algorithm for performing logical operations.

-[Search (Similarity Calculation) Algorithm of Similarity Search Unit 5] FIG. 5 shows a search (similarity calculation) algorithm by the similarity search unit 5. D = {d1, d2, ..., dn} at the start in the figure is all data stored in the data storage means 3 with similarity.
Also, the data sets {a1, a2, ..., ai}, {b
1, b2, ..., Bj} are assumed to have no data duplication in each set. However, both sets may contain the same data.

As shown in FIG. 5, in the search algorithm of the similarity search means 5, first, the similarity search means 5 receives the similarity search expression H (A) from the similarity search expression display input means 2.
-Receive L (B) (step 51).

The similarity search means 5 receives the data set corresponding to the identifier from the search result identification storage means 8 and develops it as follows (step 52). A → {a1, a2, ..., ai}, B → {b1, b
2, ..., bj} axεD (x = 1, 2, ..., i), byεD (y
= 1, 2, ..., j)

Variables R and P for the retrieval result are set to an empty set (step 53).

M (dx) = i + j V (dx) = S (dx, a1) + S (dx, a2) +,
..., + S (dx, ai) + (100-S (dx, b
1)) + (100-S (dx, b2)) +, ..., +
(100-S (dx, bj))} / M (dx) (where dxεD) is found and when V (dx) is larger than the threshold T, dx is added to the elements of the sets R and P. This operation is performed for all the elements of D (step S54).

A set obtained by removing the elements of the set of the logical product of R and D from the data set D is newly set as D (step S5).
5).

[V (dx) × M (dx) + {V (p1)
× S (dx, p1) + V (p2) × S (dx, p2)
+, ..., + V (pk) × S (dx, pk)} / 10
0] / (M (dx) + k) (where P = {p1, p
2, ..., pk}, dxεD), and this value is newly set as V (dx), and M (dx) + k
Is newly set as M (dx). When V (dx) is larger than the threshold value T, dx is added to the elements of the sets R and P. The above operation is performed for all the elements of D (step S56).

It is determined whether P is an empty set (step 5
7) If P is not an empty set, P is set as an empty set (step S58), and the process returns to step S55. If P is an empty set, data that is an element of R is output as a search result (step S59). .

This algorithm uses the "data a1, a2,
, Ai with a high degree of similarity between the two terms and data b1, b2, ..., bj with a low degree of similarity between the two terms ". a2 ...
, Ai has a high similarity between the two terms, and the data b
, B2, ..., bj, which has a low degree of similarity between two terms, recursively considers data having a high degree of similarity between two terms, and obtains the search result set R. . As a result, it is possible to accurately obtain data having a high degree of similarity with all of the plurality of data, which has been a problem of the conventional technique.

Screen images using concrete data examples of the embodiment are shown in FIGS. 6 to 10. The text input section 62 at the top of the logical search window 61 in FIG. 6 corresponds to the search expression display input means 1 of FIG. 2, and the text input section 65 at the top of the similarity search window 64 corresponds to the similarity search expression display input means 2. Further, the two list items 63 in the logical search window 61 correspond to the logical operation search result display means 6, and the two list items 66 in the similarity search window 64 correspond to the similarity search result display means 7. FIG. 6 shows a screen in an initial state before a search expression is input.

The operation contents of FIGS. 7 to 10 will be described below. In FIG. 7, enter a search expression in the text input section 62 at the top of the logical search window 61, and click the search button 7
By clicking 1, a search is performed by the logical operation search means 4 based on the specified search expression, and the result is displayed together with the identifier. In this example, the author is “Hiroshi Yamada” (OT = Hiroshi Yamada), and 1993
Registered in the year (PD = 19930101: 19931
231) Searching for data. The identifier “k1” is given to the data set obtained as the search result.

Similar to FIG. 7, FIG. 8 shows an example of performing a logical search, and the search condition is that the author is “Makoto Takahashi”.
And specifies the data registered in 1993. For the data set obtained as a search result,
The identifier "k2" is given.

Similar to FIG. 7, FIG. 9 shows an example of performing a logical search, and the search condition is that the author is “Yoshihiko Mori”.
And specifies the data registered in 1993. For the data set obtained as a search result,
The identifier "k3" is given.

In FIG. 10, the similarity search condition designated by the similarity search means 5 is entered by inputting the conditions for the similarity search in the text input section 65 at the top of the similarity search window 64 and clicking the search button 68. The search is performed based on, and the result is displayed together with the identifier. In this example, the data set designated by the identifier k1 and the identifier k2
Data that has a high degree of similarity with the data that belongs to the data set that is specified by, and that has a low degree of similarity with the data that belongs to the data set that has been specified by identifier k3 ("Hiroshi Yamada" and "Makoto Takahashi").
, Which has a high degree of similarity with the document written by, and has a low degree of similarity with the document written by "Yoshihiko Mori". An identifier k4 is given to the data set obtained as the search result.

In both the logical search window 61 and the similarity search window 64, it is possible to further narrow down using the identifier k4.

As described above, in the present embodiment, instead of using the logical operator when narrowing down by the similarity, the similarity calculation considering the similarity in a wide range is performed, and more accurately the “specified plurality of It is possible to obtain "a set of data having high (low) similarity to all the data". further,
By providing the user with both the means for narrowing down using a general logical operator and the means for performing the above similarity calculation,
It is possible to realize a device capable of finely specifying a search and improve the efficiency of a search from a large amount of data.

[0046]

As described above, according to the present invention, the similarity calculation is performed in consideration of the similarity in a wide range instead of using the logical operator when narrowing down by the similarity. A set of data having a high degree of similarity (or a low degree of similarity) with all of the data of.

[Brief description of drawings]

FIG. 1 is a block diagram showing a configuration of the present invention,

FIG. 2 is a block diagram showing a configuration of an embodiment of the present invention,

FIG. 3 is a diagram for explaining similarity search,

FIG. 4 is a diagram showing a processing flow of logical search and similarity search;

FIG. 5 is a diagram showing a search algorithm of the similarity search means 5;

FIG. 6 is a diagram (part 1) showing a screen image using a specific data example of the embodiment;

FIG. 7 is a diagram (part 2) showing a screen image using a specific data example of the embodiment;

FIG. 8 is a diagram (part 3) showing a screen image using a specific data example of the embodiment;

FIG. 9 is a diagram (part 4) showing a screen image using a specific data example of the embodiment;

FIG. 10 is a diagram (part 5) showing a screen image using a specific data example of the embodiment;

[Explanation of symbols]

a ... Similarity data storage means, b ... Similarity search expression display / input means, c ... Similarity search means, d ... Search result holding means.

Claims (2)

[Claims] 1. A data storage unit with similarity, which holds a set of data having similarity data between two pieces of data, and a similarity search formula including a condition data set which is a condition for similarity search. A similarity search expression display and input means, a similarity search means for performing a similarity search by the condition data set to obtain a search data set, and a search result holding means for holding the search data set obtained by the similarity search means The similarity search means, the two data similarity held in the data storage means with similarity, the condition data set input by the similarity search expression display input means, The values obtained based on the search data set held in the search result holding means and the predetermined calculation formula for obtaining the similarity between the condition data set and the data with similarity degree are similar to each other. Similarity with the data retrieval apparatus characterized by obtaining the search data set exceeds a predetermined threshold value defining the limit. 2. The condition data set input by the similarity search expression display / input means includes a first condition data set for searching data with high similarity and a first condition data set for searching data with low similarity. 2. A data search device with similarity, comprising two condition data sets.