JPH10105559A - Information processor - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a function, with which retrieval accuracy and prediction accuracy can be improved by evaluating, storing and utilizing the retrieved result and the predicted result and the retrieval skill of users can be shared by sharing the stored evaluated result among plural users, concerning an information processor for performing similar data retrieval and prediction while using similar data. SOLUTION: The similar data are retrieved by a similar data retriever 105, and the similarity degree of retrieved data is evaluated by a similarity degree evaluator 107 and stored in a similarity degree evaluation value storage part 112. While using the retrieved result and the evaluation values in the past stored in the similarity degree evaluation value storage part 112, an executed result output device 108 outputs certainty in the latest retrieved result.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a method of searching and processing arbitrary similar data from a group of data represented by numerical values or symbols stored in an information storage device such as a database device and providing the data to a user. The present invention relates to general data management and prediction of natural phenomena, social phenomena such as demographics, economic phenomena such as stock price fluctuations, and chemical and physical phenomena of industrial plants and the like.

[0002]

2. Description of the Related Art With the recent development of computer technology and the development of network infrastructure such as LAN and WAN, there has been an increasing movement to store and use a large amount and various kinds of data more than ever. A case has emerged where a database is used not only for routine tasks but also for irregular tasks such as analytical tasks.

Conventionally, DBMS (Database Management System)
Data has been compiled and processed using the search function provided in m). Currently DBMS is RDB (Relational Databas
The one using e) is the most popular, and RDB provides functions based on a language called SQL. SQL provides various aggregation functions as arithmetic operations on multiple tabular data. The literature is "Introduction to Database Systems,
CJ Date, translated by Joe Fujiwara, Maruzen Co., Ltd. " Hereinafter, what is described in this RDB will be referred to as conventional technology 1.

[0004] One of the purposes of analyzing the aggregated data is to predict physical and social phenomena. Other than the multiple regression analysis and autoregression analysis, MBR (Memory Based Reasoning) is a method for finding similar data based on a large amount of accumulated data and calculating and outputting the prediction result. There is. For MBR, please refer to `` Craig Stanfill, David Waltz, TOWARD MEMORY-BASED
REASOING, Communications of the ACM, Dec 1986, V
ol. 29. Number 29, pp. 1213-1228 ". Hereinafter, the one related to the MBR will be referred to as conventional technology 2.

In prior art 2, data similar to unknown data is searched for from known data, and prediction is performed by directly using these data. Compared to the classical prediction method or neural network, the operation of the prediction model is easier, and the tabular data can be used directly for prediction. There is an advantage that data accumulated in the environment can be easily used. As an example of applying Conventional Technique 2 to weather forecasting, there is “Takao Mohri, Hidehiko Tanaka, Weather Forecasting by Memory-Based Inference, Journal of the Japanese Society for Artificial Intelligence, 1995, Vol.19, No.5, pp.798-805”.

[0006]

The prior art 1 provides only a function of operating a plurality of tables, a simple processing of data in the tables, and a function of searching. For routine tasks with clear search keys, such as monthly sales for each branch or total sales for each product in each region, prepare a program in advance by a database administrator or analysis expert. Becomes possible. However, if you want similar data to the data that you already know, but the search key is not clear, the user must repeat the search while narrowing down the search key manually.To perform an efficient search,
Experience is required in how to narrow down search keys.

Prior Art 2 provides a function of automatically searching for similar data and a function of predicting unknown data using similar data. However, a very large amount of data is required for highly accurate prediction. Become. However, in a real problem, the basic structure of a prediction target may always change with time, and thus a problem arises in that old data cannot be used.

[0008]

Means for Solving the Problems As a first invention of the present application,
Means for inputting at least one record data consisting of a plurality of fields, means for storing the input data, means for specifying one or more similarity determination fields, and data for calculating similarity Means for specifying a distance definition, means for searching the stored data for data similar to the input data in accordance with the specified distance definition, and means for outputting the searched similar data with respect to the similarity determination field. Means for designating a similarity evaluation field; means for evaluating the similarity evaluation field; means for storing the evaluation result; means for controlling the output means using the stored evaluation result; There is provided a means for displaying the accumulated similarity evaluation results together with the searched similar data.

With this configuration, it is possible to judge how reliable the data is by evaluating the similarity of the found data and accumulating the results. Therefore, the effect of improving the search accuracy each time it is used can be obtained. In addition, sharing the accumulated similarity eliminates the need for the user's experience in searching.

According to a second aspect of the present invention, a means for inputting one or more record data comprising a plurality of fields, a means for storing the input data, and one or more similarity determination fields are provided. Means for specifying the distance between data for calculating the similarity,
Means for searching the stored data for data similar to the input data according to the specified distance definition, means for outputting the searched similar data, and means for specifying a similarity evaluation field with respect to the similarity determination field Means for evaluating the similarity evaluation field, means for accumulating the evaluation result, the output means using the accumulated evaluation result, and the value of the similarity evaluation field of the input record data is missing. If you have
There is provided means for inferring a missing field value using the searched similar data and the accumulated similarity evaluation result.

According to the second aspect of the present invention, when predicting unknown data using similar data as in the second prior art,
It is possible to correct the prediction result using the similarity of the data that is always obtained, and it is possible to perform highly accurate prediction even when old data must be used to obtain a large amount of data.

[0012]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS A first embodiment of the present invention will be described below with reference to the drawings. FIG. 1 shows the configuration of an information processing apparatus 100 using examples and the flow of data. Also,
FIG. 2 shows an information processing apparatus 200 using an example constituted by a server device and a client device connected via a network. 201 to 203 are client processing devices, and 204 is a server processing device. The number of server processing devices may be two or more, but here is one for convenience of explanation.

FIG. 3 shows an information processing apparatus 200 using cases.
The details are shown. Reference numeral 307 denotes a part constituting the client processing device, and reference numeral 308 denotes a part constituting the server processing device. Since the parts included in each client processing device are the same, the details of one device are shown here for convenience of explanation.

Here, a data input device 102, a similarity determination field input device 103, a similarity evaluation field input device 104, a similar data search device 105, a similarity determination device 106, a similarity evaluation device 107, and an execution result output device 108 , Search data output device 109, similarity output device 1
10 is the same as each device in FIG. Reference numeral 301 denotes a connection identifier input device; 302, a client-side transmission device;
03 is a server-side receiving device, 305 is a server-side transmitting device,
Reference numeral 306 denotes a client-side receiving device.

The principle of searching for similar data is the same between the apparatus 100 and the apparatus 300.
Will be described in accordance with

FIG. 4 shows an example of data stored in the device 308. The data 401 is an example of data for improving the production of beverages, and each row shows the date of production, the day of the week, the expected minimum temperature, the expected maximum temperature, the minimum humidity, the maximum humidity, the estimated atmospheric pressure, the previous day's production result, and the production result of the day Represents.

FIG. 5 shows an example of data provided to the device 300 to the device 300. Data 501 is input data to be searched for a similar example, data 502 is data specifying a similarity determination field, and data 503 is data specifying a similarity evaluation field. The similarity determination field is a field used when finding similar data, and the similarity evaluation field is used when evaluating the goodness of the found similar data. That is, the data 400
Is a designation when the user wants to search for data in which the actual supply amount is almost the same and other items are similar.

FIG. 6 shows an example of distance definition and distance definition management data stored in the device 308. Data 601 indicates a distance definition management table stored in the device 309, and data 602 indicates various distance definitions stored in the device 111. The data 601 describes the correspondence between which user uses which distance definition. Data 6
02 indicates the importance of each item when measuring the distance between data. The importance is represented by the numerical value of (0,1). The higher the numerical value, the higher the importance. If the importance is 0, it means that the item is not considered at all when searching for similar examples. means.

Data 700 in FIG. 7 shows the similarity evaluation result stored in the device 112. Each field value of each data stored in the device 113 is
It indicates what similarity the past search has, and the smaller the field value is, the higher the similarity is. For example, the field value 702 indicates the similarity of the record data 701 evaluated with respect to the manufacturing performance, and when compared with the record data 703, indicates that the similarity was high in the past search.

The data 800 shown in FIG.
Output. Alternatively, the device 108 represents data to be displayed. The field value 801 is the searched similar data itself, the field value 802 is the similarity calculated from the similarity determination field, and the field value 803 is the similarity evaluation field calculated from the similarity evaluation field. It means the degree to show whether or not it went well. That is, the field value 802 represents the similarity regarding the input data, and the reliability of the similarity is represented by the field value 803.

The outline of the present embodiment will be described below with reference to FIG. First, the administrator of the device 308 stores data in the storage unit 113 in advance. Here, the administrator may be a database administrator or a user himself / herself. At the time of the search, the input data 501, the similarity determination field designation data 502, and the similarity evaluation field designation data 503 are given to the devices 102, 103, and 104, respectively. The device 301 is provided with a user identifier, an identifier of the client device itself, an identifier of an application, and the like. Here, it is assumed that a user identifier is given for convenience of explanation.

The apparatus 302 stores data 501, 502, 5,
03, transmitting the user identifier to the device 303; Device 30
3 indicates the user identifier in the received data
04 to determine the distance definition actually used. Next, the device 105 searches for similar data based on the data 501, data 502, data 503, and the distance definition.

The actual search is performed by the device 106. In searching for similar data, calculation is performed based on how much the similarity determination field value in the data of interest differs from the similarity determination field value of the input data. The importance of each field is given by a distance definition. The search result is evaluated by the device 107, and the data 700 stored in the storage unit 113 is updated based on the evaluation result.

The data 801, data 802, and data 803 as a result of the search by the device 105 are sent to the device 306 by the device 305. However, the data 803 is a past evaluation value before being updated. The device 306 gives the received data to the device 108, and the device 108
09 to the data 801 and the device 110 to the data 8
02 and 803 are output.

As described above, the similarity evaluation device 117 and the similarity evaluation value storage unit 112 are provided, and the retrieval accuracy is improved by accumulating the past retrieval results each time they are used.

Further, by managing the distance definitions for each user by the devices 301, 304, and 309, it becomes possible to share the distance definitions of a plurality of users. For this reason, the feature of the present embodiment is that the result of the improved search accuracy can be shared similarly.

Next, details of the processing when the data 500 is input will be described with reference to the drawings. FIG. 9 shows the flow of the process of the present embodiment. Data 5 in processing 901
01 is input, and data 502, data 5
03 is input. However, the processes 901 and 902 are performed by the client processing device 307. Device 3
07, the data received and the identifier held by the client processing device,
Send to

The processes 903, 904, and 905 are performed by the device 30
8. In the process 903, it is determined which distance definition is to be used by referring to the data 601. In response, the process 904 actually executes a search process. Details of the process 904 are shown in FIG.

First, in step 1001, a temporary area for storing similar data is cleared. The size of the temporary area depends on the number of similar data to be found. The maximum number of similar data may be given by the user or may be determined by the administrator of the server processing device. Here, it is assumed that the administrator of the server processing apparatus determines the maximum number of similar data in advance for convenience of explanation.

In processing 1002, one record (one line) is referred to from the data 401 stored in the device 113.
In process 1003, the similarity between the input data and the data referred to in process 1002 is determined. For example, the input data and the data to be searched have similarity determination field values of X, Y,
Assuming that the importance of the similarity determination field is W, the distance d of a certain similarity determination field is d = w · (XY).
Given by The distance of the entire data is the sum D of the distances of the respective fields.
By the designation of the similarity determination field given in 02. It is also assumed that the similarity determination value W between data is 1 / (D · D + 1).
In other words, the larger the numerical value indicating the similarity determination value, the more similar the compared data is. The similarity determination value is 1 for exactly the same data.

Next, in step 1004, the similarity of the searched similar data in the similar data storage area is compared with the similarity of the similar data currently referred to. If the similarity of the similar data currently being referred to is lower than the similarity of the least similar data in the searched similar data, that is, if the similarity is closer to the input data, the similar data being referred to in process 1005 Is stored in the storage area, and the similar past data compared is deleted.

The process is repeated until all the data in the data 401 are compared by the process 1006.

In step 1007, the similarity (similarity judgment value) of the found similar data is evaluated using the similarity designation field designated by the data 503. Data evaluated at this time and determined to have low similarity is either (1) not used as similar data, or (2) the similarity evaluation value is transmitted to the client device 307 and the output process is controlled by the device 307. (3) There may be a method in which all similar data and evaluation values are displayed at the same time on the device 308 and the user determines whether or not the data is adopted or not. Here, for convenience of explanation, it is assumed that all similar data and evaluation values are output.

For example, when the similarity is evaluated in accordance with the data 503, the production result value of the input data is Z, the production result value of the similarity data i is Zi, the similarity is Wi, and Ei is the evaluation value of the data. First, it is assumed that the following (Equation 1) holds.

Z = Σi (Zi · Wi) / Σi Wi + Σi Wi · Ei (Expression 1) where i is an identification number of similar data. The first term is the actual production value of the input data, the weighted average of similar data, the second term is the correction term, and the third term is the error term. Processing 100
In step 7, the past evaluation value Ei of the similar data searched with reference to the data 700 is calculated. For example, when the similarity evaluation field is the production result and the found data of No. 2 is searched as similar data, the similarity evaluation 10 × Wi is calculated by the processing 1007 and the data 700. The similar data, the similarity determination value, and the similarity evaluation value are transmitted to the client processing device 307 in step 908.

The process 1008 is performed before transmitting data.
The similarity evaluation value Ei is updated based on the current search result. In the actual search result, the left side and the right side of (Equation 1) do not always match. Therefore, the similarity determination values for the last n times are calculated as Wij
(j = 0,1, .., n, where i is the identification number of the similar data found in the j-th search) to reduce the difference between the left side and the right side, And Ei is updated so as to reduce the square error ε between the left side and the right side of (Equation 1).

Ε = {Σj (Z−Σi (Zi · Wij) / ΣI Wij−Σi Wij · Ei) ^ 2} / 2 (2) where a ^ b represents a raised to the power of b. From this (Equation 2),
According to the following (Equation 3), a minimum solution of ε can be obtained.

Σε / σEi = (Z−Zi · Wij−Wij · Ei) · Wi (Equation 3) However, when this embodiment is used for the first time, the initial value of Ei is set to a small random value. I have.

Here, when correcting the correction term, it is possible to make a large correction to the evaluation value for those similar with respect to the judgment field value, and to make a small correction to the one not similar with respect to the judgment field value. (Equation 1) can be correctly corrected even when certain data is referred to as similar data of different input data.

The correction term Wi · Ei is similar data i
The smaller the value of Wi · Ei, the higher the similarity between data with respect to the similarity evaluation field.
The greater the Ei, the lower the similarity.

In process 908, the similar data transmitted by the device 307, the similarity judgment value, and the similarity evaluation value are passed to the device 308. Here, processes 906 and 908 are processes in the device 308. The process 906 outputs the information shown in the data 800.

As described above, in this embodiment, the accuracy of the similarity search is improved each time the search is repeated by using the result of the past search.

Next, a second embodiment of the present invention will be described. When receiving input data including an unknown field value, the apparatus 1100 searches for similar data and predicts an unknown field value at the same time. FIG. 11 shows the configuration and data flow of an information processing apparatus 1100 using a case. Here, 1105 is a client processing device, 11
06 is a server processing device, 1103 is a prediction result output device,
Reference numeral 1104 denotes a similarity evaluation field prediction device. Reference numeral 1101 denotes the devices 101, 102, 1 in FIG.
03 and 104 are the same. 302, 303, 3
05 and 306 are the same as those in FIG. 11
03 denotes the devices 304, 309, 105, 1 in FIG.
Same as 06, 107, 111, 112, 113.

FIG. 12 is an example of input data including a missing value given to the device 1101, and the actual production value is missing. In addition, data 502 and 503 are input data.
1101. Each of these data may be given simultaneously. Data 502 is a specification of a field for determining the similarity of data, and data 503 is a field including a missing value of the data.

FIG. 13 is an example of the output result of the apparatus 1100, and the missing value of the production result is predicted to be 3700.
The data 1400 in FIG. 14 is input when the true value is obtained to evaluate the prediction operation of the device 1100 and is given to the device 1101. In the second embodiment, by using the device 1103,
The prediction accuracy improves with each use.

Hereinafter, the processing of the apparatus 1100 according to the processing 1500 of FIG. 15 will be described. In processing 1501, the device 1
According to 101, the device 1100 obtains data 1200. Next, in a process 1502, data 502 and 503 are obtained. In the process 1503, the data 1200, 502, and 503 are transmitted to the device 1106. Processing 1504, 1505, 15
06 is the same as the processes 903 and 904,
00 similar data, a similarity determination value, and a similarity evaluation value are obtained. However, since the similarity evaluation field is unknown, the data 700 stored in the device 112 is not updated. Processing 1507 predicts a missing field value from similar data using (Equation 1). In the process 1508, the predicted value is transmitted to the device 1105, and the process 1508 receives the transmitted predicted value, similar data, similarity determination value, and similarity evaluation value. The process 1509 outputs the data 1300 to the user. Data 1301 represents similar data, and 1302 represents a predicted value of a missing value of input data.

Next, the processing 1600 when the true value of the prediction is obtained later will be described with reference to FIG. Apparatus 1100
Are data 1400, 5 by processing 1601 and 1602.
02 and 503 are received. The fields designated by 502 and 503 are the same as those used at the time of prediction. Process 16
03, 1604, 1605, and 1606 are processes 1503,
Same as 1504, 1505, 1506, and data 1
400 similar data, a similarity determination device, and a similarity evaluation value are obtained. However, since there is no missing value field unlike the process 1500, the data 700 stored in the device 112 is updated by the processes 1606 and 1607 according to the designation of the similarity evaluation field. However, processing 160
Steps 6 and 1607 are the same as steps 1007 and 1008.

As described above, in the process 1600, when prediction is performed using similar similar data from the next time, a prediction value closer to the true value is obtained, and the distance definition and the similar data evaluation value are converted into a plurality of values by the server processing apparatus. As a result of sharing by the user, the result of the improved prediction accuracy can be similarly shared.

[0049]

As described above, according to the present invention, by providing the data similarity evaluation device and the evaluation value storage device, there is an advantage that the retrieval accuracy is improved each time it is used, and the improvement in accuracy can be shared by a plurality of users. Further, by providing the similarity evaluation field prediction device, it is possible to predict a missing similarity evaluation field using similar data. When a true value is obtained later, the prediction accuracy is improved, and the result of the accuracy improvement can be shared by a plurality of users.

[Brief description of the drawings]

FIG. 1 is an overall view of an information processing apparatus using a case.

FIG. 2 is a hardware configuration diagram of a client / server processing device using the present invention.

FIG. 3 is an overall view of an embodiment using a client / server processing device.

FIG. 4 is a diagram showing an example of input data.

FIG. 5 is a diagram showing a format for storing a distance definition.

FIG. 6 is a diagram showing a distance definition management table.

FIG. 7 is a diagram showing an example of a format for storing a similarity determination result.

FIG. 8 is a diagram showing an example of output data (execution result) of a similar example search.

FIG. 9 is a flowchart showing processing in the apparatus 300.

FIG. 10 is a flowchart showing processing in the device 105.

FIG. 11 is an overall view of an embodiment of the present invention.

FIG. 12 is a diagram showing an example of input data given to the device 1100.

FIG. 13 is a view showing an example of output data of the apparatus 1100.

FIG. 14 is a diagram showing an example of true value input data.

FIG. 15 is a flowchart showing a prediction method according to the present invention.

FIG. 16 is a flowchart showing a method of searching for similar data in the present invention.

 ──────────────────────────────────────────────────続 き Continuing on the front page (72) Inventor Masafumi Okada 5-2-1 Omika-cho, Hitachi City, Ibaraki Prefecture Inside the Omika Plant, Hitachi, Ltd. (72) Katsumi Omori 5-chome, Omika-cho, Hitachi City, Ibaraki Prefecture No. 1 Inside the Omika Plant of Hitachi, Ltd.

Claims (8)

[Claims] 1. An information processing apparatus for searching for desired data from stored data, input means for inputting at least one record data consisting of a plurality of fields, and specifying at least one similarity determination field. Determination field specifying means; distance definition specifying means for specifying a data distance definition for calculating similarity; and for the similarity determination field, data similar to the input data according to the specified distance definition is stored data. Search means for searching from the list; evaluation field designating means for designating a similarity evaluation field; evaluation means for evaluating the similarity evaluation field; and means for controlling the search means using the evaluation result of the evaluation means. An information processing apparatus comprising: 2. The information processing apparatus according to claim 1, wherein when the value of the similarity evaluation field of the input record data is missing, the similarity data retrieved and the evaluation result are used to remove the missing data. An information processing apparatus comprising: means for inferring a field value. 3. The information processing apparatus according to claim 1, further comprising: a storage unit configured to store the evaluation result of the evaluation unit, wherein the evaluation result stored in the storage unit corresponds to a change in the evaluation result. An information processing apparatus characterized in that the information processing apparatus changes the information. 4. The information processing apparatus according to claim 1, further comprising means for displaying the accumulated evaluation results together with the searched similar data. 5. A data analysis device comprising: a client processing device; one or more server processing devices for managing a plurality of record data including a plurality of items; and a network connecting the client processing devices, wherein the client processing device comprises: One or more records to be subjected to data analysis, a first transmitting means for transmitting an instruction of a data analysis method to any one or more of the servers, and the server processing device transmits the transmitted instruction. Means for searching for data similar to the transmitted data from the data stored in each database, and means for evaluating the degree of similarity of the searched similar data by the analysis method by the server processing device. Means for accumulating the evaluation result by the server processing device; and Means for controlling data search means, second transmission means for transmitting the search result of the searched data to the client processing device, and means for outputting the transmitted analysis result by the client processing device. An information processing apparatus characterized by the above-mentioned. 6. The information processing apparatus according to claim 5, further comprising: means for displaying the stored similarity evaluation results together with the retrieved similar data. 7. A storage medium storing a program for retrieving similarity of field values and therefore similar data from one or more record data sets consisting of one or more fields, wherein one or more similarity judgments are performed. When a field, one or more similarity determination fields, and a data-to-data distance definition for calculating the similarity are specified,
Searching the stored data for data similar to the input data according to the specified distance definition, outputting the searched data, and evaluating the similarity between the input data and the searched data with respect to the similarity evaluation field. A storage medium characterized in that: 8. The storage medium according to claim 7, wherein a program having a procedure for displaying the similarity evaluation result together with the searched similar data is stored.