JP3185167B2 - Data processing system - Google Patents

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention processes a group of information represented by numerical values or symbols stored physically divided into a plurality of databases, converts the processed information into a useful expression for a user, and outputs the converted information. More particularly, the present invention relates to a data processing system that realizes high-speed processing and can execute processing without making a user aware that data is physically divided and stored.

[0002]

2. Description of the Related Art Recently, there has been a remarkable information processing technology, and the amount of data stored in a computer and the amount of data to be processed are increasing year by year. This trend is becoming more pronounced. At present, the data amount is gigabyte (= 10 9), and the number of records is 1
It is not unusual to have more than one million. Generally, data itself stored in a computer is merely a set of numerical values and symbols, and cannot be used as it is.
Therefore, in order to effectively utilize the data, various techniques have been proposed for converting the set of data into useful information and providing the information to the user. Typical techniques are: (1) Statistical methods such as regression analysis and multiple correlation analysis (hereinafter referred to as Conventional Technique 1) (2) Learning the interrelationship between data using a neuro model or fuzzy model (Hereinafter referred to as Prior Art 2) (3) A method using a knowledge acquisition method such as rule induction (hereinafter referred to as Prior Art 3) is conventionally known.

[0003] The prior art 1 is a well-known technique that is widely and generally known, and will not be described in detail here. Regarding the above prior art 2, learning of a neuro model is described in many documents, but basically, a nonlinear relationship between multiple inputs and multiple outputs is modeled by learning. In the sense of learning a non-linear relationship, a method is known in which an input / output relationship is expressed by a fuzzy model and the shape of a membership function is adjusted by learning using an algorithm similar to that of a neuro. In either the neuro or fuzzy method, the input / output relationship is represented by a model including a certain parameter, and the parameter is determined by learning to construct a model.

[0004] Regarding the above prior art 3, for example, Spr
"Machine Learning" by inger-Verlag, 463-482
"Learning Efficient Classifi
cation Procedures and Their Application to Che
ss End Games ”by J. Ross Quinlan. This paper describes an algorithm called ID3, which automatically creates a decision tree for classification from data. Creative Workstation 2050 Manual,
Page 23 of "ES / TOOL / W-RI Explanation / Operation"
On page 53, a method for expressing the relationship existing between data in the form of rules is described in detail.

Next, a description will be given of an example of creating a decision tree based on ID3. First, a set C of all cases is C = ｛short, blond, blue,: +, short, dark, blue: −, tall, dark, brown: −, tall, blond, brown: −, tall, dark, Blue: −, shot, blond, brown: −, tall, red, blue: +, tall, blond, blue: +｝ Each case is height (value is short
t, tall), hair color (value is blond, dark, red), eye color (blue, b
rown) and two attributes + and-
Belong to one of two classes. ID3 attempts to solve the problem of determining which attributes can be classified into a + or-class by determining which attributes and in what order by generating a decision tree. In generating a decision tree, attributes are selected based on which attribute determines which information amount after determination is the smallest (that is, which attribute has the largest amount of information). FIG. 18 shows an example of a decision tree obtained in the above case. The meaning of this decision tree is as follows. First, (a) classify cases by hair color. (A1) If the hair color is dark, it is-. (A2) + if the hair color is red. (A3) If the hair color is blond, then classify it by eye color. (A31) + if the eye color is blue. (A32) If the color of the eye is brown, then-. Until now, various ID3 deformation algorithms have been proposed, but basically determine the order of determination according to a certain criterion.

The prior art 3 differs from the prior arts 1 and 2 in that the result of processing the data is a rule when the result is a rule induction, and a decision tree when the result is ID3. Because it is obtained, it is particularly effective for the user to discover the relationship included in the accumulated data. These techniques are often used as techniques for effectively utilizing large amounts of data.

[0007] In parallel with the above-mentioned data utilization technology, development of parallel database technology mainly aimed at retrieving a large amount of data at high speed is progressing. Such a parallel database technology is described in, for example, a paper entitled "DBMS technology for parallel machines-aiming for practical use in the mid-90s" described on pages 91 to 106 of "Nikkei Electronics July 19, 1993". (Hereinafter referred to as prior art 4). Also, a software technology for searching a database using spreadsheet software as a user interface is described in, for example, “Nikkei Computer 1”.
This is described in a paper entitled "Connecting the RDB of a Server Directly-To Basic Data to Spreadsheet Software" described on pages 65 to 75 of the July 12, 993 issue (hereinafter referred to as Prior Art 5). ). In this paper, a server processing device and a client processing device directly used by a user are connected via a network, and the client processing device generates a database search request using an interface provided by spreadsheet software. SQL (Struc
A search command described in “tured query language” or the like is sent to the server processing device, the server processing device executes a search according to the request, and transfers the search result to the client processing device. The client processing device formats the search result transferred from the server processing device into a spreadsheet form and displays it to the user. With such a mechanism, at least search and simple calculation of statistics (average value, variance, etc.)
With regard to, the user can work as if all data is at hand (client processing device) without being aware of the distributed database at all.

[0008]

The prior arts 1 and 2 are techniques for processing data stored in an electronic computer by regression analysis, statistical processing, or learning processing.
No particular consideration was given to how to use the processed result, such as how the user would use it. On the other hand, the rule induction and ID3 of the prior art 3
In a method that uses an algorithm or other knowledge acquisition method, the processed result can be useful to the user, but this technology mainly uses what kind of data processing means to obtain useful information. And how to execute it on a parallel database as dealt with in prior art 4
It has never been considered before.

On the other hand, the parallel database technology of the prior art 4 focuses on how to execute transaction processing in a so-called online system and data retrieval processing in an offline processing at high speed using parallel hardware. There is no known means for executing a process for processing a large amount of data at a high speed by utilizing hardware parallelism as in the above-described prior arts 1 to 3 at a high speed. Therefore, the effect obtained by applying the knowledge acquisition method of the prior art 3 to the parallel database of the prior art 4 is simply limited to the original effect of the parallel database, that is, speeding up data retrieval. As described above, conventionally, there is no known method that makes use of the inherent parallel capability of hardware, and therefore, there is a problem that the knowledge acquisition method of the related art 3 cannot be executed sufficiently fast. there were. A first object of the present invention is to solve the above-mentioned problems and to provide a data processing method and apparatus which can be executed at high speed on a parallel / distributed database.

[0010] In general, extracting, collecting, and accumulating new data requires a great deal of labor, time, and cost. Therefore, it is important to effectively use existing data assets. Has become. For example, even in the same company, different information may be constructed as completely different databases for different departments. In this case, the database is physically far away, and may even be built on completely different hardware, using completely different software. In the above-described related art 3, it is assumed that data exists on a processing device that executes processing. That is, in order to apply the related art 3, it is necessary to perform a process of extracting related data on distributed hardware and collecting the related data on one processing device. At present, processing devices are often connected via a network, but it is still time-consuming to define and configure a database that collects related data each time data processing is attempted, There is a problem in that it is very costly both in terms of network load and human work. From the viewpoint of the user using the above-mentioned conventional technology 3, the user processes data in order to achieve various purposes, and it is inefficient to spend much time and cost to prepare data for that purpose. In addition, there has been a problem that the effectiveness of these technologies has not been fully utilized.

The above-mentioned prior art 5 is a technique for solving the above problem in the case of a database search. However, the object of the prior art 5 is only processing that is used in ordinary spreadsheets, and measures are taken against the advanced data processing that is the object of the prior art 3. Did not. For example, when trying to execute rule extraction processing by rule induction, a data request to the server processing apparatus is generated every time data is referenced in processing on the client processing apparatus.
There is a problem that the load on the network becomes very heavy, and the processing speed is hardly practical. In particular, when the database is located far away, for example, when one database is located in Tokyo and another database is located in Osaka, this problem becomes more prominent. A second object of the present invention is to solve the above-mentioned problem, and to use a rule on a server processing device connected via a network without using a user to be aware of the location of physical data. By using techniques such as induction, the relationships between data can be processed in a form that is useful to users, and these processes can be performed at high speed by effectively utilizing the processing capabilities of processing devices connected to the network. An object of the present invention is to provide a method and a device for performing the method.

[0012]

The present invention SUMMARY OF], in order to achieve the above object, in a data processing system connecting a client processor and a plurality of server processing apparatuses, Kura
Client processing devices are classified into the plurality of server processing devices.
A means for sending a processing request;
Server processing devices execute the classification processes in parallel.
Means and a processing result to the client processing device.
Means for communicating, the client processing device further comprising
Means for synthesizing the received processing result, and synthesizing
Means for executing predetermined processing using the results obtained.
You. The client processing device is stored in the database.
Of data items included in stored table format data
Classification rule specification method that specifies a classification rule for classifying values
And transmitting the specified classification rule to the server processing device.
And a first transmission unit that performs
Each database based on the transmitted classification rules
Classification executing means for classifying the data stored in the
Second step of transmitting the classified result to the client processing device
Transmission means, and the client processing device further comprises:
Means for synthesizing the transmitted classification results;
Analyze relationships between data items based on classified results
Analysis means, and output means for outputting the analyzed result.
are doing.

[0013]

According to the present invention, there is provided a classification rule designating means for designating a classification rule for classifying a value of a data item included in table format data, and a classification rule designated by the classification rule designating means. Transmission means for transmitting to the server processing device, classification execution means for performing classification processing on data stored in each database in the server processing device according to the transmitted classification rules, Second transmission means for transmitting the result classified by the execution means to the client processing device; means for synthesizing the classification result transmitted to the client processing device to generate a classification result for the tabular data to be processed; And analysis means for analyzing the relationship between data items using the generated classification results, so that time-consuming processing is possible. The can be performed in parallel, and without completely transparent parallel processing to the user, also without having to transfer large amounts of data itself, it is possible to perform the process of interest. Also, when applied to rule induction, the rule induction processing itself is executed on the client processing device. However, since it is sufficient for the client processing device to hold only the results classified on each database, network The target processing can be executed without increasing the load on the server. Further, the user can execute the process without any awareness of the database on which the data to be used physically exists, and obtain an appropriate process result.

[0014]

DESCRIPTION OF THE PREFERRED EMBODIMENTS First, an embodiment of the present invention will be outlined. FIG. 1 shows an overall configuration for implementing the present invention. In FIG. 1, 101 is a user who uses the database processing apparatus of the present invention, 102 is a client processing apparatus, and 103 to 1
06 is a server processing device, 107 to 110 are databases connected to the server processing devices 103 to 106, respectively, and 111 is a bus connecting the client processing device and the server processing device. Multiple databases 107 to 11
0 is an independent DBMS (Date Base Management)
System) or may be managed by a parallel DBMS that performs unified management on a network at all. Here, the former case will be described as an example. For convenience of explanation, data is shown in FIG.
Is described by a plurality of records (corresponding to rows) having a plurality of attributes as shown in FIG. 2, and the plurality of records are divided and stored in a plurality of databases (in FIG. M records having attributes are divided and stored in three databases.)

For example, in the client processing device 102, data distributed and stored in a plurality of databases 107 to 110 can be virtually displayed as one combined table image (conventionally. Technology 5). At this time, it is assumed that the user extracts a classification rule from the table image data by, for example, rule induction. Since the data stored in multiple databases is generally huge,
Transferring all data to the client processing device is generally not possible due to processing time and cost.

Therefore, in the present invention, the rule induction processing operating on the client processing device transmits a processing request for a classification result based on a certain attribute or attribute value to a plurality of server processing devices. The format of the processing request is, for example, generally known SQL (Structured Que).
ry Language) statement. The plurality of server processing devices execute the processing requested from the client processing device in parallel, and transmit the processing result to the client processing device. In the case of rule induction processing, the processing result is obtained by classifying the records in each database according to the attribute or attribute value specified by the client processing device, for example, in the form of a correspondence table of record numbers and classification results. I have. The client processing device summarizes the processing results transmitted from the plurality of server processing devices, and stores information necessary for subsequent rule induction, that is, a classification table of all records according to each attribute or attribute value on the client processing device. Can be created. Therefore, without transferring the information itself stored in the database to the client processing device,
By having only the classification results on the client processing device, the load on the network can be reduced and processing can be performed at high speed.

When the client processing device directly executes the rule induction without performing such processing, the search request is sequentially transmitted to a plurality of server processing devices as the rule induction process proceeds. As a result, the load on the network also becomes extremely large, and as a result, the time required for the rule induction processing is increased. With the means provided in the present invention, such overhead can be reduced, so that high-speed processing can be realized.

Hereinafter, the detailed operation of the embodiment of the present invention will be described. In describing the embodiment, first, the operating principle of rule induction will be described. For details, see, for example, the above-mentioned documents, Hitachi Creative Workstation 2050 Manual, “ES / TOOL /
W-RI Explanation / Operation ", pp. 23-53. FIG. 3 is an example of a case table for inputting rule induction. In the case table 200 that is an input of the rule induction, each row corresponds to one case. In this example, each credit application corresponds to one case. Each case is 1
Have one or more attributes. In the example of FIG. 3, there are four attributes of “bank credit” 201, “deposit balance” 202, “current debt” 203, and “credit assessment” 204. Each attribute has an attribute value in a predetermined format. In the example of FIG. 3, “bank credit” 20
The attribute 1 has one of two attribute values, “present” and “absent”. Such an attribute is called a “symbol attribute”. "Deposit balance" 202 and "current debt"
The attribute 203 has an attribute represented by a numerical value, that is, a numerical attribute. In a general case table, a symbol attribute and a numerical attribute are mixed. In addition, in the example of FIG. 3, some of the attributes of some of the cases are unknown (see reference numeral 205). This is called a “missing value”.

In executing the rule induction, first, it is specified which attribute is to be a conclusion attribute. FIG.
In the example, it is assumed that an attribute "credit assessment" is designated as a conclusion attribute for the purpose of creating a system for automatically determining "credit assessment". In general rule induction processing, the conclusion attribute must be a symbol attribute. Furthermore, an explanation attribute is designated which is treated as an input to explain the attribute value of the conclusion attribute in each case. In the example of FIG. 3, all attributes other than the conclusion attribute are treated as explanatory attributes. Of course, only a part of the case table can be designated as the description attribute.

In the case table as shown in FIG. 3, as the number of cases (the number of rows) increases, it becomes more difficult to handle numerical attributes as they are. Therefore, a “label assignment process” for replacing a numerical attribute with a symbol attribute may be performed prior to the rule induction process. The label assigning process refers to a process of, for example, performing appropriate section division on an attribute having a numeric attribute and assigning a symbol called a label to each section to replace a numerical value with a symbol. FIG. 4 shows an example of a case table in which all attributes are replaced with symbol attributes by performing such labeling processing. This is an example in which the “deposit balance” 208 is “large” for 10 million yen or more, “medium” for a positive one less than 10 million yen, and “small” for a negative deposit balance. Regarding “current debt”, 800 million yen or more is “large”, 800
"Middle" for items less than ¥ 100,000 and more than ¥ 40 million, 40
This is a case where a label of less than one million yen is labeled as “small”.

As described above, data items having real or integer values can be classified into several groups by designating upper and lower limits, respectively. When specifying the upper limit value and the lower limit value, the groups may be divided so that each group has the same number or almost the same number. By the labeling process, it is possible to disregard the small difference between the numerical attributes and analyze the relation between the attributes according to the symbol attribute values which are largely classified. In particular, for a case table with a large number of cases, it is effective in terms of the processing time and the usefulness of the classification rules output.
Also, for data items that have a value as a string,
Some character strings are determined in advance, and the character strings can be classified into an appropriate number of groups by converting to any one of the character strings.

Next, in the rule induction process, a process of finding a set of explanatory attribute values that best describes the attribute value of the attribute specified as the conclusion attribute, and extracting it as a rule is performed. There are various options in this processing, and by setting those options appropriately, rules more useful to the user can be extracted. These options are not described here. For details, refer to the above document, Hitachi Creative Workstation 2050 Manual, "ES / T
OOL / W-RI Explanation / Operation ", page 23 to page 53.

FIGS. 5A and 5B are examples of rules extracted by the rule induction process. Various methods have been proposed for the internal processing of rule induction, but basically, based on the number of cases where one or more attributes have a specific attribute value (or set of attribute values), Conclusion Consists of a process for finding a set of attribute values that best classifies attributes. Therefore, in executing the rule induction process, it is sufficient that there is a correspondence table indicating which symbol attribute value each attribute of each case has. That is, in the initial processing of the rule induction, if such a correspondence table is prepared, all subsequent processing can be executed based on only this correspondence table.

FIG. 6 shows an example of such a correspondence table 211. Each row of the correspondence table of FIG. 6 corresponds to one case, similarly to the case table of FIG. Each column in FIG. 6 includes attributes “bank credit” 213 and “deposit balance” 21
4. “Current debt” 215, “Credit assessment” 216
It corresponds to. Each component of the correspondence table 211 indicates a label code indicating which value the corresponding attribute value of the corresponding case has. In FIG. 6, for the sake of simplicity, a unique case number (record number) 212 is assigned to distinguish cases. Further, which symbol attribute value corresponds to the value of each component in the correspondence table 211 can be managed by another correspondence table as shown in FIG. In FIG. 7, (a) shows a correspondence table between bank credit components (no / existence) and symbol attribute values (0/1), and (b) shows deposit balance components (small / medium / large / unknown). And correspondence table of symbol attribute values (0/1/2/3), (c)
Is the component (small / medium / large) of the current debt and the symbol attribute value (0
(D) is a correspondence table of each component (impossible / acceptable) of the credit assessment and the symbol attribute value (0/1).

What is important here is that the value taken by the label code can be generally expressed by several bits, depending on how many kinds of values the attribute value for each attribute takes. In the case table itself shown in FIG. 3, for example, it is necessary to allocate a data amount of 4 bytes (two full-width kanji characters) to represent the character “impossible” for the attribute of “credit assessment”. On the other hand, the label code in the correspondence table in FIG. 6 only needs to be able to discriminate between “1” and “0”, and thus basically needs to be 1 bit. Therefore, when the rule induction is executed for a very large number of cases, efficient processing can be performed by converting the case table of FIG. 3 or 4 into the correspondence table of FIG. 6 in advance. The outline of the rule induction processing has been described above. ID3
It goes without saying that an algorithm for creating a decision tree for classification such as that described above can be executed only by referring to the above correspondence table.

The above processing can be executed in an environment as shown in FIG. 1 by using the means of the present invention. That is, in the configuration of FIG. 1, the client processing device that executes the rule induction processing and the server processing device where the data exists are physically different processing devices, and the data itself is distributed to a plurality of server processing devices. Therefore, the above processing cannot be simply executed as it is. If all relevant data can be transferred to the client processing unit and the processing can be centralized on the client processing unit, it is easy to talk about, but especially when dealing with a large amount of data, the database itself is physically This is not practical if the data transfer is far away and the data transfer is expensive.

Hereinafter, the first embodiment of the present invention will be described in detail. In the first embodiment, the rule induction process is executed at high speed and efficiently in the configurations shown in FIGS. FIG. 8 shows a configuration example of the case table 220. In this example, “case number” 221 for distinguishing between cases, and “name” 222 and “name code” 22 as attribute values of each case
3, “bank credit” 224, “deposit balance” 225, “annual income” 226, “current debt” 227, “age” 228,
"Gender" 229, "Address" 230, "Spouse status" 2
31, “Dependents” 232, “Occupation” 233, “Final education” 234, “Housing type” 235, “Credit card holdings” 236, “Usage record” 237, “Credit assessment” 238 .

FIG. 9 shows the case table 220 shown in FIG.
Is stored in a plurality of databases. In this embodiment, case numbers 1 to 1000 as shown in FIG.
Is stored in the database 1, the case numbers 1001 to 2500 are stored in the database 2, the case numbers 2501 to 4500 are stored in the database 3, and so on. Each server processing device locally manages which range of records it has, and upon receiving a search processing request from the client processing device, executes a search only within the local data range and sends the result to the client processing device. A transmission mechanism is provided. This method uses "shared nothi
This is called the "ng" method. Of course, parallel D
The following processing is the same even when there is no DBMS that integrally manages the whole like BMS, and each database is managed simply by an independent local DBMS.

FIG. 10 shows the flow of the entire process. First, in step 250, the client processing device (see FIG. 1) displays a part of the case table on the display device in the format shown in FIG. The user specifies a conclusion attribute and a description attribute on the display device (step 251). Here, attributes that are generally not considered to be clearly related to the conclusion attribute are omitted. In this example, “name” 222, “name code” 2
23. Attributes such as “address” 230 are considered to be unrelated to the conclusion attribute “credit assessment” and are omitted in advance. Here, it is assumed that all other attributes are specified as explanatory attributes.

In the next step 252, the client processing device sets a “label code correspondence table”, which is information necessary for labeling processing, for the explanation attribute and the conclusion attribute.
Create FIG. 11 shows an example of the label code correspondence table 261. In the label code correspondence table 261, reference numeral 262 indicates that a label “small” defined for an attribute “balance” corresponds to a value whose attribute value is less than 0,
A label value of “medium” corresponds to an attribute value of 0 or more and less than 1000, and a label value of “large” corresponds to an attribute value of 1000 or more. It is considered that a part in which no value is specified in the label code correspondence table 261 has no restriction specified. Also, a special label code value is assigned to a label value of “unknown” corresponding to a missing value. For example, as indicated by reference numeral 263 in the label code correspondence table 261, the label code “3” is associated with the unknown balance. In addition, as shown by reference numerals 264 and 265, the correspondence between the attribute value and the label code is specified for the symbol attribute.

Next, when the user instructs to start the rule induction process (step 253), in step 254, the client processing device requests the plurality of server processing devices for a search request including the information shown in FIG. Send

At step 255, when each server processing device (see FIG. 1) receives the search request of FIG. 11, each server processing device starts a search process. First, each case data is converted into a case-attribute value correspondence table similar to that of FIG. 6 based on the label code correspondence table of FIG. 11 (step 256). FIG. 12 shows an example. In this search processing, for all the attributes appearing in the label code correspondence table of FIG. 11, if the attributes are symbolic attributes, the attribute values are replaced with the designated label codes. Performs a process of classifying in the specified numerical range and replacing it with a label code. After converting the locally managed case table into the case-attribute value correspondence table in FIG. 12, each server processing device transmits the result to the client processing device as a search result in step 257.

The client processing device receives the processing results from the plurality of server processing devices (step 258), combines them, and creates a case-attribute value correspondence table of the entire case table on the client processing device (step 258). 25
9). In this process, the search results from the respective server processing devices may be combined in the row direction, which is completely opposite to the case table division shown in FIG. After the case-attribute value correspondence table is created, the rule induction process extracts rules by referencing only this table (step 260). The above description is not limited to the rule induction,
The same applies to the process of automatically generating a classification decision tree based on attributes such as D3.

As described above, according to the first embodiment of the present invention, when executing the rule induction process in the client processing device, the functions of the DBMS executed by the plurality of server processing devices are expanded in advance, and , It becomes possible to execute necessary search results in parallel according to a request from the client processing device.
A search request to a plurality of server processing devices and the acquisition of the result need only be executed once prior to execution of the rule induction processing main body, and the amount of data flowing on the network is significantly reduced, and the network load is reduced. Becomes possible. Further, since the search processing itself is also executed in parallel, there is an effect that the entire processing time can be reduced.
Furthermore, from the user's point of view, the processing is performed directly on the table image data on the client processing device,
Since the work of obtaining the result is made possible, there is an effect that the work can be performed more naturally without being conscious of the fact that the case table is divided and stored in a plurality of databases.

In the first embodiment of the present invention, as shown in FIG. 2, it is assumed that the case table is divided into rows and stored in a plurality of databases. That is, it can be said that a plurality of databases have been used in advance as a set to store one case table. However, in practice, there are cases where a plurality of departments construct databases independently of each other and use those databases in an integrated manner. In that case, the case table can be considered to be divided not by rows but by columns.

In the second embodiment of the present invention, processing such as rule induction is performed using data in a plurality of databases divided in units of columns. FIG. 13 shows the configuration of the database in this embodiment. The content of the database is the same as the configuration of FIG. 8 described in the first embodiment, but the family information and the financial related information are stored in separate databases. FIG. 13A shows a database 300 relating to family information, which includes “name” 301, “name code” 302, “age” 303, “sex” 304, “address” 305, “presence or absence of spouse” 306, and “dependency”. The number of family members 307, the occupation 308, the annual income 309, the final educational background 310, and the type of dwelling 311. FIG. 13B shows a finance-related information database 320 having a “name” 321 and a “name code” 3.
22, "bank credit" 323, "deposit balance" 324,
The attributes are “annual income” 325, “occupation” 326, “credit card holdings” 327, “use record” 328, and “credit assessment” 329.

These databases may be initially configured for completely different purposes, or may reside on completely different devices in terms of hardware. Further, it may exist in a place physically far away. However, these databases must be connected to a network and available online from a client processing device directly used by a user. Here, the attributes “name”, “name code”, “occupation”, and “annual income” commonly exist in the two databases. Among these, "name code" is considered as a key attribute for specifying a record. If the consistency of the two databases is not managed, the values of the attributes “occupation” and “annual income” may differ between the two databases. Therefore, when handling data in the form of the table in FIG. 8, the user must specify which database and what item each item is physically associated with.

FIG. 14 shows an example of the attribute relation table 330 that defines this relation. Table 330 in FIG.
Is composed of an attribute, a database type, and an item name, and indicates which attribute (item type) in which database (database type) corresponds to each attribute such as a description attribute, a conclusion attribute, and a key attribute. . For example, FIG.
In the example of the table 330, “bank credit”, “balance”
Attribute corresponds to the item names "credit" and "deposit balance" in the financial database, and the attributes "name" and "annual income" correspond to the item names "name" and "
It indicates that it corresponds to "annual income". Since the key attribute specifies a record, it must be included as an attribute in all databases used as a matter of course. In this table, it is assumed that the user designs and defines which database and which attribute to use in advance, considering the purpose of data processing.

FIG. 15 shows a flowchart of the subsequent rule induction processing. First, in step 340, the client processing device (see FIG. 1) displays a part of the case table on the display device in the format shown in FIG. The user specifies a conclusion attribute and a description attribute on the display device (step 341). Here, attributes that are generally not considered to be clearly related to the conclusion attribute are omitted. In this example, attributes such as “name”, “name code”, and “address” are not related to the conclusion attribute “credit assessment” and are omitted in advance. Here, it is assumed that all other attributes are specified as explanatory attributes.

In the next step 342, the client processing device sets a “label code correspondence table”, which is information necessary for labeling processing, for the explanation attribute and the conclusion attribute.
Create FIG. 16 shows an example of this label code correspondence table. FIG. 16A is a label code correspondence table 360 for the family information database, and FIG. 16B is a label code correspondence table 370 for the financial information database.
The meaning of the correspondence table is almost the same as that described with reference to FIG. 11, but differs in that an item for specifying a key attribute is included. Otherwise, the processing up to this point is the same as that described in the first embodiment. Here, the attribute of each database used for the rule induction can be determined with reference to the attribute association table of FIG.

Next, when the user instructs the start of the rule induction process (step 343), the client processing device sends the two server processing devices the labels shown in FIGS. 16A and 16B, respectively. A search request including the code correspondence tables 360 and 370 is transmitted.

In step 345, when each server processing device (see FIG. 1) receives the search request shown in FIG. 16, each server processing device starts a search process. First, each case data is converted into a case-attribute value correspondence table similar to that of FIG. 6 based on the label code correspondence tables of FIGS. 16A and 16B (step 346). FIG.
7 shows an example. In this search processing, FIG.
And (b) label code correspondence tables 360 and 370
If the attribute is a symbolic attribute, replace the attribute value with the specified label code.If the attribute is a numeric attribute, classify it in the numerical range specified in the label code correspondence table and assign it to the label code. Perform the replacement process. Here, the difference from FIG. 12 in the first embodiment is that the case table is divided in the column direction and stored in a plurality of databases. That is, information of “name code” which is a key attribute designated as unique identification information is added. Each server processing device converts the locally managed case table into the case-attribute value correspondence table of FIG. 17, and then transmits the result as a search result to the client processing device in step 347.

The client processing device receives the processing results from the plurality of server processing devices (step 348), synthesizes the “name code” information, which is record identification information, as a key, and sets the case-attribute value correspondence of the entire case table. A table is created on the client processing device (step 349).

What is important here is that since the consistency of the two databases may not be managed, there may be a record that exists only in one of the case tables. In such a case, if an uncorresponding part in the synthesized case-attribute value correspondence table is filled with a special value representing a missing value, the case-attribute value correspondence for the entire case table based on the key attribute can be obtained. Tables can be combined. After the case-attribute value correspondence table is created, the rule induction process refers to only this table to extract rules (step 350), just as in the first embodiment.

The above description can be applied not only to the rule induction but also to the process of automatically generating a classification decision tree based on attributes such as ID3.
According to the above-described second embodiment of the present invention, when executing the rule induction process in the client processing device, the function of the DBMS (database management system) executed by the plurality of server processing devices is expanded in advance, and the client It becomes possible for the processing device to execute necessary search results in parallel according to a request from the client processing device. A search request to a plurality of server processing devices and the acquisition of the result need only be executed once prior to execution of the rule induction processing main body, and the amount of data flowing on the network is significantly reduced, and the network load is reduced. Becomes possible. Further, since the search processing itself is also executed in parallel, there is an effect that the entire processing time can be reduced. Further, from the user's point of view, since it is possible to execute the processing directly on the data of the table image on the client processing device and obtain the result, the case table is divided and stored in a plurality of databases. There is also the effect that work can proceed more naturally without being conscious of the fact.

The above effects are the same as those of the first embodiment.
Further, as an effect peculiar to the second embodiment, information divided and stored on a plurality of independent databases whose consistency is not guaranteed can be handled as if it were one virtual database. is there. By realizing the data processing in this manner, it is possible to perform processing such as rule induction using an interface that is exactly the same as the conventional statistical analysis processing and the construction of a neuro model or a fuzzy model by learning. Since various types of data processing can be freely executed and the results can be analyzed with the unified interface as described above, there is also an effect that the work efficiency of the user is improved.

[0047]

As described above, according to the present invention, when data processing such as rule induction is applied to data divided and stored in a plurality of databases, the data processing capability of the plurality of databases is provided. Can be used in parallel, so that high-speed processing can be performed. Further, it is not necessary to collect data itself used for data processing on a predetermined device (client processing device), and a single search process is required for a plurality of databases, so that the load on the network can be significantly reduced. There is an effect that can be. In addition, the user can proceed with the processing for virtually defined data in a table format, and need to be aware that the actual data is physically divided and stored in multiple databases. Since there is no data processing, there is also an effect that the data processing can be easily performed.

[Brief description of the drawings]

FIG. 1 is an overall configuration diagram of a data processing system using a plurality of databases to which the present invention is applied.

FIG. 2 is a diagram for explaining a method of dividing and storing data in a plurality of databases.

FIG. 3 is a diagram illustrating an example of a case table.

FIG. 4 is a diagram illustrating an example of a case table converted into a symbol attribute by labeling processing.

FIG. 5 is a diagram illustrating an example of a rule extracted by rule induction.

FIG. 6 is a diagram showing an example of a case-attribute value correspondence table.

FIG. 7 is a diagram showing a table component value-attribute value correspondence table.

FIG. 8 is a configuration diagram of a case table in credit assessment.

FIG. 9 is a diagram showing a case table division storage method.

FIG. 10 is a processing flowchart of the first embodiment of the present invention.

FIG. 11 is a diagram showing an example of a label code correspondence table.

FIG. 12 is a diagram illustrating an example of a case-attribute value correspondence table in the server processing device.

FIG. 13 is a diagram showing a division storage method of a case table by column division.

FIG. 14 is a diagram illustrating an example of an attribute association table.

FIG. 15 is a processing flowchart of a second embodiment of the present invention.

FIG. 16 is a diagram illustrating an example of a label code correspondence table.

FIG. 17 is a diagram illustrating an example of a case-attribute value correspondence table in the server processing device.

FIG. 18 is a diagram illustrating an example of a classification tree generated by ID3.

[Explanation of symbols]

 101 user 102 client processing device 103-106 server processing device 107-110 database

──────────────────────────────────────────────────続 き Continuation of the front page (56) References JP-A-64-82146 (JP, A) JP-A-3-156571 (JP, A) JP-A-5-2610 (JP, A) JP-A-5-210 101108 (JP, A) (58) Fields investigated (Int. Cl. ⁷ , DB name) G06F 17/30 G06F 9/44 G06F 12/00 JICST file (JOIS)

Claims (4)

(57) [Claims] 1. A data processing system comprising a client processing device and a plurality of server processing devices connected to the client processing device and each connected to a database, wherein the client processing device includes a table stored in the database. a classification rule specifying means for specifying the classification rules for classifying the value of the data items included in the form of data and a first transmission means for transmitting the specified the classification rule to the server processing unit, the said server processor transmits a classification executing means for classifying the data stored in each database based on the transmitted said classification rules, the result of the classification by the classification execution means to the client processing unit and a second transfer means, wherein the client apparatus further transmitted the <br/> And means for combining the class results, executes the rule induction based on the classification results are combined by the combining means
And a means for outputting a result extracted by the rule induction executing means . 2. The client processing device, wherein the client
Connected to the central processing unit, each connected to the database.
A data processing system consisting of multiple connected server processors
In Temu, the client processing unit is stored in the database
Case table and replace numeric attributes with symbolic attributes
Label code correspondence table and the case table
Classification processing request to classify based on the label code correspondence table
Is transmitted to the plurality of server processing devices.
Means for receiving the classification processing request.
Then, the attribute values described in the case table are
By classifying based on the label code correspondence table,
Means for generating an example-attribute value correspondence table;
Second transmission for transmitting a response table to the client processing device
Means, and the client processing device further comprises:
Means for synthesizing the result of the attribute value correspondence table, and the synthesized result
Means to run the rule induction process on the basis of the
Extracted by the rule induction executing means.
An output unit for outputting a result . 3. The classification rule specifies an upper limit value and a lower limit value for a data item having a numeric value, and specifies character string conversion for a data item having a character string as a value. The data processing system according to claim 1 or 2, wherein Wherein said analyzing means, the data processing system of claim 1, wherein the outputting the relationships between data items, as at least one rule.