JP2013196609A - Data analysis support device and data analysis support processing program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To improve precision of analysis when data on different organizations are integrated even if a deficit due to differences in data attribute among the respective organization is caused.SOLUTION: A data analysis support device includes data-by-organization storage means of storing data tables by a plurality of organizations for storing records as data having at least one kind of attribute to be analyzed with respect to the plurality of organizations; distance calculation means of calculating distances between groups of records, shown by the data-by-organization tables, of the plurality of organizations having the same attribute on the basis of a value of the common attribute by using the number of attributes common among the records and the total value of the common attributes; and analysis processing means of taking an analysis on the basis of the calculated distances.

Description

  Embodiments described herein relate generally to a data analysis support apparatus and a data analysis support processing program that support analysis of data related to different organizations.

  Conventionally, organizations that have office error data with the same tendency have been clustered by integrating and analyzing each of office error data, which is aggregated data related to different organizations such as multiple types of financial institutions, for example. Yes.

  Here, if there is an attribute that does not match between the aggregated data of a specific organization and the aggregated data of another organization among the aggregated data of different organizations, for example, the specific organization mentioned above aggregates information on the cause of the administrative error. On the other hand, if other organizations do not aggregate this cause information, the cause information related to administrative errors in other organizations described above is analyzed by integrating the aggregated data of multiple organizations. This is missing information.

  Conventionally, as a countermeasure to missing information, from the aggregated data group accumulated in the database, the missing data in which some feature defects exist is detected, and the aggregated data group is free of the missing data and the missing data. The data is divided into normal data, normal data similar to the missing data is obtained using a predetermined similarity scale, and the missing data is obtained using the feature data corresponding to the missing features of the missing data in the obtained normal data as complementary data. Some of them are substituted into features and complemented.

JP 2002-215646 A

  As described above, in the method of obtaining normal data similar to missing data using a predetermined similarity scale, if there is a deficiency in the aggregated data of a certain organization, the aggregated data of another organization is used for the deficient data of that organization. Therefore, the reliability of the aggregated data after the supplementation is not necessary and sufficient, and the accuracy of the analysis was not sufficient.

  The problem to be solved by the present invention is to improve the accuracy of analysis when these data are integrated, even if a defect occurs due to the difference in data attributes among the data of different organizations. Another object of the present invention is to provide a data analysis support device and a data analysis support processing program that can be executed.

  According to the embodiment, the data analysis support apparatus stores an organization-specific data table for managing records, which are data having at least one type of attribute, for each of a plurality of organizations to be analyzed. Based on the value of the common attribute for each record set of a plurality of organizations having at least one type of common attribute between the plurality of organizations shown in the separate data table storage means and the organization-specific data table , Based on the number of attributes common to the records, and a distance calculation means for calculating a distance between the record sets based on the total value in the common attributes, based on the distance calculated by the distance calculation means And analysis processing means for performing analysis.

The block diagram which shows the function structural example of the data analysis assistance apparatus in embodiment. The figure which shows an example of the office work mistake data of the branch of a some bank in a table format. The figure which shows an example of the data of the number of office work mistakes totaled according to the branch based on the office work mistake data of the branch of a plurality of banks in a tabular form. The figure which shows an example of the flow of the processing data by the data analysis assistance apparatus in embodiment. The flowchart which shows an example of the procedure of the processing operation of the data analysis assistance apparatus in embodiment. The flowchart which shows an example of the processing operation for the same attribute extraction by the data table coupling | bond part 41 of the data analysis assistance apparatus in this embodiment. The figure which shows an example of the combined data table produced | generated by the data table coupling | bond part of the data analysis assistance apparatus in this embodiment in a table format. The flowchart which shows an example of the processing operation by the distance calculation part between records of the data analysis assistance apparatus in this embodiment. The flowchart which shows an example of the processing operation by the clustering implementation part of the data analysis assistance apparatus in this embodiment. The figure which shows the example of a setting of the initial set of a cluster center branch. The figure which shows the example which matched each branch with the cluster center branch. The figure which shows an example of the attribute and attribute value of each organization which are contained in the predetermined cluster defined with the combined data table in a table format. The figure which shows an example of the calculation result of the gravity center of each attribute of each structure | tissue included in the predetermined cluster defined by the combined data table in a table format. The figure which shows an example of the gravity center of each cluster. The figure which shows an example of the recalculation result of the cluster center branch of the cluster defined by the combined data table in a table format. The figure which shows the experimental data utilized for evaluation of the precision of clustering in a tabular form. The figure which shows the administrative error collection situation of each bank used for evaluation of the accuracy of clustering in a tabular form. The figure which shows the experimental data containing a missing item utilized for evaluation of the precision of clustering in a table form. The figure which shows the correct answer rate of the number of the branches actually classified into each cluster in a table form.

Hereinafter, embodiments will be described with reference to the drawings.
In this embodiment, an organization-specific data table for managing records, which are aggregated data having at least one type of attribute, for each of a plurality of organizations to be analyzed is stored by organization and is shown in the organization-specific data table. For each record set of a plurality of organizations having at least one type of common attribute among the plurality of organizations, based on the value of the common attribute, the number of attributes common between the records, and the common Based on the total value in the attribute, the distance between the record sets is calculated, and clustering is performed on the basis of the calculated distance, with the organization corresponding to each record as a cluster.

FIG. 1 is a block diagram illustrating a functional configuration example of a data analysis support apparatus according to the embodiment. As shown in FIG. 1, the data analysis support apparatus 10 according to the embodiment includes a control unit 11 that controls processing operations of the entire apparatus, a storage device 12, a data table combining unit 41, an inter-record distance calculation unit 42, and a clustering execution unit 43. Have. The data table combination unit 41, the inter-record distance calculation unit 42, and the clustering execution unit 43 are processing units that are executed by software on a microprocessor. Information is exchanged between the units via the storage device 12 as shown in FIG. It is possible to give and receive.
Among these, the inter-record distance calculation unit 42 has a prominent feature compared to the prior art and has a main function for solving the problem.

  The storage device 12 is a storage medium such as a nonvolatile memory, and includes an organization-specific data table storage unit 31, a combined data table storage unit 32, an inter-record distance storage unit 33, and a clustering result storage unit 34.

  In the present embodiment, an example will be described in which office error data of each branch of a bank, which is an analysis target organization for clustering, is analyzed. The data analysis support apparatus 10 clusters each branch of a plurality of banks based on the feature of the occurrence of an office error, using data in which a missing value is generated by combining the aggregated data totaled at each branch of each bank.

  Banks of each bank accumulate information as administrative error data that indicates when, in what business, and who made what mistakes, such as mistakes in daily operations, such as incorrect fees and incorrect account number designation. ing.

The information on who made the mistake described above is information indicating what title / title the employee who made the mistake has made.
Further, the above-described information indicating what kind of mistake has occurred is information indicating what is the cause of the mistake and what is the amount of loss.
Although the attributes of information collected regarding administrative errors are generally the same between the lines, there are attributes specific to a specific bank, and a specific attribute is not necessarily collected at all banks.

  FIG. 2 is a diagram illustrating an example of office error data of branch offices of a plurality of banks in a table format. FIG. 2 shows administrative error data generated at each branch of three banks such as A bank, B bank, and C bank. These A bank, B bank, and C bank all collect the date of occurrence of the administrative error, the branch that has occurred, and the business in which the error occurred. In this case, the office error data of each row has the occurrence date, the occurrence branch, and the error occurrence business as the same attributes.

  On the other hand, the positions of those who have made administrative errors are collected by Banks A and C, but not by Bank B. In this case, the business error data of bank B does not have the attribute “title of the person who generated”, and the value of this attribute is a missing value.

  The causes of administrative errors are collected by Bank A and Bank B, but not by Bank C. In this case, the business error data of bank C does not have the attribute “cause of occurrence of error”, and the value of this attribute is a missing value.

  In FIG. 2, for the sake of convenience, the missing value of the missing attribute is clearly indicated as “NULL”, but in reality, in the administrative error data for each bank, the attribute itself does not exist for the uncollected data.

FIG. 3 is a diagram illustrating an example of data on the number of office errors counted for each branch based on office error data of a plurality of bank branches in a table format.
With respect to the administrative error data shown in FIG. 2, the calculation of the number of mistakes by bank branch of each bank will be described. Here, for simplification of explanation, only “error occurrence business”, “occurrence title”, and “error occurrence cause” in the office error data shown in FIG.

  In this administrative error data, in bank A, the “branch number” indicating the branch number of each branch of the bank A, the attribute “business: deposit” indicating that the business error has occurred is a deposit, and the business error has occurred. "Business: Loan" attribute that indicates a loan, attribute "Position: General employee" that indicates that the position of the office error occurrence person is a general employee, and that the position of the office error occurrence person is a part Attribute “Position: Part”, Attribute “Cause: Insufficient Capability” indicating that the cause of the administrative error is insufficient ability, Attribute “Cause: Miss” indicating that the cause of the administrative error is human error Are aggregated for each branch.

  For example, the value “31” in the column of the attribute “business: deposit” defined in the row of the branch number “A001” in the organization-specific data table shown in FIG. This indicates that out of office errors that occurred in a certain branch, the number of office errors in which the business is a deposit is 31.

  Further, assuming that the column of the attribute “title: general employee” is defined in the row of the organization-specific data table branch number “B001” shown in FIG. 3, the value of the cell of this column is “null”. " As described above, in fact, in the administrative error data for each bank, there is no attribute itself for the data that is not collected, so the value of the attribute that is not collected in each bank is also included in the result of counting the number of administrative errors. It does not appear. In FIG. 3, among the attribute names in the organization-specific data table of each bank, the attribute names that do not exist are shown lightly and the number of cases is “null”, but in reality, such attribute columns themselves exist. do not do.

  In the present embodiment, data on the number of mistakes related to each attribute aggregated for each branch is stored as an organization-specific data table in the organization-specific data table storage unit 31 of the storage device 12 shown in FIG. In the example shown in FIG. 3, the attributes of the aggregated data of each branch of the same bank are the same, and the data tables related to these branches are classified by bank as a group of organization-specific data tables.

FIG. 4 is a diagram illustrating an example of a flow of processing data by the data analysis support device in the embodiment.
The data table coupling unit 41 takes in the organization-specific data table of each branch of each bank, which is stored in the organization-specific data table storage unit 31 of the storage device 12, as input data. Further, the data table combining unit 41 specifies the same attribute between organizations, that is, between branches, from the attributes in the organization-specific data table, and combines the data tables of each organization based on the specified attributes. One combined data table is generated and stored in the combined data table storage unit 32 of the storage device 12.

  Further, the inter-record distance calculation unit 42 asks for any two records, that is, bank types, when a set of data on the number of mistakes for each attribute of one branch in the combined data table is set as one record. A distance indicating the level of similarity between records is calculated for data on the number of mistakes of each attribute relating to two branches that are not present, and the calculation result is stored in the inter-record distance storage unit 33 of the storage device 12.

  The clustering execution unit 43 clusters the records in the combined data table using the information on the distance between records stored in the inter-record distance storage unit 33, and the clustering result is stored in the clustering result storage unit 34 of the storage device 12. And output to the display device 20 such as a liquid crystal display device.

FIG. 5 is a flowchart illustrating an example of a processing operation procedure of the data analysis support apparatus according to the embodiment. The procedure described here is an outline of the processing operation, and details of each processing will be described later.
First, the data table combining unit 41 of the data analysis support device 10 extracts each attribute of the organization-specific data table stored in the organization-specific data table storage unit 31 of the storage device 12 (step S1).

  The data table combining unit 41 extracts the same attribute that is the same attribute between organizations from the organization-specific data table of each organization (step S2). As an example of a method for specifying the same attribute to be extracted, there is a method for detecting a complete match of attribute names between organization-specific data tables of each bank as shown in FIG.

  The data table combining unit 41 combines the organization-specific data tables stored in the organization-specific data table storage unit 31 of the storage device 12 using the same attribute extracted in step S2 to create a single combined data table. Generated and stored in the combined data table storage 32 of the storage device 12 (step S3). If there is an attribute that exists only in some organization-specific data tables, the data table combining unit 41 adds the attribute to the data table of the organization that does not have the attribute, and sets the attribute value of the added attribute as a missing value. (Null).

  The inter-record distance calculation unit 42 selects any two records from the records in the combined data table stored in the combined data table storage unit 32 of the storage device 12, and calculates the distance between the selected records. Calculate (step S4).

  As a remarkable feature compared with the prior art, in the present embodiment, the inter-record distance calculation unit 42 has an attribute whose value is a missing value in at least one of the selected two record attributes. The distance between the records is calculated only for the attributes that are excluded from the target and have both the values of the two records. The inter-record distance calculation unit 42 stores the calculated distance information in the inter-record distance storage unit 33 of the storage device 12. The inter-record distance calculation unit 42 performs this process for all combinations of two records in the combined data table.

  The clustering execution unit 43 performs clustering of each branch by clustering the records in the combined data table using the information on the distance between records stored in the inter-record distance storage unit 33 of the storage device 12. (Step S5). Then, the clustering execution unit 43 stores the clustering result in the clustering result storage unit 34 of the storage device 2 and outputs it to the display device 20 (step S6).

Next, details of the operation of the data table combining unit 41 will be described.
As described above, the data table combining unit 41 extracts attributes from the organization-specific data table stored in the organization-specific data table storage unit 31 of the storage device 12, identifies the same attribute between organizations, Join tables.

FIG. 6 is a flowchart illustrating an example of a processing operation for extracting the same attribute by the data table combining unit 41 of the data analysis support apparatus according to this embodiment.
The processing operation shown in FIG. 6 explains step S2 of the processing operation shown in FIG. 5 in detail, and is a processing operation for extracting the same attribute between organizations.

  Taking the organization-specific data table shown in FIG. 3 as an example, the data table combining unit 41, if there is an attribute having the same attribute name in each organization-specific data table of each branch of a different bank, Extract as the same attribute.

  The data table combining unit 41 reads the bank-specific data table from the organization-specific data table storage unit 31 of the storage device 12, and generates an attribute set T including all the attributes of all banks (step S11).

  Specifically, in step S11, the elements of the attribute set T obtained by the data table combining unit 41 from the organization-specific data table shown in FIG. 3 are the following 14 attributes.

“Business: Deposit (Bank A)”, “Business: Deposit (Bank B)”, “Business: Deposit (Bank C)”
"Business: Financing (Bank A)", "Business: Financing (Bank B)", "Business: Financing (Bank C)"
“Position: General Employee (Bank A)”, “Position: General Employee (Bank C)”
“Position: Part (Bank A)”, “Position: Part (Bank C)”
“Cause: Insufficient capacity (Bank A)”, “Cause: Insufficient capacity (Bank B)”
"Cause: Miss (Bank A)", "Cause: Miss (Bank B)"
Here, even if they have the same attribute name, they are counted as different attributes if the banks from which the attribute values are aggregated are different. For example, the attribute names “business: deposit (bank A)”, “business: deposit (bank B)”, and “business: deposit (bank C)” are “business: deposit” excluding the bank name. These “business: deposit (bank A)”, “business: deposit (bank B)”, and “business: deposit (bank C)” are individual elements in the attribute set T.

The data table combining unit 41 extracts an arbitrary attribute from the attribute set T generated in step S11 (step S12). This extracted attribute is referred to as attribute a.
If there is an attribute having the same attribute name as the attribute a extracted in step S12 among the attributes in the attribute set T, the data table combining unit 41 selects the attribute value regardless of the type of bank from which the attribute value is aggregated. Extract (step S13). The extracted attributes are referred to as attribute b, attribute c,.

  Specifically, when “business: deposit (bank A)” is extracted as attribute a in step S12, the data table combining unit 41 has the same attribute name “business: deposit” as attribute a in step S13. The attributes “business: deposit (bank B)” and “business: deposit (bank C)” are extracted as attributes b and c.

The data table combining unit 41 stores the information of the attributes a, b, c,... Extracted in step S12 and step S13 in the storage device 12 as the same attribute (step S14).
Here, an attribute having the same attribute name as the attribute a extracted from the attribute set T does not exist in the attribute set T, that is, the attribute a exists only in the organization-specific data table of a single organization. If the data does not exist in the organization-specific data table, the data table combining unit 41 stores only the attribute a in the storage device 12 as the same attribute described above.

  The data table combining unit 41 includes an attribute that has not yet been processed in step S12 or step S13 in the attribute set T generated in step S11. In other words, the attribute in the attribute set T is extracted as attribute a in step S12. If there are attributes that are not extracted as attributes b, c,... In step S13 (YES in step S15), the process returns to step S12 to return attributes that have not been extracted from the attribute set T. Any one is extracted as a new attribute a, and the processes in steps S13 and S14 are performed again for the new attribute a extracted in step S12.

  Further, when all the attributes in the attribute set T generated in step S11 are processed, the data table combining unit 41 has extracted all the attributes in the attribute set T as the attribute a in step S12. In this case, or when extracted as attributes b, c,... In step S13 (NO in step S15), the process for extracting the same attribute is terminated.

  When the data table combining unit 41 completes the process of extracting the same attribute, the attribute set obtained by the data table combining unit 41 as the same attribute from the organization-specific data table shown in FIG. There are six pairs (i), (c), (d), (e), and (f).

(A): “Business: Deposit (Bank A)”, “Business: Deposit (Bank B)”, “Business: Deposit (Bank C)”
(I): "Business: Loan (Bank A)", "Business: Finance (Bank B)", "Business: Finance (Bank C)"
(C): “Position: General Employee (Bank A)”, “Position: General Employee (Bank C)”
(D): “Position: Part (Bank A)”, “Position: Part (Bank C)”
(E): “Cause: Insufficient capacity (Bank A)”, “Cause: Insufficient capacity (Bank B)”
(F): “Cause: Miss (Bank A)”, “Cause: Miss (Bank B)”
Next, the processing operation for combining the organization-specific data tables using the same attribute stored in the storage device 12 in step S14 by the data table combining unit 41 will be described below.
When the same attribute is extracted for the organization-specific data tables of all organizations from the organization-specific data table, the data table combining unit 41 incorporates this attribute into the combined data table and sets it as one attribute in the combined database. .

  Specifically, in the group (a) of the same attributes extracted from the organization-specific data table shown in FIG. 3, the “business: deposit” attribute is included in the organization-specific data table of each branch of banks A, B, and C. Exist as the same attribute, and the attribute of “business: deposit” exists in the organization-specific data table for all banks, so this attribute is incorporated into the combined data table and The attribute is “business: deposit”.

  Similarly, in the group (b) of the same attribute extracted from the organization-specific data table shown in FIG. 3, “business: loan” is included in the organization-specific data table of each branch of banks A, B, and C. The attribute exists as the same attribute, and since this “business: loan” attribute exists in the organization-specific data table for all banks, this attribute is incorporated into the combined data table and the attribute in the combined database. “Business: Loan”.

  Further, from the organization data table shown in FIG. 3, the same attributes extracted from the organization data table of each branch of some banks are not extracted from the organization data table of each branch of other banks. If this is the case, the attribute is added to the attribute of the other bank to make it one attribute in the combined data table. At this time, all the attribute values of the added attributes described above in the other bank described above in the combined data table are set as missing values.

  Specifically, in the same attribute set (c) extracted from the organization-specific data table shown in FIG. 3, the organization-specific data table of each branch of banks A and C has the attribute “title: general employee”. Although it has the same attribute, the “position: general employee” attribute does not exist in the organization-specific data table of each branch of bank B.

  Therefore, the data table combining unit 41 incorporates the data table classified by organization of each branch of the B bank with “position: general employee” added to the combined database, and stores the line of each branch of the B bank in this combined database. All attribute values that are cell values in the column “Position: General Bank” are missing values.

  Further, in the group (d) of the same attribute extracted from the organization data table shown in FIG. 3, the “position: part” attribute exists as the same attribute in the organization data table of each branch of the A and C banks. However, the “position: part” attribute does not exist in the organization-specific data table of each branch of the B bank. Therefore, the data table combining unit 41 incorporates the data table classified by organization of each branch of the bank B with “position: part” added to the combined database, and stores “ All attribute values that are cell values in the column “Position: Part” are assumed to be missing values.

Further, in the same attribute set (e) extracted from the organization-specific data table shown in FIG. 3, the attribute “Cause: Insufficient Capability” is assumed to be the same attribute in the organization-specific data table of each branch of banks A and B. Although it exists, this “Cause: Insufficient Capability” attribute does not exist in the organization-specific data table of each branch of bank C.
Therefore, the data table combining unit 41 incorporates the data table classified by organization of each branch of bank C with “cause: insufficient capacity” added to the combined database, and stores the row of each branch of bank C in this combined database. All attribute values that are cell values in the column “Cause: Insufficient ability” are assumed to be missing values.

Further, in the same attribute group (f) extracted from the organization data table shown in FIG. 3, the attribute “Cause: Miss” exists as the same attribute in the organization data table of each bank of A and B banks. However, this “Cause: Miss” attribute does not exist in the organization-specific data table of each branch of bank C.
Therefore, the data table combining unit 41 incorporates the organization-specific data table of each branch of the C bank with “Cause: Miss” added to the combined database, and stores the “ All attribute values that are cell values in the “Cause: Miss” column are missing values.

  In this way, the data table combining unit 41 generates a single combined data table by combining the organization-specific data tables of the branches of each bank shown in FIG. Store in the data table storage unit 32.

FIG. 7 is a diagram illustrating an example of a combined data table generated by the data table combining unit of the data analysis support apparatus according to the present embodiment in a table format.
Each row of this combined data table corresponds to one record of each branch of each bank, each column has a branch number of each row, and “business: deposit”, which is each attribute in the organization-specific data table of the combination source, Corresponds to “Business: Loan”, “Job title: General employee”, “Job title: Part”, “Cause: Insufficient ability”, “Cause: Mistake”.

  For example, the cell value in the column of the attribute “business: deposit” defined in the row of the branch number “A001” in the organization-specific data table of each branch of the bank A shown in FIG. The value of the cell in the column of the attribute “business: deposit” defined in the row of the branch number “A001” in the completed data table is also “31”.

  In addition, there are cell values in columns of the attributes “title: general employee” and “title: part” defined in the row of the branch number “B001” in the organization-specific data table of each branch of the bank B shown in FIG. Therefore, the value of the cell in the column of “position: general employee” or “position: part” defined in the row of the branch number “B001” in the combined data table is “null”.

  In addition, there is a cell value in the column of the attribute “cause: insufficient ability” or “cause: miss” defined in the row of the branch number “C001” of the organization-specific data table of each branch of bank C shown in FIG. Therefore, the value of the cell in the column of the attribute “cause: insufficient capacity” or “cause: miss” defined in the row of the branch number “C001” of the combined data table is “null”.

Next, details of the operation of the inter-record distance calculation unit 42 will be described.
FIG. 8 is a flowchart illustrating an example of a processing operation performed by the inter-record distance calculation unit of the data analysis support device according to this embodiment.
The processing operation shown in FIG. 8 explains step S4 of the processing operation shown in FIG. 5 in detail, and any two records when a set of cells along the row direction in the combined data table is one record. This is a processing operation for calculating a distance indicating the level of similarity between the two.

  The inter-record distance calculation unit 42 arbitrarily designates a record pair (record i and record j) that is a set of two records in the combined data table (step S21), and each of the record pairs has a value. A common attribute that is an attribute is specified (step S22).

Next, the inter-record distance calculation unit 42 calculates the distance d _{i, j} between the record i and the record j according to the following equation (1) in consideration of the common attribute specified in step S22. The calculated distance information is stored in the inter-record distance storage unit 33 of the storage device 12 together with the identification name information of each record of the record pair (step S23).

  N in the formula (1) is the number of common attributes between the record i and the record j.

  C in Equation (1) is a set of common attributes between record i and record j.

  In Expression (1), “a” is an attribute.

A _k in equation (1) is the attribute value of attribute a in record k.

  A specific example will be described. First, as a first example, when a pair of the record of the branch number “A001” and the record of the branch number “A002” shown in FIG. 7 is selected, the record of the “A001” line is missing. There are six attributes with non-value values: “Business: Deposit”, “Business: Loan”, “Job title: General employee”, “Job title: Part”, “Cause: Insufficient ability”, “Cause: Mistake”. . Similarly to the record of the “A001” row, the record of the “A002” row has the attributes having a value that is not a missing value: “business: deposit”, “business: loan”, “title: general employee”, “Position: Part”, “Cause: Insufficient ability”, and “Cause: Miss”.

  That is, since both records have six attributes, and these are all common attributes, the common attributes specified in step S21 are “business: deposit”, “business: loan”, and “title: general”. There will be six: “Employee”, “Position: Part”, “Cause: Insufficient ability”, and “Cause: Mistake”.

In this case, the inter-record distance calculation unit 42 calculates the Euclidean distance using the six attributes that are common attributes, and the following value obtained by dividing the distance by the number of common attributes 6 according to the equation (1) is the branch number “A001”. ”And the record of the branch number“ A002 ”.

  Further, as a second example, when a pair of the record of the branch number “A001” and the record of the branch number “B001” shown in FIG. 7 is selected, the record of the “A001” line is missing. There are six attributes with non-value values: “Business: Deposit”, “Business: Loan”, “Job title: General employee”, “Job title: Part”, “Cause: Insufficient ability”, “Cause: Mistake”. . On the other hand, in the record of the row “B001”, there are four attributes having a value that is not a missing value: “business: deposit”, “business: loan”, “cause: insufficient ability”, and “cause: miss”.

  That is, both records have values that are not missing values for the four attributes of “business: deposit”, “business: loan”, “cause: insufficient ability”, and “cause: miss”. The common attribute whose attribute is specified in step S21 is as follows.

  On the other hand, with regard to the two attributes “title: general employee” and “position: part” having values in the record of the line “A001”, the record of the line “B001” has a missing value. Does not become a common attribute specified in step S21.

In this case, the inter-record distance calculation unit 42 calculates the Euclidean distance using the four attributes that are common attributes, and the following value obtained by dividing this distance by the number of common attributes 4 according to the equation (1) is the branch number “A001”. ”And the record of the branch number“ B001 ”.

  Further, as a third example, when a pair of the record of the branch number “B001” and the record of the branch number “C001” shown in FIG. 7 is selected, the record of the “B001” line is missing. There are four attributes having values other than values: “business: deposit”, “business: financing”, “cause: insufficient ability”, and “cause: mistake”. On the other hand, in the record of the row “C001”, there are four attributes having a value that is not a missing value: “business: deposit”, “business: financing”, “title: general employee”, and “title: part”.

  That is, both records have values that are not missing values for the two attributes “business: deposit” and “business: loan”, and these attributes are common attributes specified in step S21.

  On the other hand, regarding the two attributes “title: general employee” and “position: part” having values in the record of the line “C001”, the record of the line “B001” has a missing value. Does not become a common attribute specified in step S21. In addition, regarding the two attributes “title: general employee” and “position: part” having values in the record of the “B001” row, the record of the “C001” row has a missing value. Is not a common attribute specified in step S21.

In this case, the inter-record distance calculation unit 42 calculates the Euclidean distance using the two attributes that are common attributes, and the following value obtained by dividing the distance by the number of common attributes 2 according to the equation (1) is the branch number “B001”. ”And the record of the branch number“ C001 ”.

  In other words, in the calculation of the distance between each branch in this embodiment, the number of terms to be added increases as the number of common attributes increases as in the conventional technique, and the distance value that is the sum of these increases unreasonably. Is preventing.

  Furthermore, in the present embodiment, as shown in Expression (1), the greater the difference in attribute values per attribute, the greater the calculated distance, and the greater the number of common attributes, the greater the calculated value. Since the distance becomes smaller, it is possible to calculate a distance with higher accuracy than in the prior art.

  After the process in step S23 for a certain record pair, the inter-record distance calculation unit 42 determines that the distance between records for all record pairs on the combined data table has not been completed (NO in step S24). ), Returning to step S21, a new pair of two records in the combined data table is arbitrarily designated, and the processes of steps S22 and S23 are performed again.

Also, the inter-record distance calculation unit 42 calculates the distance between records when the calculation of the distance between records for all record pairs on the combined data table is completed (YES in step S24). The process ends.
In this way, the inter-record distance calculation unit 42 calculates the inter-record distance for all record pairs on the combined data table.

Next, details of the operation of the clustering execution unit 43 will be described.
FIG. 9 is a flowchart illustrating an example of a processing operation performed by the clustering execution unit of the data analysis support apparatus according to this embodiment.
The processing operation shown in FIG. 9 explains step S5 of the processing operation shown in FIG. 5 in detail. The combined data table stored in the combined data table storage unit 32 of the storage device 12 is read, and this table This is a processing operation for clustering all the branches indicated by the branch number column.

  In the following, the process of clustering each branch of each bank is shown in a two-dimensional map. In the present embodiment, in order to cluster each branch, a cluster central branch is set, a cluster is set by associating a branch having a short distance to the cluster central branch, and then the center of gravity of the cluster is obtained. The branch closest to the center of gravity is set as a new cluster center branch, and if the cluster center branch before and after obtaining the center of gravity is the same, the clustering result is output assuming that correct clustering has been performed.

FIG. 10 is a diagram illustrating an example of setting an initial set of cluster central branches.
In the two-dimensional map shown in FIG. 10, each branch indicated by the branch number of each row in the combined data table is represented by a circle. In this two-dimensional map, the distance between branches represents the distance stored in the inter-record distance storage unit 33 of the storage device 12.

  The clustering execution unit 43 randomly selects the same number of branches as the number of clusters designated in advance, and sets each selected branch as a cluster center branch (step S31).

  For example, when the number of clusters is specified as 3, the clustering execution unit 43 sets each of the three black circled branches shown in FIG. 10 as the cluster center branch, taking FIG. 10 as an example.

  Next, the clustering execution unit 43 arbitrarily selects one of the branches other than the cluster central branch set in step S31 (step S32), and sets the distance between the selected branch and each cluster central branch, The data is read from the inter-record distance storage unit 33 of the storage device 12 and referred to (step S33), and a cluster is arbitrarily generated by associating the selected branch with the closest cluster center branch among the cluster center branches ( Step S34). The generated cluster elements are the cluster central branch and the other branch associated with the cluster central branch.

FIG. 11 is a diagram illustrating an example in which each branch is associated with a cluster central branch.
In the example shown in FIG. 11, three clusters such as a first cluster, a second cluster, and a third cluster are shown.

The first cluster is a cluster composed of three branches that are associated with two branches that are closest to the first cluster central branch 51 among the cluster central branches shown in FIG.
The second cluster is a cluster surrounded by a double line L1 that is composed of four branches that are associated with the three branches closest to the second cluster central branch 52 among the cluster central branches shown in FIG. is there.
The third cluster is a cluster composed of six branches that are associated with five branches that are closest to the third cluster central branch 53 among the cluster central branches shown in FIG.

  Subsequently, the clustering execution unit 43 calculates the center of gravity of each cluster generated in step S34 if all branches other than the cluster central branch for generating the cluster have been selected (YES in step S35). (Step S36).

Here, the description will focus on the second cluster consisting of four branches surrounded by the double line L1 in FIG.
FIG. 12 is a diagram showing an example of attributes and attribute values of each organization included in a predetermined cluster defined in the combined data table in a table format. In this figure, from the combined data table shown in FIG. 7, the attributes of the records of the four branches included in the second cluster and the attribute values of the attributes are shown.

  The four branches included in the second cluster correspond to the branch corresponding to the branch number “A003”, the branch corresponding to the branch number “A004”, and the branch number “B003” in the combined data table illustrated in FIG. This is the branch corresponding to the branch and branch number “C003”.

Specifically, in step S33, the clustering execution unit 43 calculates an average value of attribute values of the same attribute of each record for each attribute of the records corresponding to the four branches included in the second cluster, The average value is used as the attribute value of the center of gravity.
However, if there is a record whose attribute value is a missing value among the records related to the center of gravity to be calculated, the attribute value of this record is excluded from the average value calculation, and the average value only for the records having the attribute value Is calculated.

  For example, the average value of the “business: deposit” attribute value of each record shown in FIG. 12, that is, the attribute value of the center of gravity of “business: deposit” is the value “5” in the row of the branch number “A003”, the branch number. The sum of the value “2” in the row “A004”, the value “3” in the row of the branch number “B003”, and the value “3” in the row of the branch number “C003” is set to “business: deposit” in each record. The attribute value is a value divided by the number of records “4” existing as non-missing values, as shown in the following formula.

(5 + 2 + 3 + 3) /4=3.25
Further, the average value of the attribute value of “business: loan” of each record shown in FIG. 12, that is, the attribute value of the center of gravity of “business: loan” is the value “3” in the row of the branch number “A003”, the branch number The sum of the value “5” in the row “A004”, the value “4” in the row of the branch number “B003”, and the value “3” in the row of the branch number “C003” The attribute value is a value divided by the number of records “4” existing as non-missing values, as shown in the following formula.

(3 + 5 + 4 + 3) /4=3.75
Further, the average value of the attribute values of “title: general employee” of each record shown in FIG. 12, that is, the attribute value of the center of gravity of “title: general employee” is the value “2” in the row of the branch number “A003”, A record in which the sum of the value “4” in the row of the branch number “A004” and the value “4” in the row of the branch number “C003” is present as a value in which the attribute value of “title: general” is not a missing value in each record This is the value divided by the number “3”, and is given by the following formula.

  (2 + 4 + 4) /3≈3.33.

  Further, the average value of the attribute values of “position: part” of each record shown in FIG. 12, that is, the attribute value of the center of gravity of “position: part” is the value “6” in the row of the branch number “A003”, the branch number. The sum of the value “3” in the row of “A004” and the value “2” in the row of the branch number “C003” is used as the number of records in which the attribute value of “title: part” is not a missing value among the records “ This is the value divided by 3 ”, as shown in the following equation.

(6 + 3 + 2) /3≒3.67
Further, the average value of the attribute values of “cause: insufficient ability” of each record shown in FIG. 12, that is, the attribute value of the center of gravity of “cause: insufficient ability” is the value “3” in the row of the branch number “A003”, The sum of the value “3” in the row of the branch number “A004” and the value “2” in the row of the branch number “B003” exists as a value in which the attribute value “cause: insufficient ability” is not a missing value in each record. This is the value divided by the number of records “3”, as shown in the following formula.

(3 + 3 + 2) /3≒2.67
Also, the average value of the attribute values of “Cause: Miss” of each record shown in FIG. 12, that is, the attribute value of the center of gravity of “Cause: Miss” is the value “5” in the row of the branch number “A003”, the branch number Number of records in which the sum of the value “4” of the row “A004” and the value “5” of the row of the branch number “B003” is present as a value in which the attribute value of “cause: insufficient ability” is not a missing value This is the value divided by “3”, as shown in the following equation.

(5 + 4 + 5) /3≒4.67
FIG. 13 is a diagram illustrating an example of a calculation result of the center of gravity of each attribute of each organization (branch) included in a predetermined cluster defined in the combined data table in a table format.
FIG. 14 is a diagram illustrating an example of the center of gravity of each cluster.
In FIG. 14, the centroid of the first cluster, the centroid of the second cluster, and the centroid of the third cluster on the two-dimensional map are indicated by “x” marks.

  Finally, the clustering execution unit 43 recalculates the cluster center branch of each cluster (step S37). Specifically, the clustering execution unit 43 calculates a branch having the smallest distance from the center of gravity calculated in step S36 for the predetermined cluster among all branches including the cluster central branch set in step S31. A branch is set as a new cluster center branch, and this new cluster center branch is set for each of the cluster center branches set in step S31, thereby newly setting a cluster center set.

  In calculating the distance between each branch and the center of gravity of the cluster, the above formula (1) is used as in the calculation of the distance between the branches.

FIG. 15 is a diagram illustrating an example of a recalculation result of a cluster central branch of a cluster defined in the combined data table in a table format.
If the cluster center branch set recalculated in step S37 has changed from the original cluster center branch set set in step S31 (YES in step S38), the clustering execution unit 43 performs appropriate clustering. Is returned to step S32, one of the branches other than the cluster central branch recalculated in step S37 is arbitrarily selected, and the processes in and after step S33 based on this cluster central branch are performed. Try again.

  In the example shown in FIG. 15, since the set of cluster central branches has changed from the state shown in FIG. 10, the process returns to step S32. Specifically, as shown in FIG. 15, the cluster central branch of the first cluster is changed from the initial cluster central branch 51 to the cluster central branch 61, and the cluster central branch of the second cluster is changed to the original cluster. The central branch 52 changes to the cluster central branch 62, and the cluster central branch of the third cluster changes from the initial cluster central branch 53 to the cluster central branch 63.

  In addition, the clustering execution unit 43 determines that the cluster center branch set recalculated in step S37 does not change from the original cluster center branch set set in step S31 (NO in step S38). Thus, the clustering process is terminated, the clustering result is stored in the clustering result storage unit 34 of the storage device 12, and output to the display device 20 such as a liquid crystal display device, for example.

Next, the result of an experiment for applying the present embodiment to actual data and evaluating the accuracy of clustering will be described below.
In this experiment, the accuracy of the following three methods was compared.
(A) Method of this embodiment (I) Existing method (with missing items)
(C) Existing method (no missing items)
As the existing methods (a) and (c), the following commonly used methods were used.

“An attribute where at least one record is a missing value is not used for analysis.”
However, for the method (c), data without missing items was used as input data. This corresponds to a case where all input data can be used, and indicates the upper limit of the accuracy of the clustering method.

  Subsequently, data used in the experiment will be described. FIG. 16 is a diagram showing, in a tabular form, experimental data used for evaluating the accuracy of clustering. This is a total of 30 administrative errors for three banks, Bank A, Bank B, and Bank C, and is used for bank type column, branch number column, branch type column, and clustering. , Having a column of miss attribute values.

  However, the attribute value used for clustering was not the aggregate value of the number of mistakes, but the ratio of the number of mistakes. For example, the attribute value 0.291 of “operation a” in the record corresponding to the branch number “A01” of bank A in the data shown in FIG. 16 is “occurrence at branch A01” for “the number of all mistakes that occurred at branch A01”. Of “number of mistakes in job a”. That is, when all attribute values of business a to business e of a record are summed, the total value is 1.

  The data shown in FIG. 16 is input data for the method (c). On the other hand, the input data to the methods (a) and (b) was created by generating a pseudo defect in the data shown in FIG.

FIG. 17 is a diagram showing, in a tabular form, the collection status of office errors at each bank used for evaluating the accuracy of clustering.
Based on FIG. 17, information on the “person in charge” item and information on the “situation” item are collected by each bank, but bank B does not collect information on the “business” item. Bank C does not collect information on the “Cause” item, and the corresponding part in FIG.

FIG. 18 is a diagram showing, in a tabular form, experimental data including missing items used for evaluating the accuracy of clustering.
For the evaluation of the clustering result, the values in the “branch type” column shown in FIGS. 16 and 18 are used. The “branch type” is a type of branch used in common among the three banks, and includes three types of attribute values “large”, “small”, and “special”.

  Here, the clusters generated by each method implemented with 3 clusters are regarded as a set of “large”, “small”, and “special” respectively, and branches that are actually correctly classified with respect to the total number of branches. Is calculated for all combinations of the three clusters and the three branch types, and the highest value is the correct answer rate of the method.

  For example, the branch type to be classified into the first cluster is “large”, the branch type to be classified into the second cluster is “small”, and the branch type to be classified into the third cluster is “special”. ”And the branch type actually classified by a certain method is“ large ”branch,“ large ”branch,“ small ”branch, Five branches consisting of “special” branches and “special” branches are classified. For the second cluster, two branches consisting of “small” branches and “small” branches are classified. Suppose that for the cluster of 3, four branches are classified into a “large” branch, a “special” branch, a “special” branch, and a “special” branch.

First cluster: large, large, small, special, special Second cluster: small, small Third cluster: large, special, special, special In this case, the number of “large” branches classified in the first cluster Is 2, the number of “small” branches classified in the second cluster is 2, and the number of “special” branches classified in the third cluster is 3, so the total number of branches is The correct answer rate, which is the ratio of the number of branches that are actually correctly classified in each cluster, is (2 + 2 + 3) / 11 = 7/11. This correct answer rate is all three clusters and all three branch types. When the correct answer rate is the highest for the combination, the calculation result is set as the correct answer rate of the clustering result by this method.

  Next, the accuracy rate of the number of branches actually classified correctly in each cluster with respect to the total number of branches, indicating the accuracy of each method by method (a), method (b), and method (c). It shows in FIG.

  As shown in this example, the accuracy of “(a) the method of this embodiment” exceeds the accuracy of “(a) the existing method (with missing items)”, and this accuracy is higher than that of the existing method. It can be said that the method of form is robust against data including missing items.

As described above, in the present embodiment, an organization-specific data table for managing records, which are aggregated data having at least one type of attribute, for each of a plurality of organizations to be analyzed is stored. Based on the value of the common attribute for each record set of the plurality of organizations having at least one type of common attribute among the plurality of organizations shown in the organization-specific data table, Based on the number and the total value of the common attribute, the distance between the record sets is calculated, and based on the calculated distance, clustering is performed with the organization corresponding to each record as a cluster.
Therefore, since the collected attributes are different for each organization, even if the aggregated data is not consistent among all organizations, the collected data can be obtained by utilizing the attribute information common to the organizations. It can be used effectively and enables high-precision analysis that integrates data from multiple organizations.

According to each of these embodiments, even if a defect occurs due to the difference in attribute of data among the data of different organizations, the accuracy of analysis when these data are integrated is improved. Therefore, it is possible to provide a data analysis support device that can be used.
In the embodiment described above, the organization-specific data table is a data table for managing records, which are aggregated data having at least one kind of attribute, for each of a plurality of organizations to be analyzed by organization. However, the data managed by the organization-specific data table may be quantitative data or qualitative data.

  In the present embodiment, the clustering execution unit 43 of the data analysis support apparatus 10 sets each record set of a plurality of organizations having at least one kind of common attribute among the plurality of organizations, which is indicated by the organization-specific data table. Based on the distance between the two, it has been described that the clustering process is performed with the organization corresponding to each record as a cluster, but not limited to this, if the analysis process using the distance between a set of records is performed, An analysis execution unit may be provided instead of the clustering execution unit 43, and the analysis execution unit may perform analysis processing using a self-organizing map using, for example, a distance between record sets, or a multidimensional scale configuration. Analysis processing by a method may be performed.

  Note that the method described in the above embodiment includes a magnetic disk (floppy (registered trademark) disk, hard disk, etc.), an optical disk (CD-ROM, DVD, etc.), a magneto-optical disk (MO) as programs that can be executed by a computer. ), And can be distributed in a storage medium such as a semiconductor memory.

  In addition, as long as the storage medium can store a program and can be read by a computer, the storage format may be any form.

  In addition, an OS (operating system) running on a computer based on an instruction of a program installed in the computer from a storage medium, MW (middleware) such as database management software, network software, and the like realize the above-described embodiment. A part of each process may be executed.

  Further, the storage medium in the present invention is not limited to a medium independent of a computer, but also includes a storage medium in which a program transmitted via a LAN, the Internet, or the like is downloaded and stored or temporarily stored.

  Further, the number of storage media is not limited to one, and the case where the processing in the above embodiment is executed from a plurality of media is also included in the storage media in the present invention, and the media configuration may be any configuration.

  The computer according to the present invention executes each process in the above-described embodiment based on a program stored in a storage medium, and is a single device such as a personal computer or a system in which a plurality of devices are connected to a network. Any configuration may be used.

  In addition, the computer in the present invention is not limited to a personal computer, but includes an arithmetic processing device, a microcomputer, and the like included in an information processing device, and is a generic term for devices and devices that can realize the functions of the present invention by a program. .

  Although several embodiments of the invention have been described, these embodiments are presented by way of example and are not intended to limit the scope of the invention. These novel embodiments can be implemented in various other forms, and various omissions, replacements, and changes can be made without departing from the scope of the invention. These embodiments and modifications thereof are included in the scope and gist of the invention, and are included in the invention described in the claims and the equivalents thereof.

  DESCRIPTION OF SYMBOLS 10 ... Data analysis support apparatus, 11 ... Control part, 12 ... Storage apparatus, 20 ... Display apparatus, 31 ... Organization-specific data table storage part, 32 ... Combined data table storage part, 33 ... Inter-record distance storage part, 41 ... Data table combining unit, 42... Inter-record distance calculating unit, 43.

Claims (4)

An organization-specific data table storage means for storing an organization-specific data table for managing records that are data having at least one type of attribute for each of a plurality of organizations to be analyzed;
Based on the value of the common attribute, the records are common to each record set of the plurality of organizations having at least one type of common attribute among the plurality of organizations shown in the organization-specific data table. Distance calculating means for calculating the distance between the set of records based on the number of attributes and the aggregate value in the common attribute;
A data analysis support apparatus comprising: analysis processing means for performing analysis based on the distance calculated by the distance calculation means. The distance calculating means includes
For each record set of a plurality of organizations having a common attribute shown in the organization-specific data table, the attribute value common to these records in one record and the common record in the other record The data analysis support apparatus according to claim 1, wherein a distance between the records is calculated based on a difference value between attribute values and an inverse number of the common attributes. The analysis processing means includes
Based on the distance between the record sets calculated by the distance calculation means, a plurality of cluster centers are arbitrarily set as a set of cluster centers from among the cluster elements corresponding to the respective records, and the cluster centers For each of the sets, a cluster is arbitrarily set by associating at least one cluster element having a distance close to the cluster center, the center of gravity of the set cluster is calculated, and the cluster element having the closest distance to the calculated center of gravity is calculated Is set to the new cluster center of the cluster, a new set of cluster centers is set, and the newly set cluster center set is set before the calculation of the centroid for setting the cluster center. If it is not the same as the set of cluster centers, then To configure the cluster in that associating at least one cluster element distance is close to the cluster center again, calculate the centroids of the clusters the set as a new center of gravity,
In addition, if the set of the newly set cluster centers is the same as the set of cluster centers set before the calculation of the center of gravity for setting the cluster centers, the information on the set latest clusters is used as a clustering result. The data analysis support apparatus according to claim 1, wherein the data analysis support apparatus outputs the data analysis support apparatus. A computer having an organization-specific data table storage device that stores an organization-specific data table for managing records, which are data having at least one type of attribute, for each of a plurality of organizations to be analyzed.
Based on the value of the common attribute, the records are common to each record set of the plurality of organizations having at least one type of common attribute among the plurality of organizations shown in the organization-specific data table. As a distance calculation means for calculating the distance between the record sets based on the number of attributes and the aggregate value in the common attribute, and an analysis processing means for performing an analysis based on the distance calculated by the distance calculation means Data analysis support processing program to make it function.

Images