JP6604207B2 - Relation information generation method, apparatus, and program - Google Patents

Description

  The present invention relates to a relationship information generation method, a relationship information generation device, and a relationship information generation program.

  A plurality of databases are integrated when a name identification process for master data management or when an information system (data warehouse) is constructed by integrating a plurality of core business systems.

  As a technique related to database integration, a method of narrowing down comparison candidates for item pairs to be identified is proposed by focusing on a character code range. In this method, a character code group similar to the character code group designated by the user or the like is searched from a plurality of character code groups. At this time, when each character code included in the designated character code group is within a predetermined numerical range, a character code group including a character code not belonging to the predetermined numerical range is selected from a plurality of character code groups. Extract the character code group.

  In addition, a method for converting the code of one master into the code of the other master has been proposed. In this method, items relating to the same matter are discriminated from both masters by collating the items of the record of the integrated master with the items of the record of the local master. And the record regarding the same object is discriminate | determined from an integrated master and a local master by collating the content of the item discriminate | determined regarding the same matter. In addition, a code conversion table is stored in which a code of a predetermined item indicated in the record of the integrated master determined to be related to the same object and a code of the predetermined item indicated in the record of the local master are associated with each other. Then, the input data is converted based on the code conversion table.

  Also, a method for extracting a column having a correspondence relationship between two two-dimensional data has been proposed. In this method, a column pair to be analyzed in the migration source data is selected, and a column pair to be analyzed in the migration destination data is selected. Further, the support level and the certainty factor are calculated for each row in the column pair of the migration source data, and the support level and the certainty factor are calculated for each row for each column pair of the migration destination data. Then, the difference between the support and the certainty between the rows is calculated in the migration source data, and the difference between the support and the certainty between the rows is calculated for each column pair in the migration destination data. Further, a difference calculation between the difference value in the migration source data and the difference value in the migration destination data is performed, and the migration destination column pair corresponding to the migration source column pair is determined based on the difference calculation result.

JP 2012-234343 A JP 2008-250861 A JP 2012-38066 A

  In the prior art focused on the character code described above, a profile for each data item such as a character code range, a character string length, an appearance frequency of a characteristic character string, the number of partial character strings, and the like is created. The degree of similarity is compared. However, in the integration of databases in which what items are recorded in what format, it is difficult to appropriately profile data items in order to compare the similarity between items. In addition, when the data related to one item is divided into multiple items in the database, it becomes more difficult to create an appropriate profile, and the number of combinations of data items for which the degree of similarity is compared increases and is calculated. Cost increases. Note that when data relating to one item is divided into a plurality of items, for example, data relating to the item “address” is divided into two items “address 1” and “address 2” and recorded. This is the case.

  Also, in the prior art in which a code of one master is converted into a code of the other master using a code conversion table stored in advance, an appropriate code conversion table needs to be defined. However, in this prior art, there is no specific disclosure of item association for generating a code conversion table, and for example, it is necessary to associate items manually. In addition, when data related to one item is divided into a plurality of items, it is difficult to generate a code conversion table corresponding to all of the possible division variations.

  Furthermore, in the conventional technique for extracting a column having a correspondence relationship between two two-dimensional data, a case where data is divided into a plurality of columns is considered. However, it is premised that items (name, address, TEL, etc.) to be matched between two two-dimensional data are known in advance. Therefore, when different types of databases such as master and transaction are integrated, the contents of data recorded in the database are mixed (customer list, company list, product database, paper, etc.) or unknown. Not applicable in case.

  That is, in the conventional technology, depending on the attributes (type, item, data content, etc.) of the database, it is not possible to appropriately identify the corresponding item pair between the databases, or the amount of calculation required to identify the item pair is large. There is a problem that it will increase.

  An object of the present invention is to specify an item pair corresponding between databases with a small amount of calculation regardless of the attributes of the database.

  As one aspect, the present invention extracts a partial character string from a record included in a first database including a plurality of records in which a character string is associated with each of a plurality of items. And the item corresponding to the partial character string extracted from the said 1st database among the said several items about the said 1st database is specified. In addition, in a second database including a plurality of records in which a character string is associated with each of a plurality of items, the appearance of an item corresponding to a partial character string that matches the partial character string extracted from the first database Calculate the number. The number of appearances is calculated in association with an item corresponding to the partial character string extracted from the first database. Then, based on the calculated number of appearances, information on the correspondence relationship between the items included in the first database and the items included in the second database is generated.

  As one aspect, there is an effect that an item pair corresponding between databases can be identified with a small amount of calculation regardless of the attributes of the database.

It is a functional block diagram of a relationship information generation device. It is a figure which shows an example of a common dictionary database. It is a figure which shows an example of a name collation origin table | surface. It is a figure for demonstrating extraction of a partial character string. It is a figure for demonstrating extraction of a partial character string. It is a figure which shows an example of an extraction position list. It is a figure which shows an example of a pattern list. It is a figure for demonstrating collation of a pattern. It is a figure which shows an example of an appearance position list. It is a figure which shows an example of an item group pair list. It is a graph showing an example of an item group pair. It is a figure which shows an example of a total table. It is a figure for demonstrating the division | segmentation and mapping of an item. It is a block diagram which shows schematic structure of the computer which functions as a relationship information generation apparatus. It is a flowchart which shows an example of a relationship information generation process. It is a flowchart which shows an example of a character string extraction process. It is a flowchart which shows an example of a pattern production | generation process. It is a flowchart which shows an example of a pattern matching process. It is a flowchart which shows an example of a correspondence extraction process. It is a figure for demonstrating extraction of the item group pair of the item group of a name collation destination table, and the item group of a name collation source table. It is a graph showing the other example of an item group pair. It is a figure which shows an example of the database of a tree structure.

  Hereinafter, an example of an embodiment according to the present invention will be described in detail with reference to the drawings.

  As shown in FIG. 1, the relationship information generation apparatus 10 according to the present embodiment receives a common dictionary database (DB) 20 and a name identification source DB 30 as input, and shows the correspondence between items in the common dictionary DB 20 and items in the name identification source DB 30. The correspondence information shown is generated and output.

  The common dictionary DB 20 is a general-purpose database provided from a data bank or the like. For example, as shown in FIG. 2, the common dictionary DB 20 can include a personal name dictionary 21, an address dictionary 22, a company name dictionary 23, and the like. In each dictionary, each column corresponds to each item, and each row corresponds to one record. Each record has a value (character string) corresponding to each item.

  The name identification source DB 30 is a database created and managed by a user such as a company or an organization. In the example of FIG. 1, the name identification source DB 30 includes a name identification source table A31, a name identification source table B32, and a name identification source table C33. FIG. 3 shows an example of the name identification source table A31. In the name identification source table A31, each column corresponds to each item, and each row corresponds to one record. Each record has a value (character string) corresponding to each item. Similarly to the name identification source table A31, the name identification source table B32 and the name identification source table C33 are also tabular databases, but the contents of items and data (records) included in each name identification source table are different.

  The common dictionary DB 20 and the name identification source DB 30 are stored in a predetermined storage area of the relationship information generation device 10, an external device connected to the relationship information generation device 10, or a storage medium such as a CD-ROM or a USB memory.

  In this embodiment, when the name identification source table A31, the name identification source table B32, and the name identification source table C33 are integrated into one name identification source table, items included in each dictionary in the common dictionary DB 20 as items of the name identification source table Is used. Therefore, the relationship information generation device 10 identifies the association between the items included in each dictionary in the common dictionary DB 20 and the items included in each name identification source table in the name identification source DB 30. In the following, the correspondence between the items of each dictionary and the items of the name identification source table A31 will be described. However, the items of the name identification source table B32 and the name identification source table C33 are also associated with the items of each dictionary. Thereby, a plurality of name identification source tables can be integrated into one name identification destination table via the items of each dictionary.

  Functionally, the relationship information generation device 10 includes a character string extraction unit 11, a pattern generation unit 12, a pattern matching unit 13, and a correspondence relationship extraction unit 14, as shown in FIG. The character string extraction unit 11 is an example of the extraction unit of the present invention, the pattern generation unit 12 is an example of the specifying unit of the present invention, and the pattern matching unit 13 and the correspondence relationship extraction unit 14 are the calculation unit of the present invention. And an example of a generation unit.

  The character string extraction unit 11 extracts a plurality of partial character strings having a predetermined number of characters from records recorded in each dictionary in the common dictionary DB 20. At this time, the character string extraction unit 11 includes a partial character string extracted across the items in the plurality of partial character strings to be extracted.

  Specifically, as shown in FIG. 4, the character string extraction unit 11 takes out one of the records in each dictionary (in FIG. 4, one record of the address dictionary 22 is illustrated), and the character string corresponding to each item. Are connected to generate a concatenated character string 41. The character string extraction unit 11 extracts a randomly selected n-gram as a partial character string 42 from the concatenated character string 41. In the example of FIG. 4, n = 3, and partial character strings 42 of “211”, “88 gods”, “river river Kawa”, “Nakahara Ward”, and “Tanaka 4” are extracted.

  By extracting the partial character string 42 from the concatenated character string 41, the partial character string 42 across the items is also extracted. In the above example, the partial character string 42 “88 god” is extracted across the item “〒” and the item “prefecture”, and the partial character string 42 “kawa river” is It is extracted across the “prefecture” and the item “city”.

  Thus, the reason for extracting the partial character string 42 across items will be described.

  In each database, the notation method of items corresponding to a certain item is not common, and there are cases where various writing methods are used. For example, each database includes data related to common items such as name, address, and telephone number. Regarding these common items, the items included in one database are “name”, “kana”, “prefecture”, “city”, “machi chome”, “address”, and “TEL”. . The items included in the other database are “last name”, “first name”, “address 1”, “address 2”, and “phone”. In this case, the item “name” corresponds to the item “name” in one database, but corresponds to the items “first name” and “first name” in the other database. In addition, regarding the matter of address, in one database, the items of “prefecture”, “city”, “machi chome”, and “address” correspond, but in the other database, “address 1” and “address The item “2” corresponds. Further, regarding the matter of telephone number, the item “TEL” corresponds to one database, while the item “phone” corresponds to the other database.

  When collating the above two databases, name = last name + first name, prefecture + city + town + street + address = address 1 + address 2, and TEL = telephone It is necessary to associate. Therefore, in order to include not only the case of each item but also the group of items as a target of association, a partial character string 42 across items is extracted.

  Note that the length of the partial character string 42 may not be constant. For example, as illustrated in FIG. 5, the character string extraction unit 11 may extract a partial character string 42 by mixing 3-gram, 4-gram, 5-gram, 6-gram, and the like. Further, as shown in part A of FIG. 5, the character string extraction unit 11 may extract the partial character string 42 so that parts of different partial character strings 42 overlap each other.

  In addition, when the partial character string 42 is extracted at random, a random number can be used. However, a certain item and a continuous item may be selected more easily or less easily than others. For example, a rule is defined such that a partial character string 42 extending over three items or more is defined, and the character string extracting unit 11 performs the partial character string 42 according to this rule as shown in a part B of FIG. May be extracted. Conversely, a rule may be set such that partial character string 42 that extends over three items is not extracted. Further, for a specific item, a rule for prohibiting the extraction of the partial character string 42 across items may be defined. For example, it is possible to define a rule that a partial character string 42 that straddles the item “〒” and the item “prefecture” is not extracted. In this case, “88 God” in the above example is not extracted.

  FIG. 4 shows an example in which the partial character string 42 is randomly extracted from each record or sampled record. However, as shown in C of FIG. 5, all the partial character strings 42 are extracted from all the records. You may do it.

  The character string extraction unit 11 records a set of the extracted partial character string 42 and the extracted position of the partial character string 42 as extracted position information in, for example, an extracted position list 43 as shown in FIG. The extraction position of the partial character string 42 is information indicating which partial character string 42 is extracted from which position of which record in which dictionary. For example, the extraction position is specified by information such as dictionary identification information, record number, number of characters from the beginning of the record, and the number of characters in the partial character string 42. In the example of FIG. 6, the extraction position (22, 1, 1, 3) of the partial character string “211” included in the extraction position information of the first line is one line of the dictionary (address dictionary 22) with the identification information 22. This represents the position indicated by three characters from the first character of the eye record. Note that the information for specifying the extraction position is not limited to the above example, such as using information on the number of characters from the beginning of the record instead of the number of characters.

  The pattern generation unit 12 specifies an item group corresponding to each of the partial character strings 42 extracted by the character string extraction unit 11. If the partial character string 42 is not extracted across items, the item group includes one item. Specifically, the pattern generation unit 12 specifies the position of the extracted partial character string 42 based on the extracted position information included in the extracted position list 43, and sets the value including the partial character string 42 at that position. Identify the corresponding item group.

  Further, the pattern generation unit 12 generates a pattern in which each partial character string 42 included in the extraction position list 43 and an item group corresponding to the specified partial character string 42 are combined. For example, the pattern generation unit 12 specifies an item group including the items “〒” and “prefecture” from the partial character string “88 god”, such as <“88 god” → (〒, prefecture)>. A pattern can be generated. In this example, the left side of the arrow (→) represents the partial character string 42 and the right side represents the corresponding item group. The pattern generation unit 12 records the generated pattern in a pattern list 44 as shown in FIG. 7, for example. In the example of FIG. 7, the number of items included in the item group is two at the maximum. When only one item is included in the item group, “−” indicates that there is no second item. Represents.

  Furthermore, the pattern generation unit 12 considers items that are not written in a manner such as a zip code or TEL, and a pattern obtained by replacing a part of the partial character string 42 with a wild card (?) Or a partial character string. A pattern in which a part of 42 is deleted may be added to the pattern list 44. For example, when the pattern generation unit 12 generates a pattern of “<211-8588” → (〒, −)>, <“211? 8588” → (〒, −)> and <“211588” → ( A pattern such as 〒,-)> can be added.

  The pattern collation unit 13 collates the pattern generated by the pattern generation unit 12 with the record in the name identification source table A31. Specifically, the pattern matching unit 13 searches for a character string that matches the partial character string 42 included in each pattern generated by the pattern generation unit 12 from the records included in the name identification source table 31A. FIG. 8 shows an example of the partial character string 45 retrieved from the name identification source table A31 using the patterns included in the pattern list 44 shown in FIG. In FIG. 8, the name identification source table A31 is illustrated in the form of a table, but the pattern matching unit 13 concatenates each record of the name identification source table A31 in the same manner as the extraction of the partial character string 42 from each dictionary. Matches after converting to a string.

  Further, the pattern matching unit 13 uses a set of the pattern used for matching and the appearance position of the partial character string 45 matching the partial character string 42 included in the pattern as appearance position information, for example, as shown in FIG. Record in the appearance position list 46. The appearance position of the partial character string 45 is information indicating at which position of which record of which name identification source table the partial character string 45 appears. The appearance position can be specified by the same information as the extraction position recorded in the extraction position list 43. In the example of FIG. 9, the appearance position (31, 2, 18, 3) included in the appearance position information of the first line is the record of the second line of the name identification source table (name identification source table A31) with the identification information 31. The position shown by 3 characters from the 18th character is shown.

  The pattern matching unit 13 specifies an item group corresponding to a value including the partial character string 45 indicated by each appearance position included in the appearance position list 46 from the name identification source table A31. The item group specifying method is the same as the method in which the pattern generation unit 12 specifies the item group corresponding to the partial character string 42 extracted from each dictionary. The pattern matching unit 13 uses, as an item group pair, an item group included in the pattern used for matching and an item group specified for the partial character string 45 searched from the name identification source table A31 based on the pattern, for example. It records in the item group pair list 47 as shown in FIG. The item group included in the pattern used for matching is an item group specified by the pattern generation unit 12 with respect to the partial character string 42 extracted from each dictionary, and is hereinafter also referred to as a “dictionary item group”. . The item group specified for the partial character string 45 retrieved from the name identification source table A31 based on the pattern is also referred to as a “name identification source table item group” hereinafter.

  For example, in the name identification source table A31, it is assumed that an item corresponding to the partial character string 45 that matches the partial character string “211” of the pattern <“211” → (〒, −)> is specified as “postal”. In this case, the pattern matching unit 13 records the dictionary item group (〒,-) and the name identification source table item group (mail,-) as an item group pair in the item group pair list 47.

  FIG. 11 shows an example in which the information on the item group pairs recorded in the item group pair list 47 is represented by a graph. In the graph shown in FIG. 11, for each of the dictionary item group and the name identification source table item group, the item group pairs recorded in the item group pair list 47 are connected by lines. Further, the number of times written along the line connecting the item groups is the number of times that the item group pair appears in the item group pair list 47.

  The correspondence relationship extraction unit 14 correlates each item included in each dictionary and each item included in the name collation source table A31 based on the number of appearances of the item group pair of the dictionary item group and the name collation source table item group. Is extracted, and correspondence information is generated and output.

  Specifically, the correspondence extraction unit 14 includes each item included in each dictionary (hereinafter also referred to as “dictionary item”) and each item included in each name identification source table A31 (hereinafter “name identification source table item”). Counts the number of occurrences included in the same item group pair. Then, the correspondence relationship extraction unit 14 creates a tabulation table 48 as shown in FIG. 12, for example. As a totaling method, for example, for each occurrence of an item group pair, the correspondence relationship extracting unit 14 includes each dictionary item of the dictionary item group included in the item group pair and each name identification source table item of the name identification source table item group. The counter is incremented by 1 for all the combinations. More specifically, for each occurrence of an item group pair of a dictionary item group (A, B) and a name identification source table item group (C, D), a set of A and C, A The counter of the row corresponding to each of the group of D and D, the group of B and C, and the group of B and D is incremented by one.

  Further, the correspondence relationship extraction unit 14 divides the entire items included in each dictionary in the common dictionary DB 20 into M pieces while maintaining the order of the items. Similarly, the entire items included in the name identification source table A31 are divided into N items while maintaining the order of the items. Note that the order of items is retained in consideration of the fact that consecutive items correspond to one item, and therefore the order of the dictionary may be arbitrary. FIG. 13 shows an example (M = 6, N = 5) in which the entire dictionary item is divided into six and the entire name identification source table item is divided into five. Hereinafter, one divided unit is referred to as a “division item”, each division item of the dictionary item is “division item m (m = 1,..., M)”, and each division item of the name identification source table item is “ This is called a divided item n (n = 1,..., N) ”. Also, 1 ≦ M ≦ total number of dictionary items, 1 ≦ N ≦ total number of name identification source table items.

  The correspondence extracting unit 14 gives a mapping f: m → n for associating the divided item m with the divided item n, for example, as shown in FIG. Then, the correspondence relationship extraction unit 14 calculates the reliability S for each division with different M and N and for each mapping f with different correspondence. The reliability S is an index indicating the probability of the division and the mapping f, and can be defined by the following equation (1), for example. Note that f (m) is a divided item n associated with the divided item m by the mapping f.

    S = H (f (m))-H (f (m) | m) (1)

  H (f (m)) is the entropy of the division item n corresponding to f (m), and is defined by the following equation (2). H (f (m) | m) is the entropy of the divided item n corresponding to f (m) conditioned by the divided item m, and is defined by the following equation (3).

H (f (m)) = − Σ _{f (m)} P (f (m)) log P (f (m)) (2)
H (f (m) | m) = − Σ _{f (m), m} P (f (m), m) log P (f (m) | m)
... (3)

P (f (m)) is the appearance probability of the divided item n corresponding to f (m). Specifically, P (f (m)) is the total of the counters in the row including each name identification source table item included in the divided item n corresponding to f (m) with respect to the total number of counters in the total table 48. Obtained as a percentage. P (f (m), m) is the simultaneous appearance probability of the divided item n and the divided item m corresponding to f (m). Specifically, P (f (m), m) is each dictionary item included in the divided item m and each divided item n corresponding to f (m) with respect to the total of the counters in the summary table 48. It is obtained as the ratio of the total of the counter of the row corresponding to the pair with the name identification source table item. P (f (m) | m) is a conditional probability that the divided item n corresponding to f (m) is associated with the divided item m. Specifically, P (f (m) | m) corresponds to each dictionary item included in the divided item m and f (m) with respect to the total of the counters of the lines including the dictionary items included in the divided item m. It is obtained as a ratio of the total of the counters of the rows corresponding to the pairs with each name identification source table item included in the divided item n. Σ _{f (m)} is the sum for each of the divided items n associated with the divided item m by the mapping f, and Σ _{f (m), m} is the divided item m associated with the mapped item f. And the sum of each of the groups of divided items n corresponding to f (m).

  The correspondence relationship extraction unit 14 specifies a division and mapping f having the maximum reliability S using a conventionally known optimization algorithm and the like, and the division item m and the division item n obtained by the division and mapping f are determined. A correspondence is generated and output as correspondence information.

  The relationship information generating apparatus 10 can be realized by a computer 50 shown in FIG. 14, for example. The computer 50 includes a CPU 51, a memory 52 as a temporary storage area, and a nonvolatile storage unit 53. The computer 50 also includes an input / output device 54, a read / write (R / W) unit 55 that controls reading and writing of data with respect to the recording medium 59, and a communication interface (I / F) 56. The CPU 51, the memory 52, the storage unit 53, the input / output device 54, the R / W unit 55, and the communication I / F 56 are connected to each other via a bus 57.

  The storage unit 53 can be realized by a hard disk drive (HDD), a solid state drive (SSD), a flash memory, or the like. The storage unit 53 as a storage medium stores a relationship information generation program 60 for causing the computer 50 to function as the relationship information generation device 10. The relationship information generation program 60 includes a character string extraction process 61, a pattern generation process 62, a pattern matching process 63, and a correspondence relationship extraction process 64.

  The CPU 51 reads the relationship information generation program 60 from the storage unit 53 and expands it in the memory 52, and sequentially executes the processes included in the relationship information generation program 60. The CPU 51 operates as the character string extraction unit 11 illustrated in FIG. 1 by executing the character string extraction process 61. Further, the CPU 51 operates as the pattern generation unit 12 illustrated in FIG. 1 by executing the pattern generation process 62. Further, the CPU 51 operates as the pattern matching unit 13 illustrated in FIG. 1 by executing the pattern matching process 63. Further, the CPU 51 operates as the correspondence relationship extraction unit 14 illustrated in FIG. 1 by executing the correspondence relationship extraction process 64. As a result, the computer 50 that has executed the relationship information generation program 60 functions as the relationship information generation device 10.

  The function realized by the relationship information generation program 60 can also be realized by, for example, a semiconductor integrated circuit, more specifically, an ASIC (Application Specific Integrated Circuit).

  Next, the operation of the relationship information generation device 10 according to this embodiment will be described. The relationship information generation device 10 reads the common dictionary DB 20 and the name identification source DB 30 and executes the relationship information generation process shown in FIG.

  First, in step S10, a character string extraction process shown in detail in FIG. 16 is executed.

  In step S11, the character string extraction unit 11 determines whether or not the extraction of records from each dictionary in the common dictionary DB 20 has been completed. If the record extraction has not been completed, the process proceeds to step S12, and the character string extraction unit 11 extracts one record from any dictionary at random or sequentially from the top.

  Next, in step S <b> 13, the character string extraction unit 11 concatenates each character string corresponding to each item of the extracted record to generate a concatenated character string 41.

  Next, in step S <b> 14, the character string extraction unit 11 extracts a randomly selected n-gram as a partial character string 42 from the concatenated character string 41.

  Next, in step S15, the character string extraction unit 11 uses a set of the extracted partial character string 42 and the extracted position of the partial character string 42 as extraction position information, for example, an extraction position list 43 as shown in FIG. And the process returns to step S11.

  If the character string extraction unit 11 determines in step S11 that the extraction of a predetermined number of records has been completed, or if it has determined that the extraction of all the records in the last row has been completed, the process is as follows. Returning to the relationship information generation process shown in FIG.

  Next, in step S20, a pattern generation process shown in detail in FIG. 17 is executed.

  In step S <b> 21, the pattern generation unit 12 determines whether extraction of all extraction position information included in the extraction position list 43 has been completed. If extraction of extraction position information has not been completed, the process proceeds to step S22, and the pattern generation unit 12 extracts one piece of extraction position information that has not yet been extracted from the extraction position list 43.

  Next, in step S23, the pattern generation unit 12 acquires the corresponding record from the corresponding dictionary based on the extracted extraction position information. Then, the pattern generation unit 12 specifies the partial character string 42 extracted by the character string extraction unit 11 from the acquired record based on the extracted extraction position information, and corresponds to the value including the partial character string 42. Identify items.

  Next, in step S24, the pattern generation unit 12 combines the partial character string 42 included in the extracted position information extracted in step S22 and the item group corresponding to the partial character string 42 identified in step S23. Generate a pattern. Then, the pattern generation unit 12 records the generated pattern in a pattern list 44 as shown in FIG. 7, for example.

  Next, in step S25, the pattern generation unit 12 deletes a pattern obtained by replacing a part of the partial character string 42 of the pattern generated in step S25 with a wild card (?) Or a part of the partial character string 42. A pattern is generated and added to the pattern list 44. If the partial character string 42 does not include a part that can be replaced with a wild card or a part that can be deleted, the process of this step is omitted. Then, the process returns to step S21.

  If the pattern generation unit 12 determines in step S21 that all extraction position information has been extracted, the process returns to the relationship information generation process shown in FIG.

  In the above description, a case where partial character string extraction is executed by partial character string extraction processing and an item corresponding to the partial character string is specified by pattern generation processing has been described. In addition, an item corresponding to the partial character string may be specified.

  Next, in step S30, a pattern matching process shown in detail in FIG. 18 is executed.

  In step S31, the pattern collation unit 13 constructs a pattern collator that collates each pattern included in the pattern list 44 with each record of the name identification source table A31. In normal pattern matching, if i patterns and j records are matched, i × j matching is performed, but here, i patterns can be matched to one record at a time. A simple pattern matcher (reference: Navarro & Raffinot, "Flexible Pattern Matching in String", pp. 41-74, Cambridge Press, 2002.). By using such a pattern collator, the amount of calculation can be further reduced.

  Next, in step S32, the pattern matching unit 13 determines whether or not all the records included in the name identification source table A31 have been extracted. If the record extraction has not been completed, the process proceeds to step S33, and the pattern matching unit 13 extracts one record that has not yet been extracted from the name identification source table A31.

  Next, in step S34, the pattern matching unit 13 generates a concatenated character string from the extracted record and inputs it to the pattern collator constructed in step S31.

  Next, in step S35, the pattern collation unit 13 uses the pattern used as the collation as the output of the pattern collator and the partial character that matches the partial character string 42 included in the pattern in the record extracted in step S33. A pair with the appearance position of the column 45 is acquired. The pattern matching unit 13 records the combination of the pattern and the appearance position as appearance position information in, for example, the appearance position list 46 as shown in FIG. 9, and the process returns to step S32.

  If the pattern matching unit 13 determines in step S32 that all the records in the name identification source table A31 have been extracted, the process proceeds to step S36.

  In step S <b> 36, the pattern matching unit 13 determines whether or not all the appearance position information included in the appearance position list 46 has been extracted. If the extraction of the appearance position information has not ended, the process proceeds to step S37, and the pattern matching unit 13 extracts one piece of appearance position information that has not yet been extracted from the appearance position list 46.

  Next, in step S38, the pattern matching unit 13 acquires a corresponding record from the name identification source table A31 based on the extracted appearance position information. Then, the pattern matching unit 13 specifies the partial character string 45 at the position indicated by the appearance position information from the acquired record, and specifies the item group corresponding to the value including the partial character string 45.

  Next, in step S39, the pattern matching unit 13 records the dictionary item group and the name identification source table item group as an item group pair in, for example, an item group pair list 47 as shown in FIG. 10, and the process proceeds to step S36. Return.

  If the pattern matching unit 13 determines in step S36 that all the appearance position information has been extracted, the process returns to the relation information generation process shown in FIG.

  Next, in step S40, the correspondence extraction process shown in detail in FIG. 19 is executed.

  In step S <b> 41, the correspondence relationship extraction unit 14 determines whether extraction of all item group pairs included in the item group pair list 47 has been completed. If the extraction of the item group pair is not completed, the process proceeds to step S42, and the correspondence relationship extraction unit 14 extracts one item group pair that has not been extracted from the item group pair list 47.

  Next, in step S43, the correspondence relationship extraction unit 14 calculates a summary table for all combinations of each dictionary item of the dictionary item group and each name identification source table item of the name identification source table item group included in the extracted item group pair. The counter of 48 is incremented by 1, and the process returns to step S41.

  If the correspondence extracting unit 14 determines in step S41 that all item group pairs have been extracted, the process proceeds to step S44.

  In step S44, the correspondence relationship extraction unit 14 divides the entire items included in each dictionary in the common dictionary DB 20 into M pieces while maintaining the order of the items, and sets the entire items included in the name identification source table A31 to each item. Are divided into N pieces while maintaining the order. Then, the correspondence relationship extraction unit 14 defines a mapping f: m → n that associates the divided item m with the divided item n.

  Next, in step S45, the correspondence relationship extraction unit 14 calculates the reliability S for the mapping f in the division at M and N, for example, using equation (1).

  Next, in step S46, the correspondence relationship extraction unit 14 determines whether or not the end condition for calculating the reliability S is satisfied. The termination condition depends on the optimization algorithm. For example, when the reliability S is calculated for all maps f in all M and N divisions, when steps S44 and S45 are repeated a predetermined number of times, the difference between the reliability S calculated this time and the previous time is predetermined. It can be determined that the end condition is satisfied when the value is equal to or less than the value. If the termination condition for calculating the reliability S is not satisfied, the process returns to step S44, and the correspondence relationship extraction unit 14 changes the mapping f or M and N and the mapping f to calculate the reliability S. repeat. If the termination condition for calculating the reliability S is satisfied, the process proceeds to step S47.

  In step S47, the correspondence relationship extraction unit 14 generates and outputs the association with the division item m and the division item n obtained by the division and mapping f having the maximum reliability S as the correspondence relationship information. Then, the process returns to the relation information generation process shown in FIG. 15, and the relation information generation process ends.

  As described above, according to the relationship information generating apparatus 10 according to the present embodiment, a partial character string is extracted from a record included in the common dictionary DB, and a dictionary item group corresponding to the partial character string is specified. In the name identification source table, the name identification source table item group corresponding to the partial character string that matches the partial character string extracted from the common dictionary is specified. Then, based on the number of appearances of the dictionary item group and the name identification source table item group pair, the division item m obtained by dividing the entire dictionary item into M pieces, and the division item n obtained by dividing the entire name identification source table item into N pieces, Associate. This makes it possible to specify a corresponding item pair between databases regardless of database attributes such as the type of database, items, and data contents. In addition, compared to a case where a profile for each item is created or the association between items is performed manually, it is possible to identify an item pair corresponding to each other with a small amount of calculation.

  Further, when extracting a partial character string from each database, a partial character string straddling items is included. Accordingly, it is possible to specify a corresponding item pair between databases in consideration of a case where one item is divided into a plurality of items.

  In the above embodiment, assuming that a plurality of name identification source tables are integrated into one name identification target table using items in the common dictionary, the correspondence relationship between the items in the common dictionary and the items in the name identification source table However, the present invention is not limited to this. When the items in the name identification destination table are known, the items in the name identification destination table may be directly associated with the items in the name identification source table. In this case, the common dictionary DB 20 in the above embodiment may be read as a name identification table.

  For example, as shown in FIG. 20, a partial character string 45 that matches the partial character string 42 extracted from the name identification destination table 25 is searched from the name identification source table A31, and an item group of the name identification destination table 25 and an item group of the name identification source table 31 The item group pair is extracted. FIG. 21 shows an example in which the information of the item group pairs extracted in this way is represented by a graph. In FIG. 21, as the number of appearances of the item group pair increases, the line connecting the name identification destination table item group and the name identification source table item group is represented by a thick line. Assume that the reliability S in the correspondence extraction unit 14 is calculated using the information in FIG. In this case, for example, the mapping including the correspondence of name identification table item “name” → name identification source table item “customer name abbreviation” is the name identification table item “kana” → name identification source table item “customer name abbreviation” The number of appearances is higher than that of the map including the association “”, so that the reliability is high.

  Also, the name collation table items “Postal 1” and “Postal 2” may be one division item m. In this case, the reliability S of the mapping including the correspondence of the name collation table item “postal 1” + “postal 2” → name collation source table item “postal” includes the name collation table items “postal 1” and “postal 2”. And the number of appearances of each pair of name identification source table item “mail” is reflected. Therefore, for example, the reliability is expected to be higher than a mapping including a one-to-one correspondence of name identification destination table item “Postal 1” → name identification source table item “Postal”, and appropriate association can be performed. In addition, the association of name identification table item “address” → name identification source table item “address 1” + “address 2” is also compared with the association of name identification table item “address” → name identification source table item “address 1”. The reliability of the mapping is expected to increase, and appropriate association can be performed.

  Further, for example, there may be a mapping including an incorrect association of name collation table item “transaction bank” → name collation source table item “customer name abbreviation”. However, in this case, it is highly likely that this association will be excluded because the number of occurrences of the pair of name collation table item “bank” and name collation table item “customer name abbreviation” is expected to be extremely small. .

  Moreover, although the said embodiment demonstrated the case where the database which matches an item was a table format, it is not limited to this. For example, the present invention can also be applied to a database having a tree structure such as XML (Extensible Markup Language). FIG. 22 shows an example of a database having a tree structure. In the example of FIG. 22, a leaf node connected from a node (business partner, location, telephone number) representing each item recorded in the name identification source table indicates an item corresponding to the item, and the node indicating the item is , Keep the value of each record for that item. In this case, a leaf node included in each partial tree can be regarded as an item group. The correspondence extraction unit 14 can divide each item into divided items by cutting between nodes. For example, as shown in FIG. 22, when cut at a broken line portion, it is divided into five division items of “name”, “kana”, “〒”, “address”, and “postal”. In addition, when cutting at a dashed line, it is divided into six division items of “name”, “kana”, “〒”, “address 1”, “address 2”, and “mail”.

  In the above-described embodiment, the aspect in which the relationship information generation program 60 is stored (installed) in the storage unit 53 in advance has been described. However, the present invention is not limited to this. The related information generation program according to the present invention can be provided in a form recorded on a recording medium such as a CD-ROM, a DVD-ROM, or a USB memory.

  Regarding the above embodiment, the following additional notes are disclosed.

(Appendix 1)
On the computer,
Extracting a partial character string from a record included in a first database including a plurality of records in which a character string is associated with each of a plurality of items;
Of the plurality of items for the first database, specify an item corresponding to the partial character string extracted from the first database;
In a second database including a plurality of records in which a character string is associated with each of a plurality of items, the number of occurrences of the item corresponding to the partial character string that matches the partial character string extracted from the first database is calculated. , Calculating in association with the item corresponding to the partial character string extracted from the first database,
A relation information generation method for executing processing including generating information on a correspondence relationship between an item included in the first database and an item included in the second database based on the calculated number of appearances.

(Appendix 2)
The supplementary note 1, wherein when extracting a partial character string from a record included in the first database, the partial character string is extracted from a concatenated character string obtained by concatenating character strings associated with each of the plurality of items. Relation information generation method.

(Appendix 3)
The relation information generating method according to supplementary note 2, wherein when the partial character string is extracted from the concatenated character string, an item corresponding to the extracted partial character string is extracted so as to span a plurality of items.

(Appendix 4)
The m-th divided item when the entire plurality of items included in the first database are divided into M divided items, and the entire plurality of items included in the second database are divided into N divided items. The reliability that the n-th divided item is in correspondence with the divided item is the item of the second database included in the n-th divided item and the m-th divided item. Calculated based on the number of occurrences of the item associated with the item in the first database, and the combination of the m-th divided item and the n-th divided item with the highest calculated reliability is the correspondence 4. The relation information generation method according to any one of supplementary notes 1 to 3, which is generated as information related to the relation.

(Appendix 5)
An extraction unit that extracts a partial character string from a record included in a first database including a plurality of records in which a character string is associated with each of a plurality of items;
A specifying unit for specifying an item corresponding to a partial character string extracted from the first database among the plurality of items for the first database;
In a second database including a plurality of records in which a character string is associated with each of a plurality of items, the number of occurrences of the item corresponding to the partial character string that matches the partial character string extracted from the first database is calculated. A calculation unit that calculates the item corresponding to the item corresponding to the partial character string extracted from the first database;
Based on the calculated number of occurrences, a generation unit that generates information about the correspondence between items included in the first database and items included in the second database;
A relational information generation device including

(Appendix 6)
The extraction unit extracts the partial character string from a concatenated character string obtained by concatenating the character strings associated with each of the plurality of items when extracting the partial character string from a record included in the first database. The related information generation device according to appendix 5.

(Appendix 7)
The relation information generating device according to supplementary note 6, wherein when the extraction unit extracts the partial character string from the concatenated character string, an item corresponding to the extracted partial character string is extracted so as to span a plurality of items.

(Appendix 8)
The generator generates an m-th divided item when the entire plurality of items included in the first database are divided into M divided items, and an entire plurality of items included in the second database as N The reliability of the fact that there is a correspondence relationship with the nth divided item when dividing into the number of divided items is the item of the second database included in the nth divided item and the mth divided item Calculated based on the number of appearances of the item associated with the item of the first database included in the item, and the m-th divided item and the n-th divided item with the highest calculated reliability The relation information generation apparatus according to any one of supplementary notes 5 to 7, which generates a combination as information related to the correspondence relationship.

(Appendix 9)
On the computer,
Extracting a partial character string from a record included in a first database including a plurality of records in which a character string is associated with each of a plurality of items;
Of the plurality of items for the first database, specify an item corresponding to the partial character string extracted from the first database;
In a second database including a plurality of records in which a character string is associated with each of a plurality of items, the number of occurrences of the item corresponding to the partial character string that matches the partial character string extracted from the first database is calculated. , Calculating in association with the item corresponding to the partial character string extracted from the first database,
A relation information generation program for executing processing including generating information on a correspondence relationship between an item included in the first database and an item included in the second database based on the calculated number of appearances.

(Appendix 10)
The supplementary note 9 wherein the partial character string is extracted from a concatenated character string obtained by concatenating the character strings associated with each of the plurality of items when the partial character string is extracted from the record included in the first database. Relationship information generation program.

(Appendix 11)
The related information generation program according to supplementary note 10, wherein when the partial character string is extracted from the concatenated character string, an item corresponding to the extracted partial character string is extracted so as to span a plurality of items.

(Appendix 12)
The m-th divided item when the entire plurality of items included in the first database are divided into M divided items, and the entire plurality of items included in the second database are divided into N divided items. The reliability that the n-th divided item is in correspondence with the divided item is the item of the second database included in the n-th divided item and the m-th divided item. Calculated based on the number of occurrences of the item associated with the item in the first database, and the combination of the m-th divided item and the n-th divided item with the highest calculated reliability is the correspondence The relation information generation program according to any one of supplementary notes 9 to 11, which is generated as information relating to the relation.

(Appendix 13)
On the computer,
Extracting a partial character string from a record included in a first database including a plurality of records in which a character string is associated with each of a plurality of items;
Of the plurality of items for the first database, specify an item corresponding to the partial character string extracted from the first database;
In a second database including a plurality of records in which a character string is associated with each of a plurality of items, the number of occurrences of the item corresponding to the partial character string that matches the partial character string extracted from the first database is calculated. , Calculating in association with the item corresponding to the partial character string extracted from the first database,
Based on the calculated number of occurrences, a relation information generation program that executes a process including generating information on a correspondence relationship between an item included in the first database and an item included in the second database Stored storage medium.

DESCRIPTION OF SYMBOLS 10 Relation information generation apparatus 11 Character string extraction part 12 Pattern generation part 13 Pattern collation part 14 Correspondence relation extraction part 20 Common dictionary database 25 Name collation table 30 Name collation source database 31A, 31B, 31C Name collation source table 41 Concatenated character string 42 Partial character Column 45 Partial character string 50 Computer 51 CPU
52 Memory 53 Storage Unit 59 Recording Medium 60 Related Information Generation Program

Claims (6)

On the computer,
Extracting a partial character string from a record included in a first database including a plurality of records in which a character string is associated with each of a plurality of items;
Of the plurality of items for the first database, specify an item corresponding to the partial character string extracted from the first database;
In a second database including a plurality of records in which a character string is associated with each of a plurality of items, the number of occurrences of the item corresponding to the partial character string that matches the partial character string extracted from the first database is calculated. , Calculating in association with the item corresponding to the partial character string extracted from the first database,
A relation information generation method for executing processing including generating information on a correspondence relationship between an item included in the first database and an item included in the second database based on the calculated number of appearances.   The partial character string is extracted from a concatenated character string obtained by concatenating character strings associated with each of the plurality of items when extracting a partial character string from a record included in the first database. Related information generation method.   The relation information generation method according to claim 2, wherein when extracting the partial character string from the concatenated character string, an item corresponding to the extracted partial character string is extracted so as to span a plurality of items.   The m-th divided item when the entire plurality of items included in the first database are divided into M divided items, and the entire plurality of items included in the second database are divided into N divided items. The reliability that the n-th divided item is in correspondence with the divided item is the item of the second database included in the n-th divided item and the m-th divided item. Calculated based on the number of occurrences of the item associated with the item in the first database, and the combination of the m-th divided item and the n-th divided item with the highest calculated reliability is the correspondence The relation information generation method according to any one of claims 1 to 3, wherein the relation information is generated as information related to a relation. An extraction unit that extracts a partial character string from a record included in a first database including a plurality of records in which a character string is associated with each of a plurality of items;
A specifying unit for specifying an item corresponding to a partial character string extracted from the first database among the plurality of items for the first database;
In a second database including a plurality of records in which a character string is associated with each of a plurality of items, the number of occurrences of the item corresponding to the partial character string that matches the partial character string extracted from the first database is calculated. A calculation unit that calculates the item corresponding to the item corresponding to the partial character string extracted from the first database;
Based on the calculated number of occurrences, a generation unit that generates information about the correspondence between items included in the first database and items included in the second database;
A relational information generation device including On the computer,
Extracting a partial character string from a record included in a first database including a plurality of records in which a character string is associated with each of a plurality of items;
Of the plurality of items for the first database, specify an item corresponding to the partial character string extracted from the first database;
In a second database including a plurality of records in which a character string is associated with each of a plurality of items, the number of occurrences of the item corresponding to the partial character string that matches the partial character string extracted from the first database is calculated. , Calculating in association with the item corresponding to the partial character string extracted from the first database,
A relation information generation program for executing processing including generating information on a correspondence relationship between an item included in the first database and an item included in the second database based on the calculated number of appearances.

Images