JP2013246547A - Data conversion device - Google Patents

Abstract

PROBLEM TO BE SOLVED: To suppress the deterioration of quality as test data about document data after personal information is masked.SOLUTION: An original document acquisition section 32 of a data conversion device 14 acquires the document data of an original from an active unit 10. A personal information detection section 34 detects personal information from among the document data of the original. As for the personal information detected by the personal information detection section 34, a displacement data acquisition section 38 acquires displacement data showing the hash value. A conversion document output section 42 outputs the document data obtained by displacing the personal information in the document data of the original with the displacement data as test data to recording media 16.

Description

  The present invention relates to a data processing technique, and more particularly to a technique for converting the contents of document data.

  In the operation phase of the information system, data accumulated in the production environment (in other words, commercial environment) may be extracted as test data for maintenance work. Then, the extracted data may be introduced into a development environment (in other words, a test environment), and various tests may be performed in the development environment.

  Data stored in the production environment may contain personal information. Currently, protection of personal information is emphasized, and techniques for masking personal information have been proposed.

JP 2006-221560 A JP 2010-152459 A

  As described above, data extracted from the production environment may be used as test data in the development environment. The inventor has come up with the idea that the quality of test data may be reduced when personal information contained in data extracted from the production environment is masked.

  The present invention has been made on the basis of the above-mentioned idea of the present inventor, and its main object is to provide a technique for suppressing deterioration in quality as test data for document data after masking personal information. It is.

  In order to solve the above problems, a data conversion apparatus according to an aspect of the present invention includes a detection unit that detects personal information from original document data, and a hash value of the personal information detected by the detection unit. A replacement data acquisition unit that acquires the replacement data to be shown, and an output unit that outputs document data obtained by replacing the personal information in the original document data with the replacement data.

  It should be noted that any combination of the above-described constituent elements, and the expression of the present invention converted between a method, a system, a program, a recording medium storing the program, and the like are also effective as an aspect of the present invention.

  According to the present invention, it is possible to suppress deterioration in quality as test data for document data after masking personal information.

It is a figure which shows the structure of the information system of embodiment. It is a block diagram which shows the function structure of the data converter of FIG. It is a figure which shows the extraction result of personal information. It is a figure which shows a correspondence rule. It is a figure which shows a replacement rule. It is a figure which shows a user setting screen. It is a flowchart which shows operation | movement of a data converter. It is a flowchart which shows the operation | movement following Fig.7 (a). It is a figure which shows typically the mask process of a some table. It is a figure which shows the mask process of log data typically.

  FIG. 1 shows a configuration of an information system 100 according to the embodiment. The information system 100 provides, for example, an information processing service for a retailer or a financial company, and an SI company takes charge of development, operation, and maintenance thereof. The information system 100 includes a production machine 10, a development machine 12, and a data conversion device 14.

  The production machine 10 is an information processing apparatus such as a web server, an application server, or a database server installed in a production environment. The production machine 10 provides commercial information processing services for customer companies and end users, and holds document data including various information corresponding to personal information of customer companies and end users. This document data includes table data managed by the database server. Also includes CSV files, free format log files, fixed length files, and the like.

  The development machine 12 is an information processing apparatus installed in the development environment. The development machine 12 is an information processing apparatus for performing work such as trouble analysis, bug repair, and function addition (hereinafter also collectively referred to as “maintenance work”) for the application installed in the production machine 10. It is. In the embodiment, in order to increase the efficiency of maintenance work in the development machine 12, document data corresponding to document data held in the production environment is used as test data used for the maintenance work.

  The data conversion apparatus 14 acquires document data in the production environment (hereinafter also referred to as “original document data”) from the production machine 10. Then, the personal information contained in the original document data is converted into document data masked (hereinafter also referred to as “test document data”) and recorded on the recording medium 16. The data converter 14 may be a general PC. The person in charge of the SI company reads the test document data recorded on the recording medium 16 into the development machine 12 and performs maintenance work on the development machine 12.

  FIG. 2 is a block diagram showing a functional configuration of the data converter 14 shown in FIG. The data conversion device 14 includes a data holding unit 20 that is a storage area for holding various types of data, and a data processing unit 30 that executes various types of data processing. The data holding unit 20 includes an extracted data holding unit 22, a correspondence relation holding unit 24, a replacement rule holding unit 26, and an exclusion rule holding unit 28. The data processing unit 30 includes an original document acquisition unit 32, a personal information detection unit 34, a replacement rule determination unit 36, a replacement data acquisition unit 38, a document conversion unit 40, a converted document output unit 42, and a user setting support. Part 44 is included.

  Each block shown in the block diagram of the present specification can be realized in terms of hardware by an element such as a CPU of a computer or a mechanical device, and in terms of software, it can be realized by a computer program or the like. The functional block realized by those cooperation is drawn. Therefore, those skilled in the art will understand that these functional blocks can be realized in various forms by a combination of hardware and software. For example, each block in FIG. 2 may be stored as a program module in a recording medium and installed in the storage of the data conversion apparatus 14 via the recording medium. And in the data converter 14, the function of each block may be implement | achieved by reading the program module corresponding to each block to a main memory at any time, and performing with CPU.

  The extracted data holding unit 22 holds the extraction result of the personal information included in the original document data. FIG. 3 shows the extraction result of personal information. The record number in the original document data is recorded in the record number field. For example, it may be a number indicating the line position of the CSV file, or may be an identification number assigned to each record of a table managed in the database. In the item name field, the name of the information item of the original document data in which personal information is set is recorded. A character string detected as personal information is set in the character string field. In the position field, the position where the character string of the personal information is set in each information item of the original document data, specifically, the number of bytes when the head is 1 is set. In the detection type field, information for identifying the type of personal information, for example, information indicating whether the personal information is a person name, a place name, a telephone number, an organization, an email address, or the like is recorded.

  Returning to FIG. 2, the correspondence relationship holding unit 24 determines the detection type of personal information and the type / mode when the personal information is replaced with another character string (in this embodiment, the character string includes a numeric string). A correspondence rule that associates the indicated mask pattern is held. This correspondence rule is predetermined in the data conversion device 14, but can be changed by the user via a user setting support unit 44 described later. FIG. 4 shows the correspondence rules. As shown in the figure, in principle, a mask pattern that matches the attribute of a character string detected as personal information (for example, a random Chinese character string mask pattern for personal information of Chinese characters) is associated.

  If the mask pattern is a “custom pattern” that allows the user to set an arbitrary format, a custom character string indicating the character string mode determined by the user is further held. For example, a custom character string “<2n>-<4n>-<4n>” is a random number string of length 2, “−”, a random number string of length 4, “−”, “length 4” The random character string "is a concatenated character string. The custom character string “<5a> @ <3a>. <2a>. <2a>” is a random ASCII character string of length 5, “@”, a random ASCII character string of length 3, “.”. , A random ASCII character string of length 2, “.”, And a random ASCII character string of length 2 are concatenated.

  Returning to FIG. 2, the replacement rule holding unit 26 holds a replacement rule in which each information item in the original document data is associated with a mask pattern. FIG. 5 shows the replacement rule. In the item name field, the name of the information item in the original document data is recorded. Information indicating whether the information item includes personal information (TRUE) or not (FALSE) is recorded in the target field. In the maximum detection type field, the type of personal information determined most frequently as the detection type of the information item is recorded, and the number of detections of the detection type is recorded in the detection number field. A mask pattern for replacing personal information is recorded in the mask pattern field, and a custom character string is further recorded for the custom pattern.

  In the replacement rule of FIG. 5, a hash value is specified as a mask pattern for the item name “ACCOUNT_NUMBER”. This is because the data of the item name “ACCOUNT_NUMBER” is also used in other tables not shown in FIG. 5 and serves as a key for associating both tables. Maintaining the relevance of multiple tables in the original document data by masking the personal information with the hash value will be described later with reference to FIG.

  Returning to FIG. 2, the exclusion rule holding unit 28 holds an exclusion rule for identifying a character string to be excluded from the masking process among the character strings detected as personal information. It is assumed that the exclusion rule of this embodiment defines one or more character strings (hereinafter also referred to as “non-maskable character strings”) that should not be masked. In the present embodiment, a character string that completely matches a character string that is not to be masked among character strings detected as personal information is excluded from mask processing. As a modified example, a character string partially including an unmasked character string may be excluded from the mask process. When the non-maskable character string is indicated by a regular expression, the characters included in the regular expression are excluded. The column may be excluded from the mask process.

  The original document acquisition unit 32 acquires original document data from the production machine 10. As described above, the original document data may be records stored in a database table, or may be various file data such as a CSV file, a fixed length file, a free format log file, and the like.

  The personal information detection unit 34 detects personal information included in the data from the original document data, and records the detection result in the extracted data holding unit 22 in the manner shown in FIG. The personal information detection unit 34 may be realized by a known personal information extraction unit. For example, it may be realized by “TRUE TELLER Personal Information Filter (registered trademark)” which is a software product provided by Nomura Research Institute, Ltd.

  The replacement rule determining unit 36 refers to the detection result of the personal information stored in the extracted data holding unit 22 and the corresponding rule stored in the correspondence holding unit 24, and replaces the personal information included in the original document data. And is recorded in the replacement rule holding unit 26. Specifically, for each information item of the original document data, it is determined whether or not personal information is detected (for example, whether or not a detection type is recorded), and the determination result is recorded. For each information item of the original document data, the number of detections of each detection type is counted to determine and record the maximum detection type. Then, the mask pattern (and custom character string) associated with the maximum detection type is specified and recorded in accordance with the correspondence rule stored in the correspondence relationship holding unit 24.

  In order to approximate the character string before the mask (a character string including personal information, hereinafter also referred to as “original character string”) and the attribute of the character string after the mask, In the detection type determination, the detection type specified by the personal information detection unit 34 is further detailed according to the character attribute. For example, if the maximum detection type in the personal information detection result is [person name] and the character string set in the character string field is kanji, the maximum detection type [person name] KANJI is recorded. If the maximum detection type in the personal information detection result is [person name] and the character string set in the character string field is hiragana, the maximum detection type [person name] KANA is recorded.

  The replacement data acquisition unit 38 refers to the replacement rules stored in the replacement rule holding unit 26 and replaces data for masking personal information for each record of the original document data and for each information item (hereinafter referred to as “replacement data”). , Also referred to as “mask data”). For example, when the mask pattern is a random character string (kanji), a random kanji character string having a length corresponding to the original character string length (the same length in this embodiment) is acquired as mask data.

  When the mask pattern is a hash value, an original character string is input to a hash function (SHA-2 in the embodiment), and a character string indicating a hash value as an output result of the hash function (hereinafter referred to as “hash character string”). Is also obtained as mask data. This hash function may be another type of hash function, for example, SHA-1 or MD5. The hash character string may be a HEX character string in which a hash value of a predetermined length is expressed in hexadecimal or a numeric string. Further, the replacement data acquisition unit 38 may acquire, as mask data, the result of trimming the hash character string to a length corresponding to the original character string length.

  As will be described later with reference to FIG. 6, a hash value may be specified even when the mask pattern is a custom pattern. At this time, the replacement data acquisition unit 38 acquires a hash value of the original character string, and acquires, as mask data, a result of trimming the character string indicating the hash value to a length specified by the custom character string.

  The document conversion unit 40 replaces the character string detected as the personal information by the personal information detection unit 34 with the mask data acquired by the replacement data acquisition unit 38 for each record of the original document data and for each information item. . Thus, the original document data is converted into test document data in which personal information is masked.

  Further, the document conversion unit 40 determines whether or not the original character string to be converted in the original document data matches the non-maskable character string determined by the exclusion rule of the exclusion rule holding unit 28. The original character string is replaced with mask data. If they match, the replacement of the original character string with the mask data is suppressed. In other words, the mask process for the original character string is skipped, and the process proceeds to the mask process for the next character string to be converted.

  The converted document output unit 42 records the test document data generated by the document conversion unit 40 on the recording medium 16. For example, if the original document data is table data in a database, the personal data masked table data is stored in the recording medium 16 as test document data. When the original document data is a free format log file, the log file data after masking the personal information is stored in the recording medium 16 as test document data. The test document data recorded on the recording medium 16 is read into the development machine 12 and used in maintenance work on the development machine 12 (for example, as input data or verification data for testing).

  The user setting support unit 44 supports the user's setting operation for the correspondence rules held in the correspondence relationship holding unit 24 and the replacement rules held in the replacement rule holding unit 26. Specifically, a user setting screen for editing the correspondence rule and the replacement rule is displayed on a predetermined display, and user input information for the user setting screen is displayed in the correspondence rule and replacement rule holding unit 26 of the correspondence relationship holding unit 24. Reflect in the replacement rule.

  FIG. 6 shows a user setting screen. This figure shows a user setting screen for editing the correspondence rule held in the correspondence relationship holding unit 24. When the user inputs the contents of FIG. 6, the correspondence is updated so that the detection type of [person name] KANJI and the mask pattern of the 8-character hash character string are associated with each other. It is also possible to select a hash character string from the pull-down menu of the mask pattern.

  The user setting support unit 44 displays a user setting screen for allowing the user to input a character string that is not to be masked. Then, the non-maskable character string input by the user on the user setting screen is acquired, and the character string is added to the exclusion rule of the exclusion rule holding unit 28.

The operation of the data converter 14 having the above configuration will be described below.
FIG. 7A is a flowchart showing the operation of the data converter 14. When the user operation for instructing the determination of the mask pattern for the document data in the production environment is detected in the data conversion device 14 (Y in S10), the original document acquisition unit 32 performs the production of the original document data designated by the user operation. Obtained from the machine 10 (S12). The personal information detection unit 34 detects personal information described in the original document data, and records attribute information including the description position in the extracted data holding unit 22 (S14). The replacement rule determination unit 36 determines the mask pattern of each personal information based on the attribute of the personal information detected by the personal information detection unit 34 and the correspondence rule held in the correspondence relationship holding unit 24, and each personal information The replacement rule is recorded in the replacement rule holding unit 26 (S16). If the user operation for instructing the determination of the mask pattern has not been detected (N in S10), S12 to S16 are skipped.

  When the user operation for instructing the change of the mask processing setting is detected in the data conversion device 14 (Y in S18), the user setting support unit 44 displays a user setting screen (S20). The user setting support unit 44 reflects the update information of the correspondence rule input on the user setting screen in the correspondence relationship holding unit 24 or the update information of the replacement rule input on the user setting screen in the replacement rule holding unit 26. Reflect (S22). If the user operation for instructing the setting change of the mask process is not detected (N in S18), S20 and S22 are skipped. Although not shown in FIG. 7A, when an unmasked character string is input on the user setting screen, the user setting support unit 44 records the unmasked character string in the exclusion rule holding unit 28.

  Typically, the user of the data conversion device 14 sets a hash value as a mask pattern for items of personal information that are described in a plurality of locations in the original document data and should maintain their relevance. More specifically, a hash value is set as a mask pattern for key information (for example, a foreign key in the first table and a primary key in the second table) for associating a plurality of tables in a relational database. Thereby, the relationship between items in the original document data, for example, the relation between a plurality of tables in the relational database can be maintained even in the test document data after masking.

  FIG. 7B is a flowchart showing the operation following FIG. When the user operation for instructing the start of the mask process is detected in the data conversion device 14 (Y in S24), the replacement data acquisition unit 38 sets the attribute for each original character string of the personal information described in the original document data. The mask data based on the mask pattern corresponding to is acquired. If the original character string detected as personal information does not match the non-maskable character string recorded in the exclusion rule holding unit 28 (N in S26), the document conversion unit 40 converts the original character string into mask data. Replace (S28). Specifically, data corresponding to the length of the original character string (that is, the original character string itself) is replaced with the character string of the mask data from the beginning position of the original character string recorded in the extracted data holding unit 22. If the original character string detected as personal information matches the non-maskable character string (Y in S26), S28 is skipped.

  When the replacement process for the original character string of all personal information described in the original document data or the replacement skip is completed (Y in S30), the converted document output unit 42 outputs the test document data to the recording medium 16 (S32). ). If the original character string of unprocessed personal information remains (N in S30), the process returns to S26, and the replacement data acquisition unit 38 acquires mask data for the unprocessed original character string. If no user operation for instructing the start of the mask process is detected (N in S24), S26 to S32 are skipped.

  According to the data conversion apparatus 14 of the present embodiment, it is not necessary for the user to define the correspondence between each information item of the original document data and the mask pattern one by one. That is, the personal information included in the original document data is automatically detected, and the mask pattern of each personal information is automatically determined based on the correspondence rule between the attribute of the personal information and the mask pattern. As a result, it is possible to suppress the occurrence of artificial mistakes (typically leakage of mask settings). For example, a spare item is provided in the original document data, and the spare item is unused at the time of initial development, and may be changed to retain personal information when a function is added later. When the mask pattern is manually set, the mask of the spare item is overlooked, and the mask setting is likely to be leaked. The data converter 14 can suppress the occurrence of human error by automating the detection of a character string to be masked and the determination of mask data to be replaced with the character string.

  In addition, on the user setting screen, not only can the replacement rule be edited, but the user can also edit the corresponding rule that is the basis of the replacement rule. By editing the correspondence rule, the user can collectively set a mask pattern for a plurality of information items of original document data having the same personal information attribute, and the efficiency of user work for mask processing can be realized.

  Further, according to the data conversion device 14, a plurality of personal information items described in different locations in the original document data, and each of the plurality of personal information items that should be maintained in relation to each other even after masking is obtained. Mask with the hash value of the indicated string. Thereby, the relationship between the information items in the original document data can be maintained even in the test document data after the personal information is masked, and the deterioration of the quality as the test data can be suppressed.

  FIG. 8 schematically shows mask processing of a plurality of tables. The name information table and the balance information table are related to each other using the account number as a key. FIG. 8A shows a name information table and a balance information table as original document data. For example, the balance of the name “Sasaki” is “300,000”. FIG. 8B shows the result of masking the account number of the nominal information table and the account number of the balance information table with random values, and the association between the nominal information table and the balance information table is lost. Yes.

  On the other hand, FIG. 8C shows the result of masking each of the account number of the nominal information table and the account number of the balance information table with a hash value. As shown in the figure, the relationship between the account number in the name information table and the account number in the balance information table is maintained while the original values are kept secret. For example, the balance of the name “XXX (Sasaki mask result)” is “300,000”, which can be identified even after masking. By using the name information table and the balance information table of FIG. 8C as test document data, it becomes easy to perform a test according to the production environment even in the development environment.

  Further, according to the data converter 14, personal information can be automatically masked even for free format file data (for example, log data) in which the position where the personal information is described is not determined. Moreover, according to the data converter 14, the user can designate an arbitrary character string as a non-maskable character string, and can support both the protection of personal information and the suppression of deterioration in the quality of test data.

  FIG. 9 schematically illustrates log data mask processing. FIG. 9A shows original log data in a production environment including two log messages. These two log messages have different formats. FIG. 9B shows test data after the personal information included in the log data of FIG. 9A is masked. As described above, the personal information detection unit 34 records the attribute / description position of the personal information included in the original log data. Then, the replacement data acquisition unit 38 acquires mask data corresponding to the attribute of the personal information, and the document conversion unit 40 specifies the description position of the personal information and replaces it with the mask data. Thereby, the masking of the personal information in the free format file data can be realized.

  In FIG. 9B, the product name “Yamada 300” that should not be masked is also masked as personal information. The user can output the product name “Yamada 300” as it is in the test log data by designating the character string “Yamada 300” as a non-maskable character string.

  Although not mentioned above, the personal information detection unit 34 may further output a personal information detection list in which character strings detected as personal information are recorded, as shown in FIG. 9C. Then, the personal information detection list may be displayed on the display and presented to the user. This list may be substituted by the personal information extraction result stored in the extracted data holding unit 22. By presenting the personal information detection list to the user, it is possible to assist the user in appropriately specifying the non-maskable character string.

  The present invention has been described based on the embodiments. This embodiment is an exemplification, and it will be understood by those skilled in the art that various modifications can be made to combinations of the respective constituent elements and processing processes, and such modifications are also within the scope of the present invention. is there. A modification is shown below.

A first modification will be described.
In the above embodiment, the user designates the personal information item to be masked with the hash value, but the replacement rule determining unit 36 automatically determines the personal information item to be masked with the hash value and records it in the replacement rule. May be. For example, referential integrity constraints set between a plurality of tables in the relational database may be detected by referring to the definition information of the relational database. A hash value may be set as a mask pattern for columns for which referential integrity constraints are set (typically, both the foreign key column in the first table and the primary key column in the second table).

A second modification will be described.
As described above in part in the above embodiment, when mask data is obtained by trimming a hash character string to a length corresponding to the original character string length, different original character strings are converted to the same mask data. It is possible to end up. Therefore, as a modification, the replacement data acquisition unit 38 may hold a table (hereinafter, “allocation history table”) in which the original character string is associated with the mask data obtained by trimming the hash value.

  The replacement data acquisition unit 38 refers to the allocation history table when acquiring mask data (here, a character string obtained by trimming a hash character string) of a certain original character string (referred to as “the original character string”). Then, it is determined whether the mask data has been assigned to the character string that matches the original character string. If it has been assigned, the mask data assigned to the character string that matches the original character string is assigned to the original character string.

  If a character string that matches the original character string is not recorded in the allocation history table, the replacement data acquisition unit 38 acquires mask data obtained by trimming the hash value of the original character string. Then, with reference to the allocation history table, it is determined whether or not the mask data has already been allocated to another original character string. If not assigned, the mask data is assigned to the original character string and recorded in the assignment history table. If it has been assigned to another original character string, the hash value of the original character string is input to the hash function, and the hash value that is the output result is obtained as new mask data. Thereafter, the above process is repeated until unique mask data is acquired.

  According to this modification, when a mask pattern is set to a hash value and the hash character string is trimmed, it is possible to avoid assigning overlapping mask data to different original character strings. Thereby, it is possible to avoid the occurrence of relevance in the test document data for a plurality of information items that are not relevant in the original document data.

  It should also be understood by those skilled in the art that the functions to be fulfilled by the constituent elements recited in the claims are realized by the individual constituent elements shown in the embodiments and the modified examples or by their linkage.

  DESCRIPTION OF SYMBOLS 14 Data converter, 22 Extraction data holding part, 24 Correspondence relation holding part, 26 Replacement rule holding part, 28 Exclusion rule holding part, 32 Original document acquisition part, 34 Personal information detection part, 36 Replacement rule determination part, 38 Replacement data An acquisition unit, 40 document conversion unit, 42 conversion document output unit, 44 user setting support unit.

Claims (5)

A detection unit that detects personal information from the original document data;
For the personal information detected by the detection unit, a replacement data acquisition unit that acquires replacement data indicating the hash value;
An output unit for outputting document data obtained by replacing personal information in the original document data with the replacement data;
A data conversion device comprising: It further includes a holding unit that holds a replacement rule that defines replacement data according to the attribute of personal information,
The replacement rule is described in a plurality of locations in the original document data, and for personal information that should maintain the relevance of each other, the hash value of the personal information is defined as the replacement data,
The data conversion apparatus according to claim 1, wherein the replacement data acquisition unit acquires replacement data for the detected personal information according to the replacement rule. The original document data includes data of a plurality of tables associated with personal information as a key,
The replacement data acquisition unit acquires the hash value of personal information as the key as the replacement data, thereby maintaining association of the plurality of tables in the document data output from the output unit. Item 3. The data conversion device according to Item 1 or 2. It further includes an exclusion rule holding unit that defines a specific character string to be excluded from replacement,
The output unit suppresses replacement with the replacement data for a character string detected as personal information by the detection unit that matches a character string determined by the exclusion rule. The data conversion device according to any one of 1 to 3. A function to detect personal information from the original document data;
A function of acquiring replacement data indicating a hash value of the personal information detected by the function of detecting;
A function of outputting document data obtained by replacing personal information in the original document data with the replacement data;
A computer program for realizing a computer.

Images