JP2015041145A - Personal information detection device and computer program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To make processing of identifying a mask object portion within a document efficient.SOLUTION: An original document acquisition part 32 acquires a document in which a character string relating to a plurality of items is recorded. A determination part 54 determines whether personal information is included in the character string of remaining items obtained by excluding a personal information item designated by a user, and detects the item having the personal information included as a personal information item. A detection result output part 56 outputs the personal information item designated by the user and the personal information item detected by the determination part 54 as an item to undergo mask processing of personal information. The determination part 54 sequentially determines whether the personal information is included in the character string of each item of a plurality of records within the document, and when the personal information is included in a specific item of a certain record, determines the specific item as the personal information item, and excludes the specific item from the object for determination of whether the personal information is included as to the character string of items of subsequent records.

Description

  The present invention relates to a data processing technique, and more particularly to a technique for detecting personal information recorded in a document.

  In the operation phase of the information system, data accumulated in the production environment (in other words, commercial environment) may be extracted as test data for maintenance work. Then, the extracted data may be introduced into a development environment (in other words, a test environment), and various tests may be performed in the development environment.

  Data stored in the production environment may contain personal information. Currently, protection of personal information is emphasized, and techniques for masking personal information have been proposed.

JP 2013-105274 A JP 2011-034264 A

  Before the process of masking personal information in the document, it is necessary to identify where in the document the character string indicating the personal information to be masked is written. By the way, documents including personal information accumulated in the production environment may be very large in size, and it may take a lot of time to identify the location where the personal information is written in the document. there were. The period allowed for the test is limited in the system development process, and the present inventor considered that it is necessary to improve the efficiency of the process of identifying the mask target portion in the document.

  The present invention has been made on the basis of the above-mentioned problem recognition of the present inventor, and its main object is to provide a technique for improving the efficiency of the process of identifying a mask target portion in a document.

  In order to solve the above problems, a personal information detection device according to an aspect of the present invention includes a document acquisition unit that acquires a document in which character strings related to a plurality of items are recorded, and items specified by a user, Whether or not personal information is included in the character string of other items excluding the personal information items specified by the user and the specified acquisition unit that acquires information indicating personal information items including personal information among the items A determination unit that determines and detects an item including personal information as a personal information item, and the personal information item specified by the user and the personal information item detected by the determination unit are items to be masked with personal information. An output unit for outputting. The document acquired by the document acquisition unit includes a plurality of records, each record includes a character string related to a plurality of items, and the determination unit determines whether the character string of each item of the plurality of records includes personal information. When it is determined that personal information is included in a specific item of a record, the item is detected as a personal information item, and the character string of the item in the remaining records includes personal information. Is excluded from the object to be determined.

  It should be noted that any combination of the above-described constituent elements, and the expression of the present invention converted between a method, a system, a program, a recording medium storing the program, and the like are also effective as an aspect of the present invention.

  According to the present invention, it is possible to improve the efficiency of processing for identifying a mask target portion in a document.

It is a figure which shows the structure of the information system of 1st Embodiment. It is a block diagram which shows the function structure of the data converter of FIG. It is a figure which shows the extraction result of personal information. It is a figure which shows a correspondence rule. It is a figure which shows a replacement rule. It is a figure which shows a user setting screen. It is a flowchart which shows operation | movement of a data converter. It is a flowchart which shows the operation | movement following Fig.7 (a). It is a figure which shows typically the mask process of a some table. It is a figure which shows the mask process of log data typically. It is a figure which shows the original document data before masking personal information. It is a block diagram which shows the function structure of the data converter of 2nd Embodiment.

  Hereinafter, as a first embodiment, a technique for suppressing deterioration in quality as test data for a document after masking personal information will be described. As a second embodiment, a technique for improving the efficiency of the process of identifying a mask target portion in a document in order to suppress the total time required for the masking process of personal information will be described.

(First embodiment)
FIG. 1 shows a configuration of an information system 100 according to the first embodiment. The information system 100 provides, for example, an information processing service for a retailer or a financial company, and an SI company takes charge of development, operation, and maintenance thereof. The information system 100 includes a production machine 10, a development machine 12, and a data conversion device 14.

  The production machine 10 is an information processing apparatus such as a web server, an application server, or a database server installed in a production environment. The production machine 10 provides commercial information processing services for customer companies and end users, and holds document data including various information corresponding to personal information of customer companies and end users. This document data includes table data managed by the database server. Also includes CSV files, free format log files, fixed length files, and the like.

  The development machine 12 is an information processing apparatus installed in the development environment. The development machine 12 is an information processing apparatus for performing work such as trouble analysis, bug repair, and function addition (hereinafter also collectively referred to as “maintenance work”) for the application installed in the production machine 10. It is. In the embodiment, in order to increase the efficiency of maintenance work in the development machine 12, document data corresponding to document data held in the production environment is used as test data used for the maintenance work.

  The data conversion apparatus 14 acquires document data in the production environment (hereinafter also referred to as “original document data”) from the production machine 10. Then, the personal information contained in the original document data is converted into document data masked (hereinafter also referred to as “test document data”) and recorded on the recording medium 16. The data converter 14 may be a general PC. The person in charge of the SI company reads the test document data recorded on the recording medium 16 into the development machine 12 and performs maintenance work on the development machine 12.

  FIG. 2 is a block diagram showing a functional configuration of the data converter 14 shown in FIG. The data conversion device 14 includes a data holding unit 20 that is a storage area for holding various types of data, and a data processing unit 30 that executes various types of data processing. The data holding unit 20 includes an extracted data holding unit 22, a correspondence relation holding unit 24, a replacement rule holding unit 26, and an exclusion rule holding unit 28. The data processing unit 30 includes an original document acquisition unit 32, a personal information detection unit 34, a replacement rule determination unit 36, a replacement data acquisition unit 38, a document conversion unit 40, a converted document output unit 42, and a user setting support. Part 44 is included.

  Each block shown in the block diagram of the present specification can be realized in terms of hardware by an element such as a CPU of a computer or a mechanical device, and in terms of software, it can be realized by a computer program or the like. The functional block realized by those cooperation is drawn. Therefore, those skilled in the art will understand that these functional blocks can be realized in various forms by a combination of hardware and software. For example, each block in FIG. 2 may be stored as a program module in a recording medium and installed in the storage of the data conversion apparatus 14 via the recording medium. And in the data converter 14, the function of each block may be implement | achieved by reading the program module corresponding to each block to a main memory at any time, and performing with CPU.

  The extracted data holding unit 22 holds the extraction result of the personal information included in the original document data. FIG. 3 shows the extraction result of personal information. The record number in the original document data is recorded in the record number field. For example, it may be a number indicating the line position of the CSV file, or may be an identification number assigned to each record of a table managed in the database. In the item name field, the name of the information item of the original document data in which personal information is set is recorded. A character string detected as personal information is set in the character string field. In the position field, the position where the character string of the personal information is set in each information item of the original document data, specifically, the number of bytes when the head is 1 is set. In the detection type field, information for identifying the type of personal information, for example, information indicating whether the personal information is a person name, a place name, a telephone number, an organization, an email address, or the like is recorded.

  Returning to FIG. 2, the correspondence relationship holding unit 24 determines the detection type of personal information and the type / mode when the personal information is replaced with another character string (in this embodiment, the character string includes a numeric string). A correspondence rule that associates the indicated mask pattern is held. This correspondence rule is predetermined in the data conversion device 14, but can be changed by the user via a user setting support unit 44 described later. FIG. 4 shows the correspondence rules. As shown in the figure, in principle, a mask pattern that matches the attribute of a character string detected as personal information (for example, a random Chinese character string mask pattern for personal information of Chinese characters) is associated.

  If the mask pattern is a “custom pattern” that allows the user to set an arbitrary format, a custom character string indicating the character string mode determined by the user is further held. For example, a custom character string “<2n>-<4n>-<4n>” is a random number string of length 2, “−”, a random number string of length 4, “−”, “length 4” The random character string "is a concatenated character string. The custom character string “<5a> @ <3a>. <2a>. <2a>” is a random ASCII character string of length 5, “@”, a random ASCII character string of length 3, “.”. , A random ASCII character string of length 2, “.”, And a random ASCII character string of length 2 are concatenated.

  Returning to FIG. 2, the replacement rule holding unit 26 holds a replacement rule in which each information item in the original document data is associated with a mask pattern. FIG. 5 shows the replacement rule. In the item name field, the name of the information item in the original document data is recorded. Information indicating whether the information item includes personal information (TRUE) or not (FALSE) is recorded in the target field. In the maximum detection type field, the type of personal information determined most frequently as the detection type of the information item is recorded, and the number of detections of the detection type is recorded in the detection number field. A mask pattern for replacing personal information is recorded in the mask pattern field, and a custom character string is further recorded for the custom pattern.

  In the replacement rule of FIG. 5, a hash value is specified as a mask pattern for the item name “ACCOUNT_NUMBER”. This is because the data of the item name “ACCOUNT_NUMBER” is also used in other tables not shown in FIG. 5 and serves as a key for associating both tables. Maintaining the relevance of multiple tables in the original document data by masking the personal information with the hash value will be described later with reference to FIG.

  Returning to FIG. 2, the exclusion rule holding unit 28 holds an exclusion rule for identifying a character string to be excluded from the masking process among the character strings detected as personal information. It is assumed that the exclusion rule of this embodiment defines one or more character strings (hereinafter also referred to as “non-maskable character strings”) that should not be masked. In the present embodiment, a character string that completely matches a character string that is not to be masked among character strings detected as personal information is excluded from mask processing. As a modified example, a character string partially including an unmasked character string may be excluded from the mask process. When the non-maskable character string is indicated by a regular expression, the characters included in the regular expression are excluded. The column may be excluded from the mask process.

  The original document acquisition unit 32 acquires original document data from the production machine 10. As described above, the original document data may be records stored in a database table, or may be various file data such as a CSV file, a fixed length file, a free format log file, and the like.

  The personal information detection unit 34 detects personal information included in the data from the original document data, and records the detection result in the extracted data holding unit 22 in the manner shown in FIG. The personal information detection unit 34 may be realized by a known personal information extraction unit. For example, it may be realized by “TRUE TELLER Personal Information Filter (registered trademark)” which is a software product provided by Nomura Research Institute, Ltd.

  The replacement rule determining unit 36 refers to the detection result of the personal information stored in the extracted data holding unit 22 and the corresponding rule stored in the correspondence holding unit 24, and replaces the personal information included in the original document data. And is recorded in the replacement rule holding unit 26. Specifically, for each information item of the original document data, it is determined whether or not personal information is detected (for example, whether or not a detection type is recorded), and the determination result is recorded. For each information item of the original document data, the number of detections of each detection type is counted to determine and record the maximum detection type. Then, the mask pattern (and custom character string) associated with the maximum detection type is specified and recorded in accordance with the correspondence rule stored in the correspondence relationship holding unit 24.

  In order to approximate the character string before the mask (a character string including personal information, hereinafter also referred to as “original character string”) and the attribute of the character string after the mask, In the detection type determination, the detection type specified by the personal information detection unit 34 is further detailed according to the character attribute. For example, if the maximum detection type in the personal information detection result is [person name] and the character string set in the character string field is kanji, the maximum detection type [person name] KANJI is recorded. If the maximum detection type in the personal information detection result is [person name] and the character string set in the character string field is hiragana, the maximum detection type [person name] KANA is recorded.

  The replacement data acquisition unit 38 refers to the replacement rules stored in the replacement rule holding unit 26 and replaces data for masking personal information for each record of the original document data and for each information item (hereinafter referred to as “replacement data”). , Also referred to as “mask data”). For example, when the mask pattern is a random character string (kanji), a random kanji character string having a length corresponding to the original character string length (the same length in this embodiment) is acquired as mask data.

  When the mask pattern is a hash value, an original character string is input to a hash function (SHA-2 in the embodiment), and a character string indicating a hash value as an output result of the hash function (hereinafter referred to as “hash character string”). Is also obtained as mask data. This hash function may be another type of hash function, for example, SHA-1 or MD5. The hash character string may be a HEX character string in which a hash value of a predetermined length is expressed in hexadecimal or a numeric string. Further, the replacement data acquisition unit 38 may acquire, as mask data, the result of trimming the hash character string to a length corresponding to the original character string length.

  As will be described later with reference to FIG. 6, a hash value may be specified even when the mask pattern is a custom pattern. At this time, the replacement data acquisition unit 38 acquires a hash value of the original character string, and acquires, as mask data, a result of trimming the character string indicating the hash value to a length specified by the custom character string.

  The document conversion unit 40 replaces the character string detected as the personal information by the personal information detection unit 34 with the mask data acquired by the replacement data acquisition unit 38 for each record of the original document data and for each information item. . Thus, the original document data is converted into test document data in which personal information is masked.

  Further, the document conversion unit 40 determines whether or not the original character string to be converted in the original document data matches the non-maskable character string determined by the exclusion rule of the exclusion rule holding unit 28. The original character string is replaced with mask data. If they match, the replacement of the original character string with the mask data is suppressed. In other words, the mask process for the original character string is skipped, and the process proceeds to the mask process for the next character string to be converted.

  The converted document output unit 42 records the test document data generated by the document conversion unit 40 on the recording medium 16. For example, if the original document data is table data in a database, the personal data masked table data is stored in the recording medium 16 as test document data. When the original document data is a free format log file, the log file data after masking the personal information is stored in the recording medium 16 as test document data. The test document data recorded on the recording medium 16 is read into the development machine 12 and used in maintenance work on the development machine 12 (for example, as input data or verification data for testing).

  The user setting support unit 44 supports the user's setting operation for the correspondence rules held in the correspondence relationship holding unit 24 and the replacement rules held in the replacement rule holding unit 26. Specifically, a user setting screen for editing the correspondence rule and the replacement rule is displayed on a predetermined display, and user input information for the user setting screen is displayed in the correspondence rule and replacement rule holding unit 26 of the correspondence relationship holding unit 24. Reflect in the replacement rule.

  FIG. 6 shows a user setting screen. This figure shows a user setting screen for editing the correspondence rule held in the correspondence relationship holding unit 24. When the user inputs the contents of FIG. 6, the correspondence is updated so that the detection type of [person name] KANJI and the mask pattern of the 8-character hash character string are associated with each other. It is also possible to select a hash character string from the pull-down menu of the mask pattern.

  The user setting support unit 44 displays a user setting screen for allowing the user to input a character string that is not to be masked. Then, the non-maskable character string input by the user on the user setting screen is acquired, and the character string is added to the exclusion rule of the exclusion rule holding unit 28.

The operation of the data converter 14 having the above configuration will be described below.
FIG. 7A is a flowchart showing the operation of the data converter 14. When the user operation for instructing the determination of the mask pattern for the document data in the production environment is detected in the data conversion device 14 (Y in S10), the original document acquisition unit 32 performs the production of the original document data designated by the user operation. Obtained from the machine 10 (S12). The personal information detection unit 34 detects personal information described in the original document data, and records attribute information including the description position in the extracted data holding unit 22 (S14). The replacement rule determination unit 36 determines the mask pattern of each personal information based on the attribute of the personal information detected by the personal information detection unit 34 and the correspondence rule held in the correspondence relationship holding unit 24, and each personal information The replacement rule is recorded in the replacement rule holding unit 26 (S16). If the user operation for instructing the determination of the mask pattern is not detected (N in S10), S12 to S16 are skipped.

  When the user operation for instructing the change of the mask processing setting is detected in the data conversion device 14 (Y in S18), the user setting support unit 44 displays a user setting screen (S20). The user setting support unit 44 reflects the update information of the correspondence rule input on the user setting screen in the correspondence relationship holding unit 24 or the update information of the replacement rule input on the user setting screen in the replacement rule holding unit 26. Reflect (S22). If the user operation for instructing the setting change of the mask process is not detected (N in S18), S20 and S22 are skipped. Although not shown in FIG. 7A, when an unmasked character string is input on the user setting screen, the user setting support unit 44 records the unmasked character string in the exclusion rule holding unit 28.

  Typically, the user of the data conversion device 14 sets a hash value as a mask pattern for items of personal information that are described in a plurality of locations in the original document data and should maintain their relevance. More specifically, a hash value is set as a mask pattern for key information (for example, a foreign key in the first table and a primary key in the second table) for associating a plurality of tables in a relational database. Thereby, the relationship between items in the original document data, for example, the relation between a plurality of tables in the relational database can be maintained even in the test document data after masking.

  FIG. 7B is a flowchart showing the operation following FIG. When the user operation for instructing the start of the mask process is detected in the data conversion device 14 (Y in S24), the replacement data acquisition unit 38 sets the attribute for each original character string of the personal information described in the original document data. The mask data based on the mask pattern corresponding to is acquired. If the original character string detected as personal information does not match the non-maskable character string recorded in the exclusion rule holding unit 28 (N in S26), the document conversion unit 40 converts the original character string into mask data. Replace (S28). Specifically, data corresponding to the length of the original character string (that is, the original character string itself) is replaced with the character string of the mask data from the beginning position of the original character string recorded in the extracted data holding unit 22. If the original character string detected as personal information matches the non-maskable character string (Y in S26), S28 is skipped.

  When the replacement process for the original character string of all personal information described in the original document data or the replacement skip is completed (Y in S30), the converted document output unit 42 outputs the test document data to the recording medium 16 (S32). ). If the original character string of unprocessed personal information remains (N in S30), the process returns to S26, and the replacement data acquisition unit 38 acquires mask data for the unprocessed original character string. If no user operation for instructing the start of the mask process is detected (N in S24), S26 to S32 are skipped.

  According to the data conversion apparatus 14 of the present embodiment, it is not necessary for the user to define the correspondence between each information item of the original document data and the mask pattern one by one. That is, the personal information included in the original document data is automatically detected, and the mask pattern of each personal information is automatically determined based on the correspondence rule between the attribute of the personal information and the mask pattern. As a result, it is possible to suppress the occurrence of artificial mistakes (typically leakage of mask settings). For example, a spare item is provided in the original document data, and the spare item is unused at the time of initial development, and may be changed to retain personal information when a function is added later. When the mask pattern is manually set, the mask of the spare item is overlooked, and the mask setting is likely to be leaked. The data converter 14 can suppress the occurrence of human error by automating the detection of a character string to be masked and the determination of mask data to be replaced with the character string.

  In addition, on the user setting screen, not only can the replacement rule be edited, but the user can also edit the corresponding rule that is the basis of the replacement rule. By editing the correspondence rule, the user can collectively set a mask pattern for a plurality of information items of original document data having the same personal information attribute, and the efficiency of user work for mask processing can be realized.

  Further, according to the data conversion device 14, a plurality of personal information items described in different locations in the original document data, and each of the plurality of personal information items that should be maintained in relation to each other even after masking is obtained. Mask with the hash value of the indicated string. Thereby, the relationship between the information items in the original document data can be maintained even in the test document data after the personal information is masked, and the deterioration of the quality as the test data can be suppressed.

  FIG. 8 schematically shows mask processing of a plurality of tables. The name information table and the balance information table are related to each other using the account number as a key. FIG. 8A shows a name information table and a balance information table as original document data. For example, the balance of the name “Sasaki” is “300,000”. FIG. 8B shows the result of masking the account number of the nominal information table and the account number of the balance information table with random values, and the association between the nominal information table and the balance information table is lost. Yes.

  On the other hand, FIG. 8C shows the result of masking each of the account number of the nominal information table and the account number of the balance information table with a hash value. As shown in the figure, the relationship between the account number in the name information table and the account number in the balance information table is maintained while the original values are kept secret. For example, the balance of the name “XXX (Sasaki mask result)” is “300,000”, which can be identified even after masking. By using the name information table and the balance information table of FIG. 8C as test document data, it becomes easy to perform a test according to the production environment even in the development environment.

  Further, according to the data converter 14, personal information can be automatically masked even for free format file data (for example, log data) in which the position where the personal information is described is not determined. Moreover, according to the data converter 14, the user can designate an arbitrary character string as a non-maskable character string, and can support both the protection of personal information and the suppression of deterioration in the quality of test data.

  FIG. 9 schematically illustrates log data mask processing. FIG. 9A shows original log data in a production environment including two log messages. These two log messages have different formats. FIG. 9B shows test data after the personal information included in the log data of FIG. 9A is masked. As described above, the personal information detection unit 34 records the attribute / description position of the personal information included in the original log data. Then, the replacement data acquisition unit 38 acquires mask data corresponding to the attribute of the personal information, and the document conversion unit 40 specifies the description position of the personal information and replaces it with the mask data. Thereby, the masking of the personal information in the free format file data can be realized.

  In FIG. 9B, the product name “Yamada 300” that should not be masked is also masked as personal information. The user can output the product name “Yamada 300” as it is in the test log data by designating the character string “Yamada 300” as a non-maskable character string.

  Although not mentioned above, the personal information detection unit 34 may further output a personal information detection list in which character strings detected as personal information are recorded, as shown in FIG. 9C. Then, the personal information detection list may be displayed on the display and presented to the user. This list may be substituted by the personal information extraction result stored in the extracted data holding unit 22. By presenting the personal information detection list to the user, it is possible to assist the user in appropriately specifying the non-maskable character string.

  The present invention has been described based on the first embodiment. This embodiment is an exemplification, and it will be understood by those skilled in the art that various modifications can be made to combinations of the respective constituent elements and processing processes, and such modifications are also within the scope of the present invention. is there. A modification is shown below.

A first modification of the first embodiment will be described.
In the above embodiment, the user designates the personal information item to be masked with the hash value, but the replacement rule determining unit 36 automatically determines the personal information item to be masked with the hash value and records it in the replacement rule. May be. For example, referential integrity constraints set between a plurality of tables in the relational database may be detected by referring to the definition information of the relational database. A hash value may be set as a mask pattern for columns for which referential integrity constraints are set (typically, both the foreign key column in the first table and the primary key column in the second table).

A second modification of the first embodiment will be described.
As described above in part in the above embodiment, when mask data is obtained by trimming a hash character string to a length corresponding to the original character string length, different original character strings are converted to the same mask data. It is possible to end up. Therefore, as a modification, the replacement data acquisition unit 38 may hold a table (hereinafter, “allocation history table”) in which the original character string is associated with the mask data obtained by trimming the hash value.

  The replacement data acquisition unit 38 refers to the allocation history table when acquiring mask data (here, a character string obtained by trimming a hash character string) of a certain original character string (referred to as “the original character string”). Then, it is determined whether the mask data has been assigned to the character string that matches the original character string. If it has been assigned, the mask data assigned to the character string that matches the original character string is assigned to the original character string.

  If a character string that matches the original character string is not recorded in the allocation history table, the replacement data acquisition unit 38 acquires mask data obtained by trimming the hash value of the original character string. Then, with reference to the allocation history table, it is determined whether or not the mask data has already been allocated to another original character string. If not assigned, the mask data is assigned to the original character string and recorded in the assignment history table. If it has been assigned to another original character string, the hash value of the original character string is input to the hash function, and the hash value that is the output result is obtained as new mask data. Thereafter, the above process is repeated until unique mask data is acquired.

  According to this modification, when a mask pattern is set to a hash value and the hash character string is trimmed, it is possible to avoid assigning overlapping mask data to different original character strings. Thereby, it is possible to avoid the occurrence of relevance in the test document data for a plurality of information items that are not relevant in the original document data.

(Second Embodiment)
As described above, in the second embodiment, in order to reduce the total time required for the masking process of personal information described in the first embodiment, the mask target portion in the document, in other words, the document A technique for improving the efficiency of the process of specifying the recording position of personal information in the network will be described. The configuration of the information system in the second embodiment is the same as the configuration of the information system 100 shown in FIG.

  FIG. 10 shows original document data before masking personal information. In the second embodiment, data in a table format of a relational database is exemplified as original document data. However, the original document data is not limited to the table format, but may be a document data / document file of another format in which information is electronically recorded by dividing into a plurality of items such as a CSV format and a spreadsheet.

  In the original document data of FIG. 10, four records (ID001 to 004) are recorded, and each record includes character strings regarding five items of ID, name, account number, address, and remarks. The item can be referred to as a heading or index indicating the type or category of data, and corresponds to, for example, a column name in the table. In the actual large-scale information system 100, the scale of the original document data acquired from the production machine 10 may be several million records or more. In addition, the number of items in each record may be several tens to several hundreds.

  Whether the personal information detection process so far acquires (scans) a character string recorded in each of a plurality of items in units of records and includes any of enormous keywords indicating personal information stored in advance in the database. Had confirmed. This confirmation process was repeated for the number of records. Therefore, as the number of records in the document increases and the number of items in each record increases, the time required for the personal information detection process increases.

Therefore, in the data converter 14 of the second embodiment,
(1) Due to the design of the original document data (in other words, the production machine 10), items that should obviously contain personal information are excluded from the personal information detection process.
(2) Items for which the presence of personal information has been detected will be excluded from personal information detection processing thereafter.
The combination of these two ideas makes the personal information detection process more efficient. As a result, the personal information masking process for the original document data of enormous size stored in the actual production machine 10 is supported so that it can be completed within the time allowed for the actual system development.

  FIG. 11 is a block diagram illustrating a functional configuration of the data conversion apparatus 14 according to the second embodiment. Among the functional blocks shown in FIG. 11, functional blocks that are the same as or correspond to those already described in the first embodiment are denoted by the same reference numerals. Hereinafter, the contents already described in the first embodiment are omitted as appropriate.

  The data holding unit 20 of the data conversion device 14 further includes a determination criterion holding unit 50. The personal information detection unit 34 of the data processing unit 30 includes a user designation acquisition unit 52, a determination unit 54, and a detection result output unit 56.

  The determination criterion holding unit 50 holds data serving as a reference for the determination unit 54 described later to determine whether or not personal information is included in a character string. In the embodiment, a plurality of keywords indicating personal information are held as reference data. In other words, the determination reference holding unit 50 holds dictionary data of personal information. For example, keywords “Yamada”, “Taro”, etc. may be held as personal information related to the name of the person. In addition, keywords “Tokyo”, “Shibuya-ku”, and the like may be held as personal information regarding the address.

  The original document acquisition unit 32 is document data including a plurality of records as shown in FIG. 10, and acquires and reads original document data in which character strings relating to a plurality of items are electronically recorded in each record from the production machine 10. .

  The user designation acquisition unit 52 designates an item (hereinafter referred to as “personal information designation item”) that should include personal information in the design of the original document data among a plurality of items contained in the original document data. Is acquired from the person in charge of maintenance / test work (hereinafter simply referred to as “user”). For example, the user may record information indicating that the name column, the account number column, and the address column in the table of FIG. 10 are personal information designation items in a predetermined electronic file. The user designation acquisition unit 52 may read the electronic file and identify the name column, the account number column, and the address column as personal information designation items.

  For the remaining items (hereinafter referred to as “scan target items”) excluding the personal information designation items designated by the user among the plurality of items included in the original document data, the determination unit 54 has a character string indicating personal information. It is determined for each record whether or not it is recorded. Specifically, the determination unit 54 reads a character string recorded in each of one or more items to be scanned for each record of original document data. Then, it is determined whether or not the read character string includes any of the keywords of the personal information stored in the determination criterion holding unit 50.

  In the embodiment, the determination unit 54 horizontally scans one record of the original document data illustrated in FIG. 10 and determines the next record when the determination on the character strings of all scan target items included in the one record is completed. Move on. That is, the presence / absence of personal information is sequentially determined for each item in units of one record.

  However, when the determination unit 54 determines that the keyword of the personal information is included in the character string of a specific scan target item in a certain record, the scan target item is referred to as an item including personal information (hereinafter referred to as “personal information detection item”). )). The personal information detection item can be said to be an item in which personal information is recorded in at least one record of the original document data. When determining the personal information detection item, the determination unit 54 excludes the item from the scan target item. In other words, the determination unit 54 excludes the personal information detection item from the target for determining the presence / absence of personal information in the determination process for the remaining records in which the presence / absence of personal information has not been determined.

  As described above, the determination unit 54 determines whether or not the personal information exists for both the personal information specification item statically specified by the user and the personal information detection item dynamically detected by the autonomous determination processing. To skip. Typically, as the determination process of the original document data in units of records proceeds, the number of personal information detection items increases, while the number of items to be scanned decreases.

  As a modification, the determination unit 54 scans the original document data illustrated in FIG. 10 vertically in units of items, and when the determination on the character strings of all the records for one item is completed, the determination unit 54 proceeds to determination of the next item. Good. That is, the presence / absence of personal information may be sequentially determined over a plurality of records in units of one item. Also in this case, when the presence of personal information is detected in a certain item, the item may be determined as a personal information detection item, the determination of the item may be terminated, and the determination of the next item may be started.

  The detection result output unit 56 sends the personal information detection result, which is information indicating both the personal information specification item acquired by the user specification acquisition unit 52 and the personal information detection item detected by the determination unit 54, to an external function block. Output. The personal information detection result is an item in which personal information is stored among a plurality of items included in the original document data, and can be said to be information indicating an item to be masked with respect to the stored character string.

  The functional block that is the output destination of the personal information detection result performs masking on the character string recorded in the personal information designation item and the personal information detection item of the original document data. Specifically, all of the character strings recorded in the personal information designation item and the personal information detection item are masked according to a predetermined rule, in other words, another character string from which personal information is excluded (hereinafter, “ Also called “mask character string”.

  For example, the detection result output unit 56 may output the personal information detection result to the replacement rule determination unit 36. The replacement rule determination unit 36 may uniformly determine a replacement rule for each character string of the personal information designation item and the personal information detection item for each item. The document conversion unit 40 may convert the character string recorded in each of the personal information designation item and the personal information detection item into a mask character string according to the replacement rule determined by the replacement rule determination unit 36.

The operation of the data converter 14 having the above configuration will be described.
When a user operation instructing a mask process for document data stored in the production environment is detected by the data conversion device 14, the original document acquisition unit 32 reads the original document data designated by the user operation (here, the table of FIG. 10). Data) is acquired from the production machine 10 via the communication network. The user designation acquisition unit 52 acquires information indicating personal information designation items designated in advance by the user. Here, it is assumed that the name column, account number column, and address column in FIG. 10 are designated as personal information designation items.

  The determination unit 54 determines whether the character string of each item includes personal information for each record of the original document data. At that time, the character string of the personal information designation item designated by the user is excluded from the determination target. Therefore, the presence / absence of personal information is determined for the ID column and the remarks column of FIG. In addition, once the determination unit 54 detects the fact that a character string indicating personal information is stored for a certain item, the determination unit 54 determines that item as a personal information detection item, and determines the character string of the personal information detection item in subsequent records. Exclude from the target. For example, it is detected that personal information (for example, a telephone number) is included in the remarks column for the record of ID003 in FIG. 10, and the remarks column is determined as a personal information detection item. For records subsequent to ID004, the presence / absence of personal information is determined only for the ID column.

  The detection result output unit 56 converts the personal information designation item and the personal information detection item, in other words, information indicating the item in which the mask target character string is stored into other functional blocks that perform mask processing, for example, the replacement rule determination unit 36. To notify. You may notify by recording the information on the memory area which the replacement rule determination part 36 refers. Thereafter, a conversion process (for example, S16 and after in FIG. 7A) for replacing the personal information included in the original document data with the mask character string is executed. In the original document data of FIG. 10, masking processing may be performed on the character strings of the name column, the account number column, the address column, and the remarks column. Finally, test document data in which personal information in the original document data is masked is generated.

  According to the data conversion apparatus 14 of the embodiment, the personal information designation item that is statically designated is excluded from the items that determine the presence / absence of personal information in the original document data, and the dynamically detected individual We will exclude information detection items additionally. As a result, while proceeding with the personal information scan for the original document data in units of items, the number of items to be scanned is reduced in advance and dynamically reduced so that the masked portion of the personal information in the original document data It is possible to increase the efficiency of the process of identifying In other words, it is possible to efficiently execute the process for specifying the recording position of the personal information in the original document data. As a result, even if the size of the original document data (the number of records and the number of items) increases, it is possible to suppress an increase in the time required for identifying the mask target portion in the document.

Further, according to the data conversion device 14, the possibility of the detection failure of the personal information recorded in the original document data can be made as close to zero as possible. Here, the accuracy of the automatic detection of personal information by the determination unit 54 is considered as follows. First of all, it is difficult to specify an accurate number of detection rates using a dictionary. For example, Japanese surnames can be almost covered as for surnames, but there is no information on the coverage rate because the names vary greatly. Therefore, assume that the detection rate by the dictionary is 80%, and the number of records to be detected is 10,000. Here, the presence / absence of personal information is detected for one specific item. The detection accuracy of personal information in this case is
1-((1-0.8) ^ 10000) ("^" represents power)
And almost 100%. In the case of 10 records, it becomes 99.99998976%.

  In this way, the possibility of omission of detection of personal information depends on the data volume, but can be said to be almost zero. In the data conversion apparatus 14 according to the embodiment, by excluding personal information designation items designated by the user from the personal information detection target, the number of scan target items is reduced in advance, and there is a possibility that omission of detection of personal information may occur. make low. Further, items that are difficult for humans to determine whether or not personal information is recorded, such as the remarks column in FIG. 10, are complemented by automatically detecting the presence of personal information by the determination unit 54. In other words, the data conversion device 14 can make the occurrence of omission of detection of personal information as close as possible to 0 as a synergistic effect combining the user's judgment based on the design and the machine detection based on the dictionary.

  The present invention has been described based on the second embodiment. This embodiment is an exemplification, and it will be understood by those skilled in the art that various modifications can be made to combinations of the respective constituent elements and processing processes, and such modifications are also within the scope of the present invention. is there. A modification is shown below.

A first modification of the second embodiment will be described.
Although not mentioned in the above embodiment, the personal information detection process of the determination unit 54 may be multiprocessed, and a plurality of personal information detection processes may be simultaneously executed by a plurality of processes. Specifically, when the personal information detection process of the determination unit 54 is to be executed, the operating system (OS) of the data conversion apparatus 14 may generate a plurality of processes that execute the personal information detection process. And each of several CPU with which the data converter 14 is provided may perform each process in parallel. It is desirable that the number of processes can be arbitrarily set by the user and can be changed.

  In addition to or instead of multiprocessing, the personal information detection process of the determination unit 54 may be multithreaded, and a plurality of personal information detection processes may be simultaneously executed by a plurality of threads. Specifically, when the personal information detection process of the determination unit 54 is to be executed, the OS of the data conversion apparatus 14 may generate a plurality of threads that execute the personal information detection process in parallel. The CPU included in the data conversion device 14 may execute each thread in parallel. It is desirable that the number of threads can be arbitrarily set by the user and can be changed.

  For example, each of a plurality of processes may execute the personal information detection process for a plurality of different original document data in parallel. Of course, the execution subject may be a plurality of threads. In addition, each of the plurality of threads may be a part of a record of the same original document data (document file), and the personal information detection process for different record groups may be executed in parallel. Of course, the execution subject may be a plurality of processes. According to this modification, personal information detection processing for multiple files is executed in parallel by multi-process and multi-threading, and personal information detection processing for multiple records in the same file is executed in parallel to increase the speed. Can do.

  When multiple threads execute personal information detection processing for different record groups in the same original document data, if a thread detects that personal information is included in a specific item, the detected personal information is detected. Notify other thread of item information. Notification may be performed using a known inter-thread communication mechanism (for example, setting a flag in a memory area that can be referred to by a plurality of threads). Thereafter, the other threads exclude the notified personal information detection items from the scan target items. The same applies to the case where the execution subject is a plurality of processes, and the personal information detection items may be mutually notified using a known inter-process communication mechanism.

A second modification of the second embodiment will be described.
The determination criterion holding unit 50 may hold a keyword indicating personal information in association with a category indicating the type (for example, the detection type in FIG. 3). For example, the keywords “Yamada”, “Taro”, etc. may be held in association with the category “person name”. Further, the keywords “Tokyo”, “Shibuya-ku”, etc. may be held in association with the category “address”.

  The determination unit 54 may execute the personal information recording position specifying process for the personal information designation item and the personal information detection item for each record after specifying the personal information detection item in the original document data. Specifically, referring to the determination criterion holding unit 50, the position in the document (in the record) in which the keyword of the personal information is described and the category of the keyword may be specified. Then, the personal information extraction result shown in FIG. 3, that is, the combination of record number, item name, character string, position, and detection type may be output to the replacement rule determination unit 36. When only one category of information is stored in one item and no position specification is required (typically when all of the character string is replaced), the original document is used in the determination process described in the embodiment. When the personal information detection item in the data is specified, the category may be specified together.

A third modification of the second embodiment will be described.
In the above embodiment, both the personal information detection function and the masking function are provided in one apparatus, but these functions may be shared and executed by a plurality of physically different apparatuses. That is, the first apparatus that executes the personal information detection process described in the second embodiment and the second apparatus that executes the personal information mask process described in the first embodiment configure the communication network. The data conversion apparatus 14 described in the second embodiment may be realized by cooperating with each other.

  Any combination of the first embodiment, the second embodiment, and the modifications described above is also useful as an embodiment of the present invention. The new embodiment generated by the combination has the effects of the combined embodiment and the modified examples. Further, the functions to be fulfilled by the constituent elements described in the claims are as follows. The constituent elements shown in the first embodiment, the second embodiment, and the modified examples of the embodiments are united or their cooperation. Those skilled in the art will also understand that

  10 production machine, 14 data conversion device, 34 personal information detection unit, 50 judgment reference holding unit, 52 user designation acquisition unit, 54 judgment unit, 56 detection result output unit, 100 information system.

Claims (2)

A document acquisition unit for acquiring a document in which character strings related to a plurality of items are recorded;
A designation acquisition unit for acquiring information indicating personal information items including personal information among the items specified by the user;
A determination unit for determining whether or not personal information is included in a character string of another item excluding the personal information item specified by the user, and detecting an item including the personal information as a personal information item;
An output unit that outputs the personal information item specified by the user and the personal information item detected by the determination unit as an item to be masked with personal information;
With
The document acquired by the document acquisition unit includes a plurality of records, and each record includes a character string related to the plurality of items.
The determination unit sequentially determines whether or not personal information is included in a character string of each item of the plurality of records, and when determining that a specific item of a record includes personal information, A personal information detection apparatus, characterized in that it is detected as a personal information item, and the character string of the item in the remaining records is excluded from a target for determining whether or not personal information is included. The ability to retrieve documents that contain text strings for multiple items;
A function for obtaining information indicating a personal information item that is designated by a user and includes personal information among the plurality of items;
A determination function for determining whether or not personal information is included in a character string of another item excluding the personal information item specified by the user, and detecting an item including the personal information as a personal information item;
An output unit for outputting the personal information item specified by the user and the personal information item detected by the determination function as an item to be subjected to the masking process of the personal information;
Is realized on a computer,
The document acquired by the acquiring function includes a plurality of records, and each record includes a character string related to the plurality of items.
The determination function sequentially determines whether or not personal information is included in a character string of each item of the plurality of records, and when it is determined that a specific item of a record includes personal information, the item is A computer program that is detected as a personal information item and that is excluded from a target for determining whether or not personal information is included in the character string of the item in the remaining records.

Images