JP7053987B2 - Data processing equipment, data processing methods and data processing programs - Google Patents

Description

The present invention relates to a data processing apparatus, a data processing method and a data processing program.

One organization may collect and hold data that is of high value to another. In that case, one organization may provide data to a specific other organization on the condition that it is confidentially controlled so as not to be leaked to the outside. For example, a business operator that collects DNA data showing a base sequence of DNA (Deoxyribonucleic Acid) for various organisms may provide DNA data to a research institute that analyzes DNA.

An organization may provide similar data to multiple other organizations. On the other hand, as a result of improper data confidentiality management in any of the organizations to which the data is provided, there is a possibility that the data will be leaked to the outside in violation of the contract. The provider's organization may become aware of the fact of the leak by obtaining data from sources other than the provider's organization, such as the Internet. In that case, it is preferable that the providing organization can identify from which of the plurality of providing organizations the data was leaked. Identifying the source of the leak is useful for preventing recurrence, as the organization that leaked the data may not be aware of the fact of the leak.

However, if the same data is provided to a plurality of other organizations as it is, it becomes difficult to identify the leak source from the leaked data. Therefore, there has been proposed a method of processing data in a different manner depending on the organization to which the data is provided when the data is provided.

For example, a transmission device has been proposed that enables the leakage source of multicast-distributed data to be identified by using a digital watermark. The proposed transmission device copies the original data to generate two types of copy data with different digital watermarks, and divides each copy data into a plurality of data sections in chronological order. The transmitting device selects one of the two types of copy data for each of the plurality of data sections, thereby assigning different digital watermarking time-series patterns to different receiving devices.

In addition, an information processing device has been proposed that makes it possible to identify the source of leakage of tabular data including numerical columns such as height and weight. The proposed information processing apparatus adds noise such that the sum becomes zero for a plurality of numerical values described in a numerical value column when tabular data is provided. The information processing apparatus associates a plurality of different providers with a plurality of different noise patterns by generating noise from the identifier of the provider.

Further, a character information editing device for embedding information for detecting unauthorized use in character information has been proposed. The proposed character information editing device holds a character dictionary showing isomorphic character pairs having similar shapes and different character codes. The character information editing device searches for one character of the isomorphic character pair from the character information, and replaces the searched character with the other character of the isomorphic character pair.

Further, a document processing device for embedding invisible control information in a document image has been proposed. The proposed document processing apparatus expresses control information as a binary bit string and allocates one bit to a space between two adjacent characters. The document processing device adjusts the character spacing so that the length of the blank changes depending on whether the bit value is "0" or "1".

Japanese Unexamined Patent Publication No. 2009-21279 Japanese Unexamined Patent Publication No. 2013-191121 Japanese Unexamined Patent Publication No. 2000-352928 Japanese Unexamined Patent Publication No. 2010-124451

However, if noise is added to the provided data, the authenticity of the values contained in the data will not be guaranteed, and the value of the data to the provider will be greatly reduced depending on the characteristics and usage of the data. Sometimes. For example, in DNA data, if a part of the base sequence is modified, the value of the entire DNA data may be significantly reduced.

In one aspect, it is an object of the present invention to provide a data processing apparatus, a data processing method, and a data processing program that reduce the influence on the usefulness of data in measures against leakage.

In one aspect, a data processing apparatus having a storage unit and a processing unit is provided. The processing unit generates a plurality of deletion patterns indicating records to be deleted from the data containing a plurality of records, and deletes the records contained in the data based on each of the multiple deletion patterns, thereby causing a plurality of data. Generate a subset. The storage unit stores correspondence information in which each of the plurality of subsets is associated with the destination identifier indicating the destination of the subset.

Also, in one aspect, a computer-executed data processing method is provided. Further, in one embodiment, a data processing program to be executed by a computer is provided.

On one side, the impact on the usefulness of the data in anti-leakage measures is reduced.

It is a figure explaining the data processing apparatus of 1st Embodiment. It is a figure which shows the hardware example of the information processing apparatus of 2nd Embodiment. It is a block diagram which shows the functional example of an information processing apparatus. It is a figure which shows the example of the original data table. It is a figure which shows the example of a parameter table. It is a figure which shows the example of the frequency distribution table. It is a figure which shows the example of a group table. It is a figure which shows the example of the identifiable data table. It is a figure which shows the example of the correspondence table. It is a figure which shows the example of the correction frequency distribution table. It is a figure which shows the example of the provided data table. It is a figure which shows the example of the leakage data table. It is a flowchart which shows the procedure example of the original data analysis. It is a flowchart which shows the procedure example of the provided data generation. It is a flowchart which shows the procedure example of the leakage source estimation.

Hereinafter, the present embodiment will be described with reference to the drawings.
[First Embodiment]
The first embodiment will be described.

FIG. 1 is a diagram illustrating a data processing apparatus according to the first embodiment.
The data processing apparatus 10 of the first embodiment generates a plurality of subsets to be provided to a plurality of destinations from a certain data. Further, when the data leakage is found after the processed data is provided, the data processing apparatus 10 estimates the leakage source from the plurality of provision destinations. However, data processing and leakage source estimation can be performed by different devices. The data processing device 10 may be a client computer or a server computer.

The data processing device 10 has a storage unit 11 and a processing unit 12. The storage unit 11 may be a volatile semiconductor memory such as a RAM (Random Access Memory) or a non-volatile storage such as an HDD (Hard Disk Drive). The processing unit 12 is, for example, a processor such as a CPU (Central Processing Unit) or a DSP (Digital Signal Processor). However, the processing unit 12 may include an electronic circuit for a specific purpose such as an ASIC (Application Specific Integrated Circuit) or an FPGA (Field Programmable Gate Array). The processor executes a program stored in a memory such as RAM. A collection of multiple processors may be referred to as a "multiprocessor" or simply a "processor."

The storage unit 11 stores the correspondence information 13. The correspondence information 13 associates a plurality of subsets of data with a plurality of destination identifiers such as the destination identifiers 14a, 14b, and 14c. The plurality of subsets of data is specified by the plurality of deletion patterns, for example, the deletion patterns 15a, 15b, 15c. Each deletion pattern indicates a record to be deleted from data containing a plurality of records. The record to be deleted is a part of the record in the data, and is specified by using, for example, a record identifier for identifying each record in the data. Each destination identifier indicates an individual or organization to which the data is provided.

The processing unit 12 determines a record to be deleted from the data and generates a plurality of different deletion patterns. The processing unit 12 generates a plurality of subsets of the data by deleting some records included in the data based on each of the plurality of deletion patterns.

The destination indicated by a destination identifier is provided with a subset generated from the data based on the deletion pattern associated with the destination identifier. The subset provided does not contain the records that the delete pattern specifies for deletion. It is preferable that different destination identifiers are associated with different deletion patterns. However, different deletion patterns may be such that the set of records to be deleted does not completely match, and some records to be deleted may be duplicated.

For example, the deletion pattern 15a is associated with the provision destination identifier 14a. The deletion pattern 15a specifies records # 4, # 5, and # 12 in the data as deletion targets. Further, the deletion pattern 15b is associated with the provision destination identifier 14b. The deletion pattern 15b specifies records # 3, # 7, and # 10 in the data as deletion targets. Further, the deletion pattern 15c is associated with the provision destination identifier 14c. The deletion pattern 15c specifies records # 2, # 8, and # 9 in the data as deletion targets.

The processing unit 12 acquires the leaked data 16. The leaked data 16 is, for example, input from the outside of the data processing apparatus 10 and stored in the storage unit 11. The leaked data 16 includes a part of the plurality of records included in the original data. The leaked data 16 is a set of records suspected of being leaked from any one of a plurality of destinations, and is a whole or a part of a subset provided to the one destination. The leaked data 16 is obtained from a source different from the information source such as the Internet or a list shop.

When the leaked data 16 is acquired, the processing unit 12 searches for a deletion pattern in which none of the records included in the leaked data 16 is designated as a deletion target from the plurality of deletion patterns indicated by the corresponding information 13. This is because the record specified as the deletion target is not provided and cannot be leaked. The processing unit 12 extracts the provider identifier associated with the searched deletion pattern from the corresponding information 13, and estimates that the provider indicated by the extracted provider identifier is the leak source of the leaked data 16. The processing unit 12 generates and outputs leak source information 17 indicating an estimated leak source. The leak source information 17 includes, for example, an extracted provider identifier.

For example, the leaked data 16 contains records # 2, # 4, # 5, # 8. The deletion pattern 15a registered in the correspondence information 13 specifies records # 4 and # 5 as deletion targets. Therefore, the provider of the provider identifier 14a associated with the deletion pattern 15a is excluded from the candidates for the leak source because the leaked data 16 cannot be leaked. Further, the deletion pattern 15c specifies records # 2 and # 8 as deletion targets. Therefore, the provider of the provider identifier 14c associated with the deletion pattern 15c is excluded from the candidates for the leak source because the leaked data 16 cannot be leaked. On the other hand, the deletion pattern 15b does not specify any of the records # 2, # 4, # 5, and # 8 as the deletion target. Therefore, the provider of the provider identifier 14b associated with the deletion pattern 15b is presumed to be the leak source because the leaked data 16 can be leaked.

However, the estimation of the leakage source may be performed by a unit other than the processing unit 12 of the data processing device 10, or may be performed by an information processing device other than the data processing device 10.
According to the data processing apparatus 10 of the first embodiment, when data is provided to a plurality of destinations, correspondence information 13 indicating a correspondence relationship between a plurality of subsets and a plurality of destination identifiers is generated. .. When the leaked data 16 is acquired, a provider identifier that is not designated as a deletion target for any record included in the leaked data 16 is searched based on the correspondence information 13, and the searched provider identifier is the leaked data 16. It is output as information indicating the source of the leak.

This makes it possible to estimate the leakage source from the leakage data 16 even when there are a plurality of data provision destinations. Therefore, it is possible to strengthen the protection of data by requesting the leakage source to prevent recurrence. Further, as compared with the method of adding noise to the value of the record, the authenticity of the record can be ensured and the influence on the usefulness of the data can be reduced.

[Second Embodiment]
Next, a second embodiment will be described.
The information processing apparatus 100 of the second embodiment processes the original data held by the organization to generate the provided data to be provided to each of the plurality of other organizations. Further, when the leaked data is acquired, the information processing apparatus 100 analyzes the leaked data and estimates the organization of the leak source. In the description of the second embodiment, DNA data including a plurality of base sequence records is assumed as the original data. The provider organization is assumed to be a business operator that collects a large amount of DNA data, and the provider organization is assumed to be a research institution that conducts research and development using DNA data. DNA data is provided for a fee or free of charge, provided that it is not leaked to the outside.

FIG. 2 is a diagram showing a hardware example of the information processing apparatus according to the second embodiment.
The information processing apparatus 100 includes a CPU 101, a RAM 102, an HDD 103, an image signal processing unit 104, an input signal processing unit 105, a medium reader 106, and a communication interface 107. The information processing device 100 corresponds to the data processing device 10 of the first embodiment. The CPU 101 corresponds to the processing unit 12 of the first embodiment. The RAM 102 or the HDD 103 corresponds to the storage unit 11 of the first embodiment. The processing unit 12 may be realized by the CPUs of a plurality of information processing devices connected to each other via a network or the like, depending on the processing.

The CPU 101 is a processor that executes a program instruction. The CPU 101 loads at least a part of the programs and data stored in the HDD 103 into the RAM 102 and executes the program. The CPU 101 may include a plurality of processor cores, the information processing unit 100 may have a plurality of processors, and the processes described below may be executed in parallel using the plurality of processors or processor cores. .. Also, a set of multiple processors may be referred to as a "multiprocessor" or simply a "processor".

The RAM 102 is a volatile semiconductor memory that temporarily stores a program executed by the CPU 101 and data used by the CPU 101 for calculation. The information processing apparatus 100 may include a type of memory other than the RAM, or may include a plurality of memories.

The HDD 103 is a non-volatile storage device that stores software programs such as an OS (Operating System) and middleware, and data. The information processing device 100 may be provided with other types of storage devices such as a flash memory and an SSD (Solid State Drive), or may be provided with a plurality of non-volatile storage devices.

The image signal processing unit 104 outputs an image to the display 111 connected to the information processing apparatus 100 in accordance with a command from the CPU 101. As the display 111, any kind of display such as a CRT (Cathode Ray Tube) display, a liquid crystal display (LCD: Liquid Crystal Display), a plasma display, and an organic EL (OEL: Organic Electro-Luminescence) display can be used.

The input signal processing unit 105 acquires an input signal from the input device 112 connected to the information processing device 100 and outputs the input signal to the CPU 101. As the input device 112, any kind of input device such as a mouse, a touch panel, a touch pad, a trackball, a keyboard, a remote controller, and a button switch can be used. Further, a plurality of types of input devices may be connected to the information processing apparatus 100.

The medium reader 106 is a reading device that reads programs and data recorded on the recording medium 113. As the recording medium 113, for example, a magnetic disk, an optical disk, a magneto-optical disk (MO: Magneto-Optical disk), a semiconductor memory, or the like can be used. The magnetic disk includes a flexible disk (FD) and an HDD. Optical discs include CDs (Compact Discs) and DVDs (Digital Versatile Discs).

The medium reader 106, for example, copies a program or data read from the recording medium 113 to another recording medium such as the RAM 102 or the HDD 103. The read program is executed by, for example, the CPU 101. The recording medium 113 may be a portable recording medium and may be used for distribution of programs and data. Further, the recording medium 113 and the HDD 103 may be referred to as a computer-readable recording medium.

The communication interface 107 is an interface that communicates with another information processing device via the network 114. The communication interface 107 may be a wired communication interface connected to a wired communication device such as a switch or a router, or may be a wireless communication interface connected to a wireless communication device such as a base station or an access point.

FIG. 3 is a block diagram showing a functional example of the information processing apparatus.
The information processing apparatus 100 includes an original data storage unit 121, a parameter storage unit 122, a provision data storage unit 123, a correspondence table storage unit 124, and a leaked data storage unit 125. These storage units are implemented using, for example, the storage area of the RAM 102 or the HDD 103. Further, the information processing apparatus 100 has a deleted record number calculation unit 131, an identifiable record extraction unit 132, a grouping unit 133, a deletion pattern generation unit 134, a record deletion unit 135, a record frequency reduction unit 136, and a leakage source estimation unit 137. .. These processing units are implemented using, for example, a program executed by the CPU 101.

The original data storage unit 121 stores original data including a plurality of records showing the base sequence of DNA. The base sequence of DNA is a sequence containing four types of bases, A (adenine), G (guanine), C (cytosine), and T (thymine). The lengths of the plurality of base sequences in the original data do not have to be the same. In the original data, each base sequence may be represented using the above four types of characters, or may be represented using the corresponding numerical values or other symbols.

The parameter storage unit 122 stores the parameter value used when generating the provided data from the original data. The parameter value is input to the information processing apparatus 100 by the user, for example. The information processing apparatus 100 generates a subset of the original data as the provided data by deleting some records from the original data. The parameter value is used to determine the number of records to delete. The parameter value includes the maximum number of destinations, which is the maximum number of possible destinations, and the estimation ability that indicates the probability that the leak source can be uniquely identified from the leaked data. The larger the maximum number of destinations, the larger the number of deleted records. In addition, the higher the estimation ability desired by the user, the larger the number of deleted records. However, it is also possible for the user to directly input the number of deleted records as a parameter value.

The provision data storage unit 123 stores the provision data generated by deleting some records from the original data. The data provided differs depending on the destination. The provision data storage unit 123 can store a plurality of sets of provision data corresponding to a plurality of provision destinations. The provided data stored in the provided data storage unit 123 may be transmitted to the information processing device of the providing destination via the network 114. Further, the provided data stored in the provided data storage unit 123 may be written in the portable recording medium and sent to the providing destination.

The correspondence table storage unit 124 stores a correspondence table in which the provision destination identifier that identifies the provision destination and the deletion pattern indicating the record deleted from the original data are associated with each other. The correspondence table is created and retained when the provided data is generated, and is referred to when the source of the leak is estimated.

The leaked data storage unit 125 stores leaked data obtained from an information source other than the organization to which the data is provided. The leaked data is, for example, obtained by the user and input to the information processing apparatus 100. The leaked data may be obtained from a server device published on a network such as the Internet. In addition, the leaked data may be obtained from an information provider such as a so-called list shop. The leaked data is leaked from any one of a plurality of provision destinations, and corresponds to a part or all of the provision data corresponding to the provision destination. The provided data may be leaked due to fraudulent activity by employees inside the provider. In addition, the provided data may be leaked by a security attack from the outside of the provider. The more records included in the leaked data, the easier it is to estimate the source of the leak.

The deleted record number calculation unit 131 calculates the number of deleted records using the parameter value stored in the parameter storage unit 122. For example, assume that any number of records are randomly leaked from the provided data. That is, it is assumed that all patterns of the subset that can be extracted from the provided data are subject to leakage with equal probability. Then, it can be considered that the following inequality is approximately established between the number of deleted records r, the maximum number of destinations p, and the estimation capacity c. r ≧ log ₂ ((p-1) / (1-c)). The deleted record number calculation unit 131 may adopt the smallest integer satisfying this inequality as the deleted record number r. The deleted record number calculation unit 131 notifies the grouping unit 133 and the deletion pattern generation unit 134 of the calculated number of deleted records.

The identifiable record extraction unit 132 compares a plurality of records included in the original data stored in the original data storage unit 121, counts the number of appearances (frequency) of the same base sequence, and generates a frequency distribution. The identifiable record extraction unit 132 handles a record having a base sequence having a frequency of 1, that is, a record having a base sequence that appears only once in the original data as an identifiable record. The identifiable record can be uniquely identified by the base sequence itself. On the other hand, the identifiable record extraction unit 132 treats a record other than the identifiable record, that is, a record having a base sequence having a frequency greater than 1, as an unidentifiable record.

The identifiable record extraction unit 132 notifies the grouping unit 133 of the identifiable records. Further, the identifiable record extraction unit 132 notifies the record deletion unit 135 of the generated frequency distribution. It should be noted that each base sequence contained in the DNA data is often long, and the probability that a record having the same base sequence will appear is small. Therefore, it is expected that most of the records included in the original data are identifiable records and the number of unidentifiable records is small.

The grouping unit 133 classifies the identifiable records extracted from the original data by the identifiable record extraction unit 132 into a plurality of groups. The grouping reference may be set in advance in the grouping unit 133, or may be appropriately input by the user to the information processing apparatus 100.

In grouping, it is preferable to group a set of identifiable records that may be of interest to the organization to which they are provided. That is, the organization to which the information is provided may not be interested in all the records contained in the data provided, and may extract some records from the data provided and use them for research and development. Records that have not been extracted may be discarded by the organization to which they are provided, and are less likely to be leaked than the extracted records. Therefore, the grouping unit 133 groups the identifiable records based on the interest of the provider so that the leakage source can be estimated even if the records are leaked from only a specific group.

For example, suppose that in some research and development, the base at the beginning of the base sequence is important, so only records with a specific base at the beginning of the base sequence may be used. In this case, the grouping unit 133 may classify the identifiable records into a maximum of four groups based on the first base of the base sequence. That is, a group whose base sequence starts with A, a group G, a group C, and a group T can be generated. Knowledge of how to use such records is entered, for example, by the user.

The grouping unit 133 sorts a plurality of identifiable records based on the grouping result and the number of deleted records notified from the deleted record number calculation unit 131. The sort of identifiable records sorts the identifiable records that belong to the same group so that they do not appear consecutively as much as possible. As a result, when generating the provision data corresponding to a certain provision destination, the identifiable records belonging to different groups as much as possible are selected as the records to be deleted. The grouping unit 133 notifies the deletion pattern generation unit 134 of the sorted list of identifiable records.

The deletion pattern generation unit 134 acquires a provider identifier that identifies the provider organization. The provider identifier is, for example, entered by the user into the information processing apparatus 100. The deletion pattern generation unit 134 selects as many identifiable records as the number of deleted records notified by the deletion record number calculation unit 131 from the list of identifiable records notified by the grouping unit 133. For example, the deletion pattern generation unit 134 selects identifiable records by the number of deleted records in order from the beginning of the list of identifiable records. The deletion pattern generation unit 134 determines the selected identifiable record as the deletion target record, and generates a deletion pattern indicating a set of deletion target records. The deletion pattern generation unit 134 associates the provision destination identifier with the deletion pattern and registers it in the correspondence table stored in the correspondence table storage unit 124. Further, the deletion pattern generation unit 134 notifies the record deletion unit 135 of the generated deletion pattern.

The record deletion unit 135 acquires the frequency distribution in which the base sequence and the frequency are associated with each other from the identifiable record extraction unit 132, and deletes the base sequence indicated by the deletion pattern notified from the deletion pattern generation unit 134 from the acquired frequency distribution. To generate a modified frequency distribution. The record deletion unit 135 notifies the record frequency reduction unit 136 of the correction frequency distribution.

The record frequency reduction unit 136 acquires a correction frequency distribution from the record deletion unit 135, and detects a base sequence having a frequency greater than 1 from the correction frequency distribution, that is, a base sequence of an unidentifiable record. The record frequency reduction unit 136 reduces the frequency of the detected base sequence so as to match the deletion rate of the identifiable record. This is because if only the identifiable records are to be deleted and the unidentifiable records are not to be deleted, the ratio of the unidentifiable records included in the provided data will be larger than that of the original data. By reducing the frequency of the base sequence corresponding to the unidentifiable record, the influence of the deletion of the identifiable record can be reduced, and the characteristics of the provided data can be brought closer to the characteristics of the original data.

For example, assuming that the number of identifiable records contained in the original data is n, the number of deleted records is r, and the frequency of a certain base sequence is _nt ( _nt is an integer larger than 1), the reduction frequency _rt is as follows. It is calculated. _{rt = r / n × n t .} However, if the right-hand side of this equation is not divisible, the reduction frequency _rt is made an integer by rounding, rounding down, or rounding up. The record frequency reduction unit 136 changes the frequency _nt in the modified frequency distribution to _{nt −rt} . The record frequency reduction unit 136 stores the modified frequency distribution after frequency adjustment in the provided data storage unit 123 as provided data. At this time, if the record ID is included in the correction frequency distribution, the record ID is deleted.

The processes of the deleted record number calculation unit 131, the identifiable record extraction unit 132, and the grouping unit 133 may be executed once for one original data. On the other hand, the processing of the deletion pattern generation unit 134, the record deletion unit 135, and the record frequency reduction unit 136 is executed once for one provider identifier. When three provision destinations exist, the processing of the deletion pattern generation unit 134, the record deletion unit 135, and the record frequency reduction unit 136 is executed three times, and three sets of provision data corresponding to the three provision destinations are generated.

The leak source estimation unit 137 analyzes the leak data stored in the leak data storage unit 125 and estimates the leak source. The leakage source estimation unit 137 identifies the identifiable record included in the leakage data by referring to the original data stored in the original data storage unit 121 or the frequency distribution generated by the identifiable record extraction unit 132. The leak source estimation unit 137 collates the deletion pattern included in the correspondence table stored in the correspondence table storage unit 124 with the identifiable record appearing in the leak data, and narrows down the leak source candidates. Identifiable records that were not included in the provided data cannot be leaked from the organization that received the provided data. Therefore, the provider whose deletion pattern includes the identifiable record appearing in the leaked data can be excluded from the leak source.

The leak source estimation unit 137 outputs information indicating the estimated leak source. For example, the leak source estimation unit 137 outputs a provider identifier indicating a provider estimated to be a leak source. The leak source estimation unit 137 may display information indicating the leak source on the display 111, or may transmit information indicating the leak source to another information processing device via the network 114. If the leak source cannot be narrowed down to one, the leak source estimation unit 137 may output a message indicating an estimation failure or may output information indicating a leak source candidate.

In FIG. 3, the leak source estimation unit 137 is incorporated in the information processing device 100, but is separated from the information processing device 100 and incorporated into another information processing device as a device for estimating the leak source, for example, via the network. May be connected to the information processing apparatus 100.

FIG. 4 is a diagram showing an example of the original data table.
The original data table 141 is stored in the original data storage unit 121. The original data table 141 includes items of record ID and base sequence. An identification number for identifying a record is registered in the record ID item. In the base sequence item, a character string representing each base of the base sequence using A, G, C, T or a corresponding number or symbol is registered.

Here, for the sake of simplicity, all the base sequences are sequences of length 3. However, the base sequence may have a larger length and may not have a uniform length. Further, here, the identification number of the serial number is used as the record ID. However, other numerical values or symbols such as the hash value of the base sequence may be used as the record ID. Further, here, each record in the original data table 141 includes a record ID. However, as long as the record ID and the base sequence are associated with each other, each record does not have to explicitly include the record ID. For example, the record ID of the record may be specified from the position of each record, such as the offset from the beginning of the original data table 141.

Further, although the DNA data is used as the original data in the second embodiment, the information processing apparatus 100 can handle other types of original data. Confidential information that is not desirable to be disclosed may be used as the original data. For example, a personal information set containing a plurality of personal information items such as age, gender, and address, in which one record represents personal information for one person. It also includes a set of position information that includes a plurality of measurement information items such as longitude, latitude, and time, and one record represents the position of a person or object at one point in time.

The original data is preferably sample data that includes information about some people or things as a sample, rather than 100% data that comprehensively contains information about all people or things that make up the target domain. If the original data is sample data, it is expected that the value of the provided data will not decrease even if a small number of records are deleted from the original data.

FIG. 5 is a diagram showing an example of a parameter table.
The parameter table 142 is stored in the parameter storage unit 122. The parameter table 142 stores the parameter name and the parameter value in association with each other. The parameters include the maximum number of destinations p, the estimation capacity c, and the number of deleted records r.

The maximum number of destinations p and the estimation capacity c are input by the user. The number of deleted records r is calculated from the maximum number of destinations p and the estimation capacity c, and is the smallest integer satisfying r ≧ log ₂ ((p-1) / (1-c)). However, the user may directly input the number of deleted records r. As an example, the maximum number of destinations p = 3 and the estimation capacity c = 0.75 are input by the user. This indicates that there are up to three organizations to which the data is provided, and that there is a 75% probability that the leak source can be identified from the leaked data. In this case, the number of deleted records r = 3 is determined in order to achieve the estimation capacity c = 0.75 under the maximum number of providers p = 3.

FIG. 6 is a diagram showing an example of a frequency distribution table.
The frequency distribution table 143 is generated from the original data table 141 by the identifiable record extraction unit 132. The frequency distribution table 143 includes items of base sequence, frequency and record ID. A character string indicating the base sequence is registered in the base sequence item. In the frequency item, the number of times the base sequence appears in the original data table 141 is registered. In the record ID item, the record ID of the record including the base sequence is registered. However, the record ID of the identifiable record including the base sequence having a frequency of 1 may be registered, and the record ID of the unidentifiable record containing the base sequence having a frequency greater than 1 may be omitted.

For example, among the records # 1 to # 12 included in the original data table 141, the records # 1, # 6, and # 11 are unidentifiable records having the same value other than the record ID. Since the records # 1, # 6 and # 11 include the base sequence "AAA", the frequency of the base sequence "AAA" is 3. On the other hand, among the records # 1 to # 12, records # 2 to # 5, # 7 to # 10, and # 12 are identifiable records in which the same value other than the record ID does not exist. The frequency of the base sequence contained in each of these nine records is 1. Therefore, one base sequence of frequency 3 and nine base sequences of frequency 1 are registered in the frequency distribution table 143.

FIG. 7 is a diagram showing an example of a group table.
The group table 144 is generated from the frequency distribution table 143 by the grouping unit 133. The group table 144 includes items of group ID, record ID and base sequence. An identifier that identifies the group is registered in the item of the group ID. The record ID of the identifiable record is registered in the record ID item. In the base sequence item, a character string indicating the base sequence included in the identifiable record is registered.

In the grouping, only the identifiable records having a frequency of 1 are extracted from the frequency distribution table 143, and the identifiable records are classified into a plurality of groups. One group contains one or more identifiable records. Therefore, in the group table 144, one or more record IDs are associated with one group ID.

Here, it is assumed that the grouping unit 133 is set so as to classify the identifiable records based on the base at the beginning of the base sequence. Group GC is a group showing a base sequence in which the first base is " _C ", and includes four identifiable records of records # 2, # 4, # 8, and # 10. The group GG is a group showing a base sequence in which the first base is " _G ", and includes three identifiable records of records # 3, # 9, and # 12. The group GT is a group showing a base sequence in which the first base is " _T ", and includes two identifiable records of records # 5 and # 7. Here, since there is no identifiable record containing the base sequence in which the first base is "A", the group corresponding to "A" is not generated.

The grouping unit 133 can adopt various grouping criteria according to the type of the original data. Appropriate grouping criteria are selected based on the user's knowledge. For example, the identifiable records of the personal information set can be classified by the age group such as teens, 20s, and 30s, or can be classified by the administrative division name included in the address. Further, for example, the identifiable records of the location information set can be classified by a time zone such as 10 o'clock, 11 o'clock, 12 o'clock, or can be classified by a range of longitude and latitude.

FIG. 8 is a diagram showing an example of an identifiable data table.
The identifiable data table 145 is generated from the group table 144 by the grouping unit 133. The identifiable data table 145 contains a record ID and a base sequence for identifiable records. In the identifiable data table 145, a plurality of identifiable records are sorted based on the group table 144. As much as possible, the identifiable records belonging to the same group are arranged so as not to be consecutive.

For example, from the beginning of the identifiable data table 145, the record # 4 of the group _GC , the record # 12 of the group _GG , the record # 5 of the group _GT , the record # 10 of the group _GC , and the record # of the group _GG . 3. It is lined up with record # 7 of group _GT . This is a selection of identifiable records (circularly) one by one from the groups _GC , _GG , and _GT . This is followed by record # 8 in group _GC , record # 9 in group _GG , and record # 2 in group _GC .

FIG. 9 is a diagram showing an example of a correspondence table.
The correspondence table 146 is stored in the correspondence table storage unit 124. Correspondence table 146 includes items of destination identifier and deletion pattern. The identifier input by the user is registered in the item of the provider identifier. The name of the organization of the provider may be used as the identifier of the provider. The record ID of the identifiable record to be deleted is listed in the deletion pattern item.

The deletion pattern can be generated by extracting identifiable records by the number of deleted records r in order from the top of the identifiable data table 145. For example, when the destination identifier X is first input, records # 4, # 12, and # 5 are selected in order from the top of the identifiable data table 145, and the deletion pattern “4,5,12” is generated. Next, when the destination identifier Y is input, records # 10, # 3, # 7 are selected in order from the top of the identifiable data table 145, and the deletion pattern “3, 7, 10” is generated. Next, when the provider identifier Z is input, records # 8, # 9, and # 2 are selected in order from the top of the identifiable data table 145, and the deletion pattern “2,8,9” is generated.

In the second embodiment, the deletion pattern is generated so that the deletion target records do not overlap between different providers. However, it is sufficient that the deletion patterns do not exactly match between different providers, and some records to be deleted may be duplicated. Further, when the number of identifiable records n is sufficiently large, that is, when the number of identifiable records n is sufficiently larger than the product of the number of deleted records r and the maximum number of destinations p, the record to be deleted is selected by a simple method. You can also do it. For example, even by a method of randomly selecting r records from n identifiable records, there is a high possibility that the records to be deleted do not overlap between different providers.

FIG. 10 is a diagram showing an example of a correction frequency distribution table.
The modified frequency distribution table 147 is generated from the frequency distribution table 143 and the correspondence table 146 by the record deletion unit 135. The modified frequency distribution table 147 is a copy of the frequency distribution table 143 with some identifiable records deleted. The correction frequency distribution table 147 is generated for each provider identifier. When the provider identifier X is input by the user, the frequency distribution table 143 is copied, and the deletion pattern "4,5,12" corresponding to the provider identifier X is searched from the correspondence table 146. The modified frequency distribution table 147 is obtained by deleting records # 4, # 5, and # 12 from the copy of the frequency distribution table 143.

FIG. 11 is a diagram showing an example of the provided data table.
The provided data table 148 is generated from the modified frequency distribution table 147 by the record frequency reducing unit 136 and stored in the provided data storage unit 123. The provision data table 148 is generated for each provision destination identifier. In generating the provided data table 148 from the modified frequency distribution table 147, the record ID is deleted leaving the base sequence and the frequency. However, the record ID may be left in the provided data table 148.

Further, in generating the provided data table 148 from the modified frequency distribution table 147, the frequency of the base sequence corresponding to the unidentifiable record is modified. The reduction frequency _rt = _r / n × n _t is calculated from the frequency nt of a certain base sequence, the number of identifiable records n, and the number of deleted records r, and the frequency _nt is changed to _{nt −rt} . For example, the reduction frequency _rt = 1 is calculated from the frequency n _t = 3, the number of identifiable records n = 9, the number of deleted records r = 3, and the frequency nt = 3 of the base sequence “AAA”, and the frequency n _t = 3 is n _{t −rt} . = 3-1 = 2 is changed. The base sequence and frequency of the identifiable record are the same as those of the modified frequency distribution table 147.

FIG. 12 is a diagram showing an example of a leaked data table.
The leaked data table 149 is stored in the leaked data storage unit 125. The leaked data table 149 includes items of base sequence, frequency and record ID. However, the record ID is not included in the obtained leaked data itself, but is specified and added by the information processing apparatus 100 from the base sequence. The leak source estimation unit 137 searches the original data table 141 or the frequency distribution table 143 for the record ID corresponding to the base sequence, and adds the record ID to the base sequence of the leak data table 149. When a plurality of record IDs corresponding to the base sequence are searched, that is, when the base sequence is included in the indistinguishable record, the record ID can be omitted for the base sequence as in the frequency distribution table 143. ..

The source of the leak can be estimated from the identifiable records appearing in the leaked data table 149 and the correspondence table 146. For example, records # 2, # 4, # 5, # 8 appear as identifiable records in the leaked data table 149. The deletion pattern corresponding to the destination identifier X includes records # 4 and # 5. Therefore, it is unlikely that the destination indicated by the destination identifier X is the leak source. Similarly, the deletion pattern corresponding to the destination identifier Z includes records # 2 and # 8. Therefore, the destination indicated by the provider identifier Z is unlikely to be the leak source. On the other hand, the deletion pattern corresponding to the provider identifier Y does not include any of the records # 2, # 4, # 5, # 8. Therefore, the destination indicated by the destination identifier Y can leak these identifiable records, and may be the leak source.

In this way, the information processing apparatus 100 can estimate the leakage source from the leakage data itself. Further, the above-mentioned leaked data includes only a part of the records # 2, # 4, # 5, # 8, # 9, and # 12 received by the provider indicated by the provider identifier Y. However, the source of the leak can be estimated even if the leaked data is a subset of the provided data.

Next, the processing procedure of the information processing apparatus 100 will be described.
FIG. 13 is a flowchart showing an example of the procedure for analyzing the original data.
The source data analysis is performed once for a set of source data.

(S10) The deleted record number calculation unit 131 acquires the maximum number of provided destinations p and the estimation capacity c from the parameter table 142 stored in the parameter storage unit 122.
(S11) The deleted record number calculation unit 131 determines the deleted record number r from the maximum number of provided destinations p acquired in step S10 and the estimation capacity c. The number of deleted records r is, for example, the smallest integer satisfying r ≧ log ₂ ((p-1) / (1-c)).

(S12) The identifiable record extraction unit 132 generates the frequency distribution table 143 from the original data table 141 stored in the original data storage unit 121. At this time, the identifiable record extraction unit 132 compares the records included in the original data table 141 with each other and counts the frequency for each base sequence. Further, the identifiable record extraction unit 132 adds the record ID of the record including the base sequence to the base sequence having a frequency of 1.

(S13) The identifiable record extraction unit 132 extracts the base sequence having a frequency of 1 and the record ID from the frequency distribution table 143 as identifiable records.
(S14) The grouping unit 133 classifies the identifiable records extracted in step S13 into a plurality of groups according to a predetermined grouping standard, and generates a group table 144. For example, the grouping unit 133 collects identifiable records having the same first base in the base sequence to form a group.

(S15) The grouping unit 133 determines whether an identifiable record remains in the group table 144, that is, whether an identifiable record remains in at least one of the plurality of groups generated in step S14. If there is an identifiable record in the group table 144, the process proceeds to step S16, and if not, the original data analysis ends.

(S16) The grouping unit 133 sorts the plurality of groups generated in step S14 in descending order of the number of remaining identifiable records. The grouping unit ₁₃₃ changes the group names to G1, G2, ... _In descending order of the number of identifiable records.

(S17) The grouping unit 133 is initialized to the variable i = 1.
(S18) In the grouping unit 133, is the value of the variable i equal to or less than the number of deleted records r determined in step S11, and is there any identifiable record remaining in the group Gi (whether the group G _i is an empty set ₎ ? to decide. If this condition is satisfied, the process proceeds to step S19. If this condition is not satisfied, that is, if the value of the variable _i is larger than the number of deleted records r or the group Gi is an empty set, the process proceeds to step S15.

( _S19 ) The grouping unit 133 extracts one identifiable record from the group Gi and registers the extracted identifiable record at the end of the identifiable data table 145. The identifiable records to be _extracted from the group Gi are selected at random, for example.

(S20) The grouping unit 133 increases the value of the variable i by one. Then, the process proceeds to step S18.
For example, in the case of the group table 144 in FIG. 7, since | GC | = 4, | _{G G |} = 3, | G _T | = 2, G ₁ = GC, G ₂ = _{G G ,} G ₃ = Sorted with _GT . In the inner loop, record # 4 is selected from group _{G 1} = GC when i = 1, record # 12 is selected from group _{G 2} = GG when i = 2, and group when i = 3. Record # 5 is selected from G ₃ = _GT . When i = 4, since i> r, the inner loop is terminated.

Next, since | GC | = 3, | _{G G |} = 2, | _GT | = 1, it is sorted as _{G 1} = GC, G ₂ = G _G , G ₃ = _GT . Similar to the above, in the inner loop, record # 10 is selected from group _{G 1} = GC when i = 1, record # 3 is selected from group _{G 2} = GG when i = 2, and i =. When it is 3, record # 7 is selected from the group G ₃ = _GT . When i = 4, since i> r, the inner loop is terminated.

Next, since | GC | = 2, | _{G G |} = 1, | G _T | = 0, it is sorted as _{G 1} = GC, G ₂ = G _G , G ₃ = _GT . In the inner loop, record # 8 is selected from group _{G 1} = GC when i = 1, and record # 9 is selected from group G ₂ = G _G when i = 2. When i = 3, the inner loop is terminated because the group G ₃ = _GT = φ. Next, since | GC | = 1, | _{G G |} = | _GT | = 0, it is sorted as _{G 1} = GC, G ₂ = G _G , G ₃ = _GT . Record # 2 is selected from group _{G 1} = GC when i = 1 in the inner loop. When i = 2, the inner loop is terminated because the group G ₂ = G _G = φ. Then, since | GC | = | _GG | = | _GT | = ₀ , the original data analysis is completed.

FIG. 14 is a flowchart showing an example of the procedure for generating the provided data.
The provided data generation is executed once for one provided destination identifier.
(S30) The deletion pattern generation unit 134 acquires the provider identifier d. The destination identifier d is specified by the user, for example, when he / she wants to generate a set of provided data.

(S31) The deletion pattern generation unit 134 determines whether or not the provision destination identifier d is registered in the correspondence table 146 stored in the correspondence table storage unit 124. If the provider identifier d is registered, the process proceeds to step S32, and if it is not registered, the process proceeds to step S33.

(S32) The deletion pattern generation unit 134 searches the correspondence table 146 for the deletion pattern P corresponding to the provider identifier d. Then, the process proceeds to step S35.
(S33) The deletion pattern generation unit 134 extracts identifiable records by the number of deleted records r in order from the top of the identifiable data table 145.

(S34) The deletion pattern generation unit 134 generates a deletion pattern P that lists the record IDs of the r identifiable records extracted in step S33. The deletion pattern generation unit 134 registers the provision destination identifier d and the deletion pattern P in association with each other in the correspondence table 146.

(S35) The record deletion unit 135 copies the frequency distribution table 143. The record deletion unit 135 deletes some identifiable records from the copy of the frequency distribution table 143 according to the deletion pattern P in step S32 or step S34. That is, the identifiable record having the record ID listed in the deletion pattern P is deleted from the copy of the frequency distribution table 143. As a result, the modified frequency distribution table 147 is generated.

(S36) The record frequency reduction unit 136 selects one base sequence corresponding to the unidentifiable record, that is, a base sequence having a frequency greater than 1, from the modified frequency distribution table 147.
(S37) The record frequency reduction unit 136 determines the reduction frequency rt from the number of identifiable records n, the number of deleted records _r , and the frequency _nt of the selected base sequence. The reduction frequency rt is, for example, a non-negative integer calculated by _rt = _{r / n × n t .}

(S38) The record frequency reduction unit 136 changes the frequency of the selected base sequence in the modified frequency distribution table 147 from n _t to n _{t −rt} .
(S39) The record frequency reduction unit 136 determines whether or not all of the base sequences corresponding to the unidentifiable records have been selected from the modified frequency distribution table 147. If all the relevant base sequences are selected, the process proceeds to step S40, and if not, the process proceeds to step S36.

(S40) The record frequency reduction unit 136 generates the provided data table 148 by deleting the item of the record ID from the modified frequency distribution table 147. The record frequency reduction unit 136 stores the provided data table 148 in the provided data storage unit 123.

FIG. 15 is a flowchart showing an example of a procedure for estimating the leakage source.
The leak source estimation is performed once for one set of leaked data.
(S50) The leakage source estimation unit 137 acquires the leakage data table 149 stored in the leakage data storage unit 125. At this point, the leaked data does not include the record ID.

(S51) The leakage source estimation unit 137 refers to the correspondence table 146 stored in the correspondence table storage unit 124, and identifies a set of destinations that have provided DNA data so far. The set of providers specified here is a set of leak source candidates in the initial state.

(S52) The leak source estimation unit 137 selects one base sequence from the plurality of base sequences included in the leak data table 149.
(S53) The leakage source estimation unit 137 searches the original data table 141 or the frequency distribution table 143 for the record ID of the record including the selected base sequence.

(S54) The leakage source estimation unit 137 determines whether the record ID is specified as one in step S53, that is, whether the selected base sequence corresponds to the identifiable record. If one record ID is specified, the process proceeds to step S55, and if two or more record IDs are searched, the process proceeds to step S52.

(S55) The leakage source estimation unit 137 searches the correspondence table 146 for a deletion pattern including the specified record ID. The leak source estimation unit 137 excludes the provider corresponding to the searched deletion pattern from the set of leak source candidates identified in step S51.

(S56) The leakage source estimation unit 137 determines whether or not all the base sequences included in the leakage data table 149 have been selected. If all the base sequences are selected, the process proceeds to step S57, and if there are unselected base sequences, the process proceeds to step S52.

(S57) The leak source estimation unit 137 estimates that the provider that remains without being excluded in step S55 is the leak source. Preferably, at this point, the leak source candidates are narrowed down to one. The leak source estimation unit 137 outputs leak source information indicating the leak source, such as a leak source provider identifier. For example, the leak source estimation unit 137 causes the display 111 to display the leak source information.

According to the information processing apparatus 100 of the second embodiment, even when data is provided to a plurality of organizations, the organization of the leakage source can be estimated from the leakage data itself. Therefore, it is possible to strengthen the protection of data by requesting the organization of the leakage source to prevent recurrence. Further, as a method of imparting distinctiveness to the provided data, a method of deleting a small number of records from the data is used. Therefore, as compared with the method of adding noise to the value of the record, the authenticity of the record can be ensured, and the value of the data to the provider can be prevented from deteriorating. Further, as compared with the method of changing the modification information such as the length of the blank, the distinctiveness is not lost even if the providing organization performs an operation such as normalizing the data so that the modification information disappears.

Further, since the number of deleted records is determined from the desired estimation ability, it becomes easy to adjust the balance between the probability that the leakage source can be uniquely identified and the usefulness of the data. In addition, records are classified into a plurality of groups from the viewpoint of how the data is used by the organization to which the records are provided, and records belonging to different groups are mixedly deleted. Therefore, even if the providing destination organization uses only the records belonging to one group and leaks only the records belonging to the one group, there is a high possibility that the leakage source can be identified. In addition, since the unidentifiable records are also deleted so as to be consistent with the deletion ratio of the identifiable records, it is possible to maintain the characteristics that the deleted data is close to the data before the deletion.

10 Data processing equipment 11 Storage unit 12 Processing unit 13 Correspondence information 14a, 14b, 14c Provider identifier 15a, 15b, 15c Deletion pattern 16 Leakage data 17 Leakage source information

Claims (7)

Multiple subsets of the data by generating multiple deletion patterns indicating the records to be deleted from the data containing the plurality of records and deleting the records contained in the data based on each of the plurality of deletion patterns. And the processing unit that generates
A storage unit that stores correspondence information in which each of the plurality of deletion patterns is associated with a delivery destination identifier indicating a delivery destination of a subset generated based on the deletion pattern .
In the generation of the plurality of deletion patterns, the processing unit accepts the number of provision destinations and the presumable probability that the leakage source can be estimated from the leaked data, and is based on the number of provision destinations and the presumable probability. Each delete pattern determines the number of records to be deleted,
Data processing equipment. In the generation of the plurality of deletion patterns, the processing unit extracts a plurality of identifiable records that do not contain the same value as the other records from the data, and each deletion pattern is selected from the plurality of identifiable records. Select the record to be deleted,
The data processing apparatus according to claim 1 . In the generation of the plurality of subsets, the processing unit performs a part of the plurality of unidentifiable records other than the plurality of identifiable records among the plurality of records, depending on the number of records specified to be deleted. Delete additional records,
The data processing apparatus according to claim 2 . In the generation of the plurality of deletion patterns, the processing unit classifies at least a part of the plurality of records into a plurality of groups based on a predetermined criterion, and each deletion pattern specifies two or more records as deletion targets. Mix records belonging to different groups inside,
The data processing apparatus according to claim 1 . Multiple subsets of the data by generating multiple deletion patterns indicating the records to be deleted from the data containing the plurality of records and deleting the records contained in the data based on each of the plurality of deletion patterns. And the processing unit that generates
A storage unit that stores correspondence information in which each of the plurality of deletion patterns is associated with a delivery destination identifier indicating a delivery destination of a subset generated based on the deletion pattern .
The processing unit acquires the leaked data, searches for a deletion pattern in which the record included in the leaked data is not designated as a deletion target from the plurality of deletion patterns indicated by the corresponding information, and the above-mentioned. Extract the destination identifier corresponding to the searched deletion pattern,
Data processing equipment. It is a data processing method executed by a computer.
Generates multiple deletion patterns that indicate records that are deleted from data that contains multiple records.
By deleting the records contained in the data based on each of the plurality of deletion patterns, a plurality of subsets of the data are generated.
Correspondence information in which each of the plurality of deletion patterns is associated with the provision destination identifier indicating the provision destination of the subset generated based on the deletion pattern is stored in the storage unit.
In the generation of the plurality of deletion patterns, the number of provision destinations and the estimable probability that the leakage source can be estimated from the leaked data are accepted, and each deletion pattern is designated as the deletion target based on the number of provision destinations and the presumable probability. Determine the number of records,
Data processing method. On the computer
Generates multiple deletion patterns that indicate records that are deleted from data that contains multiple records.
By deleting the records contained in the data based on each of the plurality of deletion patterns, a plurality of subsets of the data are generated.
Corresponding information in which each of the plurality of deletion patterns is associated with a delivery destination identifier indicating a delivery destination of a subset generated based on the deletion pattern is stored in the storage unit.
Processing is executed, and in the generation of the plurality of deletion patterns, the number of provision destinations and the presumable probability that the leakage source can be estimated from the leaked data are accepted, and each deletion pattern is based on the number of provision destinations and the presumable probability. Determines the number of records to be deleted,
Data processing program.

Images