RU2780957C1 - Method for depersonalising personal data - Google Patents

Abstract

FIELD: data processing.

SUBSTANCE: invention relates to a method for depersonalising structured personal data. Method includes representing the data as a pre-compiled source table, wherein each row contains personal data of a single subject divided into a set of attributes corresponding to the structural units of the personal data; and converting each attribute of the source table in multiple stages, including dividing the set of elements of the attribute into blocks and permuting the blocks at each stage, wherein the composition and order of the intrablock data do not change in the course of the conversions; multiple conversion keys for all stages of conversion of each attribute are formed and stored on the server, wherein each key contains the number of blocks at each stage, the vector of elements in each block, and the permutation matrix at each stage; wherein the attributes of the source table are converted using the set conversion keys for each attribute so that the number of blocks for each attribute and the number of elements in each block are unique for each attribute and for each stage.

EFFECT: higher security of personal data.

8 cl, 2 tbl

Description

The invention relates to the field of protecting information stored in personal data information systems (ISPD) from unauthorized access by depersonalizing personal data, which are presented in the form of an initial table.

In the present description, the terms will be used in the following interpretation.

Structured personal data refers to data containing information of various types about the subject, which is presented in the form of separate structural units.

A structural unit of personal data is understood as information about a subject of a certain type (age, passport number, addresses, etc.).

The initial table of personal data is understood as a table, each row of which contains the personal data of one subject, corresponding to the structural units of personal data, presented in the form of records of a certain form.

A personal data attribute is a named structural unit in the original personal data table. The attribute contains a set of the same type of structural units of personal data of different subjects, presented as a column of the original table of personal data, which has the name of the attribute.

The table of impersonal personal data is understood as a table, the dimension of which, the composition of the attributes and the semantics of the data coincide with the dimension and semantics of the original table of personal data, but the composition of the rows of the table has been changed, the links between the types of personal data of the subjects have been destroyed.

Known System and method of masking data [US2004181670], which provide for the transformation (masking) of identifying personal data and their presentation in a table structurally identical to the original table of personal data. Access to personal data is provided to users with established roles (permissions) for the reverse transformation of anonymized data. The disadvantages of this method are:

- the possibility of indirect identification of the subject of personal data by not converted data, since not all personal data is converted;

- the inability to process personal data without prior reverse transformation of anonymized data (de-anonymization) ;

- the need for direct impact on personal data in the source table for their transformation, which can cause errors during direct and reverse transformations and violate the integrity of the data;

- the need to separate data users according to the roles (authorities) provided to them, which complicates the work with a large number of users.

Known System and methods of de-identification of records in the data source [US7269578], which provide for the transformation of personal data depending on the requirements for anonymity (security) data.

The disadvantages of the method are:

- direct impact on the original personal data, which may lead to distortion of information at the stages of direct or reverse transformation or the lack of the possibility of reverse transformation;

- difficulties in establishing and changing the requirements for data anonymity, for different users;

- lack of opportunities for processing personal data using anonymized (distorted) data without prior de-anonymization.

Known is a secure database system [EP0884670], based on the separation of identification and personal data of subjects and storing them in various tables and on various media. The identities, along with the keys linking the various tables, are encrypted by the clients.

The disadvantages of the method are:

- separation of data according to different tables, which reduces the reliability of the entire storage and access system, as well as increasing the processing time for requests for personal data;

- the use of encryption by clients, which leads to the need to manage the key management process (creation and distribution, key change);

- working with encrypted data complicates the process of comparing data in different tables;

- the impossibility of searching for data associated with various subjects without first decrypting all the data.

The closest to the proposed invention is the method of depersonalization of personal data [RU2538913], which consists in the fact that the individual attributes of personal data placed in the source table are divided into subsets (blocks), then a cyclic shift of the subsets is performed, after which a cyclic shift is performed in each subset the elements in it. The result is a table of depersonalized data. The parameters of the cyclic shifts are random numbers for each attribute and constitute the table transformation key.

The disadvantages of the method are:

- weak scattering of attribute elements over the entire set of attribute elements, since after transformation all elements remain inside their subsets, which increases the probability of reverse transformation without knowing the key, with known data on a set of individuals whose personal data are in the source table;

- a limited number of possible transformation options associated with the use of only cyclic shifts;

- one-time transformations that do not provide a high level of security of anonymized data due to the relatively small number of transformation options;

- restriction on the number of entries in the table, the number of entries must be very large to achieve good resistance to reverse transformations;

- difficulties in adding new records associated with the need to depersonalize the data of the entire table.

The main differences of the proposed method from the closest are:

- the possibility of forming the structure of the initial table of personal data by determining the set and composition of attributes;

- no changes inside personal data blocks generated during splitting;

- the possibility of multi-stage splitting with a rearrangement of personal data blocks generated during splitting, when the number of stages is determined by the applied indicators of the quality of transformation and security of personal data;

- the possibility of applying permutations other than cyclic shifts;

- the possibility of generating transformation keys for each stage.

The claimed method coincides with the closest in the following indicators:

- splitting the initial set of personal data into non-overlapping blocks;

- the possibility, in the form of a special case, to carry out cyclic permutations of a plurality of blocks of personal data attributes.

The technical problem to be solved by the claimed technical solution is to ensure the storage and processing of personal data in an anonymized form while maintaining the data semantics of the source table, the ability to process requests for anonymized personal data in the form of their numbers as part of attributes.

The technical result achieved by the implementation of the method is to increase the security of personal data, increase the security of structured personal data stored in an anonymized form, ensure the speed and convenience of their processing without reverse conversion in real time.

The specified technical result is achieved by the fact that the method of depersonalization of structured personal data consists in presenting the data in the form of a pre-formed source table, each row of which contains the personal data of one subject, divided into a set of attributes corresponding to the given units of the personal data structure, and a multi-stage transformation of each attribute of the source table. table, which includes splitting a set of attribute elements into blocks and permuting blocks at each stage, while the composition and order of intra-block data do not change during transformations, a set of transformation keys is formed and stored on the server for all stages of transformation of each attribute, where each key contains the number of blocks per at each stage, a vector of elements in each block, and a permutation matrix at each stage, while the transformation of the attributes of the source table is performed using the given transformation keys for each attribute in such a way that that the number of blocks for each attribute and the number of elements in each block are unique for each attribute and each stage.

In addition, the number of transformation steps for each attribute of the source table is unlimited.

In addition, a plurality of transformation keys is generated and stored on the depersonalization server for all stages of transformation of each attribute, where each key is unique and contains information for transformation at the stage.

In addition, the original table of personal data can be obtained from any structured form of personal data representation received from remote sources, and have a variable number of attributes that does not match the number of structural units in personal data received from remote sources, and provides an increase in the security of the anonymized table. personal data, preserving the semantics of attribute elements.

In addition, to increase data security, the initial personal data table can be created on a separate server and, upon request, transferred to the depersonalization server, which is part of the personal data information system (ISPD), where transformation keys are generated and stored, after depersonalization, the original personal data table data is destroyed, since it is possible to restore it according to the table of anonymized personal data.

In addition, during transformations, the elements of each attribute are moved on the entire set of attribute elements, which provides more effective protection against de-anonymization with a direct enumeration of options and the use of a set of known personal data by increasing the number of possible options for anonymization.

In addition, it is possible to use various options for storing the table of anonymized personal data.

In addition, it is possible to process personal data using an anonymized data table without first de-anonymizing the entire table due to the possibility of de-anonymizing only the requested data.

The initial personal data table (PD) - T ₀ ( N , M ) - contains N records (rows) and M attributes (columns). All strings and attributes have unique numbers from 1 to N and from 1 to M , respectively.

Further, for convenience, we will consider the attribute number j of the source table, which contains an ordered numbered set of N elements (data on subjects) – C _{j 0} = { c _{j 0} ( m )}. Here c _{j 0} ( m ) is the unique number of the element having the ordinal number m (standing in place m ) in the set C _{j 0} . In the source table T ₀ ( N , M ) the ordinal number of the element is equal to the number of the row in which this element is located, and the unique element number is always equal to its ordinal number in the source table: c _{j 0} ( m ) = m , (1 < c _{j 0} ( m ) ≤ N ; 1 ≤ m ≤ N ). The ongoing transformations lead to a mismatch between the unique and ordinal numbers of elements.

Subject data may not be present in individual attribute elements, in which case the attribute element is a special set of characters.

The transformation of the data set of attribute number j ( j = 1, 2, …, M ) occurs in several stages, the number of which is R _j (1 ≤ R _j  < ∞).

Each step number r ( r = 1, 2, …, R _j ) contains two steps.

последовательно занумерованных элементов c номерами от

до

(i = 1, 2, …, Z _jr ;

). Также

, т.е. блоки разбиения содержат все элементы атрибута.At the first step, the attribute data set is divided into non-overlapping data blocks. The number of blocks is Z _jr , where 1 < Z _jr ≤ N . All blocks have serial numbers. Each block number i contains

sequentially numbered elements with numbers from

before

( i = 1, 2, …, Z _jr ;

). Also

, i.e. breakboxes contain all attribute elements.

,

, …,

) являются в общем случае уникальными для каждого атрибута и каждого этапа.The number of blocks Z _jr of the attribute j at stage r and the vector of the number of elements in each block k _jr = (

,

, …,

) are generally unique for each attribute and each stage.

, (i = 1, 2; j = 1, 2, …, Z _jr ), где x _{1 j}  – порядковый номер блока при разбиении (1 ≤ x _{1 j}  ≤ Z _jr , как правило, x _{1 j}  = j), а x _{2 j} – номер блока разбиения, стоящего на месте, указанном в верхней строке в столбце j, после перестановки (1 ≤ x _{2 j} ≤ Z _jr ), т.е. x _{2 n} – номер блока разбиения, имеющего после перестановки номер n (номер x _{1 n} ). В результате получается преобразованное множество атрибута j –

, где r – номер этапа (r = 1, 2, …, R _j ).At the second step of stage r , the attribute blocks are permuted, given by the permutation matrix

, ( i = 1, 2; j = 1, 2, …, Z _jr ), where x _{1 j}  is the serial number of the block when splitting (1 ≤ x _{1 j} ≤ Z _jr , as a rule, x _{1 j} = j ), and x _{2 j} is the number of the splitting block, which stands at the place indicated in the top line in column j , after permutation ( 1 ≤ x _{2 j} ≤ Z _jr ), i.e. x _{2 n} is the number of the partition block that has number n after the permutation (number x _{1 n} ). The result is a transformed set of attribute j −

, where r is the stage number ( r = 1, 2, …, R _j ).

,

, …,

. Множество

есть окончательный результат преобразования множества данных атрибута номер j. После окончания проведения преобразований всех атрибутов получается таблица обезличенных ПД – T ^*(N, M).The initial set for transformation at the first stage is the data set of the attribute j of the original table - C _{j 0} , the initial sets for partition transformations at stages 2, 3, …, R _j are the sets

,

, …,

. Lots of

is the final result of the transformation of the data set of attribute number j . After the completion of the transformations of all attributes, a table of impersonal PD is obtained - T ^* ( N , M ).

Transformation of the original PD table at each stage changes the serial numbers of the elements of all attributes, destroying the links between the elements of the row of the original table.

,

, …,

) и матрицей

. При этом c _jr (n) = c _{j ( r –1)}(m). Let it be required to calculate the new ordinal number n of the attribute element j after the execution of stage r , which after the end of the stage ( r - 1) has an ordinal number m and a unique number - c _{j ( r - 1)} ( m ). The partition and permutation are given by the vector k _jr = (

,

, …,

) and matrix

. In this case, c _jr ( n ) = c _{j ( r –1)} ( m ).

The number of the block h , which contains the element number m after splitting, is calculated from the condition:

.

.

The element number inside block h is:

.

.

After permutation, block number h will have serial number z , which is calculated from the condition: x _{2 z} = h .

The element number after the permutation is:

.

.

Thus, a new serial number of the element is determined after the stage. This number becomes the element number before the next conversion step. The unique number of the element does not change and always remains equal to the ordinal number of this element in the original table T ₀ .

The attribute transformation parameters are set by the set: K _jr = { Z _jr , k _jr , X _jr }, which is the transformation key of attribute j at stage r . The vector K _j = ( K _{j 1} , K _{j 2} , …, K _jRj ) specifies the set of keys for all stages of transformation of the attribute j ( j = 1, 2, …, M ).

The set of keys for all attributes and all stages of transformation of the source table: K ( T ₀ ( N , M )) = { K _j }, ( j = 1, 2, …, M ) is the transformation key of the source table.

Evaluation of the security of the transformation.

The security of the table T ^* ( N , M ) against attacks aimed at de-anonymization, in the absence of data on the transformation key, can be estimated by the number of possible transformation options.

The number of partitions of N attribute elements into k blocks is equal to the number of possible combinations from ( N - 1) to ( k - 1) and is equal to:

.

.

The number of possible splits of N attribute elements is:

.

.

The possible number of permutations of k blocks in a stage is k !. Therefore, the total number of possible transformation options for an attribute containing N elements at one stage is:

.

.

If the complete transformation of one attribute number j is carried out in R _j stages, then the number of options for the complete transformation of this attribute is:

.

.

For the original table T ₀ ( N , M ) containing M attributes, the number of transformation options is:

.

.

Thus, the number of possible transformations of the table T ₀ ( N , M ) is very large (real values of N lie in the range from 10 ³ to 10 ⁸ ). For example, at N = 100, M = 10, R _j  = 1, Z _{j 1} = 10 ( j = 1, 2, …, 10) we get U ₀ (10, 100) ≈ 10 ⁵ ×17310308 ≥ 10 ¹² . For a fixed k = 10 U (100) ≥ 10 ¹³ . Whence T (100, 10) ≥ 10 ¹³⁰ .

For a more significant increase in the number of options for small values of N , you can increase the number M by splitting the elements of one attribute into several new structural parts, each of which will be part of its own attribute (creating new attributes). For example, you can divide the subject's phone into 5 parts.

The inventive method is implemented as follows.

To carry out depersonalization, an initial table is created from structured personal data stored in various remote storages, each row of which contains data about one subject of personal data. To create a table, data is collected from remote storages (sources) of personal data by transferring personal data via open or secure communication channels to the server for generating the initial table. Each source also transmits information about the structure of personal data of this source. All data is sent to the server for generating the initial personal data table, implemented either in hardware or software, which provides communication, information protection and the formation of a table of initial data of a given structure by integrating and necessary transformations of data coming from remote sources. To receive and convert data from remote sources, specialized software should be used, which is based on data conversion programs (PDF to Word, XML converters, Microsoft Excel spreadsheets); programming languages for creating applications: C++, C#, PHP, Java, Python.

The generated initial table of personal data, upon request, is transmitted to the depersonalization server, which generates a table of depersonalized personal data. Then the original table is destroyed.

Attribute transformation keys are created and stored on the depersonalization server. To create, store and protect keys, software is used that generates conversion keys according to specified algorithms, creates a separate key file and protects this file from unauthorized access by such methods as encryption, moving to another storage, etc. The software can use the capabilities of DriveCrypt, PGP Personal Desktop to store and protect key files, as well as programming languages to create key generating applications: C++, C#, PHP, Java, Python.

The original personal data table, using the attribute transformation keys, is converted into an anonymized personal data table containing the same attributes as the original table and preserving the semantics of records for each attribute. To generate a table of anonymized personal data, software is used that implements the claimed method of anonymization, performs a multi-stage transformation of the attributes of the original table using the specified transformation keys for each attribute, and replaces the original table with the resulting table of anonymized personal data. The basis for such software are: DBMS - MySQL, Postgre SQL, Microsoft Access; programming languages for creating applications - C++, C#, PHP, Java, Python.

The depersonalization server can transfer the depersonalized data table to the personal data processing server to service requests for personal data processing from registered users.

The separation of the functions of generating the initial table, depersonalization and processing of personal data allows you to increase the performance and security of the personal data processing system.

An example of filling in the initial tables of personal data T ₀ (10, 3)

The original table consists of 10 rows ( N = 10), includes three attributes A, B, C ( M = 3) and looks like:

Initial table of personal data T ₀ (10, 3)

subject number Attribute 1 (A) Attribute 2 (B) Attribute 3 (C) one a _one b _one c _one 2 a ₂ b ₂ c ₂ 3 a ₃ b ₃ c ₃ four a _four b _four c _four 5 a ₅ b ₅ c ₅ 6 a ₆ b ₆ c ₆ 7 a ₇ b ₇ c ₇ eight a _eight b _eight c _eight 9 a ₉ b ₉ c ₉ ten a _ten b _ten c _ten

Transformation keys for attributes are set:

Attribute 1: R ₁ = 2, Z ₁₁ = 3, Z ₁₂ = 3. Attribute Key: (2, 3, (4, 2, 4), X ₁₁ , 3, (3, 5, 2), X ₁₂ ) ,

, X ₁₂ =

.where X ₁₁ =

, X ₁₂ =

.

Attribute 2: R ₂ = 1, Z ₂₁ = 4. Attribute Key: (1, 4, (2, 3, 1, 4), X ₂₁ ),

.where X ₂₁ =

.

Attribute 3: R ₃ = 2, Z ₃₁ = 5, Z ₃₂ = 3. Attribute Key: (2, 5, (2, 1, 2, 2, 3), X ₃₁ , 3, (5, 2, 3) , X ₃₂ ,),

, X ₃₂ =

.where X ₃₁ =

, X ₃₂ =

.

After converting the original table, a table of anonymized personal data is obtained:

Anonymized data table T ^* (10, 3)

subject number Attribute 1 (A) Attribute 2 (B) Attribute 3 (C) one a ₉ b ₃ c ₉ 2 a _ten b _four c _ten 3 a ₂ b ₅ c ₃ four a ₃ b ₇ c _one 5 a _four b _eight c ₂ 6 a ₇ b ₉ c ₆ 7 a _eight b _ten c ₇ eight a ₅ b ₆ c _four 9 a ₆ b _one c ₅ ten a _one b ₂ c _eight

As you can see, each row contains attribute values, not all of which correspond to the values in the original table, i.e. in each line is no longer the personal data of any subject.

The advantages of the proposed method are:

- ensuring greater data protection from de-anonymization by indirect signs, since the initial numbers of elements of each attribute after transformation can take any value from 1 to N , and the number of possible transformation options is very large;

- application for depersonalization of a limited number of the same type of transformations (partitions and permutations), which simplifies the development and operation of software;

- the ability to dynamically manage the transformation keys at each stage and for each attribute;

- the possibility of processing personal data without first de-anonymizing the entire table;

- the ability to store the attributes of the anonymized data table in various databases and on various servers;

- the ability to add new attributes and convert them without de-anonymizing the created table of anonymized personal data;

- the possibility of operational management of depersonalization / depersonalization processes;

- the possibility of forming a unified structure of the initial table of personal data, combining the structures of personal data coming from various external sources;

- the ability to split the original table into parts and form several tables of anonymized personal ones to increase the speed of access processes and request processing;

- the possibility of eliminating the original table of personal data after the formation of a table of anonymized personal data;

- the ability to store a table of anonymized personal data in the form of a distributed registry using blockchain technology.

The use of the claimed invention provides:

- a significant increase in the security of anonymized data from de-anonymization;

- improving the quality of personal data processing due to the preservation of semantics and the absence of opportunities for distortion and loss of information during depersonalization;

- increasing the speed of processing requests for personal data without de-anonymization.

Structured personal data can come from remote sources in the form of files, database tables. To carry out depersonalization, it is required, on the basis of the existing forms of representation of personal data, to compile an initial table of personal data, each row of which represents the personal data of one subject, and each column (attribute) contains a set of data of the same type corresponding to the elements of a given structure, for all subjects entered in the table.

The attributes of the source table can be elements of structural units of the initial structure of personal data (surname of the subject, city, phone, etc.), and can also be parts or combinations of elements of the initial structure, which increases the complexity of the task of unauthorized de-anonymization and reduces the requirements for the number of subjects personal data in the source table with a given number of de-anonymization options.

Claims (8)

1. A method for depersonalizing structured personal data, which consists in presenting data in the form of a pre-formed source table, each row of which contains personal data of one subject, divided into a set of attributes corresponding to units of the structure of personal data, and a multi-stage transformation of each attribute of the source table, including splitting the set attribute elements into blocks and permutations of blocks at each stage, characterized in that the composition and order of intra-block data do not change during transformations, a set of transformation keys is formed and stored on the server for all transformation stages of each attribute, where each key contains the number of blocks at each stage, a vector of elements in each block and a permutation matrix at each stage, wherein the transformation of the attributes of the source table is performed using the given transformation keys for each attribute so that the number of blocks for each attribute and the number of elements in each block are unique for each attribute and each stage. 2. The method according to claim 1, characterized in that the number of conversion steps for each attribute of the source table is unlimited. 3. The method according to claim 1, characterized in that a plurality of transformation keys is generated and stored on the depersonalization server for all transformation stages of each attribute, where each key is unique and contains information for transformation at the stage. 4. The method according to claim 1, characterized in that the original personal data table can be obtained from any structured form of personal data representation received from remote sources, and have a variable number of attributes that does not match the number of structural units in personal data received from remote sources, and providing an increase in the security of the table of anonymized personal data, while maintaining the semantics of the attribute elements. 5. The method according to claim 1, characterized in that, in order to increase data security, the initial table of personal data can be created on a separate server and, upon request, transferred to the depersonalization server, which is part of the personal data information system (PDIS), where transformation keys are generated and stored , after depersonalization, the original table of personal data is destroyed, since it is possible to restore it according to the table of depersonalized personal data. 6. The method according to claim 1, characterized in that during transformations, the elements of each attribute are moved on the entire set of its elements, which provides more effective protection against de-identification with a direct enumeration of options and the use of a set of known personal data by increasing the number of possible options for de-identification. 7. The method according to claim 1, characterized in that it is possible to use various options for storing a table of anonymized personal data. 8. The method according to claim 1, characterized in that it is possible to process personal data using an anonymized data table without first de-anonymizing the entire table due to the possibility of de-anonymizing only the requested data.