WO2021051612A1

WO2021051612A1 - Automatic data authorization desensitization method, system, device, and storage medium

Info

Publication number: WO2021051612A1
Application number: PCT/CN2019/118433
Authority: WO
Inventors: 王兴川
Original assignee: 平安科技（深圳）有限公司
Priority date: 2019-09-19
Filing date: 2019-11-14
Publication date: 2021-03-25
Also published as: CN110727954A; CN110727954B

Abstract

Provided is a method for automatic data authorization desensitization, said method comprising: creating desensitization requirements according to preset rules, and determining table access permission and data access scope according to the desensitization requirements (S110); according to the table access permission and data access scope, initiating in a DMP a request for permission to access the service table, the DMP being used for approving the permission request (S120); by means of the DMP, desensitizing the configuration of the approved permission request, the desensitization configuration comprising a desensitization rule specified for the mandatory configuration of sensitive fields (S130); synchronizing to an ETL the specified desensitization rule and the data access scope, the ETL producing target fields according to the data access scope, and automatically generating a desensitization task for the target field according to the desensitization rule (S140); checking whether the desensitization source and target fields are consistent; if consistent, then continuing to keep the desensitization source and target fields and order consistent; if not, then making the desensitization source and target fields consistent; after the inspection is completed, executing the ETL and completing the desensitization task (S150).

Description

Data authorization desensitization automation method, system, device and storage medium

This application requires the priority of the patent application whose application number is 201910885652.0, the application date is September 19, 2019, and the invention and creation title is "Data Authorization Desensitization Automation Method, Device, and Storage Medium".

Technical field

This application relates to the field of data processing, and in particular to an automated method, system, device and storage medium for data authorization and desensitization.

Background technique

With the advent of the era of big data, the commercial value of big data, the precise positioning of users, and the huge commercial value hidden in big data have been gradually discovered, but at the same time it also brings a huge challenge-the protection of personal privacy information. Personal information and personal behavior (such as location information, consumer behavior, network access behavior), etc., these are the privacy of people, and they are also a type of sensitive information that we are concerned about. How to protect people's private information based on the value mining of big data , Will also be a problem that must be solved for data desensitization.

The existing data desensitization method is only to use esProc to write SPL scripts for data report query, and to identify sensitive information fields (such as: name, certificate number, bank account, address, telephone number, business name, industrial and commercial registration number, and taxpayer identification). No.), etc., perform data desensitization and deformation through predefined desensitization rules to realize the protection of sensitive privacy data. However, the applicant realizes that in the existing data desensitization method, the data is not authorized first, and then the data is automatically desensitized. There is a great security risk, and once a field is added or deleted, the desensitization Tasks cannot be performed sequentially, requiring human involvement, and the storage before and after data desensitization is equivalent to storing 2 copies of data, which wastes storage.

Therefore, there is an urgent need for an automatic desensitization method that saves storage space and can improve the desensitization efficiency.

Summary of the invention

This application provides an automated method, electronic device, and computer-readable storage medium for data authorization and desensitization. Its main purpose is to determine table access permissions and data access scopes by creating desensitization requirements, and then use DMP according to the access permissions and data access scopes. Initiate an application for permission to access the business table. DPM is used to approve the application. DMP performs desensitization configuration on the approved permission application, compulsory configuration of sensitive fields and specifies desensitization rules to form a desensitization task, and then execute the desensitization task To ensure the smooth completion of desensitization tasks, improve desensitization efficiency, and reduce storage space.

In order to achieve the above purpose, the data authorization desensitization automated method provided in this application is applied to an electronic device, and the method includes:

S110: Create a desensitization requirement according to a preset rule, and determine table access authority and data access scope according to the desensitization requirement;

S120: Initiate an application for permission to access the service table in the DMP according to the table access permission and data access scope, and the DMP is used to approve the permission application;

S130: Perform desensitization configuration on the approved permission application through the DMP, where the desensitization configuration includes mandatory configuration designated desensitization rules for sensitive fields;

S140: Synchronize the specified desensitization rule and the data access range to ETL, the ETL generates a target field according to the data access range, and automatically generates a desensitization task for the desensitization field according to the desensitization rule ；

S150: Check whether the desensitization source and target fields are the same. If they are the same, continue to keep the desensitization source and target fields and sequence consistent; if they are inconsistent, make the desensitization source and target fields consistent. After the check is completed, execute ETL and complete the description Desensitization task.

Preferably, the desensitization requirement is a desensitization requirement created according to a preset rule, and the desensitization requirement is restricted by a data provider; the specific desensitization field of the desensitization requirement includes business sensitive fields;

The data access range includes at least a time range and a span range.

Preferably, the DMP initiates an application for permission to access the business table according to the table access permission and data access scope, and the step of using the DMP to approve the permission application includes:

The data user initiates an application for permission to access the business table;

The criteria for determining that the desensitization requirements of the data user meet the data use requirements include at least not damaging the desensitization source and not containing out-of-range instructions.

Preferably, the corresponding desensitization method in the desensitization rule includes direct desensitization and associated desensitization; wherein,

The direct desensitization is direct desensitization by applying a preset rule to the desensitization source, and has nothing to do with other fields;

The association desensitization is to apply the same desensitization rule to the associated fields between desensitization sources, so that the desensitized fields can still be associated.

Preferably, the preset rule includes at least one of the following rules:

The HASH encryption name provided for any character string is uniformly changed to "the name of the data user";

Change the last 4 digits to 1234 or shield the middle 4 digits;

All mailboxes are shielded as the preset mailboxes of the data user;

The amount category randomly floats 1% to 5%, and the address category intercepts the first 15 characters or displays only up to the level of the district.

Preferably, the process of automatically generating the desensitization task of the target field according to the desensitization rule includes:

Determine whether to create a desensitized table, view or materialized view according to the desensitization rule;

If a desensitization table is created, the desensitization table is established according to the fields of the desensitization source; if a view and a materialized view are created, a new view is generated according to the desensitization rule and the previous view is replaced.

Preferably, the process of determining whether to establish a desensitization table or a view according to the desensitization rule includes:

Pre-setting the interval of the HDFS space storage occupied by the desensitization source according to the desensitization rule;

If the desensitization source occupies less than the interval, the view is created; if it is in the interval, the materialized view is created; if it is greater than the interval, the desensitization table is created.

Preferably, after the step of checking whether the desensitization source and target fields are consistent, the method further includes:

If the desensitization source and target fields are inconsistent, judge whether to add a field or delete a field according to the order of the target field;

If the desensitization source has a new field relative to the target field, add the same field based on the target field and replace the previous desensitization table, materialized view or view. The desensitization source has a deleted field relative to the target field, then The same field is deleted on the basis of the target field, and the previous desensitized table, materialized view or view is replaced.

In order to achieve the above objective, the present application also provides an automatic electronic device for data authorization and desensitization. The electronic device includes a memory, a processor, and a computer program stored in the memory and running on the processor. The processor implements the steps in the automatic monitoring method for data authorization desensitization according to any one of claims 1-8 when the computer program is executed by the processor;

In addition, in order to achieve the above-mentioned object, the present application also provides a computer-readable storage medium, wherein the computer-readable storage medium stores a data authorization desensitization automated analysis program, and the data authorization desensitization automation program is When the processor is executed, the steps of the aforementioned automatic method for data authorization and desensitization are realized.

The data authorization and desensitization automation method, electronic device, and computer readable storage medium proposed in this application, in which the table access authority and data access scope are determined according to the desensitization requirement, and then the data user is in DMP according to the table access authority and data access scope Initiate an application for permission to access the business table, and DMP will approve the permission application. After the approval is passed, the desensitization configuration is performed in DMP, and the designated desensitization rules are mandatory for sensitive fields, and then the designated desensitization rules and data access scope are synchronized To ETL, ETL automatically generates corresponding desensitization tasks according to corresponding information, generates target fields, and checks whether the desensitization source and target fields are consistent. If they are the same, continue to keep the fields and sequence of the desensitization source table and target table consistent. If they are inconsistent, make the source and target fields consistent. After checking, execute ETL and complete the desensitization task, which greatly ensures that the changes of the fields are automatically changed synchronously, reducing human involvement, saving manpower, and improving desensitization. Efficiency reduces storage space.

Description of the drawings

FIG. 1 is a schematic diagram of an application environment of an automated method for data authorization and desensitization according to an embodiment of the present application;

Fig. 2 is a flowchart of an automated method for data authorization and desensitization according to an embodiment of the present application;

Fig. 3 is a system framework diagram in an automated electronic device for data authorization and desensitization according to an embodiment of the present application;

Fig. 4 is a schematic structural diagram of an electronic device according to an embodiment of the present application.

The realization, functional characteristics, and advantages of the purpose of this application will be further described in conjunction with the embodiments and with reference to the accompanying drawings.

detailed description

It should be understood that the specific embodiments described here are only used to explain the present application, and are not used to limit the present application.

In order to solve the aforementioned existing data desensitization methods, only use esProc to write SPL scripts for data report query, and perform data desensitization and deformation on sensitive information fields through predefined desensitization rules to realize the protection of sensitive and private data. Automatically desensitize data, which poses a great security risk. Once a field is added or deleted, the desensitization task cannot be performed sequentially. It requires human participation and wastes storage. This application starts with data authorization. Sensitive requirements determine the table access authority and data access scope, and then the data user initiates an application for permission to access the business table in DMP according to the table access authority and data access scope. DMP approves the permission application. After approval, DMP performs desensitization configuration. Configure desensitization rules for sensitive fields to form a desensitization task, and then complete the desensitization task.

The DMP (Data Management Platform) data management platform integrates scattered multi-party data into a unified technology platform, and standardizes and subdivides these data, so that users can push these subdivided results to the existing interactive marketing environment In the platform.

ETL, the abbreviation of Extract-Transform-Load in English, is used to describe the process of extracting, transforming, and loading data from the source to the destination.

Specifically, according to an embodiment of the present application, an automated method for data authorization and desensitization is provided, which is applied to the electronic device 40.

Fig. 1 is a schematic diagram of an application environment of an automated method for data authorization and desensitization according to an embodiment of the present application. As shown in FIG. 1, in the implementation environment of this embodiment, it is a computer device 110.

The computer device 110 is a computer device, such as a terminal device such as a computer.

It should be noted that the computer terminal device 110 may be a tablet computer, a notebook computer, a desktop computer, etc., which is a cenOS (linux) system, but is not limited to this. The terminal device 110 such as a computer device can be connected via Bluetooth, USB (Universal Serial Bus) or other communication connection methods, which is not limited in this application.

Fig. 2 is a flowchart of an automated method for data authorization and desensitization according to an embodiment of the present application. As shown in Figure 2, in this embodiment, the automatic data authorization desensitization method includes the following steps:

S110: Create desensitization requirements according to preset rules, and determine table access permissions and data access scopes according to the desensitization requirements;

The created desensitization requirement refers to the desensitization requirement created by the data provider according to preset rules, which is restricted by the data provider; the specific desensitization fields included in the desensitization requirement are business sensitive fields, such as amount, sales, Telephone, address, ID number, etc.;

The data access range generally includes time range, span range, etc., such as only allowing access to the records of the most recent month, only allowing access to the records of a certain department of the data user, or all records of the data user.

S120: Initiate an application for permission to access the business table in DMP according to the access permission and data access scope of the table, and the DMP is used to approve the permission application;

Among them, all professional company data that the data provider can see on the DMP platform, and the data user applies for desensitization and desensitization rules on the DMP platform, which requires the approval of the data provider;

Specifically, the data user is the person to be approved, the data provider is the approver, and the data provider determines that the desensitization requirement meets the data use requirements, it will pass the approval;

Whether it is approved or not is determined by the data provider, and all data users who have passed the approval of the DMP can apply for submitting desensitization;

The criteria for judging that the desensitization requirement meets the data use requirements are: no damage to the desensitization source, no over-range instructions (for example, there is no "phone" column in the desensitization source, and the desensitization requirement is selected as "phone"), etc., here The desensitization source is the data that is initially screened by the data user and waiting to be desensitized.

S130: Perform desensitization configuration on approved permission applications through DMP, and compulsorily configure designated desensitization rules for sensitive fields;

After the approval is passed, the desensitization configuration is performed in the DMP. For "special" sensitive fields (such as ID number, telephone number, amount, etc.), the specified desensitization rules are mandatory to ensure that the data can be used correctly. Undermine the ambiguity of data and maintain the business integrity of the data. In the desensitization method corresponding to the desensitization rules, users can choose between direct desensitization or associated desensitization; different specialized business sensitive rules may be inconsistent, and the system provides a basis It is not necessary to formulate different and unified desensitization rules in order to achieve "individualized" customized desensitization rules.

Direct desensitization means applying preset rules to desensitize the fields of a certain table, and has nothing to do with the fields of other tables. The preset rule includes at least one of the following rules:

Change the last 4 digits to 1234 or shield the middle 4 digits;

All mailboxes are shielded as the preset mailboxes of the data user;

Association desensitization is to apply the same desensitization rule to the associated fields between desensitization sources, so that the desensitized fields can still be associated, that is, the associated fields between tables apply the same desensitization rule to achieve desensitization. The sensitive fields can still be associated without destroying the consistency of the data.

Among them, sensitive fields are fields that must be desensitized identified by the data provider. For such fields, DMP must choose to specify one of several desensitization rules before providing it, otherwise it cannot be submitted. Such fields are directly desensitized.

S140: Synchronize the specified desensitization rules and data access scope to ETL, and ETL generates target fields according to the data access scope, and automatically generates desensitization tasks for the target fields according to the desensitization rules;

Among them, the process of the ETL automatically generating the corresponding desensitization task according to the corresponding information includes:

If you create a desensitization table, create a desensitization table based on the fields of the desensitization source; if you create a view and a materialized view, generate a new view according to the desensitization rule and replace the previous view.

Specifically, after configuring the desensitization rules, the corresponding rules and data access scope are synchronized to ETL, and ETL automatically generates corresponding desensitization tasks according to the received desensitization rules and data access scope, including automatically creating target fields , That is, a desensitized table, materialized view, or view to prepare for desensitizing data.

In order to improve the efficiency of subsequent use and reduce storage space, the process of determining whether to establish a desensitized table, a materialized view or a view according to the desensitization rule includes:

According to the desensitization rules, preset the interval in which the desensitization source occupies the HDFS space storage;

If the desensitization source occupies less than the interval, the view is created, if it is in the interval, the materialized view is created, and if it is greater than the interval, the desensitization table is created. For example, when the original Hive table occupies less than 100G of HDFS space, the view is created, and if it is greater than 100G and less than 500G, then a materialized view is created. If a single table is greater than 500G, a desensitized table is created, and the desensitized data is directly inserted into the desensitized table. , The specific rules are based on business needs.

S150: Check whether the desensitization source and target fields are the same. If they are the same, continue to keep the desensitization source and target fields and sequence consistent; if they are inconsistent, make the desensitization source and target fields consistent, execute ETL after checking and complete desensitization Task; the desensitization source is the data that is initially filtered by the data consumer and waiting to be desensitized.

The process of checking whether the desensitization source and target fields are consistent includes:

If the desensitization source field is changed, judge whether to add a field or delete a field according to the order of the target field;

If the desensitization source has a new field relative to the target field, add the same field based on the target field and replace the previous desensitization table, materialized view or view. If the desensitization source has a deleted field relative to the target field, then Delete the same field on the basis of the target field, and replace the previous desensitized table, materialized view, or view.

Specifically, the data after applying the desensitization rule is also stored in Hive;

Among them, the desensitization source is the data that is initially filtered by the data user and is waiting to be desensitized, and the target field is the accurate desensitization data that is immediately desensitized. The data is stored in Hive in the form of a table. The target field table is a Hive table, which cannot be processed like a relational database. Therefore, the order of the fields is particularly important. Otherwise, the desensitized fields may not be desensitized, and the fields that should not be desensitized are desensitized. Make sure that the desensitization source table and the target field table have the same meaning in a specific order. Make sure that the field that uses the desensitization rule is the field that the user really needs to desensitize, and it needs to be judged according to the order of the fields. Therefore, before the desensitization task runs, it is necessary to keep the desensitization source table and the target field table and sequence consistent. If they are inconsistent, the task will fail or the data will be misplaced. Therefore, check whether the desensitization source table and the target field table are Before running the desensitization task, the desensitization can be carried out smoothly after the two sides are consistent. Otherwise, the task will fail and human intervention will be required, which will increase the workload of task operation and maintenance.

If the desensitization task is to create a view, if there is a field change, that is, add or delete, it will be judged according to the order of the fields. If the desensitization source adds a new field at the end, it can be added on the original basis. Field, replace the previous view, otherwise you need to delete and create a new hive table. After completion, perform subsequent ETL processes such as view replacement or materialized view rebuild and desensitization table data reloading.

The automatic data authorization desensitization method in this embodiment judges the table access authority and data access scope according to the desensitization requirement, and then the data user initiates an application for permission to access the business table in the DMP according to the table access authority and data access scope, and the DMP applies for the authority After the approval is passed, the desensitization configuration is performed in the DMP, and the designated desensitization rules are mandatory for sensitive fields, and then the designated desensitization rules and data access scope are synchronized to the ETL, and the ETL automatically generates the corresponding information according to the corresponding information. For the desensitization task, generate the target field, check whether the desensitization source and target fields are consistent, if they are the same, continue to keep the fields and sequence of the desensitization source table and the target table consistent, if they are inconsistent, make the source and target fields consistent, check After completion, the ETL is executed and the desensitization task is completed, which greatly ensures the automatic synchronization of changes to the field, reduces human participation, saves manpower, improves the desensitization efficiency, and reduces storage space.

On the other hand, this application also provides a data authorization and desensitization automation system. Figure 3 is a framework diagram of a data authorization and desensitization automation system according to an embodiment of the application. The system corresponds to the data authorization and desensitization automation method and can be set in the data Authorized desensitization in automated electronic devices.

As shown in FIG. 3, the data authorization desensitization automation system includes a DMP module 310, an ETL module 320, an original library module 330, and a desensitization library module 340.

The DMP module 310 is used to accept the permission application for accessing the business table initiated by the data user, and the permission application includes the table access permission and the data access scope; then the DMP module 310 uses the data according to the data user’s access permission and access scope. The authority application of the party is approved. If approved, the DMP module 310 starts to perform desensitization configuration, that is, compulsorily configure the designated desensitization rule for the desensitization field, and synchronize the desensitization rule to the ETL module 320.

Among them, the table access authority and data access scope are determined according to the desensitization requirements, which are created by the data user according to preset rules. The specific desensitization fields included are some business sensitive fields, such as amount, sales, phone number, and address. , ID number, etc. The scope of data access generally includes time range, span range, etc., such as only allowing access to the records of the most recent month, only allowing access to the records of a certain department of the data user, or all the records of the data user.

All the professional company data that the data provider can see on the DMP module 310, the data user applies for desensitization and desensitization rules in the DMP module 310, the approval of the data provider is required; specifically, the data user is the person to be approved , The data provider is the approver, and the data provider determines that the desensitization requirement meets the data use requirements, it will pass the approval; whether it is passed is determined by the data provider, and the data user approved by the DMP module 310 can Apply for desensitization; the criteria for determining that the desensitization requirement meets the data use requirements are: no damage to the desensitization source and no over-range instructions, such as: there is no "phone" column in the desensitization source, and the desensitization requirement is selected as "phone". The desensitization source here is the data that is initially screened by the data user and waiting to be desensitized.

The ETL module 320 generates a target field according to the data access range, automatically generates a desensitization task for the target field according to a desensitization rule, and executes the desensitization task; and, before performing the desensitization task, the ETL module needs to check the source of the desensitization Whether it is consistent with the target field, if they are consistent, continue to keep the desensitization source and target fields and sequence consistent, if they are inconsistent, make the desensitization source and target fields consistent, and perform the desensitization task after the inspection is completed.

In short, the ETL module 320 automatically generates and executes desensitization tasks according to the desensitization rules synchronized by the DMP module 310; wherein, the process of the ETL module 320 automatically generates corresponding desensitization tasks according to corresponding information includes:

Determine whether to create a desensitized table, view, or materialized view according to the desensitization rule; if you create a desensitization table, create a desensitization table based on the field of the desensitization source; if you create a view and a materialized view, generate a new view according to the desensitization rule, And replace the previous view.

Among them, the desensitization methods corresponding to the desensitization rules include direct desensitization and associated desensitization; among them, direct desensitization refers to direct desensitization by applying preset rules to the desensitization source, and has nothing to do with other fields; and associated desensitization refers to desensitization. The associated fields between sensitive sources apply the same desensitization rule, so that the desensitized fields can still be associated.

The preset rules include at least one of the following rules:

Change the last 4 digits to 1234 or shield the middle 4 digits;

All mailboxes are shielded as the preset mailboxes of the data user;

In a specific implementation of this embodiment, the ETL module presets the interval of the HDFS space storage occupied by the desensitization source according to the desensitization rule; if the HDFS space storage occupied by the desensitization source is less than the interval, Create a view; if it is within the interval, create a materialized view; if it is greater than the interval, create a desensitization table.

After the ETL module checks whether the desensitization source and target fields are consistent, if the desensitization source and target fields are inconsistent, it will determine whether to add or delete fields according to the order of the target fields; if the desensitization source is newly added relative to the target field Field, the same field is added on the basis of the target field, and the previous desensitization table, materialized view, or view is replaced. If the desensitization source has a deleted field relative to the target field, it will be deleted on the basis of the target field The same field, and replace the previous desensitized table, materialized view or view.

The original library module 330 is used to store desensitization source data, and the desensitization library module 340 is used to store data such as desensitization tasks and target fields, combined with the DMP module 310 and the ETL module 320 to complete the desensitization tasks.

FIG. 4 is a schematic diagram of the electronic device of this application. In this embodiment, the electronic device 40 may be a terminal device with arithmetic functions such as a server, a tablet computer, a portable computer, a desktop computer, and the like.

The electronic device 40 includes a processor 41, a memory 42, a computer program 43, a network interface, and a communication bus.

The electronic device 40 may be a tablet computer, a desktop computer, or a smart phone, but is not limited thereto.

The memory 42 includes at least one type of readable storage medium. The at least one type of readable storage medium may be a non-volatile storage medium such as flash memory, hard disk, multimedia card, card-type memory, and the like. In some embodiments, the readable storage medium may be an internal storage unit of the electronic device 40, such as a hard disk of the electronic device 40. In other embodiments, the readable storage medium may also be an external memory of the electronic device 40, such as a plug-in hard disk equipped on the electronic device 40, a smart memory card (Smart Media Card, SMC), and a secure digital (Secure Digital, SD) card, flash card (Flash Card), etc.

In this embodiment, the readable storage medium of the memory 42 is generally used to store the computer program 43 installed in the electronic device 40, the key generation unit 310, the key management unit 320, the transmission unit 330, the alarm unit 340, and so on.

The processor 41 may be a central processing unit (CPU), microprocessor or other data processing chip in some embodiments, and is used to run the program code stored in the memory 42 or process data, such as data authorization desensitization Automation program 43 and so on.

The network interface may optionally include a standard wired interface and a wireless interface (such as a WI-FI interface), and is generally used to establish a communication connection between the electronic device 40 and other electronic devices.

The communication bus is used to realize the connection and communication between these components.

FIG. 4 only shows the electronic device 40 with the components 41-43, but it should be understood that it is not required to implement all the illustrated components, and more or fewer components may be implemented instead.

In the embodiment of the electronic device shown in FIG. 4, the memory 42 as a computer storage medium can store an operating system and a data authorization desensitization automation program 43; the processor 41 executes the data authorization desensitization automation program stored in the memory 42 At 43 o'clock, the following steps are implemented:

S140: Synchronize the specified desensitization rule and the data access range to ETL, the ETL generates a target field according to the data access range, and automatically generates a desensitization task for the target field according to the desensitization rule;

In addition, the embodiment of the present application also proposes a computer-readable storage medium. The computer-readable storage medium includes a data authorization and desensitization automation program, and when the data authorization and desensitization automation program is executed by a processor, the following operations are implemented:

The specific implementation of the computer-readable storage medium of the present application is substantially the same as the specific implementation of the above-mentioned data authorization and desensitization automation method and electronic device, and will not be repeated here.

It should be noted that in this article, the terms "include", "include" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, device, article or method including a series of elements not only includes those elements, It also includes other elements not explicitly listed, or elements inherent to the process, device, article, or method. If there are no more restrictions, the element defined by the sentence "including a..." does not exclude the existence of other identical elements in the process, device, article, or method that includes the element.

The serial numbers of the foregoing embodiments of the present application are only for description, and do not represent the advantages and disadvantages of the embodiments. Through the description of the above implementation manners, those skilled in the art can clearly understand that the above-mentioned embodiment method can be implemented by means of software plus the necessary general hardware platform, of course, it can also be implemented by hardware, but in many cases the former is better.的实施方式。 Based on this understanding, the technical solution of this application essentially or the part that contributes to the existing technology can be embodied in the form of a software product, and the computer software product is stored in a storage medium (such as ROM/RAM) as described above. , Magnetic disks, optical disks), including several instructions to make a terminal device (which may be a computer, a server, or a network device, etc.) execute the method described in each embodiment of the present application.

The above are only the preferred embodiments of the application, and do not limit the scope of the patent for this application. Any equivalent structure or equivalent process transformation made using the content of the description and drawings of the application, or directly or indirectly applied to other related technical fields , The same reason is included in the scope of patent protection of this application.

Claims

An automated method for data authorization desensitization, applied to an electronic device, characterized in that the method includes:

S110: Create a desensitization requirement according to a preset rule, and determine table access authority and data access scope according to the desensitization requirement;

S120: Initiate an application for permission to access the service table in the DMP according to the table access permission and data access scope, and the DMP is used to approve the permission application;

S130: Perform desensitization configuration on the approved permission application through the DMP, where the desensitization configuration includes mandatory configuration designated desensitization rules for sensitive fields;

S140: Synchronize the specified desensitization rule and the data access range to ETL, the ETL generates a target field according to the data access range, and automatically generates a desensitization task for the target field according to the desensitization rule;

S150: Check whether the desensitization source and target fields are the same. If they are the same, continue to keep the desensitization source and target fields and sequence consistent; if they are inconsistent, make the desensitization source and target fields consistent. After the check is completed, execute ETL and complete the description Desensitization task.
The data authorization and desensitization automation method according to claim 1, wherein the desensitization requirement is created by a data provider according to the preset rule; the specific desensitization field of the desensitization requirement includes business sensitive fields;

The data access range includes at least a time range and a span range.
The data authorization and desensitization automation method according to claim 1, wherein the DMP initiates an application for permission to access the business table according to the table access permission and the data access range, and the DMP is used to perform the permission application The approval steps include:

The data user initiates an application for permission to access the business table;

The criteria for determining that the desensitization requirements of the data user meet the data use requirements include at least not damaging the desensitization source and not containing out-of-range instructions.
The automated method for data authorization desensitization according to claim 3, wherein:

The data desensitization source is data that is initially screened by the data user and waiting to be desensitized;

The target field is accurate desensitization data that is immediately desensitized.
The data authorization desensitization automation method according to claim 4, wherein:

Both the data desensitization source and the target field are stored in Hive in the form of a table.
The data authorization desensitization automation method according to claim 1, wherein the desensitization method corresponding to the desensitization rule includes direct desensitization and associated desensitization; wherein,

The direct desensitization is direct desensitization by applying a preset rule to the desensitization source, and has nothing to do with other fields;

The association desensitization is to apply the same desensitization rule to the associated fields between desensitization sources, so that the desensitized fields can still be associated.
The data authorization and desensitization automation method according to claim 6, wherein the preset rule includes at least one of the following rules:

The HASH encryption name provided for any character string is uniformly changed to "the name of the data user";

Change the last 4 digits to 1234 or shield the middle 4 digits;

All mailboxes are shielded as the preset mailboxes of the data user;

The amount category randomly floats 1% to 5%, and the address category intercepts the first 15 characters or displays only up to the level of the district.
The data authorization desensitization automation method according to claim 1, wherein the process of automatically generating the desensitization task of the target field according to the desensitization rule comprises:

Determine whether to create a desensitized table, view or materialized view according to the desensitization rule;

If a desensitization table is created, the desensitization table is established according to the fields of the desensitization source; if a view and a materialized view are created, a new view is generated according to the desensitization rule and the previous view is replaced.
The data authorization desensitization automation method according to claim 8, wherein the process of determining whether to establish a desensitization table, a materialized view or a view according to the desensitization rule comprises:

Pre-setting the interval of the HDFS space storage occupied by the desensitization source according to the desensitization rule;

If the desensitization source occupies less than the interval, the view is created; if it is in the interval, the materialized view is created; if it is greater than the interval, the desensitization table is created.
The data authorization desensitization automation method according to claim 8, wherein after the step of checking whether the desensitization source and target fields are consistent, the method further comprises:

If the desensitization source and target fields are inconsistent, judge whether to add a field or delete a field according to the order of the target field;

If the desensitization source has a new field relative to the target field, add the same field based on the target field and replace the previous desensitization table, materialized view or view. If the desensitization source has a deleted field relative to the target field, Then the same field is deleted on the basis of the target field, and the previous desensitized table, materialized view or view is replaced.
A data authorization desensitization automation system, which is characterized by comprising a DMP module, an ETL module, an original library module and a desensitization library module, wherein,

The DMP module is used to accept the permission access application for accessing the business table initiated by the data user according to the table access permission and data access scope, and to approve the permission according to the permission application. If the approval is passed, the DMP module starts to disconnect Sensitive configuration, the desensitization configuration includes mandatory configuration of designated desensitization rules for sensitive fields, and synchronization of the desensitization rules to the ETL module;

The ETL module is configured to generate a target field according to the data access range, automatically generate a desensitization task for the target field according to the desensitization rule, and execute the desensitization task; and, the ETL module is executing all the desensitization tasks. Before describing the desensitization task, check whether the desensitization source and target fields are consistent. If they are the same, continue to keep the desensitization source and target fields and sequence consistent; if they are inconsistent, make the desensitization source and target fields consistent. Describe the desensitization task;

The original library module is used to store the desensitization source;

The desensitization library module is used to store target fields and desensitization tasks.
The data authorization desensitization automated system according to claim 11, wherein:

The data access range includes at least a time range and a span range.
The data authorization desensitization automated system according to claim 11, wherein:

The criteria for the DMP module to approve the permission application include: not destroying the desensitization source, and not containing out-of-range instructions.
The data authorization desensitization automated system according to claim 11, wherein:

The desensitization methods corresponding to the desensitization rules include direct desensitization and associated desensitization; wherein,

The direct desensitization is direct desensitization by applying a preset rule to the desensitization source, and has nothing to do with other fields;

The association desensitization is to apply the same desensitization rule to the associated fields between desensitization sources, so that the desensitized fields can still be associated.
The data authorization desensitization automation system according to claim 14, wherein:

The preset rule includes at least one of the following rules:

The HASH encryption name provided for any character string is uniformly changed to "the name of the data user";

Change the last 4 digits to 1234 or shield the middle 4 digits;

All mailboxes are shielded as the preset mailboxes of the data user;

The amount category randomly floats 1% to 5%, and the address category intercepts the first 15 characters or displays only up to the level of the district.
The data authorization desensitization automation system according to claim 11, wherein the ETL module automatically generates the desensitization task of the target field according to the desensitization rule,

Determine whether to create a desensitized table, view or materialized view according to the desensitization rule;

If a desensitization table is created, the desensitization table is established according to the fields of the desensitization source; if a view and a materialized view are created, a new view is generated according to the desensitization rule and the previous view is replaced.
The data authorization desensitization automation system according to claim 16, wherein the ETL module presets the interval of the HDFS space storage occupied by the desensitization source according to the desensitization rule;

If the desensitization source occupies less than the interval, the view is created; if it is in the interval, the materialized view is created; if it is greater than the interval, the desensitization table is created.
The data authorization desensitization automation system according to claim 11, wherein the ETL module checks whether the desensitization source and target fields are consistent,

If the desensitization source and target fields are inconsistent, judge whether to add a field or delete a field according to the order of the target field;

If the desensitization source has a new field relative to the target field, add the same field based on the target field and replace the previous desensitization table, materialized view or view. If the desensitization source has a deleted field relative to the target field, Then the same field is deleted on the basis of the target field, and the previous desensitized table, materialized view or view is replaced.
An electronic device, characterized in that the electronic device comprises: a memory, a processor, and a computer program stored in the memory and running on the processor, and the processor realizes the rights when the computer program is executed. The steps in the data authorization desensitization automated method described in any one of 1-10 are required.
A computer-readable storage medium, characterized in that, a data authorization desensitization automated analysis program is stored in the computer-readable storage medium, and when the data-based authorization desensitization automated analysis program is executed by a processor, the implementation is as claimed in the claims The steps of the data authorization desensitization automated method described in any one of 1 to 10.