CN113204791A - Desensitization method and desensitization device for desensitization data to be recovered - Google Patents
Desensitization method and desensitization device for desensitization data to be recovered Download PDFInfo
- Publication number
- CN113204791A CN113204791A CN202110592653.3A CN202110592653A CN113204791A CN 113204791 A CN113204791 A CN 113204791A CN 202110592653 A CN202110592653 A CN 202110592653A CN 113204791 A CN113204791 A CN 113204791A
- Authority
- CN
- China
- Prior art keywords
- data
- desensitization
- sensitive
- data recovery
- recovered
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000586 desensitisation Methods 0.000 title claims abstract description 186
- 238000000034 method Methods 0.000 title claims abstract description 65
- 238000011084 recovery Methods 0.000 claims abstract description 166
- 238000013515 script Methods 0.000 claims abstract description 54
- 238000012795 verification Methods 0.000 claims description 46
- 238000012545 processing Methods 0.000 claims description 12
- 238000004590 computer program Methods 0.000 claims description 11
- 239000000284 extract Substances 0.000 claims description 9
- 238000003860 storage Methods 0.000 claims description 6
- 238000004891 communication Methods 0.000 claims description 5
- 230000008569 process Effects 0.000 abstract description 21
- 238000005516 engineering process Methods 0.000 abstract description 5
- 230000002349 favourable effect Effects 0.000 abstract description 4
- 238000004519 manufacturing process Methods 0.000 description 16
- 238000012360 testing method Methods 0.000 description 14
- 238000010586 diagram Methods 0.000 description 12
- 238000012827 research and development Methods 0.000 description 8
- 244000035744 Hura crepitans Species 0.000 description 7
- 238000011161 development Methods 0.000 description 4
- 230000018109 developmental process Effects 0.000 description 4
- 230000005540 biological transmission Effects 0.000 description 3
- 238000004422 calculation algorithm Methods 0.000 description 3
- 238000013507 mapping Methods 0.000 description 3
- 238000013075 data extraction Methods 0.000 description 2
- 238000013524 data verification Methods 0.000 description 2
- 238000000605 extraction Methods 0.000 description 2
- 230000006870 function Effects 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 238000005192 partition Methods 0.000 description 2
- 230000000750 progressive effect Effects 0.000 description 2
- 230000001737 promoting effect Effects 0.000 description 2
- 230000001681 protective effect Effects 0.000 description 2
- 230000002159 abnormal effect Effects 0.000 description 1
- 230000032683 aging Effects 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000008901 benefit Effects 0.000 description 1
- 238000004364 calculation method Methods 0.000 description 1
- 238000004140 cleaning Methods 0.000 description 1
- 230000002860 competitive effect Effects 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 230000010354 integration Effects 0.000 description 1
- 230000003993 interaction Effects 0.000 description 1
- 238000002955 isolation Methods 0.000 description 1
- 238000007726 management method Methods 0.000 description 1
- 239000000463 material Substances 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 238000002360 preparation method Methods 0.000 description 1
- 230000010076 replication Effects 0.000 description 1
- 238000011160 research Methods 0.000 description 1
- 238000012549 training Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F21/00—Security arrangements for protecting computers, components thereof, programs or data against unauthorised activity
- G06F21/60—Protecting data
- G06F21/62—Protecting access to data via a platform, e.g. using keys or access control rules
- G06F21/6218—Protecting access to data via a platform, e.g. using keys or access control rules to a system of files or objects, e.g. local or distributed file system or database
- G06F21/6245—Protecting personal data, e.g. for financial or medical purposes
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/07—Responding to the occurrence of a fault, e.g. fault tolerance
- G06F11/14—Error detection or correction of the data by redundancy in operation
- G06F11/1402—Saving, restoring, recovering or retrying
- G06F11/1446—Point-in-time backing up or restoration of persistent data
- G06F11/1448—Management of the data involved in backup or backup restore
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/36—Preventing errors by testing or debugging software
- G06F11/3664—Environments for testing or debugging software
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F21/00—Security arrangements for protecting computers, components thereof, programs or data against unauthorised activity
- G06F21/60—Protecting data
- G06F21/602—Providing cryptographic facilities or services
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q40/00—Finance; Insurance; Tax strategies; Processing of corporate or income taxes
- G06Q40/02—Banking, e.g. interest calculation or account maintenance
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- Computer Hardware Design (AREA)
- Health & Medical Sciences (AREA)
- General Health & Medical Sciences (AREA)
- Bioethics (AREA)
- Business, Economics & Management (AREA)
- Finance (AREA)
- Computer Security & Cryptography (AREA)
- Software Systems (AREA)
- Quality & Reliability (AREA)
- Accounting & Taxation (AREA)
- Medical Informatics (AREA)
- Development Economics (AREA)
- Economics (AREA)
- Marketing (AREA)
- Strategic Management (AREA)
- Technology Law (AREA)
- General Business, Economics & Management (AREA)
- Databases & Information Systems (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The invention provides a desensitization method and a desensitization device for sensitive data to be recovered, and relates to the fields of big data security technology and finance. The method comprises the following steps: creating a corresponding data recovery task according to the received data recovery request; acquiring sensitive data to be restored according to the data restoration request and backing up the sensitive data to be restored; and desensitizing the backed-up sensitive data to be restored according to the data restoration task and the data desensitization script corresponding to the sensitive data to be restored. The desensitization method and the desensitization device for the sensitive data to be recovered solve the problems that the existing data desensitization process is complicated in process, low in efficiency, high in cost and the like due to excessive manual intervention, are favorable for improving the automation and safe controllable level in the large data recovery desensitization implementation process of a commercial bank, and improve the sensitive data protection capability.
Description
Technical Field
The invention relates to the field of big data security technology and finance, in particular to a desensitization method and a desensitization device for sensitive data to be recovered.
Background
The financial industry is an industry with highly concentrated sensitive data, and grasps and stores a large amount of sensitive information (such as basic information of client names, certificate numbers and the like, and transaction information) of enterprises and individual clients. With the deep integration of finance and science and technology, commercial banks accelerate the development of building a big data platform and a related data application system, and data assets are multiplied. On one hand, the bank deeply excavates and utilizes the data value, and the data is widely applied to the aspects of system research and development, test, application innovation and the like, so that the data really becomes a production element for driving business innovation, and the competitive leading advantage is obtained; on the other hand, the current domestic and foreign protection requirements on data security or information security are increasingly fine and strict, and the loss of sensitive information or data leakage to banks under high-voltage situations is higher and higher. Desensitizing the production of sensitive data is an effective method in the industry to secure the sensitive data.
However, the current data desensitization method has the problems of excessive manual intervention, tedious process, low efficiency, high cost and the like. How to create a method which can meet the requirements of large-scale use and production data such as research, development and test or application innovation of a bank system and the like and can effectively guarantee the security of sensitive data is a technical problem to be solved urgently by a commercial bank.
Disclosure of Invention
In order to solve the problems in the prior art, the application provides a desensitization method for sensitive data to be recovered, and relates to the fields of big data security technology and finance. The desensitization method for the sensitive data to be recovered comprises the following steps:
creating a corresponding data recovery task according to the received data recovery request;
acquiring sensitive data to be restored according to the data restoration request and backing up the sensitive data to be restored;
and desensitizing the backed-up sensitive data to be restored according to the data restoration task and the data desensitization script corresponding to the sensitive data to be restored.
In an embodiment, before creating the corresponding data recovery task according to the received data recovery request, the method further includes:
determining corresponding sensitive data to be recovered according to the data recovery request;
judging whether the information of the sensitive data to be recovered is consistent with the corresponding information in the data recovery request and whether the data recovery request is a repeated request; and if the information of the sensitive data to be recovered is consistent with the corresponding information in the data recovery request and the data recovery request is not a repeated request, creating a corresponding data recovery task according to the received data recovery request.
In an embodiment, the creating a corresponding data recovery task according to the received data recovery request includes:
determining an execution condition of the data recovery task according to a data recovery condition in the data recovery request, wherein the execution condition comprises: job trigger conditions, job priorities and job concurrency;
performing rationality verification on the execution conditions according to all data recovery tasks to be executed and the system resource occupation state;
and when the verification is passed, creating the data recovery task according to the sensitive data to be recovered and the execution condition.
In an embodiment, desensitizing the backed-up sensitive data to be restored according to the data restoring task and the data desensitizing script corresponding to the sensitive data to be restored includes:
when the execution condition is met, judging whether the backed-up sensitive data to be restored is matched with the data desensitization script or not;
and if so, calling the data desensitization script to identify and extract the sensitive information in the backed-up desensitization data to be restored to obtain desensitization data.
In an embodiment, after the invoking the data desensitization script to identify and extract sensitive information in the backed-up sensitive data to be restored, the method further includes:
calling a preset data desensitization verification script to verify the accuracy of the desensitization data;
and when the verification is passed, naming the desensitization data according to the naming rule of the desensitization data to be recovered.
The application also discloses a desensitization device for recovering sensitive data, which is used for realizing the desensitization method for sensitive data of the application, and the device comprises:
the data recovery task creating module is used for creating a corresponding data recovery task according to the received data recovery request;
the data backup module is used for acquiring the sensitive data to be recovered according to the data recovery request and backing up the sensitive data to be recovered;
and the data desensitization module is used for desensitizing the backed-up sensitive data to be restored according to the data restoration task and the data desensitization script corresponding to the sensitive data to be restored.
In an embodiment, the desensitization apparatus to recover sensitive data further includes:
the to-be-recovered sensitive data acquisition module is used for determining corresponding to-be-recovered sensitive data according to the data recovery request;
the data recovery request verification module is used for judging whether the information of the sensitive data to be recovered is consistent with the corresponding information in the data recovery request and whether the data recovery request is a repeated request; and if the information of the sensitive data to be recovered is consistent with the corresponding information in the data recovery request and the data recovery request is not a repeated request, creating a corresponding data recovery task according to the received data recovery request.
In one embodiment, the data recovery task creation module includes:
an execution condition determining unit, configured to determine an execution condition of the data recovery task according to a data recovery condition in the data recovery request, where the execution condition includes: job trigger conditions, job priorities and job concurrency;
the execution condition verification unit is used for verifying the rationality of the execution conditions according to all data recovery tasks to be executed and the occupation state of system resources;
and the data recovery task creating unit is used for creating the data recovery task according to the sensitive data to be recovered and the execution condition when the verification is passed.
In one embodiment, the data desensitization module comprises:
the script verification unit is used for judging whether the backed-up sensitive data to be restored is matched with the data desensitization script or not when the execution condition is met;
and the data desensitization unit is used for calling the data desensitization script to identify and extract the sensitive information in the backed-up sensitive data to be restored to obtain desensitized data when the sensitive data to be restored is matched with the data desensitization script.
In an embodiment, the desensitization apparatus to recover sensitive data further includes an accuracy verification module, configured to:
calling a preset data desensitization verification script to verify the accuracy of the desensitization data;
and when the verification is passed, naming the desensitization data according to the naming rule of the desensitization data to be recovered.
The desensitization method of waiting to resume sensitive data and the desensitization device of waiting to resume sensitive data that this application provided can effectively satisfy large-scale use production data demands such as commercial bank research and development test or application innovation, solved present data desensitization process face manual intervention too much, the process is loaded down with trivial details, inefficiency, cost are higher wait to solve the problem urgently, be favorable to promoting automation, safe controllable level among the commercial bank big data recovery desensitization implementation process, promote sensitive data protective capability.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below, it is obvious that the drawings in the following description are only some embodiments of the present invention, and for those skilled in the art, other drawings can be obtained according to the drawings without creative efforts.
Fig. 1 is a schematic diagram of a desensitization method of the present application for recovering sensitive data.
Fig. 2 is another schematic diagram of the desensitization method of the present application for recovering sensitive data.
Fig. 3 is another schematic diagram of the desensitization method of the present application for recovering sensitive data.
Fig. 4 is another schematic diagram of the desensitization method of the present application for recovering sensitive data.
FIG. 5 is a diagram illustrating the determination of whether the desensitization script matches the desensitization data to be recovered.
Fig. 6 is another schematic diagram of the desensitization method of the present application for recovering sensitive data.
Fig. 7 is a schematic diagram of a desensitization device of the present application for recovering sensitive data.
Fig. 8 is a schematic diagram of a desensitization device of the present application for recovering sensitive data.
Fig. 9 is a schematic diagram of a desensitization device of the present application for recovering sensitive data.
Fig. 10 is a schematic diagram of a desensitization device of the present application for recovering sensitive data.
Fig. 11 is a schematic diagram of a desensitization device of the present application for recovering sensitive data.
Fig. 12 is a schematic diagram of an electronic device.
Detailed Description
The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
As shown in fig. 1, the present application provides a desensitization method for sensitive data to be recovered, the method comprising the steps of:
step S101, a corresponding data recovery task is created according to the received data recovery request.
And the data recovery request is provided by a worker according to the actual requirement. The data recovery request typically includes information such as the purpose of the data recovery, the type of data recovery, the content of the recovered data, the conditions of the data recovery, and the items involved. The data recovery type is, for example, full library recovery, single table recovery or partition extraction recovery, and the data recovery condition is, for example, a data backup time requirement.
After receiving the data recovery request, creating a corresponding data recovery task according to information such as a data recovery purpose, a data recovery type, recovered data content, data recovery conditions, related items and the like in the data recovery request, wherein the information mainly comprises configuration data recovery parameters and the like.
And S102, acquiring sensitive data to be recovered according to the data recovery request and backing up the sensitive data to be recovered.
Specifically, data to be desensitized in the production environment is backed up into the desensitization implementation environment according to the data recovery request. When the data to be desensitized is backed up, the actual requirements of data recovery application are combined, the factors of system service processing, resource use and the like are comprehensively considered, and batch data synchronization tools such as service system replication and the like of a large data platform can be used to complete the backup of the data to be desensitized. During backup, the conditions of operation, resource occupation, service processing and the like of the service system in the process of data backup to be desensitized are monitored in the whole process, and the normal use of the service system is not influenced.
And S103, desensitizing the backed-up sensitive data to be restored according to the data restoration task and the data desensitization script corresponding to the sensitive data to be restored.
Specifically, sensitive information in the sensitive data to be recovered is identified, and the type of the sensitive information is identified. Generally, the identification and type identification of sensitive information can be realized through a preset rule and a sensitive information intelligent identification model obtained through a large amount of sensitive data training. A data desensitization script is then automatically generated based on the type identification and the script format defined by the desensitization tool. The desensitization tool calls the data desensitization script to realize desensitization of the sensitive data to be recovered.
The desensitization tool also performs quality verification on the generated data desensitization script to ensure the quality and accuracy of the data desensitization script.
The desensitization tool of the application is mainly designed and developed based on a data desensitization strategy of a bank. When a bank formulates a data desensitization strategy, a desensitization range, sensitive information classification, service test requirements, an information protection effect and the like need to be comprehensively considered, and different algorithms such as random mapping or drifting, key modulus weighting calculation, fixed value assignment and the like are respectively adopted, for example: the test process often requires maintaining diversity of sensitive data such as client names, certificate numbers and the like, which requires adopting a random mapping or drift algorithm to solve, and ensuring that the data after desensitization can maintain diversity, uniqueness and irreversibility. Meanwhile, different algorithms may use keys or other mapping relation tables to perform encryption processing, perform related processing such as decryption when data desensitization is implemented, automatically clear related information after implementation is completed, and ensure safety in the implementation process.
The desensitization method for sensitive data to be recovered can effectively meet the requirements of large-scale use and production data of commercial bank research and development tests or application innovation and the like, solves the problems of excessive manual intervention, complex process, low efficiency, high cost and the like in the current data desensitization process, is favorable for improving the automation and safety controllable level in the large data recovery desensitization implementation process of the commercial bank, and improves the sensitive data protection capability.
In an embodiment, as shown in fig. 2, before creating the corresponding data recovery task according to the received data recovery request, the method further includes:
step S201, determining corresponding sensitive data to be recovered according to the data recovery request and performing data verification.
Specifically, the sensitive data to be recovered is determined according to the data recovery content in the data recovery request, and data verification is performed on the data recovery content and the sensitive data to be recovered corresponding to the data recovery content.
Step S202, judging whether the data recovery request passes the verification; specifically, the checking includes determining whether the information of the sensitive data to be recovered is consistent with the corresponding information in the data recovery request and whether the data recovery request is a repeat request;
wherein, judging whether the information of the sensitive data to be recovered is consistent with the corresponding information in the data recovery request comprises: and verifying whether the data recovery content is consistent with the sensitive data to be recovered and the structure information of the software asset management platform registration table or not, whether the desensitization type is identified or not and the like through the database name/table English name. Determining whether the data recovery request is a repeat request comprises: and (4) checking the repeatability of the data recovery request in combination with historical data recovery records to fully utilize the recovered sensitive data, avoid repeated recovery and reduce the workload of the system.
In step S202, if the verification passes, that is, the information of the sensitive data to be recovered is consistent with the corresponding information in the data recovery request, and the data recovery request is not a repeat request, then step S101 is executed;
if the verification fails, that is, the information of the sensitive data to be recovered is inconsistent with the corresponding information in the data recovery request and/or the data recovery request is a repeat request, executing step S203;
step S203, reject the data recovery request.
In an embodiment, as shown in fig. 3, step S101, creating a corresponding data recovery task according to a received data recovery request specifically includes the following steps:
step S1011, determining an execution condition of the data recovery task according to a data recovery condition in the data recovery request, where the execution condition includes: job trigger conditions, job priority, and job concurrency.
Firstly, a data recovery list is generated according to a data recovery request, the list comprises a data recovery task to be executed, then execution conditions of the data recovery task are set, wherein the job trigger conditions comprise conditions such as job running date and data backup date, and the job trigger conditions are mainly determined according to information such as data backup time requirements in the data recovery conditions.
And after the job triggering condition is configured, configuring the job priority and the job concurrency according to the job triggering condition.
Step S1012, determining whether the rationality verification of the execution condition passes, where the rationality verification is to perform the rationality verification on the execution condition according to all data recovery tasks to be executed and the system resource occupation condition.
Specifically, the execution conditions (particularly, job execution date and data backup date) of the data recovery task of the whole business system, and the resource occupation conditions of the business system and the network are comprehensively considered, so that the reasonability verification is performed on the job trigger condition, the job priority and the job concurrency.
When the rationality verification passes, step S1013 is executed; when the rationality verification does not pass, step S1014 is executed.
And S1013, creating the data recovery task according to the sensitive data to be recovered and the execution conditions, wherein at this time, the data recovery task corresponding to the data recovery request is formally created successfully.
Step 1014, adjusting the execution condition to make it not affect the normal operation of the service system. For example, the concurrency of the data recovery task is integrally controlled, the super-large-scale data table is further split according to the preset splitting requirement, and the like.
In an embodiment, as shown in fig. 4, in step S103, desensitizing the backed-up sensitive data to be restored according to the data restoring task and the data desensitizing script corresponding to the sensitive data to be restored, specifically including the following steps:
and step S1031, when the execution condition is met, judging whether the backed-up sensitive data to be restored is matched with the data desensitization script or not.
In step S1031, the data recovery task in the data recovery list is first analyzed to obtain the desensitization data to be recovered and the corresponding data desensitization script, and it is determined whether the data desensitization script matches the desensitization data to be recovered.
Specifically, as shown in fig. 5, the operation step of determining matching includes:
acquiring information of sensitive data to be restored, which is backed up to a production environment, including information of a hadoop cluster library, a table structure, a field, a data partition and the like;
acquiring information of a data desensitization script, wherein the information comprises script information such as field information of a table to be desensitized, sensitive information classification and desensitization strategies;
comparing whether the information of the data desensitization script is consistent with the information of the data desensitization script to be restored in the production environment or not;
if the data is consistent with the data desensitization script, indicating that the desensitization data to be recovered is matched with the data desensitization script, and continuing to execute the step S1032; if the data desensitization scripts are inconsistent with the data desensitization data, it is indicated that the desensitization data to be recovered are not matched with the data desensitization scripts, and at the moment, early warning prompting, data recovery task interruption or other processing aiming at abnormal conditions are carried out.
And S1032, calling the data desensitization script to identify and extract the sensitive information in the backed-up desensitization data to be restored to obtain desensitization data.
Specifically, a data-based desensitization tool automatically generates an HQL program which can be executed in a production environment of a big data platform and conforms to the running characteristics of big data by combining the information of the sensitive data to be restored and the information of the corresponding data desensitization script, and the HQL program is specifically executed by a table creation extraction desensitization statement.
And then, executing the HQL program, performing online data extraction and desensitization operation on the desensitization data to be recovered in the production environment backed up to the big data platform, and recovering the desensitization data to the sandbox environment of the big data platform.
The sandbox environment of the big data platform is mainly constructed to simplify a data interaction flow between production and research and development test environments, reduce mutual influence between data extraction and batch aging of the production environments, improve the overall efficiency of research and development test environment data preparation and related applications, and support gray scale verification and release. When a sandbox environment is built, a distributed cluster resource isolation technology is adopted, an independent sandbox resource area is built in a backup cluster of a large data platform production environment, sandbox environment resources are independent, and use of other users is not influenced. In the application, the sandbox environment is mainly used for storing desensitized data and data recovery desensitization logs and carrying out work such as desensitization result verification.
In an embodiment, as shown in fig. 6, after the step S103 of invoking the data desensitization script to identify and extract sensitive information in the backed-up sensitive data to be restored, the method further includes:
step S104, calling a preset data desensitization verification script to carry out accuracy verification on desensitization data;
specifically, the accuracy of the desensitization result is verified by using a data desensitization verification tool, so that the quality of desensitization data is ensured. For example, sensitive information identification is performed on desensitization data, and whether the desensitization data further includes sensitive information is judged.
And step S105, naming the desensitization data according to the naming rule of the desensitization data to be recovered when the verification is passed, namely when the desensitization data does not contain sensitive information.
Specifically, the desensitization data are named according to the original library name _ original table name _ user name _ specific identification naming rule with the desensitization data recovery function, so that the desensitization data can be automatically uploaded to a test library in the following process.
In the application, after desensitization data are obtained, the desensitization data are automatically fed back to a user side which provides a data recovery request. In addition, desensitization data has other applications, such as: and carrying out HDFS file transmission work by utilizing a Hadoop cluster distcp transmission tool of the big data platform, and transmitting desensitization data to a development environment and/or a test environment cluster corresponding to the data recovery request. In order to avoid occupying too much network resources, the network bandwidth is often limited by configuring parameters according to factors such as the size of transmission data, the available bandwidth and the like.
And then, according to the naming rule (original library name _ original table name _ user name _ specific identification) of the desensitization data, automatically analyzing, identifying and splitting information of the desensitization data, such as library names, table names, user names and the like by using an analysis program, and automatically loading the desensitization data into corresponding research and development environments and/or test environment clusters.
After the desensitization data are transmitted to the development environment or the test environment of the big data platform, data cleaning operation is started, and the desensitization data in the sandbox environment of the big data platform are cleaned, so that the storage space of the production environment is saved.
The desensitization method for the sensitive data to be recovered can effectively meet the requirements of large-scale use and production data of research and development tests or application innovation of commercial banks, solves the problems that the existing data desensitization process is too much in manual intervention, complex in process, low in efficiency, high in cost and the like, is urgent, is beneficial to improving the automation and safety controllable level in the implementation process of large data recovery desensitization of commercial banks, and improves the protection capability of sensitive data.
Based on the same inventive concept, the embodiment of the present application further provides a desensitizing device for recovering sensitive data, which can be used to implement the method described in the above embodiments, as described in the following embodiments. Because the principle of solving the problem of the desensitization device for the sensitive data to be recovered is similar to that of the desensitization method for the sensitive data to be recovered, the implementation of the desensitization device for the sensitive data to be recovered can refer to the implementation of the desensitization method for the sensitive data to be recovered, and repeated parts are not described in detail. As used hereinafter, the term "unit" or "module" may be a combination of software and/or hardware that implements a predetermined function. While the system described in the embodiments below is preferably implemented in software, implementations in hardware, or a combination of software and hardware are also possible and contemplated.
As shown in fig. 7, the desensitization apparatus to recover sensitive data of the present application includes:
a data recovery task creating module 601, configured to create a corresponding data recovery task according to the received data recovery request;
a data backup module 602, configured to obtain, according to the data recovery request, sensitive data to be recovered and backup the sensitive data to be recovered;
and the data desensitization module 603 is configured to desensitize the backed-up sensitive data to be restored according to the data restoration task and the data desensitization script corresponding to the sensitive data to be restored.
In an embodiment, as shown in fig. 8, the desensitization apparatus to recover sensitive data further includes:
the to-be-recovered sensitive data acquisition module 701 is configured to determine, according to the data recovery request, corresponding to-be-recovered sensitive data;
a data recovery request verification module 702, configured to determine whether the information of the sensitive data to be recovered is consistent with corresponding information in the data recovery request and whether the data recovery request is a repeat request; and if the information of the sensitive data to be recovered is consistent with the corresponding information in the data recovery request and the data recovery request is not a repeated request, creating a corresponding data recovery task according to the received data recovery request.
In one embodiment, as shown in fig. 9, the data recovery task creating module 601 includes:
an execution condition determining unit 6011, configured to determine an execution condition of the data recovery task according to a data recovery condition in the data recovery request, where the execution condition includes: job trigger conditions, job priorities and job concurrency;
an execution condition verification unit 6012, configured to perform rationality verification on the execution condition according to all data recovery tasks to be executed and a system resource occupation state;
a data recovery task creating unit 6013, configured to create, when the verification passes, the data recovery task according to the sensitive data to be recovered and the execution condition.
In one embodiment, as shown in FIG. 10, the data desensitization module 603 includes:
a script verification unit 6031, configured to determine whether the backed-up desensitized data to be restored matches the data desensitized script when the execution condition is satisfied;
and a data desensitization unit 6032, configured to, when the to-be-restored sensitive data matches the data desensitization script, call the data desensitization script to identify and extract the backed-up sensitive information in the to-be-restored sensitive data, to obtain desensitized data.
In an embodiment, as shown in fig. 11, the desensitization apparatus to recover sensitive data further includes an accuracy verification module 604 for:
calling a preset data desensitization verification script to verify the accuracy of the desensitization data;
and when the verification is passed, naming the desensitization data according to the naming rule of the desensitization data to be recovered.
The desensitization device of waiting to resume sensitive data that this application provided can effectively satisfy large-scale use production data demands such as commercial bank research and development test or application innovation, has solved present data desensitization process and has faced manual intervention too much, the process is loaded down with trivial details, inefficiency, the higher waiting problem of solving of cost, is favorable to promoting automation, safe controllable level among the big data recovery desensitization implementation process of commercial bank, promotes sensitive data protective capability.
The present invention further provides an electronic device including the desensitizing apparatus for desensitizing sensitive data to be recovered in the foregoing embodiment, referring to fig. 12, the electronic device 100 specifically includes:
a central processing unit (processor)110, a memory (memory)120, a communication module (Communications)130, an input unit 140, an output unit 150, and a power supply 160.
The memory (memory)120, the communication module (Communications)130, the input unit 140, the output unit 150 and the power supply 160 are respectively connected to the central processing unit (processor) 110. The memory 120 stores a computer program, the central processing unit 110 can call the computer program, and the central processing unit 110 executes the computer program to implement all the steps of the desensitization method for recovering sensitive data in the above embodiments.
Embodiments of the present application also provide a computer storage medium for storing a computer program executable by a processor. The computer program is used for realizing any desensitization method for recovering sensitive data provided by the invention when being executed by a processor.
The embodiments in the present specification are described in a progressive manner, and the same and similar parts among the embodiments are referred to each other, and each embodiment focuses on the differences from the other embodiments. In particular, for the hardware + program class embodiment, since it is substantially similar to the method embodiment, the description is simple, and the relevant points can be referred to the partial description of the method embodiment.
As will be appreciated by one skilled in the art, embodiments of the present description may be provided as a method, system, or computer program product. Accordingly, embodiments of the present description may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, embodiments of the present description may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and so forth) having computer-usable program code embodied therein. The embodiments in the present specification are described in a progressive manner, and the same and similar parts among the embodiments are referred to each other, and each embodiment focuses on the differences from the other embodiments. In particular, for the system embodiment, since it is substantially similar to the method embodiment, the description is simple, and for the relevant points, reference may be made to the partial description of the method embodiment. In the description herein, references to the description of the term "one embodiment," "some embodiments," "an example," "a specific example," or "some examples," etc., mean that a particular feature, structure, material, or characteristic described in connection with the embodiment or example is included in at least one embodiment or example of an embodiment of the specification.
In this specification, the schematic representations of the terms used above are not necessarily intended to refer to the same embodiment or example. Furthermore, various embodiments or examples and features of different embodiments or examples described in this specification can be combined and combined by one skilled in the art without contradiction. The above description is only an example of the embodiments of the present disclosure, and is not intended to limit the embodiments of the present disclosure. Various modifications and variations to the embodiments described herein will be apparent to those skilled in the art. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the embodiments of the present specification should be included in the scope of the claims of the embodiments of the present specification.
Claims (12)
1. A method of desensitizing sensitive data to be recovered, comprising:
creating a corresponding data recovery task according to the received data recovery request;
acquiring sensitive data to be restored according to the data restoration request and backing up the sensitive data to be restored;
and desensitizing the backed-up sensitive data to be restored according to the data restoration task and the data desensitization script corresponding to the sensitive data to be restored.
2. The desensitization method according to claim 1, wherein before creating the corresponding data recovery task according to the received data recovery request, further comprising:
determining corresponding sensitive data to be recovered according to the data recovery request;
judging whether the information of the sensitive data to be recovered is consistent with the corresponding information in the data recovery request and whether the data recovery request is a repeated request; and if the information of the sensitive data to be recovered is consistent with the corresponding information in the data recovery request and the data recovery request is not a repeated request, creating a corresponding data recovery task according to the received data recovery request.
3. The desensitization method according to claim 2, wherein said creating a corresponding data recovery task according to a received data recovery request comprises:
determining an execution condition of the data recovery task according to a data recovery condition in the data recovery request, wherein the execution condition comprises: job trigger conditions, job priorities and job concurrency;
performing rationality verification on the execution conditions according to all data recovery tasks to be executed and the system resource occupation state;
and when the verification is passed, creating the data recovery task according to the sensitive data to be recovered and the execution condition.
4. The desensitization method of the sensitive data to be restored according to claim 3, wherein the desensitizing the backed-up sensitive data to be restored according to the data restoration task and the data desensitization script corresponding to the sensitive data to be restored comprises:
when the execution condition is met, judging whether the backed-up sensitive data to be restored is matched with the data desensitization script or not;
and if so, calling the data desensitization script to identify and extract the sensitive information in the backed-up desensitization data to be restored to obtain desensitization data.
5. The desensitization method according to claim 4, wherein after the invoking of the data desensitization script identifies and extracts sensitive information in the backed-up data to be restored, the method further comprises:
calling a preset data desensitization verification script to verify the accuracy of the desensitization data;
and when the verification is passed, naming the desensitization data according to the naming rule of the desensitization data to be recovered.
6. A desensitizing device to recover sensitive data, comprising:
the data recovery task creating module is used for creating a corresponding data recovery task according to the received data recovery request;
the data backup module is used for acquiring the sensitive data to be recovered according to the data recovery request and backing up the sensitive data to be recovered;
and the data desensitization module is used for desensitizing the backed-up sensitive data to be restored according to the data restoration task and the data desensitization script corresponding to the sensitive data to be restored.
7. A desensitizing device to recover sensitive data according to claim 6, further comprising:
the to-be-recovered sensitive data acquisition module is used for determining corresponding to-be-recovered sensitive data according to the data recovery request;
the data recovery request verification module is used for judging whether the information of the sensitive data to be recovered is consistent with the corresponding information in the data recovery request and whether the data recovery request is a repeated request; and if the information of the sensitive data to be recovered is consistent with the corresponding information in the data recovery request and the data recovery request is not a repeated request, creating a corresponding data recovery task according to the received data recovery request.
8. A desensitization device to recover sensitive data according to claim 7, wherein said data recovery task creation module comprises:
an execution condition determining unit, configured to determine an execution condition of the data recovery task according to a data recovery condition in the data recovery request, where the execution condition includes: job trigger conditions, job priorities and job concurrency;
the execution condition verification unit is used for verifying the rationality of the execution conditions according to all data recovery tasks to be executed and the occupation state of system resources;
and the data recovery task creating unit is used for creating the data recovery task according to the sensitive data to be recovered and the execution condition when the verification is passed.
9. Desensitization device to recover sensitive data according to claim 8, characterized in that said data desensitization module comprises:
the script verification unit is used for judging whether the backed-up sensitive data to be restored is matched with the data desensitization script or not when the execution condition is met;
and the data desensitization unit is used for calling the data desensitization script to identify and extract the sensitive information in the backed-up sensitive data to be restored to obtain desensitized data when the sensitive data to be restored is matched with the data desensitization script.
10. A desensitization device to recover sensitive data according to claim 9, further comprising an accuracy verification module for:
calling a preset data desensitization verification script to verify the accuracy of the desensitization data;
and when the verification is passed, naming the desensitization data according to the naming rule of the desensitization data to be recovered.
11. An electronic device, comprising:
a central processing unit, a memory and a communication module, wherein the memory stores a computer program which can be called by the central processing unit, and the central processing unit executes the computer program to realize the desensitization method of the sensitive data to be recovered according to any one of claims 1 to 5.
12. A computer storage medium storing a computer program, wherein the computer program, when executed by a processor, implements a method of desensitization of sensitive data to be recovered according to any of claims 1-5.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110592653.3A CN113204791A (en) | 2021-05-28 | 2021-05-28 | Desensitization method and desensitization device for desensitization data to be recovered |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110592653.3A CN113204791A (en) | 2021-05-28 | 2021-05-28 | Desensitization method and desensitization device for desensitization data to be recovered |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN113204791A true CN113204791A (en) | 2021-08-03 |
Family
ID=77023551
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110592653.3A Pending CN113204791A (en) | 2021-05-28 | 2021-05-28 | Desensitization method and desensitization device for desensitization data to be recovered |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113204791A (en) |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106599713A (en) * | 2016-11-11 | 2017-04-26 | 中国电子科技网络信息安全有限公司 | Database masking system and method based on big data |
| CN109189613A (en) * | 2018-09-20 | 2019-01-11 | 快云信息科技有限公司 | A kind of database data recovery method and relevant apparatus |
| CN111191281A (en) * | 2019-12-25 | 2020-05-22 | 平安信托有限责任公司 | Data desensitization processing method and device, computer equipment and storage medium |
| CN111858546A (en) * | 2020-06-22 | 2020-10-30 | 网联清算有限公司 | Data processing method, device and system |
-
2021
- 2021-05-28 CN CN202110592653.3A patent/CN113204791A/en active Pending
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106599713A (en) * | 2016-11-11 | 2017-04-26 | 中国电子科技网络信息安全有限公司 | Database masking system and method based on big data |
| CN109189613A (en) * | 2018-09-20 | 2019-01-11 | 快云信息科技有限公司 | A kind of database data recovery method and relevant apparatus |
| CN111191281A (en) * | 2019-12-25 | 2020-05-22 | 平安信托有限责任公司 | Data desensitization processing method and device, computer equipment and storage medium |
| CN111858546A (en) * | 2020-06-22 | 2020-10-30 | 网联清算有限公司 | Data processing method, device and system |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN112463144B (en) | Distributed storage command line service method, system, terminal and storage medium | |
| US7840517B2 (en) | Performance evaluating apparatus, method, and computer-readable medium | |
| US11159604B2 (en) | Processing an operation with a plurality of processing steps | |
| US20200389416A1 (en) | Checkpoint-inclusive resource allocation | |
| CN111726257A (en) | RPA robot management method, platform and storage medium | |
| Grottke et al. | Recovery from software failures caused by mandelbugs | |
| CN111381940B (en) | Distributed data processing method and device | |
| US11734050B2 (en) | Elastic cloud service with data driven cost reduction for VMS in the cloud | |
| US11726893B2 (en) | System for automatically evaluating a change in a large population of processing jobs | |
| US11068487B2 (en) | Event-stream searching using compiled rule patterns | |
| US20200233674A1 (en) | Automatically configuring boot order in recovery operations | |
| CN113377719A (en) | Method and system for acquiring abnormal shutdown time of system | |
| CN113204791A (en) | Desensitization method and desensitization device for desensitization data to be recovered | |
| CN107958414B (en) | Method and system for eliminating long transactions of CICS (common integrated circuit chip) system | |
| CN114006815A (en) | Automatic deployment method and device for cloud platform nodes, nodes and storage medium | |
| CN111399999A (en) | Computer resource processing method and device, readable storage medium and computer equipment | |
| CN115756549A (en) | Method and device for downloading data of big data middlebox and storage medium | |
| CN115794583A (en) | Kernel analysis method and device | |
| CN116263717A (en) | Order service processing method and device based on event | |
| CN115220887A (en) | Processing method of scheduling information, task processing system, processor and electronic equipment | |
| CN115185744A (en) | Validity verification method and device of backup data and computer equipment | |
| CN114741162A (en) | Service arranging method, device, storage medium and equipment | |
| CN111475320B (en) | High-availability detection method of computing platform, computing platform and storage medium | |
| CN111679899A (en) | Task scheduling method, device, platform equipment and storage medium | |
| CN112596750A (en) | Application testing method and device, electronic equipment and computer readable storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |