CN112668047A - Method and system for realizing static desensitization of data increment based on timing task - Google Patents
Method and system for realizing static desensitization of data increment based on timing task Download PDFInfo
- Publication number
- CN112668047A CN112668047A CN202011576819.4A CN202011576819A CN112668047A CN 112668047 A CN112668047 A CN 112668047A CN 202011576819 A CN202011576819 A CN 202011576819A CN 112668047 A CN112668047 A CN 112668047A
- Authority
- CN
- China
- Prior art keywords
- data
- timing task
- desensitization
- task
- timing
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000586 desensitisation Methods 0.000 title claims abstract description 84
- 230000003068 static effect Effects 0.000 title claims abstract description 46
- 238000000034 method Methods 0.000 title claims abstract description 30
- 238000012216 screening Methods 0.000 claims abstract description 19
- 238000005516 engineering process Methods 0.000 claims description 10
- 230000006870 function Effects 0.000 description 9
- 238000010586 diagram Methods 0.000 description 2
- 238000012544 monitoring process Methods 0.000 description 2
- 230000008569 process Effects 0.000 description 2
- 230000035945 sensitivity Effects 0.000 description 2
- 238000004891 communication Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 230000007246 mechanism Effects 0.000 description 1
- 238000005065 mining Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 230000000737 periodic effect Effects 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
Images
Landscapes
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The invention discloses a method and a system for realizing data increment static desensitization based on a timing task, belonging to the technical field of data security, aiming at solving the technical problem of realizing real-time static desensitization on increment data and protecting data privacy more safely and timely, and adopting the technical scheme that: the method comprises the following specific steps: and (3) creating a timing task: deploying a timing task registration center, configuring a timing task executor and registering the timing task executor to the registration center, creating a timing task, setting an execution period and task parameters of the timing task, and starting the timing task; screening incremental data: the timing task screens out incremental data periodically; data desensitization is performed: and calling an API (application programming interface) for performing incremental data desensitization periodically by the timing task to realize incremental desensitization. The system includes a creation module, a screening module, and a desensitization module.
Description
Technical Field
The invention relates to the technical field of big data security, in particular to a method and a system for realizing static desensitization of data increment based on a timing task.
Background
According to the monitoring of international data Information (IDC) companies, the global data volume is about doubled every two years, the arrival of big data leads a revolution, the rapid development of big data and cloud service technology is promoted, the requirements of governments and enterprises on the safety of the big data are more and more strong in the face of the application of the big data in the aspects of storage, mining, analysis and the like, and the data safety plays an increasingly important role. Therefore, how to protect data privacy and sensitive data more safely and timely becomes a challenge point of increasing attention in the industry.
The static desensitization function of data of present data security product can only desensitize current data, can't accomplish real-time tracking increment data (newly-increased data, update data) and desensitize, and sensitivity and the privacy of increment data can't obtain timely protection like this, have privacy security hidden danger.
Disclosure of Invention
The technical task of the invention is to provide a method and a system for realizing data increment static desensitization based on a timing task, so as to solve the problem of how to realize real-time static desensitization on increment data and protect data privacy more safely and timely.
The technical task of the invention is realized in the following way, and the method for realizing the static desensitization of the data increment based on the timing task specifically comprises the following steps:
and (3) creating a timing task: deploying a timing task registration center, configuring a timing task executor and registering the timing task executor to the registration center, creating a timing task, setting an execution period and task parameters of the timing task, and starting the timing task;
screening incremental data: the timing task screens out incremental data periodically;
data desensitization is performed: and calling an API (application programming interface) for performing incremental data desensitization periodically by the timing task to realize incremental desensitization.
Preferably, the timing task is developed secondarily based on an xxl-joba open source framework, becomes a timing task executor through deployment of the configuration item, and sends a registration request to a registration center in real time to ensure real-time effectiveness of the timing task executor.
Preferably, the timing task definition method specifically includes the following steps:
adding @ XxlJob annotation to complete the registration of the timing task;
and the timing task executes the task periodically according to the set timing task time parameter.
Preferably, the screening increment data are specifically as follows:
after the timing task is started, connecting a source database to obtain data of a data table to be desensitized;
screening out all data which are not desensitized according to a time field specified by the desensitization task; wherein the time field includes a creation time field of the data table and an update time field of the data table.
Preferably, the data desensitization is performed specifically as follows:
after the desensitization algorithm identification is obtained, desensitizing and caching metadata to be desensitized;
connecting a target database, and detecting whether a data table with the same name exists:
if not, firstly establishing a table;
after the table is established, desensitizing the data according to a corresponding static desensitization algorithm, and updating the desensitized data into a target database to complete a static desensitization task; the target database comprises relational databases of MySQL, Oracle, SQLSserver and DB 2.
Preferably, after the desensitization task is completed, the connection access to the source database and the target database is automatically closed, and the access amount of the databases is controlled.
A data increment static desensitization system based on a timing task is developed by adopting a Java technology and is realized based on a Springboot framework, a timing increment desensitization function is issued as an API (application programming interface) interface, and front-end developers and application service are allowed to call; the front-end developer is used for developing a web front-end page, developing a static page and a function based on jsp and js related technologies in a web technology, supporting a customer to freely select options of full desensitization, incremental desensitization, timing execution and manual execution, realizing selection of a database and a data table, and supporting a relational database of MySQL, Oracle, Sqlserver and DB 2.
Preferably, the system comprises, in combination,
the system comprises a creating module, a timing task registration center, a timing task executor and a timing task execution module, wherein the creating module is used for deploying the timing task registration center, configuring the timing task executor and registering the timing task executor to the registration center, creating a timing task, setting an execution period and task parameters of the timing task and starting the timing task;
the screening module is used for regularly screening the incremental data by the timing task;
and the desensitization module is used for desensitizing the data according to the corresponding static desensitization algorithm and updating the desensitized data into the target database.
Preferably, the timing task is developed for the second time based on an xxl-joba open source framework, becomes a timing task actuator through deployment of a configuration item, and sends a registration request to a registration center in real time to ensure real-time effectiveness of the timing task actuator;
the timing task definition method specifically comprises the following steps:
adding @ XxlJob annotation to complete the registration of the timing task;
and the timing task executes the task periodically according to the set timing task time parameter.
A computer readable storage medium having stored thereon computer executable instructions which, when executed by a processor, implement a timed task based data increment static desensitization method as described above.
The method and the system for realizing the static desensitization of the data increment based on the timing task have the following advantages:
the method is realized based on a timer timing task, a task executor is arranged, the timing task is registered to realize a periodic retrieval data table, incremental data are retrieved, static desensitization is performed on the incremental data, real-time static desensitization of the incremental data is realized, the privacy of user data is protected more timely and safely, and the safety period of the user data is strengthened;
the invention can track the incremental data in real time and complete the monitoring mechanism of data safety;
the invention can desensitize the incremental data in time, and the privacy and sensitivity of the data are more firmly protected;
the method and the device realize the static desensitization of the incremental data based on the timing task, greatly simplify the work flow of a database security administrator, and can more concisely manage and control the incremental data;
(V) the invention supports static desensitization of multiple data sources;
and sixthly, the incremental data can be tracked in real time by setting the timing task, the incremental data is updated to the target database after being desensitized, the user can track the incremental data in real time by setting the timing task, and the incremental data is subjected to static desensitization and then is stored to the target database by the timing task at regular periods.
Drawings
The invention is further described below with reference to the accompanying drawings.
FIG. 1 is a block diagram schematic diagram of a data increment static desensitization implementation method based on a timing task.
Detailed Description
The method and system for implementing data increment static desensitization based on timing task according to the present invention are described in detail below with reference to the drawings and specific embodiments.
Example 1:
as shown in fig. 1, the method for implementing data increment static desensitization based on timing task includes the following steps:
s1, creating a timing task: deploying a timing task registration center, configuring a timing task executor and registering the timing task executor to the registration center, creating a timing task, setting an execution period and task parameters of the timing task, and starting the timing task;
s2, screening incremental data: the timing task screens out incremental data periodically;
s3, performing data desensitization: and calling an API (application programming interface) for performing incremental data desensitization periodically by the timing task to realize incremental desensitization.
The timing task in the embodiment is developed for the second time based on an xxl-joba open source framework, becomes a timing task executor through deployment of a configuration item, and sends a registration request to a registration center in real time to ensure real-time effectiveness of the timing task executor.
The method for defining the timing task in the embodiment specifically comprises the following steps:
adding an @ XxlJob annotation to complete the registration of a timing task;
secondly, the timing task executes the task periodically according to the set time parameter of the timing task.
In this embodiment, the screening increment data in step S2 is specifically as follows:
s201, after the timing task is started, connecting a source database to obtain data of a data table to be desensitized;
s202, screening out all data which are not desensitized according to a time field specified by a desensitization task; wherein the time field includes a creation time field of the data table and an update time field of the data table.
The execution of data desensitization in step S3 in this embodiment is specifically as follows:
s301, after the desensitization algorithm identification is obtained, desensitization is carried out on metadata to be desensitized and caching is carried out;
s302, connecting a target database, and detecting whether a data table with the same name exists:
if not, firstly establishing a table;
s303, after the table building is completed, desensitizing the data according to a corresponding static desensitization algorithm, and updating the desensitized data into a target database to complete a static desensitization task; the target database comprises relational databases of MySQL, Oracle, SQLSserver and DB 2.
After the desensitization task in this embodiment is completed, the connection access to the source database and the target database is automatically closed, and the access amount of the databases is controlled.
Example 2:
the data increment static desensitization system based on the timing task is developed by adopting Java technology and is realized based on a Springboot framework, and a timing increment desensitization function is issued as an API (application programming interface) to allow front-end developers and application services to call; the front-end developer is used for developing a web front-end page, developing a static page and a function based on jsp and js related technologies in a web technology, supporting a client to freely select options of full desensitization, incremental desensitization, timing execution and manual execution, realizing selection of a database and a data table, and supporting a relational database of MySQL, Oracle, Sqlserver and DB 2; the system comprises a plurality of devices which are connected with each other,
the system comprises a creating module, a timing task registration center, a timing task executor and a timing task execution module, wherein the creating module is used for deploying the timing task registration center, configuring the timing task executor and registering the timing task executor to the registration center, creating a timing task, setting an execution period and task parameters of the timing task and starting the timing task;
the screening module is used for regularly screening the incremental data by the timing task; the working process of the screening module is as follows:
(1) after the timing task is started, connecting a source database to obtain data of a data table to be desensitized;
(2) screening out all data which are not desensitized according to the time field appointed by the desensitization task; wherein the time field includes a creation time field of the data table and an update time field of the data table.
And the desensitization module is used for desensitizing the data according to the corresponding static desensitization algorithm and updating the desensitized data into the target database. The operation process of the desensitization module is as follows:
(1) after the desensitization algorithm identification is obtained, desensitizing and caching metadata to be desensitized;
(2) connecting a target database, and detecting whether a data table with the same name exists:
if not, firstly establishing a table;
(3) after the table building is finished, desensitizing the data according to a corresponding static desensitization algorithm, and updating the desensitized data into a target database to finish a static desensitization task; the target database comprises relational databases of MySQL, Oracle, SQLSserver and DB 2.
The timing task in the embodiment is developed for the second time based on an xxl-joba open source framework, becomes a timing task actuator through deployment of a configuration item, and sends a registration request to a registration center in real time to ensure real-time effectiveness of the timing task actuator;
the method for defining the timing task in the embodiment specifically comprises the following steps:
adding an @ XxlJob annotation to complete the registration of a timing task;
secondly, the timing task executes the task periodically according to the set time parameter of the timing task.
Example 3:
embodiments of the present invention also provide a computer-readable storage medium having stored thereon a plurality of instructions, which are loadable by a processor and cause the processor to execute a XXXX method in any of the embodiments of the present invention. Specifically, a system or an apparatus equipped with a storage medium on which software program codes that realize the functions of any of the above-described embodiments are stored may be provided, and a computer (or a CPU or MPU) of the system or the apparatus is caused to read out and execute the program codes stored in the storage medium.
In this case, the program code itself read from the storage medium can realize the functions of any of the above-described embodiments, and thus the program code and the storage medium storing the program code constitute a part of the present invention.
Examples of storage media that can be used to provide program code include floppy disks, hard disks, magneto-optical disks, optical disks (e.g., CD-ROMs, CD-R, CD-RWs, DVD-ROMs, DVD-Rs, and systems M, DVD-RWs, DVD + RWs) for implementing incremental static desensitization of data based on timing tasks, magnetic tape, non-volatile memory cards, and ROMs. Alternatively, the program code may be downloaded from a server computer via a communications network.
Further, it should be clear that the functions of any one of the above-described embodiments may be implemented not only by executing the program code read out by the computer, but also by causing an operating system or the like operating on the computer to perform a part or all of the actual operations based on instructions of the program code.
Further, it is to be understood that the program code read out from the storage medium is written to a memory provided in an expansion board inserted into the computer or to a memory provided in an expansion unit connected to the computer, and then causes a CPU or the like mounted on the expansion board or the expansion unit to perform part or all of the actual operations based on instructions of the program code, thereby realizing the functions of any of the above-described embodiments.
Finally, it should be noted that: the above embodiments are only used to illustrate the technical solution of the present invention, and not to limit the same; while the invention has been described in detail and with reference to the foregoing embodiments, it will be understood by those skilled in the art that: the technical solutions described in the foregoing embodiments may still be modified, or some or all of the technical features may be equivalently replaced; and the modifications or the substitutions do not make the essence of the corresponding technical solutions depart from the scope of the technical solutions of the embodiments of the present invention.
Claims (10)
1. A data increment static desensitization implementation method based on a timing task is characterized by comprising the following steps:
and (3) creating a timing task: deploying a timing task registration center, configuring a timing task executor and registering the timing task executor to the registration center, creating a timing task, setting an execution period and task parameters of the timing task, and starting the timing task;
screening incremental data: the timing task screens out incremental data periodically;
data desensitization is performed: and calling an API (application programming interface) for performing incremental data desensitization periodically by the timing task to realize incremental desensitization.
2. The method for implementing static desensitization of data increment based on timed tasks according to claim 1, wherein the timed tasks are secondarily developed based on xxl-joba open source framework, become timed task executors through deployment of configuration items, and send registration requests to a registry in real time.
3. The method for implementing static desensitization of data increment based on timed task according to claim 1, wherein the timed task definition method is as follows:
adding @ XxlJob annotation to complete the registration of the timing task;
and the timing task executes the task periodically according to the set timing task time parameter.
4. The method for implementing data increment static desensitization based on timed tasks according to claim 1, characterized in that the screening increment data is as follows:
after the timing task is started, connecting a source database to obtain data of a data table to be desensitized;
screening out all data which are not desensitized according to a time field specified by the desensitization task; wherein the time field includes a creation time field of the data table and an update time field of the data table.
5. The method for implementing static desensitization of data increments based on timed tasks according to claim 1, characterized in that the data desensitization is implemented as follows:
after the desensitization algorithm identification is obtained, desensitizing and caching metadata to be desensitized;
connecting a target database, and detecting whether a data table with the same name exists:
if not, firstly establishing a table;
after the table is established, desensitizing the data according to a corresponding static desensitization algorithm, and updating the desensitized data into a target database to complete a static desensitization task; the target database comprises relational databases of MySQL, Oracle, SQLSserver and DB 2.
6. The method for implementing data increment static desensitization based on timed tasks according to any one of claims 1 to 5, wherein after the desensitization task is completed, the connection access to the source database and the target database is automatically closed, and the access volume of the databases is controlled.
7. A data increment static desensitization system based on a timing task is characterized in that the system is developed by adopting Java technology and is realized based on a Springboot framework, a timing increment desensitization function is issued as an API (application programming interface) interface, and front-end developers and application service calling are allowed; the front-end developer is used for developing a web front-end page, developing a static page and a function based on jsp and js related technologies in a web technology, supporting a customer to freely select options of full desensitization, incremental desensitization, timing execution and manual execution, realizing selection of a database and a data table, and supporting a relational database of MySQL, Oracle, Sqlserver and DB 2.
8. A timed task based data increment static desensitization system according to claim 7, wherein the system includes,
the system comprises a creating module, a timing task registration center, a timing task executor and a timing task execution module, wherein the creating module is used for deploying the timing task registration center, configuring the timing task executor and registering the timing task executor to the registration center, creating a timing task, setting an execution period and task parameters of the timing task and starting the timing task;
the screening module is used for regularly screening the incremental data by the timing task;
and the desensitization module is used for desensitizing the data according to the corresponding static desensitization algorithm and updating the desensitized data into the target database.
9. The timed task based data increment static desensitization system according to claim 8, wherein the timed task is secondarily developed based on xxl-joba open source framework, becomes a timed task executor through deployment of configuration items, and sends a registration request to the registry in real time;
the timing task definition method specifically comprises the following steps:
adding @ XxlJob annotation to complete the registration of the timing task;
and the timing task executes the task periodically according to the set timing task time parameter.
10. A computer-readable storage medium having computer-executable instructions stored thereon that, when executed by a processor, perform a method for timed task based static desensitization of data increments as recited in claims 1-6.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011576819.4A CN112668047A (en) | 2020-12-28 | 2020-12-28 | Method and system for realizing static desensitization of data increment based on timing task |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011576819.4A CN112668047A (en) | 2020-12-28 | 2020-12-28 | Method and system for realizing static desensitization of data increment based on timing task |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN112668047A true CN112668047A (en) | 2021-04-16 |
Family
ID=75410509
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202011576819.4A Pending CN112668047A (en) | 2020-12-28 | 2020-12-28 | Method and system for realizing static desensitization of data increment based on timing task |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112668047A (en) |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114201772A (en) * | 2021-11-19 | 2022-03-18 | 广东航宇卫星科技有限公司 | Stock data desensitization method and device |
| CN115878097A (en) * | 2023-03-03 | 2023-03-31 | 智慧眼科技股份有限公司 | Method and system for creating timing task |
| CN117520020A (en) * | 2024-01-05 | 2024-02-06 | 同盾科技有限公司 | Data interaction method, device and system for realizing privacy calculation |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110245470A (en) * | 2019-04-25 | 2019-09-17 | 深圳壹账通智能科技有限公司 | Intelligent data desensitization method, device, computer equipment and storage medium |
| CN110362393A (en) * | 2019-07-18 | 2019-10-22 | 北京明略软件系统有限公司 | The detection method and device of increment task |
| CN110781515A (en) * | 2019-10-25 | 2020-02-11 | 上海凯馨信息科技有限公司 | Static data desensitization method and desensitization device |
| CN111125214A (en) * | 2019-12-02 | 2020-05-08 | 武汉虹信技术服务有限责任公司 | Lightweight incremental data synchronization method and device and computer readable medium |
| CN111400762A (en) * | 2020-03-18 | 2020-07-10 | 上海凯馨信息科技有限公司 | Dynamic desensitization method for oracle database |
| CN111858546A (en) * | 2020-06-22 | 2020-10-30 | 网联清算有限公司 | Data processing method, device and system |
-
2020
- 2020-12-28 CN CN202011576819.4A patent/CN112668047A/en active Pending
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110245470A (en) * | 2019-04-25 | 2019-09-17 | 深圳壹账通智能科技有限公司 | Intelligent data desensitization method, device, computer equipment and storage medium |
| CN110362393A (en) * | 2019-07-18 | 2019-10-22 | 北京明略软件系统有限公司 | The detection method and device of increment task |
| CN110781515A (en) * | 2019-10-25 | 2020-02-11 | 上海凯馨信息科技有限公司 | Static data desensitization method and desensitization device |
| CN111125214A (en) * | 2019-12-02 | 2020-05-08 | 武汉虹信技术服务有限责任公司 | Lightweight incremental data synchronization method and device and computer readable medium |
| CN111400762A (en) * | 2020-03-18 | 2020-07-10 | 上海凯馨信息科技有限公司 | Dynamic desensitization method for oracle database |
| CN111858546A (en) * | 2020-06-22 | 2020-10-30 | 网联清算有限公司 | Data processing method, device and system |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114201772A (en) * | 2021-11-19 | 2022-03-18 | 广东航宇卫星科技有限公司 | Stock data desensitization method and device |
| CN115878097A (en) * | 2023-03-03 | 2023-03-31 | 智慧眼科技股份有限公司 | Method and system for creating timing task |
| CN117520020A (en) * | 2024-01-05 | 2024-02-06 | 同盾科技有限公司 | Data interaction method, device and system for realizing privacy calculation |
| CN117520020B (en) * | 2024-01-05 | 2024-03-29 | 同盾科技有限公司 | Data interaction method, device and system for realizing privacy calculation |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN112668047A (en) | Method and system for realizing static desensitization of data increment based on timing task | |
| US11063983B2 (en) | Componentized security policy generation | |
| US7765460B2 (en) | Out-of-band change detection | |
| CN102150105B (en) | Deployment and management of virtual containers | |
| US8443361B2 (en) | Systems and methods for tracking a history of changes associated with software packages in a computing system | |
| CN108776643B (en) | Target code merging control method and system based on version control process | |
| US20170017795A1 (en) | Enterprise level security orchestration | |
| CN113256296B (en) | Intelligent contract execution method, system, device and storage medium | |
| US20210149682A1 (en) | System and method for implementing a filesystem agent management solution | |
| CN111258850A (en) | Method and device for updating software information based on Linux system | |
| AU2017276243B2 (en) | System And Method For Generating Service Operation Implementation | |
| CN110298179B (en) | Open source framework security vulnerability detection method and device | |
| US6961940B2 (en) | Dynamically adapting events to capabilities of a management system | |
| KR101283884B1 (en) | Apparatus, method and computer readable recording medium for safeguard | |
| US9047161B1 (en) | Discovery, consolidation, and archival of multiple operating system software licenses | |
| CN105453057A (en) | Storing difference information in a backup system | |
| US11770295B2 (en) | Platform for establishing computing node clusters in different environments | |
| US11671440B1 (en) | Detection failure monitoring system | |
| WO2019018316A1 (en) | Enterprise level cybersecurity automatic remediation | |
| CN114978963A (en) | Network system monitoring analysis method and device, electronic equipment and storage medium | |
| US6868540B2 (en) | Recycling events to take advantage of capabilities of a management system | |
| CN117015759A (en) | Automatic updating of VM sets | |
| CN110221952B (en) | Service data processing method and device and service data processing system | |
| CN109828781B (en) | Source code version positioning method, device, medium and equipment for problem troubleshooting | |
| CN113609478A (en) | IOS platform application program tampering detection method and device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20210416 |