CN112668047A - Method and system for realizing static desensitization of data increment based on timing task - Google Patents

Method and system for realizing static desensitization of data increment based on timing task Download PDF

Info

Publication number
CN112668047A
CN112668047A CN202011576819.4A CN202011576819A CN112668047A CN 112668047 A CN112668047 A CN 112668047A CN 202011576819 A CN202011576819 A CN 202011576819A CN 112668047 A CN112668047 A CN 112668047A
Authority
CN
China
Prior art keywords
data
timing task
desensitization
task
timing
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202011576819.4A
Other languages
Chinese (zh)
Inventor
王哲
张滨
孙思清
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Inspur Cloud Information Technology Co Ltd
Original Assignee
Inspur Cloud Information Technology Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Inspur Cloud Information Technology Co Ltd filed Critical Inspur Cloud Information Technology Co Ltd
Priority to CN202011576819.4A priority Critical patent/CN112668047A/en
Publication of CN112668047A publication Critical patent/CN112668047A/en
Pending legal-status Critical Current

Links

Images

Landscapes

  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)

Abstract

The invention discloses a method and a system for realizing data increment static desensitization based on a timing task, belonging to the technical field of data security, aiming at solving the technical problem of realizing real-time static desensitization on increment data and protecting data privacy more safely and timely, and adopting the technical scheme that: the method comprises the following specific steps: and (3) creating a timing task: deploying a timing task registration center, configuring a timing task executor and registering the timing task executor to the registration center, creating a timing task, setting an execution period and task parameters of the timing task, and starting the timing task; screening incremental data: the timing task screens out incremental data periodically; data desensitization is performed: and calling an API (application programming interface) for performing incremental data desensitization periodically by the timing task to realize incremental desensitization. The system includes a creation module, a screening module, and a desensitization module.

Description

Method and system for realizing static desensitization of data increment based on timing task
Technical Field
The invention relates to the technical field of big data security, in particular to a method and a system for realizing static desensitization of data increment based on a timing task.
Background
According to the monitoring of international data Information (IDC) companies, the global data volume is about doubled every two years, the arrival of big data leads a revolution, the rapid development of big data and cloud service technology is promoted, the requirements of governments and enterprises on the safety of the big data are more and more strong in the face of the application of the big data in the aspects of storage, mining, analysis and the like, and the data safety plays an increasingly important role. Therefore, how to protect data privacy and sensitive data more safely and timely becomes a challenge point of increasing attention in the industry.
The static desensitization function of data of present data security product can only desensitize current data, can't accomplish real-time tracking increment data (newly-increased data, update data) and desensitize, and sensitivity and the privacy of increment data can't obtain timely protection like this, have privacy security hidden danger.
Disclosure of Invention
The technical task of the invention is to provide a method and a system for realizing data increment static desensitization based on a timing task, so as to solve the problem of how to realize real-time static desensitization on increment data and protect data privacy more safely and timely.
The technical task of the invention is realized in the following way, and the method for realizing the static desensitization of the data increment based on the timing task specifically comprises the following steps:
and (3) creating a timing task: deploying a timing task registration center, configuring a timing task executor and registering the timing task executor to the registration center, creating a timing task, setting an execution period and task parameters of the timing task, and starting the timing task;
screening incremental data: the timing task screens out incremental data periodically;
data desensitization is performed: and calling an API (application programming interface) for performing incremental data desensitization periodically by the timing task to realize incremental desensitization.
Preferably, the timing task is developed secondarily based on an xxl-joba open source framework, becomes a timing task executor through deployment of the configuration item, and sends a registration request to a registration center in real time to ensure real-time effectiveness of the timing task executor.
Preferably, the timing task definition method specifically includes the following steps:
adding @ XxlJob annotation to complete the registration of the timing task;
and the timing task executes the task periodically according to the set timing task time parameter.
Preferably, the screening increment data are specifically as follows:
after the timing task is started, connecting a source database to obtain data of a data table to be desensitized;
screening out all data which are not desensitized according to a time field specified by the desensitization task; wherein the time field includes a creation time field of the data table and an update time field of the data table.
Preferably, the data desensitization is performed specifically as follows:
after the desensitization algorithm identification is obtained, desensitizing and caching metadata to be desensitized;
connecting a target database, and detecting whether a data table with the same name exists:
if not, firstly establishing a table;
after the table is established, desensitizing the data according to a corresponding static desensitization algorithm, and updating the desensitized data into a target database to complete a static desensitization task; the target database comprises relational databases of MySQL, Oracle, SQLSserver and DB 2.
Preferably, after the desensitization task is completed, the connection access to the source database and the target database is automatically closed, and the access amount of the databases is controlled.
A data increment static desensitization system based on a timing task is developed by adopting a Java technology and is realized based on a Springboot framework, a timing increment desensitization function is issued as an API (application programming interface) interface, and front-end developers and application service are allowed to call; the front-end developer is used for developing a web front-end page, developing a static page and a function based on jsp and js related technologies in a web technology, supporting a customer to freely select options of full desensitization, incremental desensitization, timing execution and manual execution, realizing selection of a database and a data table, and supporting a relational database of MySQL, Oracle, Sqlserver and DB 2.
Preferably, the system comprises, in combination,
the system comprises a creating module, a timing task registration center, a timing task executor and a timing task execution module, wherein the creating module is used for deploying the timing task registration center, configuring the timing task executor and registering the timing task executor to the registration center, creating a timing task, setting an execution period and task parameters of the timing task and starting the timing task;
the screening module is used for regularly screening the incremental data by the timing task;
and the desensitization module is used for desensitizing the data according to the corresponding static desensitization algorithm and updating the desensitized data into the target database.
Preferably, the timing task is developed for the second time based on an xxl-joba open source framework, becomes a timing task actuator through deployment of a configuration item, and sends a registration request to a registration center in real time to ensure real-time effectiveness of the timing task actuator;
the timing task definition method specifically comprises the following steps:
adding @ XxlJob annotation to complete the registration of the timing task;
and the timing task executes the task periodically according to the set timing task time parameter.
A computer readable storage medium having stored thereon computer executable instructions which, when executed by a processor, implement a timed task based data increment static desensitization method as described above.
The method and the system for realizing the static desensitization of the data increment based on the timing task have the following advantages:
the method is realized based on a timer timing task, a task executor is arranged, the timing task is registered to realize a periodic retrieval data table, incremental data are retrieved, static desensitization is performed on the incremental data, real-time static desensitization of the incremental data is realized, the privacy of user data is protected more timely and safely, and the safety period of the user data is strengthened;
the invention can track the incremental data in real time and complete the monitoring mechanism of data safety;
the invention can desensitize the incremental data in time, and the privacy and sensitivity of the data are more firmly protected;
the method and the device realize the static desensitization of the incremental data based on the timing task, greatly simplify the work flow of a database security administrator, and can more concisely manage and control the incremental data;
(V) the invention supports static desensitization of multiple data sources;
and sixthly, the incremental data can be tracked in real time by setting the timing task, the incremental data is updated to the target database after being desensitized, the user can track the incremental data in real time by setting the timing task, and the incremental data is subjected to static desensitization and then is stored to the target database by the timing task at regular periods.
Drawings
The invention is further described below with reference to the accompanying drawings.
FIG. 1 is a block diagram schematic diagram of a data increment static desensitization implementation method based on a timing task.
Detailed Description
The method and system for implementing data increment static desensitization based on timing task according to the present invention are described in detail below with reference to the drawings and specific embodiments.
Example 1:
as shown in fig. 1, the method for implementing data increment static desensitization based on timing task includes the following steps:
s1, creating a timing task: deploying a timing task registration center, configuring a timing task executor and registering the timing task executor to the registration center, creating a timing task, setting an execution period and task parameters of the timing task, and starting the timing task;
s2, screening incremental data: the timing task screens out incremental data periodically;
s3, performing data desensitization: and calling an API (application programming interface) for performing incremental data desensitization periodically by the timing task to realize incremental desensitization.
The timing task in the embodiment is developed for the second time based on an xxl-joba open source framework, becomes a timing task executor through deployment of a configuration item, and sends a registration request to a registration center in real time to ensure real-time effectiveness of the timing task executor.
The method for defining the timing task in the embodiment specifically comprises the following steps:
adding an @ XxlJob annotation to complete the registration of a timing task;
secondly, the timing task executes the task periodically according to the set time parameter of the timing task.
In this embodiment, the screening increment data in step S2 is specifically as follows:
s201, after the timing task is started, connecting a source database to obtain data of a data table to be desensitized;
s202, screening out all data which are not desensitized according to a time field specified by a desensitization task; wherein the time field includes a creation time field of the data table and an update time field of the data table.
The execution of data desensitization in step S3 in this embodiment is specifically as follows:
s301, after the desensitization algorithm identification is obtained, desensitization is carried out on metadata to be desensitized and caching is carried out;
s302, connecting a target database, and detecting whether a data table with the same name exists:
if not, firstly establishing a table;
s303, after the table building is completed, desensitizing the data according to a corresponding static desensitization algorithm, and updating the desensitized data into a target database to complete a static desensitization task; the target database comprises relational databases of MySQL, Oracle, SQLSserver and DB 2.
After the desensitization task in this embodiment is completed, the connection access to the source database and the target database is automatically closed, and the access amount of the databases is controlled.
Example 2:
the data increment static desensitization system based on the timing task is developed by adopting Java technology and is realized based on a Springboot framework, and a timing increment desensitization function is issued as an API (application programming interface) to allow front-end developers and application services to call; the front-end developer is used for developing a web front-end page, developing a static page and a function based on jsp and js related technologies in a web technology, supporting a client to freely select options of full desensitization, incremental desensitization, timing execution and manual execution, realizing selection of a database and a data table, and supporting a relational database of MySQL, Oracle, Sqlserver and DB 2; the system comprises a plurality of devices which are connected with each other,
the system comprises a creating module, a timing task registration center, a timing task executor and a timing task execution module, wherein the creating module is used for deploying the timing task registration center, configuring the timing task executor and registering the timing task executor to the registration center, creating a timing task, setting an execution period and task parameters of the timing task and starting the timing task;
the screening module is used for regularly screening the incremental data by the timing task; the working process of the screening module is as follows:
(1) after the timing task is started, connecting a source database to obtain data of a data table to be desensitized;
(2) screening out all data which are not desensitized according to the time field appointed by the desensitization task; wherein the time field includes a creation time field of the data table and an update time field of the data table.
And the desensitization module is used for desensitizing the data according to the corresponding static desensitization algorithm and updating the desensitized data into the target database. The operation process of the desensitization module is as follows:
(1) after the desensitization algorithm identification is obtained, desensitizing and caching metadata to be desensitized;
(2) connecting a target database, and detecting whether a data table with the same name exists:
if not, firstly establishing a table;
(3) after the table building is finished, desensitizing the data according to a corresponding static desensitization algorithm, and updating the desensitized data into a target database to finish a static desensitization task; the target database comprises relational databases of MySQL, Oracle, SQLSserver and DB 2.
The timing task in the embodiment is developed for the second time based on an xxl-joba open source framework, becomes a timing task actuator through deployment of a configuration item, and sends a registration request to a registration center in real time to ensure real-time effectiveness of the timing task actuator;
the method for defining the timing task in the embodiment specifically comprises the following steps:
adding an @ XxlJob annotation to complete the registration of a timing task;
secondly, the timing task executes the task periodically according to the set time parameter of the timing task.
Example 3:
embodiments of the present invention also provide a computer-readable storage medium having stored thereon a plurality of instructions, which are loadable by a processor and cause the processor to execute a XXXX method in any of the embodiments of the present invention. Specifically, a system or an apparatus equipped with a storage medium on which software program codes that realize the functions of any of the above-described embodiments are stored may be provided, and a computer (or a CPU or MPU) of the system or the apparatus is caused to read out and execute the program codes stored in the storage medium.
In this case, the program code itself read from the storage medium can realize the functions of any of the above-described embodiments, and thus the program code and the storage medium storing the program code constitute a part of the present invention.
Examples of storage media that can be used to provide program code include floppy disks, hard disks, magneto-optical disks, optical disks (e.g., CD-ROMs, CD-R, CD-RWs, DVD-ROMs, DVD-Rs, and systems M, DVD-RWs, DVD + RWs) for implementing incremental static desensitization of data based on timing tasks, magnetic tape, non-volatile memory cards, and ROMs. Alternatively, the program code may be downloaded from a server computer via a communications network.
Further, it should be clear that the functions of any one of the above-described embodiments may be implemented not only by executing the program code read out by the computer, but also by causing an operating system or the like operating on the computer to perform a part or all of the actual operations based on instructions of the program code.
Further, it is to be understood that the program code read out from the storage medium is written to a memory provided in an expansion board inserted into the computer or to a memory provided in an expansion unit connected to the computer, and then causes a CPU or the like mounted on the expansion board or the expansion unit to perform part or all of the actual operations based on instructions of the program code, thereby realizing the functions of any of the above-described embodiments.
Finally, it should be noted that: the above embodiments are only used to illustrate the technical solution of the present invention, and not to limit the same; while the invention has been described in detail and with reference to the foregoing embodiments, it will be understood by those skilled in the art that: the technical solutions described in the foregoing embodiments may still be modified, or some or all of the technical features may be equivalently replaced; and the modifications or the substitutions do not make the essence of the corresponding technical solutions depart from the scope of the technical solutions of the embodiments of the present invention.

Claims (10)

1. A data increment static desensitization implementation method based on a timing task is characterized by comprising the following steps:
and (3) creating a timing task: deploying a timing task registration center, configuring a timing task executor and registering the timing task executor to the registration center, creating a timing task, setting an execution period and task parameters of the timing task, and starting the timing task;
screening incremental data: the timing task screens out incremental data periodically;
data desensitization is performed: and calling an API (application programming interface) for performing incremental data desensitization periodically by the timing task to realize incremental desensitization.
2. The method for implementing static desensitization of data increment based on timed tasks according to claim 1, wherein the timed tasks are secondarily developed based on xxl-joba open source framework, become timed task executors through deployment of configuration items, and send registration requests to a registry in real time.
3. The method for implementing static desensitization of data increment based on timed task according to claim 1, wherein the timed task definition method is as follows:
adding @ XxlJob annotation to complete the registration of the timing task;
and the timing task executes the task periodically according to the set timing task time parameter.
4. The method for implementing data increment static desensitization based on timed tasks according to claim 1, characterized in that the screening increment data is as follows:
after the timing task is started, connecting a source database to obtain data of a data table to be desensitized;
screening out all data which are not desensitized according to a time field specified by the desensitization task; wherein the time field includes a creation time field of the data table and an update time field of the data table.
5. The method for implementing static desensitization of data increments based on timed tasks according to claim 1, characterized in that the data desensitization is implemented as follows:
after the desensitization algorithm identification is obtained, desensitizing and caching metadata to be desensitized;
connecting a target database, and detecting whether a data table with the same name exists:
if not, firstly establishing a table;
after the table is established, desensitizing the data according to a corresponding static desensitization algorithm, and updating the desensitized data into a target database to complete a static desensitization task; the target database comprises relational databases of MySQL, Oracle, SQLSserver and DB 2.
6. The method for implementing data increment static desensitization based on timed tasks according to any one of claims 1 to 5, wherein after the desensitization task is completed, the connection access to the source database and the target database is automatically closed, and the access volume of the databases is controlled.
7. A data increment static desensitization system based on a timing task is characterized in that the system is developed by adopting Java technology and is realized based on a Springboot framework, a timing increment desensitization function is issued as an API (application programming interface) interface, and front-end developers and application service calling are allowed; the front-end developer is used for developing a web front-end page, developing a static page and a function based on jsp and js related technologies in a web technology, supporting a customer to freely select options of full desensitization, incremental desensitization, timing execution and manual execution, realizing selection of a database and a data table, and supporting a relational database of MySQL, Oracle, Sqlserver and DB 2.
8. A timed task based data increment static desensitization system according to claim 7, wherein the system includes,
the system comprises a creating module, a timing task registration center, a timing task executor and a timing task execution module, wherein the creating module is used for deploying the timing task registration center, configuring the timing task executor and registering the timing task executor to the registration center, creating a timing task, setting an execution period and task parameters of the timing task and starting the timing task;
the screening module is used for regularly screening the incremental data by the timing task;
and the desensitization module is used for desensitizing the data according to the corresponding static desensitization algorithm and updating the desensitized data into the target database.
9. The timed task based data increment static desensitization system according to claim 8, wherein the timed task is secondarily developed based on xxl-joba open source framework, becomes a timed task executor through deployment of configuration items, and sends a registration request to the registry in real time;
the timing task definition method specifically comprises the following steps:
adding @ XxlJob annotation to complete the registration of the timing task;
and the timing task executes the task periodically according to the set timing task time parameter.
10. A computer-readable storage medium having computer-executable instructions stored thereon that, when executed by a processor, perform a method for timed task based static desensitization of data increments as recited in claims 1-6.
CN202011576819.4A 2020-12-28 2020-12-28 Method and system for realizing static desensitization of data increment based on timing task Pending CN112668047A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202011576819.4A CN112668047A (en) 2020-12-28 2020-12-28 Method and system for realizing static desensitization of data increment based on timing task

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202011576819.4A CN112668047A (en) 2020-12-28 2020-12-28 Method and system for realizing static desensitization of data increment based on timing task

Publications (1)

Publication Number Publication Date
CN112668047A true CN112668047A (en) 2021-04-16

Family

ID=75410509

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202011576819.4A Pending CN112668047A (en) 2020-12-28 2020-12-28 Method and system for realizing static desensitization of data increment based on timing task

Country Status (1)

Country Link
CN (1) CN112668047A (en)

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114201772A (en) * 2021-11-19 2022-03-18 广东航宇卫星科技有限公司 Stock data desensitization method and device
CN115878097A (en) * 2023-03-03 2023-03-31 智慧眼科技股份有限公司 Method and system for creating timing task
CN117520020A (en) * 2024-01-05 2024-02-06 同盾科技有限公司 Data interaction method, device and system for realizing privacy calculation

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110245470A (en) * 2019-04-25 2019-09-17 深圳壹账通智能科技有限公司 Intelligent data desensitization method, device, computer equipment and storage medium
CN110362393A (en) * 2019-07-18 2019-10-22 北京明略软件系统有限公司 The detection method and device of increment task
CN110781515A (en) * 2019-10-25 2020-02-11 上海凯馨信息科技有限公司 Static data desensitization method and desensitization device
CN111125214A (en) * 2019-12-02 2020-05-08 武汉虹信技术服务有限责任公司 Lightweight incremental data synchronization method and device and computer readable medium
CN111400762A (en) * 2020-03-18 2020-07-10 上海凯馨信息科技有限公司 Dynamic desensitization method for oracle database
CN111858546A (en) * 2020-06-22 2020-10-30 网联清算有限公司 Data processing method, device and system

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110245470A (en) * 2019-04-25 2019-09-17 深圳壹账通智能科技有限公司 Intelligent data desensitization method, device, computer equipment and storage medium
CN110362393A (en) * 2019-07-18 2019-10-22 北京明略软件系统有限公司 The detection method and device of increment task
CN110781515A (en) * 2019-10-25 2020-02-11 上海凯馨信息科技有限公司 Static data desensitization method and desensitization device
CN111125214A (en) * 2019-12-02 2020-05-08 武汉虹信技术服务有限责任公司 Lightweight incremental data synchronization method and device and computer readable medium
CN111400762A (en) * 2020-03-18 2020-07-10 上海凯馨信息科技有限公司 Dynamic desensitization method for oracle database
CN111858546A (en) * 2020-06-22 2020-10-30 网联清算有限公司 Data processing method, device and system

Cited By (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114201772A (en) * 2021-11-19 2022-03-18 广东航宇卫星科技有限公司 Stock data desensitization method and device
CN115878097A (en) * 2023-03-03 2023-03-31 智慧眼科技股份有限公司 Method and system for creating timing task
CN117520020A (en) * 2024-01-05 2024-02-06 同盾科技有限公司 Data interaction method, device and system for realizing privacy calculation
CN117520020B (en) * 2024-01-05 2024-03-29 同盾科技有限公司 Data interaction method, device and system for realizing privacy calculation

Similar Documents

Publication Publication Date Title
CN112668047A (en) Method and system for realizing static desensitization of data increment based on timing task
US11063983B2 (en) Componentized security policy generation
US7765460B2 (en) Out-of-band change detection
CN102150105B (en) Deployment and management of virtual containers
US8443361B2 (en) Systems and methods for tracking a history of changes associated with software packages in a computing system
CN108776643B (en) Target code merging control method and system based on version control process
US20170017795A1 (en) Enterprise level security orchestration
CN113256296B (en) Intelligent contract execution method, system, device and storage medium
US20210149682A1 (en) System and method for implementing a filesystem agent management solution
CN111258850A (en) Method and device for updating software information based on Linux system
AU2017276243B2 (en) System And Method For Generating Service Operation Implementation
CN110298179B (en) Open source framework security vulnerability detection method and device
US6961940B2 (en) Dynamically adapting events to capabilities of a management system
KR101283884B1 (en) Apparatus, method and computer readable recording medium for safeguard
US9047161B1 (en) Discovery, consolidation, and archival of multiple operating system software licenses
CN105453057A (en) Storing difference information in a backup system
US11770295B2 (en) Platform for establishing computing node clusters in different environments
US11671440B1 (en) Detection failure monitoring system
WO2019018316A1 (en) Enterprise level cybersecurity automatic remediation
CN114978963A (en) Network system monitoring analysis method and device, electronic equipment and storage medium
US6868540B2 (en) Recycling events to take advantage of capabilities of a management system
CN117015759A (en) Automatic updating of VM sets
CN110221952B (en) Service data processing method and device and service data processing system
CN109828781B (en) Source code version positioning method, device, medium and equipment for problem troubleshooting
CN113609478A (en) IOS platform application program tampering detection method and device

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
RJ01 Rejection of invention patent application after publication
RJ01 Rejection of invention patent application after publication

Application publication date: 20210416