CN116302405A - Data desensitization method and device - Google Patents

Data desensitization method and device Download PDF

Info

Publication number
CN116302405A
CN116302405A CN202310149597.5A CN202310149597A CN116302405A CN 116302405 A CN116302405 A CN 116302405A CN 202310149597 A CN202310149597 A CN 202310149597A CN 116302405 A CN116302405 A CN 116302405A
Authority
CN
China
Prior art keywords
data processing
data
combination
desensitization
processing step
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202310149597.5A
Other languages
Chinese (zh)
Inventor
李燕强
李承文
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
China Construction Bank Corp
CCB Finetech Co Ltd
Original Assignee
China Construction Bank Corp
CCB Finetech Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by China Construction Bank Corp, CCB Finetech Co Ltd filed Critical China Construction Bank Corp
Priority to CN202310149597.5A priority Critical patent/CN116302405A/en
Publication of CN116302405A publication Critical patent/CN116302405A/en
Pending legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00—Arrangements for program control, e.g. control units
    • G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/46—Multiprogramming arrangements
    • G06F9/48—Program initiating; Program switching, e.g. by interrupt
    • G06F9/4806—Task transfer initiation or dispatching
    • G06F9/4843—Task transfer initiation or dispatching by program, e.g. task dispatcher, supervisor, operating system
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F21/00—Security arrangements for protecting computers, components thereof, programs or data against unauthorised activity
    • G06F21/60—Protecting data
    • G06F21/62—Protecting access to data via a platform, e.g. using keys or access control rules
    • G06F21/6218—Protecting access to data via a platform, e.g. using keys or access control rules to a system of files or objects, e.g. local or distributed file system or database
    • G06F21/6245—Protecting personal data, e.g. for financial or medical purposes
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00—Arrangements for program control, e.g. control units
    • G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/44—Arrangements for executing specific programs
    • G06F9/445—Program loading or initiating
    • G06F9/44505—Configuring for program initiating, e.g. using registry, configuration files
    • Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
    • Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
    • Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management

Landscapes

  • Engineering & Computer Science (AREA)
  • Software Systems (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Health & Medical Sciences (AREA)
  • Bioethics (AREA)
  • General Health & Medical Sciences (AREA)
  • Medical Informatics (AREA)
  • Databases & Information Systems (AREA)
  • Computer Hardware Design (AREA)
  • Computer Security & Cryptography (AREA)
  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)

Abstract

The invention discloses a data desensitizing method and device, and relates to the technical field of big data. One embodiment of the method comprises the following steps: determining a data desensitization flow according to the data desensitization request, and acquiring at least one data processing step included in the data desensitization flow, wherein the data processing step has a step type; according to the step types and the dimension of the execution sequence, carrying out combination processing on the data processing steps to generate a data processing step combination; for each data processing step combination, acquiring configuration information of the corresponding data processing step, and generating an operation instruction of the data processing step combination through the configuration information; and executing the operation instruction of each data processing step combination according to the data desensitization flow so as to perform data desensitization. According to the embodiment, the data desensitization can be performed through the combination of the data processing steps, so that the full-flow automation and the higher configurable degree of the data desensitization are realized, the research and development cost is reduced, and the multiplexing rate is improved.

Description

Data desensitization method and device
Technical Field
The invention relates to the technical field of big data, in particular to a data desensitizing method and device.
Background
In order to avoid the problem of personal privacy information disclosure caused by using production data in system development tests, the production data needs to be subjected to data desensitization. The scheme of data desensitization at present is that for different data desensitization services, research and development are required to be repeated according to service requirements, and then data desensitization is carried out.
In the process of implementing the present invention, the inventor finds that at least the following problems exist in the prior art:
the research and development cost is high, the multiplexing rate is low, and the full-flow automation and configuration of data desensitization cannot be realized.
Disclosure of Invention
In view of the above, the embodiment of the invention provides a data desensitizing method and device, which can perform data desensitization through the combination of data processing steps, realize full-flow automation and higher configurable degree of data desensitization, reduce research and development cost and improve multiplexing rate.
To achieve the above object, according to one aspect of an embodiment of the present invention, there is provided a data desensitizing method.
A method of desensitizing data, comprising: determining a data desensitization flow according to a data desensitization request, and acquiring at least one data processing step included in the data desensitization flow, wherein the data desensitization flow also comprises an execution sequence of the data processing steps, and the data processing steps have step types; according to the step types and the dimension of the execution sequence, carrying out combination processing on the data processing steps to generate a data processing step combination; for each data processing step combination, acquiring configuration information of the corresponding data processing step, and generating an operation instruction of the data processing step combination through the configuration information; and executing the operation instruction of each data processing step combination according to the data desensitization flow so as to perform data desensitization.
Optionally, the step combination processing is performed on the data processing steps according to the step type and the dimension of the execution sequence, so as to generate a data processing step combination, which includes: selecting at least one data processing step to be combined from the data processing steps according to the step types and the dimension of the execution sequence, wherein the data processing steps to be combined are the data processing steps with the same step types and the same execution sequence; and carrying out combination processing on the data processing steps to be combined to generate the data processing step combination.
Optionally, in the case that the combination of data processing steps includes a data desensitizing step, the acquiring configuration information of the corresponding data processing step, generating, by using the configuration information, an operation instruction of the combination of data processing steps includes: acquiring configuration information of a data desensitization step included in the data processing step combination, wherein the configuration information of the data desensitization step comprises a desensitization database identifier, a database table identifier to be desensitized, a data desensitization field and a corresponding data desensitization rule; and generating an operation instruction of the data processing step combination according to the configuration information of the data desensitizing step.
Optionally, in the case that the data processing step combination includes a data unloading step, the acquiring configuration information of the corresponding data processing step, generating, by using the configuration information, an operation instruction of the data processing step combination includes: acquiring configuration information of a data unloading step included in the data processing step combination, wherein the configuration information of the data unloading step comprises an unloading database identifier and a database table identifier to be unloaded; and generating an operation instruction of the data processing step combination according to the configuration information of the data unloading step.
Optionally, in the case that the data processing step combination includes a data loading step, the acquiring configuration information of the corresponding data processing step, generating, by using the configuration information, an operation instruction of the data processing step combination includes: acquiring configuration information of a data loading step included in the data processing step combination, wherein the configuration information of the data loading step comprises a loading database identifier and a database table identifier to be loaded; and generating an operation instruction of the data processing step combination according to the configuration information of the data loading step.
Optionally, the executing the operation instruction of each data processing step combination according to the data desensitization flow, so as to perform data desensitization, including: for each data processing step combination, determining the execution sequence of the data processing step combination according to the execution sequence of the data processing steps included in the data processing step combination; and executing the operation instructions of each data processing step combination according to the execution sequence so as to perform data desensitization.
Optionally, before the generating the operation instruction of the data processing step combination through the configuration information, the method further includes: determining the execution authority required for data desensitization according to each data processing step; and performing authority verification on the execution environment according to the execution authority, and determining that the verification passes.
According to another aspect of an embodiment of the present invention, there is provided a data desensitizing apparatus.
A data desensitizing apparatus comprising: the data processing step acquisition module is used for determining a data desensitization flow according to a data desensitization request and acquiring at least one data processing step included in the data desensitization flow, wherein the data desensitization flow also comprises the execution sequence of the data processing steps, and the data processing steps are of step types; the data processing step combination generating module is used for carrying out combination processing on the data processing steps according to the step types and the dimension of the execution sequence to generate a data processing step combination; the operation instruction generation module is used for acquiring configuration information of the corresponding data processing steps for each data processing step combination, and generating operation instructions of the data processing step combination through the configuration information; and the data desensitization module is used for executing the operation instructions of the data processing step combinations according to the data desensitization flow so as to perform data desensitization.
Optionally, the data processing step combination generating module is further configured to: selecting at least one data processing step to be combined from the data processing steps according to the step types and the dimension of the execution sequence, wherein the data processing steps to be combined are the data processing steps with the same step types and the same execution sequence; and carrying out combination processing on the data processing steps to be combined to generate the data processing step combination.
Optionally, in the case that the combination of data processing steps includes a data desensitizing step, the operation instruction generating module is further configured to: acquiring configuration information of a data desensitization step included in the data processing step combination, wherein the configuration information of the data desensitization step comprises a desensitization database identifier, a database table identifier to be desensitized, a data desensitization field and a corresponding data desensitization rule; and generating an operation instruction of the data processing step combination according to the configuration information of the data desensitizing step.
Optionally, in the case that the combination of data processing steps includes a data unloading step, the operation instruction generation module is further configured to: acquiring configuration information of a data unloading step included in the data processing step combination, wherein the configuration information of the data unloading step comprises an unloading database identifier and a database table identifier to be unloaded; and generating an operation instruction of the data processing step combination according to the configuration information of the data unloading step.
Optionally, in the case that the combination of data processing steps includes a data loading step, the operation instruction generation module is further configured to: acquiring configuration information of a data loading step included in the data processing step combination, wherein the configuration information of the data loading step comprises a loading database identifier and a database table identifier to be loaded; and generating an operation instruction of the data processing step combination according to the configuration information of the data loading step.
Optionally, the data desensitization module is further configured to: for each data processing step combination, determining the execution sequence of the data processing step combination according to the execution sequence of the data processing steps included in the data processing step combination; and executing the operation instructions of each data processing step combination according to the execution sequence so as to perform data desensitization.
Optionally, the system further comprises a permission verification module for: determining the execution authority required for data desensitization according to each data processing step; and performing authority verification on the execution environment according to the execution authority, and determining that the verification passes.
According to yet another aspect of an embodiment of the present invention, an electronic device is provided.
An electronic device, comprising: one or more processors; and a memory for storing one or more programs which, when executed by the one or more processors, cause the one or more processors to implement the data desensitization method provided by the embodiments of the present invention.
According to yet another aspect of an embodiment of the present invention, a computer-readable medium is provided.
A computer readable medium having stored thereon a computer program which, when executed by a processor, implements a data desensitization method provided by embodiments of the present invention.
According to yet another aspect of an embodiment of the present invention, a computer program product is provided.
A computer program product comprising a computer program which when executed by a processor implements a data desensitization method provided by embodiments of the invention.
One embodiment of the above invention has the following advantages or benefits: determining a data desensitization flow according to the data desensitization request, and acquiring at least one data processing step included in the data desensitization flow, wherein the data desensitization flow also includes an execution sequence of the data processing steps, and the data processing steps have step types; according to the step types and the dimension of the execution sequence, carrying out combination processing on the data processing steps to generate a data processing step combination; for each data processing step combination, acquiring configuration information of the corresponding data processing step, and generating an operation instruction of the data processing step combination through the configuration information; according to the data desensitization flow, the operation instructions of the data processing step combination are executed to perform the technical scheme of data desensitization, the data desensitization can be performed through the data processing step combination, the full-flow automation and the higher configurable degree of the data desensitization are realized, the research and development cost is reduced, and the multiplexing rate is improved.
Further effects of the above-described non-conventional alternatives are described below in connection with the embodiments.
Drawings
The drawings are included to provide a better understanding of the invention and are not to be construed as unduly limiting the invention. Wherein:
FIG. 1 is a schematic diagram of the main steps of a data desensitization method according to an embodiment of the invention;
FIG. 2 is a flow diagram of a method of data desensitization according to an embodiment of the invention;
FIG. 3 is a schematic diagram of the main modules of a data desensitizing apparatus according to an embodiment of the present invention;
FIG. 4 is an exemplary system architecture diagram in which embodiments of the present invention may be applied;
fig. 5 is a schematic diagram of a computer system suitable for use in implementing an embodiment of the invention.
Detailed Description
Exemplary embodiments of the present invention will now be described with reference to the accompanying drawings, in which various details of the embodiments of the present invention are included to facilitate understanding, and are to be considered merely exemplary. Accordingly, those of ordinary skill in the art will recognize that various changes and modifications of the embodiments described herein can be made without departing from the scope and spirit of the invention. Also, descriptions of well-known functions and constructions are omitted in the following description for clarity and conciseness.
In the technical scheme of the invention, the aspects of the related personal information of the user, such as acquisition, collection, updating, analysis, processing, use, transmission, storage and the like, all conform to the rules of related laws and regulations, are used for legal purposes, and do not violate the popular public order. Necessary measures are taken for the personal information of the user, illegal access to the personal information data of the user is prevented, and the personal information security, network security and national security of the user are maintained.
FIG. 1 is a schematic diagram of the main steps of a data desensitization method according to an embodiment of the present invention.
As shown in fig. 1, the data desensitizing method according to an embodiment of the present invention mainly includes the following steps S101 to S104.
Step S101: the data desensitization process is determined according to the data desensitization request, at least one data processing step included in the data desensitization process is acquired, the data desensitization process can also include an execution sequence of the data processing steps, and the data processing steps can have step types.
The data processing steps can be divided into a data desensitizing step, a data loading step and a data unloading step according to the step types, and the execution sequence of the data processing steps can be marked by an execution sequence mark. The data desensitization request may include a pre-written automated desensitization flow configuration file, where the automated desensitization flow configuration file includes configuration information for the data desensitization flow and each data processing step.
Step S102: and carrying out combination processing on the data processing steps according to the step types and the dimension of the execution sequence to generate a data processing step combination.
In one embodiment, the combination of data processing steps according to the step type and dimension of the execution sequence, generating a combination of data processing steps, may include: according to the step types and the dimension of the execution sequence, selecting at least one data processing step to be combined from the data processing steps, wherein the data processing steps to be combined are the data processing steps with the same step types and the same execution sequence; and carrying out combination processing on the data processing steps to be combined to generate a data processing step combination.
Specifically, the step types and the execution sequences of the data processing steps included in the data desensitization flow are acquired, the data processing steps are classified according to the step types and the execution sequences, and the step types and the execution sequences of the data processing steps in each class are the same. For each category, selecting at least one data processing step to be combined, and carrying out combination processing on the data processing steps to be combined to generate a data processing step combination.
In another embodiment, the step type and execution order of each data processing step included in the data desensitization procedure are acquired, the data processing steps are classified according to the step type and execution order, and the step type and execution order of the data processing steps in each class are the same. And selecting at least one data processing step to be combined from one or more categories with different step types, and carrying out combination processing on the data processing steps to be combined to generate a data processing step combination.
In one embodiment, before generating the operation instruction of the data processing step combination through the configuration information, the method may further include: determining the execution authority required for data desensitization according to each data processing step; and performing authority verification on the execution environment according to the execution authority, and determining that the verification passes.
In particular, the execution rights may include data loading rights, data unloading rights, and data readable rights, data writable rights, and the like. The execution environment may be jvm (virtual machine), database client, linux shell (user interface of system), etc.
Step S103: for each data processing step combination, configuration information of the corresponding data processing step is obtained, and an operation instruction of the data processing step combination is generated through the configuration information.
In one embodiment, in the case that the combination of data processing steps includes a data desensitizing step, acquiring configuration information of the corresponding data processing step, generating an operation instruction of the combination of data processing steps by the configuration information may include: acquiring configuration information of a data desensitization step included in the data processing step combination, wherein the configuration information of the data desensitization step comprises a desensitization database identifier, a database table identifier to be desensitized, a data desensitization field and a corresponding data desensitization rule; and generating operation instructions of the data processing step combination according to the configuration information of the data desensitizing step. The desensitization refers to the data deformation of certain sensitive information through a desensitization rule, so that the sensitive privacy data is reliably protected.
Specifically, an operation instruction is generated according to the desensitization database identifier, the database table identifier to be desensitized, the data desensitization field and the corresponding data desensitization rule. The operation instruction is used for acquiring a database table to be desensitized corresponding to the database table identifier to be desensitized from a desensitization database corresponding to the database identifier to be desensitized, and desensitizing data desensitization fields in the database table to be desensitized according to the data desensitization rule. The data desensitization rule comprises an identification card number desensitization rule, a mobile phone number desensitization rule, a name desensitization rule, an address desensitization rule, a transaction detail desensitization rule and the like, wherein the identification card number desensitization rule is used for desensitizing 18 digits, the mobile phone number desensitization rule is used for desensitizing 11 digits, and the name desensitization rule is used for reserving surnames and desensitizing the first names to Chinese digits. The configuration can be carried out through the desensitization rule number, and the configuration of the desensitization rule can be flexibly adjusted aiming at data entities with different structures.
In one embodiment, in a case where the combination of data processing steps includes a data unloading step, acquiring configuration information of the corresponding data processing step, generating an operation instruction of the combination of data processing steps by the configuration information may include: acquiring configuration information of a data unloading step included in a data processing step combination, wherein the configuration information of the data unloading step comprises an unloading database identifier and a database table identifier to be unloaded; and generating an operation instruction of the data processing step combination according to the configuration information of the data unloading step. Wherein data offloading refers to the process of exporting data from a software product, such as a database.
Specifically, an operation instruction is generated according to configuration information such as an unloading database identifier, a database table identifier to be unloaded and the like. The operation instruction is used for reading connection information of a source database (namely an unloading database), automatically logging in the database, and exporting a database table to be unloaded into file data by using database export tools such as a data pump, an sqlload (a data loading tool) and the like according to the identification of the database table to be unloaded.
In one embodiment, in the case that the combination of data processing steps includes the data loading step, acquiring configuration information of the corresponding data processing step, generating the operation instruction of the combination of data processing steps through the configuration information may include: acquiring configuration information of a data loading step included in a data processing step combination, wherein the configuration information of the data loading step comprises a loading database identifier and a database table identifier to be loaded; and generating an operation instruction of the data processing step combination according to the configuration information of the data loading step. Where data loading refers to the process of importing data into a software product such as a database.
Specifically, an operation instruction is generated according to configuration information such as a loading database identifier, a database table identifier to be loaded and the like. The operation instruction is used for reading the connection information of the database (i.e. the loading database) at the target end, automatically logging in the database, and importing the data file into the database by using database importing tools such as a data pump, an sqlload and the like according to the table identification of the database to be loaded. If the database has established a variant structure of the relevant data entity, only data may be optionally imported, and if the database is established a variant structure of the relevant data entity, both data structures and data may be optionally imported.
The database identifier and the database table identifier can be unique identifiers, time identifiers or classified identifiers, and the embodiment of the invention supports fuzzy configuration of the database identifier and the database table identifier.
In one embodiment, the combination of data processing steps may include a data loading step, a data unloading step and a data desensitizing step, the designated database table is obtained through the data unloading step, the designated database table is desensitized through the data desensitizing step, the desensitized designated database table is loaded through the data loading step, so that the designated database table can be desensitized using the combination of data processing steps, and multiplexing of the combination of data processing steps can be achieved.
Step S104: and executing the operation instruction of each data processing step combination according to the data desensitization flow so as to perform data desensitization.
In one embodiment, executing the operation instructions of each combination of data processing steps to perform data desensitization according to a data desensitization procedure may include: for each data processing step combination, determining the execution sequence of the data processing step combination according to the execution sequence of the data processing steps included in the data processing step combination; the operation instructions of the combination of the data processing steps are executed in the execution order to perform data desensitization.
Specifically, the execution order of the combination of data processing steps may be determined according to the execution order of the data processing steps included therein, or may be determined according to the degree of server usage and the execution order of the combination of other data processing steps.
In one embodiment, each combination of data processing steps and data processing steps may be re-executed by themselves in the event of execution failure until execution is successful.
Fig. 2 is a flow diagram of a method of data desensitization according to an embodiment of the invention.
As shown in FIG. 2, in one embodiment, an automation flow configuration file for a full flow is written according to data desensitization requirements, generating operational instructions. And initializing in a production environment, and executing data unloading step combination to obtain a production data original file. In a desensitization environment, the data desensitization step combination is executed to perform data automation desensitization, and a desensitized data file is generated.
The embodiment of the invention realizes an automatic configurable data desensitization method, realizes full-flow automatic configurable of source data unloading, data loading and data desensitization through language writing such as java (a lightweight programming language) and shell (a programming language) storage processes, simultaneously can realize automatic retry of errors of all steps aiming at the characteristics of large data volume, complex table structure and number checking of a financial system, can select independent execution or multi-step free combination execution of all steps, also supports diversified configuration processing modes such as processing according to a whole library, processing according to a single table, processing according to a designated partition, classifying according to fuzzy matching table names and the like, and greatly improves the automatic configurable degree of the data desensitization of the financial system.
Fig. 3 is a schematic diagram of the main modules of the data desensitizing apparatus according to an embodiment of the present invention.
As shown in fig. 3, the data desensitizing apparatus 300 according to an embodiment of the present invention mainly includes: the device comprises a data processing step acquisition module 301, a data processing step combination generation module 302, an operation instruction generation module 303 and a data desensitization module 304.
The data processing step obtaining module 301 is configured to determine a data desensitization procedure according to the data desensitization request, and obtain at least one data processing step included in the data desensitization procedure, where the data desensitization procedure may further include an execution sequence of the data processing steps, and the data processing steps may have a step type.
The data processing step combination generating module 302 is configured to perform combination processing on the data processing steps according to the step types and the dimension of the execution sequence, and generate a data processing step combination.
The operation instruction generating module 303 is configured to, for each combination of data processing steps, obtain configuration information of the corresponding data processing step, and generate an operation instruction of the combination of data processing steps according to the configuration information.
The data desensitizing module 304 is configured to execute the operation instruction of each data processing step combination according to the data desensitizing procedure, so as to perform data desensitization.
In one embodiment, the data processing step combination generation module 302 is specifically configured to: according to the step types and the dimension of the execution sequence, selecting at least one data processing step to be combined from the data processing steps, wherein the data processing steps to be combined are the data processing steps with the same step types and the same execution sequence; and carrying out combination processing on the data processing steps to be combined to generate a data processing step combination.
In one embodiment, where the combination of data processing steps includes a data desensitization step, the operational instruction generation module 303 is specifically configured to: acquiring configuration information of a data desensitization step included in the data processing step combination, wherein the configuration information of the data desensitization step comprises a desensitization database identifier, a database table identifier to be desensitized, a data desensitization field and a corresponding data desensitization rule; and generating operation instructions of the data processing step combination according to the configuration information of the data desensitizing step.
In one embodiment, in the case where the combination of data processing steps includes a data offloading step, the operation instruction generation module 303 is specifically configured to: acquiring configuration information of a data unloading step included in a data processing step combination, wherein the configuration information of the data unloading step comprises an unloading database identifier and a database table identifier to be unloaded; and generating an operation instruction of the data processing step combination according to the configuration information of the data unloading step.
In one embodiment, in the case where the combination of data processing steps includes a data loading step, the operation instruction generation module 303 is specifically configured to: acquiring configuration information of a data loading step included in a data processing step combination, wherein the configuration information of the data loading step comprises a loading database identifier and a database table identifier to be loaded; and generating an operation instruction of the data processing step combination according to the configuration information of the data loading step.
In one embodiment, the data desensitization module 304 is specifically configured to: for each data processing step combination, determining the execution sequence of the data processing step combination according to the execution sequence of the data processing steps included in the data processing step combination; the operation instructions of the combination of the data processing steps are executed in the execution order to perform data desensitization.
In one embodiment, the system may further include a rights verification module (not shown in the figure) for: determining the execution authority required for data desensitization according to each data processing step; and performing authority verification on the execution environment according to the execution authority, and determining that the verification passes.
In addition, the specific implementation of the data desensitizing apparatus in the embodiment of the present invention has been described in detail in the above data desensitizing method, so the description will not be repeated here.
Fig. 4 illustrates an exemplary system architecture 400 in which a data desensitization method or data desensitization apparatus of embodiments of the present invention may be applied.
As shown in fig. 4, the system architecture 400 may include terminal devices 401, 402, 403, a network 404, and a server 405. The network 404 is used as a medium to provide communication links between the terminal devices 401, 402, 403 and the server 405. The network 404 may include various connection types, such as wired, wireless communication links, or fiber optic cables, among others.
A user may interact with the server 405 via the network 404 using the terminal devices 401, 402, 403 to receive or send messages or the like. Various communication client applications, such as a data desensitization class application, a data processing application, a configuration class application, an instant messaging tool, a mailbox client, social platform software, etc. (for example only) may be installed on the terminal devices 401, 402, 403.
The terminal devices 401, 402, 403 may be various electronic devices having a display screen and supporting web browsing, including but not limited to smartphones, tablets, laptop and desktop computers, and the like.
The server 405 may be a server providing various services, such as a background management server (by way of example only) providing support for data desensitization class websites browsed by users using the terminal devices 401, 402, 403. The background management server can determine a data desensitization flow according to the received data desensitization request and the like, and acquire at least one data processing step included in the data desensitization flow, wherein the data desensitization flow also includes an execution sequence of the data processing steps, and the data processing steps have step types; according to the step types and the dimension of the execution sequence, carrying out combination processing on the data processing steps to generate a data processing step combination; for each data processing step combination, acquiring configuration information of the corresponding data processing step, and generating an operation instruction of the data processing step combination through the configuration information; according to the data desensitization flow, the operation instructions of each data processing step combination are executed to perform data desensitization and other processing, and the processing result (such as the data desensitization result—only an example) is fed back to the terminal device.
It should be noted that, the data desensitizing method provided by the embodiment of the present invention is generally performed by the server 405, and accordingly, the data desensitizing device is generally disposed in the server 405.
It should be understood that the number of terminal devices, networks and servers in fig. 4 is merely illustrative. There may be any number of terminal devices, networks, and servers, as desired for implementation.
Referring now to FIG. 5, there is illustrated a schematic diagram of a computer system 500 suitable for use in implementing a terminal device or server in accordance with an embodiment of the present invention. The terminal device or server shown in fig. 5 is only an example, and should not impose any limitation on the functions and scope of use of the embodiments of the present invention.
As shown in fig. 5, the computer system 500 includes a Central Processing Unit (CPU) 501, which can perform various appropriate actions and processes according to a program stored in a Read Only Memory (ROM) 502 or a program loaded from a storage section 508 into a Random Access Memory (RAM) 503. In the RAM 503, various programs and data required for the operation of the system 500 are also stored. The CPU 501, ROM 502, and RAM 503 are connected to each other through a bus 504. An input/output (I/O) interface 505 is also connected to bus 504.
The following components are connected to the I/O interface 505: an input section 506 including a keyboard, a mouse, and the like; an output portion 507 including a Cathode Ray Tube (CRT), a Liquid Crystal Display (LCD), and the like, and a speaker, and the like; a storage portion 508 including a hard disk and the like; and a communication section 509 including a network interface card such as a LAN card, a modem, or the like. The communication section 509 performs communication processing via a network such as the internet. The drive 510 is also connected to the I/O interface 505 as needed. A removable medium 511 such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, or the like is mounted on the drive 510 as needed so that a computer program read therefrom is mounted into the storage section 508 as needed.
In particular, according to embodiments of the present disclosure, the processes described above with reference to flowcharts may be implemented as computer software programs. For example, embodiments of the present disclosure include a computer program product comprising a computer program embodied on a computer readable medium, the computer program comprising program code for performing the method shown in the flow chart. In such an embodiment, the computer program may be downloaded and installed from a network via the communication portion 509, and/or installed from the removable media 511. The above-described functions defined in the system of the present invention are performed when the computer program is executed by a Central Processing Unit (CPU) 501.
The computer readable medium shown in the present invention may be a computer readable signal medium or a computer readable storage medium, or any combination of the two. The computer readable storage medium can be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or a combination of any of the foregoing. More specific examples of the computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In the context of this document, a computer readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device. In the present invention, however, the computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, with the computer-readable program code embodied therein. Such a propagated data signal may take any of a variety of forms, including, but not limited to, electro-magnetic, optical, or any suitable combination of the foregoing. A computer readable signal medium may also be any computer readable medium that is not a computer readable storage medium and that can communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device. Program code embodied on a computer readable medium may be transmitted using any appropriate medium, including but not limited to: wireless, wire, fiber optic cable, RF, etc., or any suitable combination of the foregoing.
The flowcharts and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments of the present invention. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that, in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams or flowchart illustration, and combinations of blocks in the block diagrams or flowchart illustration, can be implemented by special purpose hardware-based systems which perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.
The modules involved in the embodiments of the present invention may be implemented in software or in hardware. The described modules may also be provided in a processor, for example, as: the processor comprises a data processing step acquisition module, a data processing step combination generation module, an operation instruction generation module and a data desensitization module. The names of these modules do not constitute limitations on the module itself in some cases, and for example, the data processing step acquisition module may also be described as "a module for determining a data desensitization flow from a data desensitization request and acquiring not less than one data processing step included in the data desensitization flow".
As another aspect, the present invention also provides a computer-readable medium that may be contained in the apparatus described in the above embodiments; or may be present alone without being fitted into the device. The computer readable medium carries one or more programs which, when executed by a device, cause the device to include: determining a data desensitization flow according to the data desensitization request, and acquiring at least one data processing step included in the data desensitization flow, wherein the data desensitization flow also comprises an execution sequence of the data processing steps, and the data processing steps have step types; according to the step types and the dimension of the execution sequence, carrying out combination processing on the data processing steps to generate a data processing step combination; for each data processing step combination, acquiring configuration information of the corresponding data processing step, and generating an operation instruction of the data processing step combination through the configuration information; and executing the operation instruction of each data processing step combination according to the data desensitization flow so as to perform data desensitization.
As a further aspect, the invention also provides a computer program product comprising a computer program which, when executed by a processor, implements a data desensitisation method of an embodiment of the invention.
According to the technical scheme of the embodiment of the invention, a data desensitization flow is determined according to the data desensitization request, at least one data processing step included in the data desensitization flow is acquired, the data desensitization flow also includes the execution sequence of the data processing steps, and the data processing steps have step types; according to the step types and the dimension of the execution sequence, carrying out combination processing on the data processing steps to generate a data processing step combination; for each data processing step combination, acquiring configuration information of the corresponding data processing step, and generating an operation instruction of the data processing step combination through the configuration information; and executing the operation instruction of each data processing step combination according to the data desensitization flow so as to perform data desensitization. The data desensitization can be performed through the combination of the data processing steps, so that the full-flow automation and higher configurable degree of the data desensitization are realized, the research and development cost is reduced, and the multiplexing rate is improved.
The above embodiments do not limit the scope of the present invention. It will be apparent to those skilled in the art that various modifications, combinations, sub-combinations and alternatives can occur depending upon design requirements and other factors. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the scope of the present invention.

Claims (13)

1. A method of desensitizing data, comprising:
determining a data desensitization flow according to a data desensitization request, and acquiring at least one data processing step included in the data desensitization flow, wherein the data desensitization flow also comprises an execution sequence of the data processing steps, and the data processing steps have step types;
according to the step types and the dimension of the execution sequence, carrying out combination processing on the data processing steps to generate a data processing step combination;
for each data processing step combination, acquiring configuration information of the corresponding data processing step, and generating an operation instruction of the data processing step combination through the configuration information;
and executing the operation instruction of each data processing step combination according to the data desensitization flow so as to perform data desensitization.
2. The method of claim 1, wherein the step of combining the data processing steps in terms of step type and dimension of execution order to generate a combination of data processing steps comprises:
selecting at least one data processing step to be combined from the data processing steps according to the step types and the dimension of the execution sequence, wherein the data processing steps to be combined are the data processing steps with the same step types and the same execution sequence;
and carrying out combination processing on the data processing steps to be combined to generate the data processing step combination.
3. The method according to claim 1, wherein, in case the combination of data processing steps includes a data desensitizing step, the acquiring configuration information of the corresponding data processing step, generating an operation instruction of the combination of data processing steps from the configuration information, includes:
acquiring configuration information of a data desensitization step included in the data processing step combination, wherein the configuration information of the data desensitization step comprises a desensitization database identifier, a database table identifier to be desensitized, a data desensitization field and a corresponding data desensitization rule;
and generating an operation instruction of the data processing step combination according to the configuration information of the data desensitizing step.
4. The method according to claim 1, wherein, in the case where the combination of data processing steps includes a data unloading step, the acquiring configuration information of the corresponding data processing step, generating an operation instruction of the combination of data processing steps from the configuration information, includes:
acquiring configuration information of a data unloading step included in the data processing step combination, wherein the configuration information of the data unloading step comprises an unloading database identifier and a database table identifier to be unloaded;
and generating an operation instruction of the data processing step combination according to the configuration information of the data unloading step.
5. The method according to claim 1, wherein, in the case where the combination of data processing steps includes a data loading step, the acquiring configuration information of the corresponding data processing step, generating an operation instruction of the combination of data processing steps from the configuration information, includes:
acquiring configuration information of a data loading step included in the data processing step combination, wherein the configuration information of the data loading step comprises a loading database identifier and a database table identifier to be loaded;
and generating an operation instruction of the data processing step combination according to the configuration information of the data loading step.
6. The method of claim 1, wherein said executing the operation instructions of each of said combination of data processing steps according to said data desensitization procedure to perform data desensitization comprises:
for each data processing step combination, determining the execution sequence of the data processing step combination according to the execution sequence of the data processing steps included in the data processing step combination;
and executing the operation instructions of each data processing step combination according to the execution sequence so as to perform data desensitization.
7. The method of claim 1, further comprising, prior to generating the operation instructions for the combination of data processing steps from the configuration information:
determining the execution authority required for data desensitization according to each data processing step;
and performing authority verification on the execution environment according to the execution authority, and determining that the verification passes.
8. A data desensitizing apparatus, comprising:
the data processing step acquisition module is used for determining a data desensitization flow according to a data desensitization request and acquiring at least one data processing step included in the data desensitization flow, wherein the data desensitization flow also comprises the execution sequence of the data processing steps, and the data processing steps are of step types;
the data processing step combination generating module is used for carrying out combination processing on the data processing steps according to the step types and the dimension of the execution sequence to generate a data processing step combination;
the operation instruction generation module is used for acquiring configuration information of the corresponding data processing steps for each data processing step combination, and generating operation instructions of the data processing step combination through the configuration information;
and the data desensitization module is used for executing the operation instructions of the data processing step combinations according to the data desensitization flow so as to perform data desensitization.
9. The apparatus of claim 8, wherein the data processing step combination generation module is further configured to:
selecting at least one data processing step to be combined from the data processing steps according to the step types and the dimension of the execution sequence, wherein the data processing steps to be combined are the data processing steps with the same step types and the same execution sequence;
and carrying out combination processing on the data processing steps to be combined to generate the data processing step combination.
10. The apparatus of claim 8, wherein the data desensitizing module is further configured to:
for each data processing step combination, determining the execution sequence of the data processing step combination according to the execution sequence of the data processing steps included in the data processing step combination;
and executing the operation instructions of each data processing step combination according to the execution sequence so as to perform data desensitization.
11. An electronic device, comprising:
one or more processors;
storage means for storing one or more programs,
when executed by the one or more processors, causes the one or more processors to implement the method of any of claims 1-7.
12. A computer readable medium, on which a computer program is stored, characterized in that the computer program, when being executed by a processor, implements the method according to any of claims 1-7.
13. A computer program product comprising a computer program which, when executed by a processor, implements the method according to any of claims 1-7.
CN202310149597.5A 2023-02-22 2023-02-22 Data desensitization method and device Pending CN116302405A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202310149597.5A CN116302405A (en) 2023-02-22 2023-02-22 Data desensitization method and device

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202310149597.5A CN116302405A (en) 2023-02-22 2023-02-22 Data desensitization method and device

Publications (1)

Publication Number Publication Date
CN116302405A true CN116302405A (en) 2023-06-23

Family

ID=86837107

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202310149597.5A Pending CN116302405A (en) 2023-02-22 2023-02-22 Data desensitization method and device

Country Status (1)

Country Link
CN (1) CN116302405A (en)

Similar Documents

Publication Publication Date Title
CN107302597B (en) Message file pushing method and device
CN110704833A (en) Data permission configuration method, device, electronic device and storage medium
CN110706093A (en) Accounting processing method and device
CN112559024A (en) Method and device for generating transaction code change list
CN113076153A (en) Interface calling method and device
CN115168341A (en) Service processing method, system, medium and equipment
CN113282591B (en) Authority filtering method, authority filtering device, computer equipment and storage medium
CN108628909B (en) Information pushing method and device
CN115277857A (en) Method and device for interface verification, electronic equipment and storage medium
CN111460273B (en) Information pushing method and device
CN116302405A (en) Data desensitization method and device
CN112579428B (en) Interface testing method, device, electronic equipment and storage medium
CN112732471B (en) Error correction method and error correction device for interface return data
CN113704222A (en) Method and device for processing service request
CN109656519B (en) Method and device for automatically accessing service data
CN113448602A (en) Version updating method and device
CN112261072A (en) Service calling method, device, equipment and storage medium
CN111209014A (en) Parameter checking method and device
CN110661792B (en) Service ordered calling method and device, electronic equipment and storage medium
CN113900895B (en) Information processing method, information processing apparatus, storage medium, and electronic device
CN111724244B (en) Objection error correction method and device
CN116610295A (en) Interface service providing method, device, electronic equipment, system and storage medium
CN112784247A (en) Authority verification method and device for application program
CN113760921A (en) Method and device for creating dictionary value
CN113779078A (en) Cache data acquisition method and device

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination