CN110795756A - Data desensitization method and device, computer equipment and computer readable storage medium - Google Patents
Data desensitization method and device, computer equipment and computer readable storage medium Download PDFInfo
- Publication number
- CN110795756A CN110795756A CN201910910381.XA CN201910910381A CN110795756A CN 110795756 A CN110795756 A CN 110795756A CN 201910910381 A CN201910910381 A CN 201910910381A CN 110795756 A CN110795756 A CN 110795756A
- Authority
- CN
- China
- Prior art keywords
- data
- desensitization
- target
- target data
- security level
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Withdrawn
Links
- 238000000586 desensitisation Methods 0.000 title claims abstract description 270
- 238000000034 method Methods 0.000 title claims abstract description 55
- 238000004519 manufacturing process Methods 0.000 claims abstract description 26
- 238000005070 sampling Methods 0.000 claims description 11
- 238000013507 mapping Methods 0.000 claims description 8
- 238000004590 computer program Methods 0.000 claims description 5
- 230000014509 gene expression Effects 0.000 description 5
- 238000012423 maintenance Methods 0.000 description 5
- 238000012545 processing Methods 0.000 description 5
- 238000010586 diagram Methods 0.000 description 4
- 238000004422 calculation algorithm Methods 0.000 description 3
- 238000004891 communication Methods 0.000 description 3
- 238000005516 engineering process Methods 0.000 description 3
- 230000006870 function Effects 0.000 description 3
- 238000004458 analytical method Methods 0.000 description 2
- 239000003999 initiator Substances 0.000 description 2
- 230000003068 static effect Effects 0.000 description 2
- 238000013459 approach Methods 0.000 description 1
- 230000001174 ascending effect Effects 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000019771 cognition Effects 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 238000000605 extraction Methods 0.000 description 1
- 230000003993 interaction Effects 0.000 description 1
- 238000010295 mobile communication Methods 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 230000008569 process Effects 0.000 description 1
- 230000008707 rearrangement Effects 0.000 description 1
- 230000004044 response Effects 0.000 description 1
- 239000007787 solid Substances 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F21/00—Security arrangements for protecting computers, components thereof, programs or data against unauthorised activity
- G06F21/60—Protecting data
- G06F21/62—Protecting access to data via a platform, e.g. using keys or access control rules
- G06F21/6218—Protecting access to data via a platform, e.g. using keys or access control rules to a system of files or objects, e.g. local or distributed file system or database
- G06F21/6245—Protecting personal data, e.g. for financial or medical purposes
- G06F21/6254—Protecting personal data, e.g. for financial or medical purposes by anonymising data, e.g. decorrelating personal data from the owner's identification
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/80—Information retrieval; Database structures therefor; File system structures therefor of semi-structured data, e.g. markup language structured data such as SGML, XML or HTML
- G06F16/81—Indexing, e.g. XML tags; Data structures therefor; Storage structures
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Databases & Information Systems (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- Software Systems (AREA)
- Health & Medical Sciences (AREA)
- Bioethics (AREA)
- General Health & Medical Sciences (AREA)
- Data Mining & Analysis (AREA)
- Medical Informatics (AREA)
- Computer Hardware Design (AREA)
- Computer Security & Cryptography (AREA)
- Storage Device Security (AREA)
Abstract
The invention discloses a data desensitization method, a data desensitization device, computer equipment and a computer readable storage medium, wherein the method comprises the following steps: in a production environment, receiving a data access request, wherein the data access request is used for indicating that a requester requests to access target data in a data server; judging whether to perform dynamic desensitization on the target data; if the target data is judged to be subjected to dynamic desensitization, reading the target data, and performing dynamic desensitization on the target data to obtain desensitization data; desensitization data is fed back to the requestor. In a production environment, the server can perform dynamic desensitization on target data requested by a request party, and the desensitized data does not need to be stored in the server for a long time, so that the storage space of the server is released, and the resource utilization rate is improved. Meanwhile, dynamic desensitization can be achieved without being separated from a production environment, real-time desensitization of the query and calling results of sensitive data is achieved, and the returned data is enabled to be usable and safe.
Description
Technical Field
The embodiments of the present invention relate to data desensitization technologies, and in particular, to a data desensitization method, apparatus, computer device, and computer-readable storage medium.
Background
With the continuous development of information technology, data desensitization technology is applied to more and more fields. Most of the currently used data desensitization modes are desensitization of original data at a server side to obtain desensitization data, and the original data and the desensitization data are stored. When the user accesses sensitive data, desensitization data is sent to the user.
However, with the coming of big data era, more and more data are stored on the network side, desensitized data occupy a large amount of storage resources, and the resource utilization rate is low.
Disclosure of Invention
The invention provides a data desensitization method, a data desensitization device, computer equipment and a computer readable storage medium, which are used for realizing dynamic desensitization on target data and improving the resource utilization rate.
In a first aspect, an embodiment of the present invention provides a data desensitization method, including:
in a production environment, receiving a data access request, wherein the data access request is used for indicating that a requester requests to access target data in a data server;
judging whether to perform dynamic desensitization on the target data;
if the target data is judged to be subjected to dynamic desensitization, reading the target data, and performing dynamic desensitization on the target data to obtain desensitization data;
desensitization data is fed back to the requestor.
In a second aspect, an embodiment of the present invention further provides a data desensitization apparatus, including: the system comprises a data platform, a desensitization judgment engine and a dynamic desensitization engine, wherein:
the data platform is used for receiving a data access request in the production environment, wherein the data access request is used for indicating that a requester requests to access target data in the data server;
the desensitization judgment engine is used for judging whether to perform dynamic desensitization on the target data;
the dynamic desensitization engine is used for reading the target data and performing dynamic desensitization on the target data to obtain desensitization data if the desensitization judgment engine judges that the dynamic desensitization is performed on the target data;
and the data platform is used for feeding back desensitization data obtained by the dynamic desensitization engine to the requester.
In a third aspect, an embodiment of the present invention further provides a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, where the processor executes the program to implement the data desensitization method according to the first aspect.
In a fourth aspect, the embodiment of the present invention further provides a computer-readable storage medium, on which a computer program is stored, which when executed by a processor implements the data desensitization method according to the first aspect.
In the data desensitization method, the data desensitization device, the computer equipment and the computer readable storage medium provided by the embodiment of the invention, in a production environment, a data access request is received, wherein the data access request is used for indicating that a requester requests to access target data in a data server; judging whether to perform dynamic desensitization on the target data; if the target data is judged to be subjected to dynamic desensitization, reading the target data, and performing dynamic desensitization on the target data to obtain desensitization data; desensitization data is fed back to the requestor. Compared with the prior art that desensitization data needs to be stored in the server, so that the desensitization data occupies a large amount of system resources, the data desensitization scheme provided by the embodiment of the invention can perform dynamic desensitization on target data requested by a requester by the server in a production environment, and the desensitized data does not need to be stored in the server for a long time, so that the storage space of the server is released, and the resource utilization rate is improved. Meanwhile, dynamic desensitization can be achieved without being separated from a production environment, real-time desensitization of the query and calling results of sensitive data is achieved, and the returned data is enabled to be usable and safe.
Drawings
FIG. 1 is a flow chart of a data desensitization method according to a first embodiment of the invention;
FIG. 2 is a flow chart of a data desensitization method according to a second embodiment of the present invention;
FIG. 3 is a flow chart of a data desensitization method according to a third embodiment of the invention;
FIG. 4 is a flow chart of a data desensitization method according to a fourth embodiment of the present invention;
FIG. 5 is a schematic structural diagram of a data desensitization apparatus according to a fifth embodiment of the present invention;
fig. 6 is a schematic structural diagram of a computer device in the sixth embodiment of the present invention.
Detailed Description
The present invention will be described in further detail with reference to the accompanying drawings and examples. It is to be understood that the specific embodiments described herein are merely illustrative of the invention and are not limiting of the invention. It should be further noted that, for the convenience of description, only some of the structures related to the present invention are shown in the drawings, not all of the structures.
The existing desensitization mode is mainly static desensitization, during the static desensitization, data mask scrambling is carried out aiming at different data types through a plurality of desensitization algorithms such as shielding, deformation, replacement, random, Format Preserving Encryption (FPE) and strong encryption algorithm (such as AES), and desensitization data obtained by desensitization are stored in a server. When the user requests to access desensitization data, desensitization data which is desensitized is read from the server and fed back to the user. With the coming of big data era, more and more data are stored at the network side, desensitization data occupy a large amount of storage resources, and the resource utilization rate is low.
The embodiment of the invention provides a dynamic desensitization method, wherein a server does not locally store desensitized data, but carries out real-time desensitization on the desensitized data, and feeds the desensitized data back to a user, so that the utilization rate of a storage space of the server is improved. The present solution is specifically illustrated below by means of several examples:
example one
Fig. 1 is a flowchart of a data desensitization method according to an embodiment of the present invention, where this embodiment is applicable to a situation of performing data dynamic desensitization in a production environment, and the method may be executed by a server, and specifically includes the following steps:
in the production environment, a data access request is received, wherein the data access request is used for indicating that a requester requests to access target data in a data server.
Alternatively, the data access request may be initiated by an application on the terminal towards the server. For example, an application running on the terminal needs to access target data in the server, and the application sends a data access request to the server through a communication module of the terminal. At this time, the application program needs to present the target data, but the target data contains some sensitive data which needs to be hidden, at this time, the server cannot show all plaintext of the target data to the application, and after desensitization is performed on the target data, the desensitization data is fed back to the application on the terminal.
Optionally, the data access request may also be initiated by an operation and maintenance person. For example, the operation and maintenance personnel need to maintain the table structure, perform system tuning, and the like. At the moment, the operation and maintenance personnel can launch target data containing the form to the server through the terminal. The form contains sensitive data that needs to be suppressed and the attention of the operation and maintenance personnel is the form structure rather than the contents of the form. At this point, the operation and maintenance personnel should be avoided from retrieving or exporting the real data. Therefore, after desensitizing the target data, the server feeds the desensitized data back to the application on the terminal.
Illustratively, the data access request may also be issued by a business system. The data desensitization method provided by the application can be applied to a certain service system to ensure the data security of the service system. When other business systems perform data interaction with the business system, a data access request can be sent to the business system. When target data accessed by other service systems contains private data, desensitization processing needs to be carried out on the exchanged data, and desensitization data after desensitization is fed back to other service systems. In the conventional data desensitization mode, the server desensitizes all data, the desensitized data exist in the local storage space, and when other business systems access the sensitive data, the desensitized target data in the local storage space are read through the interface. This may cause desensitization data to stay in the server for a long time, and occupy a large amount of server storage resources.
The data accessed by the data access request can be determined according to the use requirement, including but not limited to: company operation, finance, personnel and other data, such as mobile phone number, fixed telephone, certificate number, mail box, communication address, bank card number, driving license-file number, driving license-license number, driving license-vehicle identification code, etc.
In one usage scenario, a cloud storage server for storing data may be configured on the network side. The server stores a database capable of satisfying data access requests. A data access request may be received by a data platform. The data platform is an extraction platform for providing target data. The target data includes company operation, finance, personnel and other data.
And step 120, judging whether to perform dynamic desensitization on the target data.
The data type of the target data or whether the target data contains sensitive data can be identified through the regular expression. And if the data type of the target data is needed to be desensitized or the target data contains sensitive data, determining to perform dynamic desensitization on the target data. Step 140 is performed.
If the data type of the target data is such that no desensitization is required, then dynamic desensitization of the target data is removed. Alternatively, if the target data does not contain sensitive data, dynamic desensitization of the target data is removed. Step 130 is performed.
In the above usage scenario, the data platform may locate the location information of the target data after receiving the data access request. And after acquiring the target data according to the position information, the data platform sends the target data to a desensitization judgment engine, and the desensitization judgment engine judges whether to perform dynamic desensitization on the target data. The desensitization judgment engine is used for receiving target data sent by the data platform and identifying whether data needing desensitization exists in the target data according to a certain rule (such as a regular expression for identifying sensitive data).
And step 130, if the target data is judged not to be subjected to dynamic desensitization, feeding the target data back to the requester.
And if the sensitive data does not exist in the target data, the acquired target data can be sent to the requester.
In the use scenario, after the data platform acquires the target data, the desensitization judgment engine judges that the target data is not subjected to dynamic desensitization. The data platform feeds the target data back to the requester.
And 140, if the target data is judged to be subjected to dynamic desensitization, reading the target data, and performing dynamic desensitization on the target data to obtain desensitization data.
If step 120 determines that dynamic desensitization is to be performed on the target data, the target data is read. The entire content contained in the target data is read and dynamic desensitization is performed on the entire content. For example, if the target data is a spreadsheet, then the entire spreadsheet is read and desensitized to the data in each cell in the spreadsheet, resulting in desensitized data.
In the above usage scenario, a dynamic desensitization engine may be configured to read the target data and desensitize the target data when the desensitization determination engine determines to desensitize the target data.
Step 150, feeding desensitization data back to the requestor.
And when the dynamic desensitization engine completes desensitization on the target data and obtains desensitization data, returning the desensitization data to the requester.
In the use scene, after desensitization is carried out on target data by the dynamic desensitization engine, desensitization data are sent to the desensitization judgment engine, and the desensitization judgment engine sends the desensitization data to the data platform and feeds the desensitization data back to the requester by the data platform.
According to the data desensitization method provided by the embodiment of the invention, in a production environment, a data access request is received, wherein the data access request is used for indicating that a requester requests to access target data in a data server; judging whether to perform dynamic desensitization on the target data; if the target data is judged to be subjected to dynamic desensitization, reading the target data, and performing dynamic desensitization on the target data to obtain desensitization data; desensitization data is fed back to the requestor. Compared with the prior art that desensitization data needs to be stored in the server, so that the desensitization data occupies a large amount of system resources, the data desensitization method provided by the embodiment of the invention can be used for dynamically desensitizing target data requested by a requester by the server in a production environment, and the desensitized data does not need to be stored in the server for a long time, so that the storage space of the server is released, and the resource utilization rate is improved. Meanwhile, dynamic desensitization can be achieved without being separated from a production environment, real-time desensitization of the query and calling results of sensitive data is achieved, and the returned data is enabled to be usable and safe.
Example two
Fig. 2 is a flowchart of a data desensitization method according to an embodiment of the present invention, which is further illustrated as the above embodiment, when the above embodiment is implemented in a production environment, it is found that a large amount of data exists in target data requested by a requester, and at this time, a dynamic desensitization engine reads all the target data at one time, and the processing efficiency is low, and a requirement for desensitizing multiple target data cannot be met. Especially in a production environment, when a plurality of requesters request target data with large data volume, the desensitization efficiency of the dynamic desensitization engine has a large influence on the feedback rate of desensitization data. Based on this, on the basis of the above embodiment, the method includes:
step 210, in the production environment, receiving a data access request, where the data access request is used to indicate that a requester requests to access target data in a data server.
Step 220, judging whether to perform dynamic desensitization on the target data.
And step 230, if the target data is judged not to be subjected to dynamic desensitization, feeding the target data back to the requester.
And 240, reading the target data if the target data is judged to be subjected to dynamic desensitization, and splitting the spreadsheet into a plurality of sub-forms if the target data is the spreadsheet.
When the target data is a spreadsheet, the spreadsheet may be split using sax (simple API for xml) analysis to obtain a plurality of sub-tables. The format of the sub-table is an extensible markup language XML file. The SAX analysis enables splitting a spreadsheet of target data into stream data composed of multiple sub-tables. For each sub-table in the streaming data, the occupied memory space is small, a dynamic desensitization engine is not needed to provide high memory space, and the cost is reduced.
And step 250, carrying out dynamic desensitization according to the sub-table to obtain desensitization data.
And dynamically desensitizing each obtained XML-formatted sub-table. The specific desensitization algorithm can set a corresponding desensitization regular expression according to the data type contained in the target data, replace the sensitive data in the target data with a specific symbol, and meanwhile reserve the non-sensitive data in the target data.
Alternatively, step 250 may be performed by:
step one, determining the security level of the target column according to the header content or the data content in the sub-table.
Wherein the target column is any column within the sub-table. The sub-table includes a header and data, the header being located in a first row of the sub-table, and the data being recorded starting from a second row of the sub-table. When each row of the sub-table includes a plurality of columns, the same column of different rows may record the same type of data. For example, the first column records the serial number, the second column records the user name, the third column records the user mobile phone number, and the fourth column records the user identification number. Different rows in the same column identify data content of the same data type (e.g., telephone number, etc.) for different subjects (e.g., users).
In one implementation, the header content includes the column name of the head row of the target column. At this time, the security level of each column may be determined according to the mapping relationship between each column of the header row and the top-level field in the electronic form, and the specific logic is as follows:
1) the column names of the first row of the target columns in the sub-table are read in sequence.
The first row is typically used for recording headers. The columns of the header row are labeled with the data type identified for each column. For example: the content of the first row and the first column is "serial number", the content of the second row and the first column is "name", the content of the third row and the first column is "telephone", and the like. The column names of the columns of the first row are read in turn.
2) The data type of the target column is determined from the column name.
The column names are text data, and the data types of the text data corresponding to the preset security level database are determined according to the text data, wherein the data types may be character strings, and also can be ID numbers used for representing certain data types. The security level database stores a preset mapping relation between the data type and the security level.
3) And determining the security level of the target column according to the preset mapping relation between the data type and the security level.
And searching the security level corresponding to the data type of the target column from the security level database.
The above mode can realize that the target column security level is determined based on the data content recorded in the head line of the target column. The security level of the target column can be determined according to the first row of the target column, the security level can be determined rapidly, and the data processing efficiency is improved.
In another implementation, the data content includes data recorded in each row of the target column except the first row. At this time, when desensitizing the sub-table, a groovy script may be used for desensitizing, and the logic expression of the groovy script implements step 251, which may be specifically implemented in the following manner:
1) and acquiring the data types of all rows except the first row in the target column in the sub-table.
The rows in the target column that follow the second row are the rows other than the first row. The data type of each row can be determined by reading the data characteristics of the data in each row. The data types of the data stored in the rows may be different, for example, the data types in the second row to the fourth row are telephone numbers, and the data types in the fifth row and the sixth row are identification numbers.
2) Counting a data type count, the data type count indicating a number of rows in the target column having the same data type.
In the target column, the number of occurrences of each data type is counted by data type, and the number of occurrences may be determined as the number of rows having the same data type.
3) And determining the data type of the target column according to the size of the data type count.
And sorting the data type counts in an ascending order or a descending order, and determining the data type identified by the data type count with the maximum vertical direction as the data type of the target column.
4) And determining the security level of the target column according to the data type.
And searching the security level corresponding to the data type of the target column from the security level database.
The above approach enables the security level of the target column to be determined for the actual data stored by the target column. For the conditions of the types of data stored in some target columns and the types of headers recorded in the head rows of the target columns, the security level of the target columns can be determined more accurately, and the reliability is improved.
And step two, desensitizing the data in the target column according to the security level to obtain desensitized data.
Different security levels may configure different desensitization regular expressions. As the level of security increases, the more sensitive data is masked.
Step 260, feeding desensitization data back to the requestor.
Optionally, sensitive data scanning is performed on the report on the data platform periodically, unidentified data is identified, and safety management of the data is realized based on the life cycle of the data. As more and more data is received by the data platform, data types that are not defined in the security level database may be received. By regularly scanning sensitive data, the security level database can be updated, so that the requested target data needing desensitization all have corresponding security level data, and the reliability is improved.
Further, after determining the security level of the target column, the method further includes:
if the target column is configured with the predefined security level, judging whether the predefined security level is the same as the determined security level; and if not, modifying the predefined security level according to the security level.
The predefined security level may be a security level configured manually on the data. The manually configured security level may be identified in a security level database, such as by adding a manually tagged identification. If the security level obtained in the above manner is the same as the manually determined security level, the accuracy of the manual determination can be verified. And the result of manual judgment can be used in the subsequent grading process in a limited way. If the security level obtained in the above manner is different from the security level determined manually, the manual marking identifier may be deleted and the related person may be notified. The implementation method can effectively prompt relevant personnel, and the accuracy of security level identification is guaranteed by the cognition of the relevant personnel on the security level of the data in the security level database.
EXAMPLE III
Fig. 3 is a flowchart of a data desensitization method according to a third embodiment of the present invention, which further illustrates the third embodiment, before reading target data, further includes: determining sampling data according to the target data; judging whether grading is needed to be carried out on the target data according to the sampling data; if the target data needs to be graded, grading the target data; accordingly, reading the target data includes: and if the target data does not need to be graded and the dynamic desensitization of the target data is judged, reading the target data. Illustratively, the method may be carried out by the following steps:
in the production environment, a data access request is received, where the data access request is used to indicate that a requester requests to access target data in a data server, step 310.
And step 320, judging whether dynamic desensitization is carried out on the target data.
And step 330, if the target data is judged not to be subjected to dynamic desensitization, feeding the target data back to the requester.
Step 340, determining sample data according to the target data.
After the target data is determined, in order to ensure that the security level of the target column in the target data can be determined in the above embodiment, it is necessary to determine whether the security level database already stores the security level of the data included in the target data. If all the target data are read, too many system resources are consumed. Therefore, the data can be extracted from each of the target data in a sampling mode to obtain sampling data.
In the above usage scenario, the sample data may be determined by the desensitization determination engine from the target data.
And step 350, judging whether the target data needs to be graded according to the sampling data.
And searching the data type of each data in the sampling data from the security level database to determine whether a corresponding security level exists. If a corresponding security level exists, then step 370 is performed without the need to rank the target data. If there is no corresponding security level, the target data needs to be ranked, and step 360 is performed.
In the above usage scenario, the desensitization determination engine determines whether or not the target data needs to be ranked based on the sampled data.
And step 360, if the target data needs to be graded, grading the target data.
The target data can be ranked according to the mapping relation between the pre-configured data type and the security level.
In the above usage scenario, if the desensitization determination engine determines that the target data needs to be ranked, the desensitization determination engine sends the target data (or the sampled data) to the data ranking engine and the data top-level engine, which are used for ranking the target data (or the sampled data) according to a pre-configured mapping relationship between the data type and the security level.
And 370, if the target data does not need to be graded and the dynamic desensitization of the target data is judged, reading the target data and performing the dynamic desensitization of the target data to obtain desensitization data.
In the above usage scenario, if the desensitization determination engine determines that the target data does not need to be graded and determines that the target data is to be subjected to dynamic desensitization, the target data is sent to the dynamic desensitization engine. And reading the target data by the dynamic desensitization engine, and performing dynamic desensitization on the target data to obtain desensitization data.
Step 380, feeding desensitization data back to the requestor.
The data desensitization method provided by the embodiment can ensure that the target data has a security level and improve reliability.
Example four
Fig. 4 is a flowchart of a data desensitization method according to a fourth embodiment of the present invention, which, as further described in the foregoing embodiment, further includes, after receiving a data access request: determining target data according to the data access request; storing the target data into a relay storage space, wherein the relay storage space is used for storing the target data; correspondingly, judging whether to perform dynamic desensitization on the target data comprises the following steps: reading target data in the relay storage space; and judging whether dynamic desensitization is carried out on the target data. Illustratively, the method may be carried out by the following steps:
in the production environment, a data access request is received, where the data access request is used to indicate that a requester requests to access target data in a data server.
And step 420, determining target data according to the data access request.
And step 430, storing the target data into a relay storage space, wherein the relay storage space is used for storing the target data.
Consider that in a scenario of multiple data requests, i.e., a production environment, multiple initiators may have initiated data requests at the same time. At this time, considering that the storage space of the dynamic desensitization engine is limited, the target data may be transmitted to the relay server by the desensitization determination engine. The relay server may be used to store the target data. When the target data is divided into the plurality of sub-tables in the above embodiment, the relay server may store the divided plurality of sub-tables.
And step 440, reading the target data in the relay storage space.
And the dynamic desensitization engine reads the sub-table from the relay server through asynchronous communication and desensitizes the sub-table to obtain desensitization data.
And step 450, judging whether to perform dynamic desensitization on the target data.
Step 460, if it is determined that dynamic desensitization is not performed on the target data, feeding the target data back to the requesting party.
And 470, if the target data is judged to be subjected to dynamic desensitization, reading the target data, and performing dynamic desensitization on the target data to obtain desensitization data.
And after the dynamic desensitization engine obtains desensitization data, storing the desensitization data in the relay server.
Step 480, feeding desensitization data back to the requestor.
And the desensitization judgment engine reads desensitization data from the relay server and sends the desensitization data to the data platform. And finally, feeding desensitization data back to the initiator by the data platform.
The data desensitization method provided by the embodiment can improve the response speed of multiple requests through the relay server, and improve the processing efficiency.
EXAMPLE five
Fig. 5 is a schematic structural diagram of a data desensitization apparatus according to a fifth embodiment of the present invention, where the apparatus is located in a server, and the apparatus is capable of executing a data desensitization method according to any embodiment of the present invention, and has corresponding functional modules and beneficial effects of the execution method. Illustratively, the apparatus comprises: a data platform 51, a desensitization determination engine 52, and a dynamic desensitization engine 53, wherein:
the data platform 51 is configured to receive, in the production environment, a data access request indicating that a requestor requests access to target data in a data server.
The desensitization determination engine 52 is configured to determine whether to dynamically desensitize the target data.
The dynamic desensitization engine 53 is configured to, if the desensitization determination engine 52 determines that dynamic desensitization is performed on the target data, read the target data, and perform dynamic desensitization on the target data to obtain desensitization data.
The data platform 51 is used for feeding back desensitization data obtained by the dynamic desensitization engine 53 to the requesting party.
Further, the dynamic desensitization engine 53 is configured to:
and if the target data is the electronic form, splitting the electronic form into a plurality of sub-forms.
And carrying out dynamic desensitization according to the sub-table to obtain desensitization data.
Further, the dynamic desensitization engine 53 is configured to:
and determining the security level of the target column according to the header content or the data content in the sub-table, wherein the target column is any column in the sub-table.
And desensitizing the data in the target column according to the security level to obtain desensitized data.
Further, the header content includes the column name of the head row of the target column. Accordingly, the dynamic desensitization engine 53 is operable to:
the column names of the first row of the target columns in the sub-table are read in sequence.
The data type of the target column is determined from the column name.
And determining the security level of the target column according to the preset mapping relation between the data type and the security level.
Further, the data content includes data recorded in each row of the target column except the first row, and accordingly, the dynamic desensitization engine 53 is configured to:
and acquiring the data types of all rows except the first row in the target column in the sub-table.
Counting a data type count, the data type count indicating a number of rows in the target column having the same data type.
And determining the data type of the target column according to the size of the data type count.
And determining the security level of the target column according to the data type.
The system further comprises a data grading engine, and the data top-level engine is used for grading the target data.
The data rating engine is further to:
and if the target column is configured with the predefined security level, judging whether the predefined security level is the same as the determined security level.
And if not, modifying the predefined security level according to the security level.
Further, the desensitization determination engine 52 is configured to:
sample data is determined based on the target data.
And judging whether the target data needs to be graded according to the sampling data.
And if the target data needs to be graded, sending the target data to a data grading engine, and grading the target data by the data grading engine.
The desensitization determination engine 52 is operable to: and if the target data does not need to be graded and the dynamic desensitization of the target data is judged, reading the target data.
Further, the system also comprises a relay storage server, and the relay storage server is provided with a relay storage space.
The data platform 51 is used to determine target data according to the data access request.
And storing the target data into a relay storage space of the relay storage server, wherein the relay storage space is used for storing the target data.
Accordingly, the desensitization determination engine 52 accesses the relay storage server and reads the target data in the relay storage space. And judging whether dynamic desensitization is carried out on the target data.
In the data desensitization device provided in the embodiment of the present invention, the data platform 51 receives a data access request in a production environment, where the data access request is used to indicate that a requester requests to access target data in a data server; the desensitization determination engine 52 determines whether to perform dynamic desensitization on the target data; if the dynamic desensitization is determined to be performed on the target data, the dynamic desensitization engine 53 reads the target data, and performs dynamic desensitization on the target data to obtain desensitization data; desensitization data is fed back to the requestor through the data platform 51. Compared with the prior art that desensitization data needs to be stored in the server, so that the desensitization data occupies a large amount of system resources, the data desensitization method provided by the embodiment of the invention can be used for dynamically desensitizing target data requested by a requester by the server in a production environment, and the desensitized data does not need to be stored in the server for a long time, so that the storage space of the server is released, and the resource utilization rate is improved. Meanwhile, dynamic desensitization can be achieved without being separated from a production environment, real-time desensitization of the query and calling results of sensitive data is achieved, and the returned data is enabled to be usable and safe.
EXAMPLE six
Fig. 6 is a schematic structural diagram of a computer device according to a sixth embodiment of the present invention, where the computer device may be a server, and as shown in fig. 6, the computer device includes a processor 610, a memory 620, an input device 630, and an output device 640; the number of processors 610 in the computer device may be one or more, and one processor 610 is taken as an example in fig. 6; the processor 610, the memory 620, the input device 630 and the output device 640 in the computer apparatus may be connected by a bus or other means, and the connection by the bus is exemplified in fig. 6.
The memory 620, which is a computer-readable storage medium, can be used to store software programs, computer-executable programs, and modules, such as program instructions/modules (e.g., data platform, desensitization determination engine, data rating engine, or dynamic desensitization engine) corresponding to the data desensitization methods of embodiments of the present invention. The processor 610 executes various functional applications of the computer device and data processing by executing software programs, instructions and modules stored in the memory 620, namely, implements the data desensitization method described above.
The memory 620 may mainly include a storage program area and a storage data area, wherein the storage program area may store an operating system, an application program required for at least one function; the storage data area may store data created according to the use of the terminal, and the like. Further, the memory 620 may include high speed random access memory, and may also include non-volatile memory, such as at least one magnetic disk storage device, flash memory device, or other non-volatile solid state storage device. In some examples, the memory 620 may further include memory located remotely from the processor 610, which may be connected to a computer device through a network. Examples of such networks include, but are not limited to, the internet, intranets, local area networks, mobile communication networks, and combinations thereof.
The input means 630 may be used to receive input numeric or character information and generate key signal inputs related to user settings and function control of the computer apparatus. The output device 640 may include a display device such as a display screen.
EXAMPLE seven
An embodiment of the present invention further provides a computer-readable storage medium, on which a computer program is stored, which when executed by a processor is configured to perform a data desensitization method, the method including:
in a production environment, receiving a data access request, wherein the data access request is used for indicating that a requester requests to access target data in a data server;
judging whether to perform dynamic desensitization on the target data;
if the target data is judged to be subjected to dynamic desensitization, reading the target data, and performing dynamic desensitization on the target data to obtain desensitization data;
desensitization data is fed back to the requestor.
Further, performing dynamic desensitization on the target data to obtain desensitization data, including:
if the target data is the electronic form, splitting the electronic form into a plurality of sub-forms;
and carrying out dynamic desensitization according to the sub-table to obtain desensitization data.
Further, performing dynamic desensitization according to the sub-table to obtain desensitization data, including:
determining the security level of a target column according to header content or data content in the sub-table, wherein the target column is any column in the sub-table;
and desensitizing the data in the target column according to the security level to obtain desensitized data.
Further, the header content includes the column name of the head row of the target column; correspondingly, the security level of the target column is determined according to the header content in the sub-table, and the method comprises the following steps:
sequentially reading the column names of the first row of the target columns in the sub-table;
determining the data type of the target column according to the column name;
and determining the security level of the target column according to the preset mapping relation between the data type and the security level.
Further, the data content includes data recorded in each row except the first row in the target column, and correspondingly, the determining the security level of the target column according to the data content in the sub-table includes:
acquiring the data types of all rows except the first row in a target column in a sub-table;
counting a data type count, wherein the data type count represents the number of rows in the target column with the same data type;
determining the data type of the target column according to the size of the data type count;
and determining the security level of the target column according to the data type.
Further, after determining the security level of the target column, the method further includes:
if the target column is configured with the predefined security level, judging whether the predefined security level is the same as the determined security level;
and if not, modifying the predefined security level according to the security level.
Further, before reading the target data, the method further includes:
determining sampling data according to the target data;
judging whether grading is needed to be carried out on the target data according to the sampling data;
if the target data needs to be graded, grading the target data;
accordingly, reading the target data includes:
and if the target data does not need to be graded and the dynamic desensitization of the target data is judged, reading the target data.
Further, after receiving the data access request, the method further includes:
determining target data according to the data access request;
storing the target data into a relay storage space, wherein the relay storage space is used for storing the target data;
correspondingly, judging whether to perform dynamic desensitization on the target data comprises the following steps:
reading target data in the relay storage space;
and judging whether dynamic desensitization is carried out on the target data.
Of course, the storage medium provided by the embodiment of the present invention contains computer-executable instructions, and the computer-executable instructions are not limited to the above method operations, and can also execute the relevant operations in the data desensitization method provided by any embodiment of the present invention.
From the above description of the embodiments, it is obvious for those skilled in the art that the present invention can be implemented by software and necessary general hardware, and certainly, can also be implemented by hardware, but the former is a better embodiment in many cases. Based on such understanding, the technical solutions of the present invention may be embodied in the form of a software product, which may be stored in a computer-readable storage medium, such as a floppy disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a FLASH Memory (FLASH), a hard disk or an optical disk of a computer, and includes several instructions for enabling a computer device (which may be a personal computer, a server, or a network device) to execute the methods according to the embodiments of the present invention.
It should be noted that, in the embodiment of the above search apparatus, each included unit and module are merely divided according to functional logic, but are not limited to the above division as long as the corresponding functions can be implemented; in addition, specific names of the functional units are only for convenience of distinguishing from each other, and are not used for limiting the protection scope of the present invention.
It is to be noted that the foregoing is only illustrative of the preferred embodiments of the present invention and the technical principles employed. It will be understood by those skilled in the art that the present invention is not limited to the particular embodiments illustrated herein, but is capable of various obvious changes, rearrangements and substitutions as will now become apparent to those skilled in the art without departing from the scope of the invention. Therefore, although the present invention has been described in greater detail by the above embodiments, the present invention is not limited to the above embodiments, and may include other equivalent embodiments without departing from the spirit of the present invention, and the scope of the present invention is determined by the scope of the appended claims.
Claims (11)
1. A method of data desensitization, comprising:
in a production environment, receiving a data access request, wherein the data access request is used for representing that a requester requests to access target data in a data server;
judging whether to perform dynamic desensitization on the target data;
if the target data is judged to be subjected to dynamic desensitization, reading the target data, and performing dynamic desensitization on the target data to obtain desensitization data;
feeding back the desensitization data to the requestor.
2. A method of data desensitization according to claim 1, wherein said dynamically desensitizing said target data to obtain desensitized data comprises:
if the target data is the electronic form, splitting the electronic form into a plurality of sub-forms;
and carrying out dynamic desensitization according to the sub-table to obtain desensitization data.
3. A method of data desensitization according to claim 2, wherein said performing dynamic desensitization according to sub-tables to obtain desensitization data comprises:
determining the security level of a target column according to header content or data content in a sub-table, wherein the target column is any column in the sub-table;
and desensitizing the data in the target column according to the security level to obtain desensitized data.
4. A data desensitization method according to claim 3, wherein said header contents include a column name of a target column head row; correspondingly, the determining the security level of the target column according to the header content in the sub-table includes:
sequentially reading the column names of the first row of the target columns in the sub-table;
determining the data type of the target column according to the column name;
and determining the security level of the target column according to the preset mapping relation between the data type and the security level.
5. A data desensitization method according to claim 4, wherein said data content includes data recorded in rows other than the first row of said target column, and wherein said determining a security level for a target column based on data content in a sub-table accordingly comprises:
acquiring the data types of all rows except the first row in the target column in the sub-table;
counting a data type count, the data type count representing a number of rows in the target column having a same data type;
determining the data type of the target column according to the size of the data type count;
and determining the security level of the target column according to the data type.
6. A data desensitization method according to claim 4 or 5, further comprising, after determining the security level of the target column:
if the target column is configured with a predefined security level, judging whether the predefined security level is the same as the determined security level;
and if not, modifying the predefined security level according to the security level.
7. A data desensitization method according to claim 1 or 3, wherein prior to reading said target data, further comprising:
determining sampling data according to the target data;
judging whether grading needs to be carried out on the target data according to the sampling data;
if the target data needs to be graded, grading the target data;
correspondingly, the reading the target data includes:
and if the target data does not need to be graded and the dynamic desensitization of the target data is judged, reading the target data.
8. The data desensitization method according to claim 1, further comprising, after receiving a data access request:
determining target data according to the data access request;
storing the target data to a relay storage space, wherein the relay storage space is used for storing the target data;
correspondingly, the determining whether to perform dynamic desensitization on the target data includes:
reading target data in the relay storage space;
and judging whether to perform dynamic desensitization on the target data.
9. A data desensitization apparatus, comprising: the system comprises a data platform, a desensitization judgment engine and a dynamic desensitization engine, wherein:
the data platform is used for receiving a data access request in a production environment, wherein the data access request is used for representing that a requester requests to access target data in a data server;
the desensitization judgment engine is used for judging whether to perform dynamic desensitization on the target data;
the dynamic desensitization engine is used for reading the target data and performing dynamic desensitization on the target data to obtain desensitization data if the desensitization judgment engine judges that the target data is subjected to dynamic desensitization;
and the data platform is used for feeding back the desensitization data obtained by the dynamic desensitization engine to the requester.
10. A computer device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, wherein the processor when executing the program implements a data desensitization method according to any of claims 1-8.
11. A computer-readable storage medium, on which a computer program is stored which, when being executed by a processor, carries out a method of desensitizing data according to any one of claims 1 to 8.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910910381.XA CN110795756A (en) | 2019-09-25 | 2019-09-25 | Data desensitization method and device, computer equipment and computer readable storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910910381.XA CN110795756A (en) | 2019-09-25 | 2019-09-25 | Data desensitization method and device, computer equipment and computer readable storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN110795756A true CN110795756A (en) | 2020-02-14 |
Family
ID=69439686
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910910381.XA Withdrawn CN110795756A (en) | 2019-09-25 | 2019-09-25 | Data desensitization method and device, computer equipment and computer readable storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110795756A (en) |
Cited By (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112037004A (en) * | 2020-07-14 | 2020-12-04 | 北京文思海辉金信软件有限公司 | Business processing result presentation method and device, computer equipment and storage medium |
| CN112417406A (en) * | 2020-12-04 | 2021-02-26 | 中国电子信息产业集团有限公司第六研究所 | Data desensitization method and device, readable storage medium and electronic equipment |
| CN112948877A (en) * | 2021-03-03 | 2021-06-11 | 北京中安星云软件技术有限公司 | Dynamic database desensitization method and system based on TCP (Transmission control protocol) proxy |
| CN113472757A (en) * | 2021-06-18 | 2021-10-01 | 上汽通用五菱汽车股份有限公司 | Vehicle data processing method, processing platform and readable storage medium |
| CN114282591A (en) * | 2021-11-18 | 2022-04-05 | 厦门市美亚柏科信息股份有限公司 | Dynamic security level real-time division method, terminal equipment and storage medium |
| WO2022088754A1 (en) * | 2020-10-27 | 2022-05-05 | 华为技术有限公司 | File desensitization method and apparatus, and storage medium |
| WO2022143758A1 (en) * | 2020-12-30 | 2022-07-07 | 华为技术有限公司 | Data desensitization method and apparatus, and storage system |
| CN115495769A (en) * | 2022-11-16 | 2022-12-20 | 江苏曼荼罗软件股份有限公司 | Data desensitization method, system, readable storage medium and device |
| WO2023097521A1 (en) * | 2021-11-30 | 2023-06-08 | 西门子股份公司 | Data model generation method and apparatus |
| CN114282591B (en) * | 2021-11-18 | 2024-09-27 | 厦门市美亚柏科信息股份有限公司 | Dynamic security level real-time division method, terminal equipment and storage medium |
-
2019
- 2019-09-25 CN CN201910910381.XA patent/CN110795756A/en not_active Withdrawn
Cited By (13)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112037004A (en) * | 2020-07-14 | 2020-12-04 | 北京文思海辉金信软件有限公司 | Business processing result presentation method and device, computer equipment and storage medium |
| WO2022088754A1 (en) * | 2020-10-27 | 2022-05-05 | 华为技术有限公司 | File desensitization method and apparatus, and storage medium |
| CN112417406A (en) * | 2020-12-04 | 2021-02-26 | 中国电子信息产业集团有限公司第六研究所 | Data desensitization method and device, readable storage medium and electronic equipment |
| EP4261723A4 (en) * | 2020-12-30 | 2024-06-19 | Huawei Technologies Co., Ltd. | Data desensitization method and apparatus, and storage system |
| WO2022143758A1 (en) * | 2020-12-30 | 2022-07-07 | 华为技术有限公司 | Data desensitization method and apparatus, and storage system |
| CN112948877A (en) * | 2021-03-03 | 2021-06-11 | 北京中安星云软件技术有限公司 | Dynamic database desensitization method and system based on TCP (Transmission control protocol) proxy |
| CN113472757B (en) * | 2021-06-18 | 2022-06-24 | 上汽通用五菱汽车股份有限公司 | Vehicle data processing method, processing platform and readable storage medium |
| CN113472757A (en) * | 2021-06-18 | 2021-10-01 | 上汽通用五菱汽车股份有限公司 | Vehicle data processing method, processing platform and readable storage medium |
| CN114282591A (en) * | 2021-11-18 | 2022-04-05 | 厦门市美亚柏科信息股份有限公司 | Dynamic security level real-time division method, terminal equipment and storage medium |
| CN114282591B (en) * | 2021-11-18 | 2024-09-27 | 厦门市美亚柏科信息股份有限公司 | Dynamic security level real-time division method, terminal equipment and storage medium |
| WO2023097521A1 (en) * | 2021-11-30 | 2023-06-08 | 西门子股份公司 | Data model generation method and apparatus |
| CN115495769A (en) * | 2022-11-16 | 2022-12-20 | 江苏曼荼罗软件股份有限公司 | Data desensitization method, system, readable storage medium and device |
| CN115495769B (en) * | 2022-11-16 | 2023-03-10 | 江苏曼荼罗软件股份有限公司 | Data desensitization method, system, readable storage medium and device |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110795756A (en) | Data desensitization method and device, computer equipment and computer readable storage medium | |
| EP4099170B1 (en) | Method and apparatus of auditing log, electronic device, and medium | |
| US20150234927A1 (en) | Application search method, apparatus, and terminal | |
| CN108090351B (en) | Method and apparatus for processing request message | |
| CN112636957B (en) | Early warning method and device based on log, server and storage medium | |
| CN110377651B (en) | Batch data processing method, device, equipment and storage medium | |
| CN107784205B (en) | User product auditing method, device, server and storage medium | |
| CN111586695B (en) | Short message identification method and related equipment | |
| CN111488594A (en) | Authority checking method and device based on cloud server, storage medium and terminal | |
| CN113204345A (en) | Page generation method and device, electronic equipment and storage medium | |
| CN114598597B (en) | Multisource log analysis method, multisource log analysis device, computer equipment and medium | |
| CN115544558A (en) | Sensitive information detection method and device, computer equipment and storage medium | |
| CN114490715A (en) | Data extraction method and device, electronic equipment and storage medium | |
| EP4216076B1 (en) | Method and apparatus of processing an observation information, electronic device and storage medium | |
| CN107666431B (en) | Bookmark communication message acquisition method and device | |
| CN116775488A (en) | Abnormal data determination method, device, equipment, medium and product | |
| CN113032836B (en) | Data desensitization method and apparatus | |
| CN103605480B (en) | Web server and disk resource access control method thereof | |
| CN107977381B (en) | Data configuration method, index management method, related device and computing equipment | |
| CN114722004A (en) | Message retrieval method and device, electronic equipment and storage medium | |
| CN114168557A (en) | Processing method and device for access log, computer equipment and storage medium | |
| CN112347066B (en) | Log processing method and device, server and computer readable storage medium | |
| CN111914252A (en) | File security detection method and device and electronic equipment | |
| CN112528339A (en) | Data desensitization method based on Cach é database and electronic equipment | |
| CN117892348A (en) | Management method and device for application program interface assets and electronic equipment |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| WW01 | Invention patent application withdrawn after publication | ||
| WW01 | Invention patent application withdrawn after publication |
Application publication date: 20200214 |