CN114154198A

CN114154198A - Data processing method and device

Info

Publication number: CN114154198A
Application number: CN202111474426.7A
Authority: CN
Inventors: 王伟杰; 黄记新; 王幼芝
Original assignee: CCB Finetech Co Ltd
Current assignee: CCB Finetech Co Ltd
Priority date: 2021-12-03
Filing date: 2021-12-03
Publication date: 2022-03-08

Abstract

The embodiment of the application provides a data processing method and a data processing device, wherein the method comprises the following steps: and acquiring at least one preset rule, wherein each preset rule corresponds to each safety level. The method comprises the steps of obtaining a first field to be identified, wherein the first field comprises a plurality of field data. And determining a target security level corresponding to the first field according to at least one preset rule and field data in the first field. The preset rules matched with the field data of the first field are determined in at least one preset rule, and each preset rule corresponds to the corresponding safety level, so that the preset rule of the first field can be determined according to the matched preset rule, the safety level of the field can be effectively and quickly determined automatically, and the efficiency of determining the safety level of the data is effectively improved.

Description

Data processing method and device

Technical Field

The present disclosure relates to computer technologies, and in particular, to a data processing method and apparatus.

Background

With the continuous development of big data related technology, protection aiming at data security is currently performed, and the protection becomes a very important part in data processing.

When data protection is performed, a corresponding security level is usually set for data, and currently, in the related art, when a security level is set for data, security level labeling is usually performed manually for data.

However, in the case of a large amount of data, the implementation manner of manually labeling the data security level may cause inefficient operation of determining the data security level.

Disclosure of Invention

The embodiment of the application provides a data processing method and device, and aims to solve the problem of low operation efficiency in determining a data security level.

In a first aspect, an embodiment of the present application provides a data processing method, including:

acquiring at least one preset rule, wherein each preset rule corresponds to a respective security level;

acquiring a first field to be identified, wherein the first field comprises a plurality of field data;

and matching according to the at least one preset rule and the field data in the first field, and determining the target security level corresponding to the first field.

In a possible design, the determining, according to the at least one preset rule and the field data in the first field, the target security level corresponding to the first field includes:

acquiring a first preset number of field data to be matched from the plurality of field data of the first field;

according to the field data to be matched with the first preset number, determining a target rule matched with the first field in the at least one preset rule;

and determining a target security level corresponding to the first field according to the security level corresponding to the target rule.

In one possible design, for any one of the preset rules, the preset rule includes a preset regular expression;

the determining, according to the first preset number of field data to be matched, a target rule matched with the first field in the at least one preset rule includes:

acquiring a preset regular expression in the preset rule;

determining the matching results of the first preset number of field data to be matched and the preset regular expression, wherein the matching results are matching success or matching failure;

and if the matching result is that the number of the successfully matched field data to be matched is greater than or equal to a second preset number, determining the preset rule as the target rule, wherein the second preset number is less than or equal to the first preset number.

In one possible design, the determining, according to the security level corresponding to the target rule, the target security level corresponding to the first field includes:

acquiring the number of the target rules;

if the number of the target rules is 1, determining the security level corresponding to the target rules as the security level corresponding to the first field;

and if the number of the target rules is greater than 1, determining the maximum security level corresponding to the target rules as the security level corresponding to the first field.

In one possible design, after determining the target security level corresponding to the first field, the method further includes:

and updating the target security level corresponding to the first field at regular time by taking preset time length as a period.

In one possible design, a field message of the first field is stored in a field message queue, the field message indicating a security level identifying the first field; the acquiring a first field to be identified includes:

scanning the field message queue, and acquiring a first field in the field message queue according to the field message of the first field; alternatively, the first and second electrodes may be,

and acquiring the first field according to a received data query request, wherein the data query request comprises target field data to be queried, and the target field data belongs to the first field.

In one possible design, the first field belongs to a first table, the first table belongs to a first database; table messages of the first table are stored in a table message queue, wherein the table messages are used for indicating security levels for identifying various fields in the first table; storing database messages of the first database in a database message queue, the database messages being used to indicate security levels identifying respective fields in the first database;

the scanning the field message queue, and before acquiring the first field according to the field message of the first field in the field message queue, the method further includes:

acquiring at least one database to be scanned, and storing the database message of each database into the database message queue;

scanning the database message queue, acquiring at least one table to be scanned in a first database according to the database message of the first database in the database message queue, and storing the table message corresponding to each table into a table message queue;

and scanning the table message queue, acquiring at least one field to be scanned in the first table according to the table message of the first table in the table message queue, and storing the field message corresponding to each field in the field message queue.

In a second aspect, an embodiment of the present application provides a data processing apparatus, including:

the system comprises a first acquisition module, a second acquisition module and a first processing module, wherein the first acquisition module is used for acquiring at least one preset rule, and each preset rule corresponds to a respective security level;

the second acquisition module is used for acquiring a first field to be identified, and the first field comprises a plurality of field data;

and the determining module is used for matching according to the at least one preset rule and the field data in the first field and determining the target security level corresponding to the first field.

In one possible design, the determining module is specifically configured to:

the determining module is specifically configured to:

acquiring a preset regular expression in the preset rule;

In one possible design, the determining module is specifically configured to:

acquiring the number of the target rules;

In one possible design, the determining module is further configured to:

and after the target security level corresponding to the first field is determined, regularly updating the target security level corresponding to the first field by taking preset time length as a period.

In one possible design, a field message of the first field is stored in a field message queue, the field message indicating a security level identifying the first field; the second obtaining module is specifically configured to:

the second obtaining module is further configured to:

before the field message queue is scanned and the first field in the field message queue is acquired according to the field message of the first field,

In a third aspect, an embodiment of the present application provides a data processing apparatus, including:

a memory for storing a program;

a processor for executing the program stored by the memory, the processor being adapted to perform the method as described above in the first aspect and any one of the various possible designs of the first aspect when the program is executed.

In a fourth aspect, embodiments of the present application provide a computer-readable storage medium, comprising instructions which, when executed on a computer, cause the computer to perform the method as described above in the first aspect and any one of the various possible designs of the first aspect.

In a fifth aspect, embodiments of the present application provide a computer program product comprising a computer program that, when executed by a processor, implements the method as described above in the first aspect and any one of various possible designs of the first aspect.

Drawings

In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings needed to be used in the description of the embodiments or the prior art will be briefly introduced below, and it is obvious that the drawings in the following description are some embodiments of the present application, and for those skilled in the art, other drawings can be obtained according to these drawings without inventive exercise.

Fig. 1 is a schematic diagram of data protection in the related art according to an embodiment of the present application;

fig. 2 is a flowchart of a data processing method according to an embodiment of the present application;

fig. 3 is a second flowchart of a data processing method according to an embodiment of the present application;

fig. 4 is a schematic diagram illustrating an implementation of a rule configuration provided in an embodiment of the present application;

fig. 5 is a schematic diagram illustrating an implementation of a preset rule provided in an embodiment of the present application;

FIG. 6 is a schematic diagram illustrating an implementation of a data storage structure according to an embodiment of the present application;

fig. 7 is a schematic diagram illustrating an implementation of a message queue according to an embodiment of the present application;

fig. 8 is a schematic diagram illustrating an implementation of determining a matching result according to an embodiment of the present application;

FIG. 9 is a schematic diagram illustrating an implementation of determining a security level of a first field according to an embodiment of the present application;

FIG. 10 is a schematic diagram of an interface of security levels of various fields provided by an embodiment of the present application;

fig. 11 is a schematic flowchart of a data processing method according to an embodiment of the present application;

FIG. 12 is a first schematic diagram illustrating an implementation of a user level and a security level provided by an embodiment of the present application;

fig. 13 is a second schematic diagram illustrating implementation of user levels and security levels provided in an embodiment of the present application;

fig. 14 is a schematic structural diagram of a data processing apparatus according to an embodiment of the present application;

fig. 15 is a schematic hardware structure diagram of a data processing device according to an embodiment of the present application.

Detailed Description

In order to make the objects, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application, and it is obvious that the described embodiments are some embodiments of the present application, but not all embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present application.

In order to better understand the technical solution of the present application, the related art related to the present application will be further described in detail below.

With the continuous development of big data related technology, the importance of data security is increasing at present, so that security protection is performed on data, which is also called as a very important part in the data processing process.

In the current context of big data, when all data are aggregated in a data warehouse, some sensitive data are necessarily present, and then corresponding data protection should be implemented for all sensitive data in the data warehouse to ensure that the sensitive data are not leaked.

Assuming that there are currently hundreds of databases, each database includes hundreds of tables, each table includes 50 fields, and each field includes 100 ten thousand records, in an actual implementation process, it is necessary to ensure that all sensitive data in the 100 × 50 × 100W 5000 billion field data is not leaked out in a large data platform. It can be seen that this is a very large workload.

The implementation of data protection in the related art can be understood with reference to fig. 1, where fig. 1 is a schematic diagram of data protection in the related art provided by the embodiment of the present application.

Referring to fig. 1, in the related art, when data protection is implemented, a clear text view right of data is usually opened for some specific crowd, that is, the specific crowd can directly view all data of a data warehouse, even sensitive data can be directly viewed. This portion of the particular population may be, for example, a data analyst, a scientist, or the like. However, although the identity of this particular group of people is relatively special, opening the clear text viewing right of the data directly still results in sensitive data being at risk of being revealed.

Meanwhile, referring to fig. 1, in the non-specific population other than the specific population described above, when viewing data, the system usually performs desensitization processing on the data, and processes the data after the desensitization processing to the non-specific population. However, the non-specific population may also need plaintext data to perform corresponding processing in some cases, for example, the developer may need the data to perform a test, and if the developer performs a test based on only desensitization data, it may not be possible to ensure that the developed product operates correctly on the non-desensitization data.

It can be understood based on the above description that the current implementation of opening the plaintext viewing right of all data only for a specific crowd results in a lack of flexibility in implementing data protection. Therefore, different security levels can be set for different data, so that flexible display of the data according to actual conditions can be realized.

Currently, in the prior art, when setting a security level for data, an operator usually audits the data and marks the data with a corresponding security level. However, as the related art is continuously developed, the amount of data is huge at present, and therefore, the implementation manner of manually marking the security level results in that the operation efficiency of determining the data security level is very low.

Aiming at the problems in the prior art, the application provides the following technical conception: by setting a plurality of preset rules, each preset rule can comprise a corresponding security level, the data and the preset rules can be matched, and the security level of the successfully matched preset rule is determined as the security level of the current data, so that the determination of the security level of the data can be automatically and efficiently realized.

Based on the above description, the data processing method provided by the present application is described below with reference to specific embodiments, and it should be noted that the execution main body of each embodiment in the present application may be, for example, a device with a data processing function, such as a server, a processor, a microprocessor, and the like, and in an actual implementation process, a specific implementation of the execution main body may be selected according to an actual requirement, which is not limited in this embodiment, and all devices with a data processing function may be used as the execution main body of each embodiment in the present application.

First, description is made with reference to fig. 2, and fig. 2 is a flowchart of a data processing method according to an embodiment of the present application.

As shown in fig. 2, the method includes:

s201, at least one preset rule is obtained, wherein each preset rule corresponds to each safety level.

In this embodiment, for example, a plurality of preset rules may be set, where each preset rule is used for matching with each subsequent data, and any preset rule may correspond to a security level, and in a possible implementation manner, for example, the dividing of the security level may include: plaintext, sensitive, confidential, and the like, wherein different security levels correspond to different degrees of data protection, and the degrees to which the data corresponding to the four security levels described above need to be protected increase sequentially. In the actual implementation process, the specific division of the security levels can be selected according to actual requirements, as long as different security levels corresponding to different data protection degrees can be implemented.

Moreover, the preset rule in this embodiment is used to describe the corresponding data, so that, for example, an expression of the data and the like can be described in the preset rule, and then matching can be performed according to the expression and the corresponding data, so as to realize matching between the preset rule and the data.

S202, obtaining a first field to be identified, wherein the first field comprises a plurality of field data.

The first field may be understood as meaning of specific data, for example, the first field may be "identification card", "name", etc., and a plurality of field data may be included in the first field, and the first field data in the first field may be, for example, a specific identification number, a specific name, etc.

In an actual implementation process, a plurality of fields may exist in the data warehouse, and the first field in this embodiment may be any one of the plurality of fields, where the implementation manner for each field is similar, and the field that needs to identify the security level may be used as the first field in this embodiment.

S203, matching is carried out according to at least one preset rule and the field data in the first field, and the target security level corresponding to the first field is determined.

The preset rule in this embodiment is used for matching with field data, so after the preset rule and the field data in the first field are obtained, for example, matching may be performed according to at least one preset rule and the field data in the first field, so as to determine the preset rule matched with the current first field, and then, for example, according to the security level of the preset rule matched with the first field, a target security level corresponding to the first field may be determined.

The data processing method provided by the embodiment of the application comprises the following steps: and acquiring at least one preset rule, wherein each preset rule corresponds to each safety level. The method comprises the steps of obtaining a first field to be identified, wherein the first field comprises a plurality of field data. And determining a target security level corresponding to the first field according to at least one preset rule and field data in the first field. The preset rules matched with the field data of the first field are determined in at least one preset rule, and each preset rule corresponds to the corresponding safety level, so that the preset rule of the first field can be determined according to the matched preset rule, the safety level of the field can be effectively and quickly determined automatically, and the efficiency of determining the safety level of the data is effectively improved.

On the basis of the foregoing embodiments, the data processing method provided by the present application is further described in detail with reference to fig. 3 to fig. 10, fig. 3 is a second flowchart of the data processing method provided by the present application, fig. 4 is an implementation schematic diagram of a rule configuration provided by the present application, fig. 5 is an implementation schematic diagram of a preset rule provided by the present application, fig. 6 is an implementation schematic diagram of a data storage structure provided by the present application, fig. 7 is an implementation schematic diagram of a message queue provided by the present application, fig. 8 is an implementation schematic diagram of determining a matching result provided by the present application, fig. 9 is an implementation schematic diagram of determining a security level of a first field provided by the present application, and fig. 10 is an interface schematic diagram of a security level of each field provided by the present application.

As shown in fig. 3, the method includes:

s301, at least one preset rule is obtained, wherein each preset rule corresponds to each safety level.

The implementation manner of S301 is similar to that of S201, and is not described herein again.

Meanwhile, it can be understood that at least one preset rule in this embodiment is preconfigured, and an implementation manner of the preconfigured preset rule is described below with reference to fig. 4.

As shown in fig. 4, when configuring the preset rule, for example, a rule name, a rule remark, a rule content, a rule classification, a security level, and a validation state may be further set.

The rule name is a name of a preset rule configured currently, and may reflect a specific meaning of the rule, and the like, which is not limited in this embodiment, for example, the rule name may be a "sensitive word", "domain name", and the like, and the specific setting of the rule name may be selected according to an actual requirement.

The rule remark may be a remark performed when the preset rule is configured, and may be, for example, an explanation for the current rule, or may also be an explanation for a usage specification of the current rule, and the like.

Specifically, the regular expression (regular expression) describes a pattern (pattern) for matching a character string, and may be used to check whether a character string contains a certain sub-character string, replace the matched sub-character string, or extract a sub-character string meeting a certain condition from a certain character string, or the like.

Therefore, different types of character strings can be described by presetting the regular expressions, for example, for the identification number, corresponding description can be performed by the regular expressions, and then the identification number can be matched with the corresponding regular expressions. For another example, for a mobile phone number, the corresponding description may also be performed through a regular expression, so that the mobile phone number may be matched with the corresponding regular expression.

Therefore, it can be understood that, when the rule matching is performed subsequently, the matching is performed with respect to the preset regular expressions therein, and in a possible implementation manner, each preset rule configured includes one preset regular expression. Referring to fig. 4, while inputting the regular expression, in order to determine whether the current regular expression is correct, for example, an input box for performing a test on the regular expression may be provided at a position indicated by 401 in fig. 4, and by setting the test input box, it may be quickly and effectively determined whether the configuration of the current regular expression is correct, so that the efficiency and the accuracy of rule configuration may be effectively improved.

The rule classification may be a specific service type of a currently configured rule, for example, a test, and further, for example, a template classification, etc., and the specific implementation of the rule classification is not limited in this embodiment, where the specific implementation of the rule classification may be set according to an actual service requirement.

And setting a security level corresponding to each preset rule, wherein the security level is the security level in the embodiment. In one possible implementation, the security levels may include "clear text, sensitive, confidential, and confidential" as shown in FIG. 4, which are successively higher, i.e., indicate that the data needs to be protected to an increasing extent.

In an actual implementation process, when rule configuration is performed, each rule includes a preset regular expression, so that for each rule, it can be determined what type of data the current rule is specifically matched with, and then how to correspondingly determine the corresponding data protection degree, so that each preset rule can be set with a corresponding security level, where specific implementation of the security level corresponding to each preset rule can be selected and set according to actual requirements, and this embodiment does not limit this.

And referring to fig. 4, for each configured security rule, before validation, an effective state may be configured, if the effective state is on, the preset rule may be immediately effective after the validation button of fig. 4 is clicked, and if the effective state is off, the preset rule may be temporarily not effective after the validation button of fig. 4 is clicked, and then the preset rule may be effective according to an effective instruction, for example.

The configuration of the preset rule is described above with reference to fig. 4, and the preset rule after configuration is understood with reference to fig. 5.

Referring to fig. 5, what is shown in fig. 5 is a plurality of preset rules configured, for example, for the preset rule indicated by 501, the rule name is "domain name", the rule classification is "template classification", the regular expression included in the rule content is the regular expression identified in fig. 5, it is understood that this is the regular expression describing the domain name, and the rule remark may be "domain name", and the security level thereof is "secret", for example, and a desensitization algorithm corresponding to the current rule may also be included in the preset rule.

Meanwhile, referring to fig. 5, for each preset rule, an operation control corresponding to each preset rule, for example, a close control and an adjust rule control shown in 502 in fig. 5, may also be provided in the operation interface, and when an operation for closing the control is detected, for example, the effective state of the corresponding preset rule may be set to be closed. And when detecting an operation for adjusting the rule control, for example, the rule editing interface shown in fig. 5 may be displayed, and then the user may adjust the preset rule on the rule editing page, and submit the adjusted preset rule after the adjustment is completed, so as to modify the preset rule.

In the actual implementation process, specific options that can be configured in the preset rule, specific implementation of each configuration information in the preset rule, operable content of the operation interface of the preset rule, and the like can all be selected according to actual requirements, which is not particularly limited in this embodiment, as long as the corresponding preset rule can be configured according to the actual requirements, the preset rule includes regular expressions, each regular expression corresponds to its respective security level, and meanwhile, corresponding modification, deletion, and check can be performed on each preset rule.

Therefore, it can be determined based on the above description that, in this embodiment, a plurality of preset regular expressions can be set by configuring the preset rule, and the preset regular expressions can describe corresponding data, so that field data and the preset regular expressions can be matched in this embodiment.

S302, a first field to be identified is obtained, and the first field comprises a plurality of field data.

The implementation of S301 is similar to that of S201, and a possible implementation of obtaining the first field to be identified is further described below.

In this embodiment, the first field to be identified may be, for example, a field currently queried by a user, and in one possible implementation manner of obtaining the first field to be identified, the first field may be obtained according to a received data query request, where the data query request includes target field data to be queried, and the target field data belongs to the first field.

For example, a user may input a corresponding data query requirement in a client, where the data query requirement may indicate what data the user specifically needs to query, and the client may generate a data query request according to the data query requirement input by the user. The current implementation manner is that when the user queries the corresponding data, the security level of the first field where the corresponding data is located is determined in real time.

Alternatively, in this embodiment, the currently stored fields may be sequentially scanned, so as to determine the security level corresponding to each field in advance, and in this implementation manner, before the user queries the data, the security level of each field is generated in advance.

Therefore, in another possible implementation manner of obtaining the first field to be identified, the field message of the first field may be stored in a field message queue, and the field message is used to indicate the security level for identifying the first field, and then, for example, the field message queue may be scanned, and the first field may be obtained according to the field message of the first field in the field message queue.

For example, the storage structure of data may be understood with reference to fig. 6, and as shown in fig. 6, for example, the data storage may include a plurality of databases, such as database 601, databases 602, …, and database 60n shown in fig. 6. And a plurality of tables may be included in each database, for example, a table 6011, a table 6012, a table 6013, a table 6014, etc. may be included in the database 601. And each table may include a plurality of fields, for example, a field indicated by 60111 may be included in table 6011, and in a possible implementation manner, a column in the table may be, for example, one field, or may also be a row in the table is one field, which is not limited in this embodiment.

And a plurality of records may be included in each field, each record being a field data, for example, the field data 601111 may be included in the field 60111 in fig. 6. It is understood that field data included in a field are all data with the same meaning, for example, if the current field 60111 is specifically a "name" field, each field data included in the field 60111 is a name; for another example, if the current field 60111 is specifically a "mobile phone number" field, the data of each field included in the field 60111 is a mobile phone number. In the actual implementation process, the number of the databases, the number of tables in the databases, the number of fields in the tables, the meaning of the fields, the number of field data in the fields, the specific data content of the field data, and the like may all be selected and set according to actual requirements, and the specific implementation manner of the data is not limited in this embodiment.

Therefore, in this embodiment, it may be that the first field belongs to a first table, and the first table belongs to a first database.

Meanwhile, it can be understood that, in the current implementation, the security levels corresponding to the fields are generated in advance, but in an actual data warehouse, the data volume is very large, the security levels corresponding to the fields are determined in advance, the workload is also relatively large, and in order to improve the processing efficiency of the security levels generated in advance for the fields, in one possible implementation, the security level of each field can be identified by using a message queue.

For example, it can be understood in conjunction with fig. 7, as shown in fig. 7, for example, three kinds of message queues, namely a database message queue, a table message queue, and a field message queue, may be provided.

Referring to fig. 7, for example, the service 1 shown in fig. 7 may be performed: the database list is obtained, and the database list may include a plurality of databases, and it is understood that, in this embodiment, each database may be scanned, so that, according to the database list, a database message of each database to be scanned may be stored in a database message queue, where the database message is used to indicate a security level for identifying each field in each table in the corresponding database. That is, each database corresponds to one message in the database message queue, and assuming that there are currently 100 databases, for example, 100 messages may be generated in the database message queue.

And, the database message queue may be scanned to obtain the database message to be identified from the database message queue, where "to be identified" in this embodiment refers to the security level to be identified, and it may be determined based on the above description that each database may include a plurality of tables, and it may be understood that, in this embodiment, each table is scanned, and, for example, referring to fig. 7, service 2 shown in fig. 7 may be executed: the method includes the steps of scanning a database message list, obtaining database messages of a first database, obtaining at least one table to be scanned in the first database according to the database messages of the first database, and then storing table messages corresponding to the at least one table to be scanned into table message queues, wherein each table corresponds to one message in the table message queues. The above assumes that there are currently 100 databases, and that 100 tables are included in each database, a maximum of 10000 messages can be included in the table message queue.

And, the table message queue may be scanned, and the table message to be identified is acquired from the table message queue, it may be determined based on the above description that each table may include a plurality of fields, and it may be understood that, in this embodiment, each field is scanned, and, with reference to fig. 7, for example, the service 3 shown in fig. 7 may be executed: the table message queue is scanned, the table message of the first table is obtained, at least one field to be scanned in the first table is obtained according to the table message of the first table, and then the field message corresponding to each field can be stored in the field message queue, wherein each field corresponds to one message in the field message queue. The above assumes that there are currently 100 databases and that 100 tables are included in each database, while assuming that 50 fields are included in each table, a maximum of 50 ten thousand messages may be included in the field message queue.

The field message of the first field to be identified may then be retrieved from the field message queue, thereby retrieving the first field, and then, for example, the service 4 shown in fig. 5 may be executed: according to the field message, the field data in the corresponding first field is obtained, and then the security level corresponding to the current first field can be determined according to the field data.

As can be understood from the above description, the service 1, the service 2, the service 3, and the service 4 are respectively provided for the data structures of 4 different levels of the database, the table, the field, and the field data. It can be determined that for the above 4 services, each service will produce messages for consumption by the downstream service and will also consume messages produced by the upstream service, so by setting the above consumption queues of different levels, parallel processing between the consumption queues can be realized. For example, while the database list is enqueued in the database message queue, the subsequent processing of the tables and fields can be performed in parallel, and the processing does not need to wait until the database scan is completed.

For example, in order to improve processing efficiency, a plurality of concurrent threads may be provided for the processing unit of each service, so that each service may execute the processing procedure in parallel by multiple lines. Meanwhile, because the quantity difference of the messages in the database message queue, the table message queue and the field message queue is relatively large, different concurrent thread quantities can be set for different services according to different magnitudes of the messages in each message queue and different consumption of consumption resources.

For example, the number of messages to be processed by the service 1 is the minimum, so the number of concurrent threads corresponding to the service 1 can be set to be the minimum, and the number of messages to be consumed by the service 4 is the maximum, so the greater the number of concurrent threads corresponding to the service 4 is, the better, in the actual implementation process, the number of concurrent threads set for each service can be selected and set according to the actual requirements, as long as the efficiency of identifying the security level of each field can be effectively improved.

Therefore, in summary, in this embodiment, the security levels corresponding to the fields may be generated in advance, the security levels of the fields are stored in the preset storage space, and then the security levels of the required fields are directly queried. Or, the security level of the first field where the first field data to be queried is located may be generated in real time when data is queried, and a specific implementation manner of the security level may be selected and set according to actual requirements, which is not limited in this embodiment.

S303, acquiring a first preset number of field data to be matched from the plurality of field data of the first field.

In this embodiment, each field may correspond to a plurality of field data, and currently, for example, a first preset number of field data to be matched may be obtained from the plurality of field data corresponding to the first field. The field data to be matched is the field data to be subsequently matched. It can be understood that the number of field data included in each field is very large, and subsequent matching processing is performed by acquiring a first preset number of field data to be matched, so that the workload of matching processing can be effectively reduced, and the efficiency of determining the security level of the field can be improved.

When a first preset number of field data to be matched is obtained from a plurality of field data corresponding to a first field, for example, N is used to represent the first preset number, for example, the first N field data may be selected as the field data to be matched, or the last N field data may also be selected as the field data to be matched, or N field data may also be randomly selected as the field data to be matched.

And, the first preset number may be, for example, 1000, that means that 1000 field data are selected from the plurality of field data corresponding to the first field as the field data to be matched. In the actual implementation process, the specific implementation of the first preset number may be selected and set according to actual requirements, which is not limited in this embodiment.

S304, acquiring a preset regular expression in the preset rule.

Based on the above description, it may be determined that any one of the preset rules in this embodiment includes a preset regular expression, in this embodiment, for example, each preset rule may correspond to its own security level, and for example, matching may be performed according to the field data and the preset regular expression in each preset rule, and when matching is successful, the security level corresponding to the preset rule may be determined as the security level of the first field corresponding to the current field data.

The implementation manners of any preset rule are similar, so that any preset rule is taken as an example to be introduced below, and the implementation manners of the other preset rules are similar, for example, a preset regular expression of the preset rule may be obtained.

S305, determining the matching result of the first preset number of field data to be matched and the preset regular expression, wherein the matching result is matching success or matching failure.

In this embodiment, a first preset number of fields to be matched and a plurality of preset regular expressions need to be matched.

Specifically, for any preset regular expression, in this embodiment, a matching result of the first preset number of field data to be matched and the preset regular expression may be determined, where the matching result is a matching success or a matching failure.

The matching can be performed for each preset regular expression, so that in this embodiment, for example, the matching results of the first preset number of field data to be matched and each preset regular expression can be determined.

S306, if the matching result is that the number of the successfully matched field data to be matched is larger than or equal to a second preset number, determining the preset rule as a target rule, wherein the second preset number is smaller than or equal to the first preset number.

For any one preset regular expression, after the first preset number, the fields to be matched, and the preset regular expression are matched, the matching result of successful matching or failed matching may be determined, for example, the number of the fields to be matched, which are successfully matched, in the first preset number of the fields to be matched may be determined, then the number of the successfully matched fields is compared with the second preset number, if it is determined that the number of the successfully matched fields is greater than or equal to the second preset number, it may be determined that the current preset regular expression and the current first field are successfully matched, and then the preset rule corresponding to the current regular expression may be determined as the target rule, where the second preset number is less than or equal to the first preset number.

In a possible implementation manner, the second preset number may be half of the first preset number, for example, that is, as long as the matching result of more than half of the fields to be matched in the field data to be matched in the first preset number is a successful matching, it may be determined that the current preset regular expression and the current first field are successfully matched. Or, for example, the second preset number may be 2/3 that is the first preset number, and the like, and this embodiment does not limit a specific implementation manner of the second preset number, and may be selected and set according to actual requirements, and it can be understood that the second preset number is a threshold for measuring whether the current preset regular expression and the first field are successfully matched, as long as the second preset number is smaller than the first preset number.

For convenience of description, the first preset number is 5, the second preset number is taken as an example, as shown in fig. 8, there are currently 5 field data to be matched, which are respectively field data 1, field data 2, field data 3, field data 4, and field data 5 shown in fig. 8, and assuming that matching is currently performed for the preset regular expression a, each field data to be matched may be matched with the preset regular expression a, so as to determine a matching result corresponding to each field data to be matched.

Referring to fig. 8, assuming that matching results corresponding to the current field data 1, the field data 2, the field data 4, and the field data 5 are all matching success, and matching results corresponding to the field data 3 are matching failure, it may be determined that 4 of the 5 field data to be matched are matching success, and therefore, the number of the field data to be matched, whose matching results are matching success, is greater than the second preset number 3, so that the current preset regular expression may be determined as the target regular expression, and further, the preset rule corresponding to the current preset regular expression a may be determined as the target rule.

The above description is directed to a processing procedure of a certain preset regular expression, and in an actual implementation process, the above procedure may be performed for each preset regular expression, so that a target rule may be determined in a plurality of preset rules.

S307, acquiring the number of the target rules.

Based on the above description, it can be determined that the target regular expression in the embodiment is a regular expression successfully matched with the first field, and each preset regular expression corresponds to a respective security level, so that the security level of the target regular expression in the embodiment can be determined as the security level corresponding to the first field. The number of the target regular expressions may be only one, or may be multiple.

The implementation modes of one target regular expression and a plurality of target regular expressions have certain difference, so that the number of target rules is obtained currently.

And S308, if the number of the target rules is 1, determining the security level corresponding to the target rules as the security level corresponding to the first field.

In a possible implementation manner, if the number of the target rules is 1, the security level corresponding to the target rule may be directly determined as the security level corresponding to the first field, for example, if the security level corresponding to the target rule is a secret, it may be determined that the security level corresponding to the current first field is also a secret.

S309, if the number of the target rules is larger than 1, determining the maximum security level corresponding to the target rules as the security level corresponding to the first field.

In another possible implementation manner, if the number of the target rules is greater than 1, it may be determined that a plurality of preset rules matching the first field currently exist, and the maximum security level corresponding to the target rule may be determined as the security level corresponding to the first field.

For example, it can be understood with reference to fig. 9 that, assuming that 3 target rules shown in fig. 9 are currently determined for the first field, which are respectively a preset rule a, a preset rule b, and a preset rule c, and that the security level corresponding to the preset rule a is absolute, the security level corresponding to the preset rule b is secret, and the security level corresponding to the preset rule c is secret, the maximum security level, that is, "absolute secret", among the security levels corresponding to the 3 preset rules can be determined as the security level corresponding to the first field, and then in the example shown in fig. 9, the security level corresponding to the first field is "absolute secret".

In a possible implementation manner, after determining the security level corresponding to each field, the security level of each field may be stored in a preset storage space, for example. Meanwhile, the security level of each field stored in the preset storage space may be updated periodically with a preset time as a period, for example, the updating is performed every day, for example, the updating is performed every month, and the like, which is not limited in this embodiment. And, the specific implementation of the update is to re-execute the above-described procedure, which is not described herein again.

And the security level of each field stored in the preset storage space can also be understood, for example, with reference to fig. 10.

As shown in fig. 10, for each field, for example, the name of the table to which the field belongs, the database to which the field belongs, the application to which the field belongs, and the preset rule for the current specific hit, the corresponding security level, and the state of the current security level being on may be recorded, and these information may be displayed in the graphical user interface, so that the user may quickly and effectively determine the relevant information of each current field.

Say 1001 in fig. 10, it indicates that the table to which field 1 belongs is table 1, the database to which it belongs is database 1, the application to which it belongs is 08021, where 08021 may be, for example, the number of the application, and the rule of hit is an identity number, the current security level is confidential, and the current security level status is open. The remaining field implementations are also described with reference to fig. 10, and will not be described again here.

Meanwhile, referring to fig. 10, for each field, an operation control corresponding to each field, for example, a close control and a security level adjustment control shown in 1002 of fig. 10, may be provided in the graphical user interface, and when an operation for the close control is detected, for example, the security level of the corresponding field may be set to be closed. And when an operation for adjusting the security level control is detected, for example, a dense editing interface can be displayed, and then the user can adjust the security level of the current field in the dense editing page. And a control for querying the security level of the corresponding field may also be provided in the graphical user interface, for example, referring to 1003 in fig. 10, the user may filter the corresponding content to be queried according to the actual requirement.

According to the data processing method provided by the embodiment of the application, the security level corresponding to each field is generated in advance, or the security level corresponding to each field is generated in real time, so that the security level corresponding to each field can be determined flexibly and effectively. When the security level corresponding to the field is specifically determined, the field data to be matched and a preset regular expression are specifically matched according to a first preset number of the field data to be matched in the field, and then the security level corresponding to a matched target regular expression is determined as the security level corresponding to the current field, wherein the regular expression is used for dynamically describing the field data, namely, no matter what change occurs to the field data, matching can be achieved as long as the field data conforms to the rule described by the corresponding regular expression, and compared with an implementation mode of matching according to a fixed preset keyword, the flexibility and effectiveness of matching can be effectively improved by matching according to the regular expression. Meanwhile, currently, a first preset number of field data to be matched are selected for matching processing, so that the workload of matching processing can be effectively reduced, and the processing efficiency of determining the security level of the field is improved.

It should be noted that the first field in this embodiment may include one field, and the target field data may be directly generated according to the display mode corresponding to the current first field and the corresponding first field data. And the first field in this embodiment may further include a plurality of fields, each field in the first field may correspond to a respective display manner, and for example, corresponding processing may be performed on the respective corresponding first field data according to the respective display manner of each field, so as to generate target field data corresponding to each field.

The data processing method provided by the embodiment of the application comprises the following steps: and receiving a data acquisition request sent by a client, wherein the data acquisition request comprises an identifier of a first user and a query instruction, and the query instruction is used for querying first field data in a first field. And acquiring the security level corresponding to the first field, and acquiring the user level of the first user according to the identifier of the first user. And determining a display mode corresponding to the first field according to the security level corresponding to the first field and the user level of the first user, wherein the display mode is a plaintext display mode or a display mode after desensitization treatment. And generating target field data according to the display mode corresponding to the first field and the first field data, and sending the target field data to the client. The security level of the first field queried by the first user is obtained, the user level of the first user is obtained, and then the display mode aiming at the first field is determined according to the security level of the first field and the user level of the first user, so that the specific data processing mode can be effectively determined according to the actual data access requirement of the user and the protection degree of the actual data requirement, and the flexibility of data protection can be effectively improved.

Based on the above-described embodiments, a flow of a specific application scenario for implementing the data processing method provided by the present application is described below with reference to fig. 11 to 13, fig. 11 is a schematic flow diagram of the data processing method provided by the embodiment of the present application, fig. 12 is a schematic diagram for implementing a user level and a security level provided by the embodiment of the present application, and fig. 13 is a schematic diagram for implementing a user level and a security level provided by the embodiment of the present application.

As shown in fig. 11:

1. the querying user may submit a data obtaining request to the querying component through the client, where the data obtaining request may include the identifier of the first user and the querying instruction introduced in the above embodiment;

2. the query component can then query the data in the queried database according to the query instruction;

3. the queried database can return a query result to the query component, and the query result comprises the first field data to be acquired by the user;

4. the query component may pass the identification of the first user and the query results to the desensitization component;

5. the desensitization component determines the table where the query result is located and the first field according to the query result, can also obtain the security level of the first field, can also obtain the user level of the first user according to the identification of the first user, and then determines the display mode of the first field data according to the security level and the user level, wherein the display mode can be plaintext display or display after desensitization processing.

Specifically, in this embodiment, each field corresponds to a respective security level, where the security level corresponding to each field may be preset or may also be generated in real time, and the implementation manner of this may refer to the above description, and is not described herein again.

In a possible implementation manner, for example, user registration corresponding to the identifier of each user may be stored in the preset storage space, and for example, the user level of the first user may be obtained in the preset storage space according to the identifier of the first user.

In a possible implementation, the user level may be divided, for example, as described above: plaintext, sensitive, confidential, etc., or user ratings may also be: the present embodiment also does not limit the specific implementation of the user level division, and the implementation may be selected and set according to actual requirements as long as different user levels corresponding to different data access permissions can be implemented.

It is understood that there is a magnitude relationship between the user level and the security level, and in a possible implementation, the user level and the security level may be completely consistent, and referring to fig. 12, the user level and the security level are 4 levels shown in fig. 12, and the 4 levels are "secret, sensitive, and plaintext" from large to small, that is, secret > sensitive > plaintext, and then the magnitude relationship between the user level and the security level is the current relationship.

As shown in fig. 12, when the user level is confidential, the security level of the data that can be viewed in the clear text includes confidential, sensitive, and clear text; when the user level is confidential, the security level of the data which can be viewed in the clear text comprises the confidentiality, the sensitivity and the clear text; when the user level is sensitive, the security level of the data which can be viewed in the clear text comprises the sensitive and the clear text; when the user rating is in plaintext, the security rating of the data that it can view in plaintext includes the plaintext.

For example, in the example of fig. 12, assuming that the current user level is secret and the security level corresponding to the first field is secret, it may be determined that the user level is equal to the security level corresponding to the first field, and thus it may be determined that the presentation manner of the first field is a plaintext presentation.

For another example, in the example of fig. 12, it is assumed that the current user level is confidential, but the security level corresponding to the first field is absolute, and it can be determined that the user level is less than the security level corresponding to the first field, so that it can be determined that the first field is presented after desensitization processing.

Alternatively, the user level and the security level may not be completely consistent, but there may be a corresponding relationship, referring to fig. 13, for example, the user level may be one level, two levels, three levels, four levels, five levels, six levels, seven levels, and eight levels shown in fig. 13, and the security level may be secret, sensitive, and plaintext shown in fig. 13, and the size relationship shown in fig. 13 may be, for example, that the one level and the two levels of the user may correspond to the secret in the security level, that the three levels and the four levels of the user may correspond to the secret in the security level, that the five levels and the six levels of the user may correspond to the sensitive in the security level, and that the seven levels and the eight levels of the user may correspond to the plaintext in the security level.

That is, when the user level is one level or two levels, the security level of the data that can be viewed in the clear includes confidential, sensitive, and clear text; when the user level is three or four, the security level of the data which can be viewed in the clear text comprises confidentiality, sensitivity and clear text; when the user level is five or six, the security level of the data which can be viewed in the clear text comprises sensitive and clear text; when the user level is seven or eight levels, the security level of the data that can be viewed in the clear includes the clear.

For another example, in the example of fig. 13, assuming that the current user level is four levels and the security level corresponding to the first field is sensitive (clear text access is available at five levels, six levels, and above), it may be determined that the user level is greater than the security level corresponding to the first field, and thus it may be determined that the presentation manner of the first field is clear text presentation.

For another example, in the example of fig. 13, it is assumed that the current user level is eight levels, but the security level corresponding to the first field is sensitive (the five levels and the six levels and above can be accessed in the clear), it may be determined that the user level is less than the security level corresponding to the first field, and thus it may be determined that the presentation manner of the first field is the desensitization presentation.

In the actual implementation process, the specific settings of the user level and the security level may be selected according to actual requirements, and the corresponding relationship between the user level and the security level may also be selected and configured according to actual requirements, so that the size relationship between each user level and each security level may be correspondingly determined, which is not limited in this embodiment.

After determining the security level corresponding to the first field and the user level of the first user, for example, it may be determined whether the user level of the current first user exceeds the security level corresponding to the first field, so as to determine a display manner corresponding to a subsequent first field, where the display manner in this embodiment may be a plaintext display, or may also be a display after desensitization processing.

For example, if the user level of the first user does not exceed the security level corresponding to the first field, it indicates that the right of the current user can directly view the data in the first field, and the presentation mode may be, for example, a plaintext presentation. Or, for example, if the user level of the first user exceeds the security level corresponding to the first field, it indicates that the current user cannot directly view the data in the first field, and the display mode is also displayed after desensitization processing, for example.

In an actual implementation process, the implementation of the specific desensitization processing may be selected and set according to actual requirements, for example, the implementation may include processing manners such as replacement, confusion, and occlusion, which is not limited in this embodiment.

6. After the display mode is determined, the target field data can be obtained according to the first field data, then the target field data is returned to the query component, and then the query component can provide the target field data for the client to display. Therefore, the query operation of the data is completed, and the operation of data protection is flexibly and effectively realized in the middle.

After the display mode corresponding to the first field is determined, target field data to be displayed can be generated according to the display mode corresponding to the first field and the inquired first field data, wherein the target field data can include, for example, the first field data after partial desensitization processing and can also include the first field data without partial desensitization processing, and then the target field data can be sent to the client so that the client can display the target field data.

And then, the obtained target field data can be sent to the client, so that the client can process the target field data to the user, and then for the user, the user can check the data required to be obtained by the user, and the data can be directly displayed or displayed after desensitization processing and depends on the user level of the user and the security level of the field of the currently checked data.

And comparing the security level corresponding to the first field with the user level of the first user to determine whether the plaintext display or the desensitization display is directly performed on the data of the first field at present, so that a specific data access mode can be determined according to the actual protection requirement of the current data and the actual access authority of the user, and the flexible processing of data protection can be effectively realized.

Fig. 14 is a schematic structural diagram of a data processing apparatus according to an embodiment of the present application. As shown in fig. 14, the apparatus 140 includes: a first obtaining module 1401, a second obtaining module 1402, and a determining module 1403.

A first obtaining module 1401, configured to obtain at least one preset rule, where each preset rule corresponds to a security level thereof;

a second obtaining module 1402, configured to obtain a first field to be identified, where the first field includes a plurality of field data;

a determining module 1403, configured to determine a target security level corresponding to the first field according to matching between the at least one preset rule and the field data in the first field.

In one possible design, the determining module 1403 is specifically configured to:

the determining module 1403 is specifically configured to:

acquiring a preset regular expression in the preset rule;

acquiring the number of the target rules;

In one possible design, the determining module 1403 is further configured to:

In one possible design, a field message of the first field is stored in a field message queue, the field message indicating a security level identifying the first field; the second obtaining module 1402 is specifically configured to:

the second obtaining module 1402 is further configured to:

The apparatus provided in this embodiment may be used to implement the technical solutions of the above method embodiments, and the implementation principles and technical effects are similar, which are not described herein again.

Fig. 15 is a schematic diagram of a hardware structure of a data processing apparatus according to an embodiment of the present application, and as shown in fig. 15, a data processing apparatus 150 according to the embodiment includes: a processor 1501 and a memory 1502; wherein

A memory 1502 for storing computer-executable instructions;

the processor 1501 is configured to execute the computer-executable instructions stored in the memory, so as to implement the steps performed by the data processing method in the foregoing embodiments. Reference may be made in particular to the description relating to the method embodiments described above.

Alternatively, the memory 1502 may be separate or integrated with the processor 1501.

When the memory 1502 is provided separately, the data processing apparatus further includes a bus 1503 for connecting the memory 1502 and the processor 1501.

An embodiment of the present application further provides a computer-readable storage medium, where a computer executing instruction is stored in the computer-readable storage medium, and when a processor executes the computer executing instruction, the data processing method executed by the data processing apparatus is implemented.

In the several embodiments provided in the present application, it should be understood that the disclosed apparatus and method may be implemented in other ways. For example, the above-described device embodiments are merely illustrative, and for example, the division of the modules is only one logical division, and other divisions may be realized in practice, for example, a plurality of modules may be combined or integrated into another system, or some features may be omitted, or not executed. In addition, the shown or discussed mutual coupling or direct coupling or communication connection may be an indirect coupling or communication connection through some interfaces, devices or modules, and may be in an electrical, mechanical or other form.

The integrated module implemented in the form of a software functional module may be stored in a computer-readable storage medium. The software functional module is stored in a storage medium and includes several instructions for enabling a computer device (which may be a personal computer, a server, or a network device) or a processor (processor) to execute some steps of the methods according to the embodiments of the present application.

It should be understood that the Processor may be a Central Processing Unit (CPU), other general purpose Processor, a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), etc. A general purpose processor may be a microprocessor or the processor may be any conventional processor or the like. The steps of a method disclosed in connection with the present invention may be embodied directly in a hardware processor, or in a combination of the hardware and software modules within the processor.

The memory may comprise a high-speed RAM memory, and may further comprise a non-volatile storage NVM, such as at least one disk memory, and may also be a usb disk, a removable hard disk, a read-only memory, a magnetic or optical disk, etc.

The bus may be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, an Extended ISA (EISA) bus, or the like. The bus may be divided into an address bus, a data bus, a control bus, etc. For ease of illustration, the buses in the figures of the present application are not limited to only one bus or one type of bus.

The storage medium may be implemented by any type or combination of volatile or non-volatile memory devices, such as Static Random Access Memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic or optical disks. A storage media may be any available media that can be accessed by a general purpose or special purpose computer.

Those of ordinary skill in the art will understand that: all or a portion of the steps of implementing the above-described method embodiments may be performed by hardware associated with program instructions. The program may be stored in a computer-readable storage medium. When executed, the program performs steps comprising the method embodiments described above; and the aforementioned storage medium includes: various media that can store program codes, such as ROM, RAM, magnetic or optical disks.

Finally, it should be noted that: the above embodiments are only used for illustrating the technical solutions of the present application, and not for limiting the same; although the present application has been described in detail with reference to the foregoing embodiments, it should be understood by those of ordinary skill in the art that: the technical solutions described in the foregoing embodiments may still be modified, or some or all of the technical features may be equivalently replaced; and the modifications or the substitutions do not make the essence of the corresponding technical solutions depart from the scope of the technical solutions of the embodiments of the present application.

Claims

1. A data processing method, comprising:

2. The method according to claim 1, wherein the determining the target security level corresponding to the first field according to the matching between the at least one preset rule and the field data in the first field comprises:

3. The method according to claim 2, wherein for any one of the preset rules, the preset rule comprises a preset regular expression;

acquiring a preset regular expression in the preset rule;

4. The method according to claim 2 or 3, wherein the determining a target security level corresponding to the first field according to the security level corresponding to the target rule comprises:

acquiring the number of the target rules;

5. The method according to any one of claims 1-4, wherein after determining the target security level corresponding to the first field, the method further comprises:

6. The method of any of claims 1-5, wherein a field message for the first field is stored in a field message queue, the field message indicating a security level identifying the first field; the acquiring a first field to be identified includes:

7. The method of claim 6, wherein the first field belongs to a first table, and wherein the first table belongs to a first database; table messages of the first table are stored in a table message queue, wherein the table messages are used for indicating security levels for identifying various fields in the first table; storing database messages of the first database in a database message queue, the database messages being used to indicate security levels identifying respective fields in the first database;

8. A data processing apparatus, comprising:

9. The apparatus of claim 8, wherein the determining module is specifically configured to:

10. The apparatus according to claim 9, wherein for any of the preset rules, a preset regular expression is included in the preset rule;

the determining module is specifically configured to:

acquiring a preset regular expression in the preset rule;

11. The apparatus according to claim 9 or 10, wherein the determining module is specifically configured to:

acquiring the number of the target rules;

12. The apparatus of any of claims 8-11, wherein the determining module is further configured to:

13. The apparatus of any of claims 8-12, wherein a field message for the first field is stored in a field message queue, the field message indicating a security level identifying the first field; the second obtaining module is specifically configured to:

14. The apparatus of claim 13, wherein the first field belongs to a first table, and wherein the first table belongs to a first database; table messages of the first table are stored in a table message queue, wherein the table messages are used for indicating security levels for identifying various fields in the first table; storing database messages of the first database in a database message queue, the database messages being used to indicate security levels identifying respective fields in the first database;

the second obtaining module is further configured to:

15. A data processing apparatus, characterized by comprising:

a memory for storing a program;

a processor for executing the program stored by the memory, the processor being configured to perform the method of any of claims 1 to 7 when the program is executed.

16. A computer-readable storage medium comprising instructions which, when executed on a computer, cause the computer to perform the method of any one of claims 1 to 7.

17. A computer program product comprising a computer program, characterized in that the computer program realizes the method of any of claims 1 to 7 when executed by a processor.