CN116702199A - Data processing method, device, server and storage medium - Google Patents

Data processing method, device, server and storage medium Download PDF

Info

Publication number
CN116702199A
CN116702199A CN202310673144.2A CN202310673144A CN116702199A CN 116702199 A CN116702199 A CN 116702199A CN 202310673144 A CN202310673144 A CN 202310673144A CN 116702199 A CN116702199 A CN 116702199A
Authority
CN
China
Prior art keywords
desensitization
data
desensitized
user
processing
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202310673144.2A
Other languages
Chinese (zh)
Inventor
闫宇
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Industrial and Commercial Bank of China Ltd ICBC
Original Assignee
Industrial and Commercial Bank of China Ltd ICBC
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Industrial and Commercial Bank of China Ltd ICBC filed Critical Industrial and Commercial Bank of China Ltd ICBC
Priority to CN202310673144.2A priority Critical patent/CN116702199A/en
Publication of CN116702199A publication Critical patent/CN116702199A/en
Pending legal-status Critical Current

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F21/00—Security arrangements for protecting computers, components thereof, programs or data against unauthorised activity
    • G06F21/60—Protecting data
    • G06F21/62—Protecting access to data via a platform, e.g. using keys or access control rules
    • G06F21/6218—Protecting access to data via a platform, e.g. using keys or access control rules to a system of files or objects, e.g. local or distributed file system or database
    • G06F21/6245—Protecting personal data, e.g. for financial or medical purposes
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/22—Indexing; Data structures therefor; Storage structures
    • G06F16/2282—Tablespace storage structures; Management thereof
    • Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
    • Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
    • Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Health & Medical Sciences (AREA)
  • Databases & Information Systems (AREA)
  • Software Systems (AREA)
  • Health & Medical Sciences (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Bioethics (AREA)
  • Medical Informatics (AREA)
  • Computer Hardware Design (AREA)
  • Computer Security & Cryptography (AREA)
  • Data Mining & Analysis (AREA)
  • Management, Administration, Business Operations System, And Electronic Commerce (AREA)

Abstract

The application provides a data processing method, a data processing device, a server and a storage medium, and relates to the field of big data. The method comprises the following steps: acquiring an original service data table from a data source system; the original service data table comprises a plurality of data; according to the type of sensitive fields and the desensitization processing requirement information in the desensitization strategy corresponding to each preset user level, carrying out desensitization processing on the data to be desensitized in the original service data table, obtaining a service data table after desensitization, and storing the service data table after desensitization into a storage area corresponding to the user level in a data lake. The method of the application can carry out desensitization treatment of different degrees for users of different levels, provides a more reasonable desensitization treatment mode, and can store the data table after the desensitization treatment in different areas based on the user level, thereby facilitating the inquiry of the subsequent data.

Description

Data processing method, device, server and storage medium
Technical Field
The present application relates to the field of big data technologies, and in particular, to a data processing method, a device, a server, and a storage medium.
Background
With the development of information technology, sensitive information lacks management and control in the data circulation process, if sensitive information leaks, unnecessary risks are brought to clients, and therefore, the sensitive information needs to be managed and controlled. In the financial industry, sensitive information includes biometric information, identity information, property information, and the like.
Related staff inquires service data from a database to analyze the service, sensitive information such as customer names, mobile phone numbers, address information, card numbers, customer numbers and the like is recorded in the service data, and the service data is usually subjected to desensitization processing, for example, the sensitive information is subjected to data deformation or shielding, the related staff cannot check the original service data, and only the service data subjected to the desensitization processing can be checked, so that the privacy of customers is effectively protected.
However, some workers need to view some original business data when carrying out business due to business requirements, but the business data presented to each worker after the current desensitization processing is the same, and the current desensitization processing mode is not reasonable.
Disclosure of Invention
The application provides a data processing method, a data processing device, a server and a storage medium, which are used for solving the problem that the existing desensitization processing mode is unreasonable.
In a first aspect, the present application provides a data processing method, including:
acquiring an original service data table from a data source system; the original service data table comprises a plurality of data;
according to the type of sensitive fields and the desensitization processing requirement information in the desensitization strategy corresponding to each preset user level, carrying out desensitization processing on the data to be desensitized in the original service data table, obtaining a service data table after desensitization, and storing the service data table after desensitization into a storage area corresponding to the user level in a data lake.
Optionally, the desensitizing processing is performed on the data to be desensitized in the original service data table according to the type of the sensitive field and the desensitizing processing requirement information in the desensitizing policy corresponding to each preconfigured user level, including:
for each user level, acquiring data to be desensitized corresponding to a sensitive field type in a desensitization strategy corresponding to the user level from the original service data table;
and determining whether the data to be desensitized meets the desensitization condition according to the desensitization processing demand information in the desensitization strategy corresponding to the user level, and carrying out desensitization processing on the data to be desensitized when the desensitization condition is met.
Optionally, if the preconfigured user level is a first level user and the corresponding desensitization policy is the first desensitization policy, the sensitive field type includes one or a combination of several of the following: user name, user address, and user contact.
Optionally, if the user level is a first level user and the desensitization policy corresponding to the user level is a first desensitization policy, determining whether the data to be desensitized meets the desensitization condition according to the desensitization processing requirement information in the desensitization policy corresponding to the user level, and performing desensitization processing on the data to be desensitized when the desensitization condition is met, where the step of performing the desensitization processing includes:
when the sensitive field type is a user name, determining whether first data to be desensitized corresponding to the user name meets a desensitization condition according to the rarely used word conversion desensitization processing requirement information in the first desensitization strategy, and performing rarely used word conversion processing on the first data to be desensitized when the desensitization condition is met;
and/or the number of the groups of groups,
when the sensitive field type is a user address, determining whether second data to be desensitized corresponding to the user address meets a desensitization condition according to the information of the desensitization processing requirement of the omitted part of the content in the desensitization processing requirement information in the first desensitization strategy, and carrying out omitted processing on the second data to be desensitized when the desensitization condition is met;
and/or;
when the sensitive field type is a user contact way, determining whether third data to be desensitized corresponding to the user contact way meets a desensitization condition according to digital conversion desensitization processing requirement information in the first desensitization strategy, and performing digital conversion processing on the third data to be desensitized when the desensitization condition is met.
Optionally, if the preconfigured user level is a second level user and the corresponding desensitization policy is a second desensitization policy, the sensitive field type includes one or a combination of several of the following: a user name and a user address.
Optionally, if the preconfigured user level is a second level user and the corresponding desensitization policy is a second desensitization policy, determining whether the data to be desensitized meets the desensitization condition according to the desensitization processing requirement information in the desensitization policy corresponding to the user level, and performing the desensitization processing on the data to be desensitized when the desensitization condition is met, where the step of determining includes:
when the sensitive field type is a user name, determining whether first data to be desensitized corresponding to the user name meets a desensitization condition according to the rarely used word conversion desensitization processing requirement information in the second desensitization strategy, and performing rarely used word conversion processing on the first data to be desensitized when the desensitization condition is met;
and/or the number of the groups of groups,
when the sensitive field type is a user address, determining whether second data to be desensitized corresponding to the user address meets a desensitization condition according to the information of the desensitization processing requirement of the omitted part of the information of the desensitization processing requirement in the second desensitization strategy, and carrying out omitted processing on the second data to be desensitized when the desensitization condition is met.
Optionally, the method further comprises:
receiving a data query request sent by an analysis platform, analyzing the request, and acquiring an identification of the analysis platform and an identification of a service data table to be queried;
acquiring a user level corresponding to the identification of an analysis platform, and acquiring a desensitized service data table corresponding to the identification of the service data table to be queried from a storage area corresponding to the user level;
and sending the desensitized service data table to the analysis platform so that the analysis platform can display the desensitized service data table.
In a second aspect, the present application provides a data processing apparatus comprising:
the acquisition unit is used for acquiring an original service data table from the data source system; the original service data table comprises a plurality of data;
the processing unit is used for carrying out desensitization processing on the data to be desensitized in the original service data table according to the type of the sensitive field and the desensitization processing requirement information in the desensitization strategy corresponding to each preset user level, obtaining a service data table after desensitization, and storing the service data table after desensitization into a storage area corresponding to the user level in a data lake.
In a third aspect, the present application provides a server comprising: a processor, a memory, and a transceiver;
a processor, memory, and transceiver circuitry interconnect;
the memory stores computer-executable instructions;
a transceiver for transceiving data;
the processor executes computer-executable instructions stored in the memory to cause the processor to perform the method as described in the first aspect.
In a fourth aspect, the present application provides a computer-readable storage medium having stored therein computer-executable instructions for performing the method according to the first aspect when executed by a processor.
In a fifth aspect, the application provides a computer program product comprising a computer program which, when executed by a processor, implements the method of the first aspect.
The application provides a data processing method, a device, a server and a storage medium, wherein an original service data table is obtained from a data source system; the original service data table comprises a plurality of data; according to the type of the sensitive field and the desensitization processing requirement information in the desensitization strategy corresponding to each preset user level, the desensitization processing is carried out on the data to be desensitized in the original service data table, the service data table after the desensitization is obtained, and the service data table after the desensitization is stored in a storage area corresponding to the user level in a data lake, so that the desensitization processing can be carried out to users of different levels in different degrees, a more reasonable desensitization processing mode is provided, and the data table after the desensitization processing can be stored in different areas based on the user level, so that the inquiry of subsequent data is facilitated.
Drawings
The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the application and together with the description, serve to explain the principles of the application.
FIG. 1 is a schematic diagram of a network architecture of a data processing method according to the present application;
FIG. 2 is a schematic flow chart of a data processing method according to the present application;
FIG. 3 is a flow chart of another data processing method according to the present application;
FIG. 4 is a schematic diagram of a data processing apparatus according to the present application;
fig. 5 is a block diagram of a server for implementing the data processing method of the present application.
Specific embodiments of the present application have been shown by way of the above drawings and will be described in more detail below. The drawings and the written description are not intended to limit the scope of the inventive concepts in any way, but rather to illustrate the inventive concepts to those skilled in the art by reference to the specific embodiments.
Detailed Description
Reference will now be made in detail to exemplary embodiments, examples of which are illustrated in the accompanying drawings. When the following description refers to the accompanying drawings, the same numbers in different drawings refer to the same or similar elements, unless otherwise indicated. The implementations described in the following exemplary examples do not represent all implementations consistent with the application. Rather, they are merely examples of apparatus and methods consistent with aspects of the application as detailed in the accompanying claims.
It should be noted that, the user information (including but not limited to user equipment information, user personal information, etc.) and the data (including but not limited to data for analysis, stored data, presented data, etc.) related to the present application are information and data authorized by the user or fully authorized by each party, and the collection, use and processing of the related data need to comply with related laws and regulations and standards, and provide corresponding operation entries for the user to select authorization or rejection.
It should be noted that the data processing method, apparatus, server and storage medium of the present application may be used in the technical field of big data, and may be used in any field other than the technical field of big data.
For a clear understanding of the technical solutions of the present application, the prior art solutions will be described in detail first.
At present, related workers inquire service data from a database to analyze the service, sensitive information such as customer names, mobile phone numbers, address information, card numbers, customer numbers and the like is recorded in the service data, and usually desensitization treatment is carried out on the service data, for example, the sensitive information is subjected to data deformation or shielding, the related workers cannot check the original service data, and only the service data subjected to the desensitization treatment can be checked, so that the privacy of the customer is effectively protected.
However, some workers need to view some original business data when carrying out business due to business requirements, but the business data presented to each worker after the current desensitization processing is the same, and the current desensitization processing mode is not reasonable.
Therefore, the inventor finds that the original business data table is obtained from the data source system in the research aiming at the problem that the existing desensitization processing mode is unreasonable, the original business data table comprises a plurality of data, the desensitization processing is carried out on the data to be desensitized in the original business data table according to the sensitive field type and the desensitization processing requirement information in the desensitization strategy corresponding to each user level which is preconfigured, the business data table after the desensitization processing is obtained, the business data table after the desensitization is stored in a storage area corresponding to the user level in a data lake, the desensitization processing can be carried out to users with different levels, a more reasonable desensitization processing mode is provided, and the data table after the desensitization processing can be stored in a region based on the user level, so that the subsequent data can be conveniently queried.
The inventor proposes the technical scheme of the embodiment of the application based on the creative discovery. The network architecture and application scenario of the data processing method provided by the embodiment of the application are described below.
As shown in fig. 1, a network architecture corresponding to a method for constructing an application provided by an embodiment of the present application includes: the system comprises a data source system 1, a server 2 and an analysis platform 3 corresponding to a data lake, wherein the server 2 is respectively in communication connection with the data source system 1 and the analysis platform 3, a plurality of data source systems are in communication connection with the data lake, and the server 2 acquires an original service data table from the data source systems; the original business data table comprises a plurality of data; the server 2 performs desensitization processing on the data to be desensitized in the original service data table according to the type of the sensitive field and the desensitization processing requirement information in the desensitization strategy corresponding to each preset user level, acquires the service data table after desensitization, stores the service data table after desensitization in a storage area corresponding to the user level in a data lake, and stores the service data table after desensitization in the storage area corresponding to the user level in the data lake, so that different levels of users can be desensitized to different degrees, a more reasonable desensitization processing mode is provided, and the data table after desensitization processing can be stored in different areas based on the user level, so that the subsequent data can be queried conveniently.
Embodiments of the present application will be described in detail below with reference to the accompanying drawings.
Fig. 2 is a schematic flow chart of a data processing method provided by the application, and the method is applied to a server. As shown in fig. 2, the method includes:
step 201, obtaining an original service data table from a data source system; the original service data table includes a plurality of data.
In this embodiment, a plurality of data source systems are communicatively coupled to the data lake, the data source systems being configured to process transactions and manage transactions, the data source systems including a credit system, a credit card transaction system, etc., the data source systems generating a large amount of data each day. Each data source system generates a data table from the generated data, sends the data table to a data lake, and the data table sending the data lake may be referred to as lake-entering data. The data lake is used for storing the data table sent by each data source system, the data lake is used for unified management, the data table obtained by the data lake from the data source system is called an original service data table, the original service data table comprises a plurality of data, and each piece of data comprises a plurality of fields including a user number, a user name, a user address, an account opening area code, an account opening network point number and a contact way.
Step 202, according to the type of sensitive field and the information of the desensitization processing requirement in the desensitization strategy corresponding to each preset user level, carrying out the desensitization processing on the data to be desensitized in the original service data table, obtaining a service data table after desensitization, and storing the service data table after desensitization into a storage area corresponding to the user level in a data lake.
In this embodiment, fields such as a user name and a user address in an original service data table belong to sensitive fields, when a related service person queries data, a desensitization process is required to be performed in advance for a sensitive data portion in the original service data table, the desensitized service data table is displayed to the related service person, different desensitization processes are required to be performed for different service persons of different levels, the service data tables after the desensitization process are presented to users of different levels, specifically, according to a type of a sensitive field and desensitization process requirement information in a desensitization strategy corresponding to each user level which is preconfigured, the desensitization process is performed on data to be desensitized in the original service data table, a service data table after the desensitization process is obtained, in order to facilitate subsequent data table retrieval, the service data table after the desensitization process is stored in a storage area corresponding to a user level in a data lake, different storage areas are divided for users of different levels in advance, and the service data table after the desensitization process corresponding to users of the same level is stored in the same area. A data lake is a large warehouse that stores a wide variety of raw data that is available for access, processing, analysis, and transmission. The data lake acquires original data tables from a plurality of data source systems, and the same original data table is provided with the data tables after desensitization processing corresponding to users with different levels.
In this embodiment, an original service data table is obtained from a data source system, where the original service data table includes a plurality of data, according to the type of a sensitive field and the information of a requirement for desensitization processing in a desensitization policy corresponding to each user level that is preconfigured, desensitization processing is performed on data to be desensitized in the original service data table, so as to obtain a service data table after desensitization processing, and the service data table after desensitization is stored in a storage area corresponding to the user level in a data lake, so that desensitization processing can be performed to users of different levels to different extents, a more reasonable desensitization processing mode is provided, and the data table after desensitization processing can be stored in a partitioned manner based on the user level, so that subsequent data can be queried conveniently.
Fig. 3 is a flow chart of another data processing method provided in the present application, where the method is applied to a server, as shown in fig. 3, and the method includes:
step 301, obtaining an original service data table from a data source system; the original service data table includes a plurality of pieces of data.
In this embodiment, a plurality of data source systems are communicatively coupled to the data lake, the data source systems being configured to process transactions and manage transactions, the data source systems including a credit system, a credit card transaction system, etc., the data source systems generating a large amount of data each day. Each data source system generates a data table from the generated data, sends the data table to a data lake, and the data table sending the data lake may be referred to as lake-entering data. The data lake is used for storing the data table sent by each data source system, the data lake is used for unified management, the data table obtained by the data lake from the data source system is called an original service data table, the original service data table comprises a plurality of data, and each piece of data comprises a plurality of fields including a user number, a user name, a user address, an account opening area code, an account opening network point number and a contact way.
Step 302, for each user level, obtaining data to be desensitized corresponding to the sensitive field type in the desensitization strategy corresponding to the user level from the original service data table.
In this embodiment, for each user level, data to be desensitized corresponding to a sensitive field type in a desensitization policy corresponding to the user level is obtained from an original service data table, the desensitization policies corresponding to users of different levels are different, the data to be desensitized corresponding to the users are also different, and if the preconfigured user level is a first level user, the corresponding desensitization policy is a first desensitization policy; if the preconfigured user level is a second level user, the corresponding desensitization strategy is a second desensitization strategy, and the types of sensitive fields and the desensitization processing requirement information corresponding to different strategies are different. The second level user is higher than the first level user in level, and can view that the data is different from the first level user and corresponds to different desensitization processing modes.
Step 303, determining whether the data to be desensitized meets the desensitization condition according to the desensitization processing requirement information in the desensitization strategy corresponding to the user level, and carrying out the desensitization processing on the data to be desensitized when the desensitization condition is met.
In this embodiment, whether the data to be desensitized meets the desensitization condition is determined according to the desensitization processing requirement information in the desensitization strategy corresponding to the user level, and when the desensitization condition is met, the desensitization processing is performed on the data to be desensitized, so as to obtain a data table subjected to the desensitization processing.
Optionally, if the preconfigured user level is a first level user and the corresponding desensitization policy is the first desensitization policy, the sensitive field type includes one or a combination of several of the following: user name, user address, and user contact.
In this embodiment, if the preconfigured user level is a first level user, the desensitization policy corresponding to the user is a first desensitization policy, and the types of sensitive fields in the first desensitization policy include one or a combination of several of the following: user name, user address, and user contact.
Optionally, if the user level is the first level user and the corresponding desensitization policy is the first desensitization policy, determining whether the data to be desensitized meets the desensitization condition according to the desensitization processing requirement information in the desensitization policy corresponding to the user level, and performing the desensitization processing on the data to be desensitized when the data to be desensitized meets the desensitization condition, where the step of performing the desensitization processing includes:
when the sensitive field type is a user name, determining whether the first data to be desensitized corresponding to the user name meets a desensitization condition according to the rarely used word conversion desensitization processing requirement information in the first desensitization strategy, and performing rarely used word conversion processing on the first data to be desensitized when the desensitization condition is met;
and/or the number of the groups of groups,
when the sensitive field type is a user address, determining whether second data to be desensitized corresponding to the user address meets a desensitization condition according to the desensitization processing requirement information of the omitted part of content in the desensitization processing requirement information in the first desensitization strategy, and carrying out omitted processing on the second data to be desensitized when the desensitization condition is met;
and/or;
when the sensitive field type is the user contact information, determining whether third data to be desensitized corresponding to the user contact information meets the desensitization condition according to the digital conversion desensitization processing requirement information in the first desensitization strategy, and performing digital conversion processing on the third data to be desensitized when the desensitization condition is met.
In this embodiment, the types of the sensitive fields are different, and the corresponding desensitization processing modes are different, and if the user level is the first level user and the corresponding desensitization policy is the first desensitization policy, the desensitization processing modes are as follows: when the sensitive field type is a user name, in order to avoid privacy disclosure of the user, the data to be desensitized corresponding to the sensitive field type is required to be desensitized, whether the data to be desensitized corresponding to the user name meet the desensitization condition is determined according to the rarely used word conversion desensitization processing requirement information in the first desensitization strategy, when the desensitization condition is met, the data to be desensitized are subjected to rarely used word conversion processing, and the real names of the users corresponding to the sensitive field such as the user name are converted into rarely used words, so that the real names are not displayed.
In this embodiment, when the type of the sensitive field is the user address, according to the information of the desensitization processing requirement of the omission part in the desensitization processing requirement information in the first desensitization policy, it is determined whether the second data to be desensitized corresponding to the user address meets the desensitization condition, and when the second data to be desensitized meets the desensitization condition, the omission processing is performed on the second data to be desensitized, specifically, the omission processing is performed on preset data in the second data to be desensitized, where the preset data may be an address text part and/or an address number part, the number part is a house number, and the real address of the user is partially omitted, so that the complete address information is not displayed.
In this embodiment, when the sensitive field is the user contact way, for example, the mobile phone number, according to the digital conversion desensitization processing requirement information in the first desensitization policy, it is determined whether the third data to be desensitized corresponding to the user contact way meets the desensitization condition, and when the desensitization condition is met, digital conversion processing is performed on the third data to be desensitized, so as to convert the real mobile phone number of the user into the false mobile phone number.
Referring to table 1, table 1 is an original service data table, where the original service data table includes a user number, a user name, a user address, an area code of an opening, an opening network point number, and a contact way, and if the user level is a first level user and the corresponding desensitization policy is a first desensitization policy, the sensitive field type includes one or a combination of several of the following: user name, user address and user contact means; if the sensitive field is a user name, determining whether the first data to be desensitized corresponding to the user name meets the desensitization condition according to the rare word conversion desensitization processing requirement information in the first desensitization strategy, and performing rare word conversion processing on the first data to be desensitized corresponding to the user name in table 1 when the first data to be desensitized meets the desensitization condition, and performing rare word conversion processing on the first data to be desensitized corresponding to the user name in table 1, wherein table 2 is a service data table after desensitization, and the table Gu Yi is converted into a service data table after desensitization. And/or if the sensitive field is a user address, determining whether the second data to be desensitized corresponding to the user address meets the desensitization condition according to the rarely used word conversion desensitization processing requirement information in the first desensitization strategy, and omitting the second data to be desensitized when the second data to be desensitized meets the desensitization condition, wherein the second data to be desensitized corresponding to the user address in table 1 is "Shanghai Pudong new area Puda field No. B room", the omitting process is performed on preset data in the second data to be desensitized, specifically, the preset data may be an address text part and/or an address number part, the number part is a house number, see table 2, "the" Chan A of the new area of Shanghai city Pudong "is converted into" Chan of the new area of Shanghai city Pudong ", and the real address is hidden". And/or if the sensitive field is the user contact way, determining whether the data to be desensitized corresponding to the user contact way meets the desensitization condition according to the digital conversion desensitization processing requirement information in the first desensitization strategy, and performing digital conversion processing on the data to be desensitized corresponding to the user contact way in table 1 when the data to be desensitized meets the desensitization condition, wherein the data to be desensitized corresponding to the user contact way in table 1 is '1995478122', and performing digital conversion processing on the data to be desensitized corresponding to the user contact way, see '1995478122' in table 2 is converted into '1556956132', and the data to be desensitized is converted into a virtual contact way. The desensitized service data table corresponding to the first-level user is shown in table 2, and the desensitization processing is not performed on the data corresponding to the non-sensitive fields such as the user number, the area code of the account opening place and the network point number of the account opening.
Table 1 original service data table
TABLE 2 service data sheet after desensitization
Optionally, if the preconfigured user level is a second level user and the corresponding desensitization policy is a second desensitization policy, the sensitive field type includes one or a combination of the following: a user name and a user address.
In this embodiment, if the preconfigured user level is the second level user, the desensitization policy corresponding to the user is the second desensitization policy, and the type of the sensitive field in the second desensitization policy includes one or a combination of several of the following: a user name and a user address.
Optionally, if the preconfigured user level is a second level user and the corresponding desensitization policy is the second desensitization policy, determining whether the data to be desensitized meets the desensitization condition according to the desensitization processing requirement information in the desensitization policy corresponding to the user level, and performing the desensitization processing on the data to be desensitized when the desensitization condition is met, where the step of performing the desensitization processing includes:
when the sensitive field type is a user name, determining whether the first data to be desensitized corresponding to the user name meets a desensitization condition according to the rarely used word conversion desensitization processing requirement information in the second desensitization strategy, and performing rarely used word conversion processing on the first data to be desensitized when the desensitization condition is met;
and/or the number of the groups of groups,
when the sensitive field type is the user address, determining whether the second data to be desensitized corresponding to the user address meets the desensitization condition according to the information of the desensitization processing requirement of the omission part in the information of the desensitization processing requirement in the second desensitization strategy, and performing omission processing on the second data to be desensitized when the desensitization condition is met.
In this embodiment, the types of the sensitive fields are different, the corresponding desensitization processing modes are different, and if the user level is the second level user and the corresponding desensitization policy is the second desensitization policy, the desensitization processing modes are as follows: when the sensitive field type is a user name, in order to avoid privacy disclosure of the user, the data to be desensitized corresponding to the sensitive field type is required to be desensitized, whether the data to be desensitized corresponding to the user name meet the desensitization condition is determined according to the rarely used word conversion desensitization processing requirement information in the first desensitization strategy, when the desensitization condition is met, the data to be desensitized are subjected to rarely used word conversion processing, and the real names of the users corresponding to the sensitive field such as the user name are converted into rarely used words, so that the real names are not displayed.
In this embodiment, when the type of the sensitive field is the user address, according to the information of the desensitization processing requirement of the omitted part of the information of the desensitization processing requirement in the second desensitization strategy, it is determined whether the second data to be desensitized corresponding to the user address meets the desensitization condition, and when the desensitization condition is met, the second data to be desensitized is omitted, and the real address of the user is partially omitted, so that the complete address information is not displayed.
Based on different user levels, the application carries out rare word conversion processing/omitting processing/digital conversion processing on the data to be desensitized corresponding to the sensitive fields according to different corresponding desensitization strategies, adopts different desensitization processing modes aiming at different field types, and finally obtains a service data table subjected to desensitization processing, and presents service data subjected to different desensitization processing for users of different levels.
Optionally, the data processing method further includes:
receiving a data query request sent by an analysis platform, analyzing the request, and acquiring an identification of the analysis platform and an identification of a service data table to be queried; acquiring a user level corresponding to the identification of the analysis platform, and acquiring a desensitized service data table corresponding to the identification of the service data table to be queried from a storage area corresponding to the user level; and sending the desensitized service data table to an analysis platform for the analysis platform to display the desensitized service data table.
In this embodiment, a data query request sent by an analysis platform is received, the data query request is analyzed, an identification of the analysis platform and an identification of a service data table to be queried are obtained, the identification of the service data table to be queried can be a name of the data table, a user level corresponding to the identification of the analysis platform is obtained, a desensitized service data table corresponding to the identification of the service data table to be queried is obtained from a storage area corresponding to the user level according to the user level, the desensitized service data table is a data table desensitized to different degrees, and the desensitized service data table is sent to the analysis platform so as to provide a more reasonable desensitization processing mode for displaying the desensitized service data table for users, and the original service data table is not displayed to the users, so that the privacy of the users is ensured, and different service data are displayed for users of different levels.
The application can carry out desensitization treatment to different degrees for users of different levels, provides a more reasonable desensitization treatment mode, and can store the data table after the desensitization treatment in different areas based on the user level, thereby facilitating the inquiry of the subsequent data.
Fig. 4 is a schematic structural diagram of a data processing apparatus according to the present application, and as shown in fig. 4, a data processing apparatus 400 according to the present embodiment includes an obtaining unit 401 and a processing unit 402.
The acquiring unit 401 is configured to acquire an original service data table from a data source system; the original service data table includes a plurality of data. The processing unit 402 is configured to perform desensitization processing on data to be desensitized in the original service data table according to the type of a sensitive field and information of a desensitization processing requirement in a desensitization policy corresponding to each preset user level, obtain a service data table after desensitization, and store the service data table after desensitization in a storage area corresponding to the user level in a data lake.
Optionally, the processing unit 402 is further configured to obtain, for each user level, data to be desensitized corresponding to a sensitive field type in the desensitization policy corresponding to the user level from the original service data table; and determining whether the data to be desensitized meets the desensitization condition according to the desensitization processing demand information in the desensitization strategy corresponding to the user level, and carrying out desensitization processing on the data to be desensitized when the desensitization condition is met.
Optionally, the processing unit 402 is further configured to determine, when the sensitive field type is a user name, whether the first data to be desensitized corresponding to the user name meets a desensitization condition according to the uncommon word conversion desensitization processing requirement information in the first desensitization policy, and perform uncommon word conversion processing on the first data to be desensitized when the desensitization condition is met; and/or when the sensitive field type is the user address, determining whether the second data to be desensitized corresponding to the user address meets the desensitization condition according to the information of the desensitization processing requirement of the omission part in the information of the desensitization processing requirement in the first desensitization strategy, and performing omission processing on the second data to be desensitized when the desensitization condition is met; and/or; when the sensitive field type is the user contact information, determining whether third data to be desensitized corresponding to the user contact information meets the desensitization condition according to the digital conversion desensitization processing requirement information in the first desensitization strategy, and performing digital conversion processing on the third data to be desensitized when the desensitization condition is met.
Optionally, the processing unit 402 is further configured to determine, when the sensitive field type is a user name, whether the first data to be desensitized corresponding to the user name meets a desensitization condition according to the uncommon word conversion desensitization processing requirement information in the second desensitization policy, and perform uncommon word conversion processing on the first data to be desensitized when the desensitization condition is met; and/or when the sensitive field type is the user address, determining whether the second data to be desensitized corresponding to the user address meets the desensitization condition according to the information of the desensitization processing requirement of the omission part in the information of the desensitization processing requirement in the second desensitization strategy, and performing omission processing on the second data to be desensitized when the desensitization condition is met.
Optionally, the data processing apparatus further comprises: and a transmitting/receiving unit.
The receiving and transmitting unit is used for receiving the data query request sent by the analysis platform, analyzing the request and acquiring the identification of the analysis platform and the identification of the service data table to be queried. The obtaining unit 401 is further configured to obtain a user level corresponding to the identifier of the analysis platform, and obtain, from a storage area corresponding to the user level, a desensitized service data table corresponding to the identifier of the service data table to be queried. The receiving and transmitting unit is also used for transmitting the desensitized service data table to the analysis platform so that the analysis platform can display the desensitized service data table.
Fig. 5 is a block diagram of a server for implementing the data processing method of the present application, and as shown in fig. 5, the server 500 includes: a memory 501, a processor 502 and a transceiver 503.
Processor 502, memory 501 and transceiver 503 are electrically interconnected;
a transceiver 503 for transmitting and receiving data;
memory 501 stores computer-executable instructions;
processor 502 executes computer-executable instructions stored in memory 501, causing processor 502 to perform the methods provided by any of the embodiments described above.
In an exemplary embodiment, there is also provided a computer-readable storage medium having stored therein computer-executable instructions for performing the method of any one of the above embodiments by a processor.
In an exemplary embodiment, a computer program product is also provided, comprising a computer program for executing the method of any of the above embodiments by a processor.
Other embodiments of the application will be apparent to those skilled in the art from consideration of the specification and practice of the application disclosed herein. This application is intended to cover any variations, uses, or adaptations of the application following, in general, the principles of the application and including such departures from the present disclosure as come within known or customary practice within the art to which the application pertains. It is intended that the specification and examples be considered as exemplary only, with a true scope and spirit of the application being indicated by the following claims.
It is to be understood that the application is not limited to the precise arrangements and instrumentalities shown in the drawings, which have been described above, and that various modifications and changes may be effected without departing from the scope thereof. The scope of the application is limited only by the appended claims.

Claims (11)

1. A method of data processing, the method comprising:
acquiring an original service data table from a data source system; the original service data table comprises a plurality of data;
according to the type of sensitive fields and the desensitization processing requirement information in the desensitization strategy corresponding to each preset user level, carrying out desensitization processing on the data to be desensitized in the original service data table, obtaining a service data table after desensitization, and storing the service data table after desensitization into a storage area corresponding to the user level in a data lake.
2. The method according to claim 1, wherein the desensitizing the data to be desensitized in the original service data table according to the type of the sensitive field and the desensitizing processing requirement information in the desensitizing policy corresponding to each preconfigured user level includes:
for each user level, acquiring data to be desensitized corresponding to a sensitive field type in a desensitization strategy corresponding to the user level from the original service data table;
and determining whether the data to be desensitized meets the desensitization condition according to the desensitization processing demand information in the desensitization strategy corresponding to the user level, and carrying out desensitization processing on the data to be desensitized when the desensitization condition is met.
3. The method according to claim 2, characterized in that:
if the preconfigured user level is a first-level user and the corresponding desensitization policy is a first desensitization policy, the sensitive field type includes one or a combination of several of the following: user name, user address, and user contact.
4. A method according to claim 3, wherein if the user level is a first-level user and the corresponding desensitization policy is a first desensitization policy, determining whether the data to be desensitized meets a desensitization condition according to the desensitization processing requirement information in the desensitization policy corresponding to the user level, and performing desensitization processing on the data to be desensitized when the desensitization condition is met, where the step of determining includes:
when the sensitive field type is a user name, determining whether first data to be desensitized corresponding to the user name meets a desensitization condition according to the rarely used word conversion desensitization processing requirement information in the first desensitization strategy, and performing rarely used word conversion processing on the first data to be desensitized when the desensitization condition is met;
and/or the number of the groups of groups,
when the sensitive field type is a user address, determining whether second data to be desensitized corresponding to the user address meets a desensitization condition according to the information of the desensitization processing requirement of the omitted part of the content in the desensitization processing requirement information in the first desensitization strategy, and carrying out omitted processing on the second data to be desensitized when the desensitization condition is met;
and/or;
when the sensitive field type is a user contact way, determining whether third data to be desensitized corresponding to the user contact way meets a desensitization condition according to digital conversion desensitization processing requirement information in the first desensitization strategy, and performing digital conversion processing on the third data to be desensitized when the desensitization condition is met.
5. The method according to claim 2, characterized in that:
if the preconfigured user level is the second level user and the corresponding desensitization policy is the second desensitization policy, the sensitive field type includes one or several of the following combinations: a user name and a user address.
6. The method according to claim 5, wherein if the preconfigured user level is a second level user and the corresponding desensitization policy is a second desensitization policy, determining whether the data to be desensitized meets a desensitization condition according to the desensitization processing requirement information in the desensitization policy corresponding to the user level, and performing desensitization processing on the data to be desensitized when the desensitization condition is met, including:
when the sensitive field type is a user name, determining whether first data to be desensitized corresponding to the user name meets a desensitization condition according to the rarely used word conversion desensitization processing requirement information in the second desensitization strategy, and performing rarely used word conversion processing on the first data to be desensitized when the desensitization condition is met;
and/or the number of the groups of groups,
when the sensitive field type is a user address, determining whether second data to be desensitized corresponding to the user address meets a desensitization condition according to the information of the desensitization processing requirement of the omitted part of the information of the desensitization processing requirement in the second desensitization strategy, and carrying out omitted processing on the second data to be desensitized when the desensitization condition is met.
7. The method as recited in claim 1, further comprising:
receiving a data query request sent by an analysis platform, analyzing the request, and acquiring an identification of the analysis platform and an identification of a service data table to be queried;
acquiring a user level corresponding to the identification of an analysis platform, and acquiring a desensitized service data table corresponding to the identification of the service data table to be queried from a storage area corresponding to the user level;
and sending the desensitized service data table to the analysis platform so that the analysis platform can display the desensitized service data table.
8. A data processing apparatus, the apparatus comprising:
the acquisition unit is used for acquiring an original service data table from the data source system; the original service data table comprises a plurality of data;
the processing unit is used for carrying out desensitization processing on the data to be desensitized in the original service data table according to the type of the sensitive field and the desensitization processing requirement information in the desensitization strategy corresponding to each preset user level, obtaining a service data table after desensitization, and storing the service data table after desensitization into a storage area corresponding to the user level in a data lake.
9. A server, comprising: a processor, and a memory communicatively coupled to the processor;
the memory stores computer-executable instructions;
the processor executes computer-executable instructions stored in the memory to implement the method of any one of claims 1 to 7.
10. A computer readable storage medium having stored therein computer executable instructions which when executed by a processor are adapted to carry out the method of any one of claims 1 to 7.
11. A computer program product comprising a computer program which, when executed by a processor, implements the method of any of claims 1 to 7.
CN202310673144.2A 2023-06-07 2023-06-07 Data processing method, device, server and storage medium Pending CN116702199A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202310673144.2A CN116702199A (en) 2023-06-07 2023-06-07 Data processing method, device, server and storage medium

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202310673144.2A CN116702199A (en) 2023-06-07 2023-06-07 Data processing method, device, server and storage medium

Publications (1)

Publication Number Publication Date
CN116702199A true CN116702199A (en) 2023-09-05

Family

ID=87835181

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202310673144.2A Pending CN116702199A (en) 2023-06-07 2023-06-07 Data processing method, device, server and storage medium

Country Status (1)

Country Link
CN (1) CN116702199A (en)

Similar Documents

Publication Publication Date Title
CN106303099B (en) A kind of method, apparatus and system for verifying user's communicating number validity
US10834254B2 (en) System and method for utilizing customer data in a communication system
US11386224B2 (en) Method and system for managing personal digital identifiers of a user in a plurality of data elements
CN107392051A (en) A kind of big data processing method and system
CN109345417B (en) Online assessment method and terminal equipment for business personnel based on identity authentication
US20080104021A1 (en) Systems and methods for controlling access to online personal information
CN107798541B (en) Monitoring method and system for online service
CN110557366B (en) Identity authentication system and method based on cross-network transmission and CTID (computer telephony integration) network card authentication
CN111461739A (en) Bank mobile comprehensive service system and device suitable for second-level legal person
CN113254969B (en) Business data processing method and device, electronic equipment and storage medium
CN109636582A (en) Credit information management method, device, equipment and storage medium
CN115952547B (en) Database desensitizing device and method based on protocol analysis
CN111427972A (en) Method and device for searching service data, service searching system and storage medium
US20130046560A1 (en) System and method for deterministic and probabilistic match with delayed confirmation
CN116702199A (en) Data processing method, device, server and storage medium
CN113271591B (en) Two-standard four-real data encryption interaction method and device based on 5G slice network
CN109636578A (en) Risk checking method, device, equipment and the readable storage medium storing program for executing of credit information
KR20010090362A (en) Integrated cyber loan system and method for inquiring data related to that
CN110263044B (en) Data storage method, device, equipment and computer readable storage medium
KR20110035556A (en) Service system and service method for offering financial information using message oriented service
JP4718131B2 (en) Personal information management system
US20050261997A1 (en) Determination of risk factors for use in a card replacement process
CN110418020B (en) List state information processing method and device, electronic terminal and storage medium
CN114003937A (en) Data desensitization method based on characteristic rule desensitization segment
US20200013512A1 (en) System and method for programmatically determining the mortality of a natural person

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination