CN113221177A - Data access method, device and system in distributed system - Google Patents

Data access method, device and system in distributed system Download PDF

Info

Publication number
CN113221177A
CN113221177A CN202110593606.0A CN202110593606A CN113221177A CN 113221177 A CN113221177 A CN 113221177A CN 202110593606 A CN202110593606 A CN 202110593606A CN 113221177 A CN113221177 A CN 113221177A
Authority
CN
China
Prior art keywords
data
target
user
sensitive
distributed system
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202110593606.0A
Other languages
Chinese (zh)
Inventor
刘聪
梁杰
高炘
黄兆康
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Industrial and Commercial Bank of China Ltd ICBC
Original Assignee
Industrial and Commercial Bank of China Ltd ICBC
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Industrial and Commercial Bank of China Ltd ICBC filed Critical Industrial and Commercial Bank of China Ltd ICBC
Priority to CN202110593606.0A priority Critical patent/CN113221177A/en
Publication of CN113221177A publication Critical patent/CN113221177A/en
Pending legal-status Critical Current

Links

Images

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F21/00Security arrangements for protecting computers, components thereof, programs or data against unauthorised activity
    • G06F21/60Protecting data
    • G06F21/62Protecting access to data via a platform, e.g. using keys or access control rules
    • G06F21/6218Protecting access to data via a platform, e.g. using keys or access control rules to a system of files or objects, e.g. local or distributed file system or database
    • G06F21/6245Protecting personal data, e.g. for financial or medical purposes
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/24Querying
    • G06F16/245Query processing
    • G06F16/2455Query execution
    • G06F16/24553Query execution of query operations
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/27Replication, distribution or synchronisation of data between databases or within a distributed database system; Distributed database system architectures therefor
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/28Databases characterised by their database models, e.g. relational or object models
    • G06F16/283Multi-dimensional databases or data warehouses, e.g. MOLAP or ROLAP
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F21/00Security arrangements for protecting computers, components thereof, programs or data against unauthorised activity
    • G06F21/60Protecting data
    • G06F21/62Protecting access to data via a platform, e.g. using keys or access control rules
    • G06F21/6218Protecting access to data via a platform, e.g. using keys or access control rules to a system of files or objects, e.g. local or distributed file system or database
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06QINFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
    • G06Q40/00Finance; Insurance; Tax strategies; Processing of corporate or income taxes
    • G06Q40/02Banking, e.g. interest calculation or account maintenance

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Databases & Information Systems (AREA)
  • General Physics & Mathematics (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • Bioethics (AREA)
  • Health & Medical Sciences (AREA)
  • Business, Economics & Management (AREA)
  • Data Mining & Analysis (AREA)
  • General Health & Medical Sciences (AREA)
  • Computer Security & Cryptography (AREA)
  • Finance (AREA)
  • Software Systems (AREA)
  • Accounting & Taxation (AREA)
  • Computer Hardware Design (AREA)
  • Technology Law (AREA)
  • Development Economics (AREA)
  • Computational Linguistics (AREA)
  • General Business, Economics & Management (AREA)
  • Medical Informatics (AREA)
  • Economics (AREA)
  • Strategic Management (AREA)
  • Marketing (AREA)
  • Computing Systems (AREA)
  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
  • Storage Device Security (AREA)

Abstract

The embodiment of the application provides a data access method, a device and a system in a distributed system, which can be used in the technical field of big data, wherein the method comprises the following steps: determining sensitive data in a target data table in a target database in a distributed system currently specified by a target user without sensitive data access authority; performing deformation processing on the sensitive data in the target data table according to a deformation rule nesting function corresponding to the sensitive data; and outputting the sensitive data after the deformation processing and the non-sensitive data in the target data table to the target user. According to the data access method and device, on the basis of guaranteeing the safety and privacy of data access in the distributed system, the efficiency and the degree of automation of data access in the distributed system are effectively improved, and the reliability of data access in the distributed system can be effectively improved.

Description

Data access method, device and system in distributed system
Technical Field
The present application relates to the field of data processing technologies, and in particular, to the field of big data technologies, and in particular, to a method, an apparatus, and a system for accessing data in a distributed system.
Background
With the rapid development of distributed system technology, enterprises begin to adopt high-performance distributed database systems to process mass data. In the data, sensitive data, secret-related data and the like are inevitably involved, and because the identities of the access users are different, the access permissions of the access users on the sensitive data and the secret-related data in the distributed system in the enterprise are different, so that a sensitive data access mode cannot be adopted, and the data obtained under different processing modes needs to be provided for different user permissions.
At present, in a data access mode in an existing distributed system, at least one level of manager is usually required to be involved to screen an access request of a common user, the manager needs to judge whether sensitive data can be provided for the access user according to identity information of the access user, permission setting of the access user in an enterprise, and the like, and the manager also needs to filter or encrypt sensitive data which is applied for access by a user without access permission.
However, the data access method in the existing distributed system is too dependent on manual operation of a manager, so that the data access method in the existing distributed system has the problems of low data access efficiency, low reliability, low automation degree and the like.
Disclosure of Invention
Aiming at the problems in the prior art, the application provides a data access method, device and system in a distributed system, which can effectively improve the efficiency and automation degree of data access in the distributed system and can effectively improve the reliability of data access in the distributed system on the basis of ensuring the safety and privacy of data access in the distributed system.
In order to solve the technical problem, the application provides the following technical scheme:
in a first aspect, the present application provides a data access method in a distributed system, including:
determining sensitive data in a target data table in a target database in a distributed system currently specified by a target user without sensitive data access authority;
performing deformation processing on the sensitive data in the target data table according to a deformation rule nesting function corresponding to the sensitive data;
and outputting the sensitive data after the deformation processing and the non-sensitive data in the target data table to the target user.
Further, before the determining that the target user without the sensitive data access right currently designates the sensitive data in the target data table in the target database in the distributed system, the method further includes:
receiving a data access request of a target user for a distributed system, wherein the data access request comprises a user identifier of the target user, an identifier of a target database in the distributed system and an identifier of a target data table in the target database;
judging whether the distributed system comprises a target data table in the target database or not according to the identification of the target database and the identification of the target data table, if so, acquiring the access condition of the target user;
and judging whether the target user has the access qualification of the target data table according to the access condition of the target user, and if so, acquiring the access authority and sensitive fields of the target data table which the target user requests to access.
Further, the obtaining the access condition of the target user includes:
determining a region field corresponding to the user identification of the target user from a preset user and condition relation table;
and determining the area name of the area field corresponding to the target user in a preset area parameter table.
Further, the determining whether the target user qualifies for accessing the target data table according to the access condition of the target user includes:
determining a region field corresponding to the identifier of the target database from a preset condition and permission relation table;
determining the area name of the area field corresponding to the target database in the area parameter table;
and judging whether the target user accords with the in-region data access qualification aiming at the target database or not according to the region name corresponding to the target user and the region name corresponding to the target database.
Further, the obtaining the access condition of the target user includes:
determining the organization number to which the user corresponding to the user identifier of the target user belongs from a preset user and condition relation table;
and determining the name of the mechanism to which the user belongs according to the mechanism number to which the user belongs and corresponding to the target user in a preset mechanism parameter table to which the user belongs.
Further, the determining whether the target user qualifies for accessing the target data table according to the access condition of the target user includes:
determining the organization number to which the user corresponding to the identifier of the target database belongs from a preset condition and authority relation table;
determining the name of the mechanism to which the user belongs of the mechanism number of the user corresponding to the target database in the mechanism parameter table to which the user belongs;
and judging whether the target user meets the mechanism access qualification of the user belonging to the target database or not according to the mechanism name of the user belonging to the target user and the mechanism name of the user belonging to the target database.
Further, the acquiring the access right and the sensitive field of the target data table to which the target user requests to access includes:
and searching the access authority and the sensitive field of the target data table in the target database which the target user requests to access from a preset user and sensitive information relation table according to the user identification of the target user, the identification of the target database in the distributed system and the identification of the target data table in the target database.
Further, the determining sensitive data in a target data table in a target database in the distributed system currently specified by a target user who does not have access right to the sensitive data includes:
judging whether the data requested to be accessed by the target user contains sensitive data or not according to the content of the sensitive field of the target data table requested to be accessed by the target user, if so, judging whether the target user has the sensitive data access right for accessing the sensitive data in the target data table or not based on the access right of the target data table requested to be accessed by the target user;
and if the target user does not have the sensitive data access authority, calling the sensitive data in the target data table.
Further, the deforming the sensitive data in the target data table according to the deformation rule nesting function corresponding to the sensitive data includes:
acquiring a deformation indication identifier and a deformation rule ID corresponding to a sensitive field of the target data table from a preset sensitive information list;
judging whether a sensitive field of the target data table needs to be deformed or not according to the deformation indication identifier, if so, calling a deformation rule nesting function corresponding to the deformation rule ID from a preset deformation rule;
and performing deformation processing on the sensitive data in the target data table based on the deformation rule nesting function.
Further, the outputting the sensitive data after the deformation processing and the non-sensitive data in the target data table to the target user includes:
and generating a data query view containing the deformed sensitive data according to the deformed sensitive data and the non-sensitive data in the target data table, and displaying the data query view to enable the target user to query.
Further, still include:
and if the data requested to be accessed by the target user does not contain sensitive data, or if the target user has the sensitive data access authority, generating a data query view according to the data in the target data table, and displaying the data query view to enable the target user to query.
In a second aspect, the present application provides a data access apparatus in a distributed system, including:
the data searching module is used for determining the sensitive data in the target data table in the target database in the distributed system currently specified by the target user without the sensitive data access authority;
the data deformation module is used for performing deformation processing on the sensitive data in the target data table according to a deformation rule nesting function corresponding to the sensitive data;
and the data output module is used for outputting the sensitive data after the deformation processing and the non-sensitive data in the target data table to the target user.
In a third aspect, the present application provides a data access system, comprising: the system comprises a control server, a database server and a data warehouse;
the database server is used for executing a data access method in the distributed system;
the control server is used for sending a data access request of the target user for the distributed system to the database server so that the database server can determine whether the target user has sensitive data access permission according to the data access request;
the data warehouse is used for storing each data table in each database in the distributed system, so that the database server determines the sensitive data in the target data table in the target database in the distributed system currently specified by the target user without the sensitive data access authority from the data warehouse.
In a fourth aspect, the present application provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the data access method in the distributed system.
In a fifth aspect, the present application provides a computer readable storage medium having stored thereon a computer program which, when executed by a processor, implements the method for data access in a distributed system as described.
According to the technical scheme, the data access method, the data access device and the data access system in the distributed system provided by the application comprise the following steps: determining sensitive data in a target data table in a target database in a distributed system currently specified by a target user without sensitive data access authority; performing deformation processing on the sensitive data in the target data table according to a deformation rule nesting function corresponding to the sensitive data; the method comprises the steps of outputting sensitive data after deformation processing and non-sensitive data in a target data table to a target user, and setting the deformation processing of the sensitive data in the target data table according to a deformation rule nesting function corresponding to a sensitive field of the sensitive data if the sensitive data is contained in the target data table, so that the sensitive data can be safely accessed automatically without the participation of higher-level management personnel, the efficiency and the degree of automation of data access in a distributed system can be effectively improved on the basis of ensuring the safety and the privacy of the data access in the distributed system, the reliability of the data access in the distributed system can be effectively improved, and the operation stability of the distributed system can be effectively ensured.
Drawings
In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings needed to be used in the description of the embodiments or the prior art will be briefly introduced below, and it is obvious that the drawings in the following description are some embodiments of the present application, and it is obvious for those skilled in the art to obtain other drawings based on these drawings without creative efforts.
Fig. 1 is a schematic structural diagram of a data access system in an embodiment of the present application.
Fig. 2 is a first flowchart illustrating a data access method in a distributed system in an embodiment of the present application.
Fig. 3 is a second flowchart illustrating a data access method in a distributed system in an embodiment of the present application.
Fig. 4 is a first flowchart 030 of a data access method in a distributed system according to an embodiment of the present application.
Fig. 5 is a third flowchart illustrating a data access method in a distributed system in an embodiment of the present application.
Fig. 6 is a second flowchart of a step 030 in a data access method in a distributed system in the embodiment of the present application.
Fig. 7 is a fourth flowchart illustrating a data access method in a distributed system in an embodiment of the present application.
Fig. 8 is a fifth flowchart illustrating a data access method in a distributed system in an embodiment of the present application.
Fig. 9 is a sixth flowchart illustrating a data access method in a distributed system in an embodiment of the present application.
Fig. 10 is a seventh flowchart illustrating a data access method in a distributed system in an embodiment of the present application.
Fig. 11 is an eighth flowchart illustrating a data access method in a distributed system in an embodiment of the present application.
Fig. 12 is a schematic structural diagram of a data access device in a distributed system in an embodiment of the present application.
Fig. 13 is a schematic diagram of an execution logic flow of the data access system provided in the application example of the present application.
Fig. 14 is a functional diagram of a refined query view provided by a data access device in a distributed system according to an application example of the present application.
Fig. 15 is a schematic structural diagram of an electronic device in an embodiment of the present application.
Detailed Description
In order to make the objects, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application, and it is obvious that the described embodiments are some embodiments of the present application, but not all embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present application.
It should be noted that the method, the apparatus, and the system for data access in the distributed system disclosed in the present application may be used in the field of big data technology, and may also be used in any field other than the field of big data technology.
Taking the Hadoop platform as an example of a distributed system, the Hive data warehouse of the Hadoop platform is a high-reliability and high-performance distributed database system, and mass data can be stored by using Hive. By utilizing the characteristic, enterprises use the Hadoop platform to store and process large data in disputes. When a user accesses mass data of a data warehouse, extremely high requirements are placed on data authority and sensitivity. The existing Hadoop platform realizes authority management based on a key authentication (Kerberos) mode, but as the capacity of the Hadoop platform is continuously enlarged, the number of service components is continuously increased, the data volume is continuously increased, and the number of users is continuously increased, fine data authority control of a large data platform becomes necessary. At present, for the existing mass data authority control method, especially in the field of banks and large-scale finance, there is a partial gap in the management of access authority for data sensitivity and user-table level granularity, that is, in the existing data access mode in the distributed system, because higher and basic managers are needed to judge and process the access request of sensitive data, the data access mode in the existing distributed system has the problems of low data access efficiency, low reliability, low automation degree and the like.
Based on the content, the data access device manages the data authority based on the big data platform in a user-table level granularity, integrates the authority control of the user on sensitive information, forms a set of big data authority management control device, and supports the data access device in the distributed system to automatically manage and control the access authority of the common authority user by providing a visual interface. Operability is provided for the fine authority control of mass data of the distributed system. The method is particularly suitable for the large data authority management of the bank industry of the Hadoop platform Hive data warehouse.
Aiming at the problems of low data access efficiency, low reliability, low automation degree and the like of a data access mode in the existing distributed system, the embodiment of the application respectively provides a data access method in the distributed system, a data access device in the distributed system, a data access system and a computer readable storage medium of electronic equipment, and determines the sensitive data in a target data table in a target database in the distributed system currently specified by a target user without sensitive data access authority; performing deformation processing on the sensitive data in the target data table according to a deformation rule nesting function corresponding to the sensitive data; the method comprises the steps of outputting sensitive data after deformation processing and non-sensitive data in a target data table to a target user, and setting the deformation processing of the sensitive data in the target data table according to a deformation rule nesting function corresponding to a sensitive field of the sensitive data if the sensitive data is contained in the target data table, so that the sensitive data can be safely accessed automatically without the participation of higher-level management personnel, the efficiency and the degree of automation of data access in a distributed system can be effectively improved on the basis of ensuring the safety and the privacy of the data access in the distributed system, the reliability of the data access in the distributed system can be effectively improved, and the operation stability of the distributed system can be effectively ensured.
In one or more embodiments of the present application, the distributed system may refer to a big data service cloud, an HBase cluster, or the like, where the HBase cluster (Hadoop Database) refers to a high-reliability, high-performance, column-oriented, scalable distributed storage system, and is composed of a Master node Master and a slave node Region Server, where the Master may also be written as an HMaster specifically, and the Region Server may also be written as an hregoion Server or a Region Server, or the like.
Based on the above, the present application further provides a data access system for implementing the data access method in the distributed system provided in one or more embodiments of the present application, referring to fig. 1, the data access system includes a control server, a database server and a data warehouse, where the database server may specifically be the distributed system data access apparatus mentioned in one or more embodiments of the present application, the data access apparatus in the distributed system may communicate with a client device owned by each user, the data warehouse in the big data service cloud and each control server accessing the big data service cloud by itself or through a third-party server, and the data access apparatus in the distributed system may be a server that receives a data access request in the distributed system for the big data service cloud from the client device or each control server, various relevant rule configuration files preset by the user can be obtained from the client device, the third-party database or locally, for example, at least one of the user information table, the regional parameter table, the professional institution parameter table (the institution parameter table to which the user belongs), the sensitive information list, the deformation rule, the user-sensitive information relation table (the user and sensitive information relation table), the condition-permission relation table (the condition and permission relation table) and the user-condition relation table (the user and condition relation table) mentioned in one or more embodiments of the present application. The control server is used for sending a data access request of the target user for the distributed system to the database server so that the database server can determine whether the target user has sensitive data access permission according to the data access request; the data warehouse is used for storing each data table in each database in the distributed system, so that the database server determines the sensitive data in the target data table in the target database in the distributed system currently specified by the target user without the sensitive data access authority from the data warehouse.
It is understood that the client devices may include smart phones, tablet electronic devices, network set-top boxes, portable computers, desktop computers, Personal Digital Assistants (PDAs), in-vehicle devices, smart wearable devices, and the like. Wherein, intelligence wearing equipment can include intelligent glasses, intelligent wrist-watch, intelligent bracelet etc..
The client device may have a communication module (i.e., a communication unit), and may be communicatively connected to a remote server to implement data transmission with the server. The server may include a server on the task scheduling center side, and in other implementation scenarios, the server may also include a server on an intermediate platform, for example, a server on a third-party server platform that is communicatively linked to the task scheduling center server. The server may include a single computer device, or may include a server cluster formed by a plurality of servers, or a server structure of a distributed apparatus.
The server and the client device may communicate using any suitable network protocol, including network protocols not yet developed at the filing date of this application. The network protocol may include, for example, a TCP/IP protocol, a UDP/IP protocol, an HTTP protocol, an HTTPS protocol, or the like. Of course, the network Protocol may also include, for example, an RPC Protocol (Remote Procedure Call Protocol), a REST Protocol (Representational State Transfer Protocol), and the like used above the above Protocol.
The following embodiments and application examples are specifically and individually described in detail.
In order to solve the problems of low data access efficiency, low reliability, low automation degree and the like in the data access mode in the existing distributed system, the application provides an embodiment of a data access method in the distributed system, and referring to fig. 2, the data access method in the distributed system executed by the data access device in the distributed system specifically includes the following contents:
step 100: and determining the sensitive data in the target data table in the target database in the distributed system currently designated by the target user without the sensitive data access authority.
In step 100, a user may first make a query request, and the device checks whether a query permission exists, and if so, extracts a corresponding query view name, performs a query on a big data platform, and returns a query result; if not, returning no inquiry authority to end the inquiry. And then, the data access device in the distributed system judges whether the query request conforms to the data application qualification in the administrative region, if not, the query request is rejected, and if so, whether sensitive information needs to be queried is judged.
It is understood that the access rights of the user to the data include: and inquiring a non-sensitive information view and inquiring a sensitive information view. If the non-sensitive information view is inquired, the access view filters the sensitive information of the inquired target data, and one or more filtering conditions can be provided; and if the sensitive information view is queried, the access view does not filter the sensitive information of the query target data. And the access rights of the users to the views are in one-to-one correspondence.
In one or more embodiments of the present application, the target user may be a super administrator user, an administrator user, a general user, and the like, and the permissions corresponding to the user identities of each type are different, the super administrator user may perform permission control on all data, and the first-level administrator user divides different data management permissions according to the organization, and has corresponding query and approval permissions and view creation permissions, for example, the data sources of the bank data risk class, the credit class, the financial accounting class, the asset management, the human resources, and the like are different. And the secondary administrator user has different inquiry and approval authorities in different organizations within the scope of each administrative area. The common user belongs to one or more of the different mechanisms in the scope and belongs to an administrative region, and the data authority is required to be applied by the related mechanism administrator when the data is authorized to be used.
Step 200: and performing deformation processing on the sensitive data in the target data table according to a deformation rule nesting function corresponding to the sensitive data.
In step 200, the contents of different fields of different tables can be labeled in advance to form a sensitive information list. Uploading the sensitive information deformation function to a big data platform, and creating a view of a source table in the big data platform to form a table view of the nested sensitive information deformation function. The view of the same source table has multiple views due to the size of the authority range.
Step 300: and outputting the sensitive data after the deformation processing and the non-sensitive data in the target data table to the target user.
As can be seen from the above description, in the data access method in the distributed system provided in the embodiment of the present application, if the target data table contains sensitive data, the sensitive data in the target data table is set to be subjected to deformation processing according to the deformation rule nesting function corresponding to the sensitive field of the sensitive data, and the sensitive data can be safely accessed without the participation of a higher-level manager, so that on the basis of ensuring the safety and privacy of data access in the distributed system, the efficiency and the degree of automation of data access in the distributed system can be effectively improved, the reliability of data access in the distributed system can be effectively improved, and the operation stability of the distributed system can be effectively ensured.
In order to improve the effectiveness and reliability of data access in the distributed system, referring to fig. 3, an embodiment of a data access method in the distributed system provided in the present application further includes the following steps before step 100 of the data access method in the distributed system:
step 010: receiving a data access request of a target user for a distributed system, wherein the data access request comprises a user identifier of the target user, an identifier of a target database in the distributed system and an identifier of a target data table in the target database.
Step 020: and judging whether the distributed system comprises the target data table in the target database or not according to the identifier of the target database and the identifier of the target data table, and if so, executing the step 030.
Step 030: and acquiring the access condition of the target user.
In step 020, whether the distributed system includes the target data table in the target database is judged according to the identifier of the target database and the identifier of the target data table, if the distributed system does not include the target data table in the target database, a notification message for indicating that the distributed system does not include the target data table in the target database is sent to the target user, and the current process is ended.
Step 040: and judging whether the target user has the access qualification of the target data table according to the access condition of the target user, if so, executing the step 050.
Step 050: and acquiring the access authority and sensitive fields of the target data table which the target user requests to access.
As can be seen from the above description, the data access method in the distributed system provided in the embodiment of the present application can effectively improve the validity and reliability of data access in the distributed system by automatically determining whether the distributed system includes the target data table in the target database and automatically determining whether the target user has the access qualification of the target data table.
In order to provide an effective data basis for determining whether the target user meets the in-region data access qualification for the target database, an embodiment of a data access method in a distributed system provided in the present application is shown in fig. 4, where the step 030 in the data access method in the distributed system specifically includes the following contents:
step 031: and determining the area field corresponding to the user identifier of the target user from a preset user and condition relation table.
Step 032: and determining the area name of the area field corresponding to the target user in a preset area parameter table.
Specifically, a pre-stored user-condition relationship table is obtained, and the area field corresponding to the target user is searched in the user-condition relationship table according to the user identifier of the target user, wherein the user-condition relationship table stores the association relationship among the user, the area and the mechanism to which the user belongs. And storing the user ID, the area code where the user is located and the serial number of the user mechanism. Such as: users 01, 1301, 003. And acquiring a preset region parameter table, and acquiring a region name corresponding to the region field in the region parameter table according to the region field corresponding to the target user. The regional parameter table is used for storing administrative region information, including provincial level, city level and county level. And (3) storing: area number, area name. Such as: hebei province branch (area code: 1300) and Shijiazhuang branch (area code: 1301) in the bank system.
As can be seen from the above description, in the data access method in the distributed system provided in the embodiment of the present application, by defining the access condition as the area name, an effective data basis can be provided for determining whether the target user meets the in-area data access qualification for the target database, so that the effectiveness and reliability of determining whether the target user has the access qualification of the target data table according to the access condition of the target user can be effectively improved.
In order to improve the effectiveness of determining whether the target user has the access qualification of the target data table according to the access condition of the target user, in an embodiment of the data access method in the distributed system provided by the present application, based on step 031 and step 032, referring to fig. 5, step 040 in the data access method in the distributed system specifically includes the following contents:
step 041: determining a region field corresponding to the identifier of the target database from a preset condition and permission relation table;
step 042: determining the area name of the area field corresponding to the target database in the area parameter table;
step 043: and judging whether the target user accords with the in-region data access qualification aiming at the target database or not according to the region name corresponding to the target user and the region name corresponding to the target database.
Specifically, whether the target user meets the in-region data access qualification for the target database is judged according to the region name corresponding to the target user and the region name corresponding to the target database, if not, a notification message used for indicating that the target user does not meet the in-region data access qualification for the target database is sent to the target user, and the current process is ended.
As can be seen from the above description, the data access method in the distributed system provided in the embodiment of the present application can further effectively improve the validity and reliability of determining whether the target user has the access qualification of the target data table according to the access condition of the target user by determining whether the target user meets the data access qualification in the region for the target database.
In order to provide an effective data basis for determining whether the target user meets the in-region data access qualification for the target database, referring to fig. 6, in an embodiment of the data access method in the distributed system provided by the present application, step 030 in the data access method in the distributed system may further specifically include the following:
step 033: and determining the organization number to which the user corresponding to the user identifier of the target user belongs from a preset user and condition relation table.
Step 034: and determining the name of the mechanism to which the user belongs according to the mechanism number to which the user belongs and corresponding to the target user in a preset mechanism parameter table to which the user belongs.
Specifically, a pre-stored user-condition relation table is obtained, and a user-belonging organization number corresponding to the target user is searched in the user-condition relation table according to the user identifier of the target user; and acquiring a preset mechanism parameter table to which the user belongs, and acquiring the name of the mechanism to which the user belongs corresponding to the mechanism number to which the user belongs in the mechanism parameter table to which the user belongs according to the mechanism number to which the user corresponding to the target user belongs.
The mechanism parameter table to which the user belongs is used for storing mechanism information to which different users belong. Including department code, department name. Such as: credit department (001), wind control department (002), accounting department (003) in the bank system, etc.
As can be seen from the above description, in the data access method in the distributed system provided in the embodiment of the present application, by defining the access condition as the name of the organization to which the user belongs, an effective data basis can be provided for determining whether the target user meets the access qualification of the organization to which the user belongs with respect to the target database, so that the comprehensiveness, effectiveness, and reliability of determining whether the target user has the access qualification of the target data table according to the access condition of the target user can be further effectively improved.
In order to improve the effectiveness of determining whether the target user has the access qualification of the target data table according to the access condition of the target user, in an embodiment of the data access method in the distributed system provided by the present application, based on step 033 and step 034, referring to fig. 7, step 040 in the data access method in the distributed system further specifically includes the following contents:
step 044: and determining the organization number to which the user belongs corresponding to the identifier of the target database from a preset condition and authority relation table.
Step 045: and determining the name of the mechanism to which the user belongs of the mechanism number of the user corresponding to the target database in the mechanism parameter table to which the user belongs.
Step 046: and judging whether the target user meets the mechanism access qualification of the user belonging to the target database or not according to the mechanism name of the user belonging to the target user and the mechanism name of the user belonging to the target database.
Specifically, according to the mechanism name to which the user corresponding to the target user belongs and the mechanism name to which the user corresponding to the target database belongs, whether the target user meets the mechanism access qualification of the user corresponding to the target database is judged, if not, a notification message used for indicating that the target user does not meet the mechanism access qualification of the user corresponding to the target database is sent to the target user, and the current process is ended.
As can be seen from the above description, the data access method in the distributed system provided in the embodiment of the present application can further effectively improve the comprehensiveness, effectiveness, and reliability of determining, according to the access condition of the target user, whether the target user has the access qualification of the target data table by determining whether the target user meets the access qualification of the organization to which the user belongs for the target database.
In order to improve the efficiency and the degree of automation for obtaining the access right and the sensitive field of the target data table of the target user in the target database, in an embodiment of the data access method in the distributed system provided by the present application, referring to fig. 8, step 050 in the data access method in the distributed system further includes the following contents:
step 051: and searching the access authority and the sensitive field of the target data table in the target database which the target user requests to access from a preset user and sensitive information relation table according to the user identification of the target user, the identification of the target database in the distributed system and the identification of the target data table in the target database.
Specifically, a pre-stored user-sensitive information relation table is obtained, and according to the user identifier of the target user, the identifier of the target database in the distributed system and the identifier of the target data table in the target database, the access authority and the sensitive field of the target data table in the target database which the target user requests to access are searched from the user-sensitive information relation table.
And the user and sensitive information relation table is used for storing the association relation between the sensitive information and the user. And storing user ID, library name, table name, authority and sensitive field. For example, user 01, library a, table a, authority 1 (1-entitled, 0 not entitled), sensitive field (name, telephone, address, field name requiring nested sensitive field separated by comma).
As can be seen from the above description, according to the data access method in the distributed system provided in the embodiment of the present application, by using the user-sensitive information relation table, the efficiency and the degree of automation for obtaining the access permission and the sensitive field of the target data table of the target user in the target database can be effectively improved, and further the efficiency and the degree of automation for data access in the distributed system can be further improved.
In order to improve the efficiency and the degree of automation for identifying the access right of the target user, referring to fig. 9, in an embodiment of the data access method in the distributed system provided by the present application, step 100 in the data access method in the distributed system specifically includes the following contents:
step 110: and judging whether the data requested to be accessed by the target user contains sensitive data or not according to the content of the sensitive field of the target data table requested to be accessed by the target user, if so, judging whether the target user has the sensitive data access right for accessing the sensitive data in the target data table or not based on the access right of the target data table requested to be accessed by the target user.
Step 120: and if the target user does not have the sensitive data access authority, calling the sensitive data in the target data table.
Specifically, whether the data to be accessed by the target user contains sensitive data is determined according to a sensitive field of a target data table in a target database which the target user requests to access (for example, if the sensitive field is null or "0", it indicates that the data to be accessed by the target user does not contain sensitive data, and if the sensitive field is "telephone", it indicates that the data to be accessed by the target user contains sensitive data, namely, a telephone number).
As can be seen from the above description, according to the data access method in the distributed system provided in the embodiment of the present application, whether the target user has the sensitive data access right is determined according to the sensitive field and the access right, so that the efficiency and the degree of automation for identifying the access right of the target user can be effectively improved, and the efficiency and the degree of automation for data access in the distributed system can be further improved.
In order to improve the efficiency and the automation degree of performing deformation processing on the sensitive data in the target data table, referring to fig. 10, an embodiment of a data access method in a distributed system provided by the present application includes the following steps in step 200:
step 210: and acquiring a deformation indication identifier and a deformation rule ID corresponding to the sensitive field of the target data table from a preset sensitive information list.
Step 220: and judging whether the sensitive field of the target data table needs to be deformed or not according to the deformation indication identifier, if so, calling a deformation rule nesting function corresponding to the deformation rule ID from a preset deformation rule.
Step 230: and performing deformation processing on the sensitive data in the target data table based on the deformation rule nesting function.
Specifically, a preset sensitive information list is obtained, and a corresponding deformation indication identifier and a corresponding deformation rule ID are obtained from the sensitive information list according to the sensitive field of the target data table.
The sensitive information list is used for storing a table ID, a library name, a table name, a field name, whether to deform (0-no, 1-yes), a deformation rule ID (a name deformation rule is 1, a mobile phone number deformation rule is 2, an address deformation rule is 3, a mailbox deformation rule is 4, and the like), and on the contrary, if a certain field does not relate to sensitive information, whether to deform the field is 0, and the deformation rule ID is empty. Such as: table ID 001, library name a, table name a, field name c1, whether morph to 1, morph rule to 1. And judging whether the sensitive field of the target data table needs to be deformed or not according to the deformation indication identifier, if so, acquiring a preset deformation rule, and acquiring a corresponding deformation rule nesting function from the deformation rule according to the deformation rule ID corresponding to the target data table.
And the deformation rule is used for storing a deformation rule nesting function corresponding to the deformation logic. And storing the deformation rule ID and the deformation function. If the name transformation rule is 1, the transformation function is name ($ x), and $ x is the identifier of the field name.
As can be seen from the above description, in the data access method in the distributed system provided in the embodiment of the present application, the deformation indication identifier and the deformation rule ID corresponding to the sensitive field of the target data table are obtained from the preset sensitive information list, and the deformation rule nesting function corresponding to the deformation rule ID is called from the preset deformation rule, so that the efficiency and the degree of automation of the deformation processing on the sensitive data in the target data table can be effectively improved, and the efficiency and the degree of automation of the data access in the distributed system can be further improved.
In order to improve the reliability and the automation degree of outputting the sensitive data in the target data table, referring to fig. 11, an embodiment of a data access method in a distributed system provided in the present application includes the following steps in step 300:
step 310: and generating a data query view containing the deformed sensitive data according to the deformed sensitive data and the non-sensitive data in the target data table, and displaying the data query view to enable the target user to query.
As can be seen from the above description, according to the data access method in the distributed system provided in the embodiment of the present application, the data query view including the sensitive data subjected to the deformation processing is generated according to the sensitive data subjected to the deformation processing and the non-sensitive data in the target data table, so that the reliability and the automation degree of outputting the sensitive data in the target data table can be effectively improved, the convenience and the intuition of querying and accessing the data by the target user can be effectively improved, the user experience of the accessing user can be effectively improved, and the convenience and the intelligence degree of data access in the distributed system can be further improved.
In order to output and process access data distinctively, an embodiment of the data access method in the distributed system provided in the present application further includes the following steps:
if it is determined in step 110 that the data requested to be accessed by the target user does not include sensitive data, or if the target user has the access right to the sensitive data, performing step 400: and generating a data query view according to the data in the target data table, and displaying the data query view to enable the target user to query.
As can be seen from the above description, according to the data access method in the distributed system provided in the embodiment of the present application, if the data requested to be accessed by the target user does not include sensitive data, or if the target user has the access permission to the sensitive data, the data query view is directly generated according to the data in the target data table, so that the access data can be output and processed in a differentiated manner, and further, on the basis of ensuring the security of data access, the efficiency and convenience of querying and accessing the data by the target user can be further improved, the user experience of the access user can be effectively improved, and further, the efficiency and convenience of data access in the distributed system can be further improved.
In terms of software, in order to solve the problems of low data access efficiency, low reliability, low automation degree, and the like in the data access method in the existing distributed system, the present application provides an embodiment of a data access apparatus in a distributed system for executing all or part of contents in the data access method in the distributed system, and referring to fig. 12, the data access apparatus in the distributed system specifically includes the following contents:
and the data searching module 10 is used for determining the sensitive data in the target data table in the target database in the distributed system currently specified by the target user without the sensitive data access right.
In the data searching module 10, a user can firstly put forward a query request, the device checks whether the query authority exists, if so, extracts the corresponding query view name, queries in the big data platform, and returns a query result; if not, returning no inquiry authority to end the inquiry. And then, the data access device in the distributed system judges whether the query request conforms to the data application qualification in the administrative region, if not, the query request is rejected, and if so, whether sensitive information needs to be queried is judged.
And the data deformation module 20 is configured to perform deformation processing on the sensitive data in the target data table according to a deformation rule nesting function corresponding to the sensitive data.
In the data transformation module 20, the contents of different fields of different tables can be labeled in advance to form a sensitive information list. Uploading the sensitive information deformation function to a big data platform, and creating a view of a source table in the big data platform to form a table view of the nested sensitive information deformation function. The view of the same source table has multiple views due to the size of the authority range.
And the data output module 30 is configured to output the sensitive data after the deformation processing and the non-sensitive data in the target data table to the target user.
The embodiment of the data access apparatus in the distributed system provided in the present application may be specifically configured to execute the processing procedure of the embodiment of the data access method in the distributed system in the foregoing embodiment, and the functions of the embodiment are not described herein again, and refer to the detailed description of the embodiment of the method.
As can be seen from the above description, in the data access device in the distributed system provided in the embodiment of the present application, if the target data table contains sensitive data, the sensitive data in the target data table is set to be deformed according to the nested function of the deformation rule corresponding to the sensitive field of the sensitive data, and the sensitive data can be safely accessed without the participation of a higher-level manager, so that on the basis of ensuring the safety and privacy of data access in the distributed system, the efficiency and the degree of automation of data access in the distributed system can be effectively improved, the reliability of data access in the distributed system can be effectively improved, and the operation stability of the distributed system can be effectively ensured.
In order to further explain the scheme, the application also provides a specific application example of the data access method in the distributed system, which is realized by the data access system with the function of finely managing the authority of mass data, the application example is managed by user-table level granularity based on the data authority of a big data platform, the authority control of a user on sensitive information is integrated to form a set of big data authority management control device, and the function of automatically managing and controlling the authority of a common authority user by the data access device in the distributed system is supported by providing a visual interface. Operability is provided for the fine authority control of mass data of the distributed system. The application example database user access authority management method and the data processing process are particularly suitable for bank industry big data authority management of a Hadoop platform Hive data warehouse.
Referring to fig. 13, the control server is configured to send a data access request of the target user for the distributed system to the database server, so that the database server determines whether the target user has a sensitive data access right according to the data access request; the Hive data warehouse or other databases are used for storing each data table in each database in the distributed system, so that the database server determines the sensitive data in the target data table in the target database in the distributed system currently designated by the target user without the sensitive data access right from the data warehouse.
The database server may specifically be the data access device in the distributed system mentioned in the foregoing embodiment, and the specific function of the refined query view provided by the data access device in the distributed system is shown in fig. 14.
The specific functions of the data access system are as follows:
one, authority control logic
The authority control logic is applied to a database server and relates to 7 modules of user information, conditions (region + professional organization), sensitive information, user-sensitive information relation, condition-authority relation, user-condition relation and query view.
1. And a user information module.
And the user information table is used for storing user information, including user ID, user name, user password, user address, user mobile phone number and other user basic information. Such as: three-person information with a user ID of 001 is stored in the user information table.
2. And a condition module. Including the area where the user is located and the professional organization where the user is located.
(1) And (5) a region module.
And the regional parameter table stores administrative region information including provincial level, city level and county level. And (3) storing: area number, area name. Such as: hebei province branch (area code: 1300) and Shijiazhuang branch (area code: 1301) in the bank system.
(2) And a professional institution module.
And the professional institution parameter table is used for storing information of different professional institutions. Including department code, department name. Such as: credit department (001), wind control department (002), accounting department (003) in the bank system, etc.
3. And the sensitive information authority module. Including the sensitive information list and the deformation rules.
(1) A list of sensitive information.
The method comprises the steps of storing a database table list and a sensitive information list which need to be controlled, namely storing a table ID, a table name, a field name, whether to deform (0-NO, 1-YES), a deformation rule ID (a name deformation rule is 1, a mobile phone number deformation rule is 2, an address deformation rule is 3, a mailbox deformation rule is 4 and the like), and on the contrary, if a certain field does not relate to sensitive information, whether to deform the field is 0 or not and the deformation rule ID is empty. Such as: table ID 001, library name a, table name a, field name c1, whether morph to 1, morph rule to 1.
(2) And (5) deformation rules.
And storing the deformation rule nesting function corresponding to the deformation logic. And storing the deformation rule ID and the deformation function. If the name transformation rule is 1, the transformation function is name ($ x), and $ x is the identifier of the field name.
4. User-sensitive information relationship table.
And the user-sensitive information relation table comprises the association relation between the sensitive information and the user. And storing user ID, library name, table name, authority and sensitive field. For example, user 01, library a, table a, authority 1 (1-entitled, 0 not entitled), sensitive field (name, telephone, address, field name requiring nested sensitive field separated by comma).
5. Condition-permission relationship table.
And the condition-authority relation table stores the data table and the association relation between the region and the professional organization. Store library name, table name, area field screening logic (standby), professional organization number. Such as: library a, table a, zone field (zoneno), zone field screening logic (substr (zoneno,2,4)), specialty agency number (003).
6. User-condition relationship table.
And the user-condition relation table stores the association relation between the user and the region and professional organization. And storing the user ID, the area code where the user is located and the serial number of the user mechanism. Such as: users 01, 1301, 003.
7. And querying the view.
And (4) inquiring a certain table by a certain user, and secondarily associating 4/5/6 three tables to form a query view statement. Such as: the user 01 and the query table a.A acquire the sensitive fields as c1, c2 and c3 according to the user-sensitive information relation table, and the related sensitive information module nests the sensitive fields into a deformation function. And associating the 6 user condition relation table with the 5 condition authority relation table to form the area of the user to the query table and the access range condition of the professional organization. A query view statement is formed.
Second, query calling
The query calling function is applied to a foreground server, and is mainly used for calling the functions of the modules of the database server to form query view statements, controlling query data in the database by mass data authority, and acquiring and displaying a returned result.
The specific process of the data access system for realizing the data access method in the distributed system is as follows:
s1: receiving a data access request of a target user for a distributed system, wherein the data access request comprises a user identifier of the target user, an identifier of a target database in the distributed system and an identifier of a target data table in the target database.
S2: judging whether the distributed system comprises the target data table in the target database or not according to the identifier of the target database and the identifier of the target data table, and if not, executing the step S3; if yes, go to step S4.
S3: and sending a notification message for indicating that the target data table in the target database is not contained in the distributed system to the target user, and ending the current process.
S4: acquiring a pre-stored user-condition relation table, and searching a region field corresponding to the target user and a serial number of an organization to which the user belongs in the user-condition relation table according to the user identification of the target user;
the user-condition relation table stores the association relation between the user and the region and the mechanism to which the user belongs. And storing the user ID, the area code where the user is located and the serial number of the user mechanism. Such as: users 01, 1301, 003.
S5: and acquiring a preset region parameter table, and acquiring a region name corresponding to the region field in the region parameter table according to the region field corresponding to the target user.
The regional parameter table is used for storing administrative region information, including provincial level, city level and county level. And (3) storing: area number, area name. Such as: hebei province branch (area code: 1300) and Shijiazhuang branch (area code: 1301) in the bank system.
S6: and acquiring a preset mechanism parameter table to which the user belongs, and acquiring the name of the mechanism to which the user belongs corresponding to the mechanism number to which the user belongs in the mechanism parameter table to which the user belongs according to the mechanism number to which the user corresponding to the target user belongs.
The mechanism parameter table to which the user belongs is used for storing mechanism information to which different users belong. Including department code, department name. Such as: credit department (001), wind control department (002), accounting department (003) in the bank system, etc.
S7: acquiring a pre-stored condition-permission relation table, and searching a region field corresponding to the target database and a serial number of an organization to which a user belongs in the condition-permission relation table according to the identifier of the target database;
the condition-authority relation table is used for storing the data table and the association relation between the region and the mechanism to which the user belongs. And storing the library name, the table name, the area field screening logic (standby) and the serial number of the organization to which the user belongs. Such as: library a, table a, zone field (zoneono), zone field filter logic (substr, 2,4)), organization number to which the user belongs (003).
S8: and acquiring a preset region parameter table, and acquiring a region name corresponding to the region field in the region parameter table according to the region field corresponding to the target database.
S9: and acquiring a preset mechanism parameter table to which the user belongs, and acquiring the name of the mechanism to which the user belongs corresponding to the mechanism number to which the user belongs in the mechanism parameter table to which the user belongs according to the mechanism number to which the user belongs corresponding to the target database.
The execution sequence between S4-S6 and S7-S9 may be sequential, or simultaneous, and is specifically set according to the actual application situation, which is not limited in the present application.
S10: judging whether the target user meets the in-region data access qualification for the target database or not according to the region name corresponding to the target user obtained in S5 and the region name corresponding to the target database obtained in S8, and if not, executing S11; if yes, go to S12.
S11: and sending a notification message for indicating that the target user does not meet the data access qualification in the region aiming at the target database to the target user, and ending the current process.
S12: judging whether the target user meets the mechanism access qualification of the user belonging to the target database or not according to the mechanism name of the user belonging to the target user obtained in the step S6 and the mechanism name of the user belonging to the target database obtained in the step S9, and if not, executing the step S13; if yes, go to S14.
S13: and sending a notification message for indicating that the target user does not meet the access qualification of the mechanism to which the user belongs for the target database to the target user, and ending the current process.
S14: and acquiring a pre-stored user-sensitive information relation table, and searching the access authority and the sensitive field of the target data table in the target database which the target user requests to access from the user-sensitive information relation table according to the user identification of the target user, the identification of the target database in the distributed system and the identification of the target data table in the target database.
The user-sensitive information relation table is used for storing the incidence relation between the sensitive information and the user. And storing user ID, library name, table name, authority and sensitive field. For example, user 01, library a, table a, authority 1 (1-entitled, 0 not entitled), sensitive field (name, telephone, address, field name requiring nested sensitive field separated by comma).
S15: according to the sensitive field of the target data table in the target database which the target user requests to access, judging whether the data to be accessed by the target user contains sensitive data (for example, if the sensitive field is null or '0', it is indicated that the data to be accessed by the target user does not contain the sensitive data, and if the sensitive field is 'telephone', it is indicated that the data to be accessed by the target user contains the sensitive data of the telephone number), if not, executing S16, and if so, executing S17.
S16: and calling the data in the target data table, generating a corresponding first query view without sensitive data, and displaying the first query view to enable the target user to query.
S17: judging whether the target user has the authority to access the sensitive data in the target data table according to the access authority of the target user in the target data table in the target database which the target user requests to access, if so, executing S18; if five, then S19 is performed.
S18: and calling the data in the target data table, generating a corresponding second query view with sensitive data, and displaying the second query view to enable the target user to query.
S19: and acquiring a preset sensitive information list, and acquiring a corresponding deformation indication identifier and a corresponding deformation rule ID from the sensitive information list according to the sensitive field of the target data table.
The sensitive information list is used for storing a table ID, a library name, a table name, a field name, whether to deform (0-no, 1-yes), a deformation rule ID (a name deformation rule is 1, a mobile phone number deformation rule is 2, an address deformation rule is 3, a mailbox deformation rule is 4, and the like), and on the contrary, if a certain field does not relate to sensitive information, whether to deform the field is 0, and the deformation rule ID is empty. Such as: table ID 001, library name a, table name a, field name c1, whether morph to 1, morph rule to 1.
S20: judging whether the sensitive field of the target data table needs to be deformed according to the deformation indication identifier, and if not, executing S21; if yes, go to S22.
S21 (same as S18): and calling the data in the target data table, generating a corresponding second query view with sensitive data, and displaying the second query view to enable the target user to query.
S22: and acquiring a preset deformation rule, and acquiring a corresponding deformation rule nesting function from the deformation rule according to the deformation rule ID corresponding to the target data table.
And the deformation rule is used for storing a deformation rule nesting function corresponding to the deformation logic. And storing the deformation rule ID and the deformation function. If the name transformation rule is 1, the transformation function is name ($ x), and $ x is the identifier of the field name.
S23: and carrying out deformation processing on the sensitive field in the target data table according to a deformation rule nesting function corresponding to the target data table, then calling the sensitive field after the deformation processing and the non-sensitive data in the target data table to generate a corresponding third query view containing the sensitive data after the deformation processing, and displaying the third query view so as to enable the target user to query.
Based on this, in a specific example of a user performing distributed system data access, a specific process implemented based on the Hadoop platform Hive is as follows:
1. user query process
A user puts forward a query request, a data access device in the distributed system checks whether query authority exists, if yes, a corresponding query view name is extracted, the query is carried out in a big data platform, and a query result is returned; if not, returning no inquiry authority to end the inquiry.
2. User authority judgment process
The user provides a request for requesting for applying for inquiring authority, the device judges whether the data application qualification in the administrative region is met, if not, the device refutes the application, and if so, the device judges whether sensitive information needs to be inquired.
And the data access device in the distributed system judges whether the sensitive information needs to be inquired, if the sensitive information does not need to be inquired, the sensitive information passes examination and approval and creates a user-view relation mapping relation to be returned to the user. And if the sensitive information needs to be inquired, examining and approving whether the sensitive information view meets the inquiry requirement of the user.
And the data access device in the distributed system judges whether the existing view meets the requirement, if so, the view passes the examination and approval, and a user-view relation mapping relation is created, and if not, a new view is created on the big data platform and a new sensitive information field is associated.
Based on the technical scheme, the data access method in the distributed system realized by the data access system provided by the application example of the application example carries out authority management on mass data according to table-user granularity, and has the following beneficial effects:
1. the problem that the authority management cannot be mapped one by a sea table-mass users in the existing mode is solved;
2. and user authority management under the branch organization and branch administrative regions is realized.
In terms of hardware, in order to solve the problems of low data access efficiency, low reliability, low automation degree, and the like in a data access manner in the existing distributed system, the present application provides an embodiment of an electronic device for implementing all or part of contents in a data access method in the distributed system, where the electronic device specifically includes the following contents:
fig. 15 is a schematic block diagram of a system configuration of an electronic device 9600 according to an embodiment of the present application. As shown in fig. 15, the electronic device 9600 can include a central processor 9100 and a memory 9140; the memory 9140 is coupled to the central processor 9100. Notably, this fig. 15 is exemplary; other types of structures may also be used in addition to or in place of the structure to implement telecommunications or other functions.
In one embodiment, the data access functions in a distributed system may be integrated into a central processor. Wherein the central processor may be configured to control:
step 100: and determining the sensitive data in the target data table in the target database in the distributed system currently designated by the target user without the sensitive data access authority.
In step 100, a user may first make a query request, and the device checks whether a query permission exists, and if so, extracts a corresponding query view name, performs a query on a big data platform, and returns a query result; if not, returning no inquiry authority to end the inquiry. And then, the data access device in the distributed system judges whether the query request conforms to the data application qualification in the administrative region, if not, the query request is rejected, and if so, whether sensitive information needs to be queried is judged.
Step 200: and performing deformation processing on the sensitive data in the target data table according to a deformation rule nesting function corresponding to the sensitive data.
In step 200, the contents of different fields of different tables can be labeled in advance to form a sensitive information list. Uploading the sensitive information deformation function to a big data platform, and creating a view of a source table in the big data platform to form a table view of the nested sensitive information deformation function. The view of the same source table has multiple views due to the size of the authority range.
Step 300: and outputting the sensitive data after the deformation processing and the non-sensitive data in the target data table to the target user.
As can be seen from the above description, in the electronic device provided in the embodiment of the present application, if the target data table contains sensitive data, the sensitive data in the target data table is set to be subjected to deformation processing according to the deformation rule nesting function corresponding to the sensitive field of the sensitive data, and the sensitive data can be safely accessed automatically without the participation of a higher-level manager, so that on the basis of ensuring the safety and privacy of data access in the distributed system, the efficiency and the degree of automation of data access in the distributed system are effectively improved, the reliability of data access in the distributed system is effectively improved, and the operation stability of the distributed system is effectively ensured.
In another embodiment, the data access apparatus in the distributed system may be configured separately from the central processor 9100, for example, the data access apparatus in the distributed system may be configured as a chip connected to the central processor 9100, and the data access function in the distributed system is realized by the control of the central processor.
As shown in fig. 15, the electronic device 9600 may further include: a communication module 9110, an input unit 9120, an audio processor 9130, a display 9160, and a power supply 9170. It is noted that the electronic device 9600 also does not necessarily include all of the components shown in fig. 15; further, the electronic device 9600 may further include components not shown in fig. 15, which can be referred to in the related art.
As shown in fig. 15, a central processor 9100, sometimes referred to as a controller or operational control, can include a microprocessor or other processor device and/or logic device, which central processor 9100 receives input and controls the operation of the various components of the electronic device 9600.
The memory 9140 can be, for example, one or more of a buffer, a flash memory, a hard drive, a removable media, a volatile memory, a non-volatile memory, or other suitable device. The information relating to the failure may be stored, and a program for executing the information may be stored. And the central processing unit 9100 can execute the program stored in the memory 9140 to realize information storage or processing, or the like.
The input unit 9120 provides input to the central processor 9100. The input unit 9120 is, for example, a key or a touch input device. Power supply 9170 is used to provide power to electronic device 9600. The display 9160 is used for displaying display objects such as images and characters. The display may be, for example, an LCD display, but is not limited thereto.
The memory 9140 can be a solid state memory, e.g., Read Only Memory (ROM), Random Access Memory (RAM), a SIM card, or the like. There may also be a memory that holds information even when power is off, can be selectively erased, and is provided with more data, an example of which is sometimes called an EPROM or the like. The memory 9140 could also be some other type of device. Memory 9140 includes a buffer memory 9141 (sometimes referred to as a buffer). The memory 9140 may include an application/function storage portion 9142, the application/function storage portion 9142 being used for storing application programs and function programs or for executing a flow of operations of the electronic device 9600 by the central processor 9100.
The memory 9140 can also include a data store 9143, the data store 9143 being used to store data, such as contacts, digital data, pictures, sounds, and/or any other data used by an electronic device. The driver storage portion 9144 of the memory 9140 may include various drivers for the electronic device for communication functions and/or for performing other functions of the electronic device (e.g., messaging applications, contact book applications, etc.).
The communication module 9110 is a transmitter/receiver 9110 that transmits and receives signals via an antenna 9111. The communication module (transmitter/receiver) 9110 is coupled to the central processor 9100 to provide input signals and receive output signals, which may be the same as in the case of a conventional mobile communication terminal.
Based on different communication technologies, a plurality of communication modules 9110, such as a cellular network module, a bluetooth module, and/or a wireless local area network module, may be provided in the same electronic device. The communication module (transmitter/receiver) 9110 is also coupled to a speaker 9131 and a microphone 9132 via an audio processor 9130 to provide audio output via the speaker 9131 and receive audio input from the microphone 9132, thereby implementing ordinary telecommunications functions. The audio processor 9130 may include any suitable buffers, decoders, amplifiers and so forth. In addition, the audio processor 9130 is also coupled to the central processor 9100, thereby enabling recording locally through the microphone 9132 and enabling locally stored sounds to be played through the speaker 9131.
An embodiment of the present application further provides a computer-readable storage medium capable of implementing all the steps in the data access method in the distributed system in the foregoing embodiment, where the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the computer program implements all the steps of the data access method in the distributed system in which an execution subject is a server or a client in the foregoing embodiment, for example, when the processor executes the computer program, the processor implements the following steps:
step 100: and determining the sensitive data in the target data table in the target database in the distributed system currently designated by the target user without the sensitive data access authority.
In step 100, a user may first make a query request, and the device checks whether a query permission exists, and if so, extracts a corresponding query view name, performs a query on a big data platform, and returns a query result; if not, returning no inquiry authority to end the inquiry. And then, the data access device in the distributed system judges whether the query request conforms to the data application qualification in the administrative region, if not, the query request is rejected, and if so, whether sensitive information needs to be queried is judged.
Step 200: and performing deformation processing on the sensitive data in the target data table according to a deformation rule nesting function corresponding to the sensitive data.
In step 200, the contents of different fields of different tables can be labeled in advance to form a sensitive information list. Uploading the sensitive information deformation function to a big data platform, and creating a view of a source table in the big data platform to form a table view of the nested sensitive information deformation function. The view of the same source table has multiple views due to the size of the authority range.
Step 300: and outputting the sensitive data after the deformation processing and the non-sensitive data in the target data table to the target user.
As can be seen from the above description, in the computer-readable storage medium provided in this embodiment of the present application, if the target data table contains sensitive data, the sensitive data in the target data table is set to be subjected to deformation processing according to the nested function of the deformation rule corresponding to the sensitive field of the sensitive data, and the sensitive data can be safely accessed without the participation of a higher-level manager, so that on the basis of ensuring the security and privacy of data access in the distributed system, the efficiency and the degree of automation of data access in the distributed system can be effectively improved, the reliability of data access in the distributed system can be effectively improved, and the operation stability of the distributed system can be effectively ensured.
As will be appreciated by one skilled in the art, embodiments of the present invention may be provided as a method, apparatus, or computer program product. Accordingly, the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present invention may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and the like) having computer-usable program code embodied therein. The present invention is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (devices), and computer program products according to embodiments of the invention. It will be understood that each flow and/or block of the flow diagrams and/or block diagrams, and combinations of flows and/or blocks in the flow diagrams and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks. These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
The principle and the implementation mode of the invention are explained by applying specific embodiments in the invention, and the description of the embodiments is only used for helping to understand the method and the core idea of the invention; meanwhile, for a person skilled in the art, according to the idea of the present invention, there may be variations in the specific embodiments and the application scope, and in summary, the content of the present specification should not be construed as a limitation to the present invention.

Claims (15)

1. A method for accessing data in a distributed system, comprising:
determining sensitive data in a target data table in a target database in a distributed system currently specified by a target user without sensitive data access authority;
performing deformation processing on the sensitive data in the target data table according to a deformation rule nesting function corresponding to the sensitive data;
and outputting the sensitive data after the deformation processing and the non-sensitive data in the target data table to the target user.
2. The method of claim 1, prior to determining the sensitive data in the target data table in the target database in the distributed system currently designated by the target user without the sensitive data access right, further comprising:
receiving a data access request of a target user for a distributed system, wherein the data access request comprises a user identifier of the target user, an identifier of a target database in the distributed system and an identifier of a target data table in the target database;
judging whether the distributed system comprises a target data table in the target database or not according to the identification of the target database and the identification of the target data table, if so, acquiring the access condition of the target user;
and judging whether the target user has the access qualification of the target data table according to the access condition of the target user, and if so, acquiring the access authority and sensitive fields of the target data table which the target user requests to access.
3. The method according to claim 2, wherein the obtaining the access condition of the target user comprises:
determining a region field corresponding to the user identification of the target user from a preset user and condition relation table;
and determining the area name of the area field corresponding to the target user in a preset area parameter table.
4. The method for accessing data in a distributed system according to claim 3, wherein said determining whether the target user qualifies for accessing the target data table according to the access condition of the target user includes:
determining a region field corresponding to the identifier of the target database from a preset condition and permission relation table;
determining the area name of the area field corresponding to the target database in the area parameter table;
and judging whether the target user accords with the in-region data access qualification aiming at the target database or not according to the region name corresponding to the target user and the region name corresponding to the target database.
5. The method according to claim 2, wherein the obtaining the access condition of the target user comprises:
determining the organization number to which the user corresponding to the user identifier of the target user belongs from a preset user and condition relation table;
and determining the name of the mechanism to which the user belongs according to the mechanism number to which the user belongs and corresponding to the target user in a preset mechanism parameter table to which the user belongs.
6. The method for accessing data in a distributed system according to claim 5, wherein said determining whether the target user qualifies for accessing the target data table according to the access condition of the target user includes:
determining the organization number to which the user corresponding to the identifier of the target database belongs from a preset condition and authority relation table;
determining the name of the mechanism to which the user belongs of the mechanism number of the user corresponding to the target database in the mechanism parameter table to which the user belongs;
and judging whether the target user meets the mechanism access qualification of the user belonging to the target database or not according to the mechanism name of the user belonging to the target user and the mechanism name of the user belonging to the target database.
7. The method according to claim 2, wherein the obtaining of the access right and the sensitive field of the target data table requested to be accessed by the target user comprises:
and searching the access authority and the sensitive field of the target data table in the target database which the target user requests to access from a preset user and sensitive information relation table according to the user identification of the target user, the identification of the target database in the distributed system and the identification of the target data table in the target database.
8. The method for data access in a distributed system according to any one of claims 2 to 7, wherein the determining sensitive data in a target data table in a target database in the distributed system currently specified by a target user without access rights to the sensitive data comprises:
judging whether the data requested to be accessed by the target user contains sensitive data or not according to the content of the sensitive field of the target data table requested to be accessed by the target user, if so, judging whether the target user has the sensitive data access right for accessing the sensitive data in the target data table or not based on the access right of the target data table requested to be accessed by the target user;
and if the target user does not have the sensitive data access authority, calling the sensitive data in the target data table.
9. The method according to claim 8, wherein the performing transformation processing on the sensitive data in the target data table according to the transformation rule nesting function corresponding to the sensitive data includes:
acquiring a deformation indication identifier and a deformation rule ID corresponding to a sensitive field of the target data table from a preset sensitive information list;
judging whether a sensitive field of the target data table needs to be deformed or not according to the deformation indication identifier, if so, calling a deformation rule nesting function corresponding to the deformation rule ID from a preset deformation rule;
and performing deformation processing on the sensitive data in the target data table based on the deformation rule nesting function.
10. The method according to claim 1, wherein outputting the sensitive data after the transformation process and the non-sensitive data in the target data table to the target user comprises:
and generating a data query view containing the deformed sensitive data according to the deformed sensitive data and the non-sensitive data in the target data table, and displaying the data query view to enable the target user to query.
11. The method for accessing data in a distributed system according to claim 8, further comprising:
and if the data requested to be accessed by the target user does not contain sensitive data, or if the target user has the sensitive data access authority, generating a data query view according to the data in the target data table, and displaying the data query view to enable the target user to query.
12. A data access apparatus in a distributed system, comprising:
the data searching module is used for determining the sensitive data in the target data table in the target database in the distributed system currently specified by the target user without the sensitive data access authority;
the data deformation module is used for performing deformation processing on the sensitive data in the target data table according to a deformation rule nesting function corresponding to the sensitive data;
and the data output module is used for outputting the sensitive data after the deformation processing and the non-sensitive data in the target data table to the target user.
13. A data access system, comprising: the system comprises a control server, a database server and a data warehouse;
the database server is used for executing the data access method in the distributed system of claims 1 to 11;
the control server is used for sending a data access request of the target user for the distributed system to the database server so that the database server can determine whether the target user has sensitive data access permission according to the data access request;
the data warehouse is used for storing each data table in each database in the distributed system, so that the database server determines the sensitive data in the target data table in the target database in the distributed system currently specified by the target user without the sensitive data access authority from the data warehouse.
14. An electronic device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, wherein the processor implements a data access method in a distributed system according to any one of claims 1 to 11 when executing the computer program.
15. A computer-readable storage medium, on which a computer program is stored, which, when being executed by a processor, carries out a method for data access in a distributed system according to any one of claims 1 to 11.
CN202110593606.0A 2021-05-28 2021-05-28 Data access method, device and system in distributed system Pending CN113221177A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202110593606.0A CN113221177A (en) 2021-05-28 2021-05-28 Data access method, device and system in distributed system

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202110593606.0A CN113221177A (en) 2021-05-28 2021-05-28 Data access method, device and system in distributed system

Publications (1)

Publication Number Publication Date
CN113221177A true CN113221177A (en) 2021-08-06

Family

ID=77099186

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202110593606.0A Pending CN113221177A (en) 2021-05-28 2021-05-28 Data access method, device and system in distributed system

Country Status (1)

Country Link
CN (1) CN113221177A (en)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114844961A (en) * 2022-04-22 2022-08-02 苏州浪潮智能科技有限公司 Method, device, equipment and storage medium for protocol intercommunication of distributed system

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN107704770A (en) * 2017-09-28 2018-02-16 平安普惠企业管理有限公司 Sensitive information desensitization method, system, equipment and readable storage medium storing program for executing
CN109409121A (en) * 2018-09-07 2019-03-01 阿里巴巴集团控股有限公司 Desensitization process method, apparatus and server
CN109815081A (en) * 2018-12-14 2019-05-28 深圳壹账通智能科技有限公司 The long range acquisition method and collection device of database performance
CN110245470A (en) * 2019-04-25 2019-09-17 深圳壹账通智能科技有限公司 Intelligent data desensitization method, device, computer equipment and storage medium
CN110245505A (en) * 2019-05-20 2019-09-17 中国平安人寿保险股份有限公司 Tables of data access method, device, computer equipment and storage medium
CN112257097A (en) * 2020-11-23 2021-01-22 浪潮云信息技术股份公司 Partition authority management method based on distributed database

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN107704770A (en) * 2017-09-28 2018-02-16 平安普惠企业管理有限公司 Sensitive information desensitization method, system, equipment and readable storage medium storing program for executing
CN109409121A (en) * 2018-09-07 2019-03-01 阿里巴巴集团控股有限公司 Desensitization process method, apparatus and server
CN109815081A (en) * 2018-12-14 2019-05-28 深圳壹账通智能科技有限公司 The long range acquisition method and collection device of database performance
CN110245470A (en) * 2019-04-25 2019-09-17 深圳壹账通智能科技有限公司 Intelligent data desensitization method, device, computer equipment and storage medium
CN110245505A (en) * 2019-05-20 2019-09-17 中国平安人寿保险股份有限公司 Tables of data access method, device, computer equipment and storage medium
CN112257097A (en) * 2020-11-23 2021-01-22 浪潮云信息技术股份公司 Partition authority management method based on distributed database

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114844961A (en) * 2022-04-22 2022-08-02 苏州浪潮智能科技有限公司 Method, device, equipment and storage medium for protocol intercommunication of distributed system
CN114844961B (en) * 2022-04-22 2023-08-11 苏州浪潮智能科技有限公司 Distributed system protocol intercommunication method, device, equipment and storage medium

Similar Documents

Publication Publication Date Title
US10848520B2 (en) Managing access to resources
US8959114B2 (en) Entitlement management in an on-demand system
CN102651775B (en) Based on method, the equipment and system of many tenants shared object management of cloud computing
CN102724221A (en) Enterprise information system using cloud computing and method for setting user authority thereof
CN110569667B (en) Access control method and device, computer equipment and storage medium
US20160307284A1 (en) Methods and systems relating to contextual information aggregation and dissemination
US20230319001A1 (en) Snippet(s) of content associated with a communication platform
CN103793656A (en) Security implemented through metadata orchestrators
US11741254B2 (en) Privacy centric data security in a cloud environment
CN113574528A (en) Providing policy-compliant storage for DID data
CN113392158A (en) Service data processing method and device and data center
CN105814864B (en) A kind of input and output I/O request processing method and file server
US9043342B2 (en) Method and system for policy driven data distribution
CN111897890A (en) Financial business processing method and device
US20220006640A1 (en) Blockchain with non-turing complete system guards
CN113221177A (en) Data access method, device and system in distributed system
CN112699407A (en) Service data access method, device, equipment and storage medium
US11449352B2 (en) Systems and methods for converting record formats
CN112818038A (en) Data management method based on combination of block chain and IPFS (Internet protocol file system) and related equipment
US10083246B2 (en) Apparatus and method for universal personal data portability
CN115766296A (en) User account authority control method, device, server and storage medium
WO2023064562A1 (en) Trust relationships to share client assets among client accounts in a software as a service platform
KR20190119233A (en) Member management service system using big data analysistem
CN113840013A (en) Document system for hierarchical management
CN112329047A (en) Private data encryption query method and device

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination