CN111291360B - Big data authority management method and system based on service management - Google Patents
Big data authority management method and system based on service management Download PDFInfo
- Publication number
- CN111291360B CN111291360B CN202010070975.7A CN202010070975A CN111291360B CN 111291360 B CN111291360 B CN 111291360B CN 202010070975 A CN202010070975 A CN 202010070975A CN 111291360 B CN111291360 B CN 111291360B
- Authority
- CN
- China
- Prior art keywords
- big data
- access
- service node
- micro
- service
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F21/00—Security arrangements for protecting computers, components thereof, programs or data against unauthorised activity
- G06F21/30—Authentication, i.e. establishing the identity or authorisation of security principals
- G06F21/45—Structures or tools for the administration of authentication
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/27—Replication, distribution or synchronisation of data between databases or within a distributed database system; Distributed database system architectures therefor
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F21/00—Security arrangements for protecting computers, components thereof, programs or data against unauthorised activity
- G06F21/30—Authentication, i.e. establishing the identity or authorisation of security principals
- G06F21/45—Structures or tools for the administration of authentication
- G06F21/46—Structures or tools for the administration of authentication by designing passwords or checking the strength of passwords
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F21/00—Security arrangements for protecting computers, components thereof, programs or data against unauthorised activity
- G06F21/60—Protecting data
- G06F21/604—Tools and structures for managing or administering access control systems
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/46—Multiprogramming arrangements
- G06F9/50—Allocation of resources, e.g. of the central processing unit [CPU]
- G06F9/5083—Techniques for rebalancing the load in a distributed system
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
- Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management
Abstract
The invention discloses a big data authority management method and a system based on service management, which are characterized in that all big data functional modules or ports are independently used as big data micro-service nodes through a service management technology, the big data micro-service nodes and the big data nodes are identified by key authentication, a plurality of big data micro-service nodes are connected to an external access control micro-service node, an access control service mainly provides external access data connection through a RestFul, and an interceptor is added at the front end of an access interface to realize safe filtering access of all accesses, so that authentication of access authorities and filtration of access content details are realized, and fine-granularity access authority management is realized.
Description
Technical Field
The invention relates to the technical field of Internet big data, in particular to a big data authority management method and system based on service management.
Background
The existing big data platform is an open source architecture, has no authority management function module, and has potential data safety hazard. Generally, the production environment is implemented by a key authentication (for example, using Kerberos), but as the number of nodes of a large data platform, the number of architecture components, the number of service applications, the number of access users, etc. increases, the following problems exist in this way: the deployment and implementation are complicated, the authority management becomes extremely complicated, the user identity authentication is only a small part of the authority management, and the authority management on file directories, databases, tables, service access interfaces and the like cannot be performed, especially under the condition that the nodes of a big data cluster are continuously increased.
Disclosure of Invention
The invention aims to provide a big data authority management method and a system based on service management, which realize the unified management of identity authentication and access authority of each module of big data based on service management, and perform authority management operation on a visual interface; fine-granularity authority management of accessed tables, rows, fields, contents and the like can be realized by realizing fine-granularity authority accessed by a large data platform based on service management; the authority unified management with other platforms and the single-point login function of user login are realized based on service management.
The technical scheme adopted by the invention is as follows:
the large data authority management system based on service management comprises a user access terminal, an access control micro service node, a large data micro service node and a large data server cluster, wherein the user access terminal is in communication connection with the access control micro service node, a plurality of large data micro service nodes are respectively connected to the access control micro service node, the corresponding large data micro service node is connected with the access end of each functional module of the large data server cluster through a service management technology, and an interceptor is arranged at the front end of the access control micro service node to safely filter all accesses and provide access authentication and authority control.
Further, the access control micro service node provides external access data connection through the RestFul. Each big data micro-service node is connected with the access end of each big data functional module through a Restful.
Further, the big data micro service node realizes the load balance and high-availability disaster recovery of the access service by configuring the load balance rule.
Further, each big data micro-service node is configured to query, add, modify, delete, and traverse service data for a big data server cluster.
Further, the big data micro service node is provided with a key mechanism, and authentication is carried out on the access control service node accessed by the RestFul through the key mechanism, so that only the appointed access control micro service node can safely access the big data micro service node through the RestFul.
Further, each big data micro service node has at least one hot-backed big data micro service node in other big data micro service nodes.
Further, the large data server cluster includes hdfs, hbase, hive, spark large databases.
Further, the invention also discloses a big data authority management method based on service management, which comprises the following steps:
step 1, a user sends an access request to an access control micro-service node through a user access terminal;
step 2, the access control micro-service node carries out security filtering on the access, and unsafe access is filtered according to the setting;
step 3, the access control micro service node obtains authentication information based on the access request to carry out access authentication, and gives different access rights based on different user accounts;
step 4, the access control micro service node forwards a corresponding access request to the big data micro service node after carrying the access key based on the given access right;
step 5, the big data micro service node obtains the sending information of the access control micro service node and obtains the access key to verify whether the access control micro service node is an authorized access control micro service node; if yes, the big data micro service node receives a response access request and executes the step 6; otherwise, refusing the access request forwarded by the access control micro service node and ending;
and 6, acquiring corresponding data from each functional module of the big data server cluster based on the access request by the big data micro-service node and feeding the corresponding data back to the user access terminal in sequence.
Further, in step 1, the access control micro service node provides external access data connection for the user access terminal through the RestFul.
Further, in step 4, the access control micro service node selects the corresponding big data micro service node to respond to the access request by configuring the load balancing rule so as to balance the load balancing of each big data micro service node and meet the high-availability disaster recovery requirement.
According to the technical scheme, all big data functional modules or ports are independently used as big data micro-service nodes, the big data micro-service nodes and the big data nodes are identified for identity authentication through key authentication, a plurality of big data micro-service nodes are connected to the micro-service nodes for external access control, the access control service mainly provides external access data connection through the RestFul, and an interceptor is added at the front end of the access interface to realize safe filtering access of all accesses, so that authentication of access rights and filtration of access content details are realized, and fine-granularity access rights management is realized.
Drawings
The invention is described in further detail below with reference to the drawings and detailed description;
FIG. 1 is a schematic diagram of a big data authority management method based on service management.
Detailed Description
As shown in fig. 1, the invention discloses a big data authority management system based on service management, which comprises a user access terminal, an access control micro service node, a big data micro service node and a big data server cluster, wherein the user access terminal is in communication connection with the access control micro service node, a plurality of big data micro service nodes are respectively connected to the access control micro service node, the corresponding big data micro service node is connected with the access ends of all functional modules of the big data server cluster through a service management technology, and the front end of the access control micro service node is provided with an interceptor for safely filtering all accesses and simultaneously providing access authentication and authority control.
Further, the access control micro service node provides external access data connection through the RestFul. Each big data micro-service node is connected with the access end of each big data functional module through a Restful.
Further, the big data micro service node realizes the load balance and high-availability disaster recovery of the access service by configuring the load balance rule.
Further, each big data micro-service node is configured to query, add, modify, delete, and traverse service data for a big data server cluster.
Further, the big data micro service node is provided with a key mechanism, and authentication is carried out on the access control service node accessed by the RestFul through the key mechanism, so that only the appointed access control micro service node can safely access the big data micro service node through the RestFul.
Further, each big data micro service node has at least one hot-backed big data micro service node in other big data micro service nodes.
Further, the large data server cluster includes hdfs, hbase, hive, spark large databases.
Further, the invention also discloses a big data authority management method based on service management, which comprises the following steps:
step 1, a user sends an access request to an access control micro-service node through a user access terminal;
step 2, the access control micro-service node carries out security filtering on the access, and unsafe access is filtered according to the setting;
step 3, the access control micro service node obtains authentication information based on the access request to carry out access authentication, and gives different access rights based on different user accounts;
step 4, the access control micro service node forwards a corresponding access request to the big data micro service node after carrying the access key based on the given access right;
step 5, the big data micro service node obtains the sending information of the access control micro service node and obtains the access key to verify whether the access control micro service node is an authorized access control micro service node; if yes, the big data micro service node receives a response access request and executes the step 6; otherwise, refusing the access request forwarded by the access control micro service node and ending;
and 6, acquiring corresponding data from each functional module of the big data server cluster based on the access request by the big data micro-service node and feeding the corresponding data back to the user access terminal in sequence.
Further, in step 1, the access control micro service node provides external access data connection for the user access terminal through the RestFul.
Further, in step 4, the access control micro service node selects the corresponding big data micro service node to respond to the access request by configuring the load balancing rule so as to balance the load balancing of each big data micro service node and meet the high-availability disaster recovery requirement.
The present invention will be described in detail below:
the system of the invention is characterized in that:
the large data micro service node is 1) connected with the access end of each large data functional module through a service management technology, the access entrance of large data such as hdfs, hbase, hive, spark is respectively connected with the corresponding micro service node through a Restful, and the authentication based on the secret key is started at the large data access end, so that only the registered large data micro service node can access the large data platform. 2) Service management: the method for realizing access to the big data platform by inquiring, adding, modifying, deleting, traversing all the service data on each big data micro-service node is realized, and the method realizes the RestFul call, can remotely call and access, and realizes remote access to the big data platform. 3) Micro service node identity verification: and a key mechanism is started to realize the identity verification of the access control service node accessed by the RestFul, so that only the access control service node can safely access the big data platform through the RestFul. 4) High availability service remediation: through a high availability mechanism of the micro service, each micro service node is guaranteed to have a hot backup micro service node, and load balancing and high availability disaster recovery of access service can be realized through configuration of load balancing rules.
And B, accessing and controlling a micro service node: 1) And (3) user authentication is accessed, namely, at the access control micro-service node, a user request which accords with user authentication (through account password authentication) and externally accesses the big data platform is received. 2) Access interception filtering: and intercepting all accesses through the access interception filtering module so as to realize further fine-granularity control of access authority of the access request content. The module can control access rights of basic operations such as inquiry, addition, modification, deletion, traversal and the like. 3) Fine-grained rights management: the access interception filtering module is configured with fine-grained access filtering rules, can perform fine-grained filtering interception on access request content, such as query SQL or search content containing sensitive information fields, can perform access blocking, and sets authority levels for the search of the sensitive fields. Thereby achieving fine-grained rights management at the row level, field level, or rights management at the row field level of a data warehouse of large data files.
The access control micro service node implements a visual web interface to manage each user: the identity account password authentication, the access right, the fine-granularity access right, and the like are used for authenticating the security right, so that unified right configuration management of the access rights of all components of the big data is realized. The invention can realize the butt joint of the rights of the big data platform and the rights of other system platforms through the rights safety management platform of the micro service, and can realize the unified management of the rights. And after the access rights are checked in and the identity is verified on other platforms (such as an office platform system for example), the corresponding rights which can be directly obtained can access the platform of the big data, namely the single-point check-in function of the big data platform and other platforms is realized.
According to the technical scheme, all big data functional modules or ports are independently used as big data micro-service nodes, the big data micro-service nodes and the big data nodes are identified for identity authentication through key authentication, a plurality of big data micro-service nodes are connected to the micro-service nodes for external access control, the access control service mainly provides external access data connection through the RestFul, and an interceptor is added at the front end of the access interface to realize safe filtering access of all accesses, so that authentication of access rights and filtration of access content details are realized, and fine-granularity access rights management is realized.
Claims (6)
1. Big data authority management system based on service management, its characterized in that: the system comprises a user access terminal, an access control micro service node, a big data micro service node and a big data server cluster, wherein the user access terminal is in communication connection with the access control micro service node; the access control micro-service nodes provide external access data connection through the RestFul, and each big data micro-service node is connected with the access end of each big data functional module through the RestFul; the big data micro service node is provided with a key mechanism, and the access control micro service node accessed by the RestFul is authenticated through the key mechanism, so that only the appointed access control micro service node can safely access the big data micro service node through the RestFul; the big data authority management system based on service governance executes a big data authority management method based on service governance, and the method comprises the following steps:
step 1, a user sends an access request to an access control micro-service node through a user access terminal;
step 2, the access control micro-service node carries out security filtering on the access request, and filters unsafe access request according to the setting;
step 3, the access control micro service node obtains authentication information based on the access request to carry out access authentication, and gives different access rights based on different user accounts;
step 4, the access control micro-service node forwards a corresponding access request to the big data micro-service node after carrying the access key based on the given access right;
step 5, the big data micro service node obtains the sending information of the access control micro service node and obtains the access key to verify whether the access control micro service node is an authorized access control micro service node; if yes, the big data micro service node responds to the access request and executes the step 6; otherwise, refusing the access request forwarded by the access control micro service node and ending;
and 6, acquiring corresponding data from each functional module of the big data server cluster based on the access request by the big data micro-service node and feeding the corresponding data back to the user access terminal in sequence.
2. The service governance-based big data rights management system of claim 1, wherein: and the big data micro service node realizes the load balance and high-availability disaster recovery of the access service by configuring the load balance rule.
3. The service governance-based big data rights management system of claim 1, wherein: each big data micro service node is used for querying, adding, modifying, deleting and traversing service data of the big data server cluster.
4. The service governance-based big data rights management system of claim 1, wherein: each big data micro service node has at least one big data micro service node of the hot backup in other big data micro service nodes.
5. The service governance-based big data rights management system of claim 1, wherein: the big data server cluster includes hdfs, hbase, hive, spark big databases.
6. The service governance-based big data rights management system of claim 1, wherein: and 4, the access control micro-service node selects a corresponding big data micro-service node to respond to the access request by configuring a load balancing rule.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010070975.7A CN111291360B (en) | 2020-01-21 | 2020-01-21 | Big data authority management method and system based on service management |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010070975.7A CN111291360B (en) | 2020-01-21 | 2020-01-21 | Big data authority management method and system based on service management |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN111291360A CN111291360A (en) | 2020-06-16 |
| CN111291360B true CN111291360B (en) | 2023-05-26 |
Family
ID=71021419
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010070975.7A Active CN111291360B (en) | 2020-01-21 | 2020-01-21 | Big data authority management method and system based on service management |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111291360B (en) |
Families Citing this family (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112149107A (en) * | 2020-09-01 | 2020-12-29 | 珠海市卓轩科技有限公司 | Unified authority management method, system, device and storage medium |
| CN112364338B (en) * | 2020-11-30 | 2024-04-09 | 杭州安恒信息技术股份有限公司 | Management method and device of micro-service framework, electronic device and storage medium |
| CN113157664B (en) * | 2021-03-18 | 2023-08-25 | 中睿信数字技术有限公司 | Data grading and authorizing method and system based on grading identification |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106357799A (en) * | 2016-10-20 | 2017-01-25 | 杭州东方通信软件技术有限公司 | Service bus intermediate system and calling method thereof |
| CN107025409A (en) * | 2017-06-27 | 2017-08-08 | 中经汇通电子商务有限公司 | A kind of data safety storaging platform |
| CN110149397A (en) * | 2019-05-20 | 2019-08-20 | 湖北亿咖通科技有限公司 | A kind of micro services integration method and device |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2015089171A1 (en) * | 2013-12-11 | 2015-06-18 | Intralinks, Inc. | Customizable secure data exchange environment |
-
2020
- 2020-01-21 CN CN202010070975.7A patent/CN111291360B/en active Active
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106357799A (en) * | 2016-10-20 | 2017-01-25 | 杭州东方通信软件技术有限公司 | Service bus intermediate system and calling method thereof |
| CN107025409A (en) * | 2017-06-27 | 2017-08-08 | 中经汇通电子商务有限公司 | A kind of data safety storaging platform |
| CN110149397A (en) * | 2019-05-20 | 2019-08-20 | 湖北亿咖通科技有限公司 | A kind of micro services integration method and device |
Non-Patent Citations (1)
| Title |
|---|
| 实时大数据开放平台;郑博等;《电信科学》;20171220;全文 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN111291360A (en) | 2020-06-16 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111291360B (en) | Big data authority management method and system based on service management | |
| US11616783B2 (en) | Security design and architecture for a multi-tenant Hadoop cluster | |
| CN107342992B (en) | System authority management method and device and computer readable storage medium | |
| US7356840B1 (en) | Method and system for implementing security filters for reporting systems | |
| EP2893686B1 (en) | Ldap-based multi-customer in-cloud identity management system | |
| US8935770B2 (en) | Authentication system, authentication method, and storage medium for realizing a multitenant service | |
| CN103095720B (en) | A kind of method for managing security of cloud storage system of dialogue-based management server | |
| US8051168B1 (en) | Method and system for security and user account integration by reporting systems with remote repositories | |
| US20050108526A1 (en) | Query server system security and privacy access profiles | |
| CN112906029B (en) | Method and system for controlling user authority through identification analysis | |
| US20150113614A1 (en) | Client based systems and methods for providing users with access to multiple data bases | |
| CN102004866A (en) | Method and device for user identity verification and access control of information system | |
| CN107689949B (en) | Database authority management method and system | |
| US11126460B2 (en) | Limiting folder and link sharing | |
| CN103188249A (en) | Concentration permission management system, authorization method and authentication method thereof | |
| US11836243B2 (en) | Centralized applications credentials management | |
| US7801967B1 (en) | Method and system for implementing database connection mapping for reporting systems | |
| CN102064953A (en) | System, device and method for configuring user right information of lightweight directory access protocol (ldap) server | |
| CN106330836B (en) | Access control method of server to client | |
| CN102201935A (en) | Access control method and device based on VIEW | |
| CN112948858A (en) | Method for supporting real-name account number authority | |
| Cisco | Using Security Manager Tools | |
| Cisco | Using Security Manager Tools | |
| Cisco | Using Security Manager Tools | |
| CN106533688A (en) | Security authentication method and device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |