CN107026825A - A kind of method and system for accessing big data system - Google Patents
A kind of method and system for accessing big data system Download PDFInfo
- Publication number
- CN107026825A CN107026825A CN201610073085.5A CN201610073085A CN107026825A CN 107026825 A CN107026825 A CN 107026825A CN 201610073085 A CN201610073085 A CN 201610073085A CN 107026825 A CN107026825 A CN 107026825A
- Authority
- CN
- China
- Prior art keywords
- operational access
- big data
- request
- data system
- access request
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L63/00—Network architectures or network communication protocols for network security
- H04L63/02—Network architectures or network communication protocols for network security for separating internal from external traffic, e.g. firewalls
- H04L63/0281—Proxies
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L63/00—Network architectures or network communication protocols for network security
- H04L63/08—Network architectures or network communication protocols for network security for authentication of entities
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L63/00—Network architectures or network communication protocols for network security
- H04L63/10—Network architectures or network communication protocols for network security for controlling access to devices or network resources
Landscapes
- Engineering & Computer Science (AREA)
- Computer Hardware Design (AREA)
- Computer Security & Cryptography (AREA)
- Computing Systems (AREA)
- General Engineering & Computer Science (AREA)
- Computer Networks & Wireless Communication (AREA)
- Signal Processing (AREA)
- Storage Device Security (AREA)
Abstract
The invention provides a kind of method and system for accessing big data system, it is related to big data security fields, this method includes：Detect whether that receiving the operational access to big data system sent application side by application programming interface API asks；When receiving the operational access request, the authentication information that the application side is provided is obtained；According to the authentication information, according to the authentication rule pre-seted, authentication detection is carried out to the application side；When the application side meets authentication condition, judge whether the operational access request is the authorization requests with authority；When it is the authorization requests with authority that the operational access, which is asked, operational access request is forwarded to the big data system.The program can largely improve the security of stored data in big data platform.
Description
Technical field
The present invention relates to big data security fields, more particularly to a kind of method for accessing big data system and it is
System.
Background technology
Big data and cloud computing start research boom in the whole world, are wideling popularize the mistake of big data and cloud computing
Cheng Zhong, big data turns into safely the key primarily captured.Security protection currently for big data system is basic
By the way of traditional：Access big data system to carry out by 4A fort machine using SSH instruments, data
Safety relies on the access control of order and national treasury.However, there are multiple access approach in big data system.Such as：
In addition to SSH, the API Access that can also be provided by big data system.Now traditional fort machine side
Protective action can not be played when formula is for API Access data.As can be seen here, prior art is pacified in big data
There is technical leak in terms of full protection, how to improve the security of big data is worth further investigated.
In addition, for the incomplete situation of log recording existence information of big data, some systems are for essence
Simple memory space or the undue weighting to business, often lack user and access record in daily record.In fact,
The information falls within the focused protection content of big data safety, user behavior record and contains potentially large number of letter
Cease, can also expose the potential safety hazard of some systems presence, how user behavior record is carried out effectively utilizing
It is unavoidable problem in big data application and research.
For the security protection of big data system, there is following weak link in prior art：Existing fort
System can not be conducted interviews control to user by API Access big data system；Operation Log distribution storage
In each server and the enough behavior record information of shortage of big data platform so that big data system can not
Comprehensive utilization processing is carried out to it.
The content of the invention
In order to overcome above mentioned problem, the present invention provides a kind of method and system for accessing big data system, next gram
Take the problem of there is potential safety hazard in big data system.
In order to solve the above-mentioned technical problem, the present invention is adopted the following technical scheme that：
On the one hand, the invention provides a kind of method for accessing big data system, including：
Detect whether by application programming interface API receive that application side sends to big data system
Operational access request；
When receiving the operational access request, the authentication information that the application side is provided is obtained；
According to the authentication information, according to the authentication rule pre-seted, the application side is carried out
Authentication is detected；
When the application side meets authentication condition, judge whether the operational access request is with power
The authorization requests of limit；
When it is the authorization requests with authority that the operational access, which is asked, operational access request is turned
It is sent to the big data system.
Alternatively, methods described also includes：
When the application side does not meet authentication condition, or, when the operational access request be without
During the unauthorized request of authority, reminder message is sent to the application side by the API.
Alternatively, it is described that operational access request is forwarded to before the big data system, methods described
Also include：
Whether the request content analyzed in the operational access request includes sensitive data；
When the request content in operational access request includes sensitive data, according to the sensitive number pre-seted
According to access strategy, judge whether to let pass to operational access request；
When judging to let pass, operational access request is forwarded to the big data system；
When judging to let pass, operational access request is blocked.
Alternatively, described when receiving the operational access request, methods described also includes：
The operational access to big data system sent to the application side carries out log recording；
The content recorded in the daily record includes：Application side's information, the execution of operational access request
Information, the accessed big data system information, the operational access ask asked target resource letter
At least one of in the raw information of breath, the API information and operational access request.
Alternatively, it is described to detect whether to receive the behaviour to big data system that application side is sent by API
Make before access request, methods described also includes：
The registration request of the application side is obtained by the API；
According to the registration request, the authentication corresponding with the registration request is distributed to the application side
Mark and data manipulation access rights.
Alternatively, it is described that operational access request is forwarded to before the big data system, methods described
Also include：
Judge that the operational access is asked the number of threads used whether to reach and can carried in the big data system
Number of threads can be used in the maximum of confession；
When the operational access asks the number of threads used to be not up to the maximum usable number of threads,
Operational access request is forwarded to the big data system；
, will when the operational access asks the number of threads used to reach the maximum usable number of threads
The operational access request is put into the waiting list of request message forwarding.
Alternatively, it is described to visit the operation when operational access request is data write operation request
Ask that request is forwarded to before the big data system, methods described also includes：
Judge that the big data system asks whether the storage quota of distribution has been filled with for the operational access；
, will be described when the big data system is that the operational access asks the storage quota of distribution not to be filled with
Operational access request is forwarded to the big data system；
, will be described when the big data system is that the operational access asks the storage quota of distribution to be filled with
Operational access request is put into the waiting list of request message forwarding.
On the other hand, present invention also offers a kind of system for accessing big data system, including：
Detection module, sends for detecting whether receiving application side by application programming interface API
The operational access of big data system is asked；
Acquisition module, for when receiving the operational access request, obtaining the body that the application side is provided
Part authentication information；
Authentication module, it is right according to the authentication rule pre-seted for according to the authentication information
The application side carries out authentication detection；
First judge module, for when the application side meets authentication condition, judging that the operation is visited
Ask whether request is the authorization requests with authority；
First forwarding module, for when it is the authorization requests with authority that the operational access, which is asked, inciting somebody to action
The operational access request is forwarded to the big data system.
Alternatively, the system also includes：
Prompting module, for operating visit when the application side does not meet authentication condition, or, when described
When asking request for unauthorized request without authority, prompting is sent to the application side by the API
Message.
Alternatively, the system also includes：
Analysis module, for analyzing whether the request content in the operational access request includes sensitive data；
Second judge module, for when the operational access request in request content include sensitive data when,
According to the sensitive data access policy pre-seted, judge whether to let pass to operational access request；
Second forwarding module, for when judging to let pass, operational access request being forwarded to described
Big data system；
Module is blocked, for when judging to let pass, being blocked to operational access request.
Alternatively, the system also includes：
Logger module, the operational access to big data system for being sent to the application side carries out day
Will is recorded；
The content recorded in the daily record includes：Application side's information, the execution of operational access request
Information, the accessed big data system information, the operational access ask asked target resource letter
At least one of in the raw information of breath, the API information and operational access request.
Alternatively, the system also includes：
Registration request acquisition module, the registration request for obtaining the application side by the API；
Mark and authority distribution module, for according to the registration request, to the application side distribute with it is described
The corresponding identity identifier of registration request and data manipulation access rights.
Alternatively, the system also includes：
Whether the 3rd judge module, the number of threads used for judging the operational access to ask reaches described
Available maximum usable number of threads in big data system；
3rd forwarding module, for asking the number of threads used to be not up to the maximum when the operational access
When number of threads can be used, operational access request is forwarded to the big data system；
First waiting list adds module, for asking the number of threads used to reach institute when the operational access
When stating maximum usable number of threads, the operational access is asked to be put into the waiting list of request message forwarding.
Alternatively, the system also includes：
4th judge module, for judging that the big data system asks the storage of distribution for the operational access
Whether quota has been filled with；
4th forwarding module, for being that the operational access asks the storage of distribution to be matched somebody with somebody when the big data system
When volume is not filled with, operational access request is forwarded to the big data system；
Second waiting list adds module, for being that the operational access asks depositing for distribution when the big data system
When storage quota has been filled with, the operational access is asked to be put into the waiting list of request message forwarding.
The beneficial effects of the invention are as follows：
Such scheme, one Agent layer based on big data system of framework on the whole, using side to big number
According to all operations of system by being transmitted to actual big data system, the visit to big data system by Agent layer
The certification asked, authentication are all completed by Agent layer, and the main reality completed by the Agent layer to big data system
Border is operated, it is ensured that the safety of big data system, and being somebody's turn to do the big data system security protection process based on Agent layer will
Big data system is analyzed attendant etc. with applications side, operation system, big data and kept apart, and is formed
Access buffer area, had both been easy to be managed collectively the user of big data platform, and the big number of access is realized again
According to the mid-event control of platform, the security of stored data in big data platform is largely improved.
Brief description of the drawings
Fig. 1 represents schematic flow sheet in first embodiment of the invention；
Fig. 2 represents schematic flow sheet one in second embodiment of the invention；
Fig. 3 represents schematic flow sheet two in second embodiment of the invention；
Fig. 4 represents schematic flow sheet three in second embodiment of the invention；
Fig. 5 represents the function structure schematic diagram of agent platform in third embodiment of the invention；
Fig. 6 represents the technical pattern schematic diagram of agent platform in third embodiment of the invention；
Fig. 7 represents the schematic flow sheet one of agent platform service and interaction in third embodiment of the invention；
Fig. 8 represents the schematic flow sheet two of agent platform service and interaction in third embodiment of the invention；
Fig. 9 represents module schematic block diagram in fourth embodiment of the invention.
Embodiment
The exemplary embodiment of the disclosure is more fully described below with reference to accompanying drawings.Although being shown in accompanying drawing
The exemplary embodiment of the disclosure, it being understood, however, that may be realized in various forms the disclosure without should be by
Embodiments set forth here is limited.It is opposite to be able to be best understood from this there is provided these embodiments
It is open, and can by the scope of the present disclosure completely convey to those skilled in the art.
First embodiment
As shown in figure 1, the invention discloses a kind of method for accessing big data system, this method includes：
Step 101：Detect whether by application programming interface API receive that application side sends to big
The operational access request of data system.
Specifically, the application side is specifically as follows the big data system including user, system application etc.
The application of system.
Step 102：When receiving the operational access request, the identity for obtaining application side's offer is recognized
Demonstrate,prove information.
Step 103：According to the authentication information, according to the authentication rule pre-seted, to described
Authentication detection is carried out using side.
Step 104：When the application side meets authentication condition, judge that the operational access request is
No is the authorization requests with authority.
Step 105：When it is the authorization requests with authority that the operational access, which is asked, by the operation
Access request is forwarded to the big data system.
In said process, applications side is mainly received by API please to big data systematic difference
Ask, and the authentication information provided application side is detected, judges whether it meets authentication condition, then
The operational access request of application side to meeting authentication condition is made whether as the authorization requests with authority
Judgement, and control only the authorization requests with authority are forwarded to big data system, the process is from two
The potential dangerous service factor to big data system of filtering aspect reduction applications, it is ensured that big data system
The data safety of system, finally gives the data response of big data system.
This programme is existed by application programming interface API to receive including user, system application etc.
The operational access to big data system of interior application side is asked, and when the request of application side is sent, to it
Authenticated, to ensure that the application side, for valid application side, protects the data safety of big data system, and most
The authorization requests with authority for meeting the application side of authentication condition at last are forwarded to big data system, with
To the response of big data system, the program one Agent layer based on big data system of framework on the whole,
And then formed an agent platform, by the Agent layer complete the external agent service based on big data system, user,
The application sides such as application program are to all operations of big data system by being transmitted to actual big number by Agent layer
According to system, Agent layer judges whether the operational access request of application side has been authorized to before forwarding is performed, only
In the case where being authorized to, could to user, using etc. the authorization requests sent of application side handle it,
Can be specifically to realize that application side accesses uniformly recognizing for big data platform based on network authenticating protocol Kerberos
Card, single-sign-on.The Agent layer is user, using the only way which must be passed for accessing big data system, and access is recognized
Card, authentication are all completed by Agent layer, and the main practical operation completed by the Agent layer to big data system,
Ensure the safety of big data system, should the big data system security protection process based on Agent layer and agent platform
Big data system is analyzed into attendant etc. with applications side, operation system, big data to keep apart, shape
Into access buffer area, both it had been easy to be managed collectively the user of big data system platform, visit is realized again
The mid-event control of big data system platform is asked, stored data in big data system platform is largely improved
Security.
Second embodiment
To make technical scheme more perfect, the present embodiment will be on the basis of first embodiment, to first
The preferred embodiment that the implementation process of the method for big data system is accessed in embodiment is described.
Here, when application side does not meet authentication condition, or, when operational access request be without
During the unauthorized request of authority, reminder message to application side can also be sent by the API.
Further, as shown in Fig. 2 based on first embodiment, turning operational access request in step 104
It is sent to before big data system, this method is also included in the request during the operational access sent to application side is asked
Appearance is judged that specific steps include：
Step 201：Whether the request content analyzed in the operational access request includes sensitive data.
Specifically, the sensitive data can be by big data system manager carry out presetting data content and
Data area.
Step 202：When the request content in operational access request includes sensitive data, according to default
The sensitive data access policy put, judges whether to let pass to operational access request.
Step 203：When judging to let pass, operational access request is forwarded to the big data system
System.
Step 204：When judging to let pass, operational access request is blocked.
Said process, by carrying out authentication and the operational access sent to application side request to application side
While authentication, Agent layer also is made whether to include the analysis of sensitive data to the request content of application side,
It is real if comprising sensitive data, request will be blocked or be let pass by i.e. fixed sensitive data access policy
Now to the protection of the Sensitive data content in big data system.Specifically, application side's operational access can be based on
The access region of request, time are controlled to operational access request, or to sensitive data identification, classification,
Realize that data desensitize, can be combined with command set control, national treasury pattern and realize to quick in big data system
Feel the security protection of data.
Further,, should in step 102 when receiving operational access request based on first embodiment
Method also includes：The operational access to big data system sent to application side carries out log recording.
Specifically, the content recorded in daily record includes：What application side's information, the operational access were asked
Execution information, the accessed big data system information, the operational access ask asked target money
At least one of in the raw information of source information, the API information and operational access request.
Here, the request of all API through Agent layer to Agent layers all should be recorded that more detailed logging.Daily record
The content included：User and program information that the request sent application side is related to are specifically included using square information
(account number, role, the IP of program and port etc.)；Accessed big data system information specifically includes interviewed
Ask the information such as systematic name, type, IP and the port of big data system；What operational access was asked performs letter
Breath specifically include operational access request that this is sent by application side to operating time of big data system, operate pass
The information such as key word, corresponding authority, operating result, failure cause；Operational access asks asked target money
Source information specifically includes the information such as resource name and the configured attribute of resource, specifically, the configured attribute
At most fill in 5 underlying attributes；Operational access request raw information specifically include request time, URL,
The information such as request method, required parameter, response time；API information is specifically included：Type, the API
The information such as domain name, IP and the port numbers at place.Correspondingly, this can also be recorded based on big data system
The title of agent platform and other associated detailed informations.
The program records user, using the operation for accessing big data system, daily record in detail by agent skill group
Content includes：Account number, role, User IP, program IP and port, big data systematic name, type,
IP and port, operation keyword, operating result, target resource name etc., meet 5W1H big number
Access record in detail according to system, be easy to carry out to user, using all operation behaviors for accessing big number system evidence
Control analysis and concentrate audit.
Further, based on first embodiment, detect whether to receive application by API in step 101
Before the operational access request to big data system that side is sent, this method also includes：Pass through the API
Obtain the registration request of the application side；According to the registration request, to application side distribution and the note
The corresponding identity identifier of volume request and data manipulation access rights.
The application sides such as institute's access big data systematic difference system in need are both needed to register in this agent platform,
And identified by this platform for certifications such as its corresponding program account numbers of distribution, authorize corresponding data permission, when
Agent platform can carry out authentication and authentication after receiving the access request of application to it, be applied after authenticating successfully
Big data system can be accessed just now, to realize that agent platform accesses application side the account used in big data platform
Number wait identity identifier managed concentratedly.Specific data manipulation access rights are authorized, it is possible to achieve
Become more meticulous mandate, to user, using catalogue, file, storehouse, the table progress fine granularity for accessing big data platform
Authorize, and judgement is identified to the operational access request of application side, be with power in asking operational access
The authorization requests of limit are responded, the security of the big data that adequately protects system.
Further, available big data when agent platform limits the API of each routine call Agent layer
The storage size that system resource, such as Thread Count and data take, the API provided when routine call Agent layer
When, agent platform can judge the number of threads currently provided as this program and the maximum workable thread of this program
Quantity.
The embodiment in the case of this kind is deployed herein to describe, based on first embodiment, on the one hand, such as Fig. 3
Shown, before operational access request is forwarded into big data system, this method also includes：
Step 301：Judge whether the number of threads that the operational access request is used reaches the big data system
Available maximum usable number of threads in system.
Step 302：When the operational access asks the number of threads used to be not up to the maximum usable line
During number of passes amount, operational access request is forwarded to the big data system.
Step 303：When the operational access asks the number of threads used to reach the maximum usable thread
During quantity, the operational access is asked to be put into the waiting list of request message forwarding.
In practical application, following several situations are primarily present：
The first situation：If the maximum that the currently used number of threads of application program is not up to setting can be used
Number of threads, then agent platform directly can provide service for this program.For example：A certain application program currently makes
Thread Count is 15, and maximum available thread number is 20, then agent platform will judge current system
There is idling-resource in system, exist for condition and ability that application program provides service.
Second of situation：If the currently used number of threads of application program has reached that the maximum of setting can be used
Number of threads, then agent platform this request can be put into waiting list, until have available thread release after locate again
Manage this request.For example：The currently used Thread Count of a certain application program is 25, and maximum available thread number
Can not be temporarily to apply journey for 20, then agent platform will judge current system resource saturation already
Sequence provides service, have to wait until obtaining enough resources.
On the other hand, as shown in figure 4, when operational access request is data write operation request, will operate
Access request is forwarded to before big data system, and this method also includes：
Step 401：Judge the big data system for the operational access ask distribution storage quota whether
It has been filled with.
Step 402：When the big data system is that the operational access asks the storage quota of distribution not to be filled with
When, operational access request is forwarded to the big data system.
Step 403：When the big data system is that the operational access asks the storage quota of distribution to be filled with
When, the operational access is asked to be put into the waiting list of request message forwarding.
In practical application, when application program writes data into big data system, agent platform can be sentenced in advance
It is whether the storage quota that this program is distributed in advance is full in disconnected big data system, if storage is full, then
Mistake, which is there may exist, when writing by force writes possibility, so when agent platform judges storage completely, that
Application program will enter waiting list, and write operation is carried out again in the case of the memory space for possessing abundance.
The side of the access big data system that can protect big data system safety based on Agent layer in the application
Method should cover the interface of big data system data comprehensively as far as possible.That is, the API that Agent layer is provided
All API that big data system is provided in itself should be covered, in order to carry out overall safety to big data system
Protection.The present invention acts on behalf of all kinds of big data system operatios by " protocol level ", in the premise that agreement is constant,
It is simple using access, it is not necessary to change programmatic agent suitable for most big data systems.
The method of above-mentioned access big data system, security protection is carried out based on Agent layer to big data system,
4A range of managements are included using big data system as a kind of new system resource, the account of big data platform is realized
Number management centralization, empowerment management become more meticulous, access registrar unitize and operated centralization of auditing, and compensate for
The defect that 4A forts system can not be protected user by API Access big data system, can be engaged in
Preceding prevention, mid-event control, the security protection of three aspect composition big data systems of post-audit, and to sensitivity
Data carry out emphasis management and control, effectively realize the security protection and supervision to big data system.
3rd embodiment
Specifically, in this embodiment, by the concrete composition of the agent platform constituted in the first two embodiment
Structure carries out expansion description.
As shown in Fig. 5, Fig. 6, Fig. 7, Fig. 8, framework one is based on big data to the program on the whole
The Agent layer of system, the application interface API sides of being employed of the Agent layer are called, in the API of Agent layer
After the operational access request for receiving application side, the function of agent platform include to the progress authentication of application side,
Access control, data security protecting, operation note etc.；Certification to application side can specifically pass through authentication center
Carry out, be specially to realize that application side accesses the unification of big data platform based on network authenticating protocol Kerberos
Certification, single-sign-on.Keeper, analyst, the maintenance person of big data system can be real by agent platform
Now the account management of big data system, empowerment management, operation are audited, agent platform is managed and supervised
Control etc..
The big data system security protection process based on Agent layer is by big data system and applications side, industry
Business system, big data analysis attendant etc. keep apart, and form access buffer area, have both been easy to big data
The user of platform is managed collectively, and the mid-event control for accessing big data platform, high degree are realized again
On improve the security of stored data in big data platform.
Further, the application programming interface API that the Agent layer is provided can be provided by Agent layer
The need for safety certification RESTful interfaces.As user or these RESTful interfaces of routine access,
Agent layer needs the application sides such as user or application program to provide authentication information, and controls only to allow using side
The operation requests authorized in operational access request are completed, the operation authorized in being asked for operational access please
Ask progress parsing to be transmitted to big data system, big data system is packaged to its response and passes to application
Side.The user of all non-certified, the access of program and any uncommitted operation will be refused all
Absolutely, to ensure by the safety of pipe big data system.
In said process, the corresponding application side of agent platform and big data system can be multiple, be answered when receiving
When being asked with side the operational access of big data system, the API of agent platform can receive multiple big datas
Access program, the plurality of big data access program may same application side send, it is also possible to be that difference is answered
Sent with side, agent platform can be voluntarily to accessing program and multiple big in multiple application sides, multiple big datas
The operational access carried out between data system is controlled and coordinated；And it is above-mentioned by operational access request be forwarded to
Big data system can be specifically to ask operational access to be forwarded to the API of big data system, by big number
Called according to the API of system, the final operational access for realizing application side asks to reach big data system,
And obtain the response of big data system.
Fourth embodiment
As shown in figure 9, the invention also discloses a kind of system for accessing big data system, including：Detect mould
Block 501, acquisition module 502, authentication module 503, the first judge module 504, the first forwarding module 505.
Wherein, detection module 501, should for detecting whether being received by application programming interface API
The operational access to big data system sent with side is asked；Acquisition module 502, for described when receiving
When operational access is asked, the authentication information that the application side is provided is obtained；Authentication module 503, is used for
According to the authentication information, according to the authentication rule pre-seted, identity is carried out to the application side
Authentication checks；First judge module 504, for when the application side meets authentication condition, judging
Whether the operational access request is the authorization requests with authority；First forwarding module 505, for working as
When the operational access request is the authorization requests with authority, operational access request is forwarded to institute
State big data system.
The program one Agent layer based on big data system of framework on the whole, user, application program etc.
Using side to all operations of big data system by being transmitted to actual big data system, the generation by Agent layer
Reason layer is user, using the only way which must be passed for accessing big data system, the certification of access, is authenticated all by Agent layer
To complete, and the main practical operation completed by the Agent layer to big data system, it is ensured that big data system
Safety, should big data system security protection process based on Agent layer by big data system and applications side,
Operation system, big data analysis attendant etc. keep apart, and form access buffer area, have both been easy to big number
It is managed collectively according to the user of platform, the mid-event control for accessing big data platform, very big journey is realized again
The security of stored data in big data platform is improved on degree.
Correspondingly, the system also includes：Prompting module.
The prompting module is used for when the application side does not meet authentication condition, or, when the operation is visited
When asking request for unauthorized request without authority, prompting is sent to the application side by the API
Message.
Further, the system also includes：Analysis module, the second judge module, the second forwarding module, resistance
Disconnected module.
Wherein, analysis module, for whether analyzing the request content in the operational access request comprising sensitivity
Data；Second judge module, for when the operational access request in request content include sensitive data when,
According to the sensitive data access policy pre-seted, judge whether to let pass to operational access request；
Second forwarding module, for when judging to let pass, operational access request to be forwarded into the big number
According to system；Module is blocked, for when judging to let pass, being blocked to operational access request.
Wherein, the system also includes：Logger module.
Logger module, the operational access to big data system for being sent to the application side carries out day
Will is recorded；The content recorded in the daily record includes：What application side's information, the operational access were asked holds
Row information, the accessed big data system information, the operational access ask asked target resource
At least one of in the raw information of information, the API information and operational access request.
Further, the system also includes：Registration request acquisition module, mark and authority distribution module.
Wherein, registration request acquisition module, the registration for obtaining the application side by the API please
Ask；Mark and authority distribution module, for according to the registration request, to the application side distribute with it is described
The corresponding identity identifier of registration request and data manipulation access rights.
The application sides such as institute's access big data systematic difference system in need are both needed to register in this agent platform,
And identified by this platform for certifications such as its corresponding program account numbers of distribution, authorize corresponding data permission, it is real
Modern platform accesses application side the identity identifiers such as the account number used in big data platform and carries out concentrate tube
Reason.Specific data manipulation access rights are authorized, it is possible to achieve become more meticulous mandate, and adequately protect big data
The security of system.
Alternatively, the system also includes：3rd judge module, the 3rd forwarding module, the first waiting list add
Enter module.
Wherein, the 3rd judge module, for judging whether the number of threads that the operational access request is used reaches
The available maximum usable number of threads into the big data system；3rd forwarding module, for working as
When stating the number of threads that uses of operational access request and being not up to the maximum usable number of threads, by the behaviour
The big data system is forwarded to as access request；First waiting list adds module, for when the operation
, please by the operational access when number of threads that access request is used reaches the maximum usable number of threads
Seek the waiting list for being put into request message forwarding.
Alternatively, the system also includes：4th judge module, the 4th forwarding module.
Wherein, the 4th judge module, for judging that the big data system asks to distribute for the operational access
Storage quota whether be filled with；4th forwarding module, for being visited when the big data system for the operation
When asking that the storage quota of request distribution is not filled with, operational access request is forwarded to the big data system；
Second waiting list adds module, for being that the operational access asks depositing for distribution when the big data system
When storage quota has been filled with, the operational access is asked to be put into the waiting list of request message forwarding.
The method of above-mentioned access big data system, security protection is carried out based on Agent layer to big data system,
4A range of managements are included using big data system as a kind of new system resource, the account of big data platform is realized
Number management centralization, empowerment management become more meticulous, access registrar unitize and operated centralization of auditing, and compensate for
The defect that 4A forts system can not be protected user by API Access big data system, can be engaged in
Preceding prevention, mid-event control, the security protection of three aspect composition big data systems of post-audit, and to sensitivity
Data carry out emphasis management and control, effectively realize the security protection and supervision to big data system.
Each embodiment in this specification is described by the way of progressive, what each embodiment was stressed
All be between the difference with other embodiment, each embodiment identical similar part mutually referring to.
Although having been described for the preferred embodiment of the embodiment of the present invention, those skilled in the art once obtain
Cicada basic creative concept, then can make other change and modification to these embodiments.So, it is appended
Claim is intended to be construed to include preferred embodiment and falls into being had altered for range of embodiment of the invention
And modification.
Finally, in addition it is also necessary to explanation, herein, such as first and second or the like relational terms are only
Only be used for by an entity or operation with another entity or operate make a distinction, and not necessarily require or
Imply between these entities or operation there is any this actual relation or order.Moreover, term " bag
Include ", "comprising" or any other variant thereof is intended to cover non-exclusive inclusion so that including one
Process, method, article or the terminal device of list of elements not only include those key elements, but also including not
There are other key elements being expressly recited, or also include being this process, method, article or terminal device
Intrinsic key element.In the absence of more restrictions, by wanting that sentence "including a ..." is limited
Element, it is not excluded that also exist in addition in the process including the key element, method, article or terminal device
Identical element.
Above-described is the preferred embodiment of the present invention, it should be pointed out that for the ordinary people of the art
For member, some improvements and modifications can also be made under the premise of principle of the present invention is not departed from, these
Improvements and modifications are also within the scope of the present invention.
Claims (14)
1. a kind of method for accessing big data system, it is characterised in that including：
Detect whether by application programming interface API receive that application side sends to big data system
Operational access request；
When receiving the operational access request, the authentication information that the application side is provided is obtained；
According to the authentication information, according to the authentication rule pre-seted, the application side is carried out
Authentication is detected；
When the application side meets authentication condition, judge whether the operational access request is with power
The authorization requests of limit；
When it is the authorization requests with authority that the operational access, which is asked, operational access request is turned
It is sent to the big data system.
2. according to the method described in claim 1, it is characterised in that methods described also includes：
When the application side does not meet authentication condition, or, when the operational access request be without
During the unauthorized request of authority, reminder message is sent to the application side by the API.
3. according to the method described in claim 1, it is characterised in that described to ask the operational access
It is forwarded to before the big data system, methods described also includes：
Whether the request content analyzed in the operational access request includes sensitive data；
When the request content in operational access request includes sensitive data, according to the sensitive number pre-seted
According to access strategy, judge whether to let pass to operational access request；
When judging to let pass, operational access request is forwarded to the big data system；
When judging to let pass, operational access request is blocked.
4. according to the method described in claim 1, it is characterised in that described to receive the operation visit
When asking request, methods described also includes：
The operational access to big data system sent to the application side carries out log recording；
The content recorded in the daily record includes：Application side's information, the execution of operational access request
Information, the accessed big data system information, the operational access ask asked target resource letter
At least one of in the raw information of breath, the API information and operational access request.
5. according to the method described in claim 1, it is characterised in that described to detect whether to connect by API
Receive before the operational access request to big data system that application side is sent, methods described also includes：
The registration request of the application side is obtained by the API；
According to the registration request, the authentication corresponding with the registration request is distributed to the application side
Mark and data manipulation access rights.
6. according to the method described in claim 1, it is characterised in that described to ask the operational access
It is forwarded to before the big data system, methods described also includes：
Judge that the operational access is asked the number of threads used whether to reach and can carried in the big data system
Number of threads can be used in the maximum of confession；
When the operational access asks the number of threads used to be not up to the maximum usable number of threads,
Operational access request is forwarded to the big data system；
, will when the operational access asks the number of threads used to reach the maximum usable number of threads
The operational access request is put into the waiting list of request message forwarding.
7. according to the method described in claim 1, it is characterised in that when operational access request is number
It is described that operational access request is forwarded to before the big data system during according to write operation request, institute
Stating method also includes：
Judge that the big data system asks whether the storage quota of distribution has been filled with for the operational access；
, will be described when the big data system is that the operational access asks the storage quota of distribution not to be filled with
Operational access request is forwarded to the big data system；
, will be described when the big data system is that the operational access asks the storage quota of distribution to be filled with
Operational access request is put into the waiting list of request message forwarding.
8. a kind of system for accessing big data system, it is characterised in that including：
Detection module, sends for detecting whether receiving application side by application programming interface API
The operational access of big data system is asked；
Acquisition module, for when receiving the operational access request, obtaining the body that the application side is provided
Part authentication information；
Authentication module, it is right according to the authentication rule pre-seted for according to the authentication information
The application side carries out authentication detection；
First judge module, for when the application side meets authentication condition, judging that the operation is visited
Ask whether request is the authorization requests with authority；
First forwarding module, for when it is the authorization requests with authority that the operational access, which is asked, inciting somebody to action
The operational access request is forwarded to the big data system.
9. system according to claim 8, it is characterised in that the system also includes：
Prompting module, for operating visit when the application side does not meet authentication condition, or, when described
When asking request for unauthorized request without authority, prompting is sent to the application side by the API
Message.
10. system according to claim 8, it is characterised in that the system also includes：
Analysis module, for analyzing whether the request content in the operational access request includes sensitive data；
Second judge module, for when the operational access request in request content include sensitive data when,
According to the sensitive data access policy pre-seted, judge whether to let pass to operational access request；
Second forwarding module, for when judging to let pass, operational access request being forwarded to described
Big data system；
Module is blocked, for when judging to let pass, being blocked to operational access request.
11. system according to claim 8, it is characterised in that the system also includes：
Logger module, the operational access to big data system for being sent to the application side carries out day
Will is recorded；
The content recorded in the daily record includes：Application side's information, the execution of operational access request
Information, the accessed big data system information, the operational access ask asked target resource letter
At least one of in the raw information of breath, the API information and operational access request.
12. system according to claim 8, it is characterised in that the system also includes：
Registration request acquisition module, the registration request for obtaining the application side by the API；
Mark and authority distribution module, for according to the registration request, to the application side distribute with it is described
The corresponding identity identifier of registration request and data manipulation access rights.
13. system according to claim 8, it is characterised in that the system also includes：
Whether the 3rd judge module, the number of threads used for judging the operational access to ask reaches described
Available maximum usable number of threads in big data system；
3rd forwarding module, for asking the number of threads used to be not up to the maximum when the operational access
When number of threads can be used, operational access request is forwarded to the big data system；
First waiting list adds module, for asking the number of threads used to reach institute when the operational access
When stating maximum usable number of threads, the operational access is asked to be put into the waiting list of request message forwarding.
14. system according to claim 8, it is characterised in that the system also includes：
4th judge module, for judging that the big data system asks the storage of distribution for the operational access
Whether quota has been filled with；
4th forwarding module, for being that the operational access asks the storage of distribution to be matched somebody with somebody when the big data system
When volume is not filled with, operational access request is forwarded to the big data system；
Second waiting list adds module, for being that the operational access asks distribution when the big data system
Storage quota when being filled with, the operational access is asked to be put into the waiting list of request message forwarding.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201610073085.5A CN107026825A (en) | 2016-02-02 | 2016-02-02 | A kind of method and system for accessing big data system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201610073085.5A CN107026825A (en) | 2016-02-02 | 2016-02-02 | A kind of method and system for accessing big data system |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN107026825A true CN107026825A (en) | 2017-08-08 |
Family
ID=59524992
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201610073085.5A Pending CN107026825A (en) | 2016-02-02 | 2016-02-02 | A kind of method and system for accessing big data system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN107026825A (en) |
Cited By (13)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107657182A (en) * | 2017-10-18 | 2018-02-02 | 成都索贝数码科技股份有限公司 | A kind of method for strengthening media data control of authority reliability |
| CN107707573A (en) * | 2017-11-22 | 2018-02-16 | 用友金融信息技术股份有限公司 | Data access method and its device and computer installation and its readable storage medium storing program for executing |
| CN108959625A (en) * | 2018-07-23 | 2018-12-07 | 郑州云海信息技术有限公司 | The acquisition methods and device of information in cloud data system |
| CN109308410A (en) * | 2018-10-16 | 2019-02-05 | 翟红鹰 | Obtain method, system and the computer readable storage medium of block chain data |
| CN109977690A (en) * | 2017-12-28 | 2019-07-05 | 中国移动通信集团陕西有限公司 | A kind of data processing method, device and medium |
| CN110022280A (en) * | 2018-01-08 | 2019-07-16 | 中国移动通信有限公司研究院 | A kind of watermark information processing method, device and computer storage medium |
| CN110392062A (en) * | 2019-08-06 | 2019-10-29 | 深圳萨摩耶互联网金融服务有限公司 | A kind of multidimensional encryption method and device based on big data |
| CN110839027A (en) * | 2019-11-14 | 2020-02-25 | 北京京东尚科信息技术有限公司 | User authentication method, device, proxy server and network service system |
| CN111737723A (en) * | 2020-08-25 | 2020-10-02 | 杭州海康威视数字技术股份有限公司 | Service processing method, device and equipment |
| CN111800509A (en) * | 2020-07-07 | 2020-10-20 | 北京尚隐科技有限公司 | Personal information access request system and method for applying same |
| CN112115484A (en) * | 2020-09-27 | 2020-12-22 | 中国工商银行股份有限公司 | Access control method, device, system and medium for application program |
| CN113938307A (en) * | 2021-10-21 | 2022-01-14 | 中国联合网络通信集团有限公司 | Information collection method and system |
| CN117978471A (en) * | 2024-01-18 | 2024-05-03 | 南方电网数字电网集团信息通信科技有限公司 | Unauthorized access detection method, device, equipment and storage medium for access request |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101562621A (en) * | 2009-05-25 | 2009-10-21 | 阿里巴巴集团控股有限公司 | User authorization method and system and device thereof |
| CN102413464A (en) * | 2011-11-24 | 2012-04-11 | 杭州东信北邮信息技术有限公司 | GBA (General Bootstrapping Architecture)-based secret key negotiation system and method of telecommunication capability open platform |
| US20120324225A1 (en) * | 2011-06-20 | 2012-12-20 | Jason Chambers | Certificate-based mutual authentication for data security |
| CN103839138A (en) * | 2014-03-08 | 2014-06-04 | 成都文昊科技有限公司 | System for supporting interaction of multiple heterogeneous systems |
-
2016
- 2016-02-02 CN CN201610073085.5A patent/CN107026825A/en active Pending
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101562621A (en) * | 2009-05-25 | 2009-10-21 | 阿里巴巴集团控股有限公司 | User authorization method and system and device thereof |
| US20120324225A1 (en) * | 2011-06-20 | 2012-12-20 | Jason Chambers | Certificate-based mutual authentication for data security |
| CN102413464A (en) * | 2011-11-24 | 2012-04-11 | 杭州东信北邮信息技术有限公司 | GBA (General Bootstrapping Architecture)-based secret key negotiation system and method of telecommunication capability open platform |
| CN103839138A (en) * | 2014-03-08 | 2014-06-04 | 成都文昊科技有限公司 | System for supporting interaction of multiple heterogeneous systems |
Non-Patent Citations (1)
| Title |
|---|
| 杨婧媛: "《统一用户信息管理服务框架的设计与实现》", 《中国优秀硕士论文全文数据库信息科技辑》 * |
Cited By (17)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107657182A (en) * | 2017-10-18 | 2018-02-02 | 成都索贝数码科技股份有限公司 | A kind of method for strengthening media data control of authority reliability |
| CN107657182B (en) * | 2017-10-18 | 2020-12-01 | 成都索贝数码科技股份有限公司 | Method for enhancing reliability of media data authority control |
| CN107707573A (en) * | 2017-11-22 | 2018-02-16 | 用友金融信息技术股份有限公司 | Data access method and its device and computer installation and its readable storage medium storing program for executing |
| CN109977690A (en) * | 2017-12-28 | 2019-07-05 | 中国移动通信集团陕西有限公司 | A kind of data processing method, device and medium |
| CN110022280A (en) * | 2018-01-08 | 2019-07-16 | 中国移动通信有限公司研究院 | A kind of watermark information processing method, device and computer storage medium |
| CN108959625A (en) * | 2018-07-23 | 2018-12-07 | 郑州云海信息技术有限公司 | The acquisition methods and device of information in cloud data system |
| CN109308410A (en) * | 2018-10-16 | 2019-02-05 | 翟红鹰 | Obtain method, system and the computer readable storage medium of block chain data |
| CN110392062A (en) * | 2019-08-06 | 2019-10-29 | 深圳萨摩耶互联网金融服务有限公司 | A kind of multidimensional encryption method and device based on big data |
| CN110839027B (en) * | 2019-11-14 | 2023-03-07 | 北京京东尚科信息技术有限公司 | User authentication method, device, proxy server and network service system |
| CN110839027A (en) * | 2019-11-14 | 2020-02-25 | 北京京东尚科信息技术有限公司 | User authentication method, device, proxy server and network service system |
| CN111800509A (en) * | 2020-07-07 | 2020-10-20 | 北京尚隐科技有限公司 | Personal information access request system and method for applying same |
| CN111737723A (en) * | 2020-08-25 | 2020-10-02 | 杭州海康威视数字技术股份有限公司 | Service processing method, device and equipment |
| CN112115484A (en) * | 2020-09-27 | 2020-12-22 | 中国工商银行股份有限公司 | Access control method, device, system and medium for application program |
| CN112115484B (en) * | 2020-09-27 | 2023-11-21 | 中国工商银行股份有限公司 | Access control method, device, system and medium for application program |
| CN113938307A (en) * | 2021-10-21 | 2022-01-14 | 中国联合网络通信集团有限公司 | Information collection method and system |
| CN113938307B (en) * | 2021-10-21 | 2023-07-14 | 中国联合网络通信集团有限公司 | Information collection method and system |
| CN117978471A (en) * | 2024-01-18 | 2024-05-03 | 南方电网数字电网集团信息通信科技有限公司 | Unauthorized access detection method, device, equipment and storage medium for access request |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN107026825A (en) | A kind of method and system for accessing big data system | |
| AU2019206006B2 (en) | System and method for biometric protocol standards | |
| CN107342992A (en) | A kind of System right management method, apparatus and computer-readable recording medium | |
| CN101286845B (en) | Control system for access between domains based on roles | |
| CN104735091B (en) | A kind of user access control method and apparatus based on linux system | |
| CN106411857B (en) | A kind of private clound GIS service access control method based on virtual isolation mech isolation test | |
| CN109670768A (en) | Right management method, device, platform and the readable storage medium storing program for executing in multi-service domain | |
| US7779248B2 (en) | Moving principals across security boundaries without service interruption | |
| CN110197058A (en) | Unified internal control method for managing security, system, medium and electronic equipment | |
| CN101375288A (en) | Extensible role based authorization for manageable resources | |
| CN107204978B (en) | A kind of access control method and device based on multi-tenant cloud environment | |
| CN102546664A (en) | User and authority management method and system for distributed file system | |
| CN107426152B (en) | Multitask security isolation system and method under cloud platform actual situation Interconnection Environment | |
| CN104794374B (en) | A kind of application rights management method and apparatus for Android system | |
| Dan et al. | Attribute based access control (ABAC)-based cross-domain access control in service-oriented architecture (SOA) | |
| CN110417820A (en) | Processing method, device and the readable storage medium storing program for executing of single-node login system | |
| CN107770192A (en) | Identity authentication method and computer-readable recording medium in multisystem | |
| JP2005234729A (en) | Unauthorized access protection system and its method | |
| CN108696540A (en) | A kind of authorizing secure system and its authorization method | |
| CN102571874A (en) | On-line audit method and device in distributed system | |
| JP2018013875A (en) | Access management system | |
| US20220060463A1 (en) | Method for managing network devices, apparatus, and computer readable storage medium | |
| CN108366068A (en) | Cloud network resource management control system based on policy language under a kind of software defined network | |
| Chadwick et al. | Multi-session separation of duties (MSoD) for RBAC | |
| Purba et al. | Assessing Privileged Access Management (PAM) using ISO 27001: 2013 Control |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20170808 |