CN107196951B - A kind of implementation method and firewall system of HDFS system firewall - Google Patents
A kind of implementation method and firewall system of HDFS system firewall Download PDFInfo
- Publication number
- CN107196951B CN107196951B CN201710439355.4A CN201710439355A CN107196951B CN 107196951 B CN107196951 B CN 107196951B CN 201710439355 A CN201710439355 A CN 201710439355A CN 107196951 B CN107196951 B CN 107196951B
- Authority
- CN
- China
- Prior art keywords
- metadata
- access request
- authentication
- server
- metadata access
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L63/00—Network architectures or network communication protocols for network security
- H04L63/02—Network architectures or network communication protocols for network security for separating internal from external traffic, e.g. firewalls
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F21/00—Security arrangements for protecting computers, components thereof, programs or data against unauthorised activity
- G06F21/60—Protecting data
- G06F21/62—Protecting access to data via a platform, e.g. using keys or access control rules
- G06F21/6218—Protecting access to data via a platform, e.g. using keys or access control rules to a system of files or objects, e.g. local or distributed file system or database
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L63/00—Network architectures or network communication protocols for network security
- H04L63/08—Network architectures or network communication protocols for network security for authentication of entities
Abstract
The present invention provides the implementation methods and firewall system of a kind of HDFS system firewall, HDFS system firewall is arranged on the channel between application server and NameNode server, this method comprises: receiving the metadata access request that the expression from application server carries out metadata access to NameNode server；Default fine granularity purview certification is carried out to metadata access request, obtains authentication result；When default fine granularity purview certification passes through, metadata access request is sent to NameNode server；Receive the metadata information for metadata access request from NameNode server；Metadata information is back to application server；When default fine granularity purview certification does not pass through, it will indicate that authenticating unsanctioned error message is back to application server.The present invention is by carrying out fine granularity scope check to client access request, being able to ascend the security protection ability of HDFS system, and shield cluster external attack in metadata information of the client to NameNode server demand file.
Description
Technical field
The present invention relates to the security technology areas of distributed file system, more particularly to a kind of HDFS system firewall
Implementation method and firewall system.
Background technique
In recent years, popularizing with big data application, (wherein, Hadoop is one by Apache fund to Hadoop system
The distributed system infrastructure of club's exploitation) and answered extensively using the ecosystem that Hadoop is constructed as Floor layer Technology
With having become big data and handle actual technology platform synonym.HDFS (Hadoop distributed file system) is used as one
A take Hadoop system as the distributed file system of basic framework, is the bottom file storage of the databases such as HBase, Hive
System.Meanwhile most of storage tools in Hadoop ecology all support the data of HDFS to store.Therefore, HDFS is big data
Foundation stone in processing technique foundation stone.
Wherein, HDFS: client/server is used, by NameNode (namenode) and DataNode (back end) two parts
It constitutes.Wherein, NameNode is host node, for the metadata information of storage file, can be one or more；DataNode
It is from node, for storing actual file block, quantity is up to thousands of.
The security protection of HDFS system is the foundation stone of Hadoop ecological safety protection.Currently, the Hadoop version of mainstream mentions
For the weak HDFS permission control based on operating system grade；Third parties' component such as Ranger is capable of providing to multiple groups of Hadoop ecology
The uniform permission administration of part；Knox provides the other access agent of Rest api class.But what primary Hadoop and Ranger was provided
Permission control is all integrated in NameNode process.That is, the deployment of the firewall of traditional HDFS system needs to be embedded in
To the cluster internal of HDFS system, moreover, cluster, which must restart, just can be carried out security protection, to influence after the completion of deployment
The operation of original cluster, moreover, the attack of external malicious user can not be shielded；And the safety that Knox then only provides http protocol is anti-
Shield and only the seeervice level authority managing and controlling of offer coarseness, can not effectively resist external attack.
It can be seen that the security protection ability of the conventional security protectiving scheme of HDFS system is weaker, day not can effectively solve
Beneficial severe big data security protection problem.
Summary of the invention
The present invention provides the implementation methods and firewall system of a kind of HDFS system firewall, to solve HDFS system
The problem of security protection ability present in conventional security protectiving scheme is weak, can not shield cluster external attack.
To solve the above-mentioned problems, according to an aspect of the present invention, the invention discloses a kind of HDFS system firewalls
Implementation method, the HDFS system includes NameNode server, and the HDFS system firewall is arranged in application server
On channel between NameNode server, which comprises
Receive first number that the expression from the application server carries out metadata access to the NameNode server
According to access request；
Default fine granularity purview certification is carried out to metadata access request, obtains authentication result；
When the default fine granularity purview certification passes through, metadata access request is sent to the NameNode
Server；
Receive the metadata information for metadata access request from the NameNode server；
The metadata information is back to the application server；
When the default fine granularity purview certification does not pass through, it is described will to indicate that the unsanctioned error message of certification is back to
Application server.
According to another aspect of the present invention, the invention also discloses a kind of firewall system of HDFS system, the HDFS
System includes NameNode server, and the firewall system is arranged between application server and the NameNode server
Channel on, the firewall system includes:
First receiving module is carried out for receiving the expression from the application server to the NameNode server
The metadata access of metadata access is requested；
Authentication module obtains authentication result for carrying out default fine granularity purview certification to metadata access request；
Sending module, for when the default fine granularity purview certification passes through, the metadata access being requested to send
To the NameNode server；
Second receiving module is directed to what the metadata access was requested from the NameNode server for receiving
Metadata information；
First return module, for the metadata information to be back to the application server；
Second return module, for when the default fine granularity purview certification does not pass through, will indicate that certification is unsanctioned
Error message is back to the application server.
Compared with prior art, the present invention includes the following advantages:
The present invention is by accessing client in metadata information of the client to NameNode server demand file
Request carries out fine granularity scope check and is completely forwarded to client access request if meeting the preset rules of competence
NameNode server does not forward client request, but returns to error message if do not met.So that user and HDFS
Interaction between server-side must could be accessed or be managed to HDFS server-side by firewall.Improve HDFS system
Security protection ability can shield cluster external attack.
Detailed description of the invention
Fig. 1 is a kind of structural block diagram of HDFS system embodiment of the invention；
Fig. 2 is a kind of step flow chart of the implementation method embodiment of HDFS system firewall of the invention；
Fig. 3 is a kind of logic diagram of HDFS system embodiment of the invention；
Fig. 4 is a kind of physics deployment diagram of HDFS system embodiment of the invention；
Fig. 5 is a kind of structural block diagram of the firewall system embodiment of HDFS system of the invention.
Specific embodiment
In order to make the foregoing objectives, features and advantages of the present invention clearer and more comprehensible, with reference to the accompanying drawing and specific real
Applying mode, the present invention is described in further detail.
Referring to Fig.1, a kind of structural block diagram of HDFS system of the invention is shown.
The HDFS system of the embodiment of the present invention includes client and server-side, wherein Client is deployed in client
API (Client Application Programming Interface, client applications programming interface), Client API
Including file access interface and NameNode client and DataNode client；Server-side include NameNode server and
DataNode server, wherein NameNode server-side includes NameNode server-side and storage HDFS file metadata
Database, DataNode server include DataNode server-side and the database for storing HDFS blocks of files.
In addition, HDFS system further includes on the channel being arranged between NameNode client and NameNode server-side
HDFS firewall.The HDFS firewall is equivalent to " an inspection on the channel being arranged between client and HDFS cluster
It stands ".Communication between HDFS client and HDFS server-side realizes that communication process is as follows based on Hadoop RPC frame:
1) user submits the information such as the file path of access by file access interface；
2) the NameNode client of file access interface intrinsic call RPC frame is led to NameNode server-side
Before letter, is intercepted by the HDFS firewall of the embodiment of the present invention, access request is checked；
3) when inspection result is that access request meets preset condition, then HDFS firewall completely turns the access request
It is sent to NameNode server-side, the data of HDFS file metadata are locally stored according to access request access for NameNode server-side
Library to obtain the metadata information for needing the HDFS blocks of files accessed, and is forwarded to NameNode visitor by HDFS firewall
Family end；When inspection result is that access request does not meet preset condition, then error message is returned to NameNode client, and will
Error message is back to user by file access interface.
4) it after file access interface takes the metadata information of HDFS blocks of files inside NameNode client, calls
RPC frame DataNode client is communicated by DataNode client with DataNode server-side, by first number of HDFS blocks of files
It is believed that breath is forwarded to DataNode server-side；
5) DataNode server-side can access according to the metadata information of HDFS blocks of files and HDFS file is locally stored
The database of block, to obtain HDFS file block data；
6) DataNode server-side is back to DataNode client, file access after will acquire HDFS file block data
Interface takes HDFS blocks of files inside DataNode client, and is transmitted to user.
The embodiment of the present invention utilizes the characteristics of HDFS file access process, requests text to NameNode server in client
When the metadata information of part, fine granularity scope check is carried out to client access request, if meeting the preset rules of competence,
Client access request is completely forwarded to HDFS cluster (NameNode server-side and storage HDFS file metadata here
Database), if do not met, do not forward client request, but return to error message.So that user must be anti-by this
Wall with flues could access or manage to HDFS server-side.The HDFS of the embodiment of the present invention can be actively using initiative type safeguard technology
Real time monitoring, alarm, stops the external data around enterprise network boundary (FireWall, IDS IPS etc.) protection to be attacked at identification
It hits, and from the data theft, broken of internal high permission user (DBA, developer, third party's outsourcing service provider)
It is bad, damage etc., from the technological layer of HDFS access instruction Precise control, a kind of active safety defensive measure is provided, in conjunction with
Independently of the safe access control rule of HDFS, user's reply can be helped to threaten from inside and outside data safety.
Referring to Fig. 2, a kind of step flow chart of the implementation method embodiment of HDFS system firewall of the invention is shown,
The HDFS system includes NameNode server, and the HDFS system firewall setting takes in application server and NameNode
On channel between business device, the method can specifically include following steps:
Step 101, it receives the expression from the application server and carries out metadata visit to the NameNode server
The metadata access request asked；
Wherein, the firewall of the embodiment of the present invention can receive the request of the metadata access from application server, wherein
Metadata access request indicates to carry out metadata access to NameNode server.
Step 102, default fine granularity purview certification is carried out to metadata access request, obtains authentication result；
Wherein, the firewall of the embodiment of the present invention can metadata access request to reach NameNode server it
Before, metadata access request is intercepted and captured, and default fine granularity purview certification is carried out to metadata access request, to obtain table
Show that certification passes through or indicate the unacceptable authentication result of certification.
Step 103, when the default fine granularity purview certification passes through, metadata access request is sent to described
NameNode server；
So, only when the default fine granularity purview certification of metadata access request passes through, the embodiment of the present invention
Firewall can just request metadata access to be sent to NameNode server to obtain metadata information.
Step 104, the metadata letter for metadata access request from the NameNode server is received
Breath；
Wherein, it after NameNode server receives metadata access request, can be asked according to the metadata access
It asks to obtain corresponding metadata information from local data base, and the metadata information is back to firewall, in this way, fire prevention
Wall can receive the metadata information for being directed to metadata access request.
Step 105, the metadata information is back to the application server；
Then, the metadata information received can be back to application server by the firewall of the embodiment of the present invention.
Step 106, when the default fine granularity purview certification does not pass through, it will indicate that authenticating unsanctioned error message returns
It is back to the application server.
On the contrary, if authentication result is that metadata access request does not pass through the default fine granularity by step 102
Purview certification, then the firewall of the embodiment of the present invention will not forward the metadata access to request to NameNode server, and
Error message will be returned to application server, wherein the error message indicates that metadata access request certification does not pass through.
By means of the technical solution of the above embodiment of the present invention, the embodiment of the present invention by application server and
Firewall is set on the channel between NameNode server, rather than firewall is embedded in HDFS cluster (such as NameNode
Server) it is internal, it avoids on HDFS cluster reboot and influences the problem of HDFS cluster is run；And by being set on above-mentioned channel
Firewall is set, is carried out before so as to be sent to NameNode server to the metadata access request from application server
It intercepts and captures, and carries out default fine granularity purview certification, just metadata access can be requested to be forwarded to only after certification passes through
NameNode server, so that application server obtains corresponding metadata information；And when certification does not pass through, then do not forward this
Metadata access is requested to NameNode server, but returns to error message to application server, improves HDFS system
Security protection intensity can shield the attack outside cluster.
On the basis of the above embodiments, optionally, another reality of the invention being shown respectively referring to Fig. 3 and Fig. 4
Apply the logic diagram and physics deployment diagram of the HDFS system of example.
Wherein, identical system module and the communication process between them can refer to mutually in Fig. 1, Fig. 3 and Fig. 4,
It is repeated no more in the present embodiment.
As shown in Figure 3, Figure 4, the HDFS firewall of the embodiment of the present invention includes NameNode server end module, center tune
Spend device module, NameNode client modules, these three modules are sequentially connected in series, and be deployed in client application server and
Between NameNode server, the client and server-side of correspondence and they and HDFS system between these three modules
Correspondence it is as shown in Figure 3.
Wherein, the Client API of Fig. 3 is deployed on the application server in Fig. 4；In addition, Fig. 3 is only shown schematically
A DataNode server in Fig. 4, other two DataNode server is not shown, but do not influence the embodiment of the present invention
Method.
In addition, from figs. 3 and 4 it can be seen that any access request that client is initiated to NameNode server all needs
First pass around HDFS firewall.It can handle user access request inside HDFS firewall, specific processing step is as follows:
1) NameNode client that, NameNode server module is responsible for receiving client send to file destination
Metadata access request, and metadata access request is parsed, obtain request content；
The request content includes at least: the user name of user A (that is, request of data side), password, user A are requested
The address of target metadata, user A are when operating the corresponding file destination of the target metadata (that is, target data)
Object run type (wherein, object run type can be reading and writing, modification etc. operation).
That is, can determine which user will be to which mesh by parsing to metadata access request here
Mark file carries out the request content of which kind of operation.
Certainly, several important request contents, tool of the present invention for request content are only schematically listed here
Hold in vivo and be not limited to above content, can also include other information.
2), the request content of intercepting and capturing is committed to central scheduler module and carries out default particulate by NameNode server module
Purview certification is spent, authentication result is obtained；
Wherein, how central scheduler can be operated according to authentication result to determine in next step.
Central scheduler module carries out default fine granularity purview certification in the request content requested the metadata access
When, it can be accomplished by the following way:
S1, according to pre-set user identity information to the metadata access request in the user name and the password into
Row authentication；
Specifically, the pre-set user identity information of the embodiment of the present invention, which is stored, has access to the data of HDFS system
The subscriber identity information of the user name of all users of permission and corresponding password, wherein the pre-set user identity information can be
One tables of data, and the pre-set user identity information can store central scheduler module local, HDFS firewall other
In module or it is stored in HDFS system.
That is, central scheduler module when obtaining the pre-set user identity information, can locally call or from
External system obtains；If pre-set user identity information is locally stored in HDFS firewall the acquisition that can save data
Between, promote authentication efficiency；It, can be with and if being stored in pre-set user identity information in the system in addition to firewall
Save the local storage space of HDFS firewall.For the specific storage mode of pre-set user identity information can according to need into
Row flexible setting and adjustment.
Here it is possible to searched whether in pre-set user identity information with the user name of the user A in request content and
One group of subscriber identity information that password matches, wherein user name and corresponding password are one group of subscriber identity informations.
If there is matched subscriber identity information, then authentication passes through, and does not otherwise pass through.
In this way, the embodiment of the present invention can be before the metadata information that client accesses in HDFS server, to access
The user identity of data carries out re-authentication, and so-called re-authentication, i.e., before the authentication for carrying out this, user is logged in
When HDFS system, the authentication of first time also can be all carried out.This ensure that the user identity safety of access data, further
Shield the attack outside cluster.
S2, when the authentication of metadata access request passes through, when acquisition operates the file destination
Action type be the object run type list of targeted subscribers；
Wherein, when the authentication that the metadata access of user A is requested passes through, such as user A needs to carry out file A
Read operation, then central scheduler module can obtain which user has file A from default file operating right list
There is the permission of read operation, and these users just constitute list of targeted subscribers.
It wherein, include a plurality of record in the default file operating right list, every describes operation file, operation
Type, operator.In this way, being assured which operator can have the permission of read operation to file A from these records.
Wherein, the pre-set user identity letter in the storage location, acquisition modes and S1 of the default file operating right list
Cease similar, details are not described herein.
S3, judges whether the user name in the metadata access request belongs to the mesh in the list of targeted subscribers
User name is marked, so that it is determined that whether metadata access request passes through authentication；
Then, central scheduler module it may determine that the metadata access request request content in user name whether
Belong to the user name in the above-mentioned list of targeted subscribers got, such as judges the user name of user A whether in the target user
In list, determine whether metadata access request passes through authentication with this.
S4, if the user name belongs to the name of the target user in the list of targeted subscribers, it is determined that the metadata is visited
Ask that the authentication of request passes through；
In this way, the embodiment of the present invention can be made whether to mesh the user of access target file by S2, S3 and S4
The inspection that file carries out the permission of corresponding operating is marked, even if in this way, the user has passed through authentication, such as the user is HDFS
The high permission user (DBA, developer, third party's outsourcing service provider etc.) of internal system, the fire prevention of the embodiment of the present invention
Wall such as can also be avoided them from stealing significant data, be destroyed, damaged at the operation by authentication, so that HDFS be made to visit
Ask that instruction obtains Precise control.
S5 determines the default of the metadata access request when the authentication and the authentication pass through
Fine granularity purview certification passes through；
Also, only request in the metadata access of user in the case where authentication and authentication pass through, this
The central scheduler module of inventive embodiments can just assert that the metadata access request of the user A has passed through default fine granularity permission
Certification.
In this way, can further protect the information of HDFS system to pacify by the double authentication of authentication and authentication
Entirely.
S6 determines the metadata when any one of the authentication and the authentication certification do not pass through
The default fine granularity purview certification of access request does not pass through.
On the contrary, if the username and password in discovery metadata access request is not belonging to pre-set user identity by S1
One group of subscriber identity information in information, and/or, judge through S3, finds the user name in the metadata access request
The target user's name being not belonging in the list of targeted subscribers can then determine the default fine granularity of the metadata access request
Purview certification does not pass through.
In this way, the embodiment of the present invention passes through S1~S6, so that it may carry out authentication to the metadata access request of user
It is whether thin by presetting come the request content for determining the metadata access request of user A with this with the double authentication of authentication
Granularity purview certification improves authentication strength and has refined certification granularity.
So by 2), if it is determined that the default fine granularity purview certification passes through, then continues specially treated 3)
Judgment step；
3) it, according to the default security information of the target data, determines in the corresponding user of the user name to the mesh
Mark file carries out before the operation of the object run type, if needs to carry out the file destination default specially treated.
Specifically, due to reading and writing of some files towards certain user in HDFS system, the operation such as change when, cannot will
The file data of original storage shows these users, needs to carry out before file data is showed user to preset special place
It manages (such as encryption, desensitization process).Wherein, the action type that encryption can be applied to includes read operation, write operation,
And the action type that desensitization process can be applied to is read operation.
Such as user A need in HDFS system file A carry out read operation when, it is necessary first to read the member of this document A
Data information, in this process, due to including some sensitive informations in file A, and user A is not to these in file A
The permission of the read operation of sensitive information, user B is to these sensitive informations in file A, and in other words, user B is in A file
Full content has the permission of read operation.Therefore, when user A will read file A, system setting needs to take off file A
Quick processing, that be supplied to user A reading can only be the file A after desensitization.
So center is adjusted here in view of the safety of the sensitive information in HDFS system in certain files in this step
Degree device module also needs to judge the user A before carrying out read operation to file A, if needs to carry out this document A default spy
Different processing (such as the specially treateds such as encryption, desensitization).
4), in addition, central scheduler module can also be by request content when above-mentioned default fine granularity purview certification passes through
It is sent to NameNode client modules；
Wherein, the present invention does not limit the execution sequence of step 3) He step 4).
5), after process step 3) and step 4), the NameNode client modules of the embodiment of the present invention can be incited somebody to action
Request content is sent to the NameNode server-side of the NameNode server, and NameNode server-side is according to metadata access
The database of HDFS file metadata is locally stored in the request content access of request, to obtain the file destination for needing to access
Metadata information, then, the metadata information that NameNode server-side will acquire are back to the NameNode client of firewall
End module；
6), the NameNode client modules of firewall receive the NameNode service from the NameNode server
The metadata information sent is held, and the metadata information is back to central scheduler module；
7), central scheduler module, will be according to above-mentioned steps 3 after the metadata information received) in whether need
The judging result of default specially treated is carried out, to file destination to determine whether to handle metadata information.
Specifically, when determine need to carry out specially treated to file destination when, then the central schedule of the embodiment of the present invention
Device module can also obtain the file destination (file A) after the default specially treated on the DataNode server
Storage address；And the metadata information in the step 6) received is updated to the storage address；
That is, when needing to carry out such as desensitization process to file A by step 3) judgement, the side of the embodiment of the present invention
Method can carry out desensitization process to this document A, and the file A ' after desensitization is stored into the database of DataNode server
Another storage address on, i.e. file A and file A ' exist, and file A can't be deleted or substitute.Then, then this is obtained
File A ' and will will be taken using the storage address in the storage address A ' of the database of DataNode server from NameNode
Metadata information in the storage address A (i.e. above-mentioned steps 6) of the file A got at business end) it is replaced, to obtain one
A new metadata information.
Certainly, for the desensitization process to file A, and operation that the file A ' after desensitization process is stored can be with
Completed, can also be completed by other modules in HDFS system by the firewall of the embodiment of the present invention, the present invention to this simultaneously
With no restrictions.
8), central scheduler module by updated metadata information (i.e. the storage address of file A ') via
NameNode server module is back to the NameNode client of the application server；
9), when the default fine granularity purview certification that the authentication result Jing Guo step 2) is the request content does not pass through (example
Do not pass through as at least having one in authentication, authentication) when, then central scheduler will directly indicate that certification is unsanctioned
Error message is back to the application server, without above-mentioned steps 2) any one step in~step 8).
Optionally, firewall in the above-described embodiments can be a gateway, be deployed in Fig. 3 on gateway
NameNode server module, central scheduler module and NameNode client modules.Also, the firewall is additionally provided with two
A network interface card, a network interface card is for connecting application server, another network is for connecting NameNode server, so that anti-
Wall with flues can be different from network protocol application server and NameNode server communicated, HDFS server is protected
Shield.
The method of realizing fireproof wall of the embodiment of the present invention, which only needs to modify a small amount of Hadoop source code, can be realized HDFS visit
Ask that instruction truncation, technical threshold is low, workload is few, it being capable of effectively save product development cost.Also, firewall of the invention with
Gateway forms deployment, supports software deployment and soft or hard integrated deployment, does not need again Hadoop cluster internal and installs additional insert
Part avoids additional cost payout to influence all very littles to HDFS cluster and client application.
It should be noted that for simple description, therefore, it is stated as a series of action groups for embodiment of the method
It closes, but those skilled in the art should understand that, embodiment of that present invention are not limited by the describe sequence of actions, because according to
According to the embodiment of the present invention, some steps may be performed in other sequences or simultaneously.Secondly, those skilled in the art also should
Know, the embodiments described in the specification are all preferred embodiments, and the related movement not necessarily present invention is implemented
Necessary to example.
It is corresponding with method provided by the embodiments of the present invention, referring to Fig. 5, show a kind of HDFS system of the present invention
Firewall system embodiment structural block diagram, the HDFS system includes NameNode server, and the firewall system is set
It sets on the channel between application server and the NameNode server, the firewall system can specifically include as follows
Module:
First receiving module 51, for receive the expression from the application server to the NameNode server into
The metadata access of row metadata access is requested；
Authentication module 52 obtains certification knot for carrying out default fine granularity purview certification to metadata access request
Fruit；
Sending module 53, for when the default fine granularity purview certification passes through, the metadata access being requested to send out
It send to the NameNode server；
Second receiving module 54 is requested from the NameNode server for the metadata access for receiving
Metadata information；
First return module 55, for the metadata information to be back to the application server；
Second return module 56, for when the default fine granularity purview certification does not pass through, will indicate that certification does not pass through
Error message be back to the application server.
Optionally, the firewall system further include:
Parsing module obtains request content for parsing to metadata access request；
The request content includes at least: the user name of request of data side, password, the requested target element in request of data side
Object run class of the address, request of data side of data when being operated to the corresponding target data of the target metadata
Type.
Optionally, the authentication module 52 includes:
Authentication submodule, for according to pre-set user identity information to the metadata access request in the use
Name in an account book and the password carry out authentication；
Acquisition submodule when the authentication for requesting when the metadata access passes through, is obtained to the number of targets
It is the list of targeted subscribers of the object run type according to action type when being operated；
Authentication submodule, for judging whether the user name in the metadata access request belongs to the mesh
Target user's name in user list is marked, so that it is determined that whether metadata access request passes through authentication；
First determines submodule, if belonging to the name of the target user in the list of targeted subscribers for the user name,
Determine that the authentication of the metadata access request passes through；
Second determines submodule, for when the authentication and the authentication pass through, determining first number
Pass through according to the default fine granularity purview certification of access request；
Third determines submodule, for not passing through when any one of the authentication and the authentication certification
When, determine that the default fine granularity purview certification of the metadata access request does not pass through.
Optionally, the firewall system further include:
Determining module, for when the default fine granularity purview certification passes through, according to the default peace of the target data
Full information, determine the corresponding user of the user name to the file destination carry out the object run type operation it
Before, if need to carry out the file destination default specially treated.
Optionally, the HDFS system further includes DataNode server, the firewall system further include:
Module is obtained, for obtaining target data after the default specially treated on the DataNode server
Storage address；
Update module, for the metadata information received to be updated to the storage address；
First return module 55, comprising:
Submodule is returned to, for the updated metadata information to be back to the application server.
For system embodiments, since it is basically similar to the method embodiment, related so being described relatively simple
Place illustrates referring to the part of embodiment of the method.
All the embodiments in this specification are described in a progressive manner, the highlights of each of the examples are with
The difference of other embodiments, the same or similar parts between the embodiments can be referred to each other.
It should be understood by those skilled in the art that, the embodiment of the embodiment of the present invention can provide as method, apparatus or calculate
Machine program product.Therefore, the embodiment of the present invention can be used complete hardware embodiment, complete software embodiment or combine software and
The form of the embodiment of hardware aspect.Moreover, the embodiment of the present invention can be used one or more wherein include computer can
With in the computer-usable storage medium (including but not limited to magnetic disk storage, CD-ROM, optical memory etc.) of program code
The form of the computer program product of implementation.
The embodiment of the present invention be referring to according to the method for the embodiment of the present invention, terminal device (system) and computer program
The flowchart and/or the block diagram of product describes.It should be understood that flowchart and/or the block diagram can be realized by computer program instructions
In each flow and/or block and flowchart and/or the block diagram in process and/or box combination.It can provide these
Computer program instructions are set to general purpose computer, special purpose computer, Embedded Processor or other programmable data processing terminals
Standby processor is to generate a machine, so that being held by the processor of computer or other programmable data processing terminal devices
Capable instruction generates for realizing in one or more flows of the flowchart and/or one or more blocks of the block diagram
The device of specified function.
These computer program instructions, which may also be stored in, is able to guide computer or other programmable data processing terminal devices
In computer-readable memory operate in a specific manner, so that instruction stored in the computer readable memory generates packet
The manufacture of command device is included, which realizes in one side of one or more flows of the flowchart and/or block diagram
The function of being specified in frame or multiple boxes.
These computer program instructions can also be loaded into computer or other programmable data processing terminal devices, so that
Series of operation steps are executed on computer or other programmable terminal equipments to generate computer implemented processing, thus
The instruction executed on computer or other programmable terminal equipments is provided for realizing in one or more flows of the flowchart
And/or in one or more blocks of the block diagram specify function the step of.
Although the preferred embodiment of the embodiment of the present invention has been described, once a person skilled in the art knows bases
This creative concept, then additional changes and modifications can be made to these embodiments.So the following claims are intended to be interpreted as
Including preferred embodiment and fall into all change and modification of range of embodiment of the invention.
Finally, it is to be noted that, herein, relational terms such as first and second and the like be used merely to by
One entity or operation are distinguished with another entity or operation, without necessarily requiring or implying these entities or operation
Between there are any actual relationship or orders.Moreover, the terms "include", "comprise" or its any other variant meaning
Covering non-exclusive inclusion, so that process, method, article or terminal device including a series of elements not only wrap
Those elements are included, but also including other elements that are not explicitly listed, or further includes for this process, method, article
Or the element that terminal device is intrinsic.In the absence of more restrictions, being wanted by what sentence "including a ..." limited
Element, it is not excluded that there is also other identical elements in process, method, article or the terminal device for including the element.
The firewall system of implementation method to a kind of HDFS system firewall provided by the present invention and a kind of HDFS above
System, is described in detail, and used herein a specific example illustrates the principle and implementation of the invention, above
The explanation of embodiment is merely used to help understand method and its core concept of the invention；Meanwhile for the general skill of this field
Art personnel, according to the thought of the present invention, there will be changes in the specific implementation manner and application range, in conclusion this
Description should not be construed as limiting the invention.
Claims (6)
1. a kind of implementation method of HDFS system firewall, which is characterized in that the HDFS system includes NameNode server,
The HDFS system firewall is arranged on the channel between application server and NameNode server, which comprises
The expression from the application server is received to visit to the metadata that the NameNode server carries out metadata access
Ask request；
Default fine granularity purview certification is carried out to metadata access request, obtains authentication result；
When the default fine granularity purview certification passes through, metadata access request is sent to the NameNode and is serviced
Device；
Receive the metadata information for metadata access request from the NameNode server；
The metadata information is back to the application server；
When the default fine granularity purview certification does not pass through, it will indicate that authenticating unsanctioned error message is back to the application
Server；
It is described that default fine granularity purview certification, before obtaining authentication result, the method are carried out to metadata access request
Further include:
Metadata access request is parsed, request content is obtained；
The request content includes at least: the user name of request of data side, password, the requested target metadata in request of data side
Object run type when being operated to the corresponding target data of the target metadata of address, request of data side；
It is described that default fine granularity purview certification is carried out to metadata access request, obtain authentication result, comprising:
According to pre-set user identity information to the user name and password progress identity in metadata access request
Certification；
When the authentication of metadata access request passes through, operation class when operating to the target data is obtained
Type is the list of targeted subscribers of the object run type；
Judge whether the user name in the metadata access request belongs to the target user in the list of targeted subscribers
Name, so that it is determined that whether metadata access request passes through authentication；
If the user name belongs to the name of the target user in the list of targeted subscribers, it is determined that the metadata access request
Authentication passes through；
When the authentication and the authentication pass through, the default fine granularity power of the metadata access request is determined
Limit certification passes through；
When any one of the authentication and the authentication certification do not pass through, the metadata access request is determined
Default fine granularity purview certification do not pass through.
2. the method according to claim 1, wherein it is described metadata access request is sent to it is described
Before NameNode server, the method also includes:
When the default fine granularity purview certification passes through, according to the default security information of the target data, determine described
The corresponding user of user name carries out the file destination before the operation of the object run type, if needs to the mesh
Mark file carries out default specially treated, and specially treated includes cryptographic operation and desensitization operation.
3. according to the method described in claim 2, it is characterized in that, the HDFS system further includes DataNode server, when
It is described that the metadata information is back to the application clothes when determining that needs carry out default specially treated to the target data
It is engaged in front of device, the method also includes:
Obtain storage address of the target data after the default specially treated on the DataNode server；
The metadata information received is updated to the storage address；
It is described that the metadata information is back to the application server, comprising:
The updated metadata information is back to the application server.
4. a kind of firewall system of HDFS system, which is characterized in that the HDFS system includes NameNode server, described
Firewall system is arranged on the channel between application server and the NameNode server, the firewall system packet
It includes:
First receiving module carries out first number to the NameNode server for receiving the expression from the application server
It is requested according to the metadata access of access；
Authentication module obtains authentication result for carrying out default fine granularity purview certification to metadata access request；
Sending module, for when the default fine granularity purview certification passes through, metadata access request to be sent to institute
State NameNode server；
Second receiving module, for receiving first number for metadata access request from the NameNode server
It is believed that breath；
First return module, for the metadata information to be back to the application server；
Second return module authenticates unsanctioned mistake for will indicate when the default fine granularity purview certification does not pass through
Information is back to the application server；
The firewall system further include:
Parsing module obtains request content for parsing to metadata access request；
The request content includes at least: the user name of request of data side, password, the requested target metadata in request of data side
Object run type when being operated to the corresponding target data of the target metadata of address, request of data side；
The authentication module includes:
Authentication submodule, for according to pre-set user identity information to the metadata access request in the user name
Authentication is carried out with the password；
Acquisition submodule, when the authentication for requesting when the metadata access passes through, obtain to the target data into
Action type when row operation is the list of targeted subscribers of the object run type；
Authentication submodule is used for judging whether the user name in the metadata access request belongs to the target
Target user's name in the list of family, so that it is determined that whether metadata access request passes through authentication；
First determines submodule, if belonging to the name of the target user in the list of targeted subscribers for the user name, it is determined that
The authentication of the metadata access request passes through；
Second determines submodule, for when the authentication and the authentication pass through, determining that the metadata is visited
Ask that the default fine granularity purview certification of request passes through；
Third determines submodule, is used for when any one of the authentication and the authentication certification do not pass through, really
The default fine granularity purview certification of the fixed metadata access request does not pass through.
5. firewall system according to claim 4, which is characterized in that the firewall system further include:
Determining module, for being believed according to the default safety of the target data when the default fine granularity purview certification passes through
Breath determines before the corresponding user of the user name carries out the operation of the object run type to the file destination, is
No to need to carry out the file destination default specially treated, specially treated includes cryptographic operation and desensitization operation.
6. firewall system according to claim 5, which is characterized in that the HDFS system further includes DataNode service
Device, the firewall system further include:
Module is obtained, for obtaining storage of the target data after the default specially treated on the DataNode server
Address；
Update module, for the metadata information received to be updated to the storage address；
First return module, comprising:
Submodule is returned to, for the updated metadata information to be back to the application server.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710439355.4A CN107196951B (en) | 2017-06-12 | 2017-06-12 | A kind of implementation method and firewall system of HDFS system firewall |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710439355.4A CN107196951B (en) | 2017-06-12 | 2017-06-12 | A kind of implementation method and firewall system of HDFS system firewall |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN107196951A CN107196951A (en) | 2017-09-22 |
| CN107196951B true CN107196951B (en) | 2019-02-26 |
Family
ID=59877982
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201710439355.4A Active CN107196951B (en) | 2017-06-12 | 2017-06-12 | A kind of implementation method and firewall system of HDFS system firewall |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN107196951B (en) |
Families Citing this family (14)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107818268A (en) * | 2017-11-15 | 2018-03-20 | 中国联合网络通信集团有限公司 | The access control method and server of big data platform |
| CN108289098B (en) * | 2018-01-12 | 2021-07-06 | 百度在线网络技术（北京）有限公司 | Authority management method and device of distributed file system, server and medium |
| CN110071870B (en) * | 2018-01-24 | 2022-03-18 | 苏宁云商集团股份有限公司 | Alluxio-based routing method and device for multiple HDFS clusters |
| CN109309686A (en) * | 2018-11-01 | 2019-02-05 | 浪潮软件集团有限公司 | Multi-tenant management method and device |
| CN109840424A (en) * | 2018-12-18 | 2019-06-04 | 合肥天源迪科信息技术有限公司 | A kind of data base encryption and the system that desensitizes |
| CN111522787B (en) * | 2019-02-01 | 2023-04-07 | 阿里巴巴集团控股有限公司 | Data processing method and device of distributed system and storage medium |
| CN110188573A (en) * | 2019-05-27 | 2019-08-30 | 深圳前海微众银行股份有限公司 | Subregion authorization method, device, equipment and computer readable storage medium |
| CN113722723A (en) * | 2020-05-25 | 2021-11-30 | 中移(苏州)软件技术有限公司 | Information processing method, system, equipment and computer storage medium |
| CN112329015A (en) * | 2020-12-23 | 2021-02-05 | 黑龙江省网络空间研究中心 | Privacy information protection system and method based on code injection |
| CN112668052A (en) * | 2020-12-30 | 2021-04-16 | 北京天融信网络安全技术有限公司 | Data desensitization method and device, storage medium and electronic equipment |
| CN112910980B (en) * | 2021-01-27 | 2022-11-15 | 中国银联股份有限公司 | Database access system and method |
| CN113343299A (en) * | 2021-06-18 | 2021-09-03 | 浪潮云信息技术股份公司 | Hive database dynamic desensitization system and implementation method |
| CN113780789A (en) * | 2021-09-02 | 2021-12-10 | 科大国创云网科技有限公司 | Unified data access service type fine-grained authority control method and system |
| CN116743511B (en) * | 2023-08-15 | 2023-11-03 | 中移(苏州)软件技术有限公司 | Authentication method, device, server and storage medium |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102307185A (en) * | 2011-06-27 | 2012-01-04 | 北京大学 | Data isolation method used in storage cloud |
| CN103209189A (en) * | 2013-04-22 | 2013-07-17 | 哈尔滨工业大学深圳研究生院 | Distributed file system-based mobile cloud storage safety access control method |
| CN104023085A (en) * | 2014-06-25 | 2014-09-03 | 武汉大学 | Security cloud storage system based on increment synchronization |
| CN104506514A (en) * | 2014-12-18 | 2015-04-08 | 华东师范大学 | Cloud storage access control method based on HDFS (Hadoop Distributed File System) |
| CN104935590A (en) * | 2015-06-10 | 2015-09-23 | 南京航空航天大学 | HDFS access control method based on role and user trust value |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| EP3065077B1 (en) * | 2015-03-05 | 2020-04-08 | Tata Consultancy Services Limited | Gap analysis of security requirements against deployed security capabilities |
-
2017
- 2017-06-12 CN CN201710439355.4A patent/CN107196951B/en active Active
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102307185A (en) * | 2011-06-27 | 2012-01-04 | 北京大学 | Data isolation method used in storage cloud |
| CN103209189A (en) * | 2013-04-22 | 2013-07-17 | 哈尔滨工业大学深圳研究生院 | Distributed file system-based mobile cloud storage safety access control method |
| CN104023085A (en) * | 2014-06-25 | 2014-09-03 | 武汉大学 | Security cloud storage system based on increment synchronization |
| CN104506514A (en) * | 2014-12-18 | 2015-04-08 | 华东师范大学 | Cloud storage access control method based on HDFS (Hadoop Distributed File System) |
| CN104935590A (en) * | 2015-06-10 | 2015-09-23 | 南京航空航天大学 | HDFS access control method based on role and user trust value |
Also Published As
| Publication number | Publication date |
|---|---|
| CN107196951A (en) | 2017-09-22 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN107196951B (en) | A kind of implementation method and firewall system of HDFS system firewall | |
| CN108322471B (en) | Multi-tenant identity and data security management cloud service | |
| US10027716B2 (en) | System and method for supporting web services in a multitenant application server environment | |
| US9225704B1 (en) | Unified management of third-party accounts | |
| US10432644B2 (en) | Access control system for enterprise cloud storage | |
| CN105659558B (en) | Computer implemented method, authorization server and computer-readable memory | |
| US9582672B2 (en) | Encrypted file storage | |
| US10565402B2 (en) | System and method for serving online synchronized content from a sandbox domain via a temporary address | |
| CN108370374B (en) | Certificate update and deployment | |
| CN106104563B (en) | The technology of network security is provided by the account opened on time just | |
| US10397213B2 (en) | Systems, methods, and software to provide access control in cloud computing environments | |
| KR20200093007A (en) | Model training system and method, and storage medium | |
| CN105164633B (en) | The configuration and verifying carried out by trusted provider | |
| CN104580364B (en) | A kind of method and apparatus of resource sharing | |
| US10560435B2 (en) | Enforcing restrictions on third-party accounts | |
| CN107005582A (en) | Public point is accessed using the voucher being stored in different directories | |
| WO2010138910A1 (en) | Secure collaborative environment | |
| JP2012533820A (en) | Plug-in authority control method and system | |
| CN106034104A (en) | Verification method, verification device and verification system for network application accessing | |
| US11063922B2 (en) | Virtual content repository | |
| TW201909072A (en) | Method, device, and apparatus for loss reporting, removing loss report, and service management of electronic account | |
| JP2021527858A (en) | Location-based access to access-controlled resources | |
| US10931650B1 (en) | Apparatus and method for building, extending and managing interactions between digital identities and digital identity applications | |
| CA3148146A1 (en) | Techniques for incentivized intrusion detection system | |
| CN113569179A (en) | Subsystem access method and device based on unified website |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |