CN103561033A - Device and method for user to have remote access to HDFS cluster - Google Patents
Device and method for user to have remote access to HDFS cluster Download PDFInfo
- Publication number
- CN103561033A CN103561033A CN201310554689.8A CN201310554689A CN103561033A CN 103561033 A CN103561033 A CN 103561033A CN 201310554689 A CN201310554689 A CN 201310554689A CN 103561033 A CN103561033 A CN 103561033A
- Authority
- CN
- China
- Prior art keywords
- user
- module
- server
- character
- hdfs cluster
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Abstract
The invention relates to a device and method for a user to have remote access to an HDFS cluster. The device comprises middleware for connecting the user and the HDFS cluster, the middleware comprises a server module and a client module which are in a mutual scheduling relation, the server module comprises a service starting module, an access request processing module, an access request transmission module and a file management module, and the client module comprises a user operation module, an IP packaging module and a server selection module. The device and method for the user to have remote access to the HDFS cluster have the services for starting data scheduling or/and data storage on a server with a public network IP on the HDFS cluster, the middleware is provided for the user needing to have remote access to the HDFS cluster, and the user has remote access to the HDFS cluster through the middleware. The device and method for the user to have remote access to the HDFS cluster not only can solve problems caused when the user has remote access to the HDFS cluster but also effectively isolate the user.
Description
Technical field
The present invention relates to data memory access field, relate in particular to a kind of device and method of user's remote access HDFS cluster.
Background technology
Hadoop is the software platform that large-scale data is processed in an exploitation and operation, is the open source software framework that Appach realizes with java language, can realize in the cluster forming at a large amount of computers mass data is carried out to Distributed Calculation.Hadoop is developed by Apache foundation, and user can develop distributed program in the situation that not understanding distributed bottom details, and the function that makes full use of cluster is carried out high-speed computation and storage.In Hadoop framework, most crucial design is exactly HDFS(Hadoop Distributed File System, is called for short HDFS) and MapReduce, HDFS is responsible for providing the storage of mass data, and MapReduce is responsible for providing the calculating to data.
HDFS is the distributed file system of Hadoop, HDFS is a host-guest architecture, and a HDFS cluster generally comprises a title node (also claiming NameNode node or name server) and a plurality of memory node (also claiming DataNode node or data storage server).In addition, the Hadoop cluster that comprises above-mentioned HDFS cluster also comprises the node group of the node of called after Secondary NameNode, the node of called after JobTracker and called after TaskTracker, wherein NameNode nodes records file be how to be split into piecemeal (block) and which DateNode node these piecemeals (block) have all stored in, NameNode node has also been preserved the state information of file system operation simultaneously; The piecemeal being split (blocks) of storing in DataNode node, the state information that Secondary NameNode node is responsible for collecting file system operation is to NameNode node; When having job invocation to Hadoop cluster, JobTracker node is responsible for the operation of task, and is responsible for a plurality of TaskTracker nodes of scheduling; TaskTracker node is responsible for some map or reduce task.
In Hadoop, HDFS has the feature of high fault tolerance, can be deployed on cheap hardware by design.HDFS can provide the data of high transfer rate application programs to conduct interviews, and is applicable to having the application program of super large data set.HDFS not only has backup functionality, the data that are stored in HDFS cluster automatically can be backed up, and HDFS also has frame perception mechanism simultaneously, and when user's data writing, HDFS can write by the nearest node of chosen distance user; When user's reading out data, HDFS can read by the nearest backup block of chosen distance user, thereby reduces bandwidth usage to greatest extent.
Although HDFS has, height is fault-tolerant, height is concurrent and save the features such as bandwidth, but in order to save to greatest extent the pressure of bandwidth and release NameNode node, when user reads and writes data from HDFS cluster, what adopt is point-to-point mode, be that NameNode node can directly dock user with the DataNode node of storage data, allow user directly from DataNode node, obtain data flow.Although do like this, reached the object that developer wants, concerning using HDFS to carry out but having played opposite effect data memory access.Due in Hadoop cluster, the server (DataNode node) that is not every storage data has public network IP, when NameNode node returns to user by the IP of DataNode node, that user obtains is the Intranet IP of DataNode node in server cluster, user can not use this Intranet IP to conduct interviews to DataNode node, so also just can not read and write data from DataNode node.Therefore,, if directly build HDFS on application cluster server, there will be user outside application cluster server can not read and write the situation of data in HDFS cluster.
Therefore under prior art condition, keeping in the impregnable situation of HDFS intrinsic advantage, in the urgent need to a kind of technical scheme that can realize data in user's remote access HDFS cluster.
Summary of the invention
The primary technical problem solving of the present invention is that a kind of device of user's remote access HDFS cluster is provided for prior art, utilizes this device can realize user remote storage is conducted interviews.
The technical problem that the present invention further will solve is a kind of method of user's remote access HDFS cluster that provides for prior art, the method is realizing when user conducts interviews to remote storage, also realized the user of access HDFS cluster is effectively isolated, user can not directly touch the IP address of server in HDFS cluster.
The present invention solves the technical scheme that above-mentioned primary technical problem adopts: a kind of device of user's remote access HDFS cluster, it is characterized in that, comprise the middleware that connects user and HDFS cluster, described middleware comprises the service end of scheduling relation module and client modules each other, wherein
Service end module comprises:
Service starts module, for starting the service of data dispatch and data storage on HDFS cluster server;
Access request processing module, the access request of sending for the treatment of user;
Access request transport module, the access request of sending for transmission user;
Document management module, the fileinfo for leading subscriber on HDFS cluster data storage server;
Client modules comprises:
User's operational module, for receiving user's access request;
IP package module, for encapsulating the IP address of the server on HDFS cluster with public network IP;
Server is selected module, for selecting from the nearest server of user network topology, and the IP address from the nearest server of user network topology choosing is recorded.
User uses the use procedure of said apparatus access HDFS cluster to be:
The server on HDFS cluster with public network IP is opened has data dispatch or/and the service of data storage,
User uses said apparatus to carry out remote access process to HDFS cluster:
(a), user uses user's operational module that in above-mentioned middleware, client modules provides to propose access request, one or more in the following operation of content choice of access request: create file, open file reading, deleted file, files listed and create file;
(b), client modules is according to the IP address encapsulating in IP package module, by server, select module successively each server to be carried out to connecting test, select the server of network delay minimum, then user's access request is sent to this server by the access request processing module in service end module and access request transport module, this server is called to adapter coupling servers;
(c), adapter coupling servers receives after the access request of user's transmission, the data storage server that searching is nearest apart from this adapter coupling servers, by service, start that module starts data dispatch on this data storage server or/and the service of data storage, then between user and this data storage server, set up TCP and be connected, thereby realize the remote access operation of user to this data storage server.
As improvement, the client modules of described middleware also comprises:
Encrypting module, for user name is encrypted, to generate identifying code;
Authentication module, verifies for the identifying code to encrypting user;
Described service end module also comprises: User Catalog is set up module, for the data storage server at HDFS cluster, to user, sets up remote access catalogue.
Like this, when user is with after the data storage server nearest apart from adapter coupling servers is connected, the User Catalog of described middleware set up module according to each user's user by name they set up a catalogue, user can only carry out all accessing operations of data storage server in HDFS cluster under this catalogue, and can not the content under other User Catalogs be conducted interviews.
The present invention solves the technical scheme that above-mentioned further technical problem adopts: a kind of method of user's remote access HDFS cluster, it is characterized in that, and comprise the following steps:
(1), on HDFS cluster, there is turn-on data scheduling on the server of public network IP or/and the service of data storage;
(2), to needing the user of remote access HDFS cluster that a middleware is provided, described middleware comprises the service end of scheduling relation module and client modules each other, wherein,
Service end module comprises:
Service starts module, for starting the service of data dispatch and data storage on server;
Access request processing module, the access request of sending for the treatment of user;
Access request transport module, the access request of sending for transmission user;
Document management module, the fileinfo for leading subscriber on HDFS cluster data storage server;
Client modules comprises:
User's operational module, for receiving user's access request;
IP package module, for encapsulating the IP address of the server on HDFS cluster with public network IP;
Server is selected module, for selecting from the nearest server of user network topology, and the IP address from the nearest server of user network topology choosing is recorded;
(3), user uses above-mentioned middleware to carry out remote access to HDFS cluster:
(3-1), user uses user's operational module that in above-mentioned middleware, client modules provides to propose access request, one or more in the following operation of content choice of access request: create file, open file reading, deleted file, files listed and create file;
(3-2), client modules is according to the IP address encapsulating in IP package module, by server, select module successively each server to be carried out to connecting test, select the server of network delay minimum, then user's access request is sent to this server by the access request processing module in service end module and access request transport module, this server is called to adapter coupling servers;
(3-3), adapter coupling servers receives after the access request of user's transmission, the data storage server that searching is nearest apart from this adapter coupling servers, by service, start that module starts data dispatch on this data storage server or/and the service of data storage, then between user and this data storage server, set up TCP and be connected, thereby realize the remote access operation of user to this data storage server.
In this scheme, in HDFS cluster, the real IP address of server is encapsulated in middleware, user cannot touch these IP addresses, and user do not need to know the real IP address of server in each HDFS cluster yet, only need to use just addressable HDFS cluster of middleware provided by the invention.
As improvement, the client modules of described middleware also comprises:
Encrypting module, for user name is encrypted, to generate identifying code;
Authentication module, verifies for the identifying code to encrypting user;
Described service end module also comprises: User Catalog is set up module, for the data storage server at HDFS cluster, to user, sets up remote access catalogue.
When user is with after the data storage server nearest apart from adapter coupling servers is connected, the User Catalog of described middleware set up module according to each user's user by name they set up a catalogue, user can only carry out all accessing operations of data storage server in HDFS cluster under this catalogue, and can not go to access the content under other User Catalogs.This improvement project can play the effect that user is isolated---and be that user is when access HDFS cluster, have a privately owned memory space, other users cannot conduct interviews to this privately owned memory space, thereby both reduced user's use difficulty, guaranteed again the safety of HDFS cluster self.
As improving further, the method in described encrypting module, user name being encrypted is:
User name is encoded according to following rule, thereby generates the identifying code of long 30 characters:
The Back ground Information that the first six of identifying code character is identifying code, comprises successively: key, user name length, every how many characters, insert idle characters, each idle character quantity, character o and character k inserting; Wherein, insert idle character be called span every how many characters, each idle character quantity of inserting is called space, and whether character o and character k are used for identifying code effectively to verify; Key is the random numeral generating between 0 to 25, in ciphering process, each character of user name is encrypted with secret key, span character of every encryption simultaneously just inserts space random character, if last identifying code length is less than 30 characters, at end, inserts random character and carry out completion;
Described authentication module is decrypted identifying code in the following way:
First read out the first six character of identifying code, order is followed successively by key, user name length, every how many characters, inserts idle character, each idle character quantity, character o and character k inserting;
If the 5th and the 6th of this identifying code is not character o and character k, illustrate that this identifying code is invalid, return to null value;
Then with key, remaining character is decoded, space character skipped in span character of every decoding, until the length of length of the plaintext out of decoding and user name finishes when identical, finally decodes and obtain complete user name.
Improve again, in described step (3-2), the method that Network Search postpones minimum server is: according to the IP address encapsulating in IP package module, successively the IP of each server is used to ping order, record receives from each server the needed time of bag of returning, the last the shortest server of required time of finding out from the recorded time, this server is from the nearest server of user network topology, and the IP of this server is recorded.
Improve again, in described step (3-3), adopt the frame perception mechanism that HDFS cluster carries to find the data storage server nearest apart from adapter coupling servers.
As preferably, described middleware offers user with the form of Jar bag.
Compared with prior art, the invention has the advantages that: the present invention not only can solve user's remote access HDFS cluster problem, also realized user has effectively been isolated simultaneously, it is the real IP address that user does not need to know each server in HDFS cluster, only need to use middleware provided by the invention is addressable HDFS cluster, thereby it is very simple that user is used.
Accompanying drawing explanation
Fig. 1 is the integrated stand composition of user's remote access HDFS cluster in the embodiment of the present invention one;
Fig. 2 is the module frame chart of middleware in the embodiment of the present invention one;
Fig. 3 is the method flow diagram of user's remote access HDFS cluster in the embodiment of the present invention one;
Fig. 4 is the flow chart of euclidean distance between node pair computational methods in HDFS cluster frame perception mechanism in the embodiment of the present invention one;
Fig. 5 is the module frame chart of middleware in the embodiment of the present invention two.
Embodiment
Below in conjunction with accompanying drawing, embodiment is described in further detail the present invention.
Embodiment mono-
First the present embodiment provides a kind of device of user's remote access HDFS cluster, shown in Figure 1, this device comprises the middleware that connects user and HDFS cluster, and described middleware comprises the service end of scheduling relation module and client modules each other, wherein
Service end module comprises:
Service starts module, for starting the service of data dispatch and data storage on HDFS cluster server;
Access request processing module, the access request of sending for the treatment of user, it is any access request that the processing here refers to the access request that interpretation user sends, then to the processing of classifying of the access request after judgement;
Access request transport module, the access request of sending for transmission user;
Document management module, the fileinfo for leading subscriber on HDFS cluster data storage server, as record and preserve the user directory address on data storage server and the fileinfo in catalogue in HDFS cluster;
Client modules comprises:
User's operational module, for receiving user's access request;
IP package module, for encapsulating the IP address of the server on HDFS cluster with public network IP;
Server is selected module, for selecting from the nearest server of user network topology, and the IP address from the nearest server of user network topology choosing is recorded, shown in Figure 2.
The present embodiment provides a kind of method of user's remote access HDFS cluster, and user, by above-mentioned middleware access HDFS cluster, specifically comprises the following steps, shown in Figure 3:
(1), on HDFS cluster, there is turn-on data scheduling on the server of public network IP or/and the service of data storage;
(2), to needing the user of remote access HDFS cluster that a middleware with said structure is provided;
(3), user uses above-mentioned middleware to carry out remote access to HDFS cluster:
(3-1), user uses user's operational module that in above-mentioned middleware, client modules provides to propose access request, one or more in the following operation of content choice of access request: create file, open file reading, deleted file, files listed and create file;
(3-2), client modules is according to the IP address encapsulating in IP package module, by server, select module successively each server to be carried out to connecting test, select the server of network delay minimum, then user's access request is sent to this server by the access request processing module in service end module and access request transport module, this server is called to adapter coupling servers;
(3-3), adapter coupling servers receives after the access request of user's transmission, the data storage server that searching is nearest apart from this adapter coupling servers, by service, start that module starts data dispatch on this data storage server or/and the service of data storage, then between user and this data storage server, set up TCP and be connected, thereby realize the remote access operation of user to this data storage server.
In described step (3-2), the method that Network Search postpones minimum server is: according to the IP address encapsulating in IP package module, successively the IP of each server is used to ping order, record receives from each server the needed time of bag of returning, the last the shortest server of required time of finding out from the recorded time, this server is from the nearest server of user network topology, and the IP of this server is recorded.
In described step (3-3), adopt the frame perception mechanism that HDFS cluster carries to find the data storage server nearest apart from adapter coupling servers.Fig. 4 has described the flow chart of euclidean distance between node pair computational methods in HDFS cluster frame perception mechanism, and it adopts tree to represent the network configuration of node, and internodal distance is calculated to the nodes sum of nearest public father node with them.For example,, apart from node H
1with node H
2nearest public father node is R
1, node H
1to father node R
1nodes be 1, node H
2to father node R
1nodes be also 1, node H
1with node H
2distance be 2; And for example, node H
1with node H
6nearest public father node be D
1, H
1to D
1nodes be 2, H
6to D
1nodes be also 2, node H
1with node H
6distance be 4.The frame perceptron that HDFS cluster carries is made as routine techniques, no longer continues to describe in detail.
In the present embodiment, described middleware offers user with the form of Jar bag.
Embodiment bis-
Different from embodiment mono-, the client modules of described middleware also comprises:
Encrypting module, for user name is encrypted, to generate identifying code;
Authentication module, verifies for the identifying code to encrypting user;
Described service end module also comprises: User Catalog is set up module, for the data storage server at HDFS cluster, to user, sets up remote access catalogue; Shown in Figure 5.
When user is with after the data storage server nearest apart from adapter coupling servers is connected, the User Catalog of described middleware set up module according to each user's user by name they set up a catalogue, user can only carry out all accessing operations of data storage server in HDFS cluster under this catalogue, and can not the content under other User Catalogs be conducted interviews.
The method in described encrypting module, user name being encrypted is:
User name is encoded according to following rule, thus generate one long be the identifying code of 30 characters:
The Back ground Information that the first six of identifying code character is identifying code, comprises successively: key, user name length, every how many characters, insert idle characters, each idle character quantity, character o and character k inserting; Wherein, insert idle character be called span every how many characters, each idle character quantity of inserting is called space, and whether character o and character k are used for identifying code effectively to verify; Key is the random numeral generating between 0 to 25, in ciphering process, each character of user name is encrypted with secret key, span character of every encryption simultaneously just inserts space random character, if last identifying code length is less than 30 characters, at end, inserts random character and carry out completion;
Described authentication module is decrypted identifying code in the following way:
First read out the first six character of identifying code, order is followed successively by key, user name length, every how many characters, inserts idle character, each idle character quantity, character o and character k inserting;
If the 5th and the 6th of this identifying code is not character o and character k, illustrate that this identifying code is invalid, return to null value;
Then with key, remaining character is decoded, space character skipped in span character of every decoding, until length of the plaintext out of decoding finishes when identical with user name length, finally decodes and obtain complete user name.
To be slx123 be described in detail the process of above-mentioned encryption and decryption as example the user name of take below:
Key key is random generation, and the key key generating in this example is " 5 ", according to alphabet, it is mapped to " F ", empty how many of Span(inserts invalid data) be also automatically to generate, here be " 7 ", so be mapped to " H ", Space(inserts the figure place of invalid data at every turn), here be " 0 ", be mapped to " A ", the length of Length(enciphered data), enciphered data is " SLX123 " here, so length is 6, be mapped to " G ".Identifying code front two character is for being confirmed whether it is correct identifying code character " o " and character " k ", utilize key key to be encrypted, formula is: (x-' A '+key) %26+ ' A ', wherein x is " o " or " k ", the key is here 5, so encrypted result is respectively " T ", " P ", it is more than the implication of first 6 of identifying code.
Next, user name is encrypted, first according to encrypting formula (' S '-' A '+key) %26+ ' A ', encrypts first data " S " for X; Then judge whether to insert idle character, span is 7 herein, and the meaning refers to that every encryption 7 bit data just insert space invalid data, has only encrypted now one, does not therefore need insertion, then directly encrypts next bit " L "; Repeat above step, after data have been encrypted completely, if 30 of ciphertext curtailment, by the space of the idle character lack of fill that automatically generates.Therefore the identifying code after encrypting is FHAGTPXQCghiPHARPFRARGJYYEFYBR.
The inverse process of decrypting process for encrypting, first obtain the information of first 6, Key=" F ", " A "=5, Span=" H ", " A "=7, Space=...=0, Length=...=6, O=(' T '-' A '+26-key) %26+ ' A '=' o ', k=(' T '-' A '+26-key) %26+ ' A '=' k '; Whether the character that first judges the 5th and the 6th is " o " and " k ", if so, illustrates that ciphertext is effective, then just starts deciphering.First decipher first (' X '-' A '+26-key) %26+ ' A '=' S ', then by span, judge whether next bit is idle character, because span is 7, known is not idle character, so continue deciphering, obtains " L ", repeats above step; When being length, decoding length of the plaintext out stops decrypting process.Plaintext is now exactly encrypted data, the user name SLX123 obtaining after decoding.
Claims (8)
1. a device for user's remote access HDFS cluster, is characterized in that, comprises the middleware that connects user and HDFS cluster, and described middleware comprises the service end of scheduling relation module and client modules each other, wherein,
Service end module comprises:
Service starts module, for starting the service of data dispatch and data storage on HDFS cluster server;
Access request processing module, the access request of sending for the treatment of user;
Access request transport module, the access request of sending for transmission user;
Document management module, the fileinfo for leading subscriber on HDFS cluster data storage server;
Client modules comprises:
User's operational module, for receiving user's access request;
IP package module, for encapsulating the IP address of the server on HDFS cluster with public network IP;
Server is selected module, for selecting from the nearest server of user network topology, and the IP address from the nearest server of user network topology choosing is recorded.
2. the device of user's remote access HDFS cluster according to claim 1, is characterized in that, the client modules of described middleware also comprises:
Encrypting module, for user name is encrypted, to generate identifying code;
Authentication module, verifies for the identifying code to encrypting user;
Described service end module also comprises: User Catalog is set up module, for the data storage server at HDFS cluster, to user, sets up remote access catalogue.
3. a method for user's remote access HDFS cluster, is characterized in that, comprises the following steps:
(1), on HDFS cluster, there is turn-on data scheduling on the server of public network IP or/and the service of data storage;
(2), subtend needs the user of remote access HDFS cluster that a middleware is provided, described middleware comprises the service end of scheduling relation module and client modules each other, wherein,
Service end module comprises:
Service starts module, for starting the service of data dispatch and data storage on server;
Access request processing module, the access request of sending for the treatment of user;
Access request transport module, the access request of sending for transmission user;
Document management module, the fileinfo for leading subscriber on HDFS cluster data storage server;
Client modules comprises:
User's operational module, for receiving user's access request;
IP package module, for encapsulating the IP address of the server on HDFS cluster with public network IP;
Server is selected module, for selecting from the nearest server of user network topology, and the IP address from the nearest server of user network topology choosing is recorded;
(3), user uses above-mentioned middleware to carry out remote access to HDFS cluster:
(3-1), user uses user's operational module that in above-mentioned middleware, client modules provides to propose access request, one or more in the following operation of content choice of access request: create file, open file reading, deleted file, files listed and create file;
(3-2), client modules is according to the IP address encapsulating in IP package module, by server, select module successively each server to be carried out to connecting test, select the server of network delay minimum, then user's access request sends this server by the access request processing module in service end module and access request transport module, and this server is called to adapter coupling servers;
(3-3), adapter coupling servers receives after the access request of user's transmission, the data storage server that searching is nearest apart from this adapter coupling servers, by service, start that module opens data dispatch on this data storage server or/and the service of data storage, then between user and this data storage server, set up TCP and be connected, thereby realize the remote access operation of user to this data storage server.
4. the method for user's remote access HDFS cluster according to claim 3, is characterized in that, the client modules of described middleware also comprises:
Encrypting module, for user name is encrypted, to generate identifying code;
Authentication module, verifies for the identifying code to encrypting user;
Described service end module also comprises: User Catalog is set up module, for the data storage server at HDFS cluster, to user, sets up remote access catalogue;
When user is with after the data storage server nearest apart from adapter coupling servers is connected, the User Catalog of described middleware set up module according to each user's user by name they set up a catalogue, user can only carry out all accessing operations of data storage server in HDFS cluster under this catalogue, and can not the content under other User Catalogs be conducted interviews.
5. the method for user's remote access HDFS cluster according to claim 4, is characterized in that, in described encrypting module, user name is encrypted to method and is:
User name is encoded according to following rule, thereby generates the identifying code of long 30 characters:
The Back ground Information that the first six of identifying code character is identifying code, comprise successively: key, user name length, every how many characters, insert idle character, each idle character quantity of inserting, character o and character k, every how many characters, insert idle character and be called span, each idle character quantity of inserting is called space, wherein whether character o and character k are used for identifying code effectively to verify, key is the random numeral generating in a numeral between 0 to 25, in ciphering process, each character of user name is encrypted with secret key, span character of every encryption simultaneously just inserts space random character, if last identifying code length is less than 30 characters, at end, insert random character and carry out completion,
Described authentication module is decrypted identifying code in the following way:
First read out the first six character of identifying code, order is followed successively by key, user name length, every how many characters, inserts idle character, each idle character quantity, character o and character k inserting;
If the 5th and the 6th of this identifying code is not character o and character k, illustrate that this identifying code is invalid, return to null value;
Then with key, remaining character is decoded, space character skipped in span character of every decoding, until decode when length of the plaintext is user name length character string, finishes, and finally decoding obtains complete user name.
6. the method for user's remote access HDFS cluster according to claim 3, it is characterized in that, in described step (3-2), the method that Network Search postpones minimum server is: according to the IP address encapsulating in IP package module, successively the IP of each server is used to ping order, record receives from each server the needed time of bag of returning, the last the shortest server of required time of finding out from the recorded time, this server is from the nearest server of user network topology, and the IP of this server is recorded.
7. the method for user's remote access HDFS cluster according to claim 3, is characterized in that, described step (3-3) adopts the frame perception mechanism that HDFS cluster carries to find the data storage server nearest apart from adapter coupling servers.
8. the method for user's remote access HDFS cluster according to claim 3, is characterized in that, described middleware offers user with the form of Jar bag.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201310554689.8A CN103561033B (en) | 2013-11-08 | 2013-11-08 | User remotely accesses the device and method of HDFS cluster |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201310554689.8A CN103561033B (en) | 2013-11-08 | 2013-11-08 | User remotely accesses the device and method of HDFS cluster |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN103561033A true CN103561033A (en) | 2014-02-05 |
| CN103561033B CN103561033B (en) | 2016-11-02 |
Family
ID=50015183
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201310554689.8A Active CN103561033B (en) | 2013-11-08 | 2013-11-08 | User remotely accesses the device and method of HDFS cluster |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN103561033B (en) |
Cited By (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103973689A (en) * | 2014-05-12 | 2014-08-06 | 浪潮电子信息产业股份有限公司 | Method for managing clusters through mobile device |
| CN104750937A (en) * | 2015-04-08 | 2015-07-01 | 西安电子科技大学 | Integrated circuit layout conversion method based on Hadoop |
| CN105490990A (en) * | 2014-09-18 | 2016-04-13 | 台湾新光保全股份有限公司 | Method for logging on service network through wireless sensing device |
| CN106027666A (en) * | 2016-06-30 | 2016-10-12 | 东方通信股份有限公司 | Scheduling method independent of platform network |
| CN106796500A (en) * | 2014-05-28 | 2017-05-31 | 蓝数据软件有限公司 | Mapped between the version of distributed file system |
| CN108255928A (en) * | 2017-11-30 | 2018-07-06 | 北京元心科技有限公司 | Distributed system method for reading data and device |
| CN112468454A (en) * | 2020-11-11 | 2021-03-09 | 许继集团有限公司 | Remote file management system and remote file management method thereof |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1571398A (en) * | 2004-04-29 | 2005-01-26 | 上海交通大学 | Network safety isolating and information exchanging system and method based on proxy mapping |
| CN101074991A (en) * | 2007-06-29 | 2007-11-21 | 中国网络通信集团公司 | Method and system for processing geographic position information and middleware in geographic information system |
| CN101127656A (en) * | 2006-08-18 | 2008-02-20 | 台湾侠诺科技股份有限公司 | An access system and method in broadband same-structure network |
| CN102946323A (en) * | 2012-10-24 | 2013-02-27 | 曙光信息产业(北京)有限公司 | Realizing method for location awareness of compute node cabinet in HDFS (Hadoop Distributed File System) and realizing system thereof |
-
2013
- 2013-11-08 CN CN201310554689.8A patent/CN103561033B/en active Active
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1571398A (en) * | 2004-04-29 | 2005-01-26 | 上海交通大学 | Network safety isolating and information exchanging system and method based on proxy mapping |
| CN101127656A (en) * | 2006-08-18 | 2008-02-20 | 台湾侠诺科技股份有限公司 | An access system and method in broadband same-structure network |
| CN101074991A (en) * | 2007-06-29 | 2007-11-21 | 中国网络通信集团公司 | Method and system for processing geographic position information and middleware in geographic information system |
| CN102946323A (en) * | 2012-10-24 | 2013-02-27 | 曙光信息产业(北京)有限公司 | Realizing method for location awareness of compute node cabinet in HDFS (Hadoop Distributed File System) and realizing system thereof |
Non-Patent Citations (1)
| Title |
|---|
| 黎平国: "基于HDFS的数字图书馆云存储系统研究", 《情报探索》, 30 September 2012 (2012-09-30) * |
Cited By (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103973689A (en) * | 2014-05-12 | 2014-08-06 | 浪潮电子信息产业股份有限公司 | Method for managing clusters through mobile device |
| CN106796500A (en) * | 2014-05-28 | 2017-05-31 | 蓝数据软件有限公司 | Mapped between the version of distributed file system |
| US11080244B2 (en) | 2014-05-28 | 2021-08-03 | Hewlett Packard Enterprise Development Lp | Inter-version mapping of distributed file systems |
| CN105490990A (en) * | 2014-09-18 | 2016-04-13 | 台湾新光保全股份有限公司 | Method for logging on service network through wireless sensing device |
| CN104750937A (en) * | 2015-04-08 | 2015-07-01 | 西安电子科技大学 | Integrated circuit layout conversion method based on Hadoop |
| CN106027666A (en) * | 2016-06-30 | 2016-10-12 | 东方通信股份有限公司 | Scheduling method independent of platform network |
| CN106027666B (en) * | 2016-06-30 | 2019-03-19 | 东方通信股份有限公司 | A kind of dispatching method independently of platform network |
| CN108255928A (en) * | 2017-11-30 | 2018-07-06 | 北京元心科技有限公司 | Distributed system method for reading data and device |
| CN112468454A (en) * | 2020-11-11 | 2021-03-09 | 许继集团有限公司 | Remote file management system and remote file management method thereof |
Also Published As
| Publication number | Publication date |
|---|---|
| CN103561033B (en) | 2016-11-02 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN103561033B (en) | User remotely accesses the device and method of HDFS cluster | |
| CN108664223B (en) | Distributed storage method and device, computer equipment and storage medium | |
| CN106127075B (en) | Encryption method can search for based on secret protection under a kind of cloud storage environment | |
| JP6671278B2 (en) | Data transfer optimization | |
| JP4972067B2 (en) | User query processing apparatus and method using encrypted modulation of queries in a database containing encrypted columns | |
| US11657171B2 (en) | Large network attached storage encryption | |
| CN109522328B (en) | Data processing method and device, medium and terminal thereof | |
| US10721058B2 (en) | Ultra-secure blockchain enabled analytics | |
| KR101282281B1 (en) | Weighted keyword searching method for perserving privacy, and apparatus thereof | |
| CN103530201A (en) | Safety data repetition removing method and system applicable to backup system | |
| CN110768787A (en) | Data encryption and decryption method and device | |
| US9847981B1 (en) | Encrypted augmentation storage | |
| CN109716317A (en) | System and method for the accurate flow of event of creation time | |
| CN101335765A (en) | Storage service middleware based on mobile caching | |
| CN111859426B (en) | Universal encrypted database connector and setting method thereof | |
| CN107911381A (en) | Access method, system, server-side and the client of application programming interface | |
| CN105279254A (en) | Layout data stream file system, operation apparatus for system and realization method for apparatus | |
| CN102438004A (en) | Method and system for acquiring metadata information of media file and multimedia player | |
| CN102664930A (en) | Off-line learning system | |
| CN104836656A (en) | Method for storing and transmitting video file | |
| CN105447146A (en) | Massive data collecting and exchanging system and method | |
| CN104753870B (en) | a kind of data transmission method and system | |
| CN108734023B (en) | System and method for accessing and integrating ciphertext database system | |
| CN104618414B (en) | A kind of implementation method of Distributed Services, service broker's device and distributed system | |
| US20090103723A1 (en) | System And Method For Secure Storage Of Data |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| C53 | Correction of patent of invention or patent application | ||
| CB02 | Change of applicant information |
Address after: West Street in the official Zhejiang city of Ningbo province Zhenhai District 315200 Village No. 777 Gen E WITHUB 42 Applicant after: Xidian-Ningbo Information Technology Institute Address before: Zhenhai District 315200 Zhejiang city of Ningbo province Chuang No. 777 West Street in the official record E WITHUB 42 Applicant before: Xidian-Ningbo Information Technology Institute |
|
| C14 | Grant of patent or utility model | ||
| GR01 | Patent grant |