CN104092733B - A kind of credible distributed file system based on HDFS - Google Patents
A kind of credible distributed file system based on HDFS Download PDFInfo
- Publication number
- CN104092733B CN104092733B CN201410281244.1A CN201410281244A CN104092733B CN 104092733 B CN104092733 B CN 104092733B CN 201410281244 A CN201410281244 A CN 201410281244A CN 104092733 B CN104092733 B CN 104092733B
- Authority
- CN
- China
- Prior art keywords
- remote
- data
- module
- remote proving
- platform
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Landscapes
- Storage Device Security (AREA)
Abstract
The credible distributed file system based on HDFS that the invention discloses a kind of, including remote proving module and data seal modules, the remote proving module include：Based on the remote proving module of integrity measurement framework IMA, remote procedure call remote proving module connects remote proving module with socket；The seal data module includes progress data seal modules after remote proving failure.The beneficial effects of the invention are as follows credible and secure mechanism is added in HDFS distributed file systems, so that carrying out the credibility for being able to confirm that interactive object when data interaction between entity node, system connection untrusted address can be prevented after system integrity is destroyed, being sealed to data after remote proving failure ensures that data are not leaked.
Description
Technical field
The present invention relates to trust computing field, more particularly to a kind of credible distributed file system based on HDFS.
Background technology
Due to the rapid development of information technology in recent years and widely available, a large amount of data are increased with exponential, Google, refined
The large enterprises such as tiger, Amazon face the challenge of mass data.Google takes the lead in being proposed Google file system GFS and MapReduce
The storage and processing of mass data are coped with, are attracted wide public concern, and expedited the emergence of a version increased income rapidly, referred to as
Hadoop.After development at full speed in recent years, Hadoop has become an important portion of Internet company's basic calculation platform
Point, in external large enterprises of the domestic large enterprises and Twitter, Facebook etc. such as Tencent, Baidu, Alibaba, have certainly
The version of oneself customization.Hadoop is designed exclusively for processed offline and large-scale data analysis, is repeatedly read in write-once
In the affairs taken, performances of the Hadoop efficiently to realize and stablize widely is supported and is applied in Internet enterprises.
Along with flourishing for cloud computing, many critical problems have also been encountered, safety problem is especially prominent.Hadoop
Distributed memory system be known as Hadoop distributed file systems, abbreviation HDFS, it by namenode, the second namenode,
The entity nodes such as back end and DFS clients form.Namenode stores the NameSpace mirror image of distributed file system
Fsimage and editor daily record edits, and maintain file and catalogue all in file system tree and whole tree in memory.
Back end is the place that data block substantially stores, and it is 3 that data block, which gives tacit consent to number of copies, can be when reading and writing the data block
It is synchronized between copy.Second namenode is responsible for NameSpace mirror image fsimage and edits the merging of daily record edits, and handle
Data after merging are returned to namenode.DFS clients are an objects of DistributedFilesystem, and HDFS is answered
A DistributedFilesystem can be generally obtained with program to be interacted with HDFS, and DFS clients are actually
The object interacted with namenode and back end.Remote process tune is frequently carried out between these multiple entity nodes
With interactive operations such as, TCP flow formula data transmission and HTTP Transmitting Data Streams, how to ensure that these data interactions are that safety can
That leans on gradually attracts attention.Since Hadoop is the LAN deployment in fire wall, the security mechanism added after 1.0 versions
It is mainly used for preferably assisting the cooperation of multi-user in LAN.However in advanced duration penetration attack Advanced
After Persistent Attack come into vogue, the safety curtain that fire wall provides gradually becomes unable to do what one wishes.
Currently, on solving cloud computing safety problem, many methods and thinking have been emerged.Reliable computing technology is answered
It is exactly the one big of cloud computing security fields research to solve the safety problem encountered in cloud computing application used in cloud computing environment
Hot spot.The purpose of trust computing is the most sensitive information of protection, as private key and symmetric key are not stolen or not by malicious code
It uses.With Hadoop assume environment be in the fire wall of safety on the contrary, trust computing assume software in use may
It can be destroyed, when attacking generation, sensitive key will be protected.Trusted Computing Group (Trusted Computing
Group, TCG) credible platform module (Trusted Platform Module, TPM) that defines is a kind of be placed in computer
New embedded security subsystem.TCG chips mainly pass through authentication public key function, integrity measurement function and remote proving function
To achieve the purpose that protect terminal security.By checking the variation of PCR metrics, possess the platform of TPM it can be found that spy is soft
The presence of the rogue programs such as part, wooden horse, and after finding malicious code, platform can order TPM to refuse the decryption to sensitive data
Operation or the operation signed or decrypted using private key.Searching full private key and will not be leaked due to the TPM owners, it is any
The malicious act for attempting to obtain private key is all infeasible.
Invention content
Under the background quickly popularized the present invention is based on distributed system and reliable computing technology, study reliable computing technology
It is dissolved into distributed system, to solve the safety problem of distributed system, proposes especially long-range using reliable computing technology
Proof technology and data sealing technique solve the solution of the safety problem of distributed file system, and specific technical solution is such as
Under.
The technical scheme is that：
A kind of credible distributed file system based on HDFS, including remote proving module and data seal modules, it is described
Remote proving module includes：Based on the remote authentication module of integrity measurement framework IMA, for carrying out platform between entity node
Status remote proves, verification platform integrality；Remote procedure call remote proving module, for being built between HDFS entity nodes
When vertical remote procedure call connection, remote proving is carried out, it is ensured that the platform status integrality of remote procedure call promoter；
Socket connects remote proving module, carries out remote proving when connecting untrusted address for entity node, avoids putting down
Platform leaking data.The seal data module is used to after remote proving failure be sealed data, by PCR value to symmetrical
Key is sealed, and is used in combination symmetric key that data are encrypted, so that data ability only under believable platform status
Can be decrypted, so that it is guaranteed that data are not leaked.
Further, in said program, the remote procedure call remote proving module carries out following steps：It is long-range first
Invocation of procedure applicant initiates socket connection request, long-range mistake to the remote procedure call port of remote procedure call server
After journey invoking server receives connection, remote procedure call applicant sends remote procedure call header and using server public affairs
Key encrypted random number n after remote procedure call server obtains random number n using private key decryption, sends platform status deposit
Applicant is returned in the value of device PCR10, random number n and measurement list.After applicant verifies reception data, determining pair
Whether side is in believable platform status, that is, confirms the platform status integrality of other side.
The socket connection remote procedure call module carries out following steps：Consumer process initiates socket company first
Whether welding system calls connect and incoming link address is as parameter, then judge link address credible in system calling
In address list, if not then judging whether trusted agent process starts, link address is sent to credible generation if starting
Otherwise reason process directly returns to mistake.
The seal data module carries out following steps：Process remote proving first is given birth to if remote proving success
AES key and use specify platform status register PCR to the sealing of key process data in a pair, and sealing data are exported to magnetic
On disk.If remote proving fails, it is encrypted using data key, and the key is abandoned after the completion of encryption.
Staticametric list of the remote proving based on integrity measurement framework IMA.
Seal data avoids directly using by being sealed AES key and data being encrypted using AES key
Data, which are encrypted, in RSA key causes efficiency to reduce.
The process of remote proving is added in the present invention before entity node and the normal communication interaction of entity node, only exists
When showing that the entity node to be communicated is credible by remote proving and dynamic measurement, just allow to carry out data interaction.This hair
It is bright entity node carry out socket connection system call when, if link address untrusted address can carry out remote proving,
Only remote proving by just socket being allowed to connect, to after avoiding entity node integrality from being destroyed data transmission to entering
The person's of invading machine.The present invention carries out seal data after remote proving failure, the data on entity node, avoids leaking data.
One aspect of the present invention can carry out remote proving to each entity, on the other hand can also be prevented after system integrity is destroyed
Socket connects untrusted address to avoid leaking data, while can be sealed to data after remote proving failure,
Finally realize practicable credible distributed file system.
Compared with prior art, the beneficial effects of the invention are as follows：One, after traditional binary system remote proving failure, add
The safety of seal data mechanism protection data is added；Two, remote proving and seal data mechanism are added in remote procedure call, are made
The entity node for obtaining interaction in distributed file system all obtains the credible security that integrity measurement framework IMA and TPM are provided；
Three, in socket connection procedure, when being added to the progress connect operations of untrusted address, remote proving is carried out
Operation, to ensure that local data by network connection to attacker's server and can not be transferred directly to attacker by invader
Server.Four, on the basis of credible distributed file system, a Dropbox application has been designed and Implemented, which is directed to can
The characteristics of believing distributed file system, can customize further safety.
Description of the drawings
Fig. 1 is composition schematic diagram of the present invention；
Fig. 2 is the platform status remote proving module design figure based on IMA；
Fig. 3 is remote procedure call remote proving module design figure；
Fig. 4 is that socket connects remote proving module design figure；
Fig. 5 is seal data module design figure.
Specific implementation mode
Below in conjunction with attached drawing, the present invention is further described, but the implementation and protection of the present invention are without being limited thereto, if having below
Not specified symbol or process is that those skilled in the art are referred to the prior art and realize.
Embodiment one
As shown in Figure 1, a kind of design of the credible distributed file system based on HDFS, including remote proving module sum number
According to seal modules, the remote proving module includes：Based on the remote proving module of integrity measurement framework IMA, for applying
Person establishes the activation certificate of connection, provides applicant's voucher for possessing legal TPM；Remote procedure call remote proving module,
When for establishing remote procedure call connection between HDFS entity nodes, remote proving is carried out, it is ensured that remote process tune
With the platform status integrality of promoter；Socket connects remote proving module, for entity node in connection untrusted address
When carry out remote proving, avoid platform data from revealing.The seal data module is used for the logarithm after remote proving failure
According to being sealed, symmetric key is sealed by PCR value, symmetric key be used in combination that data are encrypted, so that number
According to could be only decrypted under believable platform status, so that it is guaranteed that data are not leaked.
As shown in Fig. 2, the remote proving module based on integrity measurement framework IMA includes challenger and is certified
Two parts of person, challenger send random number to the person of being certified, and the person of being certified is by identity key to random number and platform status
The value of register PCR10 returns to challenger after being digitally signed.Challenger uses the identity key public key pair for the person of being certified
Data are encrypted, and whether verify random number correct.It is not then certified person and sends the measure column that integrity measurement framework is safeguarded
Table, whether challenger is consistent by the value for simulating PCR extension measurement list operation verifications PCR10, if unanimously illustrating measure column
Table is not tampered with, and is measured list by traversal and is checked whether metric is legal, illustrates that the person's of being certified is flat if all legal
Platform integrality is not destroyed, and returns to measurement results.
As shown in figure 3, the remote procedure call remote proving module includes remote procedure call client and long-range
Invocation of procedure server, remote procedure call applicant initiate set to the remote procedure call port of remote procedure call server
Word connection request is connect, after remote procedure call server receives connection, remote procedure call applicant sends remote procedure call
Header and the use encrypted random number n of server public key, remote procedure call server obtain random number n using private key decryption
Afterwards, the value of platform status register PCR10 is sent, applicant is returned in random number n and measurement list.Applicant is to receiving number
It after being verified, determines whether other side is in believable platform status, that is, confirms the platform status integrality of other side.
As shown in figure 4, the socket connection remote procedure call includes the following steps：Consumer process initiates set first
It connects word connection system to call connect and be passed to link address as parameter, then whether judges link address in system calling
In credible address list, if not then whether being actuated for judging to trusted agent process, connection is sent if starting
Trusted agent process is given in address, otherwise directly returns to mistake.The credible address list includes INADDR_ANY, INADDR_
The IP address of other interaction machines of totally ten one local address and cluster internal such as LOCALHOST.The transmission link address
Netlink has been used to carry out data interactive method as kernel and user to trusted agent.The tPid indicate trusted agent into
The process number of journey is initialized as -1 when system starts, be initialized as after trusted agent process initiation trusted agent into
The process number of journey.The ERR indicates that system is called and returns to error value.
As shown in figure 5, the seal data module includes the following steps：Process remote proving first, if long-range card
Bright success then generates a pair of of AES key and using specifying platform status register PCR to seal key process data, sealing number
According to exporting on disk.If remote proving fails, it is encrypted using data key, and being abandoned after the completion of encryption should
Key.
Claims (3)
1. a kind of credible distributed file system based on HDFS, including remote proving module and data seal modules, feature
It is
The remote proving module includes：
Remote procedure call remote proving module carries out verifying completeness of platform when carrying out communication interaction for entity node；
Based on the remote authentication module of integrity measurement framework IMA, for carrying out platform status remote proving between entity node,
Verification platform integrality；The remote procedure call remote proving module is specifically used for realizing following process：Remote process first
Applicant is called to initiate socket connection request, remote process tune to the remote procedure call port of remote procedure call server
After receiving connection with server, remote procedure call applicant is sent remote procedure call header and is added using server public key
Close random number n sends platform status register after remote procedure call server obtains random number n using private key decryption
Applicant is returned in the value of PCR10, random number n and measurement list；After applicant verifies reception data, other side is determined
It whether is in believable platform status, that is, confirms the platform status integrality of other side；
Socket connect remote proving module, for entity node to untrusted address carry out socket connection system call when into
Row verifying completeness of platform；
After the seal data module carries out remote proving failure for entity node, seal data is carried out to platform data；
The remote authentication module based on integrity measurement framework IMA is specifically used for realizing following process：Challenger sends out first
Send random number to the person of being certified, the person of being certified carries out the value of random number and platform status register PCR10 by identity key
Challenger is returned to after digital signature；Challenger is decrypted data using the identity key public key for the person of being certified, and verifies
Whether random number is correct；It is not then certified person and sends the measurement list that integrity measurement framework is safeguarded, challenger is by simulating PCR
Whether the value of extension measurement list operation verification PCR10 is consistent, if consistent illustrate that measuring list is not tampered with, and passes through traversal
Measurement list checks whether metric is legal, illustrates that the completeness of platform for the person of being certified is not destroyed if all legal, and
Return to measurement results；The socket connection remote proving module is specifically used for realizing following process：Consumer process is sent out first
It plays socket connection system to call connect and be passed to link address as parameter, then judges link address in system calling
Whether in credible address list, if not then judging whether trusted agent process starts, connection ground is sent if starting
Trusted agent process is given in location, otherwise directly returns to mistake；
The seal data module is specifically used for realizing following process：Remote proving is carried out first, if remote proving success
It then generates a pair of of AES key and using specifying platform status register PCR to seal key process data, sealing data is exported
Onto disk；If remote proving fails, it is encrypted using data key, and the key is abandoned after the completion of encryption.
2. the credible distributed file system according to claim 1 based on HDFS, it is characterised in that：The platform status
Remote proving is the staticametric result based on IMA integrity measurement frameworks.
3. the credible distributed file system according to claim 1 based on HDFS, it is characterised in that：The long-range card
Bright module and data seal modules can effectively defend high-order duration to threaten APT.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201410281244.1A CN104092733B (en) | 2014-06-20 | 2014-06-20 | A kind of credible distributed file system based on HDFS |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201410281244.1A CN104092733B (en) | 2014-06-20 | 2014-06-20 | A kind of credible distributed file system based on HDFS |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN104092733A CN104092733A (en) | 2014-10-08 |
| CN104092733B true CN104092733B (en) | 2018-09-14 |
Family
ID=51640413
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201410281244.1A Active CN104092733B (en) | 2014-06-20 | 2014-06-20 | A kind of credible distributed file system based on HDFS |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN104092733B (en) |
Families Citing this family (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106656915A (en) * | 2015-10-30 | 2017-05-10 | 深圳市中电智慧信息安全技术有限公司 | Cloud security server based on trusted computing |
| CN106650459B (en) * | 2016-11-11 | 2018-05-01 | 大唐高鸿信安（浙江）信息科技有限公司 | Safeguard the believable system and method for distributed memory system data |
| CN108763397B (en) * | 2018-05-22 | 2022-07-08 | 中国科学技术大学苏州研究院 | Data placement method of distributed file system supporting deep learning |
| US11469890B2 (en) * | 2020-02-06 | 2022-10-11 | Google Llc | Derived keys for connectionless network protocols |
Citations (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101477602A (en) * | 2009-02-10 | 2009-07-08 | 浪潮电子信息产业股份有限公司 | Remote proving method in trusted computation environment |
Family Cites Families (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN100581107C (en) * | 2008-11-04 | 2010-01-13 | 西安西电捷通无线网络通信有限公司 | Trusted platform verification method based on three-element peer authentication |
| US8601265B2 (en) * | 2010-11-22 | 2013-12-03 | Netapp, Inc. | Method and system for improving storage security in a cloud computing environment |
-
2014
- 2014-06-20 CN CN201410281244.1A patent/CN104092733B/en active Active
Patent Citations (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101477602A (en) * | 2009-02-10 | 2009-07-08 | 浪潮电子信息产业股份有限公司 | Remote proving method in trusted computation environment |
Non-Patent Citations (2)
| Title |
|---|
| Design and Implementation of a TCG-Based Integrity Measurement Architecture;Reiner Sailer et al;《USENIX ASSOCIATION PROCEEDINGS OF THE 13TH USENIX SECURITY SYMPOSIUM》;20040813;正文第223-238页 * |
| Incorporating Hardware Trust Mechanisms in Apache Hadoop;Jason C. Cohen et al;《Globecom Workshops(GC Wkshps),2012 IEEE》;20121207;正文第771-772页 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN104092733A (en) | 2014-10-08 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US10305903B2 (en) | Bypassing certificate pinning | |
| US9582668B2 (en) | Quantifying the risks of applications for mobile devices | |
| WO2019218919A1 (en) | Private key management method and apparatus in blockchain scenario, and system | |
| US10769045B1 (en) | Measuring effectiveness of intrusion detection systems using cloned computing resources | |
| Li et al. | Mayhem in the push clouds: Understanding and mitigating security hazards in mobile push-messaging services | |
| CN104092733B (en) | A kind of credible distributed file system based on HDFS | |
| CN109995776A (en) | A kind of internet data verification method and system | |
| Bates et al. | Forced perspectives: Evaluating an SSL trust enhancement at scale | |
| Onarlioglu et al. | Overhaul: Input-driven access control for better privacy on traditional operating systems | |
| Galibus et al. | Elements of cloud storage security: concepts, designs and optimized practices | |
| JP2022534677A (en) | Protecting online applications and web pages that use blockchain | |
| US20230244797A1 (en) | Data processing method and apparatus, electronic device, and medium | |
| Tejaswi et al. | Leaky kits: the increased risk of data exposure from phishing kits | |
| TWI546698B (en) | Login system based on servers, login authentication server, and authentication method thereof | |
| CN110875903B (en) | Security defense method and device | |
| Banas | Cloud forensic framework for iaas with support for volatile memory | |
| Bravi | Use of Trusted Computing techniques to counteract Cybersecurity attacks in Critical Infrastructures | |
| Hauffman et al. | Assessing the Security of Android Dating Apps | |
| Kimak | An Investigation into possible attacks on HTML5 IndexedDB and their prevention | |
| Jones | SenderKeys Identified Mail | |
| Bock | Measuring Adoption of Phishing-Resistant Authentication Methods on the Web | |
| Tyler et al. | Towards Browser Controls to Protect Cookies from Malicious Extensions | |
| Gautam et al. | Passwords Are Meant to Be Secret: A Practical Secure Password Entry Channel for Web Browsers | |
| CN111147241A (en) | Key protection method based on block chain | |
| CN117040821A (en) | Login state generation and verification method, electronic equipment and medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |