CN104320401B - A kind of big data storage based on distributed file system accesses system and method - Google Patents
A kind of big data storage based on distributed file system accesses system and method Download PDFInfo
- Publication number
- CN104320401B CN104320401B CN201410602562.3A CN201410602562A CN104320401B CN 104320401 B CN104320401 B CN 104320401B CN 201410602562 A CN201410602562 A CN 201410602562A CN 104320401 B CN104320401 B CN 104320401B
- Authority
- CN
- China
- Prior art keywords
- server
- meta data
- storage
- log
- data server
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
- H04L67/10—Protocols in which an application is distributed across nodes in the network
- H04L67/1097—Protocols in which an application is distributed across nodes in the network for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS]
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L41/00—Arrangements for maintenance, administration or management of data switching networks, e.g. of packet switching networks
- H04L41/08—Configuration management of networks or network elements
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/50—Network services
- H04L67/51—Discovery or management thereof, e.g. service location protocol [SLP] or web services
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/50—Network services
- H04L67/56—Provisioning of proxy services
- H04L67/561—Adding application-functional data or data for application control, e.g. adding metadata
Landscapes
- Engineering & Computer Science (AREA)
- Computer Networks & Wireless Communication (AREA)
- Signal Processing (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The invention discloses a kind of big data storages based on distributed file system to access system and method, wherein, big data storage access system based on distributed file system includes distributed file system, and the distributed file system includes meta data server, storage server cluster, access client and log server；The meta data server is managed monitoring to the metadata information in distributed file system and to the storage server cluster；The data included in metadata information are stored in several data memory nodes of storage server cluster by the storage server cluster；Metadata information in the meta data server back up the log server and the operation log of real-time synchronization meta data server；The access client is used for the connection status according to meta data server, and selection accesses storage server cluster by the spare Metadata Service process in meta data server or log server.
Description
Technical field
The present invention relates to data storage access technique fields, and in particular to a kind of big data based on distributed file system
Storage accesses system and method.
Background technology
When Namenode (i.e. management node) based on hadoop is for production environment, single-point problem is stored.It solves at present
The scheme of this problem is mainly：The metadata information of Namenode is stored in NFS (Network File System, network
File system) on, the host where the active and standby process of Namenode is able to access on NFS all as the client of NFS
The metadata information of storage.After the host node process of Namenode exits, the metadata that host process is safeguarded is stored in NFS
On, the slave node process of Namenode still can normally access the metadata on NFS, the slave node process of such Namenode
The work of the host node process of Namenode is just completely taken over.
Using NFS management file storages, management node server-side is likely to encounter disk bottleneck, bandwidth bottleneck, influences business
Response speed；The problems such as management node server-side memory space is unable to dynamic expansion.
First, management node server-side disk easily encounters bottleneck：Data are uniformly stored on management node server-side disk,
Multiple and different access clients initiates data access request simultaneously, and management node server-side disk is likely to encounter bottleneck, influences
Performance.
Second, management node server-side bandwidth easily encounters bottleneck：Data are uniformly to be stored in management node server-side disk
On, when multiple and different access clients initiate data access request simultaneously, the data for flowing to different access client will be from
Management node server-side network interface card flows out, and management node server-side bandwidth easily reaches bottleneck, influences performance.
Third, management node server-side memory space are unable to dynamic expansion：Data can only be stored in management node server-side
Under certain catalogue, shared service need to be stopped when management node server-side insufficient memory is extended.Behind extension storage space,
Memory space is again started up, the process of extension needs to stop upper layer application system.
4th, management node server-side single-point problem：Data are stored in management node server-side, server-side occur suspension,
It delays machine, all access clients cannot all visit again the data in server-side, and applied business can occur with server-side failure
Failure, business can stop servicing.
Invention content
The technical problems to be solved by the invention are to provide a kind of big data storage based on distributed file system and access
System and method can solve the problems, such as management server single-point in big data storage access process.
One side according to the present invention provides a kind of big data storage based on distributed file system and accesses system
System, including distributed file system, the distributed file system includes meta data server, storage server cluster, accesses
Client and log server；
The meta data server is used to be managed the metadata information in distributed file system and deposit to described
Storage server cluster is managed monitoring；
The storage server cluster is used to that the data included in metadata information to be stored in storage according to pre-defined rule
Several data memory nodes of server cluster；
The log server for the metadata information in the meta data server is spaced to schedule into
Row backup and the operation log of real-time synchronization meta data server；
The access client is used for the connection status according to meta data server, and selection passes through meta data server or day
Spare Metadata Service process in will server accesses storage server cluster.
Other side according to the present invention provides a kind of big data storage access side based on distributed file system
Method, including：
The data included in metadata are stored in several memory nodes of storage server cluster according to pre-defined rule,
And monitoring is managed to metadata information and the storage server cluster by meta data server；
Log server is spaced to schedule backs up the metadata information in meta data server, and in real time
The operation log of synchronous meta data server；
Access client is to meta data server transmission data access request；
According to the connection status of meta data server, the access client selection is taken by meta data server or daily record
The spare Metadata Service process being engaged in device accesses storage server cluster.
A kind of big data storage based on distributed file system provided by the invention accesses system and method, and data are deposited
Storage solves the server-side disk bottleneck of the prior art and bandwidth bottle on several memory nodes of storage server cluster
Neck, and the memory space of server-side is easy to dynamic expansion；Meta data server is monitored management to storage server cluster,
Log server backs up the metadata of meta data server and synchronizes its operation log, when meta data server is in different
During normal state, access client can by Metadata Service process spare in log server to storage server cluster into
Row accesses, and solves the problems, such as management node server-side single-point of the prior art.
Description of the drawings
Fig. 1 is that a kind of big data based on distributed file system of the embodiment of the present invention one stores the signal of access system
Figure；
Fig. 2 is that a kind of big data based on distributed file system of the embodiment of the present invention two stores access method flow
Figure.
Specific embodiment
The principle and features of the present invention will be described below with reference to the accompanying drawings, and the given examples are served only to explain the present invention, and
It is non-to be used to limit the scope of the present invention.
Embodiment one, a kind of big data storage access system based on distributed file system.With reference to Fig. 1 to this reality
The system for applying example offer is described in detail.
In Fig. 1, a kind of big data storage access system based on distributed file system provided in this embodiment includes dividing
Cloth file system, wherein, distributed file system includes meta data server 101, storage server cluster 102, accesses visitor
Family end 103 and log server 104.
Wherein, meta data server 101 be mainly used for being managed the metadata information in distributed file system and
Monitoring is managed to storage server cluster 102.
Storage server cluster 102, which is mainly used for being stored in the data included in metadata information according to pre-defined rule, to be deposited
Store up several data memory nodes of server cluster.
Specifically, in the present embodiment, storage server cluster 102 has several management node servers, it is each to manage
If node server manages a storage server node, storage server cluster 102 is formed, carrying out distribution to big data deposits
Storage.
When it is implemented, meta data server 101 is managed the metadata information in distributed file system, and right
Storage server cluster 102 is monitored management.Meta data server 101 is according to pre-defined rule to including in metadata information
Data carry out cutting, and data are cut into several pieces, and be stored in several block numbers evidence after cutting not according to certain strategy
On same storage server node, for example, can be allocated according to the residual memory space of currently stored server, by number
On the storage server node bigger according to remaining space is stored in as possible.
Meta data server 101 is also managed storage server cluster 102 monitoring, such as to each storage server
On data storage capacity be monitored, in order to avoid the data volume stored in storage server cross ambassador's storage server collapse；Compare again
Such as, the connection status of each storage server is monitored, to understand the connection status of storage server at any time.
In addition, also virtual IP address is configured to meta data server 101 in the virtual IP address configuration module in distributed file system, visit
Ask that client 103 can log in the visit that meta data server 101 carries out storage server cluster 102 data by virtual IP address
It asks.Wherein, meta data server managing process and virtual IP address function binding, the two are stored, are migrated simultaneously simultaneously.
Log server 104 is mainly used for being spaced the metadata information in meta data server 101 to schedule
Back up the operation log of simultaneously real-time synchronization meta data server 101.
Specifically, storage server cluster 102 has carried out distributed storage to big data, and by meta data server
101 pairs of storage server clusters 102 are managed monitoring, and log server 104 believes the metadata in meta data server 101
Breath is backed up.
Access client 103 is mainly used for 101 transmission data access request of meta data server and is taken according to metadata
Be engaged in the connection status of device 101, selection by Metadata Service spare in meta data server 101 or log server 104 into
Journey accesses storage server cluster.
Specifically, when if access client 103 wishes to access to the data in storage server cluster 102, access
Client 103 can be to 101 transmission data access request of meta data server.
When meta data server 101 is online, access client 103 is configured by virtual IP address configuration module
Virtual IP address logs in meta data server 101, and then carries out data to storage server cluster 102 by meta data server 101
Access.
When meta data server 101 is in off-line state (such as suspension, delay machine or break down), by Metadata Service
Device service processes and the virtual IP address of configuration switch to log server 104 and start, by metadata spare in log server 104
The function of service processes take over meta data server 101, i.e., at this time by Metadata Service process spare in log server 104
Storage server cluster 102 is managed, access client 103 still can be by virtual IP address configuration module to metadata
The virtual IP address that server 101 is configured logs in log server 104, and then is taken by metadata spare in log server 104
Business process carries out storage server cluster 102 access of data.
The metadata in meta data server 101 is believed as previously mentioned, log server 104 can be spaced to schedule
Breath is backed up, and the operation log of meeting real-time synchronization meta data server 101.When meta data server 101 is in offline
During state, the virtual IP address of meta data server service processes and configuration is switched into log server 104 and is started.At this point, daily record
First number that spare Metadata Service process can be backed up according to log server 104 from meta data server 101 in server 104
It is believed that breath and synchronous operation log recover newest metadata information, access for access client.Log services
Spare Metadata Service process loads the metadata recovered according to the newest metadata information recovered in device 104
Data in information start managing process, entire distributed file system are managed, and access client 103 at this time can be with
Log server 104 is logged in by former virtual IP address, by the spare Metadata Service process in log server to storage service
Device cluster 102 carries out the access of data.
Embodiment two, a kind of big data storage access method based on distributed file system.With reference to Fig. 2 to this reality
The method for applying offer is described in detail.
In Fig. 2, the data included in metadata information S201, are stored in storage server cluster according to pre-defined rule
Several memory nodes, and monitoring is managed to storage server cluster by meta data server.
Specifically, only there are one management node servers for storage server cluster in the prior art, store data in and deposit
It stores up in server cluster, when access client accesses to the data in storage server cluster, is required to save by managing
Point server network interface card carries out data outflow, therefore can there are the memory spaces and bandwidth bottleneck of management node server.At this
In embodiment, storage server cluster has several management node servers, if each management node server manages a deposit
Server node is stored up, forms storage server cluster, distributed storage is carried out to big data.
When it is implemented, meta data server is managed the metadata information in distributed file system, and to depositing
Storage server cluster is monitored management.For example, metadata information (the packet in meta data server management distributed file system
Include file directory tree tissue, attribute maintenance, the mandate of the record of file operation daily record, access client access etc.), management is entire
The NameSpace of storage system is coordinated access client and the data storage and access of storage server collection cluster, is provided to name
Claim the management of the information such as space, access control information, file and memory block map information, block position.Meta data server according to
Pre-defined rule carries out cutting to the data included in metadata information, and data are cut into several pieces, and according to certain strategy
By several block numbers after cutting according to being stored on different storage server nodes, for example, can be according to currently stored server
Residual memory space be allocated, data are stored in as possible on the bigger storage server node of remaining space.
Meta data server is also managed storage server cluster monitoring, such as to the number in each storage server
It is monitored according to amount of storage, in case the data volume stored in storage server crosses the collapse of ambassador's storage server；For another example, to each
The connection status of a storage server is monitored, to understand the connection status of storage server at any time.
In addition, also virtual IP address is configured to meta data server in distributed file system, access client can be by virtual
IP logs in meta data server, and then carries out the access of data to storage server cluster by meta data server.Wherein, it is first
Data server managing process and virtual IP address function binding, the two are stored, are migrated simultaneously simultaneously.
S202, log server are spaced to schedule backs up the metadata information in meta data server,
And the operation log of real-time synchronization meta data server.
Specifically, above-mentioned steps have carried out distributed storage to big data, and by meta data server to storage service
Device cluster is managed monitoring, this step log server backs up the metadata information in meta data server.Specifically
During implementation, log server is spaced to schedule backs up the metadata information in meta data server, for example, can
The metadata information in meta data server is backed up every other hour with every.Log server is to the behaviour of meta data server
Make daily record also real-time synchronization, for example, access client has carried out storage server cluster by meta data server update behaviour
Make, operation log, these operations of log server meeting real-time synchronization meta data server can be formed in meta data server
Daily record.
S203, access client are to meta data server transmission data access request.
Specifically, after data are stored according to pre-defined rule in storage server cluster, if access client is wished
When prestige accesses to the data in storage server cluster, access client can be accessed to meta data server transmission data please
It asks.
S204, the connection status according to meta data server, access client selection pass through meta data server or daily record
Spare Metadata Service process accesses storage server cluster in server.
Specifically, distributed file system can carry out heartbeat detection in real time to meta data server, i.e., to Metadata Service
The connection status of device is monitored in real time.When meta data server is online, access client passes through distribution
The virtual IP address of file system configuration logs in meta data server, and then storage server cluster is carried out by meta data server
The access of data.When it is implemented, when meta data server receives the data access request of access client, metadata clothes
Business device the access rights of access client can be verified first, if the access authority verification of access client by when, member
Data server can match the storage server accessed for access client according to scheduled routing rule.For example, several
Access client is simultaneously to meta data server transmission data access request, and meta data server can carry out equally loaded, by number
Between being distributed in storage server cluster according to access that pressure is in admirable proportion, in order to avoid cause storage server cluster crash.
When meta data server is in off-line state (such as suspension, delay machine or break down), by meta data server
Service processes and the virtual IP address of configuration switch to log server startup, by Metadata Service process spare in log server
The function of take over meta data server, i.e., at this time by Metadata Service process spare in log server come to storage server
Cluster is managed, and access client can still be stepped on by the virtual IP address that distributed file system is configured to meta data server
Spare Metadata Service process in the log server of land, and then by spare Metadata Service process to storage server collection
Group carries out the access of data.
As previously mentioned, log server can be spaced to schedule in meta data server metadata carry out it is standby
Part, and the operation log of meeting real-time synchronization meta data server.When access client takes storage by meta data server
After business device cluster accesses, operation log can be formed in meta data server, such as to the number in storage server cluster
According to being updated, update operation log can be formed in meta data server.
When meta data server is in off-line state, meta data server service processes and the virtual IP address of configuration are switched
To daily record startup of server.At this point, Metadata Service process spare in log server can be according to log server from first number
Newest metadata information is recovered according to the metadata information of server backup and the operation log of synchronization, loads what is recovered
Newest metadata information, accesses for access client.For example, log server every 1 hour to meta data server
In metadata information backed up, and the operation log in real-time synchronization meta data server, if meta data server at this time
In off-line state, then Metadata Service process spare in log server is according to the meta data server of backup before offline
Metadata information in 1 hour and the operation log in this 1 hour, recover the metadata information in this 1 hour.
It is extensive to load this according to the newest metadata information recovered for spare Metadata Service process in log server
The metadata information appeared again starts managing process, entire distributed file system is managed, and access client at this time can be with
The access of data is carried out to storage server cluster by Metadata Service process spare in log server.
A kind of big data storage based on distributed file system provided by the invention accesses system and method, by big data
According to several memory nodes that storage server cluster is stored in after scheduled regular cutting, distributed file system point is utilized
Cloth stores the characteristics of data, and data are stored on the different memory nodes of storage server cluster, solve the prior art
The memory space problem of server disk；When access client carries out data access, data flow stores section from different data
Point outflow solves the bandwidth bottleneck of back end, also accelerates the access speed of data；Storage in storage server cluster
Interstitial content can need dynamic increase or deletion, under the premise of current service is not influenced, storage service according to specific
The memory space of device cluster can be with dynamic retractility.
Log server is increased in existing distributed file system, is managed in real-time synchronization distributed file system
The metadata information and operation log of node (alternatively meta data server), distributed file system possess using HA
(High Availability, High Availabitity) does the switching of managing process.When meta data server breaks down, suspension or machine of delaying
When, HA does the switching of managing process, and the service processes of meta data server are switched to metadata spare in log server
Service processes, the function of spare Metadata Service process take over meta data server in log server manage distributed text
Part system, access client can access storage server cluster by Metadata Service process spare in log server.
The foregoing is merely presently preferred embodiments of the present invention, is not intended to limit the invention, it is all the present invention spirit and
Within principle, any modification, equivalent replacement, improvement and so on should all be included in the protection scope of the present invention.
Claims (7)
1. a kind of big data storage access system based on distributed file system, which is characterized in that including distributed field system
System, the distributed file system include meta data server, storage server cluster, access client and log server；
The meta data server is used to be managed the metadata information in distributed file system and the storage is taken
Business device cluster is managed monitoring；
The storage server cluster is used to that the data included in metadata information to be stored in storage service according to pre-defined rule
Several data memory nodes of device cluster；
The log server is standby for being spaced progress to schedule to the metadata information in the meta data server
Part and the operation log of real-time synchronization meta data server；
The access client is used for the connection status according to meta data server, and selection is taken by meta data server or daily record
The spare Metadata Service process being engaged in device accesses storage server cluster；
The log server further includes：Data recovery module, for when meta data server is in off-line state, according to institute
Metadata information that log server backed up from meta data server and the operation log of synchronization are stated, recovers newest metadata
Information；
Wherein, the meta data server is specifically used for, and cuts the data included in metadata information according to pre-defined rule
Divide, the data after the cutting are stored in several memory nodes of storage server cluster；
Monitoring is managed to the data storage capacity and presence of each memory node of the storage server cluster.
2. big data as described in claim 1 stores access system, which is characterized in that the distributed file system is also wrapped
It includes：
Virtual IP address configuration module, for virtual IP address to be configured to the meta data server；
Access client logs in meta data server by the virtual IP address and accesses storage server cluster.
3. big data as claimed in claim 2 stores access system, which is characterized in that the distributed file system is also wrapped
It includes：
Handover module, for when meta data server is in off-line state, by meta data server service processes and virtual IP address
Switch to log server startup.
4. it is a kind of based on distributed file system big data storage access method, which is characterized in that its use claim 1 to
3 any storage access systems of the big data based on distributed file system, including：
Step S201, according to pre-defined rule by the data included in metadata information be stored in storage server cluster several
Memory node, and monitoring is managed to the metadata information and storage server cluster by meta data server；
Step S202, log server is spaced to schedule backs up the metadata information in meta data server,
And the operation log of real-time synchronization meta data server；
Step S203, access client is to meta data server transmission data access request；
Step S204, according to the connection status of meta data server, the access client selection by meta data server or
Spare Metadata Service process in log server accesses storage server cluster；
When meta data server is in off-line state, log server is according to the metadata information backed up from meta data server
And synchronous operation log, newest metadata information is recovered, access client accesses newest member by log server
Data information；
Wherein, the step S201 includes:
The data included in metadata information are subjected to cutting according to pre-defined rule, the data after the cutting are stored in storage
Several memory nodes of server cluster, and by meta data server to each memory node of the storage server cluster
Data storage capacity and presence be managed monitoring.
5. big data as claimed in claim 4 stores access method, which is characterized in that further includes：
Virtual IP address is configured for meta data server, access client is logged in by virtual IP address described in the meta data server access
Storage server cluster.
6. big data as claimed in claim 5 stores access method, which is characterized in that is further included before performing step S204：
Heartbeat detection is carried out to meta data server；
The step S204 includes：
When meta data server is online, access client logs in meta data server by virtual IP address and accesses storage
Server cluster；When meta data server is in off-line state, access client logs in log server by virtual IP address, leads to
The spare Metadata Service process crossed in log server accesses storage server cluster.
7. big data as claimed in claim 6 stores access method, which is characterized in that further includes：
When the meta data server is in off-line state, meta data server service processes and virtual IP address are switched into daily record
Startup of server, access client log in log server by virtual IP address, are taken by the spare metadata in log server
Business process accesses storage server cluster.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201410602562.3A CN104320401B (en) | 2014-10-31 | 2014-10-31 | A kind of big data storage based on distributed file system accesses system and method |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201410602562.3A CN104320401B (en) | 2014-10-31 | 2014-10-31 | A kind of big data storage based on distributed file system accesses system and method |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN104320401A CN104320401A (en) | 2015-01-28 |
| CN104320401B true CN104320401B (en) | 2018-06-22 |
Family
ID=52375571
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201410602562.3A Active CN104320401B (en) | 2014-10-31 | 2014-10-31 | A kind of big data storage based on distributed file system accesses system and method |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN104320401B (en) |
Families Citing this family (20)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104618487B (en) * | 2015-02-06 | 2018-04-20 | 新华三技术有限公司 | A kind of Internet Protocol stores online upgrade method and device |
| CN105049504B (en) * | 2015-07-09 | 2019-03-05 | 国云科技股份有限公司 | A kind of big data transfer transmission is synchronous and storage method |
| CN105389350B (en) * | 2015-10-28 | 2019-02-15 | 浪潮(北京)电子信息产业有限公司 | A kind of metadata of distributed type file system information acquisition method |
| CN105468295B (en) * | 2015-11-14 | 2019-03-05 | 华中科技大学 | A kind of security protection access method and system for realizing object storage |
| CN106815286B (en) * | 2016-12-05 | 2020-09-11 | 大唐网络有限公司 | File processing method |
| CN106648979A (en) * | 2016-12-09 | 2017-05-10 | 曙光信息产业(北京)有限公司 | Backup method and system for distributed real-time file system |
| CN108259543B (en) * | 2016-12-29 | 2021-07-06 | 广东中科遥感技术有限公司 | Distributed cloud storage database and method for deploying same in multiple data centers |
| CN107070972B (en) * | 2016-12-30 | 2020-02-07 | 中国银联股份有限公司 | Distributed file processing method and device |
| CN107229425B (en) * | 2017-06-02 | 2020-05-19 | 浙江宇视科技有限公司 | Data storage method and device |
| CN107357922A (en) * | 2017-07-21 | 2017-11-17 | 郑州云海信息技术有限公司 | A kind of NFS of distributed file system accesses auditing method and system |
| CN107659644A (en) * | 2017-09-25 | 2018-02-02 | 郑州云海信息技术有限公司 | A kind of method, apparatus for accessing storage system, readable storage medium storing program for executing |
| CN108023953B (en) * | 2017-12-04 | 2021-07-09 | 北京星选科技有限公司 | High-availability implementation method and device for FTP service |
| CN108874319B (en) * | 2018-06-27 | 2021-06-29 | 郑州云海信息技术有限公司 | Metadata updating method, device, equipment and readable storage medium |
| CN109407977B (en) * | 2018-09-25 | 2021-08-31 | 佛山科学技术学院 | Big data distributed storage management method and system |
| CN109783462B (en) * | 2018-12-13 | 2021-01-05 | 创新科技术有限公司 | Data access method and device based on distributed file system |
| CN111212111B (en) * | 2019-12-17 | 2022-05-13 | 腾讯科技（深圳）有限公司 | Object storage service management method and electronic equipment |
| CN113225362B (en) * | 2020-02-06 | 2024-04-05 | 北京京东振世信息技术有限公司 | Server cluster system and implementation method thereof |
| CN111400100B (en) * | 2020-03-16 | 2021-05-28 | 杭州坤泽实业股份有限公司 | Management method and system for distributed software backup |
| CN115242717B (en) * | 2022-09-15 | 2022-12-09 | 四川汉唐云分布式存储技术有限公司 | Centralized storage system based on end-side calculation and control method |
| CN115964353B (en) * | 2023-03-10 | 2023-08-22 | 阿里巴巴（中国）有限公司 | Distributed file system and access metering method thereof |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US6173293B1 (en) * | 1998-03-13 | 2001-01-09 | Digital Equipment Corporation | Scalable distributed file system |
| CN101997823A (en) * | 2009-08-17 | 2011-03-30 | 联想(北京)有限公司 | Distributed file system and data access method thereof |
| CN103064927A (en) * | 2012-12-21 | 2013-04-24 | 曙光信息产业(北京)有限公司 | Data access method and device of distributed file system |
| CN103095758A (en) * | 2011-11-01 | 2013-05-08 | 北大方正集团有限公司 | Distributed file system and file data processing method thereof |
| CN103647797A (en) * | 2013-11-15 | 2014-03-19 | 北京邮电大学 | Distributed file system and data access method thereof |
-
2014
- 2014-10-31 CN CN201410602562.3A patent/CN104320401B/en active Active
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US6173293B1 (en) * | 1998-03-13 | 2001-01-09 | Digital Equipment Corporation | Scalable distributed file system |
| CN101997823A (en) * | 2009-08-17 | 2011-03-30 | 联想(北京)有限公司 | Distributed file system and data access method thereof |
| CN103095758A (en) * | 2011-11-01 | 2013-05-08 | 北大方正集团有限公司 | Distributed file system and file data processing method thereof |
| CN103064927A (en) * | 2012-12-21 | 2013-04-24 | 曙光信息产业(北京)有限公司 | Data access method and device of distributed file system |
| CN103647797A (en) * | 2013-11-15 | 2014-03-19 | 北京邮电大学 | Distributed file system and data access method thereof |
Also Published As
| Publication number | Publication date |
|---|---|
| CN104320401A (en) | 2015-01-28 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN104320401B (en) | A kind of big data storage based on distributed file system accesses system and method | |
| CN102411637B (en) | Metadata management method of distributed file system | |
| CN105389230B (en) | A kind of continuous data protection system and method for combination snapping technique | |
| WO2019154394A1 (en) | Distributed database cluster system, data synchronization method and storage medium | |
| CN105335251B (en) | A kind of fault recovery method and system | |
| CN105095317B (en) | Distributed data base service management system | |
| CN109729129A (en) | Configuration modification method, storage cluster and the computer system of storage cluster | |
| US20140164329A1 (en) | Dynamically Varying the Number of Database Replicas | |
| CN103929500A (en) | Method for data fragmentation of distributed storage system | |
| CN105471960A (en) | Information interaction system and method between private clouds and public cloud | |
| US10652100B2 (en) | Computer system and method for dynamically adapting a software-defined network | |
| CN103138912B (en) | Method of data synchronization and system | |
| CN109218100A (en) | Distributed objects storage cluster and its request responding method, system and storage medium | |
| EP3039844B1 (en) | System and method for supporting partition level journaling for synchronizing data in a distributed data grid | |
| CN103186554A (en) | Distributed data mirroring method and data storage node | |
| CN104994168A (en) | distributed storage method and distributed storage system | |
| CN103973725A (en) | Distributed collaboration method and collaboration device | |
| CN103795801A (en) | Metadata group design method based on real-time application group | |
| CN113268472B (en) | Distributed data storage system and method | |
| CN105635311A (en) | Method for synchronizing resource pool information in cloud management platform | |
| CN105262640A (en) | System and method for improving reliability of cloud platform server, and disposition framework of system | |
| CN102546776A (en) | Method for realizing off-line reading files in SAN (Storage Area Networking) shared file system | |
| CN103544322A (en) | Hotspot metadata management method based on server cluster | |
| CN105404565B (en) | A kind of dual-active data guard method and device | |
| CN104539982A (en) | Method and system for dispatching of video resource point-to-point sharing and node terminal |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |