CN107203639A - Parallel file system based on High Performance Computing - Google Patents
Parallel file system based on High Performance Computing Download PDFInfo
- Publication number
- CN107203639A CN107203639A CN201710434507.1A CN201710434507A CN107203639A CN 107203639 A CN107203639 A CN 107203639A CN 201710434507 A CN201710434507 A CN 201710434507A CN 107203639 A CN107203639 A CN 107203639A
- Authority
- CN
- China
- Prior art keywords
- file system
- data
- client
- mds
- hpc
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/10—File systems; File servers
- G06F16/18—File system types
- G06F16/1858—Parallel file systems, i.e. file systems supporting multiple processors
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Data Mining & Analysis (AREA)
- Databases & Information Systems (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The invention discloses a kind of parallel file system for being based on high-performance calculation (HPC), including some meta data servers (MDS), multiple object storage servers (OSS) and client (Client).The MDS, for being responsible for Metadata Service, while managing the NameSpace of whole file system；For one MDT of share and access between multiple MDS；Each MDT is used to preserve file metadata object；Using the Client by MDS reading and savings in MDT metadata.The OSS, for the interaction being responsible between client and physical store and data storage, is provided out input/output (I/O) interface of data；Each OSS manages one or more OST, data storage object.The Client, for accessing the file data being stored on OST by OSS；By the arbitrary node of Client carry file system, realization can be with POSIX file system interfaces, and user pellucidly accesses the data of whole file system by Client.
Description
Technical field
High-performance calculation is based on the present invention relates to computer file system design and administrative skill, more particularly to one kind
The parallel file system of (High Performance Computing, HPC) technology.
Background technology
The continuous expansion and the appearance of high-performance calculation (HPC) technology applied with supercomputing, supercomputing is pushed to
Every field, the high performance overall target of pursuit is changed into by pursuing supercomputing single parameter.High-performance calculation
It is also applied to numerical computations and data processing.With the maturation of database technology, the development of the technology such as biological gene, historical data
Geometry level expansion supercomputing is pushed to data processing field.
The fast development of hardware technology promotes supercomputing and moves towards high-end calculating, but the slow development of software engineering is then
It has impact on the further development of high-end calculating.In high-performance computing sector, relative to hardware technology, concurrent software technology decades
All there is no breakthrough progress.Especially distributed computing technology, automatic paralleling identification technology, parallel file system skill
Art etc..
With being skyrocketed through for data volume, the formation speed of data is increasingly faster, it is necessary to higher and higher bandwidth, increasingly
Big storage size, the need for single file system can not meet current high performance calculating.
Existing HPCC parallel file system, such as Lustre file system, can at most support 100000
Individual client (Client), 1000 object storage servers (OSS) and 2 meta data server (MDS) nodes.Lustre systems
1~3 functional module can be run in system simultaneously.But Lustre is generally operated on high-performance computer system, in order to
The performance of Lustre file system is improved, usual MDS, OSS are opened with Client points and operated on the different nodes of Lustre.
The metadata storage and management strategy of Lustre file system is：MDS is responsible for metadata, and there is provided an overall situation
NameSpace, Client can read the metadata being stored on metadata target device (MDT) by MDS.
MDS can have 2 in Lustre, employ Active-Standby fault tolerant mechanism, when one of MDS cisco unity malfunctions
When, another standby MDS can start service.MDT can only have the same MDT of share and access between 1, different MDS.
Its file data storage and management strategy is：OSS loads provide input/output (I/O) service, receive and service to come
The request of automatic network.The file data being stored on object storage target device (OST) can be had access to by OSS.One OSS
2 to 8 OST of correspondence.File data on OST is preserved in the form of slitting, the slitting of file can an OSS it
In, it can also be stored in multiple OSS.Lustre's has one characteristic that its data is object-based function storage, with passing
The block-based storage mode of system is different.
Lustre parallel file systems access system by Client ends, the Client Lustre file system that has been carry
Arbitrary node.Client provides the interface between Virtual File System (VFS) and Lustre systems under Linux, passes through
File in Client, user-accessible operation Lustre systems.
Although the performance and scalability of Lustre file system are all good, but also possess object-based intellectuality and deposit
The advantages of storage, the authentication mechanism of safety, fairly perfect fault tolerant mechanism, and Lustre also achieves file lock function.But
Lustre needs the support of special hardware equipment, and Lustre does not realize MDS cluster management also at present, although Lustre
Double MDS are in availability or improve many, but after system reaches certain scale, MDS still probably turns into
Bottleneck in Lustre file system；In addition, Lustre file system is also not carried out visualized graph interface, its file is used
System is not directly perceived enough.
The content of the invention
In view of this, it is a primary object of the present invention to provide a kind of parallel file system for being based on high-performance calculation (HPC)
System, to support to build ultra-large storage cluster based on cheap SATA/SAS disc drivers, overcomes existing parallel file system
System needs the deficiency that specialised hardware equipment is supported；Support the distributed file system stored based on object；Support most of high speeds
Network type；Handling capacity, scalability and high-performance are further improved, so that effective data management mechanism is realized, global number
Substituted and system rapid configuration according to shared, failure.
Another object of the present invention is to provide a kind of high-performance calculation (HPC) parallel file system, pass through integrated collection
Group's management module, supports the read-write process of WEB interface Dynamic Announce parallel file system storage cluster.
To reach above-mentioned purpose, the technical proposal of the invention is realized in this way：
A kind of parallel file system based on high-performance calculation HPC, including some meta data server MDS, multiple objects
Storage server OSS and multiple client Client；Also include metadata target device MDT and object storage target device OST；
Wherein：
The MDS, for being responsible for Metadata Service, while managing the NameSpace of whole file system；For multiple MDS
Between one MDT of share and access；Each MDT is used to preserve file metadata object；Read and protected by MDS using the Client
It is stored in MDT metadata；
The OSS, for the interaction being responsible between client and physical store and data storage, is provided out the defeated of data
Enter/export I/O interfaces；Each OSS manages one or more OST, data storage object；
The Client, for accessing the file data being stored on OST by OSS；Client carries file system
Arbitrary node, realization can be with POSIX file system interfaces, and user pellucidly accesses the number of whole file system by Client
According to.
Wherein, described HPC parallel file systems, be at best able to support 100000 client Client, 1000 it is right
As storage server OSS and 2 meta data server MDS.
Described parallel file system, connects multiple meta data server MDS, metadata target by gigabit Ethernet and sets
Standby MDT, object storage server OSS and object storage target device (OST).
Described parallel file system supports Infiniband architecture.
Described parallel file system is integrated with some drive modules；The drive module include intel network interface cards,
Mellanox network interface cards and LSI RAID cards, support to build based on cheap SATA/SAS disc drivers using the drive module
Ultra-large storage cluster.
Described parallel file system is integrated with HPC cluster management module, passes through a meta data server of service end
MDS is collected into the monitoring data of all clients in the same network segment.
Described HPC cluster management module includes monitoring program module, and the monitoring program module further comprises
Ltmetad program modules, ltmond monitoring program modules and PHP web front end modules；Wherein：
The ltmetad program modules, the operation each child of ltmetad process periodic pollings is the data of ltmond nodes
XML data that resource, parsing are collected into, storage numerical value, and are stored in rotation database by volatile data by RRD instruments
In RRD, and the XML data being collected into is transmitted to client；
The ltmond monitoring program modules, for monitoring the change of local state, and report the change of correlation, pass through list
Broadcast or multicast mode monitors the state of other ltmond nodes for running ltmond processes, and cluster state request is answered
Answer, the request is described by XML；
The PHP web front ends module, for providing Visual Dynamic clustering functionality, to realize effective monitoring cluster shape
State.
The ltmetad program modules periodically go ltmond nodes or ltmetad node poll data.
The parallel file system based on high-performance calculation (HPC) of the present invention, has the advantages that：
Using the parallel file system, it can support to build super large rule based on cheap SATA/SAS disc drivers
Mould storage cluster, the defect for overcoming existing parallel file system to need specialised hardware equipment to support；Support what is stored based on object
Distributed file system；Support most of express network types；Handling capacity, scalability and high-performance are further improved, so that
Effective data management mechanism is realized, global data is shared, failure is substituted and system rapid configuration.
In addition, the parallel file system is by integrated cluster management module, support to store with WEB interface form Dynamic Announce
The read-write situation of cluster, it is more directly perceived using process.
Brief description of the drawings
Fig. 1 is the parallel file system configuration diagram that the embodiment of the present invention is based on high-performance calculation (HPC)；
Fig. 2 is parallel file system schematic diagram of the embodiment of the present invention based on HPC；
Fig. 3 is parallel file system cluster module monitoring configuration diagram of the embodiment of the present invention based on HPC；
Fig. 4 is parallel file system cluster module monitoring structure schematic diagram of the embodiment of the present invention；
Fig. 5 is the process schematic of parallel file system cluster module collection monitoring data shown in Fig. 4；
Fig. 6 is parallel file system cluster monitoring software interface schematic diagram of the embodiment of the present invention based on HPC.
Embodiment
Below in conjunction with the accompanying drawings and embodiments of the invention to the present invention be based on high-performance calculation (HPC) parallel file system
It is described in further detail.
High-performance calculation (HPC), refers to use in multiple processors (as a part for individual machine) or a certain cluster
The computing system and environment of several computers (being operated as single computing resource) of tissue.High-performance calculation (HPC) is simultaneously composed a piece of writing
Part system is the NFS that value is applied to multi-machine surroundings, and the data of single file deposit in difference using forms such as slittings
Input/output (I/O) node on, the concurrent access of the multiple threads of multimachine is supported, while supporting point of metadata and data
Cloth is deposited, and provides single directory space.
Fig. 1 is the parallel file system configuration diagram that the embodiment of the present invention is based on high-performance calculation (HPC).
As shown in figure 1, the HPC parallel file systems, are mainly deposited including some meta data servers (MDS), multiple objects
Store up server (OSS) and multiple client (Client).Also include metadata target device (MDT) and object storage target device
(OST)。
The HPC parallel file systems, can at most support 100000 clients (Client), 1000 object storage clothes
Business device (OSS) and 2 meta data servers (MDS).Wherein：
The MDS, for being responsible for Metadata Service, while managing the NameSpace of whole file system；For multiple MDS
Between one MDT of share and access；Each MDT is used to preserve file metadata object, such as file name, bibliographic structure and access
Authority.The Client is then by MDS reading and savings in MDT metadata.
The OSS, for the interaction being responsible between client and physical store and data storage, is provided out the IO of data
Interface.Each OSS manages one or more OST, data storage object.
The Client, for accessing the file data being stored on OST by OSS；Client carries file system
Arbitrary node, realizing can be with the transparent whole file system of access by Client with POSIX file system interfaces, user
The data of system.
The above-mentioned HPC parallel file systems of the embodiment of the present invention, the file data placement strategy of use is as follows：
1) each regular file, catalogue, symbol connection and special file have a unique inode, are used as file member
Data object；
2) file data is stored in several OST storage object according to certain band pattern distribution；
3) point band size of file, storage object number, the location information, all conduct such as corresponding OST indexes of banding pattern
Data distribution attributes object is stored in metadata inode extended attribute.
The above-mentioned HPC parallel file systems of the embodiment of the present invention, the method for distributing metadata of use is as follows：
One is catalogue subtree zone method, and two be pure Hash method.Wherein：
The catalogue subtree zone method is specifically included：
1) NameSpace is divided into different catalogues, subtree；
2) the corresponding metadata of each catalogue subtree is managed by same meta data server；
3) meta data server can also manage multiple catalogue subtrees；
4) it is exactly an installable small documents system per catalogue subtree.
The pure Hash method, is specifically included：
When client creates a file, with the identifier (or pathname) of file for key assignments (Key), pass through Hash
Function selects the meta data server of establishment file.
Fig. 2 is parallel file system schematic diagram of the embodiment of the present invention based on HPC.HPC parallel files system as shown in Figure 2
System, multiple meta data servers (MDS), metadata target device (MDT), object storage server are connected by gigabit Ethernet
(OSS) and object storage target device (OST).
The HPC parallel file systems also support Infiniband architecture.The InfiniBand, also known as infinite bandwidth skill
Art, for the connectivity problem at settlement server end, such as distributed connection.Infiniband architecture is a kind of many concurrent chains of support
" Convertion cable " technology connect, in the art, every kind of link can reach the 2.5Gbps speed of service.The framework is one
Speed is 500MB/ seconds when individual link, and speed is 2GB/ second when four links, and 12 speed when link can be with
Reach 6GB/ seconds.
The HPC parallel file systems, are integrated with substantial amounts of drive module in program module, such as intel network interface cards,
Mellanox network interface cards, LSI RAID cards etc., can support to build based on cheap SATA/SAS disc drivers ultra-large
Storage cluster, the defect for overcoming existing parallel file system to need specialised hardware equipment to support.
Fig. 3 is parallel file system cluster module monitoring configuration diagram of the embodiment of the present invention based on HPC, and Fig. 4 is this
Parallel file system cluster module monitoring overall structure diagram of the inventive embodiments based on HPC.
Parallel file system of the invention based on HPC is integrated with HPC cluster management module, the HPC cluster management module energy
Enough curve maps show nearest a hour, nearest one day, nearest one week, the parallel file system in even nearest 1 year nearest January
The indexs such as cpu load situation, internal memory service condition, network condition and the hard disk occupancy of system (MDS, OSS and Client).
The HPC cluster management module, is collected in the same network segment by a meta data server (MDS) of service end
All clients (Client) monitoring data.
Here, the operation principle of described HPC cluster management module is：The HPC cluster management module, runs multiple
Ltmond processes and a main ltmetad process, using whole ltmond process collection monitoring data, and by the prison being collected into
Data summarization is controlled to ltmetad managements of process end program.The ltmetad managements of process end program is by the supervising data storage
Into rotation database (Round Robin Database, RRD), finally by PHP programs (such as Apache PHP front ends journey
Sequence) it is shown on Web browser interface and (refers to Fig. 5).
Utilized as shown in figure 4, illustrating meta data server (MDS) by running ltmetad managements of process end program
Ltmond processes from object storage server 1 (OSS1), OSS2 ..., OSSN and client Client1, Client2 ...,
The process of ClientN collection monitoring data.
Fig. 5 is the process schematic of parallel file system cluster management module collection monitoring data shown in Fig. 4.Such as Fig. 5 institutes
Show, the process of the collection monitoring data is mainly completed by the monitoring program module of parallel file system cluster management module,
Its operation principle is as follows：
Ltmetad program modules, the data money of the operation each child of ltmetad process periodic pollings (ltmond nodes)
XML data that source, parsing are collected into, storage numerical value, and are stored in rotation data by volatile data by RRD instruments (Tool)
In storehouse (RRD), and the XML data being collected into is transmitted to client.XML data source includes ltmond monitoring program modules
(operation ltmond processes), its data source is to realize positioning by IP address.
Using the ltmond monitoring program modules, the change of local state is monitored, and reports the change of correlation, passes through list
Broadcast or multicast mode monitors the state of other ltmond nodes for running ltmond processes, and cluster state request is answered
Answer, the request is described by XML.
Clustering functionality is monitored there is provided Visual Dynamic by PHP web front ends module.Carried using PHP web front end modules
It is a kind of method for allowing system manager to realize effective monitoring cluster state for Visual Dynamic cluster monitoring.It is not only by
XML data is showed in the way of html page, and can show the historical data of system in many ways.The reality of the function
Ltmetad program modules are now depended on, various data are provided for the PHP web front ends module using ltmetad program modules.
Specifically, in the present embodiment, parallel file system monitoring program module of the invention under default situations is by before PHP Web
End receives the data using 9651 ports.
As shown in figure 4, ltmetad program modules periodically go ltmond nodes or ltmetad node poll data.
One ltmetad node can set multiple data sources (Datasource), and each Datasource can have multiple backups, one
Individual failure can also go other main frames (Host) access evidence.
Here, the poll data refer to perform poll () function operation, and effect is to be suspended to current file pointer to set
In the waiting list that standby inside is defined.The poll () function is similar with the effect of select () function, but the former is more efficient.
When return on the occasion of when, represent the number for the filec descriptor for meeting response events, represent and do not have at the appointed time if returning to 0
Event occurs.Errno should be checked immediately if finding to be returned as bearing, because this is represented, an error has occurred.
With reference to Fig. 5, multiple ltmond nodes by destination host (Host) from UDP (UDP) to unicast
Ltmond nodes send data, and ltmetad nodes ask XML file to target Host ltmond, are then stored in RRDTool
Database in.In Fig. 5, the ltmetad nodes, the ltmond nodes being connected with the ltmetad nodes, RRDTool,
PHP script and Apache PHP front ends are the nodes positioned at meta data server (MDS).The node is responsible for collecting storage
The status information of the monitored each node of display.
Fig. 6 is parallel file system cluster monitoring software interface schematic diagram of the embodiment of the present invention based on HPC.Such as Fig. 6 institutes
Show, user utilize HPC cluster management module, by PHP programs can be checked at Web browser interface a nearest hour,
Nearest one day, nearest one week, the CPU of the HPC parallel file systems (MDS, OSS and Client) in even nearest 1 year nearest January
The indexs such as loading condition, internal memory service condition, network condition and hard disk occupancy.It is achieved thereby that visualized graphs are managed,
The operation conditions of the parallel file system can intuitively be checked.
The foregoing is only a preferred embodiment of the present invention, is not intended to limit the scope of the present invention.
Claims (8)
1. a kind of parallel file system based on high-performance calculation HPC, it is characterised in that including some meta data server MDS,
Multiple object storage server OSS and multiple client Client；Also include metadata target device MDT and object storage target
Equipment OST；Wherein：
The MDS, for being responsible for Metadata Service, while managing the NameSpace of whole file system；For between multiple MDS
One MDT of share and access；Each MDT is used to preserve file metadata object；Using the Client by MDS reading and savings in
MDT metadata；
The OSS, for the interaction being responsible between client and physical store and data storage, be provided out the inputs of data/
Export I/O interfaces；Each OSS manages one or more OST, data storage object；
The Client, for accessing the file data being stored on OST by OSS；Client carries file system is appointed
Meaning node, realization can be with POSIX file system interfaces, and user pellucidly accesses the data of whole file system by Client.
2. the parallel file system based on HPC according to claim 1, it is characterised in that the described HPC parallel files system
System, is at best able to support 100000 client Client, 1000 object storage server OSS and 2 meta data servers
MDS。
3. the parallel file system based on HPC according to claim 1, it is characterised in that described parallel file system, leads to
Cross gigabit Ethernet and connect multiple meta data server MDS, metadata target device MDT, object storage server OSS and object
Store target device (OST).
4. the parallel file system based on HPC according to claim 1, it is characterised in that described parallel file system branch
Hold Infiniband architecture.
5. the parallel file system based on HPC according to claim 1, it is characterised in that described parallel file system collection
Into there is some drive modules；The drive module includes intel network interface cards, Mellanox network interface cards and LSI RAID cards, is driven using described
Dynamic model block is supported to build ultra-large storage cluster based on cheap SATA/SAS disc drivers.
6. the parallel file system based on HPC according to claim 1, it is characterised in that described parallel file system collection
Into there is HPC cluster management module, all visitors in the same network segment are collected into by a meta data server MDS of service end
The monitoring data at family end.
7. the parallel file system based on HPC according to claim 6, it is characterised in that described HPC cluster management module
Including monitoring program module, the monitoring program module further comprises ltmetad program modules, ltmond monitoring program modules
With PHP web front end modules；Wherein：
The ltmetad program modules, the operation each child of ltmetad process periodic pollings is the data money of ltmond nodes
XML data that source, parsing are collected into, storage numerical value, and volatile data is stored in rotation database RRD by RRD instruments
In, and the XML data being collected into is transmitted to client；
The ltmond monitoring program modules, for monitoring the change of local state, and report the change of correlation, by unicast or
Multicast mode monitors the state of the ltmond nodes of other operation ltmond processes, and carries out response to cluster state request,
The request is described by XML；
The PHP web front ends module, for providing Visual Dynamic clustering functionality, to realize effective monitoring cluster state.
8. the parallel file system based on HPC according to claim 7, it is characterised in that the ltmetad program modules week
Go to phase property ltmond nodes or ltmetad node poll data.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710434507.1A CN107203639A (en) | 2017-06-09 | 2017-06-09 | Parallel file system based on High Performance Computing |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710434507.1A CN107203639A (en) | 2017-06-09 | 2017-06-09 | Parallel file system based on High Performance Computing |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN107203639A true CN107203639A (en) | 2017-09-26 |
Family
ID=59907485
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201710434507.1A Pending CN107203639A (en) | 2017-06-09 | 2017-06-09 | Parallel file system based on High Performance Computing |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN107203639A (en) |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109871363A (en) * | 2019-02-28 | 2019-06-11 | 苏州浪潮智能科技有限公司 | A kind of shared-file system and its building method of redundancy structure |
| CN110012080A (en) * | 2019-03-21 | 2019-07-12 | 新华三技术有限公司 | Data processing method |
| CN111444157A (en) * | 2019-01-16 | 2020-07-24 | 阿里巴巴集团控股有限公司 | Distributed file system and data access method |
| CN111522514A (en) * | 2020-04-27 | 2020-08-11 | 上海商汤智能科技有限公司 | Cluster file system, data processing method, computer device and storage medium |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102833580A (en) * | 2012-09-19 | 2012-12-19 | 南京斯坦德通信股份有限公司 | High-definition video application system and method based on infiniband |
| US20160026553A1 (en) * | 2014-07-22 | 2016-01-28 | Cray Inc. | Computer workload manager |
| US9454331B1 (en) * | 2013-06-27 | 2016-09-27 | Emc Corporation | Cluster file system with a burst buffer appliance for coordinated control of data movement among storage tiers across a plurality of object storage servers |
| CN106227839A (en) * | 2016-07-26 | 2016-12-14 | 浪潮电子信息产业股份有限公司 | The expansion method of a kind of lustre file system and device |
-
2017
- 2017-06-09 CN CN201710434507.1A patent/CN107203639A/en active Pending
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102833580A (en) * | 2012-09-19 | 2012-12-19 | 南京斯坦德通信股份有限公司 | High-definition video application system and method based on infiniband |
| US9454331B1 (en) * | 2013-06-27 | 2016-09-27 | Emc Corporation | Cluster file system with a burst buffer appliance for coordinated control of data movement among storage tiers across a plurality of object storage servers |
| US20160026553A1 (en) * | 2014-07-22 | 2016-01-28 | Cray Inc. | Computer workload manager |
| CN106227839A (en) * | 2016-07-26 | 2016-12-14 | 浪潮电子信息产业股份有限公司 | The expansion method of a kind of lustre file system and device |
Non-Patent Citations (1)
| Title |
|---|
| 张轶彬: "分布式文件系统客户端的设计与实现", 《中国优秀硕士学位论文全文数据库信息科技辑》 * |
Cited By (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111444157A (en) * | 2019-01-16 | 2020-07-24 | 阿里巴巴集团控股有限公司 | Distributed file system and data access method |
| CN111444157B (en) * | 2019-01-16 | 2023-06-20 | 阿里巴巴集团控股有限公司 | Distributed file system and data access method |
| CN109871363A (en) * | 2019-02-28 | 2019-06-11 | 苏州浪潮智能科技有限公司 | A kind of shared-file system and its building method of redundancy structure |
| CN110012080A (en) * | 2019-03-21 | 2019-07-12 | 新华三技术有限公司 | Data processing method |
| CN111522514A (en) * | 2020-04-27 | 2020-08-11 | 上海商汤智能科技有限公司 | Cluster file system, data processing method, computer device and storage medium |
| CN111522514B (en) * | 2020-04-27 | 2023-11-03 | 上海商汤智能科技有限公司 | Cluster file system, data processing method, computer equipment and storage medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN108885582B (en) | Multi-tenant memory services for memory pool architecture | |
| US10089307B2 (en) | Scalable distributed data store | |
| Hupfeld et al. | The XtreemFS architecture—a case for object‐based file systems in Grids | |
| US10394611B2 (en) | Scaling computing clusters in a distributed computing system | |
| US9081837B2 (en) | Scoped database connections | |
| US10922303B1 (en) | Early detection of corrupt data partition exports | |
| EP3398132A1 (en) | Distributed computing dependency management system | |
| US10860604B1 (en) | Scalable tracking for database udpates according to a secondary index | |
| US10127077B2 (en) | Event distribution pattern for use with a distributed data grid | |
| CN103595799B (en) | A kind of method realizing distributed shared data storehouse | |
| US11157455B2 (en) | Inofile management and access control list file handle parity | |
| US10852985B2 (en) | Persistent hole reservation | |
| CN103166785A (en) | Distributed type log analysis system based on Hadoop | |
| CN107203639A (en) | Parallel file system based on High Performance Computing | |
| US11151162B2 (en) | Timestamp consistency for synchronous replication | |
| US20120197840A1 (en) | Push replication for use with a distributed data grid | |
| CA3109311C (en) | Aggregated service status reporter | |
| US11144407B1 (en) | Synchronous database geo-mirroring using delayed visibility write operations | |
| CN202872848U (en) | Cloud storage terminal equipment based on cloud information and cloud computing services | |
| US9231957B2 (en) | Monitoring and controlling a storage environment and devices thereof | |
| Chen et al. | Big data storage architecture design in cloud computing | |
| Chakraborty et al. | A proposal for high availability of HDFS architecture based on threshold limit and saturation limit of the namenode | |
| CN113094354A (en) | Database architecture method and device, database all-in-one machine and storage medium | |
| US11757703B1 (en) | Access requests processing and failover handling across multiple fault tolerance zones | |
| Ebbers et al. | Implementing IBM InfoSphere BigInsights on IBM System X |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |