CN112230862A - Temporary accelerated storage method and system for I/O intensive operation - Google Patents
Temporary accelerated storage method and system for I/O intensive operation Download PDFInfo
- Publication number
- CN112230862A CN112230862A CN202011156404.1A CN202011156404A CN112230862A CN 112230862 A CN112230862 A CN 112230862A CN 202011156404 A CN202011156404 A CN 202011156404A CN 112230862 A CN112230862 A CN 112230862A
- Authority
- CN
- China
- Prior art keywords
- storage system
- temporary
- acceleration
- job
- storage
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 25
- 230000001133 acceleration Effects 0.000 claims abstract description 140
- 230000008901 benefit Effects 0.000 abstract description 5
- 238000012545 processing Methods 0.000 description 7
- 230000008676 import Effects 0.000 description 4
- 230000002159 abnormal effect Effects 0.000 description 2
- 210000001072 colon Anatomy 0.000 description 2
- 238000011161 development Methods 0.000 description 2
- 238000010586 diagram Methods 0.000 description 2
- 241000282414 Homo sapiens Species 0.000 description 1
- 238000004364 calculation method Methods 0.000 description 1
- 238000004891 communication Methods 0.000 description 1
- 238000010276 construction Methods 0.000 description 1
- 230000006378 damage Effects 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 238000009434 installation Methods 0.000 description 1
- 238000011160 research Methods 0.000 description 1
- 238000004904 shortening Methods 0.000 description 1
- 230000009466 transformation Effects 0.000 description 1
- 238000010200 validation analysis Methods 0.000 description 1
- 238000012795 verification Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0602—Interfaces specially adapted for storage systems specifically adapted to achieve a particular effect
- G06F3/061—Improving I/O performance
- G06F3/0611—Improving I/O performance in relation to response time
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0628—Interfaces specially adapted for storage systems making use of a particular technique
- G06F3/0629—Configuration or reconfiguration of storage systems
- G06F3/0631—Configuration or reconfiguration of storage systems by allocating resources to storage systems
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0668—Interfaces specially adapted for storage systems adopting a particular infrastructure
- G06F3/0671—In-line storage system
- G06F3/0673—Single storage device
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0668—Interfaces specially adapted for storage systems adopting a particular infrastructure
- G06F3/0671—In-line storage system
- G06F3/0673—Single storage device
- G06F3/0679—Non-volatile semiconductor memory device, e.g. flash memory, one time programmable memory [OTP]
Abstract
The invention provides a temporary accelerated storage method for I/O intensive operation, which relates to the technical field of storage and comprises the following steps: constructing a storage acceleration resource demand description framework of a system and a user layer; receiving a job script compiled by a user according to a storage acceleration resource description framework and a job management specification; constructing a temporary acceleration storage system according to a script submitted by a user; importing files or directories required by the operation from the low-speed storage system to the temporary acceleration storage system; mounting the temporary acceleration storage system and starting operation; and exporting the data generated by the operation from the temporary acceleration storage system, and destroying the temporary acceleration storage system. According to the invention, the temporary acceleration storage system is constructed when the operation is submitted, and is destroyed in time when the operation is finished and quitted, so that resources are released, the execution time of I/O intensive operation is obviously shortened, the operation efficiency of user operation is improved, and the use benefit of a high-performance computer is improved.
Description
Technical Field
The invention relates to the technical field of storage, in particular to a temporary accelerated storage method and a temporary accelerated storage system for I/O intensive operation.
Background
The high-performance computing system is used as the third scientific research method for understanding the world transformation world by human beings after theoretical science and experimental science, the application field is more and more extensive, and the performance requirement is more and more high.
Current high performance computing is developing abnormally rapidly and is more and more data centric.
Compared with the rapid development of computing performance, the development of storage performance is relatively lagged, and in consideration of factors such as high-capacity demand and price, most high-performance computers still adopt a mechanical hard disk as a main storage system (hereinafter referred to as a low-speed storage system) at present, and such a storage system is not suitable for I/O intensive operation, and especially when hundreds of various applications compete for I/O resources at the same time, the I/O factors can significantly increase the operation running time. In recent years, the storage field is gradually increased, and a plurality of fast storage devices such as SSD, NVRAM SSD, NVDIMM and the like appear, but due to their high price, a high-performance computing system cannot fully deploy such fast storage devices, and generally only a small part of the fast storage devices are used as a cache of the storage system, so that limited I/O acceleration support is provided for all jobs (including non-I/O-intensive jobs).
Disclosure of Invention
The temporary accelerated storage method and the temporary accelerated storage system for the I/O intensive operation, which are provided by the invention, are used for overcoming the defects of a storage system in high-performance calculation in the prior art, obviously shortening the execution time of the I/O intensive operation, improving the operation efficiency of user operation and improving the use benefit of a high-performance computer.
In order to solve the technical problems, the technical scheme provided by the invention is as follows:
the invention provides a temporary acceleration storage method for I/O intensive operation, which comprises the following steps: constructing a storage acceleration resource demand description framework of a system and a user layer;
receiving a job script compiled by a user according to a storage acceleration resource description framework and a job management specification;
constructing a temporary acceleration storage system according to a script submitted by a user;
importing files or directories required by the operation from the low-speed storage system to the temporary acceleration storage system;
mounting the temporary acceleration storage system and starting operation;
and exporting the data generated by the operation from the temporary acceleration storage system, and destroying the temporary acceleration storage system.
The invention provides a temporary acceleration storage method for I/O intensive operation, preferably, the script at least comprises: the number of nodes required for the I/O intensive job, the number of CPU cores, the temporary accelerated storage system capacity, and the file or directory information used by the job (the file or directory information is already stored in the existing low speed storage system).
The invention provides a temporary accelerated storage method for I/O intensive operation, which preferably comprises a KV storage system;
the "building a temporary acceleration storage system according to a script submitted by a user" specifically includes:
judging whether the number of the nodes, the number of the CPU cores and the resources for temporarily accelerating the capacity of the storage system meet the resource requirements in the script submitted by the user or not;
if the resource meets the requirement, the resource requirement in the script is reserved and configured;
and registering the resource requirements in the script into the KV storage system, and constructing a temporary acceleration storage system.
The temporary acceleration storage method for I/O intensive jobs provided by the present invention preferably includes the following specific steps:
installing a temporary acceleration storage system client on a required computing node;
and running the job in the temporary acceleration storage system, and storing data generated by the job into the temporary acceleration system.
The temporary acceleration storage method for I/O intensive jobs provided by the present invention preferably includes the following steps:
unloading the temporary accelerated storage system client on the corresponding computing node;
releasing the computing resources;
locking the temporary acceleration storage system;
exporting data generated by the job to a low-speed storage system;
the temporary accelerated storage system is destroyed.
The temporary acceleration storage system for I/O intensive operation preferably comprises a KV storage system, a storage acceleration system, an operation management system, a tool module and a program; the storage acceleration system comprises a storage acceleration system client, a storage acceleration system server and a tool module;
the program specifically comprises:
the job management system receives a job script compiled by a user according to the storage acceleration resource description framework and the job management specification;
constructing a temporary acceleration storage system according to a script submitted by a user;
importing files or directories required by the operation from the low-speed storage system to the temporary acceleration storage system;
mounting the temporary acceleration storage system and starting operation;
and exporting the data generated by the operation from the temporary acceleration storage system, and destroying the temporary acceleration storage system.
The invention provides an I/O-intensive job-oriented temporary acceleration storage system, preferably, the script at least comprises: the number of nodes required for the I/O intensive job, the number of CPU cores, the temporary accelerated storage system capacity, and the file or directory information used by the job (the file or directory information is already stored in the existing low speed storage system).
The temporary acceleration storage system for I/O intensive operation, provided by the invention, preferably comprises a KV storage system;
the "building a temporary acceleration storage system according to a script submitted by a user" specifically includes:
the operation management system judges whether the number of the nodes, the number of the CPU cores and the resources for temporarily accelerating the capacity of the storage system meet the resource requirements in the script submitted by the user, and if the resources meet the requirements, the resource requirements in the script are reserved and configured;
the operation management system calls a storage acceleration system client to register the resource requirement in the script in the KV storage system;
and after the server side of the storage acceleration system acquires the resource requirement registered in the KV storage system, calling a tool module to construct a temporary acceleration storage system.
The temporary acceleration storage system for I/O intensive operations provided by the present invention preferably, the "mounting the temporary acceleration storage system, starting the operation" specifically includes:
the tool module installs a temporary acceleration storage system client on a required computing node;
and running the job in the temporary acceleration storage system, and storing data generated by the job into the temporary acceleration system.
The temporary acceleration storage method for I/O intensive jobs provided by the present invention preferably includes the following steps:
the tool module unloads the storage acceleration system client on the corresponding computing node;
the operation management system releases the computing resource;
locking the temporary acceleration storage system;
the tool module exports data generated by the operation to a low-speed storage system;
the tool module destroys the temporary accelerated storage system.
The invention has the following advantages:
the invention provides a temporary acceleration storage method for I/O intensive operation, which comprises the following steps: constructing a storage acceleration resource demand description framework of a system and a user layer; receiving a job script compiled by a user according to a storage acceleration resource description framework and a job management specification; constructing a temporary acceleration storage system according to a script submitted by a user; importing files or directories required by the operation from the low-speed storage system to the temporary acceleration storage system; mounting the temporary acceleration storage system and starting operation; and exporting the data generated by the operation from the temporary acceleration storage system, and destroying the temporary acceleration storage system. According to the invention, the temporary acceleration storage system is constructed when the operation is submitted, and is destroyed in time when the operation is finished and quitted, so that resources are released, the execution time of I/O intensive operation is obviously shortened, the operation efficiency of user operation is improved, and the use benefit of a high-performance computer is improved.
Drawings
The invention and its features, aspects and advantages will become more apparent from reading the following detailed description of non-limiting embodiments with reference to the accompanying drawings. Like reference symbols in the various drawings indicate like elements. The drawings are not necessarily to scale, emphasis instead being placed upon illustrating the principles of the invention.
FIG. 1 is a partial flow chart of a temporary accelerated storage method for I/O intensive jobs according to embodiment 1 of the present invention;
fig. 2 is a schematic structural diagram of an I/O-intensive job-oriented temporary acceleration storage system according to embodiment 1 of the present invention.
Detailed Description
It should be noted that the embodiments and features of the embodiments in the present application may be combined with each other without conflict. It is noted that the terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of example embodiments according to the present application.
Example 1:
as shown in fig. 1, the temporary accelerated storage method for I/O-intensive jobs according to embodiment 1 of the present invention specifically includes the following steps:
step S101: a user submits a script through a job management system, wherein the script comprises information such as the number of nodes required by I/O intensive jobs, the number of CPU cores, the required storage acceleration system capacity, and files or directories (the file or directory information is stored in the existing low-speed storage system) used by the jobs, and the script must conform to the format of a storage acceleration resource requirement description frame;
step S102: the operation management system verifies whether the script information is correct, if so, the operation is executed in S103, and if not, error information is returned;
step S103: the job management system inquires whether all resources such as the number of nodes, the number of CPU cores, the capacity of the storage acceleration system and the like meet requirements, if all the resources meet the requirements, the job management system reserves and configures all the required resources, and the operation is switched to S104 for execution; if the requested resource is not satisfied, executing S103 until exiting overtime;
step S104: according to the reserved resource information, the operation management system calls a storage acceleration system client to register the resource requirements in KV storage, after the storage acceleration system server obtains the resource information, a storage acceleration system auxiliary tool (namely a tool module described below) is called for construction, and if the temporary acceleration storage system is successfully constructed, the life cycle of the temporary acceleration storage system begins, and the operation is executed in S105; if the storage acceleration system is unsuccessfully constructed and the S104 is retried, an error is reported and the system is quitted;
step S105: importing files or directory information necessary for user operation from a low-speed storage system to a corresponding position of a temporary acceleration storage system, and if the data import is normally finished, transferring to S106 for execution; if the data import is overtime or abnormal, after retrying S105 to make mistakes, reporting errors and exiting;
step S106: the auxiliary tool of the storage acceleration system installs a client of the temporary acceleration storage system on a required computing node, the successful installation is shifted to S107, otherwise, the S106 is retried to cause an error, and the error is reported and quit is performed;
step S107: starting the operation of the user operation, and storing data generated by the operation into a temporary acceleration storage system;
step S108: after the operation of the user is finished, the auxiliary tool of the storage acceleration system unloads the client of the temporary acceleration storage system on the corresponding computing node, the unloading is successfully executed in S109, otherwise, the operation is executed in S110 after the retry of S108 is failed;
step S109: the operation management system releases the computing resource;
step S110: locking the temporary acceleration storage system;
step S111: the auxiliary tool of the storage acceleration system imports and exports information according to data appointed by a user, exports data generated by operation into a low-speed storage system, if the data export is normally finished, the execution is switched to S112, if the data export is overtime or abnormal, the storage acceleration system is continuously locked, an error is reported to remind the user of manually exporting the data, and after the data export is finished, the execution is switched to S112;
step S112: and (4) destroying the temporary acceleration storage system by the auxiliary tool of the storage acceleration system, and ending the life cycle of the temporary acceleration storage system.
The temporary acceleration storage method for the I/O intensive jobs provided in embodiment 1 of the present invention constructs a temporary acceleration storage system when a job is submitted, where the storage system may be composed of NVME SSD, NVDIMM, or even a server node memory, and has the characteristics of high IOPS, high bandwidth, and the like, and is only used for destroying the I/O intensive jobs, and when the job is finished and quit, the storage system is temporarily accelerated in time to release resources for use by another I/O intensive job.
In order to ensure that the script submitted by the user can be identified by the temporary accelerated storage method for the I/O-intensive job provided in this embodiment 1, before the temporary accelerated storage method for the I/O-intensive job provided in this embodiment 1 is used, a storage accelerated resource requirement description framework on a system and user level needs to be constructed, and the script submitted by the user needs to be written according to the specification of the storage accelerated resource requirement description framework.
In this embodiment, a specific storage acceleration resource requirement description framework is shown in the following table:
the description of the fields in the table is as follows:
number 1: a list of usernames and/or IDs, a list of usernames and/or IDs delimited by colons, that allow the use of a temporary accelerated storage system;
sequence number 2: a list of user names and/or IDs, delimited by colons, prohibits the use of temporary accelerated storage systems. The user cannot belong to both the allowed user and the rejected user. Allowing all users to use the temporary acceleration storage system by default;
sequence No. 3: if so, destroying the temporary acceleration storage system after the file fails to be exported from the temporary acceleration storage system to the low-speed storage system. Otherwise, the temporary acceleration storage system is reserved for analysis and manual destruction;
number 4: the system is used for appointing the full path of the call script of the job management system, and constructing a file system/viewing the state and the like;
number 5: if the time for temporarily accelerating the operation of the storage system (including no verification of the job, import and export of data) exceeds the time, such as mounting a file system, the job will be in a designated state. Job administrators will need to release jobs;
number 6: if the time for the files or directories necessary for the job to be imported from the low speed storage system into the temporary accelerated storage system exceeds this value, the storage accelerated system will be released and the job will be in the specified state. Job administrators will need to release jobs;
number 7: if the data produced by the job is exported from the storage acceleration system to the low speed storage system for a time exceeding this value, the storage acceleration system will be released and the job will be purged;
number 8: if validation of the job submission request takes more time than this data, the submission will be denied;
number 9: GB or TB is used as a capacity application unit of the storage acceleration system;
number 10: a directory which needs to be imported from the low-speed storage system to the temporary acceleration storage system;
number 11: files which need to be imported from the low-speed storage system to the temporary acceleration storage system;
number 12: directories that need to be removed from the temporary acceleration storage system to the low speed storage system;
number 13: files that need to be removed from the temporary acceleration storage system to the low speed storage system;
fig. 2 is an architecture diagram of a system for temporary accelerated storage method for I/O-intensive jobs according to this embodiment 1, where the system includes a KV storage system, a storage acceleration system, and a job management system.
An operation management system: the method is used for scheduling and distributing the resources of the high-performance computing system, ensures that the user operation shares the high-performance computing resources fairly and reasonably, and improves the system utilization rate and the throughput rate.
The storage acceleration system comprises a client, a server and a tool module, wherein the client is called by the operation management system and interacts with the KV storage, the server also interacts with the KV storage, and the tool module is called to execute the work of constructing, destroying, mounting and the like of the temporary acceleration storage system.
The KV storage system is a highly available distributed key value (key-value) database, and the key for providing services such as resource registration and the like is the timely communication between a storage acceleration system client and a server.
It should be noted that the upper-level processing unit in this embodiment refers to a processing unit that has data dependency and needs to transmit data to the processing unit; the next-stage processing unit in this embodiment refers to a processing unit that has data dependency on the processing unit and needs the processing unit to transmit data.
The above description is only for the preferred embodiment of the present invention and is not intended to limit the scope of the present invention, and all equivalent structural changes made by using the contents of the present specification and the drawings, or any other related technical fields, are included in the scope of the present invention.
Claims (10)
1. An I/O-intensive job-oriented temporary accelerated storage method is characterized by comprising the following steps:
constructing a storage acceleration resource demand description framework of a system and a user layer;
receiving a job script compiled by a user according to a storage acceleration resource description framework and a job management specification;
constructing a temporary acceleration storage system according to a script submitted by a user;
importing files or directories required by the operation from the low-speed storage system to the temporary acceleration storage system;
mounting the temporary acceleration storage system and starting operation;
and exporting the data generated by the operation from the temporary acceleration storage system, and destroying the temporary acceleration storage system.
2. The I/O-intensive job-oriented temporary accelerated storage method according to claim 1, wherein the "script" includes at least: the number of nodes required for the I/O intensive job, the number of CPU cores, the temporary accelerated storage system capacity, and the file or directory information used by the job (the file or directory information is already stored in the existing low speed storage system).
3. The I/O-intensive job-oriented temporary accelerated storage method according to claim 2, comprising KV storage system;
the "building a temporary acceleration storage system according to a script submitted by a user" specifically includes:
judging whether the number of the nodes, the number of the CPU cores and the resources for temporarily accelerating the capacity of the storage system meet the resource requirements in the script submitted by the user or not;
if the resource meets the requirement, the resource requirement in the script is reserved and configured;
and registering the resource requirements in the script into the KV storage system, and constructing a temporary acceleration storage system.
4. The temporary acceleration storage method for I/O-intensive jobs according to claim 3, wherein the mounting the temporary acceleration storage system and starting the job specifically comprises:
installing a temporary acceleration storage system client on a required computing node;
and running the job in the temporary acceleration storage system, and storing data generated by the job into the temporary acceleration system.
5. The I/O-intensive job-oriented temporary accelerated storage method according to claim 4, wherein the "exporting data generated by the job to the temporary accelerated storage system, and destroying the temporary accelerated storage system" is specifically:
unloading the temporary accelerated storage system client on the corresponding computing node;
releasing the computing resources;
locking the temporary acceleration storage system;
exporting data generated by the job to a low-speed storage system;
the temporary accelerated storage system is destroyed.
6. A temporary acceleration storage system for I/O intensive operation is characterized by comprising a KV storage system, a storage acceleration system, an operation management system and a program; the storage acceleration system comprises a storage acceleration system client, a storage acceleration system server and a tool module;
the program specifically comprises:
the job management system receives a job script compiled by a user according to the storage acceleration resource description framework and the job management specification;
constructing a temporary acceleration storage system according to a script submitted by a user;
importing files or directories required by the operation from the low-speed storage system to the temporary acceleration storage system;
mounting the temporary acceleration storage system and starting operation;
and exporting the data generated by the operation from the temporary acceleration storage system, and destroying the temporary acceleration storage system.
7. The I/O-intensive job-oriented temporary acceleration storage system according to claim 6, wherein the "script" includes at least: the number of nodes required for the I/O intensive job, the number of CPU cores, the temporary accelerated storage system capacity, and the file or directory information used by the job (the file or directory information is already stored in the existing low speed storage system).
8. The I/O-intensive job-oriented temporary accelerated storage system of claim 7, comprising a KV storage system;
the "building a temporary acceleration storage system according to a script submitted by a user" specifically includes:
the operation management system judges whether the number of the nodes, the number of the CPU cores and the resources for temporarily accelerating the capacity of the storage system meet the resource requirements in the script submitted by the user, and if the resources meet the requirements, the resource requirements in the script are reserved and configured;
the operation management system calls a storage acceleration system client to register the resource requirement in the script in the KV storage system;
and after the server side of the storage acceleration system acquires the resource requirement registered in the KV storage system, calling a tool module to construct a temporary acceleration storage system.
9. The I/O-intensive job-oriented temporary acceleration storage system according to claim 3, wherein the "mount temporary acceleration storage system, start job" is specifically:
the tool module installs a temporary acceleration storage system client on a required computing node;
and running the job in the temporary acceleration storage system, and storing data generated by the job into the temporary acceleration system.
10. The I/O-intensive-job-oriented temporary acceleration storage system according to claim 4, wherein the "exporting data generated by the job to the temporary acceleration storage system, destroying the temporary acceleration storage system" is specifically:
the tool module unloads the temporary accelerated storage system client on the corresponding computing node;
the operation management system releases the computing resource;
locking the temporary acceleration storage system;
the tool module exports data generated by the operation to a low-speed storage system;
the tool module destroys the temporary accelerated storage system.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011156404.1A CN112230862A (en) | 2020-10-26 | 2020-10-26 | Temporary accelerated storage method and system for I/O intensive operation |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011156404.1A CN112230862A (en) | 2020-10-26 | 2020-10-26 | Temporary accelerated storage method and system for I/O intensive operation |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN112230862A true CN112230862A (en) | 2021-01-15 |
Family
ID=74109649
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202011156404.1A Pending CN112230862A (en) | 2020-10-26 | 2020-10-26 | Temporary accelerated storage method and system for I/O intensive operation |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112230862A (en) |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20060271754A1 (en) * | 2005-05-27 | 2006-11-30 | Tsukasa Shibayama | Storage system |
| CN103218415A (en) * | 2013-03-27 | 2013-07-24 | 互爱互动(北京)科技有限公司 | Data processing system and method based on data warehouse |
| US9286261B1 (en) * | 2011-11-14 | 2016-03-15 | Emc Corporation | Architecture and method for a burst buffer using flash technology |
| US20180232143A1 (en) * | 2017-02-15 | 2018-08-16 | International Business Machines Corporation | Burst buffer dynamic logical volume sizing in high performance computing environment |
| CN111416872A (en) * | 2020-03-30 | 2020-07-14 | 中国人民解放军国防科技大学 | High-speed cache file system communication method and system based on MP and RDMA |
-
2020
- 2020-10-26 CN CN202011156404.1A patent/CN112230862A/en active Pending
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20060271754A1 (en) * | 2005-05-27 | 2006-11-30 | Tsukasa Shibayama | Storage system |
| US9286261B1 (en) * | 2011-11-14 | 2016-03-15 | Emc Corporation | Architecture and method for a burst buffer using flash technology |
| CN103218415A (en) * | 2013-03-27 | 2013-07-24 | 互爱互动(北京)科技有限公司 | Data processing system and method based on data warehouse |
| US20180232143A1 (en) * | 2017-02-15 | 2018-08-16 | International Business Machines Corporation | Burst buffer dynamic logical volume sizing in high performance computing environment |
| CN111416872A (en) * | 2020-03-30 | 2020-07-14 | 中国人民解放军国防科技大学 | High-speed cache file system communication method and system based on MP and RDMA |
Non-Patent Citations (1)
| Title |
|---|
| MARC-ANDRE VEF等: "GekkoFS – A temporary distributed file system for HPC applications", 2018 IEEE INTERNATIONAL CONFERENCE ON CLUSTER COMPUTING, pages 319 - 324 * |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US7277913B2 (en) | Persistent queuing for distributed file systems | |
| CN103038788B (en) | Providing multiple network resources | |
| US7756888B2 (en) | Method and apparatus for providing heterogeneous resources for client systems | |
| US9413604B2 (en) | Instance host configuration | |
| US7543150B2 (en) | Method and system for setting up hosting environments in safety | |
| US7779413B2 (en) | Method of assigning available resources for internal and external users at start time of scheduled time period based on program reservations information | |
| US20050034130A1 (en) | Balancing workload of a grid computing environment | |
| CN112463144B (en) | Distributed storage command line service method, system, terminal and storage medium | |
| US20120072579A1 (en) | Monitoring cloud-runtime operations | |
| US20160085601A1 (en) | Transparent user mode scheduling on traditional threading systems | |
| US8966025B2 (en) | Instance configuration on remote platforms | |
| CN104065636B (en) | Data processing method and system | |
| JP2010500673A (en) | Storage management system for maintaining consistency of remote copy data (storage management system, storage management method, and computer program) | |
| WO2016118429A1 (en) | System for efficient processing of transaction requests related to an account in a database | |
| CN107710155B (en) | System and method for booting application servers in parallel | |
| US11366802B1 (en) | Batch undo processing for transaction rollbacks in a multi-writer database | |
| Lofstead et al. | D2t: Doubly distributed transactions for high performance and distributed computing | |
| JP2003006170A (en) | Method for performing program in environment of plural computers | |
| US20150186479A1 (en) | Importing data into dynamic distributed databases | |
| CN112230862A (en) | Temporary accelerated storage method and system for I/O intensive operation | |
| CN112130900B (en) | User information management method, system, equipment and medium for BMC | |
| CN114691309A (en) | Batch business processing system, method and device | |
| US11121981B1 (en) | Optimistically granting permission to host computing resources | |
| CN112860413A (en) | Centralized job scheduling system, device, electronic equipment and computer readable storage medium | |
| US20110023018A1 (en) | Software platform and method of managing application individuals in the software platform |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |