CN111177105A - Mass file writing method, device, system and medium of distributed file system - Google Patents
Mass file writing method, device, system and medium of distributed file system Download PDFInfo
- Publication number
- CN111177105A CN111177105A CN201911386425.XA CN201911386425A CN111177105A CN 111177105 A CN111177105 A CN 111177105A CN 201911386425 A CN201911386425 A CN 201911386425A CN 111177105 A CN111177105 A CN 111177105A
- Authority
- CN
- China
- Prior art keywords
- file
- user
- data
- writing
- client
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000000034 method Methods 0.000 title claims abstract description 50
- 238000004590 computer program Methods 0.000 claims description 11
- 238000005516 engineering process Methods 0.000 description 2
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000010586 diagram Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 230000000750 progressive effect Effects 0.000 description 1
- 239000007787 solid Substances 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/10—File systems; File servers
- G06F16/18—File system types
- G06F16/182—Distributed file systems
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/10—File systems; File servers
- G06F16/13—File access structures, e.g. distributed indices
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/10—File systems; File servers
- G06F16/17—Details of further file system functions
- G06F16/172—Caching, prefetching or hoarding of files
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/10—File systems; File servers
- G06F16/18—File system types
- G06F16/1847—File system types specifically adapted to static storage, e.g. adapted to flash memory or SSD
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Data Mining & Analysis (AREA)
- Databases & Information Systems (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The invention discloses a method, a device and a system for writing mass files of a distributed file system and a computer readable storage medium, wherein the method comprises the following steps: receiving data written by a user, and caching the data into a client; judging whether the total amount of currently cached data in the client reaches a preset client cache threshold value or not, if so, determining that a file written by a user is a large file, and completely writing all currently cached data and subsequently received data into an HDD (hard disk drive) until a file closing request input by the user is received; if not, judging whether a file closing request input by a user is received or not, if so, determining that the file written by the user is a small file, and writing all the currently cached data into the SDD disk; if not, returning to the step of receiving the data written by the user; the invention can improve the performance of writing small files and the overall performance of the file system in the using process, and further meets the requirements of users.
Description
Technical Field
The embodiment of the invention relates to the technical field of distributed file systems, in particular to a method, a device and a system for writing mass files of a distributed file system and a computer readable storage medium.
Background
With the rapid development and application of computers and information technologies, the scale of an industry application system is rapidly enlarged, data generated by industry application is rapidly developed explosively, and large data of industries and enterprises which reach the scale of hundreds of TB or even tens of PB is far beyond the processing capacity of the traditional computing technology and system. The distributed system is generated according to the scenes of mass data. Distributed systems are currently mainly composed of three parts: 1. the client is responsible for interacting with the user; 2. the metadata server is responsible for managing metadata information; 3. the data server is responsible for managing data. At present, for a data server, the data server mainly manages data stored on a hard disk, and a magnetic disk mainly includes an HDD and an SDD, the HDD is a traditional nonvolatile memory, and the SDD is a solid state disk. SDD has much higher read and write performance than HDDs, but SDD is expensive and the cost of deployment is high.
For a distributed file system, large files and small files are arranged in the distributed file system, and the large files and the small files can all use SDD disks as storage disks simply for improving the read-write performance, but the distributed system is higher in deployment cost; if the large file and the small file only use the HDD disk as a storage disk, the performance requirements are difficult to meet, and especially the small file has the characteristics of small occupied space, large file volume and consumption of the performance of the disk when reading and writing. Because data is written into a disk while writing in a file system, the size of a file is difficult to accurately judge when the writing is not completed, and the respective storage of large and small files is difficult to realize.
In view of the above, how to provide a method, an apparatus, a system and a computer readable storage medium for writing mass files of a distributed file system to solve the above technical problems becomes a problem to be solved by those skilled in the art.
Disclosure of Invention
The embodiment of the invention aims to provide a method, a device and a system for writing mass files of a distributed file system and a computer readable storage medium, which can improve the writing performance of small files and the overall performance of the file system in the using process and further meet the requirements of users.
In order to solve the above technical problem, an embodiment of the present invention provides a method for writing a mass file of a distributed file system, including:
receiving data written by a user, and caching the data into a client;
judging whether the total amount of currently cached data in the client reaches a preset client cache threshold value, if so, determining that the file written by the user is a large file, and writing all currently cached data and all subsequently received data into an HDD (hard disk drive) until a file closing request input by the user is received;
if not, judging whether a file closing request input by the user is received or not, if so, determining that the file written by the user is a small file, and writing all the currently cached data into the SDD disk; and if not, returning to execute the step of receiving the data written by the user and caching the data into the client.
Optionally, before the receiving the data written by the user and caching the data in the client, the method further includes:
after receiving a file creating instruction input by a user, inquiring prestored small file strategy information from a metadata server, wherein the small file strategy information comprises a small file size threshold;
and determining the client cache threshold value according to the small file strategy information.
Optionally, before receiving a file creation instruction input by a user, the method further includes:
and receiving small file strategy information input by a user, and sending the small file strategy information to the metadata server for storage.
Optionally, the method further includes:
and deleting all the data currently cached in the client after all the data currently cached in the client are written into the corresponding disk.
The embodiment of the present invention further provides a device for writing mass files of a distributed file system, which includes:
the first receiving module is used for receiving data written by a user and caching the data into a client;
the first judging module is used for judging whether the total amount of currently cached data in the client reaches a preset client cache threshold value or not, and if so, triggering the first writing module; if not, triggering a second judgment module;
the first writing module is used for determining that the file written by the user is a large file, and writing all currently cached data and all subsequently received data into an HDD (hard disk drive) until a file closing request input by the user is received;
the second judging module is used for judging whether a file closing request input by the user is received or not, and if the file closing request is received, triggering a second writing module; if not, triggering the first receiving module;
and the second writing module is used for determining that the file written by the user is a small file and writing all the currently cached data into the SDD disk.
Optionally, the method further includes:
the query module is used for querying prestored small file strategy information from the metadata server after receiving a file creation instruction input by a user, wherein the small file strategy information comprises a small file size threshold;
and the determining module is used for determining the client cache threshold according to the small file strategy information.
Optionally, the method further includes:
and the second receiving module is used for receiving the small file strategy information input by the user and sending the small file strategy information to the metadata server for storage.
Optionally, the method further includes:
and the deleting module is used for deleting all the data currently cached in the client after all the data currently cached in the client are written into the corresponding disk.
The embodiment of the present invention further provides a mass file writing system of a distributed file system, including:
a memory for storing a computer program;
a processor for implementing the steps of the method for writing mass files of the distributed file system as described in the above when executing the computer program.
An embodiment of the present invention further provides a computer-readable storage medium, where a computer program is stored on the computer-readable storage medium, and when the computer program is executed by a processor, the steps of the method for writing mass files in the distributed file system are implemented.
The embodiment of the invention provides a method, a device and a system for writing mass files of a distributed file system and a computer readable storage medium, wherein the method comprises the following steps: receiving data written by a user, and caching the data into a client; judging whether the total amount of currently cached data in the client reaches a preset client cache threshold value or not, if so, determining that a file written by a user is a large file, and completely writing all currently cached data and subsequently received data into an HDD (hard disk drive) until a file closing request input by the user is received; if not, judging whether a file closing request input by a user is received or not, if so, determining that the file written by the user is a small file, and writing all the currently cached data into the SDD disk; if not, returning to the step of receiving the data written by the user.
It can be seen that, in the present application, data written by a user is cached in a client, and whether a file written by the user is a large file or a small file is further determined according to a preset client cache threshold and a current cached data total, when the current cached data total reaches the client cache threshold, it is indicated that the file written by the user is the large file, the large file is stored in an HDD disk, when the current cached data total does not reach the client cache threshold and a file closing request input by the user is received, it is indicated that the size of the file still does not reach the client cache threshold when the file is completely written, it is determined that the file is the small file, and at this time, the file is written into an SDD, that is, the large file and the small file can be distinguished in the using process, and are stored respectively, so as to improve the small file writing performance and the overall performance of a file system, further meeting the requirements of users.
Drawings
In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings needed in the prior art and the embodiments will be briefly described below, and it is obvious that the drawings in the following description are only some embodiments of the present invention, and it is obvious for those skilled in the art to obtain other drawings without creative efforts.
Fig. 1 is a schematic flowchart of a method for writing mass files in a distributed file system according to an embodiment of the present invention;
fig. 2 is a schematic structural diagram of a mass file writing device of a distributed file system according to an embodiment of the present invention.
Detailed Description
The embodiment of the invention provides a method, a device and a system for writing mass files of a distributed file system and a computer readable storage medium, which can improve the writing performance of small files and the overall performance of the file system in the using process and further meet the requirements of users.
In order to make the objects, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are some, but not all, embodiments of the present invention. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
Referring to fig. 1, fig. 1 is a schematic flowchart of a method for writing a mass file of a distributed file system according to an embodiment of the present invention. The method comprises the following steps:
s110: receiving data written by a user, and caching the data into a client;
it should be noted that, in this embodiment, data written by the user may be cached in the client instead of being directly written into the disk, and a client cache threshold may be set, and the client cache threshold may be used to determine whether the user writes a small file or a large file at present, and then write the file into the corresponding disk after determining whether the user writes the large file or the small file.
S120: judging whether the total amount of currently cached data in the client reaches a preset client cache threshold value, if so, entering S130; if not, entering S140;
specifically, after the data written by the user is cached in the client, by comparing the total amount of data currently cached by the client with a predetermined client cache threshold, it can be determined whether the total amount of data currently cached reaches the client cache threshold, where the client cache threshold may be determined according to the size of a small file that the user wants to specify, for example, if a file smaller than 10M is used as the small file, the client cache threshold may be set to 10M, and may be specifically determined according to actual needs.
S130: determining that a file written by a user is a large file, and writing all currently cached data and all subsequently received data into an HDD (hard disk drive) until a file closing request input by the user is received;
specifically, when the total amount of data currently cached by the client reaches a preset client cache threshold, it is determined that a file written by a user is a large file, that is, data written by the user is full and still written in the client, which indicates that the file is not a small file (i.e., a large file), at this time, all data currently cached in the client are all flushed into an HDD disk of the data server, and data written by a subsequent user is directly written into the HDD disk until a file closing request input by the user is received, and the file is closed, that is, until the user finishes writing the file.
S140: judging whether a file closing request input by a user is received, and if so, entering S150; if not, returning to S110;
s150: and determining that the file written by the user is a small file, and writing all the currently cached data into the SDD disk.
Specifically, when the total amount of data currently cached in the client is smaller than the client cache threshold, it is indicated that the file written by the user is not fully cached, that is, the cache still has a remaining space, if a file closing request input by the user is received at this time, that is, the file written by the user is completely written, it is indicated that the total size of the file written by the user reaches the client cache threshold, and the small file standard is met, it is determined that the file written by the user this time is a small file, and all data currently cached in the client is written into the SDD disk in the data server. That is, the method provided by the application can determine whether the file written by the user is a large file or a small file, and store the small file in the SDD disk, so that the writing performance of the small file is improved.
Further, before receiving the data written by the user in S110 and caching the data in the client, the method may further include:
after receiving a file creating instruction input by a user, inquiring prestored small file strategy information from a metadata server, wherein the small file strategy information comprises a small file size threshold;
and determining a client cache threshold value according to the small file strategy information.
It should be noted that, the metadata server may store the small file policy information in advance, and the small file policy information includes a small file size threshold, that is, a specification of the size of the small file. The method comprises the steps that a user inputs a file creating instruction when writing a file through a client, the client knows that the user is about to write the file after receiving the file creating instruction input by the user, at the moment, small file strategy information can be inquired from a metadata server, a corresponding small file size threshold value is obtained after the inquiry is finished, then the client cache threshold value of the client is determined according to the small file size threshold value, for example, the small file size threshold value is 10M, the client cache threshold value can also be set to be 10M, so that the file smaller than 10M is specified to be a small file, and when the user writes data later, whether the file written by the user is a large file or a small file is determined according to the client cache threshold value.
Further, before receiving a file creation instruction input by the user in S110, the method may further include:
and receiving the small file strategy information input by the user, and sending the small file strategy information to the metadata server for storage.
It should be noted that, a user may determine a small file policy according to actual needs, where the small file policy includes a small file size threshold, that is, a specification of the size of a small file, and writes the small file policy information into a client, and after receiving the small file policy information input by the user, the client sends the small file policy information to a metadata server, and the metadata server stores the small file policy. Certainly, the user can also adjust the small file strategy according to actual needs, a new small file strategy can be input when the small file strategy needs to be adjusted, the client sends the new small file strategy to the metadata server, the metadata server stores the new small file strategy after receiving the new small file strategy, and meanwhile, the small file strategy stored before is deleted.
Further, the method may further include:
and deleting all the data cached currently in the client after all the data cached currently in the client is written into the corresponding disk.
It can be understood that, since data is cached in the client every time a new file is written by a user, in order to ensure that the newly written file can be cached smoothly next time and that all cached data of the same file are cached every time, so as to accurately judge the size of the file, in this embodiment, after the specific type of the file currently written by the user is determined every time and all currently cached data in the client is written into the corresponding disk, all currently cached data in the client can be deleted, thereby ensuring that the file can be cached to the client smoothly next time the user writes the file.
It can be seen that, in the present application, data written by a user is cached in a client, and whether a file written by the user is a large file or a small file is further determined according to a preset client cache threshold and a current cached data total, when the current cached data total reaches the client cache threshold, it is indicated that the file written by the user is the large file, the large file is stored in an HDD disk, when the current cached data total does not reach the client cache threshold and a file closing request input by the user is received, it is indicated that the size of the file still does not reach the client cache threshold when the file is completely written, it is determined that the file is the small file, and at this time, the file is written into an SDD, that is, the large file and the small file can be distinguished in the using process, and are stored respectively, so as to improve the small file writing performance and the overall performance of a file system, further meeting the requirements of users.
On the basis of the foregoing embodiment, an embodiment of the present invention further provides a device for writing a mass file in a distributed file system, which refers to fig. 2. The device includes:
the first receiving module 21 is configured to receive data written by a user and cache the data in the client;
the first judging module 22 is configured to judge whether a total amount of currently cached data in the client reaches a preset client cache threshold, and if so, trigger the first writing module 23; if not, triggering a second judgment module 24;
a first write-in module 23, configured to determine that a file written by a user is a large file, and write all currently cached data and all subsequently received data into the HDD disk until a file close request input by the user is received;
the second judging module 24 is configured to judge whether a file closing request input by a user is received, and if the file closing request is received, trigger the second writing module 25; if not, triggering the first receiving module 21;
and a second writing module 25, configured to determine that the file written by the user is a small file, and write all currently cached data into the SDD disk.
Further, the apparatus may further include:
the query module is used for querying prestored small file strategy information from the metadata server after receiving a file creation instruction input by a user, wherein the small file strategy information comprises a small file size threshold;
and the determining module is used for determining the client cache threshold according to the small file strategy information.
Further, the apparatus may further include:
and the second receiving module is used for receiving the small file strategy information input by the user and sending the small file strategy information to the metadata server for storage.
Further, the apparatus may further include:
and the deleting module is used for deleting all the data currently cached in the client after all the data currently cached in the client are written into the corresponding disk.
It should be noted that the method for writing the mass files of the distributed file system provided in this embodiment has the same beneficial effects as the apparatus for writing the mass files of the distributed file system provided in the foregoing embodiment, and for specific description of the method for writing the mass files of the distributed file system related in this embodiment, reference is made to the foregoing embodiment, and details of this application are not repeated herein.
On the basis of the above embodiments, an embodiment of the present invention further provides a mass file writing system of a distributed file system, where the system includes:
a memory for storing a computer program;
a processor for implementing the steps of the method for writing mass files of the distributed file system as described above when executing a computer program.
For example, the processor in this embodiment is configured to receive data written by a user, and cache the data in the client; judging whether the total amount of currently cached data in the client reaches a preset client cache threshold value or not, if so, determining that a file written by a user is a large file, and completely writing all currently cached data and subsequently received data into an HDD (hard disk drive) until a file closing request input by the user is received; if not, judging whether a file closing request input by a user is received or not, if so, determining that the file written by the user is a small file, and writing all the currently cached data into the SDD disk; if not, returning to the step of receiving the data written by the user.
On the basis of the foregoing embodiments, an embodiment of the present invention further provides a computer-readable storage medium, where a computer program is stored on the computer-readable storage medium, and when the computer program is executed by a processor, the steps of the method for writing mass files in the distributed file system are implemented.
The computer-readable storage medium may include: various media capable of storing program codes, such as a usb disk, a removable hard disk, a Read-only Memory (ROM), a Random Access Memory (RAM), a magnetic disk, or an optical disk.
The embodiments in the present description are described in a progressive manner, each embodiment focuses on differences from other embodiments, and the same and similar parts among the embodiments are referred to each other. The device disclosed by the embodiment corresponds to the method disclosed by the embodiment, so that the description is simple, and the relevant points can be referred to the method part for description.
It is further noted that, in the present specification, relational terms such as first and second, and the like are used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Also, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising an … …" does not exclude the presence of other identical elements in a process, method, article, or apparatus that comprises the element.
The previous description of the disclosed embodiments is provided to enable any person skilled in the art to make or use the present invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the generic principles defined herein may be applied to other embodiments without departing from the spirit or scope of the invention. Thus, the present invention is not intended to be limited to the embodiments shown herein but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.
Claims (10)
1. A mass file writing method of a distributed file system is characterized by comprising the following steps:
receiving data written by a user, and caching the data into a client;
judging whether the total amount of currently cached data in the client reaches a preset client cache threshold value, if so, determining that the file written by the user is a large file, and writing all currently cached data and all subsequently received data into an HDD (hard disk drive) until a file closing request input by the user is received;
if not, judging whether a file closing request input by the user is received or not, if so, determining that the file written by the user is a small file, and writing all the currently cached data into the SDD disk; and if not, returning to execute the step of receiving the data written by the user and caching the data into the client.
2. The method for writing mass files in a distributed file system according to claim 1, before receiving data written by a user and caching the data in a client, further comprising:
after receiving a file creating instruction input by a user, inquiring prestored small file strategy information from a metadata server, wherein the small file strategy information comprises a small file size threshold;
and determining the client cache threshold value according to the small file strategy information.
3. The method for writing mass files in a distributed file system according to claim 2, further comprising, before receiving a file creation command input by a user:
and receiving small file strategy information input by a user, and sending the small file strategy information to the metadata server for storage.
4. The method for writing mass files in a distributed file system according to claim 1, further comprising:
and deleting all the data currently cached in the client after all the data currently cached in the client are written into the corresponding disk.
5. A massive file writing device of a distributed file system is characterized by comprising:
the first receiving module is used for receiving data written by a user and caching the data into a client;
the first judging module is used for judging whether the total amount of currently cached data in the client reaches a preset client cache threshold value or not, and if so, triggering the first writing module; if not, triggering a second judgment module;
the first writing module is used for determining that the file written by the user is a large file, and writing all currently cached data and all subsequently received data into an HDD (hard disk drive) until a file closing request input by the user is received;
the second judging module is used for judging whether a file closing request input by the user is received or not, and if the file closing request is received, triggering a second writing module; if not, triggering the first receiving module;
and the second writing module is used for determining that the file written by the user is a small file and writing all the currently cached data into the SDD disk.
6. The apparatus for writing mass files of a distributed file system according to claim 5, further comprising:
the query module is used for querying prestored small file strategy information from the metadata server after receiving a file creation instruction input by a user, wherein the small file strategy information comprises a small file size threshold;
and the determining module is used for determining the client cache threshold according to the small file strategy information.
7. The apparatus for writing mass files of a distributed file system according to claim 6, further comprising:
and the second receiving module is used for receiving the small file strategy information input by the user and sending the small file strategy information to the metadata server for storage.
8. The apparatus for writing mass files of a distributed file system according to claim 5, further comprising:
and the deleting module is used for deleting all the data currently cached in the client after all the data currently cached in the client are written into the corresponding disk.
9. A mass file writing system of a distributed file system, comprising:
a memory for storing a computer program;
processor for implementing the steps of the method for writing mass files of a distributed file system according to any of claims 1 to 4 when executing said computer program.
10. A computer-readable storage medium, having stored thereon a computer program which, when being executed by a processor, carries out the steps of the method for writing mass files of a distributed file system according to any one of claims 1 to 4.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911386425.XA CN111177105B (en) | 2019-12-29 | 2019-12-29 | Mass file writing method, device, system and medium of distributed file system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911386425.XA CN111177105B (en) | 2019-12-29 | 2019-12-29 | Mass file writing method, device, system and medium of distributed file system |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN111177105A true CN111177105A (en) | 2020-05-19 |
| CN111177105B CN111177105B (en) | 2022-03-22 |
Family
ID=70649004
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201911386425.XA Active CN111177105B (en) | 2019-12-29 | 2019-12-29 | Mass file writing method, device, system and medium of distributed file system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111177105B (en) |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111930696A (en) * | 2020-07-22 | 2020-11-13 | 中国建设银行股份有限公司 | File transmission processing method and system based on small program |
| CN112084156A (en) * | 2020-09-24 | 2020-12-15 | 中国农业银行股份有限公司上海市分行 | Hybrid storage system and self-adaptive backup method of file |
| CN112187875A (en) * | 2020-09-09 | 2021-01-05 | 苏州浪潮智能科技有限公司 | Automatic matching method and system for multi-target cluster mounting strategy of distributed system |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106775446A (en) * | 2016-11-11 | 2017-05-31 | 中国人民解放军国防科学技术大学 | Based on the distributed file system small documents access method that solid state hard disc accelerates |
| CN108089825A (en) * | 2018-01-11 | 2018-05-29 | 郑州云海信息技术有限公司 | A kind of storage system based on distributed type assemblies |
| CN108563743A (en) * | 2018-04-12 | 2018-09-21 | 郑州云海信息技术有限公司 | A kind of file read/write method, system and equipment and storage medium |
| WO2019061132A1 (en) * | 2017-09-28 | 2019-04-04 | 深圳清华大学研究院 | Hybrid file system architecture, file storage, dynamic migration, and application thereof |
| US10503697B1 (en) * | 2016-06-30 | 2019-12-10 | EMC IP Holding Company LLC | Small file storage system |
-
2019
- 2019-12-29 CN CN201911386425.XA patent/CN111177105B/en active Active
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US10503697B1 (en) * | 2016-06-30 | 2019-12-10 | EMC IP Holding Company LLC | Small file storage system |
| CN106775446A (en) * | 2016-11-11 | 2017-05-31 | 中国人民解放军国防科学技术大学 | Based on the distributed file system small documents access method that solid state hard disc accelerates |
| WO2019061132A1 (en) * | 2017-09-28 | 2019-04-04 | 深圳清华大学研究院 | Hybrid file system architecture, file storage, dynamic migration, and application thereof |
| CN108089825A (en) * | 2018-01-11 | 2018-05-29 | 郑州云海信息技术有限公司 | A kind of storage system based on distributed type assemblies |
| CN108563743A (en) * | 2018-04-12 | 2018-09-21 | 郑州云海信息技术有限公司 | A kind of file read/write method, system and equipment and storage medium |
Non-Patent Citations (1)
| Title |
|---|
| 聂瑞华等: "一种改进的基于缓存池机制的小文件I/O策略", 《计算机工程与应用》 * |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111930696A (en) * | 2020-07-22 | 2020-11-13 | 中国建设银行股份有限公司 | File transmission processing method and system based on small program |
| CN112187875A (en) * | 2020-09-09 | 2021-01-05 | 苏州浪潮智能科技有限公司 | Automatic matching method and system for multi-target cluster mounting strategy of distributed system |
| CN112187875B (en) * | 2020-09-09 | 2022-05-13 | 苏州浪潮智能科技有限公司 | Automatic matching method and system for multi-target cluster mounting strategy of distributed system |
| CN112084156A (en) * | 2020-09-24 | 2020-12-15 | 中国农业银行股份有限公司上海市分行 | Hybrid storage system and self-adaptive backup method of file |
Also Published As
| Publication number | Publication date |
|---|---|
| CN111177105B (en) | 2022-03-22 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111177105B (en) | Mass file writing method, device, system and medium of distributed file system | |
| US11853549B2 (en) | Index storage in shingled magnetic recording (SMR) storage system with non-shingled region | |
| EP1960888B1 (en) | Optimizing write and wear performance for a memory | |
| CN106776368B (en) | Cache management method, device and system during data reading | |
| CN107729558B (en) | Method, system and device for defragmenting file system and computer storage medium | |
| US11194710B2 (en) | Garbage collection—automatic data placement | |
| JP2017521762A (en) | Storage apparatus, program, and information processing method | |
| CN107291399B (en) | Back-end storage method, device and system based on SPDK | |
| CN107967122B (en) | Data writing method, device and medium for block device | |
| CN109804359A (en) | For the system and method by write back data to storage equipment | |
| CN110727404A (en) | Data deduplication method and device based on storage end and storage medium | |
| US10346051B2 (en) | Storage media performance management | |
| CN112463054B (en) | Method and equipment for improving read-write performance | |
| CN112181902B (en) | Database storage method and device and electronic equipment | |
| US20170235781A1 (en) | Method, server and computer program stored in computer readable medium for managing log data in database | |
| CN105556462A (en) | Writing to files and file meta-data | |
| CN109508140B (en) | Storage resource management method and device, electronic equipment and system | |
| KR101643278B1 (en) | Method, apparatus, and computer program stored in computer readable medium for managing storage server in database system | |
| CN108984117B (en) | Data reading and writing method, medium and equipment | |
| CN109241011B (en) | Virtual machine file processing method and device | |
| CN106504402A (en) | Log information recording method and device | |
| CN108509295B (en) | Operation method of memory system | |
| US9836220B2 (en) | Data processing system and method of operating the same | |
| CN111752915B (en) | File on-demand copying method and system based on hybrid storage medium | |
| US11720551B1 (en) | Method and system for streaming data from portable storage devices |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |