CN116383165B - Adaptive writing archive storage method - Google Patents
Adaptive writing archive storage method Download PDFInfo
- Publication number
- CN116383165B CN116383165B CN202310652957.3A CN202310652957A CN116383165B CN 116383165 B CN116383165 B CN 116383165B CN 202310652957 A CN202310652957 A CN 202310652957A CN 116383165 B CN116383165 B CN 116383165B
- Authority
- CN
- China
- Prior art keywords
- file
- level
- writing
- files
- time
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000003860 storage Methods 0.000 title claims abstract description 35
- 238000000034 method Methods 0.000 title claims abstract description 16
- 230000003044 adaptive effect Effects 0.000 title claims abstract description 11
- 230000014759 maintenance of location Effects 0.000 claims abstract description 63
- 238000004321 preservation Methods 0.000 claims abstract description 45
- 238000001514 detection method Methods 0.000 claims abstract description 4
- 230000006835 compression Effects 0.000 claims description 18
- 238000007906 compression Methods 0.000 claims description 18
- 230000015654 memory Effects 0.000 abstract description 16
- 238000012545 processing Methods 0.000 abstract description 2
- 230000008859 change Effects 0.000 description 9
- 230000007246 mechanism Effects 0.000 description 6
- 230000009471 action Effects 0.000 description 4
- 230000008901 benefit Effects 0.000 description 4
- 238000010586 diagram Methods 0.000 description 4
- 239000010410 layer Substances 0.000 description 4
- 239000002356 single layer Substances 0.000 description 4
- 238000012360 testing method Methods 0.000 description 4
- 238000005265 energy consumption Methods 0.000 description 3
- 101100226366 Arabidopsis thaliana EXT3 gene Proteins 0.000 description 2
- 238000013459 approach Methods 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 208000034420 multiple type III exostoses Diseases 0.000 description 2
- 230000000750 progressive effect Effects 0.000 description 2
- 230000002035 prolonged effect Effects 0.000 description 2
- 230000007547 defect Effects 0.000 description 1
- 238000009826 distribution Methods 0.000 description 1
- 230000006870 function Effects 0.000 description 1
- 238000004519 manufacturing process Methods 0.000 description 1
- 230000006386 memory function Effects 0.000 description 1
- 238000000926 separation method Methods 0.000 description 1
- 230000003068 static effect Effects 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/10—File systems; File servers
- G06F16/18—File system types
- G06F16/1805—Append-only file systems, e.g. using logs or journals to store data
- G06F16/1815—Journaling file systems
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/10—File systems; File servers
- G06F16/17—Details of further file system functions
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/10—File systems; File servers
- G06F16/17—Details of further file system functions
- G06F16/172—Caching, prefetching or hoarding of files
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
- Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Data Mining & Analysis (AREA)
- Databases & Information Systems (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
An adaptive writing archive storage method belongs to the technical field of electric digital data processing, and comprises the following steps: step S1, when a file storage request exists, a log file system makes the file into a transaction structure; step S2, performing compressible detection on the file, and dividing the file into two writing modes; step S3, the writing mode judges that the slightly written files are divided into 4 reservation levels; step S4, judging whether the file preservation time is about to be used up or not at regular intervals; step S5, in the check point stage, the files are stored in the file area. The invention optimizes the memory by the log file system, solves the problem of repeatedly writing the same file, and simultaneously, gives the required file retention time to different files.
Description
Technical Field
The invention belongs to the technical field of electric digital data processing, and particularly relates to an adaptive writing archive storage method.
Background
The log file system refers to a file system in which when a change occurs in the file system, related information is written into an area called a log, and then the change is written into the main file system. In the event of a file system failure (e.g., a kernel crash or a sudden power outage), the journaling file system is more likely to maintain consistency and can recover faster.
In many journal file systems (e.g., xfs, ext3, ext4, reiser fs), when a data schema is selected, both metadata and file content are written to the journal and then committed to the primary file system. This improves security and can maintain consistency of the data by checking the metadata and file content in the log when the system fails, but this approach loses performance because all the data is written twice. For example, fig. 1 is a diagram of an archive storage structure of an ext3 file system, as shown in fig. 1, before data is written into a storage hard disk such as an SSD, the archive storage structure is first manufactured into a Transaction structure (Transaction) writing log area, the Transaction structure includes an archive itself and metadata, and index nodes of a Transaction head (H) and a Transaction tail (T) are added before and after each Transaction structure, and the index nodes function as effective data only when the log file system is checked, and the method ensures the integrity and consistency of the data when the Transaction head (H) and the Transaction tail (T) happen accidentally. After the Checkpoint stage (Checkpoint), the transaction structures written to the log area are written to the data area. When the system crashes, the original system state can be effectively recovered through a check point stage. The method improves the security of files, but simultaneously writes the same data in the log area and the data area respectively, thereby losing performance.
Therefore, it is necessary to develop an adaptive write file saving method, which effectively reduces the write operations of file saving.
Disclosure of Invention
In view of the above-mentioned shortcomings of the prior art, an object of the present invention is to provide an adaptive writing file storage method.
In order to achieve the above object, the present invention adopts the following technical scheme.
An adaptive writing archive storage method comprises the following steps:
step S1, when a file storage request exists, a log file system makes the file into a transaction structure; a transaction structure including a status index bit, metadata, and archive content; the state index bit records the block position, the block length, the file retention level and the file retention time;
step S2, compressible detection is carried out on the files, and the content of the files is divided into two writing modes according to the compression rate: mild writes applicable to compressible files, severe writes applicable to incompressible files;
step S3, the writing mode is judged to be a slightly written file, and the files are divided into 4 reservation levels according to the increasing sequence of the natural preservation time: primary retention level, secondary retention level, tertiary retention level, quaternary retention level;
step S4, regularly judging whether the file preservation time is about to be used up: if not, continuing waiting until the system judges that the file storage time is about to run out or writing into the file area when waiting for a check point stage; otherwise, the file is refreshed, the reservation level is increased by one step, the file reservation level and the file reservation time are updated in the state index bit, and if the previous reservation level is the four-level reservation level, the reservation level is reset to the four-level reservation level during refreshing;
step S5, in the check point stage, the files are stored in the file area.
Further, in step S1, the transaction structure includes a transaction header, a status index bit, metadata, file content, and a transaction trailer;
in step S2, light writing includes: metadata and high compression archival content; heavy writes, including incompressible archives.
Further, in step S3, the primary reservation level: the natural preservation time of the file is 3 minutes;
secondary retention rating: the natural preservation time of the files is 3 hours;
three-level retention level: the natural preservation time of the file is 3 days;
four-level retention rating: the natural preservation time of the files is 3 months.
Further, in step S3, the natural preservation time of the metadata is initially set to a first-level preservation level;
the natural preservation time of the file content with high compression rate is initially set as a secondary preservation level;
the writing mode is judged to be a file which is severely written, the reservation level is a four-level reservation level, and the natural preservation time is 3 months.
Further, in step S4, the method for periodically determining whether the file storage time is about to be exhausted is as follows: checking whether the stored time of the file exceeds T/2 or not every T/3 period according to the system time, wherein T is the natural storage time of the corresponding file; if yes, judging that the file preservation time is about to be used up; otherwise, the next checking period is continued to be waited.
In step S5, for the files allocated as heavy writing, the index position is updated in the metadata area by using the writing mode to the location and adopting the index changing mode as the mode of storing in the file area;
for files allocated as light writing, the staggered writing mode is adopted to write the file area in the check point stage.
The invention optimizes the memory by the log file system, solves the problem of repeatedly writing the same file, and simultaneously, endows different files with required file retention time, and has the following advantages:
1, in this scheme, the compression rate is used as an index for judging the stability of the file, that is, an index of the update frequency of the file. Mild writing is suitable for files with poor stability, and the natural storage time of writing is short. Heavy writing is suitable for files with better stability, and the natural storage time of writing is longer. Because of the slightly written files, the natural preservation time is shorter, so that the energy consumption and time required by preservation are saved, and the service life of a memory is prolonged.
2, the scheme presets a reservation level making mechanism, so that files with different reservation levels have natural preservation time suitable for the levels.
And 3, in the scheme, an increment reservation mechanism is arranged, so that whether the file storage time is about to run out is regularly judged, and the situation that the file disappears due to no change of the file exceeding time is avoided. Meanwhile, the progressive writing mode can increase the energy use efficiency, lighten the writing of the memory and prolong the service life.
4, for files distributed as heavy writing, the writing mode of using In-place is used, and the files are not rewritten once, so that the energy consumption and time required for saving are saved, and the service life of the memory is prolonged.
Drawings
FIG. 1 is a diagram of an archive architecture of an ext3 file system;
FIG. 2 is a diagram of an archive storage structure of the present invention;
fig. 3 is a flow chart of the present invention.
Detailed Description
In a computer, a structure of matching a static random access memory and a dynamic random access memory with a hard disk is generally adopted, but the access speeds of the dynamic random access memory and the hard disk are greatly different, so that data are blocked when moving between the dynamic random access memory and the hard disk, and finally, the performance of the system is reduced. The emerging phase change memory, one of the nonvolatile random access memories, is considered one of the most promising candidates for replacing dynamic random access memory as main memory due to its better scalability and non-volatility. With the multi-bit memory function, the phase change memory of the multi-layer cell is superior to the single-layer cell in terms of density, but has a limit in the number of writing times, and it is necessary to consider the writing loss like an SSD. The single-layer unit architecture can be accessed 10 ten thousand times, while the multi-layer unit architecture can only access 1/10 of the single-layer unit architecture, and the multi-layer unit architecture is not superior to the single-layer unit architecture in terms of access speed. Although phase change memories of multi-layer cells have many drawbacks, they have high advantages in terms of both manufacturing cost and unit capacity, and subsequently have a great market demand. Therefore, for multi-level cell phase change memories, the archive storage mechanism employed should reduce the write energy, thereby increasing the lifetime.
Traditional log file systems give files the same file retention time and the same writing power and energy to the files regardless of the type of file, but if the files are frequently updated files, for example, files in editing, the files may be changed for less than a few minutes, and this file retention mode regardless of the type of file takes a lot of resources and time, which is an unnecessary act.
FIG. 2 is a diagram of an archive storage structure of the present invention; fig. 3 is a flow chart of the present invention. The steps of the present invention are further described below in conjunction with fig. 2 and 3.
An adaptive writing archive storage method comprises the following steps:
step S1, when a file storage request exists, a log file system makes the file into a Transaction structure (Transaction); a transaction structure including a status index bit, metadata, and archive content; the state index bits record block location, block length, file retention level, and file retention time.
Taking the ext3 log file system as an example, a state index bit is newly added on the original transaction structure, so that the transaction structure comprises a transaction head (H), the state index bit, metadata Ma, archive content Da and a transaction tail (T); status index bits record block location, block length, file retention level and file retention time.
When the file needs to be refreshed, the current state of the transaction structure can be known from the state index bits of the transaction structure.
Metadata, which is data about a archive, is typically structured data, which is structured data (e.g., title, version, publication data, related descriptions, search points, etc.) extracted from the archive to describe its features, content, and to organize, describe, search, store, manage information, and knowledge resources.
The file content is the accessed file itself.
Step S2, compressible detection is carried out on the files, and the content of the files is divided into two writing modes according to the compression rate: mild writing for compressible files, severe writing for incompressible files.
In a non-volatile random access memory, different write energies may allow for different stabilities of the resistance or magnetic pole, thereby affecting the data retention time. For example, when the length of the data retention time is 10 years, the required writing energy thereof is 1.158nJ; whereas the length of the data retention time is 40ms, the required write energy is only 0.161nJ.
The light writing is suitable for files with poor stability, and the natural storage time of writing is short, so that lower energy consumption and writing time are obtained. Mild writing, comprising: metadata, archival content of high compression rate.
Heavy writing is suitable for files with better stability, and the natural storage time of writing is longer. The higher write energy required for heavy writing compared to light writing can cause more serious wear on the endurance of non-volatile random access memory. Heavy writes, including incompressible archives.
In the scheme, the compression rate is used as an index for judging the stability of the file, namely an index of the file updating frequency. Files with high compression rate, i.e. files that can be recompressed, such as text files, are updated more frequently, giving them a shorter natural shelf life during storage, i.e. with light writing. In contrast, files with low compression rate, i.e. files which cannot be compressed again, such as JPEG, MP3, etc., are updated less frequently, and most people of such multimedia files will not change after downloading.
In the aspect of file classification, if the adopted standard is the format of files, such as a secondary file name, the following defects exist, firstly, the format of each file needs to be classified, stored and recorded, and is difficult to traverse; second, if a new file format appears later, the files cannot be classified, and thus the reservation level of the file cannot be determined.
Step S3, the writing mode is judged to be a slightly written file, and the files are divided into 4 reservation levels according to the increasing sequence of the natural preservation time: primary retention level, secondary retention level, tertiary retention level, quaternary retention level.
According to the scheme, files with different reservation grades are provided with natural preservation time which is suitable for the grades. In reality, since the actual storage requirements of various files cannot be known, a preset reservation level setting mechanism is required.
First level retention class: the natural preservation time is 3 minutes;
secondary retention rating: the natural preservation time is 3 hours;
three-level retention level: the natural preservation time is 3 days;
four-level retention rating: the natural preservation time is 3 months.
According to the scheme, equal proportion distribution is not adopted for the natural preservation time, and the reason is that if the proportion is too small, the time difference of each grade is almost the same, so that all files can be updated easily, and the meaning of distinguishing is lost. Similarly, too long a separation will easily distinguish between the two classes.
The natural preservation time of the metadata is initially set as a first-level preservation level; because of the metadata, the update frequency is high.
The natural preservation time of the file content with high compression rate is initially set as a secondary preservation level; because of the high compression rate of the archival content, only a short shelf life is required.
Three level retention levels and four level retention levels for the purpose of a gradual increase in retention levels in subsequent steps.
The writing mode judges that the file is severely written, the reservation level is a four-level reservation level, and the natural preservation time is 3 months; instead of using three levels of retention, unnecessary refreshing is avoided, thereby extending the life of the memory.
After files are divided into 4 reservation levels according to natural preservation time, files which are not expected to appear, for example, files classified as 3 minutes are not changed any more after exceeding time, and if the files are not processed, files disappear, which is never happenable and fatal to users. Therefore, it is necessary to further set the delta reservation mechanism.
Step S4, regularly judging whether the file preservation time is about to be used up: if not, continuing waiting until the system judges that the file storage time is about to run out or writing into the file area when waiting for a check point stage; otherwise, the file is refreshed, the reservation level is increased by one step, the file reservation level and the file reservation time are updated in the state index bit, and if the previous reservation level is the four-level reservation level, the reservation level is reset to the four-level reservation level during refreshing.
When the file is made into a transaction structure, the file retention class and the file retention time are written into the state index bit. If the reservation level of the file is a primary reservation level, a secondary reservation level, and a tertiary reservation level, the reservation level is increased by one step and the file reservation level and the file reservation time are updated in the state index bit during the refresh operation. For example, metadata identified as primary retention level and high compression rate file content of secondary retention level, when a file needs to be refreshed, are refreshed using a higher write strength than the current retention level and the retention level of the file is updated in the state index bit. And if the files reaching the three-level retention level after refreshing are required to be refreshed, writing is performed by using the writing force of the four-level retention level.
The method for regularly judging whether the file preservation time is about to be used up comprises the following steps: checking whether the stored time of the file exceeds T/2 or not every T/3 period according to the system time, wherein T is the natural storage time of the corresponding file; if yes, judging that the file preservation time is about to be used up; otherwise, the next checking period is continued to be waited.
When a plurality of files are entered, each file cannot be entered at the same time, and the files must be different for a few seconds or even a few minutes. For example, there are three entries of secondary retention levels, which have been present for 80 minutes, 40 minutes, and 20 minutes, respectively. If the system time is 80 minutes from the time point of entering the first file data, a check time point is reached, and the stored time of the file does not reach the T/2 period, so that refreshing is not needed. Then, after 30 minutes, the system time is 110 minutes, and the first data entered at the earliest has been saved for 90 minutes over the T/2 period, but has not yet reached the system check time, so it will continue to maintain a level two data retention time. After another 30 minutes, the system check time point is reached (for example, the first time check point of the system is at 80 minutes, and the second time check point is at 140 minutes), at which time the second file is also over 90 minutes, and the first and second files are refreshed together and become files with three levels of retention. The third data, although being checked by the system, is only 80 minutes, and the refreshing upper limit of 90 minutes of the T/2 period is not exceeded, so that the data retention time of the second level is maintained until the file is changed or the next check of the system is encountered.
Therefore, even if the existence time of the file exceeds the T/2 period but is not updated, the file can be checked and updated by the system before the end of the T period, and the situation that the existence time of the file exceeds the reservation level of each level to cause the file to disappear can not occur.
According to the scheme, through checking the file preservation time, an increment reservation mechanism is designed, so that all files can be ensured not to exceed the upper limit of the preservation time of the reservation level per se; meanwhile, the progressive writing mode can increase the energy use efficiency, lighten the writing of the memory and prolong the service life.
Step S5, in the check point stage, the files are stored in the file area.
For files distributed as heavy writing, the writing mode of using In-place is used, the files are not rewritten once, the mode of changing index is used as the mode of storing In the file area, and the index position of storing the file content In the transaction structure is updated In the metadata area.
The in-place writing mode is to write the file for the first time when the file is written into the log file system, and to update the index directory in the file area by changing the index instead of repeating the writing for one time in the check point stage, and to store the index in the transaction structure. In the check point stage, the file contents Da and Db which are written again are not written again, but the index positions of Da and Db in the metadata area are changed into Da and Db in the transaction structure and recorded. That is, the locations of the actual stored file contents Da and Db will not change after entering the log file system and passing through the check point stage, but the Ma and Mb in the metadata area for storing the index locations of Da and Db will be updated to point to Da and Db stored in the transaction structure. The benefit of using this approach is that it reduces one write to memory, and the files that will be written in this way are heavily written files, which typically cause relatively large damage to memory (e.g., NVRAM), and thus have great benefit to memory with limited numbers of reads and writes.
For files allocated for light writing, a write mode using dislocation (Out-place) writes into the file area at the time of the checkpoint stage.
The staggered writing mode writes the file for the first time when the file is written into the log file system, and writes the saved transaction structure into the file area for storage in the check point stage.
The writing energy loss of the scheme is verified, an ext3 simulator based on a Linux system is used as a testing environment, an IOzone is used as a testing basis, and the IOzone can test the reading and writing performance of different file systems.
The first level of retention was written at 4867.37pJ.
The secondary retention level, write energy was 5440.00pJ.
Three levels of retention were written at 7706.67pJ.
Four levels of retention were written at 9248.00pJ.
The first level of refresh action retains rank, write energy is 2593.68 pJ.
The second level of retention of the refresh action, write energy was 2880.00pJ.
The three levels of refresh action preserve rank, write energy is 4013.33pJ.
The four levels of refresh action preserve rank, write energy is 4787.00pJ.
Firstly, the IO writing quantity of a traditional EXT3 system is tested, and the traditional EXT3 file system has no hierarchical writing force, so that the writing energy of the four heaviest reserved grades is uniformly used. At 3 months, the writing amount of the test library is 2520798356, and the writing energy is: 4867931958 x 9248 units pJ. The simulator was run for 6 months, and the portion of the write volume between 3 months and 6 months was based on four levels of retention data. At 6 months, the refresh operation, the writing amount is 33587968, and the writing energy is: 33587968 x 4787, units pJ. Thus, the write energy is 4.51793E+13pJ in total.
The IO write amount of the scheme is then tested.
Table 1, summary of light write amount
In the above table, when the file ratio of the high compression rate is 100%, most files are assigned to the secondary retention level, and at this time, the secondary retention level is the largest. The first level of the storage metadata retains the level of the archive, and its write size remains stable.
Then, the IO write amount at the time of refresh is calculated, and at this time, the first-level retention levels are all 0.
TABLE 2 write quantity at refresh
Table 3, total write energy contrast for the present scheme and conventional ext3 System
It can be seen from the above table that, with the increase of the high compression file ratio, the scheme has an obvious increase compared with the traditional ext3 system. For example, when the high compression file is 0, the scheme can save about 50% of writing energy by reducing the secondary writing and reasonably distributing the refresh period compared to the traditional ext3 system.
It will be understood that equivalents and modifications will occur to those skilled in the art in light of the present invention and their spirit, and all such modifications and substitutions are intended to be included within the scope of the present invention as defined in the following claims.
Claims (3)
1. An adaptive writing archive storage method is characterized by comprising the following steps:
step S1, when a file storage request exists, a log file system makes the file into a transaction structure; a transaction structure including a status index bit, metadata, and archive content; the state index bit records the block position, the block length, the file retention level and the file retention time;
step S2, compressible detection is carried out on the files, and the content of the files is divided into two writing modes according to the compression rate: mild writes applicable to compressible files, severe writes applicable to incompressible files;
step S3, the writing mode is judged to be a slightly written file, and the files are divided into 4 reservation levels according to the increasing sequence of the natural preservation time: primary retention level, secondary retention level, tertiary retention level, quaternary retention level;
in step S3, the first level reserve level: the natural preservation time of the file is 3 minutes;
secondary retention rating: the natural preservation time of the files is 3 hours;
three-level retention level: the natural preservation time of the file is 3 days;
four-level retention rating: the natural preservation time of the file is 3 months;
step S4, regularly judging whether the file preservation time is about to be used up: if not, continuing waiting until the system judges that the file storage time is about to run out or writing into the file area when waiting for a check point stage; otherwise, the file is refreshed, the reservation level is increased by one step, the file reservation level and the file reservation time are updated in the state index bit, and if the previous reservation level is the four-level reservation level, the reservation level is reset to the four-level reservation level during refreshing;
in step S4, the method for regularly judging whether the file storage time is about to run out is as follows: checking whether the stored time of the file exceeds T/2 or not every T/3 period according to the system time, wherein T is the natural storage time of the corresponding file; if yes, judging that the file preservation time is about to be used up; otherwise, continuing to wait for the next checking period;
step S5, a checking point stage, storing the file into a file area;
in step S5, for the files allocated as heavy writing, using a writing mode of using a location, and adopting a mode of changing an index as a mode of storing into a file area, updating the index position of storing file contents in the transaction structure into the metadata area;
for files allocated as light writing, the staggered writing mode is adopted to write the file area in the check point stage.
2. An adaptive write archive method according to claim 1, wherein in step S1, the transaction structure includes a transaction header, a state index bit, metadata, archive contents, and a transaction trailer;
in step S2, light writing includes: metadata and high compression archival content; heavy writes, including incompressible archives.
3. An adaptive write archive method according to claim 2, wherein in step S3, the natural preservation time of the metadata is initially set to a first-level retention level;
the natural preservation time of the file content with high compression rate is initially set as a secondary preservation level;
the writing mode is judged to be a file which is severely written, the reservation level is a four-level reservation level, and the natural preservation time is 3 months.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310652957.3A CN116383165B (en) | 2023-06-03 | 2023-06-03 | Adaptive writing archive storage method |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310652957.3A CN116383165B (en) | 2023-06-03 | 2023-06-03 | Adaptive writing archive storage method |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN116383165A CN116383165A (en) | 2023-07-04 |
| CN116383165B true CN116383165B (en) | 2023-08-11 |
Family
ID=86971577
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202310652957.3A Active CN116383165B (en) | 2023-06-03 | 2023-06-03 | Adaptive writing archive storage method |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN116383165B (en) |
Citations (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2000029971A2 (en) * | 1998-11-13 | 2000-05-25 | The Chase Manhattan Bank | A system and method for managing information retrievals from distributed archives |
| CN101216850A (en) * | 2008-01-11 | 2008-07-09 | 清华大学 | File systems accessing register dynamic collection method |
| CN101542446A (en) * | 2006-12-01 | 2009-09-23 | 微软公司 | System analysis and management |
| CN101639825A (en) * | 2008-07-31 | 2010-02-03 | 张炳煌 | File format structure applied to dynamic copybook writing system |
| CN102521078A (en) * | 2011-12-01 | 2012-06-27 | 北京邮电大学 | Educational administration resource grade backup method based on third party cloud |
| CN102831240A (en) * | 2012-09-05 | 2012-12-19 | 曙光信息产业(北京)有限公司 | Storage method and storage structure of extensible metadata documents |
| CN104090987A (en) * | 2014-07-28 | 2014-10-08 | 华中科技大学 | Historical data storage and indexing method |
| CN109241022A (en) * | 2018-09-11 | 2019-01-18 | 天津理工大学 | A kind of archive management system and its ant search algorithm based on blue-ray storage |
| CN113986821A (en) * | 2021-10-12 | 2022-01-28 | 北京新朝科技有限公司 | Archive file management method based on label |
| CN114265637A (en) * | 2021-12-24 | 2022-04-01 | 中电信数智科技有限公司 | Archive management system configuration method, system, electronic device and storage medium |
Family Cites Families (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US8549239B2 (en) * | 2007-06-20 | 2013-10-01 | Hewlett-Packard Development Company, L.P. | Network message logging and archival |
| US10318477B2 (en) * | 2010-05-26 | 2019-06-11 | Red Hat, Inc. | Managing and archiving system and application log files |
-
2023
- 2023-06-03 CN CN202310652957.3A patent/CN116383165B/en active Active
Patent Citations (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2000029971A2 (en) * | 1998-11-13 | 2000-05-25 | The Chase Manhattan Bank | A system and method for managing information retrievals from distributed archives |
| CN101542446A (en) * | 2006-12-01 | 2009-09-23 | 微软公司 | System analysis and management |
| CN101216850A (en) * | 2008-01-11 | 2008-07-09 | 清华大学 | File systems accessing register dynamic collection method |
| CN101639825A (en) * | 2008-07-31 | 2010-02-03 | 张炳煌 | File format structure applied to dynamic copybook writing system |
| CN102521078A (en) * | 2011-12-01 | 2012-06-27 | 北京邮电大学 | Educational administration resource grade backup method based on third party cloud |
| CN102831240A (en) * | 2012-09-05 | 2012-12-19 | 曙光信息产业(北京)有限公司 | Storage method and storage structure of extensible metadata documents |
| CN104090987A (en) * | 2014-07-28 | 2014-10-08 | 华中科技大学 | Historical data storage and indexing method |
| CN109241022A (en) * | 2018-09-11 | 2019-01-18 | 天津理工大学 | A kind of archive management system and its ant search algorithm based on blue-ray storage |
| CN113986821A (en) * | 2021-10-12 | 2022-01-28 | 北京新朝科技有限公司 | Archive file management method based on label |
| CN114265637A (en) * | 2021-12-24 | 2022-04-01 | 中电信数智科技有限公司 | Archive management system configuration method, system, electronic device and storage medium |
Non-Patent Citations (1)
| Title |
|---|
| 程志鹏 ; 蔚雪洁 ; 谭建明 ; .基于Snort的入侵检测系统的研究与实现.电脑开发与应用.2007,第24-25+28页. * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN116383165A (en) | 2023-07-04 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US9449005B2 (en) | Metadata storage system and management method for cluster file system | |
| US6636879B1 (en) | Space allocation in a write anywhere file system | |
| CN101510223B (en) | Data processing method and system | |
| CN103838853B (en) | Mixed file system based on different storage media | |
| US20080082525A1 (en) | File storage system, file storing method and file searching method therein | |
| CN101707633B (en) | Message-oriented middleware persistent message storing method based on file system | |
| CN112395212B (en) | Method and system for reducing garbage recovery and write amplification of key value separation storage system | |
| CN102945207A (en) | Cache management method and system for block-level data | |
| CN103473298A (en) | Data archiving method and device and storage system | |
| CN111400083A (en) | Data storage method and system and storage medium | |
| CN103257830A (en) | Storage cell accessing method and storage cell | |
| CN111414320B (en) | Method and system for constructing disk cache based on nonvolatile memory of log file system | |
| CN109165321B (en) | Consistent hash table construction method and system based on nonvolatile memory | |
| CN102024060B (en) | Method for memorizing files and restoring error deleted files | |
| CN113704217A (en) | Metadata and data organization architecture method in distributed persistent memory file system | |
| KR101438667B1 (en) | Database method for b+ tree based on PRAM | |
| CN114281762A (en) | Log storage acceleration method, device, equipment and medium | |
| CN108595589A (en) | A kind of efficient access method of magnanimity science data picture | |
| CN116383165B (en) | Adaptive writing archive storage method | |
| US6678787B2 (en) | DASD-free non-volatile updates | |
| CN113253932A (en) | Read-write control method and system for distributed storage system | |
| US11138110B1 (en) | Data management for memory devices | |
| KR101861475B1 (en) | Database method for PCM+ tree of non-volatile memory | |
| CN111444114B (en) | Method, device and system for processing data in nonvolatile memory | |
| CN107608626B (en) | Multi-level cache and cache method based on SSD RAID array |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |