CN111880731B - Data processing method and device and related components - Google Patents
Data processing method and device and related components Download PDFInfo
- Publication number
- CN111880731B CN111880731B CN202010701050.8A CN202010701050A CN111880731B CN 111880731 B CN111880731 B CN 111880731B CN 202010701050 A CN202010701050 A CN 202010701050A CN 111880731 B CN111880731 B CN 111880731B
- Authority
- CN
- China
- Prior art keywords
- data
- real
- data block
- invalid
- storage system
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0602—Interfaces specially adapted for storage systems specifically adapted to achieve a particular effect
- G06F3/0608—Saving storage space on storage systems
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/10—File systems; File servers
- G06F16/17—Details of further file system functions
- G06F16/174—Redundancy elimination performed by the file system
- G06F16/1744—Redundancy elimination performed by the file system using compression, e.g. sparse files
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0628—Interfaces specially adapted for storage systems making use of a particular technique
- G06F3/0638—Organizing or formatting or addressing of data
- G06F3/064—Management of blocks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0628—Interfaces specially adapted for storage systems making use of a particular technique
- G06F3/0646—Horizontal data movement in storage systems, i.e. moving data in between storage devices or systems
- G06F3/0647—Migration mechanisms
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0628—Interfaces specially adapted for storage systems making use of a particular technique
- G06F3/0646—Horizontal data movement in storage systems, i.e. moving data in between storage devices or systems
- G06F3/0652—Erasing, e.g. deleting, data cleaning, moving of data to a wastebasket
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
- Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Human Computer Interaction (AREA)
- Data Mining & Analysis (AREA)
- Databases & Information Systems (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The application discloses a data processing method, which is applied to a real-time compression storage system and comprises the following steps: when the update data is acquired, determining a target data block corresponding to the update data; calculating invalid information of the target data block; judging whether the real-time compression storage system meets garbage recovery conditions or not; if so, releasing invalid data in the target data block, dividing the data in the target data block into cold data and hot data, and respectively storing the cold data and the hot data into corresponding idle data blocks according to the invalid information. The method and the device can ensure that the cold data in each data block only needs to be migrated once, do not need to waste excessive resources of the real-time compression storage system, and improve the performance of the real-time compression storage system. The application also discloses a data processing device, an electronic device and a computer readable storage medium, which have the beneficial effects.
Description
Technical Field
The present application relates to the field of storage servers, and in particular, to a data processing method, apparatus, and related components.
Background
The exponential growth of data has higher and higher requirements on storage capacity, and at present, two ways are usually adopted to deal with the growth of storage capacity, one is to expand the storage capacity, but the cost is increased; the other method is to compress the data to reduce the consumption of the storage capacity so as to achieve the purpose of saving the cost. The real-time compression storage system compresses data before the data are written into the storage device, so that the occupation of storage space is reduced, and the access to the storage device (such as a magnetic disk) can be further reduced, thereby improving the performance of the storage system.
The current real-time compression storage system sequentially writes data into a storage space by taking a data block as a unit after compressing the data, generates invalid data in the data block due to the updating of the data, and when the space of the real-time compression storage system is insufficient, the real-time compression storage system needs to execute a garbage collection algorithm to release the space occupied by the invalid data to be free space so as to continuously store the new incoming data. In the process of executing the garbage collection algorithm, cold data in the data block can be migrated for multiple times along with merging of effective data in the data block, and resources of the real-time compression storage system are wasted, so that the performance of the real-time compression storage system is reduced.
Therefore, how to provide a solution to the above technical problem is a problem that needs to be solved by those skilled in the art.
Disclosure of Invention
The application aims to provide a data processing method, a data processing device, an electronic device and a computer readable storage medium, which can ensure that cold data in each data block only needs to be migrated once, do not waste excessive resources of a real-time compression storage system, and improve the performance of the real-time compression storage system.
In order to solve the above technical problem, the present application provides a data processing method, applied to a real-time compression storage system, including:
when the update data is acquired, determining a target data block corresponding to the update data;
calculating invalid information of the target data block;
judging whether the real-time compression storage system meets garbage recovery conditions or not;
if so, releasing invalid data in the target data block, dividing the valid data in the target data block into cold data and hot data, and respectively storing the cold data and the hot data into respective corresponding idle data blocks according to the invalid information.
Preferably, after determining the target data block corresponding to the update data, the data processing method further includes:
and marking invalid data corresponding to the updated data in the target data block, and recording the length of the invalid data.
Preferably, the invalid information is a ratio of a length of the invalid data to a length of the target data block.
Preferably, the process of respectively storing the cold data and the hot data into respective corresponding idle data blocks according to the invalid information specifically includes:
determining the processing sequence of the target data blocks according to the sequence of the ratio from large to small;
and sequentially storing the cold data and the hot data in each target data block to the corresponding idle data blocks respectively according to the processing sequence.
Preferably, the process of determining whether the real-time compression storage system meets the garbage collection condition specifically includes:
judging whether the free space of the real-time compression storage system is smaller than a preset value or not;
if yes, judging that the real-time compression storage system meets garbage recovery conditions;
if not, judging that the real-time compression storage system does not meet the garbage recovery condition.
Preferably, the process of dividing the valid data in the target data block into cold data and hot data specifically includes:
and dividing effective data in the target data block into cold data and hot data according to the write access frequency, wherein the data with the write access frequency of 1 is the cold data, and the data with the write access frequency of more than 1 is the hot data.
In order to solve the above technical problem, the present application further provides a data processing apparatus applied to a real-time compression storage system, including:
the determining module is used for determining a target data block corresponding to the updating data when the updating data is acquired;
the calculation module is used for calculating invalid information of the target data block;
the judging module is used for judging whether the real-time compression storage system meets garbage recycling conditions or not, and if yes, the processing module is triggered;
the processing module is configured to release invalid data in the target data block, divide the valid data in the target data block into cold data and hot data, and store the cold data and the hot data in respective corresponding idle data blocks according to the invalid information.
Preferably, the data processing apparatus further includes:
and the marking module is used for marking invalid data corresponding to the updating data in the target data block and recording the length of the invalid data.
In order to solve the above technical problem, the present application further provides an electronic device, including:
a memory for storing a computer program;
a processor for implementing the steps of the data processing method as described in any one of the above when executing the computer program.
To solve the above technical problem, the present application further provides a computer-readable storage medium having a computer program stored thereon, where the computer program is executed by a processor to implement the steps of the data processing method according to any one of the above.
The application provides a data processing method, which is applied to a real-time compression storage system, when updated data is obtained, invalid data exists in a target data block corresponding to the updated data, invalid information of the target data block is calculated at the moment, when the real-time compression storage system meets a garbage recovery condition, the invalid data of the target data block is released, so that the space of the real-time compression storage system is released, the remaining valid data in the data block is divided according to cold and hot data, the cold and hot data are respectively merged into corresponding idle data blocks, so that the cold data in each data block only needs to be migrated once, excessive resources of the real-time compression storage system do not need to be wasted, and the performance of the real-time compression storage system is improved. The application also provides a data processing device, an electronic device and a computer readable storage medium, which have the same beneficial effects as the data processing method.
Drawings
In order to more clearly illustrate the embodiments of the present application, the drawings needed for the embodiments will be briefly described below, and it is obvious that the drawings in the following description are only some embodiments of the present application, and that other drawings can be obtained by those skilled in the art without inventive effort.
FIG. 1 is a flow chart illustrating steps of a data processing method provided herein;
fig. 2 is a schematic structural diagram of a data processing apparatus provided in the present application.
Detailed Description
The core of the application is to provide a data processing method, a data processing device, an electronic device and a computer readable storage medium, which can ensure that cold data in each data block only needs to be migrated once, avoid wasting excessive resources of a real-time compression storage system, and improve the performance of the real-time compression storage system.
In order to make the objects, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application, and it is obvious that the described embodiments are some embodiments of the present application, but not all embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present application.
Referring to fig. 1, fig. 1 is a flowchart illustrating steps of a data processing method applied to a real-time compression storage system, where the data processing method includes:
s101: when the update data is acquired, determining a target data block corresponding to the update data;
specifically, before executing this step, an operation of determining that the received data is update data or new data may be further included, and if the received data is new data, the new data is written into a corresponding data block, the update data in this embodiment is specifically update data of existing old data, and if the written data is update data, a target data block corresponding to the update data is determined, where the target data block is a data block where the old data corresponding to the update data is located, and generally, the update data may be written into a data block different from the data block where the corresponding old data is located. As a preferred embodiment, for the updated data, the target data block where the corresponding old data is located can be determined by querying the metadata.
Specifically, since the updated data exists, the old data corresponding to the updated data in the target data block is invalid data, and the invalid data is considered to be meaningless and also occupy the space of the real-time compressed storage system, the step aims to determine the position of the invalid data, so that when the space of the real-time compressed storage system is small, the position of the invalid data can be quickly located, and the invalid data in the target data block is released.
It can be understood that, in this embodiment, the write access frequency of data is recorded during data writing, and specifically, the write access frequency of the data may be determined according to the write address, if the address is accessed for the first time, the write access frequency of the data is recorded as 1, and when the address is accessed again for the next time, the write access frequency of the data is recorded as 2, and the write access frequencies are sequentially accumulated, where the write address may specifically refer to an LBA.
S102: calculating invalid information of the target data block;
specifically, the invalid information in this step may be a ratio of a length of the invalid data to a length of the target data block. Before executing the step, an operation of obtaining the length of the target data block may be further included, and it is understood that the data corresponding to the update data in the target data block is invalid data, so that the invalid information of the target data block is recalculated once every time the update data corresponding to the data in the target data block is received, so as to ensure the reliability of the invalid information of the target data block.
S103: judging whether the real-time compression storage system meets garbage recovery conditions, if so, executing S104;
s104: and releasing invalid data in the target data block, dividing the valid data in the target data block into cold data and hot data, and respectively storing the cold data and the hot data into corresponding idle data blocks according to the invalid information.
Specifically, whether the real-time compression storage system meets the garbage recycling condition is judged, if yes, the garbage recycling needs to be executed on the real-time compression storage system, and if not, the data in the real-time compression storage system is not processed, wherein whether the real-time compression storage system meets the garbage recycling condition can be judged according to the size of the free space of the real-time compression storage system, if the free space of the real-time compression storage system is smaller than or equal to a preset value, the judgment is met, otherwise, the judgment is not met, and the preset value can be set to be 10% of the rated storage space.
Specifically, when the garbage collection condition is met, the invalid data in the target data block is released, so that the space of the real-time compression storage system is released. Then dividing the effective data in the target data block into cold data and hot data, merging the cold data into one free data block, merging the hot data into another free data block, so that the space occupied by the original target data block is released to become free space. As a preferred embodiment, the valid data in the target data block may be divided into cold data and hot data according to the write access frequency, where the data with the write access frequency of 1 is the cold data, and the data with the write access frequency of more than 1 is the hot data. Specifically, cold data and hot data are stored respectively in the garbage recycling process, so that the data volume transferred in the garbage recycling process can be reduced, and the performance of a real-time compression storage system is improved. Further, S103 to S104 should be repeated to control the size of the free space of the real-time compression storage system to be above the preset value.
It can be understood that, considering that a plurality of target data blocks may exist simultaneously, after cold data in a certain target data block is stored in the free data block a, cold data in a next target data block is also stored in the free data block a until the free data block a is full, and then cold data in the remaining target data blocks is written in a new free data block B, and so on, hot data, and the like, thereby avoiding excessive occupation of the free data block.
It can be seen that, in this embodiment, when the update data is acquired, it is stated that invalid data exists in a target data block corresponding to the update data, at this time, invalid information of the target data block is calculated, and when the real-time compression storage system meets a garbage collection condition, the invalid data of the target data block is released, so that a space of the real-time compression storage system is released, remaining valid data in the data block is divided according to cold data and hot data, and the cold data and the hot data are respectively merged into respective corresponding idle data blocks, so that the cold data in each data block only needs to be migrated once, excessive resources of the real-time compression storage system do not need to be wasted, and performance of the real-time compression storage system is improved.
On the basis of the above-described embodiment:
as a preferred embodiment, after determining the target data block corresponding to the update data, the data processing method further includes:
and marking invalid data corresponding to the updated data in the target data block, and recording the length of the invalid data.
Specifically, in the data writing process, after a target data block corresponding to the update data is determined, invalid data corresponding to the update data in the target data block is marked, so that invalid information of the target data block can be calculated subsequently according to the lengths of all marked invalid data in the target data block, the accuracy of the invalid information is ensured, and in addition, the invalid data in the target data block can be released subsequently.
As a preferred embodiment, the process of storing the cold data and the hot data into the respective corresponding free data blocks according to the invalidation information specifically includes:
determining the processing sequence of the target data blocks according to the sequence of the ratio from large to small;
and sequentially storing the cold data and the hot data in each target data block into the corresponding idle data blocks respectively according to the processing sequence.
Specifically, considering that a real-time compressed storage system can simultaneously include a plurality of target data blocks, in order to improve the efficiency of releasing the space in the real-time compressed storage system, the present application first determines the processing order of the target data blocks according to the sequence from the large ratio to the small ratio, and it can be understood that the larger ratio indicates that the more invalid data in the target data block is, and therefore, the target data block with the larger ratio should be processed first.
Referring to fig. 2, fig. 2 is a schematic structural diagram of a data processing apparatus applied to a real-time compression storage system, the data processing apparatus including:
the determining module 1 is configured to determine, when the update data is acquired, a target data block corresponding to the update data;
the calculation module 2 is used for calculating invalid information of the target data block;
the judging module 3 is used for judging whether the real-time compression storage system meets the garbage recycling condition, and if so, the processing module 4 is triggered;
and the processing module 4 is configured to release invalid data in the target data block, divide the valid data in the target data block into cold data and hot data, and store the cold data and the hot data in respective corresponding idle data blocks according to the invalid information.
It can be seen that, in this embodiment, when the update data is acquired, it is indicated that invalid data exists in the target data block corresponding to the update data, at this time, invalid information of the target data block is calculated, and when the real-time compression storage system meets the garbage collection condition, the invalid data of the target data block is released, so that the space of the real-time compression storage system is released, the remaining valid data in the data block is divided according to cold and hot data, and the cold and hot data are respectively merged into respective corresponding idle data blocks, so that the cold data in each data block only needs to be migrated once, excessive resources of the real-time compression storage system do not need to be wasted, and the performance of the real-time compression storage system is improved.
As a preferred embodiment, the data processing apparatus further includes:
and the marking module is used for marking invalid data corresponding to the updated data in the target data block and recording the length of the invalid data.
As a preferred embodiment, the invalid information is a ratio of the length of the invalid data to the length of the target data block.
As a preferred embodiment, the process of storing the cold data and the hot data into the respective corresponding free data blocks according to the invalidation information specifically includes:
determining the processing sequence of the target data blocks according to the sequence of the ratio from large to small;
and sequentially storing the cold data and the hot data in each target data block into the corresponding idle data blocks respectively according to the processing sequence.
As a preferred embodiment, the process of determining whether the real-time compression storage system meets the garbage collection condition specifically includes:
judging whether the free space of the real-time compression storage system is smaller than a preset value or not;
if yes, judging that the real-time compression storage system meets the garbage recovery condition;
if not, the real-time compression storage system is judged not to meet the garbage recovery condition.
As a preferred embodiment, the process of dividing the valid data in the target data block into cold data and hot data specifically includes:
and dividing effective data in the target data block into cold data and hot data according to the write access frequency, wherein the data with the write access frequency of 1 is the cold data, and the data with the write access frequency of more than 1 is the hot data.
In another aspect, the present application further provides an electronic device, including:
a memory for storing a computer program;
a processor for implementing the steps of the data processing method as described in any of the above embodiments when executing the computer program.
For an introduction of an electronic device provided in the present application, please refer to the above embodiments, which are not described herein again.
The electronic equipment provided by the application has the same beneficial effects as the data processing method.
In another aspect, the present application further provides a computer-readable storage medium, on which a computer program is stored, which, when executed by a processor, implements the steps of the data processing method as described in any one of the above embodiments.
Please refer to the above embodiments for an introduction of a computer-readable storage medium provided in the present application, which is not described herein again.
The computer-readable storage medium provided by the application has the same beneficial effects as the data processing method.
It is further noted that, in the present specification, relational terms such as first and second, and the like are used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Also, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrases "comprising a," "8230," "8230," or "comprising" does not exclude the presence of other like elements in a process, method, article, or apparatus that comprises the element.
The previous description of the disclosed embodiments is provided to enable any person skilled in the art to make or use the present application. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the generic principles defined herein may be applied to other embodiments without departing from the spirit or scope of the application. Thus, the present application is not intended to be limited to the embodiments shown herein but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.
Claims (6)
1. A data processing method is applied to a real-time compression storage system and comprises the following steps:
when the update data is acquired, determining a target data block corresponding to the update data;
calculating invalid information of the target data block;
judging whether the real-time compression storage system meets garbage recovery conditions or not;
if so, releasing invalid data in the target data block, dividing the valid data in the target data block into cold data and hot data, and respectively storing the cold data and the hot data into respective corresponding idle data blocks according to the invalid information;
after determining the target data block corresponding to the update data, the data processing method further includes:
marking invalid data corresponding to the updated data in the target data block, and recording the length of the invalid data;
the invalid information is the ratio of the length of the invalid data to the length of the target data block;
the process of respectively storing the cold data and the hot data into the respective corresponding idle data blocks according to the invalid information specifically includes:
determining the processing sequence of the target data blocks according to the sequence of the ratio from large to small;
and sequentially storing the cold data and the hot data in each target data block to the corresponding idle data blocks respectively according to the processing sequence.
2. The data processing method according to claim 1, wherein the process of determining whether the real-time compressed storage system meets the garbage collection condition specifically comprises:
judging whether the free space of the real-time compression storage system is smaller than a preset value or not;
if yes, judging that the real-time compression storage system meets garbage recovery conditions;
if not, judging that the real-time compression storage system does not meet the garbage recovery condition.
3. The data processing method according to any one of claims 1 to 2, wherein the process of dividing the valid data in the target data block into cold data and hot data specifically comprises:
and dividing effective data in the target data block into cold data and hot data according to the write access frequency, wherein the data with the write access frequency of 1 is the cold data, and the data with the write access frequency of more than 1 is the hot data.
4. A data processing device, applied to a real-time compression storage system, comprises:
the determining module is used for determining a target data block corresponding to the updating data when the updating data is acquired;
the calculation module is used for calculating invalid information of the target data block;
the judging module is used for judging whether the real-time compression storage system meets garbage recycling conditions or not, and if yes, the processing module is triggered;
the processing module is used for releasing invalid data in the target data block, dividing the valid data in the target data block into cold data and hot data, and respectively storing the cold data and the hot data into corresponding idle data blocks according to the invalid information;
the data processing apparatus further includes:
the marking module is used for marking invalid data corresponding to the updating data in the target data block and recording the length of the invalid data;
the invalid information is the ratio of the length of the invalid data to the length of the target data block;
the process of respectively storing the cold data and the hot data into the respective corresponding idle data blocks according to the invalid information specifically includes:
determining the processing sequence of the target data blocks according to the sequence of the ratio from large to small;
and sequentially storing the cold data and the hot data in each target data block to the corresponding idle data blocks respectively according to the processing sequence.
5. An electronic device, comprising:
a memory for storing a computer program;
a processor for implementing the steps of the data processing method according to any one of claims 1 to 3 when executing said computer program.
6. A computer-readable storage medium, characterized in that a computer program is stored on the computer-readable storage medium, which computer program, when being executed by a processor, carries out the steps of the data processing method according to any one of claims 1-3.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010701050.8A CN111880731B (en) | 2020-07-17 | 2020-07-17 | Data processing method and device and related components |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010701050.8A CN111880731B (en) | 2020-07-17 | 2020-07-17 | Data processing method and device and related components |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN111880731A CN111880731A (en) | 2020-11-03 |
| CN111880731B true CN111880731B (en) | 2023-03-21 |
Family
ID=73154558
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010701050.8A Active CN111880731B (en) | 2020-07-17 | 2020-07-17 | Data processing method and device and related components |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111880731B (en) |
Families Citing this family (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113126918B (en) * | 2021-04-01 | 2023-02-28 | 山东英信计算机技术有限公司 | Fragmented data reading method, device and medium |
| CN113253923B (en) * | 2021-04-28 | 2023-05-09 | 锐掣(杭州)科技有限公司 | Data processing method, device, equipment, medium and product |
| CN114138183B (en) * | 2021-10-29 | 2023-09-12 | 郑州云海信息技术有限公司 | Data layering method, device and equipment of storage system and storage medium |
| CN115827660B (en) * | 2023-02-14 | 2023-05-12 | 本原数据(北京)信息技术有限公司 | Data updating method and device, electronic equipment and nonvolatile storage medium |
Citations (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106201906A (en) * | 2016-07-11 | 2016-12-07 | 浪潮(北京)电子信息产业有限公司 | A kind of cold and hot data separation method for flash memory and system |
Family Cites Families (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104516471B (en) * | 2013-09-27 | 2017-04-12 | 国际商业机器公司 | Method and device for managing power supply of storage system |
| CN109558244A (en) * | 2018-12-03 | 2019-04-02 | 郑州云海信息技术有限公司 | Junk information recovery method, device, equipment and computer readable storage medium |
| CN109683830A (en) * | 2019-01-08 | 2019-04-26 | 郑州云海信息技术有限公司 | A kind of method for writing data, system and the associated component of full flash memory array |
| CN110069218A (en) * | 2019-04-22 | 2019-07-30 | 珠海全志科技股份有限公司 | Cold and hot data separation method, device, computer equipment and storage medium |
-
2020
- 2020-07-17 CN CN202010701050.8A patent/CN111880731B/en active Active
Patent Citations (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106201906A (en) * | 2016-07-11 | 2016-12-07 | 浪潮(北京)电子信息产业有限公司 | A kind of cold and hot data separation method for flash memory and system |
Non-Patent Citations (3)
| Title |
|---|
| HDFS数据动态分布设计与实现;蔡宇昂等;《绿色科技》(第06期);全文 * |
| The reconstruction of MODIS land surface temperature products using NSSR;Wenping Yu;《IEEE》;全文 * |
| 移动设备文件系统冷热数据管理策略;袁玉环;《信息科技》(第3期);全文 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN111880731A (en) | 2020-11-03 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111880731B (en) | Data processing method and device and related components | |
| US11853549B2 (en) | Index storage in shingled magnetic recording (SMR) storage system with non-shingled region | |
| US10802718B2 (en) | Method and device for determination of garbage collector thread number and activity management in log-structured file systems | |
| US9965394B2 (en) | Selective compression in data storage systems | |
| US10466932B2 (en) | Cache data placement for compression in data storage systems | |
| CN109614377A (en) | File delet method, device, equipment and the storage medium of distributed file system | |
| CN110879800A (en) | Data writing, compressing and reading method, data processing method and device | |
| CN110147331B (en) | Cache data processing method and system and readable storage medium | |
| EP3958120A1 (en) | Method and device for anonymous page management, terminal device, and readable storage medium | |
| CN107665098B (en) | Information processing method, storage device, and computer storage medium | |
| CN112286459A (en) | Data processing method, device, equipment and medium | |
| CN110555001A (en) | data processing method, device, terminal and medium | |
| WO2024119775A1 (en) | Raid card node updating method and system, and related apparatus | |
| CN108304259B (en) | Memory management method and system | |
| CN109558456A (en) | A kind of file migration method, apparatus, equipment and readable storage medium storing program for executing | |
| CN112799595A (en) | Data processing method, device and storage medium | |
| CN110955388A (en) | Storage server, data storage method and device thereof and storage medium | |
| CN106557277A (en) | The reading method and device of disk array | |
| CN110658999B (en) | Information updating method, device, equipment and computer readable storage medium | |
| CN110795034B (en) | Data migration method, device and equipment of storage system and readable storage medium | |
| CN110688226B (en) | Cache recovery method, device and equipment and readable storage medium | |
| CN109284237B (en) | Garbage recovery method and system in full flash memory array | |
| CN111143288A (en) | Data storage method, system and related device | |
| CN112463048A (en) | Compact processing speed adjusting method and device, electronic equipment and medium | |
| CN112925629B (en) | Bloom filter dynamic adjustment method, bloom filter dynamic adjustment system, electronic equipment and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |