CN109753240B

CN109753240B - Data storage method adopting garbage recovery mechanism

Info

Publication number: CN109753240B
Application number: CN201811442057.1A
Authority: CN
Inventors: 兰海洋; 李礼; 王雨雷; 吴春; 周正; 邱赐云
Original assignee: Shanghai V&g Information Technology Co ltd
Current assignee: Shanghai V&g Information Technology Co ltd
Priority date: 2018-11-28
Filing date: 2018-11-28
Publication date: 2021-12-21
Anticipated expiration: 2038-11-28
Also published as: CN109753240A

Abstract

The invention has proposed a data storage method adopting the garbage recovery mechanism, the file system deletes the catalogue where the data needing to be erased are located, send to the nand flash these data block address and address number where locate, after the nand flash control module receives these information, enable and erase the function, erase the data in the first block address, the counter adds 1, if the counter equals the address number, erase and finish; and the file system sends the idle block address and the write enable to the nand flash control module, and the nand flash control module starts to execute write operation when the data of the data source is valid after receiving the idle block address and the write enable. The invention improves the write-in bandwidth and improves the radar data acquisition efficiency; the write amplification effect does not exist, the operation on nand flash is reduced, and the service life of the storage equipment is prolonged; the correctness of the data is ensured.

Description

Data storage method adopting garbage recovery mechanism

Technical Field

The invention belongs to the field of digital circuit data communication, relates to a high-speed data storage method adopting a garbage recovery mechanism, and particularly relates to a data storage method adopting a garbage recovery mechanism.

Background

In the field of radar signal acquisition, a high-speed interface exists, and data acquired at high speed needs to be recorded in an SSD (solid state disk). Conventional storage methods suffer from write amplification problems due to the nature of SSDs that must be erased before data can be written.

Write Amplification (Write Amplification) is the amount of data that an SSD is written to does not match the amount of data that is actually written. Take a common nand flash as an example: the number of pages of a block is 128, each page contains 8KB of data, 127 pages of the current block have written data, and 8KB of space is available, at this time, if the 8KB of data is written, but the SSD cannot directly cover the existing data, the current block needs to be firstly erased and then written, the data of the other 127 pages needs to be read out into the memory, then the whole block is erased, and the written 8KB and the data in the previous 127 pages are put together and written into one block again. The data volume actually written in the process is 1024KB (1M data volume), the written data volume is enlarged by 128 times in the whole process, and the nand flash performs reading, erasing and writing operations in the process, so that the writing bandwidth of the nand flash is greatly reduced.

In the application of radar high-speed data acquisition projects, the data volume at the data source end is very large and continuous, so that it is very important to ensure the correctness and continuity of the data, which is directly related to the performance of the system and the success or failure of the project. However, in the conventional SSD storage mechanism with write amplification, valid data is lost during the writing process, as shown in fig. 1, a work flow of writing data in the conventional SSD is shown.

The traditional SSD data storage flow is as follows: the nand flash control module receives data of a data source, all effective data to be written into the block are read out and the data of the data source are spliced into a data block, the erasing module is enabled after the data are read out, the current block is erased, and the data block is written into the nand flash after the erasing is finished. In this process, it is necessary to perform read operation, erase operation, and then write operation on the flash, so that a lot of time is consumed, and the time consumed by the conventional method is analyzed by taking the erase timing as an example. FIG. 2 is a nand flash erase sequence.

When the erasing operation is executed, an erasing command and an address are sent to the nand flash, and the erasing is completed after waiting for the tBERS time. Whereas the time-typical value of tBERS is 1.5ms, with a maximum of 7 ms. As shown in fig. 3.

The time consumed by the reading operation is different according to the difference of the data size in the block, and if the nand flash clock is 40MHz, only one page in the current block writes data, and the size of one page is 8192B, the time consumed by the reading operation is 204.8 us; if the current block has 127 pages to which data is written, the time consumed for reading is 26 ms. This can be concluded that writing a block of data takes at least an additional 1.5ms to 33 ms. If the next packet of data has arrived within this time, the next packet of data is dropped, resulting in incomplete data.

Disclosure of Invention

Aiming at the defects of the prior art, the invention adopts a new garbage recovery mechanism by analyzing the application scene of radar signal acquisition, avoids amplification and does not need garbage recovery in the writing process. When deleting data, the file system sets a state flag, which records the state information of the file, in the directory area to be deleted and also clears the data in the whole block, so that the write amplification effect is avoided, the write bandwidth is improved, and the SSD can be applied to high-speed radar data acquisition items.

In order to achieve high-speed storage and ensure the correctness and continuity of data, a customized garbage recycling mechanism is used in the invention, data storage is carried out by taking block as a unit when the data is stored, and as the data volume of an application scene is very large, the space is not wasted too much. When some data needs to be deleted, the file system sets a state flag, which records the file state information, in the directory area to be deleted, and simultaneously sends a block address to be deleted to the nand flash control module, and the nand flash control module clears all data stored in the blocks. When data is written, the nand flash control module directly stores the data in an address corresponding to the nand flash when receiving the data of the data source, and the functions of reading and erasing are not needed, so that the write amplification effect is avoided when the data is written, the write time is greatly saved, the write bandwidth is improved, and the SSD storage device can be applied to a high-speed acquisition system. As shown in fig. 4. The invention adopts the following specific technical scheme:

a data storage method adopting a garbage collection mechanism comprises two processes of erasing data and writing data.

1. Erase data process

Step 1: when data needs to be erased, the file system sets the state flag of the data needing to be erased to be deleted;

step 2: the file system sends the block address and the address number of the data to the storage medium;

and step 3: after the storage medium control module receives the block information, enabling an erasing function;

and 4, step 4: erasing data in a first block address, and adding 1 to a counter;

and 5: judging whether the counter is equal to the number of the addresses or not, and if not, continuing to erase the data in the next block address; if yes, ending the erasing function;

step 6: and (6) ending.

2. Data writing process

Step 1: the file system sends an idle block address and write enable to the storage medium control module;

step 2: the storage medium control module receives the address and write enable;

and step 3: starting to execute the write operation when the data of the data source is valid, and finishing the write operation of the storage medium control module if the file system does not enable the write operation;

and 4, step 4: the data write to write the data is complete.

Further, the storage medium comprises a nand flash memory, an SSD memory and the like.

The key technology of the invention is that the file system deletes the directory together with the data in the flash after deleting the directory, thereby ensuring the real-time bandwidth of the written data. The invention deletes the directory together with the data, namely the file system deletes the data in the nand flash once in the process of deleting the data, thereby ensuring the speed of writing operation, improving the writing bandwidth of the equipment and prolonging the service life of the equipment. Has the following beneficial effects:

1. the write-in bandwidth is improved, and the radar data acquisition efficiency is improved.

2. The write amplification effect does not exist, the operation on nand flash is reduced, and the service life of the storage device is prolonged.

3. The correctness of the data is ensured.

Drawings

FIG. 1 is a conventional SSD storage data flow diagram;

FIG. 2 is a schematic diagram of a Nand flash erase timing sequence;

FIG. 3 is a drawing showing^tSpecification of BERS parameters;

FIG. 4 is a schematic diagram of a customized garbage collection mechanism;

FIG. 5 is a flow chart of erasing data;

FIG. 6 is a flow chart of a write data operation execution.

Detailed Description

Referring to fig. 5, during the erasing process: when data needs to be erased, the file system deletes a directory where the data needing to be erased is located, and sends a block address where the data are located and the number of addresses to the nand flash, after the nand flash control module receives the block information, the erase function is enabled, the data in the first block address are erased firstly, the counter is added with 1, and if the counter is equal to the number of the addresses, the erase is completed.

Referring to fig. 6, the data writing process: the file system sends an idle block address and a write enable to the nand flash control module, the nand flash control module receives the address and the write enable, and starts to execute write operation when data of the data source is valid, and if the file system does not enable the write operation, the nand flash control module finishes the write operation execution.

Claims

1. A data storage method adopting a garbage recovery mechanism is characterized by comprising the following steps: erasing data and writing data;

the erasing data process comprises the following steps:

1) the file system sets the state flag of the data needing to be erased into deletion;

2) the file system sends the block address and the address number of the data to the storage medium;

3) after the storage medium control module receives the block information, enabling an erasing function;

4) erasing data in a first block address, and adding 1 to a counter;

5) judging whether the counter is equal to the number of the addresses or not, and if not, continuing to erase the data in the next block address; if yes, ending the erasing function;

the data writing process comprises the following steps:

1) the file system sends an idle block address and write enable to the storage medium control module;

2) the storage medium control module receives a block address and write enable;

3) and starting to execute the write operation when the data of the data source is valid, and finishing the write operation of the storage medium control module if the file system does not enable the write operation.

2. The data storage method adopting the garbage collection mechanism according to claim 1, comprising: the storage medium comprises a nand flash memory and an SSD memory.