CN113157512B

CN113157512B - Method for realizing data verification test of large-capacity SSD on small memory machine

Info

Publication number: CN113157512B
Application number: CN202110472983.9A
Authority: CN
Inventors: 王璞; 刘正主; 李铁; 段好强
Original assignee: Shandong Sinochip Semiconductors Co Ltd
Current assignee: Shandong Sinochip Semiconductors Co Ltd
Priority date: 2021-04-29
Filing date: 2021-04-29
Publication date: 2022-12-02
Anticipated expiration: 2041-04-29
Also published as: CN113157512A

Abstract

The invention discloses a method for realizing data verification test of a high-capacity SSD on a small memory machine, which comprises two parts of random data writing and data verification, wherein during the random data writing, the storage mode of an LBA sequence is determined according to a mode parameter in a command parameter, when the mode =1, a low-configuration test machine is encountered, the LBA sequence is not recorded, when the mode =2, the LBA sequence is stored in a local file, and when the mode =3, the LBA sequence is stored in a memory through bitmap; and during data verification, generating an LBA sequence, and adopting different verification modes according to the mode of storing the LBA sequence during data writing. The invention effectively meets the requirement of the SSD device data consistency test with the continuously increased content in the industry, has low requirement on a test machine, improves the efficiency and quality of the test and effectively reduces the test cost.

Description

Method for realizing data verification test of large-capacity SSD on small memory machine

Technical Field

The invention relates to the field of hard disk testing, in particular to a method for realizing data verification testing of a high-capacity SSD on a small memory machine.

Background

In a data consistency verification test of the SSD storage device, data written into the SSD device needs to be read again for verification so as to verify the data storage function of the SSD device. In this test scenario, the mainstream test tool fio currently used in the industry has a defect when performing full disk data verification on a common pc machine with a small memory in response to the ssd device with an increasing capacity.

Aiming at the scene, the implementation scheme and the defects of the main flow testing tool fio in the industry are as follows:

and writing a certain amount of io in the firmware through a verify _ back log parameter, then verifying, and writing the io in the next batch, so that the data consistency test of the whole disk is completed by circularly writing and reading verification. The defect of this is that in the whole-disk random write verification process, if the written and verified data has garbage collection (gc) errors, but the cyclic write and read verification is normal, the software tool cannot find the errors. Full disk data checks must be performed to enable discovery.

The firmware needs to be completely written in first and then completely read and checked without setting a verify _ backlog parameter, but the scheme has small memory and low configuration for a test machine, and cannot meet the test requirement.

Disclosure of Invention

Aiming at the defects of the prior art, the invention provides a method for realizing data verification test of a high-capacity SSD on a small memory machine, which replaces fio to complete the full disk data verification test, has low requirement on the memory of a test machine, and can meet the test requirement of the current common pc machine.

In order to solve the technical problem, the technical scheme adopted by the invention is as follows: a method for realizing data verification test of a large-capacity SSD on a small memory machine is disclosed, wherein the large-capacity SSD refers to an SSD with the capacity larger than or equal to 8T, the small memory machine refers to a machine with the memory smaller than or equal to 1G, the method comprises two parts of random data writing and data verification, when the data is randomly written, the storage mode of an LBA sequence is determined according to a mode parameter in command parameters, when the mode =1, a low-configuration test machine is encountered, the LBA sequence is not recorded, when the mode =2, the LBA sequence is stored in a local file, and when the mode =3, the LBA sequence is stored in the memory through bitmap;

and during data verification, generating an LBA sequence, and adopting different verification modes according to the mode of storing the LBA sequence during data writing. When mode =1, the write operation does not record the LBA sequence, the random LBA address is directly read, and whether to perform verification is determined by reading the identification information in the content; when mode =2, the LBA sequence is stored in the local file, the locally stored LBA sequence file is opened, whether the LBA address is written before is determined, and then the LBA address is read and checked. And when mode =3, judging whether to perform read check on the randomly generated lba address by comparing the bitmap stored in the memory.

Furthermore, when data is written, n memories are applied, available memories are dynamically searched and restored and cleaned in the applied n memories, and memory management and data writing are achieved by matching with the function of the liboio library function.

Further, when data is written or data is verified, the total amount to be written, the verification amount and the specified IO packet size in the command parameters are obtained, the number of write-read rounds is specified to be equal to the total amount to be written divided by the verification amount, the number of submission times is equal to the verification amount divided by the specified IO packet size, the number of write-read rounds is used as the number of outer cycles, the number of submission times is used as the number of inner cycles, and the write-in amount is reached until the cycle is completed.

Further, the specific process of writing data is as follows:

s11), analyzing the command parameters, acquiring the total amount to be written, the check amount and the specified IO packet size in the command parameters, and enabling the number of writing and reading rounds to be equal to the total amount to be written divided by the check amount and the number of submission times to be equal to the check amount divided by the specified IO packet size;

s12), dynamically applying for n memories;

s13) searching an available memory in the applied memory, selecting a random or sequential LBA sequence according to the rw parameter, and preparing an IO message body; determining a storage mode of the LBA random sequence according to a mode parameter in the command parameter, when the mode =1, when a low-configuration test machine is encountered, the LBA random sequence is not recorded, when the mode =2, the LBA random sequence is stored in a local file, and when the mode =3, the LBA random sequence is stored in a memory through a bitmap;

s14), submitting an io structure through the io _ submit, repeatedly submitting the io which fails to be submitted due to insufficient available resources in the exception processing of the execution result of the io _ submit function, timing, if the continuous submission fails for more than 30S, quitting the exception, and if the submission is successful within 30S, continuing the process;

s15) submitting 1 io structure each time, obtaining an io execution result through io _ getevnts, if the execution result is obtained, cleaning the occupied memory, counting, and if the execution result is not obtained, continuing to execute the step S13;

s16), judging whether the number of submission times is reached, if so, finishing the circulation, and if not, continuing to execute the step S13;

s17), judging whether the number of writing and reading turns is reached, if so, finishing the circulation, and if not, continuing to execute the step S12.

Further, the specific process of data verification is as follows:

s21), analyzing the command parameters, acquiring the total amount to be written, the check amount and the specified IO packet size in the command parameters, and enabling the number of writing and reading rounds to be equal to the total amount to be written divided by the check amount and the number of submission times to be equal to the check amount divided by the specified IO packet size;

s22), according to the rw parameter, randomly or sequentially generating an LBA sequence, and according to a mode parameter during data writing, selecting a verification mode after the LBA is read; the reading operation submits an IO reading message structure through a libario library function IO _ submit, and obtains an execution result through IO _ getevnts to obtain the content of the appointed LBA address;

when mode =1, the write operation does not record the LBA random sequence, after the read operation acquires the random LBA address content, it is determined whether the magic and seed in the Verify _ header portion are the same as expected, if not, the comparison of the next LBA is skipped, if yes, the Verify _ header and the data are continuously compared,

when mode =2, storing an LBA random sequence in a local file, opening the locally stored LBA random sequence file, judging whether an LBA address is written in, if not, jumping out to continue the comparison of the next LBA, and if so, continuing to compare the Verify _ header and the data;

when mode =3, judging whether to perform reading verification on the randomly generated lba address by comparing bitmap stored in the memory;

the method comprises the steps that a Verify _ header is a check header part in an IO message structure body, magic is a magic number, seed is a time seed, the magic number and the time seed are part of data of the check header part of the IO message structure body, data is a data part in the IO message structure body, and the Verify _ header and the data form the IO message structure body;

s23), judging whether the number of times of submission is reached, if so, finishing the circulation, and if not, continuing to execute the step S22;

s24), judging whether the number of write-read rounds is reached, if so, finishing the circulation, otherwise, redefining the submission times, and executing the step S22.

Further, when rw =0, a random mode is selected to generate the LBA sequence, and in the random mode, the random sequence of the LBA is generated by the time seed; and when rw =1, selecting a sequential mode to generate the LBA sequence, and sequentially generating the LBA sequence starting from the LBA equal to 0 in the sequential mode.

The invention has the beneficial effects that:

1. the requirement on the test environment is low. The implementation method has low requirement on the memory of the tester, and the current ordinary pc machine can meet the test requirement. The mode0 approach has essentially no memory requirements. In the mode1, the LBA random sequence is stored locally, memory occupancy occurs in reading of a local file, one bit represents whether one LBA is written, and compared with fio, for a 4k LBA, fio needs to store header content of 40 bytes, and the memory occupancy ratio of the two is 40 × 8/1=320. mode3, same as mode2.

2. Full disk data verification is supported. The capacity of the current popular Nvme SSD is in unit of T, and the method can effectively support the whole disk data verification.

3. The design can be customized for the data check pattern. The data content of each LBA is divided into a verify _ header and a data part. The verify _ header may include a magic number, a time seed, an LBA position, a write time, and the like, the data may be defined as a byte position of the LBA, and the like, and effective verification of the data is achieved through a customized design of a data verification pattern.

Drawings

FIG. 1 is a flow chart for writing data;

FIG. 2 is a flow chart of data verification.

Detailed Description

The invention is further described with reference to the following figures and specific embodiments.

Example 1

The embodiment discloses a method for realizing data verification test of a large-capacity SSD on a small-memory machine, wherein the large-capacity SSD refers to an SSD with the capacity larger than or equal to 8T, and the small-memory machine refers to a machine with the memory smaller than or equal to 1G.

In the present embodiment, in a scenario of performing a data verification test on a large-capacity SSD device on a machine with a smaller memory, an asynchronous non-blocking commit io is implemented by libario library functions io _ submit (), io _ getevnts (), and the like. And dynamically applying n memories to provide storage addresses for the io message body. In the applied n memories, available memories are dynamically searched, the memories are restored and cleaned, functions are given by matching liboio library functions, efficient management of the memories is achieved, and rapid data writing is completed. And performing multi-round data writing, reading and verification according to the number of writing and reading rounds, the io submission times of each round and the like analyzed in the command parameters, so as to meet the requirements of the test scene and complete the data verification test.

Specifically, during data writing, the storage mode of the LBA sequence is determined according to a mode parameter in the command parameters, when the mode =1, the LBA sequence is not recorded in the low-configuration test machine, when the mode =2, the LBA sequence is stored in the local file, and when the mode =3, the LBA sequence is stored in the memory by bitmap.

When data is verified, an LBA sequence is generated randomly or sequentially according to the rw parameter, different verification modes are adopted according to the mode of storing the LBA sequence when data is written, when mode =1, the LBA sequence is not recorded in the writing operation, the random LBA address is directly read, and whether verification is performed or not is judged according to the marking information in the read content; when mode =2, the LBA sequence is stored in the local file, the locally stored LBA sequence file is opened, whether the LBA address was written before is determined, and then read check is performed, and when mode =3, whether read check is performed on the randomly generated LBA address is determined by comparing the bitmap stored in the memory.

In this embodiment, when the model =1, it is determined whether to perform verification by reading the time seed in the content, and if it is known that the LBA is written with data according to the time seed, the data verification is performed, otherwise, the data verification is not performed.

The method comprises two parts of data writing and data verification, and as shown in fig. 1, the specific process of the data writing is as follows:

1. and acquiring size, verify _ size and bs in the command parameters. The size/verify _ size = the number of write and read rounds, verify _ size/bs = the number of IO submissions;

wherein size represents the total amount to be written, verify _ size represents the amount of checks, checks are performed every time this capacity size is written, and bs represents the designated IO size.

2. And taking the number of the writing and reading rounds as an outer loop and the number of submitting times as an inner loop until the loop is completed to reach the writing amount.

3. And dynamically applying n memories for preparing the io message body.

4. In each internal cycle, an LBA random sequence is generated through the time seed, and the storage mode of the LBA random sequence is determined according to the mode parameter in the command parameters.

mode =1, no recording when a low configuration tester is encountered;

mode =2, will write the local file storage LBA random sequence;

mode =3, and the LBA random sequence is stored in the memory by bitmap;

5. submitting the IO structure through the IO _ submit function, repeatedly submitting the IO structure which fails to be submitted due to insufficient available resources in exception processing of an execution result of the IO _ submit function, timing, if the continuous submission failure exceeds 30s, quitting the exception, and if the submission is successful within 30s, continuing the process.

6. And submitting 1 IO structure each time, and acquiring an IO execution result through IO _ getevents. And if the execution result is obtained, cleaning the occupied memory and counting. If not, the execution continuation io _ submit is returned.

7. And finishing the cycle to finish the writing requirement.

Therefore, the write flow for checking and testing the data of the large-capacity SSD equipment on the machine with smaller memory is realized.

As shown in fig. 2, the data checking process includes:

1. and acquiring size, verify _ size and bs in the command parameters. By size/verify _ size = number of write and read rounds, verify _ size/bs = number of IO submissions.

2. And taking the number of writing and reading rounds as an outer loop and the number of submission times as an inner loop until the loop is completed to reach the reading and checking quantity.

3. According to the rw parameter, randomly or sequentially generating an LBA sequence, and according to a mode parameter during data writing, selecting a verification mode after the LBA is read; the reading operation submits an IO reading message structure through a libaio library function IO _ submit, and obtains an execution result through IO _ getevnts to obtain the content of the appointed LBA address;

when mode =3, judging whether to perform read check on the randomly generated lba address by comparing the bitmap stored in the memory;

4. and finishing the cycle, and finishing the reading and the verification.

Therefore, the reading and verification process for data verification and test of the large-capacity SSD equipment on the machine with smaller memory is realized.

In this embodiment, when rw =0, a random mode is selected to generate the LBA sequence, and in the random mode, the random sequence of the LBA is generated by the time seed; and when rw =1, selecting a sequential mode to generate the LBA sequence, and sequentially generating the LBA sequence starting from the LBA equal to 0 in the sequential mode.

The invention effectively meets the requirement of the SSD device data consistency test with the continuously increased content in the industry, has low requirement on a test machine, improves the efficiency and quality of the test and effectively reduces the test cost.

The foregoing description is only for the basic principle and the preferred embodiments of the present invention, and modifications and substitutions by those skilled in the art are included in the scope of the present invention.

Claims

1. The method for realizing the data verification test of the large-capacity SSD on the small-memory machine is characterized in that the large-capacity SSD refers to the SSD with the capacity larger than or equal to 8T, and the small-memory machine refers to the machine with the memory smaller than or equal to 1G, and the method comprises the following steps: the method comprises two parts of random data writing and data verification, wherein during random data writing, a storage mode of the LBA sequence is determined according to a mode parameter in a command parameter, when the mode =1, the LBA sequence is not recorded, when the mode =2, the LBA sequence is stored in a local file, and when the mode =3, the LBA sequence is stored in an internal memory through bitmap;

when data is verified, an LBA sequence is generated, different verification modes are adopted according to the mode of storing the LBA sequence when data is written, when mode =1, the LBA sequence is not recorded in the writing operation, the random LBA address is directly read, and whether the verification is carried out or not is judged by reading the mark information in the content; when mode =2, the LBA sequence is stored in the local file, the locally stored LBA sequence file is opened, whether the LBA address was written before is determined, and then read check is performed, and when mode =3, whether read check is performed on the randomly generated LBA address is determined by comparing the bitmap stored in the memory.

2. The method for implementing data verification test of large-capacity SSD on small-memory machine as claimed in claim 1, wherein: when writing data, applying for n memories, dynamically searching for an available memory and restoring and cleaning the memory in the applied n memories, and realizing memory management and data writing by matching with the function of a libaio library function.

3. The method for implementing data verification test of large-capacity SSD on small-memory machine as claimed in claim 1, wherein: when data is written or data is verified, the total amount to be written, the verification amount and the specified IO packet size in the command parameters are obtained, the number of write-read rounds is specified to be equal to the total amount to be written divided by the verification amount, the number of submission times is equal to the verification amount divided by the specified IO packet size, the number of write-read rounds is used as the number of outer cycles, the number of submission times is used as the number of inner cycles, and the write-in amount is reached until the cycles are completed.

4. The method for implementing data verification test of large-capacity SSD on small-memory machine as claimed in claim 1, wherein: the specific process of writing data is as follows:

s12), dynamically applying for n memories;

s13) searching an available memory in the applied memory, selecting a random or sequential LBA sequence according to the rw parameter, and preparing an IO write message body; determining a storage mode of the LBA random sequence according to a mode parameter in the command parameter, wherein when the mode =1, the LBA random sequence is not recorded, when the mode =2, the LBA random sequence is stored in a local file, and when the mode =3, the LBA random sequence is stored in an internal memory through bitmap;

s14), submitting the IO structure through the IO _ submit function, repeatedly submitting failed IOs due to insufficient available resources in the exception processing of the execution result of the IO _ submit function, timing, and if the continuous submitting failure exceeds 30S, quitting the exception, and if the submitting is successful within 30S, continuing the process;

s15) submitting 1 IO structural body each time, obtaining an IO execution result through IO _ getevnts, if the IO execution result is obtained, cleaning the occupied memory, counting, and if the IO execution result is not obtained, continuing to execute the step S13;

s17), judging whether the number of the write-read rounds is reached, if so, ending the circulation, and if not, continuing to execute the step S12.

5. The method for implementing data verification test of large capacity SSD on small memory machine as claimed in claim 1, wherein: the specific process of data verification is as follows:

when mode =1, the write operation does not record the LBA random sequence, after the read operation obtains the random LBA address content, it is determined whether the magic and seed in the Verify _ header portion are the same as expected, if not, the comparison of the next LBA is skipped, if so, the Verify _ header and the data are continuously compared,

s24), judging whether the number of the write-read rounds is reached, if so, finishing the circulation, otherwise, redefining the number of submission times, and executing the step S22.

6. The method for implementing data verification test of large capacity SSD on small memory machine according to claim 4 or 5, characterized in that: when rw =0, selecting a random mode to generate an LBA sequence, and generating the LBA random sequence through time seeds in the random mode; and when rw =1, selecting a sequential mode to generate the LBA sequence, and sequentially generating the LBA sequence starting from the LBA equal to 0 in the sequential mode.