WO2018018827A1

WO2018018827A1 - Data storage control method and data server

Info

Publication number: WO2018018827A1
Application number: PCT/CN2016/108679
Authority: WO
Inventors: 张建; 李发明
Original assignee: 深圳市中博科创信息技术有限公司
Priority date: 2016-07-25
Filing date: 2016-12-06
Publication date: 2018-02-01
Also published as: CN106201766A; CN106201766B

Abstract

A data storage control method and a data server. The method comprises: when a data server receives first data, storing the first data in an FPGA chip (S110); when the receiving of the first data is completed, scheduling an erasure correcting code stored in the FPGA chip to calculate check data for the first data stored in the FPGA chip, and calculating a storage ratio of the first data stored in the FPGA chip to the check data (S120); and splitting, according to the storage ratio, the first data stored in the FPGA chip and the check data to obtain a plurality of pieces of second data, and respectively storing the second data in various storage units of a memory (S130). The method and data server reduce the occupation of the operation space in the memory by means of the erasure correcting code, improve the data storage capability of the memory, and thus improve the calculation efficiency of the erasure correcting code.

Description

Data storage control method and data server

Technical field

The present invention relates to the field of data storage technologies, and in particular, to a data storage control method and a data server.

Background technique

Redundant Arrays Independent from Redundant Arrays Independent Disks, RAID) to the distributed storage system, the erasure code is widely stored in the storage system, used for backup and storage management of data written and read in the storage system to ensure normal reading and writing and data security. However, as the data shows an explosive growth trend, the erasure code has more and more computational data, so occupying more storage space of the storage system not only reduces the data storage capacity of the storage system, but also reduces the erasure code. The efficiency of the calculation.

Summary of the invention

The main object of the present invention is to provide a data storage control method and a data server, which aim to solve the problem of low data processing capability of the storage system.

To achieve the above objective, the present invention provides a data storage control method, and the data storage control method includes:

When the data server receives the first data, storing the first data in an FPGA chip;

After the receiving of the first data is completed, calling the erasure code stored in the FPGA chip to calculate parity data of the first data stored in the FPGA chip, and calculating the first data stored in the FPGA chip and the Verify the storage ratio of the data;

Separating the first data stored in the FPGA chip and the verification data according to the storage ratio to obtain a plurality of second data, and storing the second data in each storage unit of the memory.

Preferably, the step of storing the second data into each storage unit of the memory comprises:

Each of the second data is sequentially called and stored in a corresponding storage unit;

Determining, by each time the second data is called, whether a storage unit corresponding to the second data to be stored has a storage failure;

If yes, sending an interrupt message to the storage unit corresponding to the second data, so that the storage unit corresponding to the second data stops storing the second data according to the interrupt message;

If not, the second data that is called is stored in the corresponding storage unit.

Preferably, the step of splitting the first data stored in the FPGA chip according to the storage ratio and the verification data to obtain the second data comprises:

Determining, according to the storage ratio, a first split component of the first data stored in the FPGA chip and a second split component of the check data;

Separating the first data stored in the FPGA chip according to the first split component, and splitting the check data according to the second split component, and storing the split FPGA chip The first data and the split check data are used as the second data.

Preferably, the data storage control method further includes:

When the data server receives the data read instruction, the erasure code in the FPGA chip is invoked according to the data read instruction;

Retrieving data to be read stored in each of the storage units corresponding to the data read instruction according to the erasure code.

Preferably, the step of recombining the data to be read stored in each of the storage units corresponding to the data read instruction according to the erasure code comprises:

And sequentially reading the data to be read stored in each of the storage units corresponding to the data read instruction;

After the data to be read is read, determining whether there is invalid data in the read data to be read;

And when the failure data exists in the data to be read, recovering the failure data by using the non-failure data in the data to be read, and performing the failure data according to the erasure code and the failure after the recovery. Data is reorganized;

When there is no invalid data in the data to be read, the data to be read is reorganized according to the erasure code.

Further, the present invention further provides a data storage control method, the data storage control method comprising the following steps:

When the data server receives the first data, storing the first data in a high speed memory of the FPGA chip;

After the first data storage is completed, the erasure code stored in the FPGA chip is called to calculate the verification data of the first data stored in the FPGA chip, and the first data stored in the FPGA chip and the first data are calculated. Verify the storage ratio of the data;

And dividing the first data stored in the FPGA chip and the verification data according to the storage ratio to obtain a plurality of second data, and storing the second data in each storage unit of the memory.

Preferably, the step of storing the second data in each storage unit of the memory comprises:

Each of the second data is sequentially called, and each of the second data is allocated a corresponding storage address according to a preset rule;

And storing each of the second data to a storage unit corresponding to the storage address;

Preferably, after the step of sending an interrupt message to the storage unit corresponding to the second data, the data storage control method further includes:

And detecting, by the preset time, the storage failure recovery of the storage unit corresponding to the second data, and storing the second data in the storage unit of the storage failure recovery;

Recovering the storage failure recovery of the storage unit corresponding to the second data within a preset time, reallocating the to-be-stored unit that matches the second data, and storing the second data in the Talked in the storage unit.

Preferably, the data storage control method further includes:

Retrieving data to be read stored in each of the storage units corresponding to the data read instruction according to the preset rule and the erasure code.

Preferably, the step of recombining the data to be read stored in each of the storage units corresponding to the data read instruction according to the preset rule and the erasure code comprises:

Determining whether there is invalid data in the read data to be read;

And when there is no invalid data in the data to be read, the data to be read is reorganized according to the erasure code and the preset rule.

Preferably, after the step of determining whether there is invalid data in the data to be read that is read, the data storage control method further includes:

Obtaining the number of invalid data in the data to be read;

When the number of the invalid data in the data to be read is less than or equal to a preset value, performing the use of the non-failed data in the data to be read to recover the invalid data, and performing the non-failure according to the erasure code The step of reorganizing the data and the recovered invalid data;

When the number of invalid data in the data to be read is greater than a preset value, the prompt information is output.

In addition, to achieve the above object, the present invention further provides a data server, where the data server includes:

a first storage module, configured to store the first data in an FPGA chip when the data server receives the first data;

a splitting module, configured to calculate, after the first data receiving is completed, an erasure code stored in the FPGA chip, to calculate parity data of the first data stored in the FPGA chip, and calculate the stored in the FPGA chip First data and a storage ratio of the verification data;

a second storage module, configured to split the first data stored in the FPGA chip and the verification data according to the storage ratio to obtain a plurality of second data, and store the second data into the memory In each storage unit.

Preferably, the second storage module comprises:

a calling unit, configured to sequentially call each of the second data, and store the data in a corresponding storage unit;

a first determining unit, configured to determine, when each of the second data is invoked, whether a storage unit corresponding to the second data to be stored has a storage failure;

a sending unit, configured to send an interrupt message to the storage unit corresponding to the second data, if the storage unit corresponding to the second data to be stored has a storage failure, so that the storage unit corresponding to the second data is The interrupt message stops storing the second data;

a storage unit, configured to store the called second data into a corresponding storage unit if a storage failure corresponding to the storage unit corresponding to the second data to be stored does not occur.

Preferably, the second storage module further includes:

a determining unit, configured to determine, according to the storage ratio, a first split component of the first data stored in the FPGA chip and a second split component of the check data;

a splitting unit, configured to split the first data stored in the FPGA chip according to the first split component, and split the verification data according to the second split component, and split the split data The first data stored in the FPGA chip and the split verification data are used as the second data.

Preferably, the data server further includes:

Calling module, when the data server receives the data read instruction, calling the erasure code in the FPGA chip according to the data read instruction;

And a reassembly module, configured to recombine data to be read stored in each of the storage units corresponding to the data read instruction according to the erasure code.

Preferably, the reorganization module comprises:

a reading unit, configured to sequentially read data to be read stored in each of the storage units corresponding to the data read instruction;

a second determining unit, configured to determine, after the reading of the data to be read is completed, whether there is invalid data in the read data to be read;

a first reorganization unit, configured to recover the failure data by using the non-failure data in the data to be read when the failure data exists in the data to be read, and to the non-failure data according to the erasure code And recovering the failed data after the recovery;

And a second recombining unit, configured to recombine the data to be read according to the erasure code when there is no invalid data in the data to be read.

When the first data is received by the data server, the first data is stored in the FPGA chip, and after the first data is received, the erasure code stored in the FPGA chip is called to calculate the first data stored in the FPGA chip. Verifying the data, calculating the storage ratio of the first data and the verification data stored in the FPGA chip, splitting the first data stored in the FPGA chip according to the storage ratio, and verifying the data to obtain a plurality of second data, and the second The data is stored separately in individual memory locations of the memory. The erasure code is stored in the FPGA chip. When the data server receives the first data, the first data is temporarily stored in the FPGA chip, and the erasure code stored in the FPGA chip is called to calculate the verification corresponding to the first data. Data and a storage ratio of the first data and the verification data, after the first data and the verification data are split according to the storage ratio to obtain a plurality of second data, the second data is separately stored in the memory In each storage unit, the resolution of the data by the erasure code, the determination of the check data, and the calculation of the storage ratio of the data and the check data are all performed in the FPGA chip, and the memory is only responsible for allocating the storage unit to store the split. The divided data and the verification data can be used, thereby reducing the occupation of the memory running space of the erasure code, improving the data storage capability of the memory, and further improving the calculation efficiency of the erasure code.

DRAWINGS

1 is a schematic flow chart of a first embodiment of a data storage control method according to the present invention;

2 is a schematic flowchart of a refinement step of storing the second data in each storage unit of the memory in the data storage control method of the present invention;

3 is a schematic flowchart of a refinement step of splitting the first data stored in the FPGA chip and the verification data to obtain a plurality of second data according to the storage ratio in the data storage control method of the present invention;

4 is a schematic flow chart of a second embodiment of a data storage control method according to the present invention;

FIG. 5 is a schematic flowchart of a refinement step of step S220 in the second embodiment shown in FIG. 4 according to the present invention;

6 is a schematic diagram of functional modules of a first embodiment of a data server according to the present invention;

7 is a schematic diagram of a refinement function module of a second storage module in a second embodiment of a data server according to the present invention;

FIG. 8 is a schematic diagram of another refinement function module of the second storage module in the third embodiment of the data server of the present invention; FIG.

9 is a schematic diagram of functional modules of a fourth embodiment of a data server according to the present invention;

FIG. 10 is a schematic diagram of a refinement function module of a reassembly module in a fifth embodiment of a data server according to the present invention.

The implementation, functional features, and advantages of the present invention will be further described in conjunction with the embodiments.

detailed description

It is understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.

Based on the above problems, the present invention provides a data storage control method.

Referring to FIG. 1, FIG. 1 is a schematic flowchart diagram of a first embodiment of a data storage control method according to the present invention.

In this embodiment, the data storage control method includes:

Step S110, when the data server receives the first data, storing the first data in the FPGA chip;

In this embodiment, the erasure code is stored in the FPGA chip, and the FPGA chip is applied to the data storage server, and is independent of the CPU and the memory in the data server. When the data server receives the first data written externally, the data server temporarily stores the received first data in the high speed memory of the FPGA chip. The data server includes a PC end, a smart phone, a smart TV, and a tablet computer; the high speed memory includes a solid state hard disk (Solid) State Disk, SSD).

Step S120, after the first data receiving is completed, calling the erasure code stored in the FPGA chip to calculate the verification data of the first data stored in the FPGA chip, and calculating the first data stored in the FPGA chip and the storage of the verification data. proportion;

After determining that the first data has been received and completely stored in the high speed memory of the FPGA chip, the data server calls the erasure code stored in the FPGA chip, and the temporary storage is performed at the high speed according to the algorithm of the erasure code. The verification data of the first data in the memory, and calculating the storage ratio of the first data and the verification data in the high speed memory. For example, after the first data of a total of 3 bytes is completely stored in the high speed memory of the FPGA chip, the PC end calls the erasure code stored in the FPGA chip to calculate the check data of the first data as 1 byte, and then the calculation is performed. The storage ratio of the first data and the check data in the high speed memory is 3:1. The check data is obtained by performing check coding on the first data to obtain a character string. When part of the data in the first data is lost, the erasure code passes the check data and is not lost in the first data. The data recovers the lost data in the first data.

Step S130, splitting the first data and the verification data stored in the FPGA chip according to the storage ratio to obtain a plurality of second data, and storing the second data in each storage unit of the memory.

When determining the storage ratio of the first data and the verification data, splitting the first data and the verification data according to the storage ratio to obtain a plurality of second data, the data server is allocated from the local memory and the second data A unit to be stored with the same number of data, and the storage space size of each unit to be stored matches the byte size of the second data, and the second data is separately stored in each unit to be stored in the memory. Each of the to-be-stored units can store only one second data. For example, after the storage ratio of the first data and the check data is 3:1, the first data is split into three data segments, each data segment is 1 byte, combined with one byte check. The data is obtained by 4 bytes of second data, and the PC side allocates 4 storage units from the memory of the local end, and the storage space of the 4 storage units is one byte, and the 4 bytes are the second. The data is stored in 4 storage units.

It should be noted that when the check data is split into M segments, the first data is split into K segments, and the total memory Y occupied by the M segments is equal to the total occupied by the K segments. One of the K points of memory X, that is, Y=X*1/K.

In this embodiment, when the first data is received by the data server, the first data is stored in the FPGA chip, and after the first data is received, the erasure code stored in the FPGA chip is called to calculate the first data stored in the FPGA chip. Checking the data, and calculating the storage ratio of the first data and the verification data stored in the FPGA chip, splitting the first data stored in the FPGA chip according to the storage ratio, and verifying the data to obtain a plurality of second data, and The two data are separately stored in respective storage units of the memory. The erasure code is stored in the FPGA chip. When the data server receives the first data, the first data is temporarily stored in the FPGA chip, and the erasure code stored in the FPGA chip is called to calculate the verification corresponding to the first data. Data and a storage ratio of the first data and the verification data, after the first data and the verification data are split according to the storage ratio to obtain a plurality of second data, the second data is separately stored in the memory In each storage unit, the resolution of the data by the erasure code, the determination of the check data, and the calculation of the storage ratio of the data and the check data are all performed in the FPGA chip, and the memory is only responsible for allocating the storage unit to store the split. The divided data and the verification data can be used, thereby reducing the occupation of the memory running space of the erasure code, improving the data storage capability of the memory, and further improving the calculation efficiency of the erasure code.

Further, please refer to FIG. 2, which is a schematic flowchart of the refinement step of step S130 in the first embodiment of the present invention. The storing the second data in each storage unit of the memory includes:

Step S131, sequentially calling each second data and storing it in a corresponding storage unit;

In this embodiment, after the data server allocates the to-be-stored unit that is consistent with the second data amount and the storage space size matches the byte size of the second data, the erasure code is given to the data server. The to-be-storage unit sends a preparation message, where each preparation message includes a target second data that needs to be stored in each unit to be stored. After the preparation unit receives the preparation message, a preparation log is generated according to the preparation message. After the preparation log of the storage unit is sent, the response message is sent to the FPGA chip. When the FPGA receives the response message, the FPGA sends the storage error message to the storage unit to send a storage message, indicating that the storage unit is to be written into the preparation log. The second data of a target. The preparation log is saved in the unit to be stored, and is used to temporarily save a target second data that needs to be stored by the to-be-stored unit.

It should be noted that, before storing the second data sequentially into the corresponding to-be-stored unit, each of the second data is allocated a storage address, and the storage address is allocated to each of the second data in a manner that the number of addresses is sequentially increased. . E.g, Splitting the first data into the second data a, the second data b, the second data c, and the second data d, in which the second data a, the second data b, the second data c, and the second data d are sequentially Before being stored in the corresponding to-be-stored unit, the second data a, the second data b, the second data c, and the second data d are allocated storage addresses in such a manner that the number of addresses is sequentially increased, respectively, 0001, 0010, and 01001. 0100.

Step S132, determining, by each time the second data is called, determining whether a storage unit corresponding to the second data to be stored has a storage failure;

When storing the second data, determining whether the unit to be stored corresponding to the second data to be stored is a feedback response message, and if receiving the response information fed back by the unit to be stored, determining that the storage unit is not in storage failure If the response information with the feedback of the storage unit is not received, it is determined that the storage unit has a storage failure.

Step S133, if yes, sending an interrupt message to the storage unit corresponding to the second data, so that the storage unit corresponding to the second data stops storing the second data according to the interrupt message;

When it is determined that the storage unit has a storage failure, the erasure code is invoked to send an interrupt message to the to-be-stored unit, and the to-be-stored unit deletes the preparation log according to the interruption message, and interrupts storage of the target second data, the target The second data is waiting for the storage, and if the storage failure recovery of the storage unit is detected within a preset time, the target second data is stored in the to-be-stored unit, and if the storage unit is not detected within a preset time The storage failure recovery, the data server re-allocates the to-be-stored unit that matches the target second data from the local storage, and stores the target second data into the to-be-stored unit.

Step S134, if no, storing the called second data into the corresponding storage unit.

The second data to be called is stored in the corresponding storage unit when it is determined that the storage unit does not have a storage failure.

In this embodiment, by sequentially calling each second data and storing it in the corresponding storage unit, each time the second data is called, it is determined whether the storage unit corresponding to the second data to be stored has a storage failure, and if so, The storage unit corresponding to the second data sends an interrupt message, so that the storage unit corresponding to the second data stops storing the second data according to the interrupt message, and if not, stores the called second data in the corresponding storage unit. Since each storage unit of the memory may have a storage failure at any time during the process of storing the second data, detecting each storage unit of the memory by time, avoiding sending storage disorder during the process of storing the second data, resulting in data Lost to ensure data security.

Further, please refer to FIG. 3, which is a schematic flowchart of a refinement step of step S130 according to the first embodiment of the present invention, where the first data stored in the FPGA chip and the checksum are split according to the storage ratio. The data gets multiple second data including:

Step S135, determining a first split component of the first data stored in the FPGA chip and a second split component of the check data according to the storage ratio;

In this embodiment, after determining that the storage ratio of the first data and the check data is M:K, determining that the first split component of the first data is M, and the second split component of the check data Is K.

Step S136, splitting the first data stored in the FPGA chip according to the first split component, and splitting the check data according to the second split component, and splitting the first data stored in the split FPGA chip and splitting The post-check data is used as the second data.

Splitting the first data into M segments according to the first split component M, and splitting the check data into K segments according to the second split component K, thereby composing M+K second data .

In this embodiment, the first split component of the first data stored in the FPGA chip and the second split component of the check data are determined according to the storage ratio, and the first data stored in the FPGA chip is split according to the first split component. And splitting the verification data according to the second split component, and using the first data stored in the split FPGA chip and the split check data as the second data. Since the erasure code splits the entire segment of data into a small segment of data according to a certain storage ratio, it not only facilitates the storage of the split data, but also avoids the problem that the data in the memory cannot be recovered after the data is invalid.

Further, based on the foregoing first embodiment, referring to FIG. 4, a schematic flowchart of a second embodiment of the data storage control method of the present invention is provided. In the second embodiment, the data storage control method includes:

Step S210, when the data server receives the data read instruction, invoking the erasure code in the FPGA chip according to the data read instruction;

Step S220, reorganizing the data to be read stored in each storage unit corresponding to the data read instruction according to the erasure code.

In this embodiment, when the data server receives the data read instruction, for example, when the data server receives the data read instruction sent by the client, the erasure code stored in the FPGA chip is invoked, according to the erasure code. The data redundancy function restores the data to be read into the entire data from the reorganization direction of the storage address from small to large. The data redundancy function includes splitting the data into a plurality of segments according to a preset rule, and is capable of reconstructing the previous data according to the preset rule.

In this embodiment, when the data server receives the data read instruction, the erasure code in the FPGA chip is called according to the data read instruction, and the data to be read stored in each storage unit corresponding to the data read instruction is reorganized according to the erasure code. The data to be read is reorganized by the erasure code, so that the data to be read can be read normally, which improves the fault tolerance of the memory.

Further, please refer to FIG. 5, which is a schematic flowchart of the refinement step of step S220 in the second embodiment of the present invention. The refinement step of step S220 includes:

Step S221, sequentially reading data to be read stored in each storage unit corresponding to the data read instruction;

Step S222, after the data to be read is read, determining whether there is invalid data in the read data to be read;

In this embodiment, after the erasure code in the FPGA chip is called according to the data read instruction, the erasure code sends a data call instruction to the data to be read stored in each storage unit corresponding to the data read instruction, Each storage unit corresponding to the data read instruction sends the locally stored data to be read to the FPGA chip according to the call instruction, and is temporarily stored in the high speed memory in the FPGA chip, and the data to be read is read. After completion, it is judged whether there is invalid data in the read data to be read.

Step S223, when there is invalid data in the data to be read, recovering the invalid data by using the non-failed data in the data to be read, and reconstructing the non-failed data and the restored invalid data according to the erasure code;

When there is invalid data in the data to be read, and the number of the invalid data is less than a preset value, the erasure code recovers the invalid data through the non-failed data in the data to be read based on the data redundancy function, after restoring the invalid data, according to The data redundancy function in the erasure code restores the non-failed data and the recovered invalid data to the entire segment of data from the reorganization direction of the storage address from small to large. The preset value is an integer smaller than the segment of the check data in the data to be read. If the number of the check data in the data to be read is K, the preset value is an integer smaller than K.

It should be noted that when the number of the failed data is greater than or equal to the preset value, the erasure code cannot recover the invalid data through the non-failed data in the data to be read based on the data redundancy function, so the data to be read cannot be read normally. take.

Step S224: When there is no invalid data in the data to be read, the data to be read is reorganized according to the erasure code.

When there is no invalid data in the data to be read, the data to be read is restored to the entire data according to the data redundancy function in the erasure code from the reorganization direction of the storage address from small to large.

The present embodiment sequentially reads the data to be read stored in each storage unit corresponding to the data read command, and after the data to be read is read, determines whether there is invalid data in the read data to be read, and is to be read. When there is invalid data in the data, the invalid data is recovered by using the unfailed data in the data to be read, and the non-failed data and the restored invalid data are reorganized according to the erasure code, and when there is no invalid data in the data to be read According to the erasure code, the data to be read is reorganized, so that the invalid data can still be read normally after recovery, thereby ensuring data security.

The invention further provides a data server.

Referring to FIG. 6, FIG. 6 is a schematic diagram of functional modules of a first embodiment of a data server according to the present invention.

In this embodiment, the data server includes: a first storage module 110, a split module 120, and a second storage module 130.

The first storage module 110 is configured to store the first data in the FPGA chip when the data server receives the first data;

In this embodiment, the erasure code is stored in the FPGA chip, and the FPGA chip is applied to the data storage server, and is independent of the CPU and the memory in the data server. When the first storage module 110 in the data server receives the first data written externally, the data server temporarily stores the received first data in the high speed memory of the FPGA chip. The data server includes a PC end, a smart phone, a smart TV, and a tablet computer; the high speed memory includes a solid state hard disk (Solid) State Disk, SSD).

The splitting module 120 is configured to, after the first data receiving is completed, invoke the erasure code stored in the FPGA chip to calculate the verification data of the first data stored in the FPGA chip, and calculate the first data stored in the FPGA chip. And the storage ratio of the verification data;

After the first storage module 110 determines that the first data has been received and is completely stored in the high speed memory of the FPGA chip, the splitting module 120 in the data server calls the erasure code stored in the FPGA chip, according to the The erasure code algorithm calculates the parity data of the first data temporarily stored in the high speed memory, and calculates the storage ratio of the first data and the parity data in the high speed memory. For example, after the first data of a total of 3 bytes is completely stored in the high speed memory of the FPGA chip, the PC end calls the erasure code stored in the FPGA chip to calculate the check data of the first data as 1 byte, and then the calculation is performed. The storage ratio of the first data and the check data in the high speed memory is 3:1. The check data is obtained by performing check coding on the first data to obtain a character string. When part of the data in the first data is lost, the erasure code passes the check data and is not lost in the first data. The data recovers the lost data in the first data.

The second storage module 130 is configured to split the first data and the verification data stored in the FPGA chip according to the storage ratio to obtain a plurality of second data, and store the second data in each storage unit of the memory.

When the splitting module 120 determines the storage ratio of the first data and the check data, the second storage module 130 splits the first data and the check data according to the storage ratio to obtain a plurality of second data, and the data server A unit to be stored is allocated in the memory of the end, and the size of the storage space of each unit to be stored matches the byte size of the second data, and the second data is separately stored in the memory. In the storage unit. Each of the to-be-stored units can store only one second data. For example, after the storage ratio of the first data and the check data is 3:1, the first data is split into three data segments, each data segment is 1 byte, combined with one byte check. The data is obtained by 4 bytes of second data, and the PC side allocates 4 storage units from the memory of the local end, and the storage space of the 4 storage units is one byte, and the 4 bytes are the second. The data is stored in 4 storage units.

In this embodiment, when the first data is received by the first storage module 110 in the data server, the first data is stored in the FPGA chip. After the first data is received, the split module 120 calls the correction stored in the FPGA chip. Deleting the code to calculate the verification data of the first data stored in the FPGA chip, and calculating the storage ratio of the first data and the verification data stored in the FPGA chip, and the second storage module 130 splits the stored in the FPGA chip according to the storage ratio. A data and verification data obtain a plurality of second data, and the second data is separately stored in each storage unit of the memory. The erasure code is stored in the FPGA chip. When the data server receives the first data, the first data is temporarily stored in the FPGA chip, and the erasure code stored in the FPGA chip is called to calculate the verification corresponding to the first data. Data and a storage ratio of the first data and the verification data, after the first data and the verification data are split according to the storage ratio to obtain a plurality of second data, the second data is separately stored in the memory In each storage unit, the resolution of the data by the erasure code, the determination of the check data, and the calculation of the storage ratio of the data and the check data are all performed in the FPGA chip, and the memory is only responsible for allocating the storage unit to store the split. The divided data and the verification data can be used, thereby reducing the occupation of the memory running space of the erasure code, improving the data storage capability of the memory, and further improving the calculation efficiency of the erasure code.

Further, based on the first embodiment, a second embodiment of the data server of the present invention is proposed. In this embodiment, referring to FIG. 7, the second storage module 130 includes a calling unit 131, a first determining unit 132, and a sending unit. 133 and storage unit 134.

The calling unit 131 is configured to sequentially call each second data and store the data in a corresponding storage unit;

In this embodiment, after the data server allocates the to-be-stored unit that matches the second data amount from the local memory and the storage space size matches the byte size of the second data, the calling unit 131 invokes the The erasure code sends a preparation message to the to-be-stored unit, and each preparation message includes a target second data that needs to be stored in each to-be-stored unit. When the to-be-stored unit receives the preparation message, it prepares according to the preparation message. Logging, after the preparation log of the to-be-stored unit, sending a response message to the FPGA chip, when the FPGA receives the response message, invoking the erasure code to send a storage message to the to-be-stored unit, indicating that the to-be-stored unit writes Enter a target second data in the preparation log. The preparation log is saved in the unit to be stored, and is used to temporarily save a target second data that needs to be stored by the to-be-stored unit.

The first determining unit 132 is configured to determine, when each call to the second data, whether a storage unit corresponding to the second data to be stored has a storage failure;

When the calling unit 131 stores the second data, the first determining unit 132 determines whether the unit to be stored corresponding to the second data to be stored feeds back a response message, and if receiving the response information fed back by the unit to be stored, determining The storage unit does not have a storage failure. If the response information with the storage unit feedback is not received, it is determined that the storage unit has a storage failure.

The sending unit 133 is configured to send an interrupt message to the storage unit corresponding to the second data if the storage unit corresponding to the second data to be stored has a storage failure, so that the storage unit corresponding to the second data stops storing according to the interrupt message. Second data;

When the first determining unit 132 determines that the storage unit has a storage failure, the sending unit 133 calls the erasure code to send an interrupt message to the to-be-stored unit, and the to-be-stored unit deletes the preparation log according to the interrupt message, and interrupts the target. The second data is stored, and the target second data is waiting for storage. If the storage failure recovery of the storage unit is detected within a preset time, the target second data is stored in the to-be-stored unit if the preset time is If the storage failure recovery of the storage unit is not detected, the data server re-allocates the to-be-stored unit that matches the target second data from the local storage, and stores the target second data into the to-be-stored unit.

The storage unit 134 is configured to store the called second data into the corresponding storage unit if the storage unit corresponding to the second data to be stored does not have a storage failure.

When the first determining unit 132 determines that the storage unit does not have a storage failure, the storage unit 134 stores the called second data into the corresponding storage unit.

Further, based on the first embodiment, a third embodiment of the data server of the present invention is proposed. In this embodiment, referring to FIG. 8, the second storage module 130 includes a determining unit 135 and a splitting unit 136.

The determining unit 135 is configured to determine, according to the storage ratio, a first split component of the first data stored in the FPGA chip and a second split component of the check data;

In this embodiment, after the determining unit 135 determines that the storage ratio of the first data and the check data is M:K, determining that the first split component of the first data is M, and the second of the check data Split the component to K.

The splitting unit 136 is configured to split the first data stored in the FPGA chip according to the first split component, and split the check data according to the second split component, and store the split FPGA chip The first data and the split check data are used as the second data.

The splitting unit 136 splits the first data into M segments according to the first split component M, and splits the check data into K segments according to the second split component K, thereby forming M+K. Second data.

Further, based on the first embodiment, a fourth embodiment of the data server of the present invention is provided. In this embodiment, referring to FIG. 9, the data server further includes: a calling module 210 and a recombining module 220.

The calling module 210 is configured to invoke an erasure code in the FPGA chip according to the data read instruction when the data server receives the data read instruction;

The recombination module 220 is configured to reassemble data to be read stored in each storage unit corresponding to the data read instruction according to the erasure code.

In this embodiment, when the data server receives the data read instruction, for example, when the data server receives the data read instruction sent by the client, the calling module 210 calls the erasure code stored in the FPGA chip, and the recombination module 220: According to the data redundancy function in the erasure code, the data to be read is restored to the entire data segment from the reorganization direction of the storage address from small to large. The data redundancy function includes splitting the data into a plurality of segments according to a preset rule, and is capable of reconstructing the previous data according to the preset rule.

Further, based on the fourth embodiment, a fifth embodiment of the data server of the present invention is proposed. In this embodiment, referring to FIG. 10, the reassembly module 220 includes a reading unit 221, a second judging unit 222, and a first reorganization. Unit 223 and second recombination unit 224.

The reading unit 221 is configured to sequentially read data to be read stored in each storage unit corresponding to the data read instruction;

The second determining unit 222 is configured to determine, after the data to be read is read, whether there is invalid data in the read data to be read;

In this embodiment, after the erasure code in the FPGA chip is called according to the data read command, the read unit 221 sends the erasure code to the data to be read stored in each storage unit corresponding to the data read command. a data calling instruction, and each storage unit corresponding to the data reading instruction sends the locally stored data to be read to the FPGA chip according to the calling instruction, and temporarily stores the data to be read in the high speed memory in the FPGA chip, to be read After the data reading is completed, the second determining unit 222 determines whether there is invalid data in the read data to be read.

The first recombining unit 223 is configured to recover invalid data in the data to be read when there is invalid data in the data to be read, and perform non-failure data and the restored invalid data according to the erasure code. Reorganization

When the second determining unit 222 determines that there is invalid data in the data to be read, and the number of the invalid data is less than a preset value, the erasure code recovers the invalid data by using the non-failed data in the data to be read based on the data redundancy function. After restoring the invalid data, the un-failed data and the restored failed data are restored to the entire data according to the data redundancy function in the erasure code from the small to large recombination direction of the storage address. The preset value is an integer smaller than the segment of the check data in the data to be read. If the number of the check data in the data to be read is K, the preset value is an integer smaller than K.

The second recombining unit 224 is configured to reassemble the data to be read according to the erasure code when there is no invalid data in the data to be read.

When there is no invalid data in the data to be read by the second determining unit 222, the second recombining unit 224 restores the data to be read to the whole according to the data redundancy function in the erasure code from the reorganization direction of the storage address from small to large. Segment data.

The above are only the preferred embodiments of the present invention, and are not intended to limit the scope of the invention, and the equivalent structure or equivalent process transformations made by the description of the present invention and the drawings are directly or indirectly applied to other related technical fields. The same is included in the scope of patent protection of the present invention.

Claims

A data storage control method, characterized in that the data storage control method comprises the following steps:

When the data server receives the first data, storing the first data in an FPGA chip;

After the receiving of the first data is completed, calling the erasure code stored in the FPGA chip to calculate parity data of the first data stored in the FPGA chip, and calculating the first data stored in the FPGA chip and the Verify the storage ratio of the data;

And dividing the first data stored in the FPGA chip and the verification data according to the storage ratio to obtain a plurality of second data, and storing the second data in each storage unit of the memory.
The data storage control method according to claim 1, wherein the step of storing the second data in each storage unit of the memory comprises:

Each of the second data is sequentially called and stored in a corresponding storage unit;

Determining, by each time the second data is called, whether a storage unit corresponding to the second data to be stored has a storage failure;

If yes, sending an interrupt message to the storage unit corresponding to the second data, so that the storage unit corresponding to the second data stops storing the second data according to the interrupt message;

If not, the second data that is called is stored in the corresponding storage unit.
The data storage control method according to claim 1, wherein the step of splitting the first data stored in the FPGA chip and the verification data according to the storage ratio to obtain second data comprises:

Determining, according to the storage ratio, a first split component of the first data stored in the FPGA chip and a second split component of the check data;

Separating the first data stored in the FPGA chip according to the first split component, and splitting the check data according to the second split component, and storing the split FPGA chip The first data and the split check data are used as the second data.
The data storage control method according to claim 1, wherein the data storage control method further comprises:

When the data server receives the data read instruction, the erasure code in the FPGA chip is invoked according to the data read instruction;

Retrieving data to be read stored in each of the storage units corresponding to the data read instruction according to the erasure code.
The data storage control method according to claim 4, wherein the step of recombining the data to be read stored in each of the storage units corresponding to the data read instruction according to the erasure code comprises:

And sequentially reading the data to be read stored in each of the storage units corresponding to the data read instruction;

After the data to be read is read, determining whether there is invalid data in the read data to be read;

And when the failure data exists in the data to be read, recovering the failure data by using the non-failure data in the data to be read, and performing the failure data according to the erasure code and the failure after the recovery. Data is reorganized;

When there is no invalid data in the data to be read, the data to be read is reorganized according to the erasure code.
A data server, characterized in that the data server comprises:

a first storage module, configured to store the first data in an FPGA chip when the data server receives the first data;

a splitting module, configured to calculate, after the first data receiving is completed, an erasure code stored in the FPGA chip, to calculate parity data of the first data stored in the FPGA chip, and calculate the stored in the FPGA chip First data and a storage ratio of the verification data;

a second storage module, configured to split the first data stored in the FPGA chip and the verification data according to the storage ratio to obtain a plurality of second data, and store the second data in each of the memory In the storage unit.
The data server of claim 6, wherein the second storage module comprises:

a calling unit, configured to sequentially call each of the second data, and store the data in a corresponding storage unit;

a first determining unit, configured to determine, when each of the second data is invoked, whether a storage unit corresponding to the second data to be stored has a storage failure;

a sending unit, configured to send an interrupt message to the storage unit corresponding to the second data, if the storage unit corresponding to the second data to be stored has a storage failure, so that the storage unit corresponding to the second data is The interrupt message stops storing the second data;

a storage unit, configured to store the called second data into a corresponding storage unit if a storage failure corresponding to the storage unit corresponding to the second data to be stored does not occur.
The data server of claim 6, wherein the second storage module further comprises:

a determining unit, configured to determine, according to the storage ratio, a first split component of the first data stored in the FPGA chip and a second split component of the check data;

a splitting unit, configured to split the first data stored in the FPGA chip according to the first split component, and split the verification data according to the second split component, and split the split data The first data stored in the FPGA chip and the split verification data are used as the second data.
The data server of claim 6, wherein the data server further comprises:

Calling module, when the data server receives the data read instruction, calling the erasure code in the FPGA chip according to the data read instruction;

And a reassembly module, configured to recombine data to be read stored in each of the storage units corresponding to the data read instruction according to the erasure code.
The data server of claim 9, wherein the reassembly module comprises:

a reading unit, configured to sequentially read data to be read stored in each of the storage units corresponding to the data read instruction;

a second determining unit, configured to determine, after the reading of the data to be read is completed, whether there is invalid data in the read data to be read;

a first reorganization unit, configured to recover the failure data by using the non-failure data in the data to be read when the failure data exists in the data to be read, and to the non-failure data according to the erasure code And recovering the failed data after the recovery;

And a second recombining unit, configured to recombine the data to be read according to the erasure code when there is no invalid data in the data to be read.
A data storage control method, characterized in that the data storage control method comprises the following steps:

When the data server receives the first data, storing the first data in a high speed memory of the FPGA chip;

After the first data storage is completed, the erasure code stored in the FPGA chip is called to calculate the verification data of the first data stored in the FPGA chip, and the first data stored in the FPGA chip and the first data are calculated. Verify the storage ratio of the data;

And dividing the first data stored in the FPGA chip and the verification data according to the storage ratio to obtain a plurality of second data, and storing the second data in each storage unit of the memory.
The data storage control method according to claim 11, wherein the step of storing the second data in each storage unit of the memory comprises:

Each of the second data is sequentially called, and each of the second data is allocated a corresponding storage address according to a preset rule;

And storing each of the second data to a storage unit corresponding to the storage address;

Determining, by each time the second data is called, whether a storage unit corresponding to the second data to be stored has a storage failure;

If yes, sending an interrupt message to the storage unit corresponding to the second data, so that the storage unit corresponding to the second data stops storing the second data according to the interrupt message;

If not, the second data that is called is stored in the corresponding storage unit.
The data storage control method according to claim 12, wherein after the step of transmitting an interrupt message to the storage unit corresponding to the second data, the data storage control method further comprises:

And detecting, by the preset time, the storage failure recovery of the storage unit corresponding to the second data, and storing the second data in the storage unit of the storage failure recovery;

Recovering the storage failure recovery of the storage unit corresponding to the second data within a preset time, reallocating the to-be-stored unit that matches the second data, and storing the second data in the Talked in the storage unit.
The data storage control method according to claim 12, wherein the step of splitting the first data stored in the FPGA chip and the verification data according to the storage ratio to obtain second data comprises:

Determining, according to the storage ratio, a first split component of the first data stored in the FPGA chip and a second split component of the check data;

Separating the first data stored in the FPGA chip according to the first split component, and splitting the check data according to the second split component, and storing the split FPGA chip The first data and the split check data are used as the second data.
The data storage control method according to claim 11, wherein the data storage control method further comprises:

When the data server receives the data read instruction, the erasure code in the FPGA chip is invoked according to the data read instruction;

Retrieving data to be read stored in each of the storage units corresponding to the data read instruction according to the preset rule and the erasure code.
The data storage control method according to claim 15, wherein said retrieving said to be read in each of said storage units corresponding to said data read command according to said preset rule and said erasure code The steps to retrieve data include:

And sequentially reading the data to be read stored in each of the storage units corresponding to the data read instruction;

Determining whether there is invalid data in the read data to be read;

And when the failure data exists in the data to be read, recovering the failure data by using the non-failure data in the data to be read, and performing the failure data according to the erasure code and the failure after the recovery. Data is reorganized;

And when there is no invalid data in the data to be read, the data to be read is reorganized according to the erasure code and the preset rule.
The data storage control method according to claim 16, wherein after the step of determining whether there is invalid data in the read data to be read, the data storage control method further comprises:

Obtaining the number of invalid data in the data to be read;

When the number of the invalid data in the data to be read is less than or equal to a preset value, performing the use of the non-failed data in the data to be read to recover the invalid data, and performing the non-failure according to the erasure code The step of reorganizing the data and the recovered invalid data;

When the number of invalid data in the data to be read is greater than a preset value, the prompt information is output.