WO2021109275A1 - 一种基于fpga设备的网络定义存储方法、读取方法及系统 - Google Patents

一种基于fpga设备的网络定义存储方法、读取方法及系统 Download PDF

Info

Publication number
WO2021109275A1
WO2021109275A1 PCT/CN2019/127032 CN2019127032W WO2021109275A1 WO 2021109275 A1 WO2021109275 A1 WO 2021109275A1 CN 2019127032 W CN2019127032 W CN 2019127032W WO 2021109275 A1 WO2021109275 A1 WO 2021109275A1
Authority
WO
WIPO (PCT)
Prior art keywords
user
sub
fpga
data
storage register
Prior art date
Application number
PCT/CN2019/127032
Other languages
English (en)
French (fr)
Inventor
张继栋
卢华
朱伏生
Original Assignee
广东省新一代通信与网络创新研究院
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 广东省新一代通信与网络创新研究院 filed Critical 广东省新一代通信与网络创新研究院
Publication of WO2021109275A1 publication Critical patent/WO2021109275A1/zh

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00—Arrangements for program control, e.g. control units
    • G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/46—Multiprogramming arrangements
    • G06F9/50—Allocation of resources, e.g. of the central processing unit [CPU]
    • G06F9/5005—Allocation of resources, e.g. of the central processing unit [CPU] to service a request
    • G06F9/5027—Allocation of resources, e.g. of the central processing unit [CPU] to service a request the resource being a machine, e.g. CPUs, Servers, Terminals
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F21/00—Security arrangements for protecting computers, components thereof, programs or data against unauthorised activity
    • G06F21/60—Protecting data
    • G06F21/602—Providing cryptographic facilities or services
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F21/00—Security arrangements for protecting computers, components thereof, programs or data against unauthorised activity
    • G06F21/60—Protecting data
    • G06F21/62—Protecting access to data via a platform, e.g. using keys or access control rules
    • G06F21/6218—Protecting access to data via a platform, e.g. using keys or access control rules to a system of files or objects, e.g. local or distributed file system or database
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00—Arrangements for program control, e.g. control units
    • G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/46—Multiprogramming arrangements
    • G06F9/52—Program synchronisation; Mutual exclusion, e.g. by means of semaphores
    • G06F9/526—Mutual exclusion algorithms

Definitions

  • the present invention relates to the technical field of network definition storage, in particular to a network definition storage method based on FPGA equipment, a reading method and a corresponding network definition storage system.
  • Network-defined storage is an online storage technology, known for its high security and high reliability.
  • the user server After the user server splits the original data into a number of sub-data according to a certain length, it will be sent to the FPGA deployed on the server in order for encryption, scrambling and redundancy processing, and then the multiple sub-data blocks processed by the FPGA are uploaded to the network via the network.
  • Multiple network storage servers When the user needs to read the original data, the opposite direction is adopted. First, download the sub-data block from each network storage server to the user server, and then perform de-redundancy, descrambling, decryption, and finally reassembly through the FPGA deployed on the user server in order. The original data is returned to the user. Due to the use of redundancy technology, it can be ensured that when some network storage servers cannot be downloaded abnormally, the complete original data can be restored from the sub-data blocks downloaded from other normal network storage servers.
  • the existing network-defined storage method and system all the data of all users share the same key, which poses a great security risk; and regardless of encryption, scrambling, and redundancy processing, or de-redundancy and de-redundancy
  • the scrambling and decryption processing has strict requirements on the processing sequence of the sub-data, and the work efficiency is low.
  • the existing network-defined storage methods and systems do not support parallel processing of multiple data, let alone parallel operation of data by multiple users. Therefore, the overall operating efficiency of the system is very low, which is a great waste to the performance of the user's server CPU, memory and FPGA.
  • the present invention proposes a network-defined storage method and a reading method based on an FPGA device, and a network-defined storage system based on an FPGA device.
  • the technical scheme of the present invention adopts a multi-key, multi-process, multi-user, and multi-data working mechanism, sets an independent and unique key for each data of each user, and adopts advanced multi-process business logic to realize multi-user ,
  • the parallel operation of multiple data greatly improves the efficiency of the entire network-defined storage system.
  • the technical solution of the present invention adopts a method in which each user is assigned a user process, and each data storage/read operation of each user is assigned a user subprocess, and combined with each data of each user, a unique and unique secret is assigned.
  • the key and multiple FPGA key storage registers and multiple FPGA data storage registers efficiently realize the concurrent processing of multi-user and multi-data operation requests.
  • the present invention proposes a network definition storage method based on FPGA equipment, which includes the following steps:
  • Step S101 The user server creates a user main process for each user, and creates a user subprocess for each data to be stored for each user;
  • Step S102 Each of the user sub-processes applies for a unique key from the corresponding user main process, and the user main process splits each to-be-stored data into several sub-data;
  • Step S103 Each of the user sub-processes applies to the corresponding user main process for an idle FPGA key storage register, and writes the unique key of the user sub-process into the FPGA key storage register;
  • Step S104 Each of the user sub-processes applies to the corresponding user main process for an idle FPGA data storage register, and writes the corresponding sub-data, key storage location and processing logic information into the corresponding FPGA data storage register ;
  • Step S105 After the user sub-process completes the writing operation of the FPGA data storage register, it immediately reads the FPGA data storage register in a blocking mode; FPGA saves the key according to the sub-data read from the FPGA data storage register Position and processing logic information, take out the unique key and sub-data corresponding to the data to be stored, and perform logical processing on the sub-data according to the processing logic information, and output N sub-sub-data, where N is a positive integer; The N sub-sub-data are written into the FPGA data storage register;
  • Step S106 Upload the N sub-sub-data to N network storage servers respectively, and the user sub-process notifies the user main process to release the corresponding FPGA data storage register and FPGA key storage register;
  • Step S107 End the user sub-process and the user main process.
  • the processing logic includes encryption, scrambling and redundancy.
  • the N number is at least 3.
  • the scheduling of the user process and the user sub-process, as well as the application and release of the FPGA key storage register and the FPGA data storage register are all uniformly scheduled by the user server.
  • the present invention also proposes a method for reading network definitions based on FPGA equipment, which includes the following steps:
  • Step S201 The user server creates a user main process for each user, and creates a user subprocess for each data to be read for each user;
  • Step S202 Each of the user sub-processes reads from the user server the unique key corresponding to each data to be read and the storage location information of the N sub-sub-data corresponding to the data on the network storage server.
  • the user The sub-process downloads N sub-sub-data in the storage location according to the storage location information;
  • Step S203 Each of the user sub-processes applies to the corresponding user main process for an idle PFGA key storage register, and writes the unique key corresponding to the data to be read in step S202 into the FPGA key storage register ;
  • Step S204 Each of the user sub-processes applies to the corresponding user main process for an idle FPGA data storage register, and writes the N sub-sub-data, key storage location and processing logic information downloaded in step S202 into the FPGA data storage register;
  • Step S205 After the user subprocess completes the write operation of the FPGA data storage register, it immediately reads the FPGA data storage register in a blocking mode; the FPGA reads the FPGA data storage register according to the unique data to be read from the FPGA data storage register. The location of the key in the FPGA key storage register and processing logic information, take out the unique key and sub-sub-data corresponding to the data to be read, perform logical processing on the sub-sub-data according to the processing logic information, and output the sub-data, And write the sub-data into the FPGA data storage register;
  • Step S206 the user sub-process reads the sub-data in the FPGA data storage register of step S205 in a blocking manner, and returns the sub-data to the user, and the user sub-process notifies the user main process to release the corresponding FPGA data storage register And FPGA key storage register;
  • Step S207 End the user sub-process and the user main process.
  • the processing logic includes de-redundancy, de-scrambling and decryption.
  • the N number is at least 3.
  • the scheduling of the user process and the user sub-process, as well as the application and release of the FPGA key storage register and the FPGA data storage register are all uniformly scheduled by the user server.
  • the present invention also proposes a network definition storage system based on FPGA equipment, including: a user server and N network storage servers, the user server is configured with FPGA, CPU, and memory, the FPGA is configured with registers, and the registers include: FPGA data storage register and FPGA key storage register, the FPGA data storage register is used to read and write sub-data to be processed, the FPGA key storage register is used to read and write keys, and the user server is configured with a CPU and memory,
  • the system adopts a network definition storage method based on FPGA equipment.
  • the prior art only defines one key storage register area and one data storage register area. Each sub-data to be stored or sub-sub-data to be read must wait for the previous sub-data or sub-sub-data to be processed by the FPGA and be processed by the FPGA.
  • the data storage register of the FPGA can be written only after reading; and in the technical scheme of the present invention, multiple FPGA data storage registers are defined, and the main process of the user server performs centralized scheduling, which greatly improves the user process and the user process on the user server. The efficiency of FPGA exchange data;
  • the technical scheme of the present invention adopts a method in which each user is assigned a user process, and each data storage/read operation of each user is assigned a user sub-process, combined with each data of each user and unique and unique.
  • the key and multiple FPGA key storage registers and multiple FPGA data storage registers efficiently realize the concurrent processing of multi-user and multi-data operation requests.
  • Fig. 1 is a flow chart of the steps of a method for network definition storage based on FPGA equipment of the present invention.
  • Figure 2 is a flowchart of the steps of a method for reading network definitions based on FPGA devices of the present invention.
  • Fig. 3 is a schematic diagram of the basic steps of a network definition storage method based on FPGA equipment of the present invention.
  • Fig. 4 is a schematic diagram of the basic steps of a method for reading a network definition based on an FPGA device according to the present invention.
  • Fig. 5 is a schematic diagram of the principle of a method for network definition storage based on FPGA devices in specific implementations.
  • Fig. 6 is a schematic diagram of the principle of a method for reading a network definition based on an FPGA device in a specific embodiment.
  • a network definition storage method based on FPGA equipment includes the following steps:
  • Step S101 The user server creates a user main process for each user, and creates a user sub-process for each to-be-stored data of the user;
  • Step S102 Each of the user sub-processes applies for a unique key from the corresponding user main process, and the user main process splits each to-be-stored data into several sub-data;
  • Step S103 Each of the user sub-processes applies to the corresponding user main process for an idle FPGA key storage register, and writes the unique key of the user sub-process into the FPGA key storage register;
  • Step S104 Each of the user sub-processes applies to the corresponding user main process for an idle FPGA data storage register, and writes the corresponding sub-data, key storage location and processing logic information into the corresponding FPGA data storage register ;
  • Step S105 After the user sub-process completes the writing operation of the FPGA data storage register, it immediately reads the FPGA data storage register in a blocking mode; FPGA saves the key according to the sub-data read from the FPGA data storage register Position and processing logic information, take out the unique key and sub-data corresponding to the data to be stored, and perform logical processing on the sub-data according to the processing logic information, and output N sub-sub-data, where N is a positive integer; The N sub-sub-data are written into the FPGA data storage register;
  • Step S106 Upload the N sub-sub-data to N network storage servers respectively, and the user sub-process notifies the user main process to release the corresponding FPGA data storage register and FPGA key storage register;
  • Step S107 End the user sub-process and the user main process.
  • a method for reading network definitions based on FPGA devices includes the following steps:
  • Step S201 The user server creates a user main process for each user, and creates a user subprocess for each data to be read for each user;
  • Step S202 Each of the user sub-processes reads from the user server the unique key corresponding to each data to be read and the storage location information of the N sub-sub-data corresponding to the data on the network storage server.
  • the user The sub-process downloads N sub-sub-data in the storage location according to the storage location information;
  • Step S203 Each of the user sub-processes applies to the corresponding user main process for an idle PFGA key storage register, and writes the unique key corresponding to the data to be read in step S202 into the FPGA key storage register ;
  • Step S204 Each of the user sub-processes applies to the corresponding user main process for an idle FPGA data storage register, and writes the N sub-sub-data, key storage location and processing logic information downloaded in step S202 into the FPGA data storage register;
  • Step S205 After the user subprocess completes the write operation of the FPGA data storage register, it immediately reads the FPGA data storage register in a blocking mode; the FPGA reads the FPGA data storage register according to the unique data to be read from the FPGA data storage register. The location of the key in the FPGA key storage register and processing logic information, take out the unique key and sub-sub-data corresponding to the data to be read, perform logical processing on the sub-sub-data according to the processing logic information, and output the sub-data, And write the sub-data into the FPGA data storage register;
  • Step S206 the user sub-process reads the sub-data in the FPGA data storage register of step S205 in a blocking manner, and returns the sub-data to the user, and the user sub-process notifies the user main process to release the corresponding FPGA data storage register And FPGA key storage register;
  • Step S207 End the user sub-process and the user main process.
  • the numbers 1, 2, and 3 correspond to the steps S101, S102, and S101 in the basic storage step
  • the number 4 corresponds to the step S103 in the basic storage step
  • the number 5 corresponds to the step S103 in the basic storage step.
  • the number 6 corresponds to the step S105 in the basic storing step
  • the number 7 corresponds to the step S105 in the basic storing step
  • the number 8 corresponds to the step S106 in the basic storing step
  • the number 9 Corresponds to step S107 in the basic storage procedure described above.
  • the numbers 1, 2, and 3 correspond to the steps S201 and S202 in the basic reading steps
  • the number 4 corresponds to the step S203 in the basic reading steps
  • the number 5 corresponds to the basic reading steps.
  • the step S204 in the steps corresponds to the step S204 in the above basic reading step
  • the number 7 corresponds to the step S205 in the basic reading step above
  • the number 8 corresponds to the step S206 in the basic reading step above.
  • Step corresponding, the number 9 corresponds to the step S207 in the basic reading steps.
  • a network definition storage method based on FPGA equipment as shown in Figure 5.
  • Step 101 Two users U1 and U2 access the user server at the same time, and the main service process T of the user server creates two user processes U1T and U2T for the two users respectively. Refer to number 1 in Figure 5;
  • Step 102 Two users submit multiple file storage requests at the same time.
  • the files are U1F1, U1F2, U2F1, U2F2.
  • the storage request of U1 reaches the user process U1T of U1.
  • U1T is created by the subsequent operations of the two files U1F1 and U1F2.
  • U2's storage request reaches U2's user process U2T
  • U2T creates two user subprocesses U2F1T and U2F2T for the subsequent operations of the two files U2F1 and U2F2.
  • Step 103 The user sub-process U1F1T applies for the unique key U1F1K to the main business process T for the file U1F1 and saves it locally.
  • the file U1F1 is split into sub-data U1F1P1 and U1F1P2 according to the agreed size
  • the user sub-process U1F2T is the file U1F2
  • the main business process T applies for the unique key U1F2K
  • the file U1F2 is split into sub-data U1F2P1 and U1F2P2 according to the agreed size
  • the user sub-process U2F1T is the file U2F1 to apply for the unique key U2F1K from the main business process T, and the file U2F1 Split into sub-data U2F1P1 and U2F1P2 according to the agreed size
  • the user sub-process U2F2T applies for the unique key U2F2K to the main business process T for the file U2F2, and at the same time splits
  • Step 104 The user sub-process U1F1T applies to the main service process T for an idle FPGA key storage register KR1, and writes U1F1K to KR1; the user sub-process U1F2T applies to the main service process T for an idle FPGA key storage register KR2, And write U1F2K into KR2; user subprocess U2F1T applies to the main business process T for an idle FPGA key storage register KR3, and writes U2F1K to KR3; user subprocess U2F2T applies to the main business process T for an idle FPGA key Store register KR4, and write U2F2K into KR4. Refer to number 4 in Figure 5;
  • Step 105 The user sub-process U1F1T applies to the main business process T for free FPGA data storage registers DR1 and DR2, and writes the sub-data U1F1P1, the storage location KR1 of the key U1F1K and the processing logic into DR1, and the sub-data U1F1P2 Write DR2 together with the storage location KR1 and processing logic of the key U1F1K; the user subprocess U1F2T applies to the main business process T for free FPGA data storage registers DR3 and DR4, and saves the subdata U1F2P1 and the key U1F2K The location KR2 and processing logic are written into DR3 together.
  • the sub-data U1F2P2 and the storage location KR2 of the key U1F2K and the processing logic are written into DR4 together;
  • the user sub-process U2F1T applies to the main business process T for free FPGA data storage registers DR5 and DR6, and write the sub-data U2F1P1, the storage location KR3 of the key U2F1K and the processing logic into DR5, and write the sub-data U2F1P2, the storage location KR3 of the key U2F1K and the processing logic into DR6 together;
  • the process U2F2T applies to the main business process T for free FPGA data storage registers DR7 and DR8, and writes the sub-data U2F2P1, the storage location KR4 of the key U2F2K, and processing logic into DR7, the sub-data U2F2P2 and the key U2F2K.
  • the storage location KR4 and processing logic and other information are written into DR8 together.
  • Step 106 The FPGA scans regularly and finds that there is data to be processed in the data storage register DR1. It is read in a certain format, and the processing logic is read according to some of the fields as encryption, scrambling, and redundancy, and reads the corresponding data according to some of the fields.
  • the key is stored in the key storage register KR1, the key U1F1K and the sub-data U1F1P1 are taken out, and then encrypted, scrambled, and redundant processing is performed according to the read processing logic, and the three output blocks are processed, which is characterized in that the output sub-data
  • the number of sub-data is the same as the number of configured network storage servers.
  • the sub-data downloaded from other network storage servers can be restored using redundant algorithms.
  • the correct sub-data is preferably greater than or equal to 3 network storage servers; the sub-sub-data U1F1P1M1, U1F1P1M2 and U1F1P1M3 are written into DR1 in a certain format; the FPGA continues to scan and finds that there is data to be processed in the data storage register DR2, which is similar to the above process.
  • U1F1P2M1, U1F1P2M2, U1F1P2M3 are written to DR2; similarly, FPGA writes U1F2P1M1, U1F2P1M2, U1F2P1M3 to DR3, U1F2P2M1, U1F2P2M2, U1F2P2M1, U1F2P2M1, U1F2P2M2, U1F2U2F1P1, U1F2P2M2, U1F1P2, U2F1P5, U1F2P2M2, U1F1P1 Write DR6, write U2F2P1M1, U2F2P1M2, U2F2P1M3 to DR7, and write U2F2M1, U2F2P2M2, U2F2P2M3 to DR8. As shown in Figure 5, number 6;
  • Step 107 After the above FPGA writes the sub-sub-data U1F1P1M1, U1F1P1M2 and U1F1P1M3 into DR1, the user subprocess U1F1T that is in the blocked state of reading DR1 gets the read return immediately, so it reads U1F1P1M1, U1F1P1M2 and U1F1P1M3, and uploads them to three immediately.
  • the storage of file U1F1 is completed, and the key storage register KR1 of FPGA can also notify the main service process T to release; U1F2P1M1, U1F2P1M2, U1F2P1M3 are uploaded to S1, S2 and S3 respectively, and DR3 is released; U1F2P2M1, U1F2P2M2 U1F2P2M3 is uploaded to S1, S2, and S3, and DR4 is released.
  • the file U1F2 is stored, and the key storage register KR2 of FPGA can also notify the service main process T to release; U2F1P1M1, U2F1P1M2, U2F1P1M3 are uploaded to S1, S2, respectively And S3, and release DR5; U2F1P2M1, U2F1P2M2, U2F1P2M3 are uploaded to S1, S2, and S3 respectively, and DR6 is released.
  • the file U2F1 is stored, and the FPGA key storage register KR3 can also notify the main service process T to release; U2F2P1M1, U2F2P1M2, U2F2P1M3 are uploaded to S1, S2, and S3 respectively, and DR7 is released; U2F2P2M1, U2F2P2M2, U2F2P2M3 are uploaded to S1, S2, and S3 respectively, and DR8 is released.
  • the file U2F2 is stored and the key storage register of FPGA is KR4. It is also possible to notify the main process T of this service that it is released. As shown in Figure 5, number 7;
  • Step 108 U1F1 has been stored, end user subprocess U1F1T; U1F2 has been stored, end user subprocess U1F2T; U2F1 has been stored, end user subprocess U2F1T; U2F2 has been stored, end user subprocess U2F2T. As shown in Figure 5, number 8;
  • Step 109 The user U1 disconnects from the user server and ends the user process U1T; the user U2 disconnects from the user server and ends the user process U2T. Refer to number 9 in Figure 5.
  • the table seen in Figure 5 represents the data stored locally on the user server, including the file name submitted and saved by the user, the unique key corresponding to the file name, and the storage location of the file after being processed and uploaded to a different network storage server. .
  • Step 201 Two users U1 and U2 access the user server at the same time, and the main service process T of the user server creates two user processes U1T and U2T for the two users respectively.
  • the main service process T of the user server creates two user processes U1T and U2T for the two users respectively.
  • Step 202 Two users submit multiple file reading requests at the same time, the files are U1F1, U1F2, U2F1, U2F2, and the reading request of U1 reaches the user process U1T of U1, and U1T is the subsequent operation of the two files U1F1 and U1F2 Create two user subprocesses U1F1T and U1F2T respectively; U2's read request reaches the user process U2T of U2, U2T creates two user subprocesses U2F1T and U2F2T for the subsequent operations of the two files U2F1 and U2F2, as shown in Figure 4 2. After the user sub-processes are created, they independently read the key corresponding to the file to be read by the user and the storage location on the network storage server from the local. As shown in the table in Figure 6;
  • Step 203 The user subprocess U1F1T reads the local records, reads the key U1F1K of the file U1F1 and the storage location of the fragment data in the network storage servers S1, S2, S3, and downloads U1F1P1M1, U1F1P1M2, U1F1P1M3, and U1F1P2M1 according to the storage location.
  • Step 204 The user sub-process U1F1T applies to the main service process T for an idle FPGA key storage register KR1, and writes U1F1K to KR1; the user sub-process U1F2T applies to the main service process T for an idle FPGA key storage register KR2, And write U1F2K to KR2; user subprocess U2F1T applies to the main business process T for an idle FPGA key storage register KR3, and writes U2F1K to KR3; user subprocess U2F2T applies to the main business process T for an idle FPGA key Store register KR4, and write U2F2K into KR4. Refer to number 4 in Figure 6;
  • Step 205 The user sub-process U1F1T applies to the main business process T for free FPGA data storage registers DR1 and DR2, and converts the sub-sub-data U1F1P1M1, U1F1P1M2, U1F1P1M3, the storage location KR1 of the key U1F1K and the processing logic according to a certain format.
  • the user sub-process U1F2T applies to the main business process T for an idle FPGA data storage register DR3 And DR4, and write the sub-sub-data U1F2P1M1, U1F2P1M2, U1F2P1M3, the storage location KR2 of the key U1F2K and processing logic into DR3 together, the sub-sub-data U1F2P2M1, U1F2P2M2, U1F2P2M3 and the key U1F2K storage location KR2 and the processing logic KR2
  • the user sub-process U2F1T applies to the main business process T for free FPGA data storage registers DR5 and DR6, and the sub-sub-data U2F1P1M1, U2F1P1M2, U
  • the information is written into DR5 together, and the sub-sub-data U2F1P2M1, U2F1P2M2, U2F1P2M3, the storage location KR3 of the key U2F1K and the processing logic are written into DR6 together;
  • the user sub-process U2F2T applies to the main business process T for an idle FPGA data storage register DR7 Together with DR8, write the sub-sub-data U2F2P1M1, U2F2P1M2, U2F2P1M3, the storage location KR4 of the key U2F2K and processing logic into DR7, the sub-sub-data U2F2P2M1, U2F2P2M2, U2F2P2M3 and the key U2F2K storage location KR4 and the processing logic KR4 Wait for the information to be written into DR8 together. After each user subprocess above completes the operation of writing the DR, it will immediately read the same DR in a blocking mode. Refer to number 5 in Figure
  • Step 206 The FPGA scans regularly and finds that there is data to be processed in the data storage register DR1. It is read in a certain format, and the processing logic is read according to some of the fields as de-redundancy, descrambling, and decryption, and reads the corresponding data according to some of the fields.
  • the key is stored in the key storage register KR1, and the key U1F1K and sub-sub-data U1F1P1M1, U1F1P1M2, U1F1P1M3 are taken out, and the read-out processing logic is used for de-redundancy, descrambling, and decryption.
  • the processing and output sub-data U1F1P1 is in a certain format Write to DR1; FPGA continues to scan and finds that there is data to be processed in the data storage register DR2. Similar to the above process, write U1F1P2 to DR2; similarly, FPGA writes U1F2P1 to DR3, U1F2P2 to DR4, and U2F1P1. DR5, write U2F1P2 into DR6, write U2F2P1 into DR7, and write U2F2P2 into DR8. Refer to number 6 in Figure 6;
  • Step 207 After the above FPGA writes the sub-data U1F1P1 into DR1, the user sub-process U1F1T in the blocked state of reading DR1 gets the read return immediately, so U1F1P1 is read out and returned to the user U1, and the service main process T is notified to release the data storage Register DR1; similarly, U1F1P2 is returned to user U1, and DR2 is released.
  • the file U1F1 is read, and the key storage register KR1 of FPGA can also notify the service main process T to release; U1F2P1 is returned to user U1 and released DR3; U1F2P2 is returned to user U1, and DR4 is released.
  • the file U1F2 is read, and the key storage register KR2 of FPGA can also notify the service main process T to release; U2F1P1 is returned to user U2, and DR5 is released; U2F1P2 is returned Give to user U2 and release DR6.
  • the file U2F1 is read, and the key storage register KR3 of FPGA can also notify the main service process T to release; U2F2P1 is returned to user U2, and DR7 is released; U2F2P2 is returned to user U2, And release DR8.
  • the file U2F2 is read, and the key storage register KR4 of the FPGA can also notify the main service process T to release. Refer to number 7 in Figure 6;
  • Step 208 U1F1 has been read, end user subprocess U1F1T; U1F2 has been read, end user subprocess U1F2T; U2F1 has been read, end user subprocess U2F1T; U2F2 has been read, end user subprocess U2F2T .
  • Step 209 The user U1 disconnects from the user server and ends the user process U1T; the user U2 disconnects from the user server and ends the user process U2T. Refer to number 9 in Figure 6.
  • the table seen in Figure 6 represents the data stored locally on the user server, including the file name submitted and saved by the user, the unique key corresponding to the file name, and the storage location of the file after being processed and uploaded to a different network storage server. .
  • the network-defined storage system includes a user server with FPGA boards and multiple network storage servers; the user server is configured with a certain amount of CPU and memory to support the system overhead of multi-user and multi-process; the registers on the FPGA are set according to a certain amount. The length and location are divided into several areas, one part is used to read and write sub-data to be processed, and the other part is used to read and write keys; the user server can access the network storage server through the authentication method required by the network and the network server.
  • the embodiments of the present invention provide a data storage method, a reading method, and a network definition storage system; the technical solution of the present invention adopts a multi-key, multi-process, multi-user, and multi-data working mechanism, which is specific to each user's Each data is set with an independent and unique key, and at the same time, advanced multi-process business logic is adopted to realize the parallel operation of multi-user and multi-data, which greatly improves the efficiency of the entire network-defined storage system.

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Software Systems (AREA)
  • General Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Bioethics (AREA)
  • Computer Security & Cryptography (AREA)
  • Computer Hardware Design (AREA)
  • General Health & Medical Sciences (AREA)
  • Health & Medical Sciences (AREA)
  • Databases & Information Systems (AREA)
  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)

Abstract

本发明涉及网络定义存储技术领域，具体涉及一种基于FPGA设备的网络定义存储方法、读取方法及相应的网络定义存储系统。本发明的技术方案采用多密钥、多进程、多用户、多数据的工作机制，针对每个用户的每个数据都设置独立唯一的密钥，同时采用先进的多进程业务逻辑，实现多用户、多数据的并行操作，极大地提高了整个网络定义存储系统的效率。

Description

一种基于FPGA设备的网络定义存储方法、读取方法及系统 技术领域
本发明涉及网络定义存储技术领域，具体涉及一种基于FPGA设备的网络定义存储方法、读取方法及相应的网络定义存储系统。
背景技术
网络定义存储是一种在线存储技术，以高安全高可靠著称。
用户服务器将原始数据按一定长度拆分成若干子数据后，按顺序交由本服务器上部署的FPGA进行加密、加扰和冗余处理，然后通过网络将FPGA处理过的多个子数据块分别上传到多个网络存储服务器中。用户需要读取原始数据时，采用相反的方向，先从各个网络存储服务器下载子数据块到用户服务器，再按顺序经用户服务器上部署的FPGA进行去冗余、去扰、解密，最后重新拼装成原始数据返还给用户。由于采用冗余技术，可以确保在部分网络存储服务器出现异常无法下载的情况下，从其他正常的网络存储服务器下载的子数据块也能还原出完整的原始数据。
但是，现有的网络定义存储方法及系统中，所有用户的所有数据共用同一个密钥，有很大的安全隐患；并且无论进行加密、加扰和冗余处理，还是进行去冗余、去扰和解密处理，对子数据的处理先后顺序有严格的要求，工作效率低。并且现有的网络定义存储方法及系统不支持多数据并行处理，更不支持多用户并行操作数据。因此系统整体运行效率很低，对用户服务器的CPU、内存以及FPGA的性能是个极大地浪费。
发明内容
为了解决现有技术中存在的问题，本发明提出了一种基于FPGA设备的网络定义存储方法、读取方法，以及一种基于FPGA设备的网络定 义存储系统。
本发明的技术方案采用多密钥、多进程、多用户、多数据的工作机制，针对每个用户的每个数据都设置独立唯一的密钥，同时采用先进的多进程业务逻辑，实现多用户、多数据的并行操作，极大地提高了整个网络定义存储系统的效率。
本发明的技术方案采用每个用户分配一个用户进程、每个用户的每个数据存储/读取操作分配一个用户子进程的方式，再结合每个用户的每个数据分配唯一的不重复的密钥以及多个FPGA密钥存储寄存器和多个FPGA数据存储寄存器，很高效地实现了多用户多数据操作请求并发处理。
为了实现上述目的，本发明提出了一种基于FPGA设备的网络定义存储方法，包括以下步骤：
步骤S101、用户服务器为每个用户创建用户主进程，并为每个用户的每个待存储数据创建用户子进程；
步骤S102、每个所述用户子进程向对应的用户主进程申请唯一密钥，所述用户主进程将所述每个待存储数据拆分为若干子数据；
步骤S103、每个所述用户子进程向对应的用户主进程申请空闲的FPGA密钥存储寄存器，并将所述用户子进程的唯一密钥写入该FPGA密钥存储寄存器；
步骤S104、每个所述用户子进程向对应的用户主进程申请空闲的FPGA数据存储寄存器，并将对应的所述子数据、密钥的保存位置和处理逻辑信息写入对应的FPGA数据存储寄存器；
步骤S105、当所述用户子进程完成该FPGA数据存储寄存器的写入操作后，立即用阻塞方式读该FPGA数据存储寄存器；FPGA根据从FPGA数据存储寄存器中读出的子数据、密钥的保存位置和处理逻辑信息，取出所述待存储数据所对应的唯一密钥和子数据，并根据处理逻辑信息对所述子数据进行逻辑处理，输出N个子子数据，其中，N为正整数；将所述N个子子数据写入该FPGA数据存储寄存器；
步骤S106、把所述N个子子数据分别上传到N个网络存储服务器，所述用户子进程通知用户主进程释放相应的FPGA数据存储寄存器和FPGA密钥存储寄存器；
步骤S107、结束所述用户子进程和用户主进程。
优选的，所述的处理逻辑包括加密、加扰和加冗余。
优选的，所述的N个为至少3个。
优选的，所述用户进程、用户子进程的调度、以及所述FPGA密钥存储寄存器和所述FPGA数据存储寄存器的申请和释放，都由所述用户服务器统一调度。
本发明还提出了一种基于FPGA设备的网络定义读取方法，包括以下步骤:
步骤S201、用户服务器为每个用户创建用户主进程，并为每个用户的每个待读取数据创建用户子进程；
步骤S202、每个所述用户子进程从用户服务器中读出每个待读取数据所对应的唯一密钥和该数据对应的N个子子数据在网络存储服务器上的存储位置信息，所述用户子进程根据所述存储位置信息，下载该存储位置上的N个子子数据；
步骤S203、每个所述用户子进程向对应的用户主进程申请空闲的PFGA密钥存储寄存器，并将步骤S202中所述待读取数据所对应的唯一密钥写入该FPGA密钥存储寄存器；
步骤S204、每个所述用户子进程向对应的用户主进程申请空闲的FPGA数据存储寄存器，并将步骤S202中下载的N个子子数据、密钥存储位置和处理逻辑信息写入该FPGA数据存储寄存器；
步骤S205、当所述用户子进程完成该FPGA据存储寄存器的写入操作后，立即用阻塞方式读FPGA数据存储寄存器；FPGA根据从FPGA数据存储寄存器中读出的待读取数据所对应的唯一密钥在FPGA密钥存储寄存器中的位置以及处理逻辑信息，取出待读取数据所对应的唯一密钥以及子子数据，按照所述处理逻辑信息对子子数据进行逻辑处理，输 出子数据，并把所述子数据写入FPGA数据存储寄存器；
步骤S206、所述用户子进程通过阻塞方式读出步骤S205的FPGA数据存储寄存器中的子数据，并把该子数据返还给用户，所述用户子进程通知用户主进程释放相应的FPGA数据存储寄存器和FPGA密钥存储寄存器；
步骤S207、结束所述用户子进程和用户主进程。
优选的，所述的处理逻辑包括去冗余、去扰和解密。
优选的，所述的N个为至少3个。
优选的，所述用户进程、用户子进程的调度、以及所述FPGA密钥存储寄存器和所述FPGA数据存储寄存器的申请和释放，都由所述用户服务器统一调度。
本发明还提出了一种基于FPGA设备的网络定义存储系统，包括：用户服务器和N个网络存储服务器，所述用户服务器配置有FPGA、CPU和内存，所述FPGA配置有寄存器，所述寄存器包括FPGA数据存储寄存器和FPGA密钥存储寄存器，所述FPGA数据存储寄存器用于读写待处理子数据，所述FPGA密钥存储寄存器用于读写密钥，所述用户服务器配置有CPU和内存，所述系统采用一种基于FPGA设备的网络定义存储方法。
本发明的技术方案和现有技术相比，其有益效果在于：
1、现有技术所有用户的所有数据共用同一个密钥，有很大的安全隐患；而在本发明的技术方案中，每个用户的每个数据都使用唯一的不重复的密钥，安全性极高；
2、现有技术只定义了一个密钥存储寄存器区域和一个数据存储寄存器区域，每个待存储子数据或待读取子子数据必须等前一个子数据或子子数据被FPGA处理完毕并被读出后，才能写入FPGA的数据存储寄存器；而在本发明的技术方案中，定义了多个FPGA数据存储寄存器，由用户服务器主进程进行集中调度，极大地提高了用户服务器上用户进程与FPGA交换数据的效率；
3、本发明的技术方案采用每个用户分配一个用户进程、每个用户的每个数据存储/读取操作分配一个用户子进程的方式，再结合每个用户的每个数据分配唯一的不重复的密钥以及多个FPGA密钥存储寄存器和多个FPGA数据存储寄存器，很高效地实现了多用户多数据操作请求并发处理。
附图说明
图1是本发明一种基于FPGA设备的网络定义存储方法步骤流程图。
图2为本发明一种基于FPGA设备的网络定义读取方法步骤流程图。
图3为本发明一种基于FPGA设备的网络定义存储方法基本步骤示意图。
图4为本发明一种基于FPGA设备的网络定义读取方法基本步骤示意图。
图5为具体实施方式一种基于FPGA设备的网络定义存储方法原理示意图。
图6为具体实施方式一种基于FPGA设备的网络定义读取方法原理示意图。
具体实施方式
下面结合附图和实施例，对本发明的具体实施方式作进一步详细描述。以下实施例用于说明本发明，但不用来限制本发明的范围。
实施例1
如附图1所示，一种基于FPGA设备的网络定义存储方法，包括以下步骤：
步骤S101、用户服务器为每个用户创建用户主进程，并为所述用户的每个待存储数据创建用户子进程；
步骤S102、每个所述用户子进程向对应的用户主进程申请唯一密钥，所述用户主进程将所述每个待存储数据拆分为若干子数据；
步骤S103、每个所述用户子进程向对应的用户主进程申请空闲的 FPGA密钥存储寄存器，并将所述用户子进程的唯一密钥写入该FPGA密钥存储寄存器；
步骤S104、每个所述用户子进程向对应的用户主进程申请空闲的FPGA数据存储寄存器，并将对应的所述子数据、密钥的保存位置和处理逻辑信息写入对应的FPGA数据存储寄存器；
步骤S105、当所述用户子进程完成该FPGA数据存储寄存器的写入操作后，立即用阻塞方式读该FPGA数据存储寄存器；FPGA根据从FPGA数据存储寄存器中读出的子数据、密钥的保存位置和处理逻辑信息，取出所述待存储数据所对应的唯一密钥和子数据，并根据处理逻辑信息对所述子数据进行逻辑处理，输出N个子子数据，其中，N为正整数；将所述N个子子数据写入该FPGA数据存储寄存器；
步骤S106、把所述N个子子数据分别上传到N个网络存储服务器，所述用户子进程通知用户主进程释放相应的FPGA数据存储寄存器和FPGA密钥存储寄存器；
步骤S107、结束所述用户子进程和用户主进程。
实施例2
如附图2所示，一种基于FPGA设备的网络定义读取方法，包括以下步骤：
步骤S201、用户服务器为每个用户创建用户主进程，并为每个用户的每个待读取数据创建用户子进程；
步骤S202、每个所述用户子进程从用户服务器中读出每个待读取数据所对应的唯一密钥和该数据对应的N个子子数据在网络存储服务器上的存储位置信息，所述用户子进程根据所述存储位置信息，下载该存储位置上的N个子子数据；
步骤S203、每个所述用户子进程向对应的用户主进程申请空闲的PFGA密钥存储寄存器，并将步骤S202中所述待读取数据所对应的唯一密钥写入该FPGA密钥存储寄存器；
步骤S204、每个所述用户子进程向对应的用户主进程申请空闲的 FPGA数据存储寄存器，并将步骤S202中下载的N个子子数据、密钥存储位置和处理逻辑信息写入该FPGA数据存储寄存器；
步骤S205、当所述用户子进程完成该FPGA据存储寄存器的写入操作后，立即用阻塞方式读FPGA数据存储寄存器；FPGA根据从FPGA数据存储寄存器中读出的待读取数据所对应的唯一密钥在FPGA密钥存储寄存器中的位置以及处理逻辑信息，取出待读取数据所对应的唯一密钥以及子子数据，按照所述处理逻辑信息对子子数据进行逻辑处理，输出子数据，并把所述子数据写入FPGA数据存储寄存器；
步骤S206、所述用户子进程通过阻塞方式读出步骤S205的FPGA数据存储寄存器中的子数据，并把该子数据返还给用户，所述用户子进程通知用户主进程释放相应的FPGA数据存储寄存器和FPGA密钥存储寄存器；
步骤S207、结束所述用户子进程和用户主进程。
实施例3
一种基于FPGA设备的网络定义存储方法基本步骤
如附图3所示，标号1、2、3与上述存储基本步骤中的第S101、S102、步对应，标号4与上述存储基本步骤中的第S103步对应，标号5与上述存储基本步骤中的第S104步对应，标号6与上述存储基本步骤中的第S105步对应，标号7与上述存储基本步骤中的第S105步对应,标号8与上述存储基本步骤中的第S106步对应,标号9与上述存储基本步骤中的第S107步对应。
实施例4
一种基于FPGA设备的网络定义读取方法基本步骤
如附图4所示，标号1、2、3与上述读取基本步骤中的第S201、S202步对应，标号4与上述读取基本步骤中的第S203步对应，标号5与上述读取基本步骤中的第S204步对应，标号6与上述读取基本步骤中的第S204步对应，标号7与上述读取基本步骤中的第S205步对应，标号8与上述读取基本步骤中的第S206步对应,标号9与上述读取基本步骤中 的第S207步对应。
实施例5
一种基于FPGA设备的网络定义存储方法，如附图5所示。
假设条件：U1和U2两个用户同时访问用户服务器，两个用户同时存储多个文件，其中用户U1存储2个文件U1F1和U1F2，用户U2存储两个文件U2F1和U2F2。
步骤101：两个用户U1和U2同时访问用户服务器，用户服务器本业务主进程T为两个用户分别创建两个用户进程U1T和U2T。如附图5标号1；
步骤102：两个用户同时提交多个文件存储请求，文件分别为U1F1、U1F2、U2F1、U2F2，其中U1的存储请求到达U1的用户进程U1T，U1T为两个文件U1F1和U1F2的后续操作分别创建两个用户子进程U1F1T和U1F2T；U2的存储请求到达U2的用户进程U2T，U2T为两个文件U2F1和U2F2的后续操作分别创建两个用户子进程U2F1T和U2F2T，用户子进程创建好后，就各自独立接收对应用户提交的对应文件。如附图5标号2；
步骤103：用户子进程U1F1T为文件U1F1向本业务主进程T申请唯一密钥U1F1K并进行本地保存，同时将文件U1F1按约定大小拆分为子数据U1F1P1和U1F1P2；用户子进程U1F2T为文件U1F2向本业务主进程T申请唯一密钥U1F2K，同时将文件U1F2按约定大小拆分为子数据U1F2P1和U1F2P2；用户子进程U2F1T为文件U2F1向本业务主进程T申请唯一密钥U2F1K，同时将文件U2F1按约定大小拆分为子数据U2F1P1和U2F1P2；用户子进程U2F2T为文件U2F2向本业务主进程T申请唯一密钥U2F2K，同时将文件U2F2按约定大小拆分为子数据U2F2P1和U2F2P2。如附图5标号3；
步骤104：用户子进程U1F1T向本业务主进程T申请空闲的FPGA密钥存储寄存器KR1，并将U1F1K写入KR1；用户子进程U1F2T向本业务主进程T申请空闲的FPGA密钥存储寄存器KR2，并将U1F2K写入KR2；用户子进程U2F1T向本业务主进程T申请空闲的FPGA密钥存储寄存器KR3，并将U2F1K写入KR3；用户子进程U2F2T向本业务主进程T申请空闲的FPGA密 钥存储寄存器KR4，并将U2F2K写入KR4。如附图5标号4；
步骤105：用户子进程U1F1T向本业务主进程T申请空闲的FPGA数据存储寄存器DR1和DR2，并将子数据U1F1P1和密钥U1F1K的保存位置KR1以及处理逻辑等信息一起写入DR1，子数据U1F1P2和密钥U1F1K的保存位置KR1以及处理逻辑等信息一起写入DR2；用户子进程U1F2T向本业务主进程T申请空闲的FPGA数据存储寄存器DR3和DR4，并将子数据U1F2P1和密钥U1F2K的保存位置KR2以及处理逻辑等信息一起写入DR3，子数据U1F2P2和密钥U1F2K的保存位置KR2以及处理逻辑等信息一起写入DR4；用户子进程U2F1T向本业务主进程T申请空闲的FPGA数据存储寄存器DR5和DR6，并将子数据U2F1P1和密钥U2F1K的保存位置KR3以及处理逻辑等信息一起写入DR5，子数据U2F1P2和密钥U2F1K的保存位置KR3以及处理逻辑等信息一起写入DR6；用户子进程U2F2T向本业务主进程T申请空闲的FPGA数据存储寄存器DR7和DR8，并将子数据U2F2P1和密钥U2F2K的保存位置KR4以及处理逻辑等信息一起写入DR7，子数据U2F2P2和密钥U2F2K的保存位置KR4以及处理逻辑等信息一起写入DR8。上述每一个用户子进程完成写DR的操作后，都会立即用阻塞方式读同一个DR。如附图5标号5；
步骤106：FPGA定时扫描，发现数据存储寄存器DR1中有待处理数据，按一定格式读出，并根据其中部分字段读出处理逻辑为加密、加扰、加冗余，并根据其中部分字段读出对应密钥保存在密钥存储寄存器KR1中，取出密钥U1F1K及子数据U1F1P1，按读出的处理逻辑进行加密、加扰、加冗余处理，处理输出的三块，其特征在于，输出的子子数据个数与所配置网络存储服务器的个数相同，为实现高可靠性，即使其中某几个网络存储服务器出现异常，利用冗余算法从其他网络存储服务器下载的子子数据也能还原出正确的子数据，最好大于等于3个网络存储服务器；子子数据U1F1P1M1、U1F1P1M2和U1F1P1M3按一定格式写入DR1；FPGA继续扫描，发现数据存储寄存器DR2中有待处理数据，和上述过程类似，将U1F1P2M1、U1F1P2M2、U1F1P2M3写入DR2；同样的， FPGA分别将U1F2P1M1、U1F2P1M2、U1F2P1M3写入DR3，将U1F2P2M1、U1F2P2M2、U1F2P2M3写入DR4，将U2F1P1M1、U2F1P1M2、U2F1P1M3写入DR5，将U2F1P2M1、U2F1P2M2、U2F1P2M3写入DR6，将U2F2P1M1、U2F2P1M2、U2F2P1M3写入DR7，将U2F2P2M1、U2F2P2M2、U2F2P2M3写入DR8。如附图5标号6；
步骤107：当上述FPGA将子子数据U1F1P1M1、U1F1P1M2和U1F1P1M3写入DR1后，处于读DR1阻塞状态的用户子进程U1F1T立即得到读返回，于是读出U1F1P1M1、U1F1P1M2和U1F1P1M3，并立即分别上传到三个云存储服务器S1、S2和S3。上传完毕后，通知本业务主进程T释放数据存储寄存器DR1，同时本地保存这三个子子数据在云存储服务器的保存位置；同样的，U1F1P2M1、U1F1P2M2、U1F1P2M3分别上传到S1、S2和S3，并释放DR2，此时文件U1F1存储完成，FPGA的密钥存储寄存器KR1也可以通知本业务主进程T释放了；U1F2P1M1、U1F2P1M2、U1F2P1M3分别上传到S1、S2和S3，并释放DR3；U1F2P2M1、U1F2P2M2、U1F2P2M3分别上传到S1、S2和S3，并释放DR4，此时文件U1F2存储完成，FPGA的密钥存储寄存器KR2也可以通知本业务主进程T释放了；U2F1P1M1、U2F1P1M2、U2F1P1M3分别上传到S1、S2和S3，并释放DR5；U2F1P2M1、U2F1P2M2、U2F1P2M3分别上传到S1、S2和S3，并释放DR6，此时文件U2F1存储完成，FPGA的密钥存储寄存器KR3也可以通知本业务主进程T释放了；U2F2P1M1、U2F2P1M2、U2F2P1M3分别上传到S1、S2和S3，并释放DR7；U2F2P2M1、U2F2P2M2、U2F2P2M3分别上传到S1、S2和S3，并释放DR8，此时文件U2F2存储完成，FPGA的密钥存储寄存器KR4也可以通知本业务主进程T释放了。如附图5标号7；
步骤108：U1F1已存储完毕，结束用户子进程U1F1T；U1F2已存储完毕，结束用户子进程U1F2T；U2F1已存储完毕，结束用户子进程U2F1T；U2F2已存储完毕，结束用户子进程U2F2T。如附图5标号8；
步骤109：用户U1断开与用户服务器的连接，结束用户进程U1T；用户U2断开与用户服务器的连接，结束用户进程U2T。如附图5标号9。
其中附图5中看到的表格表示用户服务器本地保存的数据，包括用户所提交保存的文件名，文件名对应的唯一密钥，文件被处理后上传到不同的网络存储服务器的存储位置等信息。
实施例6
一种基于FPGA设备的网络定义读取方法，如附图6所示。
假设条件：U1和U2两个用户同时访问用户服务器，两个用户同时读取多个文件，其中用户U1读取2个文件U1F1和U1F2，用户U2读取两个文件U2F1和U2F2。
步骤201：两个用户U1和U2同时访问用户服务器，用户服务器本业务主进程T为两个用户分别创建两个用户进程U1T和U2T。如附图6标号1；
步骤202：两个用户同时提交多个文件读取请求，文件分别为U1F1、U1F2、U2F1、U2F2，其中U1的读取请求到达U1的用户进程U1T，U1T为两个文件U1F1和U1F2的后续操作分别创建两个用户子进程U1F1T和U1F2T；U2的读取请求到达U2的用户进程U2T，U2T为两个文件U2F1和U2F2的后续操作分别创建两个用户子进程U2F1T和U2F2T，如附图4标号2。用户子进程创建好后，就各自独立从本地读出用户所要读取的文件对应的密钥和在网络存储服务器上的保存位置。如附图6中的表格；
步骤203：用户子进程U1F1T读取本地记录，读出文件U1F1的密钥U1F1K和分片数据在网络存储服务器S1、S2、S3的存储位置，根据存储位置分别下载U1F1P1M1、U1F1P1M2、U1F1P1M3和U1F1P2M1、U1F1P2M2、U1F1P2M3；同样的，用户子进程U1F2T读取本地记录，读出文件U1F2的密钥U1F2K和分片数据在网络存储服务器S1、S2、S3的存储位置，根据存储位置分别下载U1F2P1M1、U1F2P1M2、U1F2P1M3和U1F2P2M1、U1F2P2M2、U1F2P2M3；用户子进程U2F1T读取本地记录，读出文件U2F1的密钥U2F1K和分片数据在网络存储服务器S1、S2、S3的存储位置，根据存储位置分别下载U2F1P1M1、U2F1P1M2、U2F1P1M3和U2F1P2M1、U2F1P2M2、U2F1P2M3；用户子进程U2F1T读取本地记录，读出文件U2F2 的密钥U2F2K和分片数据在网络存储服务器S1、S2、S3的存储位置，根据存储位置分别下载U2F2P1M1、U2F2P1M2、U2F2P1M3和U2F2P2M1、U2F2P2M2、U2F2P2M3。如附图6标号3；
步骤204：用户子进程U1F1T向本业务主进程T申请空闲的FPGA密钥存储寄存器KR1，并将U1F1K写入KR1；用户子进程U1F2T向本业务主进程T申请空闲的FPGA密钥存储寄存器KR2，并将U1F2K写入KR2；用户子进程U2F1T向本业务主进程T申请空闲的FPGA密钥存储寄存器KR3，并将U2F1K写入KR3；用户子进程U2F2T向本业务主进程T申请空闲的FPGA密钥存储寄存器KR4，并将U2F2K写入KR4。如附图6标号4；
步骤205：用户子进程U1F1T向本业务主进程T申请空闲的FPGA数据存储寄存器DR1和DR2，并按一定格式将子子数据U1F1P1M1、U1F1P1M2、U1F1P1M3和密钥U1F1K的保存位置KR1以及处理逻辑等信息一起写入DR1，将子子数据U1F1P2M1、U1F1P2M2、U1F1P2M3和密钥U1F1K的保存位置KR1以及处理逻辑等信息一起写入DR2；用户子进程U1F2T向本业务主进程T申请空闲的FPGA数据存储寄存器DR3和DR4，并将子子数据U1F2P1M1、U1F2P1M2、U1F2P1M3和密钥U1F2K的保存位置KR2以及处理逻辑等信息一起写入DR3，子子数据U1F2P2M1、U1F2P2M2、U1F2P2M3和密钥U1F2K的保存位置KR2以及处理逻辑等信息一起写入DR4；用户子进程U2F1T向本业务主进程T申请空闲的FPGA数据存储寄存器DR5和DR6，并将子子数据U2F1P1M1、U2F1P1M2、U2F1P1M3和密钥U2F1K的保存位置KR3以及处理逻辑等信息一起写入DR5，子子数据U2F1P2M1、U2F1P2M2、U2F1P2M3和密钥U2F1K的保存位置KR3以及处理逻辑等信息一起写入DR6；用户子进程U2F2T向本业务主进程T申请空闲的FPGA数据存储寄存器DR7和DR8，并将子子数据U2F2P1M1、U2F2P1M2、U2F2P1M3和密钥U2F2K的保存位置KR4以及处理逻辑等信息一起写入DR7，子子数据U2F2P2M1、U2F2P2M2、U2F2P2M3和密钥U2F2K的保存位置KR4以及处理逻辑等信息一起写入DR8。上述每一个用户子进程完成写DR的操作后，都会立即用阻塞方式读同一个DR。如附图6标号5；
步骤206：FPGA定时扫描，发现数据存储寄存器DR1中有待处理数据，按一定格式读出，并根据其中部分字段读出处理逻辑为去冗余、去扰、解密，并根据其中部分字段读出对应密钥保存在密钥存储寄存器KR1中，取出密钥U1F1K及子子数据U1F1P1M1、U1F1P1M2、U1F1P1M3，按读出的处理逻辑进行去冗余、去扰、解密处理，处理输出子数据U1F1P1按一定格式写入DR1；FPGA继续扫描，发现数据存储寄存器DR2中有待处理数据，和上述过程类似，将U1F1P2写入DR2；同样的，FPGA分别将U1F2P1写入DR3，将U1F2P2写入DR4，将U2F1P1写入DR5，将U2F1P2写入DR6，将U2F2P1写入DR7，将U2F2P2写入DR8。如附图6标号6；
步骤207：当上述FPGA将子数据U1F1P1写入DR1后，处于读DR1阻塞状态的用户子进程U1F1T立即得到读返回，于是读出U1F1P1并返还给用户U1，同时通知本业务主进程T释放数据存储寄存器DR1；同样的，U1F1P2返还给用户U1，并释放DR2，此时文件U1F1读取完成，FPGA的密钥存储寄存器KR1也可以通知本业务主进程T释放了；U1F2P1返还给用户U1，并释放DR3；U1F2P2返还给用户U1，并释放DR4，此时文件U1F2读取完成，FPGA的密钥存储寄存器KR2也可以通知本业务主进程T释放了；U2F1P1返还给用户U2，并释放DR5；U2F1P2返还给用户U2，并释放DR6，此时文件U2F1读取完成，FPGA的密钥存储寄存器KR3也可以通知本业务主进程T释放了；U2F2P1返还给用户U2，并释放DR7；U2F2P2返还给用户U2，并释放DR8，此时文件U2F2读取完成，FPGA的密钥存储寄存器KR4也可以通知本业务主进程T释放了。如附图6标号7；
步骤208：U1F1已读取完毕，结束用户子进程U1F1T；U1F2已读取完毕，结束用户子进程U1F2T；U2F1已读取完毕，结束用户子进程U2F1T；U2F2已读取完毕，结束用户子进程U2F2T。如附图6标号8；
步骤209：用户U1断开与用户服务器的连接，结束用户进程U1T；用户U2断开与用户服务器的连接，结束用户进程U2T。如附图6标号9。
其中附图6中看到的表格表示用户服务器本地保存的数据，包括用户所提交保存的文件名，文件名对应的唯一密钥，文件被处理后上传 到不同的网络存储服务器的存储位置等信息。
实施例7
网络定义存储系统包含一台部署有FPGA板卡的用户服务器和多个网络存储服务器；用户服务器配置有一定数量的CPU和内存，用以支撑多用户多进程的系统开销；FPGA上的寄存器按一定长度和位置划分成若干区域，一部分用于读写待处理子数据，一部分用于读写密钥；用户服务器能通过网络及网络服务器所要求的鉴权方式访问网络存储服务器。
综上，本发明实施例提供一种数据存储方法、读取方法及网络定义存储系统；本发明的技术方案采用多密钥、多进程、多用户、多数据的工作机制，针对每个用户的每个数据都设置独立唯一的密钥，同时采用先进的多进程业务逻辑，实现多用户、多数据的并行操作，极大地提高了整个网络定义存储系统的效率。
以上内容是结合具体的实施方式对本方法做的进一步详细说明，但不能认定本方法的具体实施只限于这些说明。对于本方法所属技术领域的普通技术人员来说，在不脱离本实用信息构思的前提下，还可以做出若干简单推演或替换，都应该视为本方法的保护范围。

Claims (9)

  1. 一种基于FPGA设备的网络定义存储方法，其特征在于，包括以下步骤：
    步骤S101、用户服务器为每个用户创建用户主进程，并为每个用户的每个待存储数据创建用户子进程；
    步骤S102、每个所述用户子进程向对应的用户主进程申请唯一密钥，所述用户主进程将所述每个待存储数据拆分为若干子数据；
    步骤S103、每个所述用户子进程向对应的用户主进程申请空闲的FPGA密钥存储寄存器，并将所述用户子进程的唯一密钥写入该FPGA密钥存储寄存器；
    步骤S104、每个所述用户子进程向对应的用户主进程申请空闲的FPGA数据存储寄存器，并将对应的所述子数据、密钥的保存位置和处理逻辑信息写入对应的FPGA数据存储寄存器；
    步骤S105、当所述用户子进程完成该FPGA数据存储寄存器的写入操作后，立即用阻塞方式读该FPGA数据存储寄存器；FPGA根据从FPGA数据存储寄存器中读出的子数据、密钥的保存位置和处理逻辑信息，取出所述待存储数据所对应的唯一密钥和子数据，并根据处理逻辑信息对所述子数据进行逻辑处理，输出N个子子数据，其中，N为正整数；将所述N个子子数据写入该FPGA数据存储寄存器；
    步骤S106、把所述N个子子数据分别上传到N个网络存储服务器，所述用户子进程通知用户主进程释放相应的FPGA数据存储寄存器和FPGA密钥存储寄存器；
    步骤S107、结束所述用户子进程和用户主进程。
  2. 根据权利要求1所述的一种基于FPGA设备的网络定义存储方法，其特征在于，所述的处理逻辑包括加密、加扰和加冗余。
  3. 根据权利要求1所述的一种基于FPGA设备的网络定义存储方法，其特征在于，所述的N个为至少3个。
  4. 根据权利要求1所述的一种基于FPGA设备的网络定义存储方法，其特征在于，所述用户进程、用户子进程的调度、以及所述FPGA密钥存储寄存器和所述FPGA数据存储寄存器的申请和释放，都由所述用户服务器统一调度。
  5. 一种基于FPGA设备的网络定义读取方法，其特征在于，包括以下步骤:
    步骤S201、用户服务器为每个用户创建用户主进程，并为每个用户的每个待读取数据创建用户子进程；
    步骤S202、每个所述用户子进程从用户服务器中读出每个待读取数据所对应的唯一密钥和该数据对应的N个子子数据在网络存储服务器上的存储位置信息，所述用户子进程根据所述存储位置信息，下载该存储位置上的N个子子数据；
    步骤S203、每个所述用户子进程向对应的用户主进程申请空闲的PFGA密钥存储寄存器，并将步骤S202中所述待读取数据所对应的唯一密钥写入该FPGA密钥存储寄存器；
    步骤S204、每个所述用户子进程向对应的用户主进程申请空闲的FPGA数据存储寄存器，并将步骤S202中下载的N个子子数据、密钥存储位置和处理逻辑信息写入该FPGA数据存储寄存器；
    步骤S205、当所述用户子进程完成该FPGA据存储寄存器的写入操作后，立即用阻塞方式读FPGA数据存储寄存器；FPGA根据从FPGA数据存储寄存器中读出的待读取数据所对应的唯一密钥在FPGA密钥存储寄存器中的位置以及处理逻辑信息，取出待读取数据所对应的唯一密钥以及子子数据，按照所述处理逻辑信息对子子数据进行逻辑处理，输出子数据，并把所述子数据写入FPGA数据存储寄存器；
    步骤S206、所述用户子进程通过阻塞方式读出步骤S205的FPGA数据存储寄存器中的子数据，并把该子数据返还给用户，所述用户子 进程通知用户主进程释放相应的FPGA数据存储寄存器和FPGA密钥存储寄存器；
    步骤S207、结束所述用户子进程和用户主进程。
  6. 根据权利要求5所述的一种基于FPGA设备的网络定义读取方法，其特征在于，所述的处理逻辑包括去冗余、去扰和解密。
  7. 根据权利要求5所述的一种基于FPGA设备的网络定义读取方法，其特征在于，所述的N个为至少3个。
  8. 根据权利要求5所述的一种基于FPGA设备的网络定义读取方法，其特征在于，所述用户进程、用户子进程的调度、以及所述FPGA密钥存储寄存器和所述FPGA数据存储寄存器的申请和释放，都由所述用户服务器统一调度。
  9. 一种基于FPGA设备的网络定义存储系统，包括：用户服务器和N个网络存储服务器，所述用户服务器配置有FPGA、CPU和内存，所述FPGA配置有寄存器，所述寄存器包括FPGA数据存储寄存器和FPGA密钥存储寄存器，所述FPGA数据存储寄存器用于读写待处理子数据，所述FPGA密钥存储寄存器用于读写密钥，所述用户服务器配置有CPU和内存，其特征在于，所述系统采用权利要求1至4任意一项所述的一种基于FPGA设备的网络定义存储方法。
PCT/CN2019/127032 2019-12-05 2019-12-20 一种基于fpga设备的网络定义存储方法、读取方法及系统 WO2021109275A1 (zh)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN201911239370.XA CN110955525B (zh) 2019-12-05 2019-12-05 一种基于fpga设备的网络定义存储方法、读取方法及系统
CN201911239370.X 2019-12-05

Publications (1)

Publication Number Publication Date
WO2021109275A1 true WO2021109275A1 (zh) 2021-06-10

Family

ID=69979935

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2019/127032 WO2021109275A1 (zh) 2019-12-05 2019-12-20 一种基于fpga设备的网络定义存储方法、读取方法及系统

Country Status (2)

Country Link
CN (1) CN110955525B (zh)
WO (1) WO2021109275A1 (zh)

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114978950A (zh) * 2022-06-02 2022-08-30 江苏新质信息科技有限公司 基于fpga、cpu协同的网络算法调用方法及系统
CN116521249A (zh) * 2023-07-03 2023-08-01 北京左江科技股份有限公司 一种基于进程文件描述符的内核态报文分发方法

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20120110397A1 (en) * 2010-10-29 2012-05-03 Kabushiki Kaisha Toshiba Data transmission system, storage medium and data transmission program
CN102546181A (zh) * 2012-01-09 2012-07-04 西安电子科技大学 基于密钥池的云存储加解密方法
CN103488958A (zh) * 2012-06-20 2014-01-01 微软公司 管理具有隔离组件的现场可编程门阵列的使用
CN104852949A (zh) * 2014-02-14 2015-08-19 航天信息股份有限公司 基于混合加密机制的云存储数据管理方法和系统

Family Cites Families (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN106971369B (zh) * 2017-03-02 2020-06-12 南京师范大学 一种基于gpu的地形可视域分析的数据调度与分发方法
CN107526963A (zh) * 2017-08-30 2017-12-29 深圳市风云实业有限公司 密钥查找设备、方法和计算机可读存储介质
CN108182263A (zh) * 2018-01-05 2018-06-19 郑州云海信息技术有限公司 一种数据中心综合管理系统的数据存储方法
IT201800005506A1 (it) * 2018-05-18 2019-11-18 Sistema di elaborazione, relativo circuito integrato e procedimento

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20120110397A1 (en) * 2010-10-29 2012-05-03 Kabushiki Kaisha Toshiba Data transmission system, storage medium and data transmission program
CN102546181A (zh) * 2012-01-09 2012-07-04 西安电子科技大学 基于密钥池的云存储加解密方法
CN103488958A (zh) * 2012-06-20 2014-01-01 微软公司 管理具有隔离组件的现场可编程门阵列的使用
CN104852949A (zh) * 2014-02-14 2015-08-19 航天信息股份有限公司 基于混合加密机制的云存储数据管理方法和系统

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
WAN, YI: "Design and Uses of FC Encryption Cards in Storage Area Network (SAN)", MASTER DISSERTATION, 15 November 2009 (2009-11-15), pages 1 - 81, XP055818166 *

Cited By (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114978950A (zh) * 2022-06-02 2022-08-30 江苏新质信息科技有限公司 基于fpga、cpu协同的网络算法调用方法及系统
CN114978950B (zh) * 2022-06-02 2023-10-27 江苏新质信息科技有限公司 基于fpga、cpu协同的网络算法调用方法及系统
CN116521249A (zh) * 2023-07-03 2023-08-01 北京左江科技股份有限公司 一种基于进程文件描述符的内核态报文分发方法
CN116521249B (zh) * 2023-07-03 2023-10-10 北京左江科技股份有限公司 一种基于进程文件描述符的内核态报文分发方法

Also Published As

Publication number Publication date
CN110955525A (zh) 2020-04-03
CN110955525B (zh) 2022-12-20

Similar Documents

Publication Publication Date Title
US8966288B2 (en) System and method for providing encryption in storage operations in a storage network, such as for use by application service providers that provide data storage services
US7277941B2 (en) System and method for providing encryption in a storage network by storing a secured encryption key with encrypted archive data in an archive storage device
CN103530201B (zh) 一种适用于备份系统的安全数据去重方法和系统
US8225109B1 (en) Method and apparatus for generating a compressed and encrypted baseline backup
CN110784463B (zh) 一种基于区块链的文件存储和访问方法
US6993661B1 (en) System and method that provides for the efficient and effective sanitizing of disk storage units and the like
US8489893B2 (en) Encryption key rotation messages written and observed by storage controllers via storage media
CN102208001B (zh) 硬件支持的虚拟化密码服务
JP4648687B2 (ja) データストレージシステムにおける暗号化変換の方法と装置
AU2016203740B2 (en) Simultaneous state-based cryptographic splitting in a secure storage appliance
KR101047213B1 (ko) 암호화 장치, 암호화 방법 및 컴퓨터 판독가능한 기록 매체
US9256499B2 (en) Method and apparatus of securely processing data for file backup, de-duplication, and restoration
WO2005065084A2 (en) System and method for providing encryption in pipelined storage operations in a storage network
US20090282262A1 (en) Information Processing Apparatus, Information Processing System, and Encryption Information Management Method
US20090172417A1 (en) Key management method for remote copying
US11005663B2 (en) Secure audit scheme in a distributed data storage system
JP2009064178A (ja) ストレージ装置及びデータの管理方法
CN104331345B (zh) 一种数据恢复方法
CN102855452A (zh) 基于加密组块的快速数据加密策略遵从
WO2021109275A1 (zh) 一种基于fpga设备的网络定义存储方法、读取方法及系统
US8943328B2 (en) Key rotation for encrypted storage media
JP2002084269A (ja) 秘密鍵のリカバリ方法および保管方法
JP2009042892A (ja) データ管理システム，データ管理方法
US8615492B1 (en) Techniques for providing multiplexed data for backup
CN115079960B (zh) 数据的处理方法、加速卡及数据处理系统

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 19954780

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

32PN Ep: public notification in the ep bulletin as address of the adressee cannot be established

Free format text: NOTING OF LOSS OF RIGHTS PURSUANT TO RULE 112(1) EPC (EPO FORM 1205A DATED 091122)

122 Ep: pct application non-entry in european phase

Ref document number: 19954780

Country of ref document: EP

Kind code of ref document: A1