WO2024055679A1 - 数据存储方法、装置、系统、芯片及加速设备 - Google Patents
数据存储方法、装置、系统、芯片及加速设备 Download PDFInfo
- Publication number
- WO2024055679A1 WO2024055679A1 PCT/CN2023/102954 CN2023102954W WO2024055679A1 WO 2024055679 A1 WO2024055679 A1 WO 2024055679A1 CN 2023102954 W CN2023102954 W CN 2023102954W WO 2024055679 A1 WO2024055679 A1 WO 2024055679A1
- Authority
- WO
- WIPO (PCT)
- Prior art keywords
- data
- write operation
- storage
- log
- operation instruction
- Prior art date
Links
- 230000001133 acceleration Effects 0.000 title claims abstract description 200
- 238000013500 data storage Methods 0.000 title claims abstract description 103
- 238000000034 method Methods 0.000 title claims abstract description 97
- 238000012545 processing Methods 0.000 claims description 32
- 238000012986 modification Methods 0.000 claims description 21
- 230000004048 modification Effects 0.000 claims description 21
- 230000003111 delayed effect Effects 0.000 claims description 19
- 238000004590 computer program Methods 0.000 claims description 10
- 230000008569 process Effects 0.000 abstract description 37
- 230000006870 function Effects 0.000 description 26
- 238000010586 diagram Methods 0.000 description 11
- 230000003993 interaction Effects 0.000 description 8
- 238000004458 analytical method Methods 0.000 description 6
- 238000005516 engineering process Methods 0.000 description 5
- 230000009286 beneficial effect Effects 0.000 description 4
- 102100034033 Alpha-adducin Human genes 0.000 description 3
- 101000799076 Homo sapiens Alpha-adducin Proteins 0.000 description 3
- 101000629598 Rattus norvegicus Sterol regulatory element-binding protein 1 Proteins 0.000 description 3
- 230000002452 interceptive effect Effects 0.000 description 3
- 102100024348 Beta-adducin Human genes 0.000 description 2
- 101000689619 Homo sapiens Beta-adducin Proteins 0.000 description 2
- 238000013473 artificial intelligence Methods 0.000 description 2
- 238000011022 operating instruction Methods 0.000 description 2
- 239000004065 semiconductor Substances 0.000 description 2
- 230000011664 signaling Effects 0.000 description 2
- 239000007787 solid Substances 0.000 description 2
- 230000005540 biological transmission Effects 0.000 description 1
- 238000004364 calculation method Methods 0.000 description 1
- 238000004891 communication Methods 0.000 description 1
- 238000012790 confirmation Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 238000007726 management method Methods 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 230000002093 peripheral effect Effects 0.000 description 1
- 230000002085 persistent effect Effects 0.000 description 1
- 238000003672 processing method Methods 0.000 description 1
- 238000004904 shortening Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/46—Multiprogramming arrangements
- G06F9/50—Allocation of resources, e.g. of the central processing unit [CPU]
Definitions
- the present application relates to the field of computer technology, and in particular, to a data storage method, device, system, chip and acceleration equipment.
- an acceleration device can be configured for the host, and some data processing functions can be offloaded from the host to the acceleration device.
- the data access function can be offloaded from the host to the acceleration device, and the host can access data from the storage device through the acceleration device.
- various data access instructions sent by the host to the acceleration device are executed by the main processor of the acceleration device, consuming the computing resources of the main processor of the acceleration device and affecting the performance of the acceleration device.
- Embodiments of the present application provide a data storage method, device, system, chip and acceleration device, which can save computing resources of the main processor of the acceleration device and improve the execution efficiency of write operation instructions.
- inventions of the present application provide a data storage method, which can be applied to an acceleration device.
- the acceleration device is connected between a host and a storage device.
- the acceleration device may include an interface and a network processor.
- the method may include the following steps: an interface in the acceleration device receives a write operation instruction sent by the host, and forwards the write operation instruction to a network processor in the acceleration device; the network processor generates a pre-write operation log based on the write operation instruction, and Save the write-ahead operation log to the log storage area in the storage device.
- the pre-write operation log is used to save the to-be-written data carried in the above-mentioned write operation instructions to the data storage area in the storage device during playback through the storage controller of the storage device.
- the network processor in the acceleration device After the interface in the acceleration device receives the write operation instruction sent by the host, the network processor in the acceleration device generates a pre-write operation log based on the received write operation instruction, and saves the pre-write operation log to the storage. In the log storage area of the device, the above two steps are executed faster.
- the acceleration device does not need to process the write operation instructions to write the data to be written into the data storage area of the storage device, and there is no need to wait for the storage controller of the storage device to The data to be written is written into the data storage area, so it can reduce the time for the acceleration device to execute write operation instructions and improve the execution efficiency of the acceleration device for write operation instructions; and, by processing the write operation instructions through the network processor in the acceleration device, no acceleration is required
- the main processor in the device processes the write operation instructions, which can save the computing resources of the main processor in the acceleration device and improve the performance of the acceleration device.
- the network processor saves the pre-write operation log to a log storage area in the storage device. If the network processor determines that the pre-write operation log is saved, it sends the operation execution to the host through the interface. Complete notification.
- the network processor saves the pre-write operation log to the log storage area of the storage device, and when it is determined that the pre-write operation log is saved, it sends an operation execution completion notification to the host without waiting for the storage controller of the storage device. Play back the pre-write operation log, write the data to be written into the data storage area, and then send the operation completion notification to the host. Therefore, the delay for the host and the acceleration device to execute the write operation instruction can be shortened; and the process does not need to be accelerated.
- the main processor of the device interacts multiple times with the network processor to execute the write operation instruction, which can further shorten the delay in executing the write operation instruction.
- non-modification operation instruction includes any operation instructions except write operation instructions.
- the main processor in the acceleration device or the storage controller of the storage device receives the non-modification type operation instruction, it can perform the corresponding operation according to the specific type of the non-modification type operation instruction.
- non-modification operation instructions may include read operation instructions.
- the main processor in the acceleration device or the storage controller of the storage device receives the read operation instruction, the data in the storage device may be retrieved from the data according to the address information carried in the read operation instruction. Read the corresponding data from the storage area.
- the log storage area of the storage device can be set with at least one storage space, and each storage space in the at least one storage space corresponds to There is a filter set.
- the network processor may save the pre-write operation log to the first storage space in the log storage area, where the first storage space may be at least one storage space. any storage space in .
- the pre-write operation log includes the first address information carried in the above-mentioned write operation instruction.
- the network processor can update the first filter corresponding to the first storage space. The updated first filter is used to represent the first address information corresponding to The pre-write operation log is stored in the first storage space, waiting to be played back through the storage controller of the storage device.
- the network processor When the network processor saves the pre-write operation log into the first storage space in the log storage area, the first filter corresponding to the first storage space is stored in the acceleration device.
- the network processor receives the read operation instruction sent by the host and obtains the second address information carried in the read operation instruction, where the read operation instruction is used to instruct to obtain data from the data storage area according to the second address information.
- the network processor may determine, based on the first filter, whether the pre-write operation log corresponding to the second address information is stored in the first storage space. If it is determined that the pre-write operation log corresponding to the second address information is stored in the first storage space, The network processor may then send a deferred read notification to the storage controller of the storage device. The delayed read notification is used to instruct the storage controller to wait for the completion of playback of the pre-write operation log corresponding to the second address information before executing the above read operation instruction.
- the log storage area is divided into one or more storage spaces, a filter is set for each storage space, and based on the filter, it is determined whether the pre-write operation log corresponding to the address information carried by the read operation instruction is located in the storage space.
- the space has not yet been replayed. If playback has not yet been performed, the execution of the read operation instruction is delayed, thereby avoiding the problem of being unable to read data or reading incorrect data when the storage controller executes the read operation instruction.
- the first filter corresponding to the first storage space is saved in the acceleration device. If the network processor determines that the first storage space is full, the first filter may be saved to the storage device to save storage space in the acceleration device.
- the network processor determines, based on the first filter, that the pre-write operation log corresponding to the second address information is not saved in the first storage space
- the read operation instruction is sent to the storage device of the storage device.
- the controller so that the storage controller determines whether the pre-write operation log corresponding to the second address information is stored in the log storage area based on the filter in the storage device, waiting for playback by the storage controller of the storage device, so that it can be more effectively Prevents the storage controller from reading data that has not yet been written to the data storage area.
- the pre-write operation log may include the number of the write operation instruction.
- the number of the write operation instructions may be an increasing number according to the time sequence in which the network processor receives each write operation instruction.
- the number of the write operation instruction is saved in each first target location. Compared with setting the element in each first target location to a set value in the related art, the second address information corresponding to the second address information is determined based on the first filter. When the pre-write operation log is stored in the first storage space, the probability of misjudgment can be reduced.
- the network processor receives the data sent by the storage controller of the storage device.
- the data swap request can obtain the third address information and playback progress identifier carried in the data swap request, where the data swap request is used to instruct the data saved in the data storage area to be saved to the acceleration device according to the third address information.
- the playback progress indicator refers to the maximum number contained in the write-ahead operation log that the storage controller has completed playback.
- the network processor may obtain the latest operation identification from the first filter according to the third address information.
- the latest operation identifier refers to the number of the write operation instruction corresponding to the last received third address information before the data swap request is received. If the latest operation identifier is greater than the playback progress identifier, it means that the pre-write operation log corresponding to the third address information is still stored in the first storage space and has not been played back yet, and the data corresponding to the third address information is about to be modified, then the network processor can Ignore the data swap request and do not perform the data swap operation to avoid swapping the old data that is about to be modified into the cache area of the acceleration device. Through the above process, the same address information can be saved in the cache area of the acceleration device. The data is consistent with the data stored in the data storage area of the storage device.
- the network processor can obtain the latest operation identification from the first filter in the following manner: the network processor determines at least one second target location in the first filter based on the third address information, The number of the write operation instruction stored in each second target location in the at least one second target location is obtained respectively, at least one write operation number is obtained, and the smallest write operation number among the at least one write operation number is used as the latest operation identifier.
- the network processor determines that the pre-write operation log corresponding to the third address information is still saved in the first storage space and has not yet been processed. Play back and ignore the data swap request to reduce the number of misjudgments that occur, that is, to reduce the number of prewrite operation logs corresponding to the third address information that are not saved in the first storage space but are misjudged as still being saved in the first storage space. frequency.
- the log storage area of the storage device includes a primary log storage area and a backup log storage area.
- the network processor can save the prewrite operation log to the main log storage area and the backup log storage area in the storage device respectively, thereby improving the accuracy of the stored data. sex and effectiveness.
- inventions of the present application provide a data storage device, which can be installed in a network processor of a data storage system, and an acceleration device can be connected between the host and the storage device.
- the device may include:
- a log generation unit configured to receive a write operation instruction of the host forwarded by an interface in the acceleration device, and generate a pre-write operation log based on the write operation instruction
- the log saving unit is used to save the pre-write operation log to the log storage area in the storage device, wherein the pre-write operation log is used to save the to-be-written data carried in the write operation instruction during playback through the storage controller of the storage device.
- the input data is saved to the data storage area in the storage device.
- the log saving unit may also be configured to send an operation execution completion notification to the host when it is determined that saving the pre-write operation log is completed.
- the log storage area may include at least one storage space, and each storage space in the at least one storage space is provided with a filter; the pre-write operation log includes the write operation instructions.
- the first address information carried; the log storage unit is specifically used for:
- the first filter is stored in the acceleration device, and the device may further include a read instruction execution unit, and the read instruction execution unit is used for:
- a delayed read notification is sent to the storage controller of the storage device, and the delayed read notification is used to instruct the storage controller to wait for the pre-write operation log corresponding to the second address information to be played back before executing the read operation instruction.
- the pre-write operation log includes the number of the write operation instruction; the number of the write operation instruction is a number that increases according to the time sequence of the network processor receiving each write operation instruction; the log saving unit, Specifically, it can be used to: determine at least one first target location in the first filter according to the first address information, and save the number of the write operation instruction to each first target location in the at least one first target location. .
- the device may further include a data swapping unit, and the data swapping unit may be used for:
- the data swap request sent by the storage controller of the storage device, and obtain the third address information and playback progress identifier carried in the data swap request;
- the playback progress identifier indicates that the storage controller has completed the pre-write operation for playback The maximum number contained in the log;
- the data swap request is used to instruct to save the data stored in the data storage area to the cache area of the acceleration device according to the third address information;
- the latest operation identifier is obtained from the first filter; the latest operation identifier refers to the write operation instruction corresponding to the third address information received last time before the data swap request is received. number;
- embodiments of the present application further provide a chip, including a processor and a power supply circuit.
- the power supply circuit is used to supply power to the processor.
- the processor is used to execute a computer program and execute data obtained through an interface. Any method executed by the network processor described in the first aspect.
- embodiments of the present application also provide an acceleration device, the acceleration device is connected between a host and a storage device, and the acceleration device includes an interface and a network processor;
- the interface is used to receive data and provide the received data to the network processor;
- the network processor is configured to execute a computer program to implement any method performed by the network processor described in the first aspect.
- embodiments of the present application further provide a data storage system, including a host, a storage device and an acceleration device; the acceleration device is connected between the host and the storage device, and the acceleration device adopts the acceleration device recorded in the fourth aspect. .
- embodiments of the present application provide a computer-readable storage medium in which computer-executable instructions are stored.
- the computer-executable instructions are used to cause a computer to execute any of the above-mentioned methods provided in the first aspect. method.
- Figure 1 is a schematic structural diagram of a data storage system in related technologies
- Figure 2 is an application scenario diagram of a data storage system provided by an embodiment of the present application
- Figure 3 is a schematic structural diagram of a data storage system provided by an embodiment of the present application.
- Figure 4 is an interactive schematic diagram of a data storage method provided by an embodiment of the present application.
- Figure 5 is an interactive schematic diagram of another data storage method provided by an embodiment of the present application.
- Figure 6 is an interactive schematic diagram of another data storage method provided by an embodiment of the present application.
- Figure 7 is a flow chart of a data storage method provided by an embodiment of the present application.
- Figure 8 is a schematic diagram of a log storage area provided by an embodiment of the present application.
- Figure 9 is a flow chart of another data storage method provided by an embodiment of the present application.
- Figure 10 is a processing flow chart of a read operation instruction provided by an embodiment of the present application.
- Figure 11 is a schematic diagram of a filter provided by an embodiment of the present application.
- Figure 12 is a processing flow chart of a data exchange request provided by an embodiment of the present application.
- FIG13 is a structural block diagram of a data storage device provided in an embodiment of the present application.
- Figure 14 is a structural block diagram of a chip provided by an embodiment of the present application.
- Figure 15 is a structural block diagram of an acceleration device provided by an embodiment of the present application.
- Acceleration device used to offload some functions of the host. For example, data processing functions in the network, storage or operating system that are not suitable for host processing can be offloaded to the acceleration device to release the computing power of the host.
- Acceleration devices may include, but are not limited to, data processing units (DPUs), infrastructure processors (IPUs), system on chips (SoCs), iNICs or smartNICs and other computing devices with offload capabilities. unit. Among them, iNIC or smartNIC can be understood as an intelligent network card.
- NP Network processor
- RDMA remote direct memory access
- File system A software system used to manage and store file information. It is a system that organizes and allocates the space of file storage devices, is responsible for file storage, and protects and retrieves stored files.
- the data storage area of the file system can include a metadata storage area and a data storage area.
- the data storage area is an area responsible for storing files or data in files. In the file data area, files are stored in the form of file data blocks. storage.
- the metadata storage area is an area that supports the file system architecture. It is used to store metadata such as index and validity of files, as well as attribute data of the file system itself. It can also be understood that the data in the file system is divided into data types. Data and metadata types of data.
- Data refers to the actual data in the file, that is, the content of the file; while metadata refers to system data used to describe the characteristics of a file, such as access permissions, file owners, and file data blocks.
- Distribution information etc.
- the distribution information may include index nodes (inode), etc. If you need to operate a file in the file system, you must first obtain its metadata before you can locate the file's location and obtain the file's content or related attributes.
- “plurality” means two or more.
- “plurality” in the embodiments of this application can also be understood as “at least two”.
- “At least one” can be understood as one or more, for example, one, two or more.
- including at least one means including one, two or more, and it does not limit which ones are included.
- A, B and C One then it can include A, B, C, A and B, A and C, B and C, or A and B and C.
- “And/or” describes the relationship between related objects, indicating that there can be three relationships.
- a and/or B can mean: A exists alone, A and B exist simultaneously, and B exists alone.
- the character "/" unless otherwise specified, generally indicates that the related objects are in an "or" relationship.
- ordinal numbers such as “first” and “second” mentioned in the embodiments of this application are used to distinguish multiple objects and are not used to limit the order, timing, priority or importance of multiple objects.
- Data storage systems can be applied to application scenarios such as cloud storage or cloud computing.
- an acceleration device can be used to offload the data access function of the host.
- the data storage system shown in Figure 1 includes a host 100, an acceleration device 200 and a storage device 300.
- the acceleration device 200 is connected between the host 100 and the storage device 300.
- the host 100 can access the storage device 300 through the acceleration device 200.
- the host 100 and the acceleration device 200 can be used as components of a physical machine, and the physical machine and the storage device 300 can communicate remotely through a network.
- the host 100 can be understood as the core processor of the physical machine, and is the computing core and control core of the physical machine.
- Physical machines may include, but are not limited to, physical servers in a cloud computing cluster, computing devices in a computing device cluster, or servers in a network management center.
- the host 100 can receive data input by the user through the client and process the data.
- the storage device 300 in the data storage system can be set outside the host 100 and exchange data with the host 100 through the network.
- the storage device 300 may include a storage controller 310 and a disk enclosure 320.
- the storage controller 310 may be used to manage and control the disk enclosure 320.
- the storage controller 310 and the disk frame 320 can be connected through a bus, or can communicate remotely through a network.
- the acceleration device 200 is used to offload some functions of the host 100 .
- the acceleration device 200 can offload the data access function of the host 100 .
- the acceleration device 200 offloads some functions of the host 100, and the host 100 can be dedicated to management and control functions, thereby improving the performance of the host 100.
- the acceleration device 200 may include a main processor 210 and an application specific integrated circuit (ASIC) chip 220.
- the main processor 210 can be implemented by a central processing unit (Central Processing unit, CPU) or other general-purpose processor.
- the ASIC chip 220 may include a hardware queue 221 and a network engine 222.
- the network engine 222 may be understood as a network interface, and the network engine 222 may be implemented using a remote memory access (rdma over converged ethernet, RoCE) interface.
- the data storage system shown in Figure 1 can be used to process file input and output (IO) operations in a file system.
- the host 100 may send the file operation command to the hardware queue 221 in the acceleration device 200 through the queue channel.
- the main processor 210 of the acceleration device 200 reads the file operation instruction from the hardware queue 221, performs the file IO operation, generates a data access instruction based on the file IO operation, and then stores the data through the network engine 222 in the acceleration device 200.
- the fetch instruction is sent to the storage controller 310, and the storage controller 310 interacts with the disk enclosure 320 to execute the data access instruction. After the execution of the data access instruction is completed, the storage controller 310 returns an access completion message to the acceleration device 200.
- the acceleration device 200 receives the access completion message sent by the storage controller 310 through the network engine 222, and transmits it to the main processing of the acceleration device 200.
- the main processor 210 processes the access completion message, and then notifies the host 100 that the file operation instruction execution is completed through the hardware queue 221 in the ASIC chip 220.
- the processing of file IO operations by the acceleration device 200 is mainly implemented by the main processor 210, which will consume the computing resources of the main processor 210 and affect the performance of the acceleration device 200; and, when the acceleration device 200 There are four interactions between the main processor 210 and the ASIC chip 220, which increases the delay of the file IO operation.
- the data storage system may include multiple physical machines, such as physical machine 410, physical machine 420, physical machine 430, etc.
- Each physical machine can include a host and an acceleration device.
- the acceleration device can communicate with the storage controller and disk enclosure remotely through the network.
- Each physical machine can interact with storage controllers or disk enclosures for data or signaling through acceleration devices.
- Figure 2 takes two storage controllers and two disk frames as an example for illustration, such as storage controller 510, storage controller 520, disk frame 530 and disk frame 540.
- the number of storage controllers and disk enclosures can be more than 2 or less than 2.
- the storage controller and disk enclosures can be located in one storage device, or they can be set up separately and communicate remotely through the network. Storage controllers can be used to manage and control disk enclosures.
- Figure 3 takes an example of interaction between a physical machine and a storage device.
- the physical machine 600 may include a host 610 and an acceleration device 620 connected to the host 610.
- the acceleration device 620 can be directly inserted into a card slot on the motherboard of the host 610 and exchange data or signaling with the host 610 through the PCIe bus.
- the PCIe bus can be replaced by a bus of Compute Express Link (CXL), Universal Serial Bus (Universal Serial Bus, USB) protocol or other protocols.
- CXL Compute Express Link
- USB Universal Serial Bus
- the storage device 700 may include a storage controller 710 and multiple disk enclosures connected to the storage controller 710, such as the disk enclosure 720, the disk enclosure 730, etc.
- the storage controller 710 may be used to manage and control the multiple disk enclosures.
- Figure 3 only takes two disk frames as an example. In actual use, the number of disk frames can be more than 2 or less than 2.
- the disk frame can use a solid state disk (SSD).
- SSD is a hard disk made of an array of solid-state electronic storage chips. It can include a control unit and a storage unit.
- the control unit can receive a network processor or a storage controller. Send instructions to manage and control the storage unit.
- the storage unit can use a flash memory (flash) chip or a dynamic random access memory (DRAM) chip.
- the acceleration device 620 may include a main processor 621 , a network processor 622 and an internal memory 623 .
- the main processor 621 can be a CPU or a programmable logic device (PLD) chip.
- the PLD can be a complex programmable logical device (CPLD) or a field-programmable gate array (field-programmable). gate array (FPGA), general array logic (GAL), or any combination thereof.
- the main processor 621 is used to run the operating system of the acceleration device 620 and software programs run based on the operating system.
- the network processor 622 is used to receive and process file IO operations sent by the host 610.
- the cache area in the internal memory 623 can be used to save the data cache table and the metadata cache table corresponding to the file system.
- the area used to save the data cache table corresponding to the file system can be called a data cache, and is used to save the metadata cache table corresponding to the file system.
- the area of the data cache can be called the metadata cache.
- data items are stored in the data cache, and the data items can be used to store the storage address of the data in the disk frame.
- the metadata cache can save the metadata of the file system.
- the metadata can adopt a tree structure, including multi-level parent node information and child node information.
- the host 610 can send the operation instruction of the file IO operation to the acceleration device 620.
- the acceleration device 620 receives the operation instruction sent by the host 610 through the interface, and forwards the operation instruction to the network processor 622 in the acceleration device.
- the network processor 622 may generate a pre-write operation log based on the write operation instruction.
- the write operation instruction refers to an operation instruction carrying data to be written, which may include but is not limited to an operation instruction for modifying data.
- Write-ahead operation logs can also be called write-ahead logging (WAL).
- WAL write-ahead logging
- the network processor 622 can save the pre-write operation log to a log storage area in the storage device 700.
- the pre-write operation log is used to store the write operation during playback through the storage controller of the storage device 700.
- the data to be written carried in the instruction is saved to the data storage area in the storage device.
- the log storage area and the data storage area may be stored in the disk frame of the storage device 700 .
- the log storage area is used to save write-ahead operation logs, and can also be called the WAL area; the data storage area is used to save file system data, and can also be called the persistent log (PLOG) area.
- the disk frame 720 shown in FIG. 3 is provided with a log storage area 721 and a data storage area 722.
- the disk frame 730 is provided with a log storage area 731 and a data storage area 732.
- the log storage area 721 and the log storage area 731 may be used to save different write-ahead operation logs.
- the log storage area 721 and the log storage area 731 can be active and backup for each other, that is, the log storage area 721 and the log storage area 731 can store the same pre-write operation log.
- the active and standby WAL areas may also be located in the same disk frame.
- the acceleration device processes the received write operation instructions through the network processor, which can save the computing resources of the main processor of the acceleration device, improve the performance of the acceleration device, and eliminate the need for multiple connections between the processor and the network processor. interactions, thereby shortening the latency of performing IO operations.
- the network processor generates a prewrite operation log based on the received write operation instruction, and saves the prewrite operation log to the log storage area in the storage device. These two steps are relatively simple and execute quickly, so they can save time.
- the complex process of replaying the write-ahead operation log is performed by the storage controller, and the playback can be completed in batches in the storage controller without affecting the latency of the IO operation performed by the acceleration device.
- the embodiment of the present application also provides a data storage method that can be applied to the data storage system shown in Figures 2 and 3.
- the operation instructions for file IO operations sent by the host received by the network processor can be divided into two categories according to the operation type: higher execution frequency and higher processor performance and latency requirements
- Operation instructions can be divided into the first type of operation instructions, such as file creation, modification, read IO, write IO, etc.; operation instructions with low execution frequency, or relatively low processor performance and latency requirements can be divided into
- the second type of operation instructions such as rename (rename), count (mount), initialization, etc.
- Different processing methods can be adopted to meet the different needs of different operating instructions.
- the host when it needs to operate the file system, it can send operation instructions for file IO operations to the acceleration device.
- the interface in the acceleration device After receiving the operation instruction sent by the host, the interface in the acceleration device forwards the operation instruction to the network processor in the acceleration device.
- the network processor After receiving the operation instruction sent by the host, the network processor can perform semantic analysis on the operation instruction according to the format, determine the operation type corresponding to the operation instruction, and process the operation instruction according to the operation type corresponding to the operation instruction.
- Figure 4 exemplarily shows an interaction flow chart between various devices when processing second-type operation instructions in a data storage method. As shown in Figure 4, the method may include the following steps:
- the host sends an operation command to the acceleration device.
- the acceleration device determines, through the network processor, that the operation type of the operation instruction is the second type of operation instruction.
- the interface in the acceleration device After receiving the operation instruction sent by the host, the interface in the acceleration device forwards the operation instruction to the network processor in the acceleration device.
- the network processor can determine the operation type corresponding to the operation instruction through semantic analysis. For example, the network processor determines that the operation instruction is an initialization operation instruction through semantic analysis, and thus can determine that the operation instruction is a second type operation instruction.
- the acceleration device sends an operation instruction to the storage controller.
- the network processor of the acceleration device can forward the second type of operation instructions to the storage controller in the storage device, and the storage controller processes them. For example, if the received second type of operation instruction is an initialization operation instruction, the storage controller executes the initialization operation instruction to initialize the disk enclosure indicated by the initialization operation instruction.
- the network processor can forward the second type of operation instructions to the main processor in the acceleration device, and the main processor The processor executes the second type of operating instructions.
- the network processor can forward the received initialization operation instruction to the main processor in the acceleration device for processing.
- the main processor in the acceleration device executes the initialization operation instruction, determines the disk frame that needs to be initialized, and controls the disk frame through the storage controller. The disk enclosure is initialized.
- Figure 5 exemplarily shows an interaction flow chart between various devices when processing read operation instructions in the first type of operation instructions in a data storage method. As shown in Figure 5, the method may include the following steps:
- S501 The host sends an operation command to the acceleration device.
- the acceleration device determines that the operation type of the operation instruction is a read operation instruction through the network processor, and obtains the address information carried in the read operation instruction.
- the interface in the acceleration device After receiving the operation instruction sent by the host, the interface in the acceleration device forwards the operation instruction to the network processor in the acceleration device.
- the network processor determines that the operation type corresponding to the operation instruction is a read operation instruction.
- the read operation instruction indicates to obtain data from the data storage area of the storage device according to the address information carried by the read operation instruction.
- the acceleration device searches the cache area of the acceleration device through the network processor based on the address information, and determines that the cache area does not contain the address information.
- the acceleration device sends the read operation instruction to the storage controller.
- the address information may be the logical address of the data to be read.
- the network processor in the acceleration device After the network processor in the acceleration device obtains the logical address carried in the read operation instruction, it queries whether the cache area of the acceleration device contains the logical address. If it does, it can obtain the data corresponding to the logical address from the cache area; if it does not contain , then the network processor in the acceleration device can forward the read operation instruction to the storage controller in the storage device, and the storage controller processes it.
- the storage controller can read data from the data storage area in the disk enclosure according to the address information carried in the read operation instruction.
- the network processor in the acceleration device can also send the read operation instruction to the main processor in the acceleration device, and the main processor processes it.
- the main processor in the acceleration device can read data from the data storage area in the disk frame according to the address information carried in the read operation instruction.
- read operation instructions can be divided into read data operation instructions and read metadata operation instructions according to the data type of the data to be read.
- the read data operation instruction is used to read data of data type;
- the read metadata operation instruction is used to read data of metadata type.
- the network processor can query whether the data cache in the cache area contains the logical address after obtaining the logical address.
- the network processor can query whether the metadata cache in the cache area contains the logical address after obtaining the logical address.
- non-modification type operation instructions can include any operation instructions except write operation instructions.
- non-modification operation instructions may refer to operation instructions that do not carry data to be written.
- non-modification operation instructions have low latency requirements and consume less computing resources on the processor, they will basically not affect the performance of the acceleration device. Therefore, after the network processor receives the non-modification operation instructions sent by the host, , the non-modification type operation instructions can be sent to the main processor of the acceleration device for processing, or the non-modification type operation instructions can be sent to the storage controller of the storage device for processing.
- Figure 6 exemplarily shows an interaction flow chart between various devices when processing write operation instructions in the first type of operation instructions in a data storage method. As shown in Figure 6, the method may include the following steps:
- the host sends an operation command to the acceleration device.
- the acceleration device determines that the operation type of the operation instruction is a write operation instruction through the network processor, and generates a pre-write operation log according to the address information and data to be written carried in the write operation instruction.
- the interface in the acceleration device After receiving the operation instruction sent by the host, the interface in the acceleration device forwards the operation instruction to the network processor in the acceleration device.
- the network processor determines that the operation type corresponding to the operation instruction is a write operation instruction.
- the write operation instruction instructs to write data or update data to the data storage area of the storage device according to the address information carried by the write operation instruction.
- the address information may be a logical address corresponding to the data to be written.
- the network processor can obtain the logical address and the data to be written carried in the write operation instruction, and generate a prewrite operation log including the logical address and the data to be written.
- write operation instructions can be divided into write data operation instructions and write metadata operation instructions according to the data type of the data to be written.
- the write data operation instruction is used to write data of data type;
- the write metadata operation instruction is used to write data of metadata type.
- the network processor can obtain the logical address carried in the write operation instruction and query whether the logical address is stored in the data cache of the acceleration device. In one embodiment, if the logical address is stored in the data cache, it means that the associated data item of the write data operation instruction is stored in the data cache, and the network processor can clear the associated data items of the write data operation instruction, that is, clear The associated data item corresponding to the logical address does not need to store the data to be written and the logical address carried in the write data operation instruction in the data cache, so as to avoid the data cache being frequently flushed due to a large number of write data operations and reducing resource consumption.
- the network processor can update the associated data item corresponding to the logical address according to the data to be written. If the logical address is not saved in the data cache, the network processor can create a new data item in the data cache to save the data to be written in the write data operation instruction and the logical address corresponding to the data to be written.
- the network processor can obtain the data to be written and the logical address corresponding to the data to be written carried in the write metadata operation instruction, and combine the data to be written and the data to be written.
- the logical address corresponding to the data is added to the metadata cache of the acceleration device, where the data to be written is metadata type data.
- the acceleration device can save the contents of the metadata cache to the storage area in the storage controller used to save file system metadata.
- S604 The acceleration device sends an operation execution completion notification to the host.
- the acceleration device can save the generated prewrite operation log to the log storage area in the disk enclosure through the network processor, and after determining that the prewrite operation log is saved, sends an operation execution completion notification to the host.
- the network processor can write the prewrite operation log into the disk frame through the RDMA channel.
- the RDMA channel is implemented based on the RDMA protocol.
- the network processor transmits the prewrite operation log to the disk chassis.
- the disk chassis saves the received prewrite operation log to the log storage area, and after the saving is completed, sends it to the network.
- the processor returns an ACK message.
- the ACK message is a confirmation message and is used to notify the network processor that the prewrite operation log has been written.
- the network processor receives the ACK message fed back by the disk frame, determines that the pre-write operation log has been saved, and then sends an operation execution completion notification to the host.
- the prewrite operation log is used to save the data to be written carried in the write operation instruction to the data storage area in the disk enclosure during playback through the storage controller.
- the data storage area may include a metadata area and a file data area. If the data to be written is metadata type data, it can be written to the metadata area; if the data to be written is data type data, it can be written to the file data area.
- the above-mentioned process of generating the pre-write operation log based on the write operation command and writing the pre-write operation log to the disk frame of the storage device is executed by the network processor of the acceleration device. After the network processor completes the execution, it directly returns the operation execution to the host. Complete notification.
- This process does not require multiple interactions between the network processor of the acceleration device and the processor, which can reduce the number of interactions between the network processor and the processor, effectively reduce the delay in executing operation instructions, achieve a fast access path to IO, and improve The overall performance of the data storage system.
- the log storage area can be called the WAL area.
- a primary WAL area and a backup WAL area may be provided in the disk frame, and prewrite operation logs may be saved to both the primary WAL area and the backup WAL area.
- the primary WAL area and the backup WAL area can be located in the same disk frame or in different disk frames.
- S605 The storage controller sends a read log notification to the disk enclosure.
- S606 The disk enclosure sends the prewrite operation log to the storage controller.
- the storage controller When the storage controller performs log playback, it can obtain the prewrite operation log from the disk enclosure.
- the storage controller sends a read log notification to the disk enclosure, and the disk enclosure sends a prewrite operation log to the storage controller based on the received read log notification.
- the storage controller can specify to read the prewrite operation log from the primary WAL area or the standby WAL area.
- the storage controller can clear the log stored in the primary WAL area and the standby WAL area at the same time.
- Write ahead operation log For example, if the storage controller specifies to read the write-ahead operation log from the primary WAL area, and the log saved in the primary WAL area is incorrect and cannot be read normally, the write-ahead operation log can be read from the backup WAL area.
- S607 The storage controller plays back the received write-ahead operation log.
- the pre-write operation log includes the data to be written carried by the write operation instruction and the logical address corresponding to the data to be written.
- the storage controller can save the to-be-written data carried by the write operation instruction to the data storage area in the disk enclosure according to the logical address carried by the write operation instruction. For example, the storage controller determines the physical address corresponding to the logical address in the data storage area based on the logical address carried by the write operation instruction, and writes the data to be written to the data storage area based on the determined physical address.
- the data storage method executed by the acceleration device may include the following steps as shown in Figure 7:
- S701 The interface in the acceleration device receives the write operation instruction sent by the host, and forwards the write operation instruction to the network processor in the acceleration device.
- the network processor in the acceleration device generates a pre-write operation log based on the write operation instruction.
- the pre-write operation log includes the data to be written carried by the write operation instruction and the address information corresponding to the data to be written.
- S703 The network processor in the acceleration device saves the prewrite operation log to the log storage area in the storage device.
- the pre-write operation log is used to save the data to be written carried in the write operation instruction to the data storage area in the storage device during playback through the storage controller of the storage device.
- the network processor determines that the write-ahead operation log is saved, it sends an operation execution completion notification to the host.
- the computing resources of the main processor of the acceleration device can be saved and the performance of the acceleration device can be improved.
- the network processor generates a prewrite operation log based on the received write operation instruction, and saves the prewrite operation log to the log storage area in the storage device. These two steps are relatively simple and execute quickly, so they can save time.
- the complex process of replaying the write-ahead operation log is performed by the storage controller, and the playback can be completed in batches in the storage controller without increasing the delay in performing IO operations on the acceleration device.
- the interface in the acceleration device receives the read operation instruction sent by the host, and can send the read operation instruction to the storage controller through the network processor in the acceleration device, and the storage controller performs the read operation instruction according to the address carried in the read operation instruction.
- the information reads the required data from the data storage area of the disk enclosure.
- the address information carried in the read operation instruction can be called read operation address information.
- the storage controller executes the read operation instruction, the data to be written corresponding to the read operation address information may still be stored in the pre-write operation log, that is, the pre-write operation log is still in the log storage area and has not yet been played back. For example, at the first moment, the host sends a first operation instruction to the acceleration device.
- the first operation instruction is a write operation instruction.
- the write operation instruction instructs to write the first data to address a.
- the network processor receives the first operation instruction. Based on The first operation instruction generates a first pre-write operation log, and writes the first pre-write operation log into the log storage area in the disk frame.
- the first pre-write operation log includes first data and an address a corresponding to the first data.
- the host sends a second operation instruction to the acceleration device.
- the second operation instruction is a read operation instruction.
- the read operation instruction instructs to read the data in address a. Since the first pre-write operation corresponding to address a The log is still in the log storage area and has not yet been played back. The first data corresponding to address a has not yet been written to the data storage area. If the storage controller executes a read operation command at this time, the data may not be read, or incorrect data may be read.
- the embodiment of the present application divides the log storage area in the disk frame into at least one storage space.
- the storage space is used to save the data generated based on the write operation instruction.
- Write-ahead operation logs each storage space can save one or more write-ahead operation logs, and each storage space can be called a data file space (segment).
- a filter is set for each storage space, and the filter is used to determine whether a certain address information is included in the write-ahead operation log saved in the corresponding storage space and has not yet been played back.
- the log storage area 810 in the disk enclosure 800 may include multiple storage spaces. Only three storage spaces are shown in FIG.
- the first filter is a filter corresponding to the storage space 811
- the second filter is a filter corresponding to the storage space 812
- the third filter is a filter corresponding to the storage space 813 .
- the filter can be a bloom filter (bloom filter), a quotient filter or other filters.
- the process of the network processor in the acceleration device writing the prewrite operation log to the disk frame may include the following steps:
- S901 Generate a pre-write operation log based on the received write operation instruction.
- the network processor can generate a pre-write operation log according to the data to be written and the first address information carried in the write operation instruction.
- the pre-write operation log includes the first address information carried in the write operation instruction, where the first address information may be a logical address corresponding to the data to be written.
- the current storage space may be any storage space among multiple storage spaces.
- the network processor writes the prewrite operation log to each storage space in the log storage area in turn. For example, the network processor first writes the prewrite operation log to the storage space 811 shown in Figure 8. At this time, the network processor writes the prewrite operation log to each storage space in the log storage area. Space 811 is the current storage space. If the storage space 811 is full after writing 30 write-ahead operation logs, then After generating the 31st write-ahead operation log, the network processor can write the 31st write-ahead operation log into the storage space 812. At this time, the storage space 812 is the current storage space. After the storage space 812 is full, the newly generated prewrite operation log can be written into the storage space 813. At this time, the storage space 813 is the current storage space.
- the updated current filter is used to represent that the pre-write operation log corresponding to the first address information is stored in the current storage space, waiting for playback through the storage controller.
- the network processor can process the first address information through a set set of hash functions to determine a set of hash values.
- the hash value is used to indicate the corresponding position of the first address information in the filter. , you can update the element at the corresponding position in the filter.
- the number of hash values obtained is consistent with the number of hash functions. If there are three hash functions, after processing the first address information, three hash values can be obtained. Based on the three hash values, the three corresponding positions of the first address information in the filter are determined. For example, assuming that the filter includes a total of 64 elements from 0 to 63, and each element occupies 1 bit, it can also be understood that the filter includes 64 positions. In the initial state, the elements at the 64 positions in the filter are all 0.
- the first address information is processed and the three hash values obtained are 8, 15, and 19 respectively, which means that the first address information corresponds to position 8, position 15, and position 19 respectively in the filter.
- Position 8 position The elements at position 15 and 19 are both set to 1.
- the elements at position 8, position 15 and position 19 are all 1, it means that the pre-write operation log corresponding to the first address information is stored in the current storage space.
- the to-be-written operation log corresponding to the first address information is The data has not yet been written to the data storage area.
- the current filter corresponding to the current storage space can be saved in the internal memory of the acceleration device.
- the network processor saves the contents of the current filter to the disk frame. For example, when the disk enclosure detects that the current storage space is full, it can send a storage space full message to the network processor. The network processor receives the storage space full message sent by the disk enclosure and determines that the current storage space is full. Save the contents of the current filter to the disk frame, and the disk frame stores the current filter in correspondence with the corresponding storage space. For example, storage space 811 and storage space 812 in Figure 8 are already full, the first filter corresponding to storage space 811 and the second filter corresponding to storage space 812 have been saved in the disk frame, but storage space 813 has not yet been filled.
- Storage space 813 is the current storage space.
- the third filter corresponding to storage space 813 is the current filter. The content of the third filter is still there. It is stored in the internal memory of the acceleration device but not in the disk frame, so the third filter in the disk frame is represented by a dotted box.
- the storage controller can play back the prewrite operation logs saved in each storage space in turn.
- the storage controller You can control the disk enclosure to clear the storage space, clear the content stored in the filter corresponding to the storage space, and reset the elements at all positions in the filter corresponding to the storage space to 0.
- the process of accelerating the processing of read operation instructions sent by the host by the network processor in the device may include the following steps:
- the network processor receives the operation instruction sent by the host and can determine the operation type corresponding to the operation instruction through semantic analysis. If it is determined that the received operation instruction is a read operation instruction, the second address information carried in the read operation instruction can be obtained.
- the read operation instruction is used to instruct to obtain data from the data storage area according to the second address information.
- the second address information may be the logical address of the data to be read.
- S1003 Read data corresponding to the second address information from the cache area, and send the read data to the host.
- the network processor searches the cache area of the acceleration device according to the second address information and determines whether the cache area contains the second address information. If the cache area of the acceleration device contains the second address information, the data corresponding to the second address information can be obtained from the cache area. and sent to the host.
- the network processor can determine the pre-written data corresponding to the second address information based on the current filter saved in the acceleration device. Whether the operation log is saved in the current storage space.
- the network processor can process the second address information through a set of hash functions to obtain a set of hash values, That is, the hash value corresponding to the second address information.
- the network processor may determine whether the element at the position indicated by the hash value corresponding to the second address information is 1 in the current filter.
- the same hash value may be obtained.
- the prewrite operation log corresponding to address b is saved in the current storage space, and the prewrite operation log corresponding to address c is not saved in the current storage space. Since a hash function is used to process address b and address c, the same result is obtained.
- the hash value of the element at the position corresponding to the hash value is 1. Therefore, based on the hash value, it can be concluded that the prewrite operation log corresponding to address c is stored in the current storage space, thus causing misjudgment.
- the address can be determined.
- the write-ahead operation log corresponding to the information is saved in the current storage space. If at least one element among the elements at the positions indicated by multiple hash values is not 1, it can be determined that the prewrite operation log corresponding to the address information is not saved in the current storage space.
- the second address information is processed and the obtained hash values include 3. If the elements at the positions indicated by the 3 hash values corresponding to the second address information are all 1, then the pre-write operation corresponding to the second address information can be determined. The log is saved in the current storage space. If one or more elements in the positions indicated by the three hash values corresponding to the second address information are not 1, it can be determined that the prewrite operation log corresponding to the second address information is not saved in the current storage space.
- the network processor determines that the pre-write operation log corresponding to the second address information is not saved in the current storage space, it can send a read operation instruction to the storage controller of the storage device, so that the storage controller is based on the filter saved in the disk frame. , determine whether the pre-write operation log corresponding to the second address information is stored in the log storage area in the disk frame, waiting to be played back through the storage controller.
- the current filter saved in the acceleration device is the third filter corresponding to storage space 813 in Figure 8
- storage space 813 is the current storage space.
- the network processor determines that the pre-write operation log corresponding to the second address information is not saved in the storage space 813, it can send the read operation instruction to the storage controller, and the storage controller can obtain the filter saved in the disk frame, as shown in Figure 8 First filter and second filter shown.
- the storage controller may determine whether the pre-write operation log corresponding to the second address information is stored in the storage space 811 based on the first filter, and may determine whether the pre-write operation log corresponding to the second address information is stored in the storage space 811 based on the second filter. Saved in storage space 812.
- the storage controller can execute the read operation instruction and read from the data storage area based on the second address information. data. Otherwise, the storage controller can wait for the completion of playback of the pre-write operation log corresponding to the second address information before executing the read operation instruction.
- the delayed read notification may include the above read operation instructions.
- the delayed read notification is used to instruct the storage controller to wait for the completion of playback of the pre-write operation log corresponding to the second address information before executing the read operation instruction.
- the delayed read notification may be a read operation instruction that adds a delayed read flag.
- the network processor determines that the pre-write operation log corresponding to the second address information is stored in the current storage space and has not been played back, indicating that the read operation instruction conflicts with a certain write operation instruction before the read operation instruction, or Say, there is a dependency. At this time, the network processor can add a delayed read flag to the read operation instruction, and send the read operation instruction with the delayed read flag added to the storage controller.
- the storage controller can wait for the pre-write operation log playback corresponding to the second address information to be completed before executing the read operation instruction.
- each operation instruction received by the network processor has a unique number.
- the number of each operation instruction may be a number that increases according to the time sequence in which the network processor receives each operation instruction, or is called a sequence number. (sequence).
- the pre-write operation log generated based on the write operation instruction may include the number of the corresponding write operation instruction.
- the delayed read notification sent by the network processor received by the storage controller includes the read operation instruction and the number of the read operation instruction. When the storage controller performs log playback, it plays back the logs in sequence according to the time sequence of each write operation instruction.
- the storage controller may determine the number of the write operation instruction included in the write-ahead operation log being played back, and the number may be called a playback progress identifier.
- the storage controller compares the playback progress indicator with the number of the read operation instruction in the delayed read notification. If the playback progress indicator is smaller than the number of the read operation instruction, it means that the prewrite operation log corresponding to the second address information may not have been played back yet. , the storage controller continues to wait. If the playback progress indicator is greater than the number of the read operation instruction, it means that the pre-write operation logs corresponding to all write operation instructions before the read operation instruction have been played back, and the pre-write operation log corresponding to the second address information must have been played back, then the storage controller The read operation instruction can be executed to read data from the data storage area based on the second address information.
- the length of the elements at each position in the filter can be increased, for example, the length of each element can be increased from 1 bit to 4 bytes (32bit) or 8 bytes (64bit).
- Each operation instruction received by the network processor has a unique number, and the pre-write operation log generated based on the write operation instruction may include the number of the corresponding write operation instruction.
- each filter The length of the elements at each position can be determined according to actual needs. The longer the length of the element, the greater the maximum number it can store. The maximum number should be greater than or equal to the number of operation instructions sent by the host to the network processor within the set cycle duration. At the beginning of the next cycle, the number of operation instructions sent by the host to the network processor starts from 0 and is renumbered.
- the first target position can be determined in the current filter based on the first address information, and the number of the write operation instruction can be saved in the current filter. the first target position.
- a set of hash functions can be used to process the first address information to obtain one or more hash values, and one or more first targets are determined in the current filter based on the one or more hash values. location, and save the number of the write operation instruction corresponding to the first address information to each first target location. Saving the number of the write operation instruction corresponding to the first address information to the first target location can significantly reduce the probability of misjudgment compared with setting the element in the first target location to 1.
- the number S11 of the write operation instruction can be saved to position 8, position 15 and position 19 respectively.
- the number of other instructions may be saved elsewhere in the current filter, or the number of any instruction may not be saved. If a location does not hold the number of any instruction, the element corresponding to that location can have an initial value of 0.
- the number S11 of the write operation instruction is stored in multiple locations to avoid misjudgment. This has been explained above and will not be repeated here.
- the network processor can obtain the number of the write operation instruction corresponding to the second address information from the current filter based on the second address information.
- the number Can be called a conflicting operation number. This conflicting operation number can be included in the deferred execution notification sent to the storage controller.
- the storage controller When the storage controller performs log playback, it can determine the number of the write operation instruction contained in the prewrite operation log being played back. This number can be called the playback progress indicator. Each time the playback of a pre-write operation log is completed, the storage controller compares the playback progress identifier with the conflict operation identifier. If the playback progress identifier and the conflict operation identifier are the same, it means that the pre-write operation log corresponding to the second address information has been played back. , the storage controller can execute the read operation instruction and read data from the data storage area based on the second address information. If the playback progress identifier is different from the conflict operation identifier, it means that the prewrite operation log corresponding to the second address information has not been played back yet, and the storage controller continues to wait.
- the log storage area is divided into one or more storage spaces, a filter is set for each storage space, and based on the filter, it is determined whether the pre-write operation log corresponding to the address information carried by the read operation instruction is located in the storage space.
- the space has not yet been played back, which can avoid the problem of being unable to read data or reading incorrect data when the storage controller executes read operation instructions.
- the storage controller can swap hotspot data frequently read by the host into the cache area in the acceleration device.
- the storage controller can identify hotspot data. When the hotspot data is identified, it can read the hotspot data from the data storage area and write the hotspot data to the cache area in the acceleration device. Alternatively, it can obtain the hotspot data from the disk enclosure and use it in the data storage area. Save the address of the hotspot data and write the address of the hotspot data into the cache area in the acceleration device.
- the storage controller When the storage controller performs data swapping, the following problems may occur: In some embodiments, when the storage controller determines that data A is hot data and obtains data A from the data storage area, the network processor in the acceleration device The prewrite operation log corresponding to the write operation instruction that modifies data A may have been saved to the log storage area, and data A is about to be modified. At this time, the storage controller cannot know that data A is about to be modified, and still writes data A to the cache area in the acceleration device, which will cause the unmodified old data to be replaced into the cache area in the acceleration device.
- the network processor may have already modified the pre-write operation corresponding to the write operation instruction of data A.
- the log is saved to the log storage area.
- the storage controller cannot know that data A is about to be modified, and will still write the address ADD1 that saves data A into the cache area in the acceleration device.
- a new address ADD2 may be configured for the modified data A, and the modified data A may be saved to the new address ADD2, causing the host or acceleration device to read data according to the address ADD1 saved in the data cache. What is received is the unmodified old data, which is equivalent to swapping the unmodified old data into the cache area of the acceleration device.
- each write operation instruction received by the network processor in the acceleration device has a unique number, and the number of each operation instruction is The number may be a number that increases according to the time sequence in which the network processor receives each write operation instruction.
- the pre-write operation log generated based on the write operation instruction may include the number of the corresponding write operation instruction.
- the network processor in the acceleration device can perform the following steps as shown in Figure 12:
- the storage controller may send a data swap request to the network processor of the acceleration device.
- the data swap request may include the third address information corresponding to data A.
- the data swap request may include data A and third address information corresponding to data A.
- the data swap request is used to instruct the network processor to save the data stored in the data storage area to the cache area of the acceleration device according to the third address information.
- the third address information may be a logical address corresponding to data A, and the logical address may be determined based on the physical address of data A used for saving in the data storage area.
- the data swap request can also include a playback progress identifier.
- the playback progress identifier refers to the maximum number S_MAX contained in the prewrite operation log that the storage controller has completed playback. That is, the last prewrite operation log that the storage controller has completed playback contains. The number of the write operation instruction.
- S1202 Obtain the latest operation identifier from the current filter according to the third address information.
- the latest operation identifier refers to the number of the write operation instruction corresponding to the last received third address information before the data swap request is received.
- the network processor can process the third address information through a set of hash functions to obtain at least one hash value, that is, the hash value corresponding to the third address information.
- the network processor may respectively obtain the number of the write operation instruction stored in at least one second target location indicated by at least one hash value corresponding to the third address information from the current filter, and obtain one or more latest operation identifiers.
- the network processor can determine whether to execute the data swap request based on the playback progress identifier and the latest operation identifier.
- a latest operation identifier when a latest operation identifier is obtained in step S1202, if the latest operation identifier is greater than the playback progress identifier S_MAX carried in the data swap request, it means that the pre-write operation log corresponding to the third address information is still stored in The storage space corresponding to the current filter has not yet been played back or is being played back.
- the data A corresponding to the third address information is about to be modified or is being modified.
- the network processor ignores the data swap request and does not perform data swap. Avoid swapping old data or parts of old data into the cache area of the acceleration device.
- step S1202 When multiple latest operation identifiers are obtained in step S1202, if each latest operation identifier is greater than the playback progress identifier S_MAX carried in the data swap request, it can be determined that the latest operation identifier corresponding to the third address information is greater than the playback progress identifier S_MAX, It means that the data A corresponding to the third address information is about to be modified or is being modified.
- the network processor can ignore the data swap request and not perform data swap.
- the network processor can perform the data swap operation and save the data A saved in the data storage area to the cache area of the acceleration device according to the third address information.
- step S1202 if the network processor obtains multiple hash values corresponding to the third address information, multiple second target locations can be determined in the current filter based on the multiple hash values.
- the network processor may separately obtain the number of the write operation instruction stored in each second target location among the plurality of second target locations, obtain multiple write operation numbers, and use the smallest write operation number among the multiple write operation numbers as the latest Operation ID. If the latest operation identifier is less than the playback progress identifier, the data swap request is executed. If the latest operation identifier is greater than the playback progress identifier, the data swap request is ignored and no data swap is performed.
- the network processor uses the monotonically increasing operation identifier and compares the latest operation identifier with the playback progress identifier to determine whether a certain address information contains the latest pre-write operation log that has not been played back. If there is a pre-write operation log that has not yet been played back, If the latest write-ahead operation log is used, the data swap request corresponding to the address information will not be executed to avoid swapping the old data that will be modified into the cache area of the acceleration device. For the same address information, the cache area of the acceleration device can be The saved data is consistent with the data saved in the data storage area of the storage device.
- embodiments of the present application also provide a data storage device.
- the data storage device can be set in the network processor of the acceleration device, and the acceleration device can be connected between the host and the storage device.
- the data storage device 1300 may include a log generating unit 1301 and a log saving unit 1302 .
- the data storage device 1300 can be used to implement the functions of the above method embodiments, and therefore can achieve the beneficial effects of the above method embodiments.
- the log generation unit 1301 can be used to receive the write operation instruction of the host forwarded by the interface in the acceleration device, and generate a pre-write operation log based on the write operation instruction; the log saving unit 1302 can be used to save the pre-write operation log to The log storage area in the storage device; the pre-write operation log is used to save the data to be written carried in the write operation instruction to the data storage area in the storage device during playback through the storage controller of the storage device.
- the log saving unit 1302 may also be configured to: if it is determined that the pre-write operation log is saved, sending an operation execution completion notification to the host.
- the log saving unit 1302 may be specifically configured to: save the pre-write operation log to the first storage space in the log storage area; the pre-write operation log includes the first address information carried in the write operation instruction;
- the first storage space is any storage space in at least one storage space; according to the first address information in the pre-write operation log, the first filter corresponding to the first storage space is updated; the updated first filter is used to characterize the first
- the pre-write operation log corresponding to an address information is stored in the first storage space, waiting to be played back through the storage controller of the storage device.
- the first filter is stored in the acceleration device.
- the data storage device 1300 may also include a read instruction execution unit, and the read instruction execution unit is connected to the log saving unit 1302 .
- the read instruction execution unit may be used to: receive a read operation instruction sent by the host, and obtain the second address information carried in the read operation instruction; the read operation instruction is used to instruct to obtain data from the data storage area according to the second address information; if based on the first A filter determines that the pre-write operation log corresponding to the second address information is stored in the first storage space, and then sends a delayed read notification to the storage controller of the storage device.
- the delayed read notification is used to instruct the storage controller to wait for the second address information. After the corresponding write-ahead operation log playback is completed, the read operation command is executed.
- the log saving unit 1302 may also be used to: if it is determined that the first storage space is full, save the first filter to the storage device.
- the read instruction execution unit may also be configured to: if it is determined based on the first filter that the pre-write operation log corresponding to the second address information is not saved in the first storage space, send the read operation instruction to The storage controller of the storage device determines whether the pre-write operation log corresponding to the second address information is stored in the log storage area based on the filter in the storage device, waiting to be played back by the storage controller of the storage device.
- the pre-write operation log includes the number of the write operation instruction; the log saving unit 1302 may be specifically configured to: determine at least one first target in the first filter according to the first address information. location, and respectively save the number of the write operation instruction to each first target location in the at least one first target location.
- the number of the write operation instructions is a number that increases according to the time sequence in which the network processor receives each write operation instruction.
- the data storage device 1300 may also include a data swapping unit, and the data swapping unit is connected to the log saving unit.
- the data swap-in unit may be used to: receive a data swap-in request sent by the storage controller of the storage device, and obtain the third address information and playback progress identifier carried in the data swap-in request; the playback progress identifier indicates that the storage controller has completed playback The maximum number contained in the prewrite operation log; the data swap request is used to instruct to save the data saved in the data storage area to the cache area of the acceleration device according to the third address information; according to the third address information, from the first filter Get the latest operation identifier; the latest operation identifier refers to the number of the write operation instruction corresponding to the last received third address information before receiving the data swap request; if the latest operation identifier is greater than the playback progress identifier, the data swap is ignored ask.
- the data swapping unit may be specifically configured to: determine at least one second target location in the first filter according to the third address information; obtain each second target location in the at least one second target location respectively.
- the number of write operation instructions stored in the target location is used to obtain at least one write operation number; among the at least one write operation number, the smallest write operation number is used as the latest operation identification.
- the log saving unit 1302 may be configured to: save the prewrite operation log to the main log storage area and the backup log storage area in the storage device.
- the embodiment of the present application also provides a chip, which may be a computing chip, and the chip may be applied to the network processor of the above embodiment.
- the chip can be used to implement the functions of the method embodiment shown in Figure 7, and therefore can achieve the beneficial effects of the above method embodiment.
- the structure of the chip 1400 can be as shown in Figure 14, including a processor 1401 and a power supply circuit 1402 connected to the processor 1401.
- the processor 1401 and the power supply circuit 1402 can be connected to each other through a bus.
- the processor 1401 can be a digital signal processor (DSP), ASIC, field programmable gate array (field programmable gate array, FPGA) or other Programmed logic devices, discrete gate or transistor logic devices, discrete hardware components, or other specific integrated circuits wait.
- the bus may be a peripheral component interconnect (PCI) bus or an extended industry standard architecture (EISA) bus.
- PCI peripheral component interconnect
- EISA extended industry standard architecture
- the bus can be divided into address bus, data bus, control bus, etc.
- the power supply circuit 1402 is used to power the processor 1401 through the bus.
- the processor 1401 can be connected to a memory provided outside the chip, or connected to a memory provided inside the chip, and run software programs and modules stored in the memory to execute various functional applications and data processing of the chip 1400, as described in this application.
- the data storage method provided by the embodiment.
- the processor 1401 may include one or more processing units, and different processing units may be independent devices or integrated into one or more processors.
- the processor 1401 may also include a controller, which may generate operation control signals based on instruction operation codes and timing signals to complete the control of fetching and executing instructions.
- the embodiment of the present application also provides an acceleration device, which can be connected between the host and the storage device.
- the acceleration device can be used to implement the functions of the above method embodiments, and therefore can achieve the beneficial effects of the above method embodiments.
- the structure of the acceleration device 1500 can be as shown in Figure 15, including an interface 1501 and a network processor 1502 connected to the interface 1501.
- the interface 1501 and the network processor 1502 can be connected to each other through a bus.
- the interface 1501 can be a bus interface, a data line interface or other communication interfaces, used to communicate with the host, receive data sent by the host, and provide the received data to Network processor 1502.
- the network processor 1502 may be an NP that supports a programmable architecture and has remote direct data access capabilities.
- the network processor 1502 may adopt the chip shown in FIG. 14 .
- the bus can be a PCI bus or an EISA bus, etc.
- the bus can be divided into address bus, data bus, control bus, etc.
- the network processor 1502 may be connected to a memory provided outside the acceleration device, or connected to a memory provided inside the acceleration device, run software programs and modules stored in the memory, and interact with the host and the storage device. Thus, the data storage method provided by the embodiment of this application is executed.
- the network processor 1502 may include one or more processing units, and different processing units may be independent devices or integrated into one or more processors.
- the network processor 1502 may also include a controller.
- the controller may generate operation control signals according to the instruction operation code and timing signals to complete the control of fetching and executing instructions.
- the structure illustrated in the embodiment of the present application does not constitute a specific limitation on the acceleration device.
- the acceleration device may include more or less components than shown in the figures, or some components may be combined, or some components may be separated, or may be arranged differently.
- the components illustrated may be implemented in hardware, software, or a combination of software and hardware.
- the embodiment of the present application also provides a data storage system.
- the structure of the data storage system can be shown in Figure 3 and includes a host 610, an acceleration device 620 and a storage device 700.
- the acceleration device 620 is connected between the host 610 and the storage device 700.
- the acceleration device 620 may include a network processor 622, and the network processor 622 may be used to execute computer programs to implement the functions of the above method embodiments.
- the method steps in the embodiments of the present application can be implemented by hardware, or by a processor executing computer programs or instructions.
- a computer program or instructions may constitute a computer program product.
- An embodiment of the present application also provides a computer program product including computer-executable instructions.
- the computer-executable instructions are used to cause the computer to perform the functions in the above method embodiment.
- Computer-executable instructions may be stored in a computer-readable storage medium.
- Embodiments of the present application further provide a computer-readable storage medium in which executable instructions are stored.
- the computer-executable instructions are used to cause the computer to perform the functions in the above method embodiment.
- the computer-readable storage medium can be random access memory (random access memory, RAM), flash memory, read-only memory (read-only memory, ROM), programmable read-only memory (programmableROM, PROM), Erasable programmable read-only memory (erasable PROM, EPROM), electrically erasable programmable read-only memory (electrically ePROM, EEPROM), register, hard disk, removable hard disk, CD-ROM or any other form known in the art Computer-readable storage media.
- RAM random access memory
- ROM read-only memory
- programmable read-only memory programmable read-only memory
- PROM Erasable programmable read-only memory
- EPROM Erasable programmable read-only memory
- electrically erasable programmable read-only memory electrically erasable programmable read-only memory (electrically ePROM, EEPROM), register, hard disk, removable hard disk, CD-ROM or any other form known in the art Computer-readable storage media.
- Computer-executable instructions may be stored in or transmitted from one computer-readable storage medium to another, for example, the computer program or instructions may be transmitted from a website, computer, server, or A data center transmits data via wired or wireless means to another website site, computer, server, or data center.
- the computer-readable storage medium may be any available medium that can be accessed by a computer or a data storage device such as a server or data center that integrates one or more available media.
- the available media may be magnetic media, such as floppy disks, hard disks, and magnetic tapes; they may also be optical media, such as digital video Digital video disc (DVD); it can also be a semiconductor medium, such as a solid state drive.
- the processor can be used to execute the program instructions and implement the above method flow.
- the processor may include but is not limited to at least one of the following: CPU, microprocessor, digital signal processor (digital signal processor, DSP), microcontroller unit (microcontroller unit, MCU), or artificial intelligence processor, etc.
- a computing device that runs software, each computing device may include one or more cores for executing software instructions to perform operations or processing.
- the processor can be built into an SoC, DPU or ASIC, or it can be an independent semiconductor chip.
- the processor may further include necessary hardware accelerators, such as FPGA, PLD, or logic circuits that implement dedicated logic operations.
- the hardware can be a CPU, microprocessor, DSP, MCU, artificial intelligence processor, ASIC, SoC, FPGA, PLD, dedicated digital circuit, hardware accelerator or non-integrated discrete device Any one or any combination thereof, which can run necessary software or not rely on software to perform the above method process.
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Software Systems (AREA)
- Human Computer Interaction (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
本申请公开了一种数据存储方法、装置、系统、芯片及加速设备,涉及计算机技术领域。本申请的加速设备通过网络处理器处理主机发送的写操作指令,可以节约加速设备的主处理器的计算资源,提高加速设备的性能。网络处理器基于接收到的写操作指令生成预写操作日志,将预写操作日志保存至存储设备中的日志存储区域,上述两个步骤执行速度较快,因此可以减少加速设备执行写操作指令的时间,提高加速设备对写操作指令的执行效率。
Description
相关申请的交叉引用
本申请要求在2022年09月14日提交中华人民共和国知识产权局、申请号为202211115045.4、发明名称为“数据存储方法、装置、系统、网络处理器及加速设备”的中国专利申请的优先权,其全部内容通过引用结合在本申请中。
本申请涉及计算机技术领域,尤其涉及一种数据存储方法、装置、系统、芯片及加速设备。
随着计算机技术和网络技术的发展,计算设备可提供的服务越来越多,对计算设备的主机(host)的算力需要也越来越高。为了节约计算设备的主机的算力,提升计算设备的性能,可以为主机配置加速设备,将部分数据处理功能从主机卸载至加速设备上。例如,在数据存储系统中,可以将数据存取功能从主机卸载到加速设备上,主机可以通过加速设备从存储设备存取数据。
在数据存取过程中,主机向加速设备发送的各类数据存取指令均由加速设备的主处理器执行,消耗加速设备的主处理器的计算资源,影响加速设备的性能。
发明内容
本申请实施例提供一种数据存储方法、装置、系统、芯片及加速设备,可以节约加速设备的主处理器的计算资源,并提高写操作指令的执行效率。
第一方面,本申请实施例提供一种数据存储方法,该方法可以应用于加速设备,加速设备连接在主机和存储设备之间,加速设备可以包括接口和网络处理器。该方法可以包括如下步骤:加速设备中的接口接收主机发送的写操作指令,把写操作指令转发给所述加速设备中的网络处理器;网络处理器基于写操作指令生成预写操作日志,并将预写操作日志保存至存储设备中的日志存储区域。其中,预写操作日志用于在通过存储设备的存储控制器进行回放时,将上述写操作指令中携带的待写入数据保存至存储设备中的数据存储区域。
上述数据存储方法中,加速设备中的接口接收到主机发送的写操作指令后,由加速设备中的网络处理器基于接收到的写操作指令生成预写操作日志,将预写操作日志保存至存储设备中的日志存储区域,上述两个步骤执行速度较快,加速设备无需对写操作指令进行处理,将待写入数据写入存储设备的数据存储区域,也无需等待存储设备的存储控制器将待写入数据写入数据存储区域,因此可以减少加速设备执行写操作指令的时间,提高加速设备对写操作指令的执行效率;并且,通过加速设备中的网络处理器处理写操作指令,无需加速设备中的主处理器对写操作指令进行处理,可以节约加速设备中的主处理器的计算资源,提高加速设备的性能。
在一种可能的实施方式中,网络处理器将预写操作日志保存至存储设备中的日志存储区域,若网络处理器确定该预写操作日志保存完成,则通过所述接口向主机发送操作执行完成通知。
上述实施方式中,网络处理器将预写操作日志保存至存储设备的日志存储区域,并在确定该预写操作日志保存完成时,向主机发送操作执行完成通知,无需等待存储设备的存储控制器对预写操作日志进行回放,将待写入数据写入数据存储区域后再向主机发送操作执行完成通知,因此,可以缩短主机和加速设备执行写操作指令的时延;并且,该过程无需加速设备的主处理器与网络处理器之间进行多次交互来执行写操作指令,从而可以进一步缩短执行写操作指令的时延。
在一种可能的实施方式中,若所述接口接收到主机发送的非修改类操作指令,则将非修改类操作指令发送至加速设备中的主处理器,或存储设备的存储控制器进行处理,其中,非修改类操作指令包括除写操作指令之外的任一操作指令。示例性地,加速设备中的主处理器或存储设备的存储控制器接收到非修改类操作指令,可以根据非修改类操作指令的具体类型执行对应的操作。例如,非修改类操作指令可以包括读操作指令,加速设备中的主处理器或存储设备的存储控制器接收到读操作指令,可以根据读操作指令中携带的地址信息,从存储设备中的数据存储区域读取对应的数据。
上述实施方式中,考虑到非修改类操作指令对时延要求较低,且对处理器的计算资源消耗较少,因
此可以将非修改类操作指令发送至加速设备的主处理器进行处理。
在一种可能的实施方式中,为了避免存储控制器读取尚未写入到数据存储区域的数据,存储设备的日志存储区域可以设置至少一个存储空间,至少一个存储空间中的每个存储空间对应设置有一个过滤器。网络处理器在将预写操作日志保存至存储设备中的日志存储区域时,可以将预写操作日志保存至日志存储区域中的第一存储空间,其中,第一存储空间可以是至少一个存储空间中的任一存储空间。预写操作日志中包括上述写操作指令中携带的第一地址信息。网络处理器在将预写操作日志保存至日志存储区域中的第一存储空间之后,可以更新第一存储空间对应的第一过滤器,更新后的第一过滤器用于表征第一地址信息对应的预写操作日志保存在第一存储空间内,等待通过存储设备的存储控制器进行回放。
在网络处理器向日志存储区域中的第一存储空间内保存预写操作日志时,第一存储空间对应的第一过滤器保存在加速设备中。网络处理器接收到主机发送的读操作指令,获取读操作指令中携带的第二地址信息,其中,读操作指令用于指示根据第二地址信息从数据存储区域获取数据。
网络处理器可以基于第一过滤器,确定第二地址信息对应的预写操作日志是否保存在第一存储空间内,如果确定第二地址信息对应的预写操作日志保存在第一存储空间内,则网络处理器可以向存储设备的存储控制器发送延迟读通知。延迟读通知用于指示存储控制器等待第二地址信息对应的预写操作日志回放完成后,再执行上述读操作指令。
上述实施方式中,通过将日志存储区域划分为一个或多个存储空间,为每个存储空间对应设置一个过滤器,基于过滤器确定读操作指令携带的地址信息对应的预写操作日志是否位于存储空间尚未进行回放。如果尚未进行回放,则延迟执行读操作指令,从而可以避免存储控制器执行读操作指令时,无法读取到数据或读取到错误数据的问题。
在一种可能的实施方式中,在网络处理器向日志存储区域中的第一存储空间内保存预写操作日志时,第一存储空间对应的第一过滤器保存在加速设备中。如果网络处理器确定第一存储空间已满,则可以将第一过滤器保存至存储设备中,以节省加速设备中的存储空间。
在一种可能的实施方式中,如果网络处理器基于第一过滤器,确定第二地址信息对应的预写操作日志未保存在第一存储空间内,则将读操作指令发送至存储设备的存储控制器,以使存储控制器基于存储设备中的过滤器,确定第二地址信息对应的预写操作日志是否保存在日志存储区域,等待通过存储设备的存储控制器进行回放,从而可以更有效地避免存储控制器读取尚未写入到数据存储区域的数据。
在一种可能的实施方式中,预写操作日志中可以包括写操作指令的编号。写操作指令的编号可以采用按照网络处理器接收到各个写操作指令的时间顺序递增的编号。在根据第一地址信息,更新第一存储空间对应的第一过滤器时,可以根据第一地址信息,在第一过滤器中确定至少一个第一目标位置,将写操作指令的编号分别保存至每个第一目标位置中。
在每个第一目标位置中保存写操作指令的编号,与相关技术中将每个第一目标位置中的元素置为设定值相比,在基于第一过滤器判断第二地址信息对应的预写操作日志是否保存在第一存储空间内时,可以降低发生误判的概率。
在一种可能的实施方式中,为避免存储控制器在将热点数据换入加速设备中的缓存区域时,可能换入即将被修改的旧数据,网络处理器接收到存储设备的存储控制器发送的数据换入请求,可以获取数据换入请求中携带的第三地址信息和回放进度标识,其中,数据换入请求用于指示根据第三地址信息,将数据存储区域保存的数据保存至加速设备的缓存区域,回放进度标识指存储控制器已完成回放的预写操作日志中包含的最大编号。
网络处理器可以根据第三地址信息,从第一过滤器中获取最新操作标识。其中,最新操作标识指在接收到数据换入请求之前,最后一次接收到的第三地址信息对应的写操作指令的编号。如果所述最新操作标识大于回放进度标识,说明第三地址信息对应的预写操作日志仍保存在第一存储空间,尚未进行回放,第三地址信息对应的数据即将被修改,则网络处理器可以忽略该数据换入请求,不执行数据换入操作,以避免将即将被修改的旧数据换入到加速设备的缓存区域中,通过上述过程,针对同一地址信息,可以使加速设备的缓存区域保存的数据与存储设备的数据存储区域保存的数据保持一致性。
在一种可能的实施方式中,网络处理器可以通过如下方式从第一过滤器中获取最新操作标识:网络处理器根据第三地址信息,在第一过滤器中确定至少一个第二目标位置,分别获取至少一个第二目标位置中的每个第二目标位置所保存的写操作指令的编号,得到至少一个写操作编号,将至少一个写操作编号中,最小的写操作编号作为最新操作标识。
通过上述过程,可以保证每个第二目标位置中保存的写操作编号均大于回放进度标识时,网络处理器才确定第三地址信息对应的预写操作日志仍保存在第一存储空间,尚未进行回放,忽略该数据换入请求,以减少发生误判的次数,即减少第三地址信息对应的预写操作日志未保存在第一存储空间,却被误判为仍保存在第一存储空间的次数。
在一种可能的实施方式中,存储设备的日志存储区域包括主日志存储区域和备日志存储区域。网络处理器在将预写操作日志保存至存储设备中的日志存储区域时,可以分别将预写操作日志保存至存储设备中的主日志存储区域和备日志存储区域,从而提高存储的数据的准确性和有效性。
第二方面,本申请实施例提供一种数据存储装置,该装置可以设置在数据存储系统的网络处理器内,加速设备可以连接在主机和存储设备之间。该装置可以包括:
日志生成单元,用于接收所述加速设备中的接口转发的所述主机的写操作指令,并基于写操作指令生成预写操作日志;
日志保存单元,用于将预写操作日志保存至存储设备中的日志存储区域,其中,预写操作日志用于在通过存储设备的存储控制器进行回放时,将写操作指令中携带的待写入数据保存至存储设备中的数据存储区域。
在一种可能的实施方式中,所述日志保存单元,还可以用于:在确定预写操作日志保存完成时,向主机发送操作执行完成通知。
在一种可能的实施方式中,日志存储区域可以包括至少一个存储空间,至少一个存储空间中的每个存储空间对应设置有一个过滤器;所述预写操作日志中包括所述写操作指令中携带的第一地址信息;所述日志保存单元,具体用于:
将所述预写操作日志保存至第一存储空间,并更新所述第一存储空间对应的第一过滤器;所述第一存储空间为所述至少一个存储空间中的任一存储空间;更新后的第一过滤器用于表征第一地址信息对应的预写操作日志保存在第一存储空间内,等待通过存储设备的存储控制器进行回放。
在一种可能的实施方式中,第一过滤器保存在加速设备中,所述装置还可以包括读指令执行单元,读指令执行单元用于:
接收到主机发送的读操作指令,获取读操作指令中携带的第二地址信息;所述读操作指令用于指示根据所述第二地址信息从所述数据存储区域获取数据;
若基于第一过滤器,确定第二地址信息对应的预写操作日志保存在第一存储空间内,则向存储设备的存储控制器发送延迟读通知,所述延迟读通知用于指示所述存储控制器等待第二地址信息对应的预写操作日志回放完成后,执行该读操作指令。
在一种可能的实施方式中,预写操作日志中包括所述写操作指令的编号;写操作指令的编号为按照网络处理器接收到各个写操作指令的时间顺序递增的编号;日志保存单元,具体可以用于:根据第一地址信息,在第一过滤器中确定至少一个第一目标位置,并将该写操作指令的编号分别保存至至少一个第一目标位置中的每个第一目标位置。
在一种可能的实施方式中,所述装置还可以包括数据换入单元,数据换入单元可以用于:
接收到存储设备的存储控制器发送的数据换入请求,获取数据换入请求中携带的第三地址信息和回放进度标识;所述回放进度标识指所述存储控制器已完成回放的预写操作日志中包含的最大编号;所述数据换入请求用于指示根据所述第三地址信息,将所述数据存储区域保存的数据保存至所述加速设备的缓存区域;
根据第三地址信息,从第一过滤器中获取最新操作标识;所述最新操作标识指在接收到所述数据换入请求之前,最后一次接收到的所述第三地址信息对应的写操作指令的编号;
若最新操作标识大于回放进度标识,则忽略该数据换入请求。
第三方面,本申请实施例还提供一种芯片,包括处理器和供电电路,所述供电电路用于为所述处理器供电,所述处理器用于执行计算机程序,对通过接口获得的数据执行上述第一方面记载的网络处理器所执行的任一种方法。
第四方面,本申请实施例还提供一种加速设备,所述加速设备连接在主机和存储设备之间,所述加速设备包括接口和网络处理器;
所述接口用于接收数据,把接收的数据提供给所述网络处理器;
所述网络处理器用于执行计算机程序,以实现上述第一方面记载的网络处理器所执行的任一种方法。
第五方面,本申请实施例还提供一种数据存储系统,包括主机、存储设备和加速设备;所述加速设备连接在主机和存储设备之间,所述加速设备采用第四方面记载的加速设备。
第六方面,本申请实施例提供一种计算机可读存储介质,所述计算机可读存储介质内存储有计算机可执行指令,该计算机可执行指令用于使计算机执行上述第一方面提供的任一种方法。
上述第二方面至第六方面中任一方面可以达到的技术效果可以参照上述第一方面中有益效果的描述,此处不再重复赘述。
图1为相关技术中的数据存储系统的结构示意图;
图2为本申请实施例提供的一种数据存储系统的应用场景图;
图3为本申请实施例提供的一种数据存储系统的结构示意图;
图4为本申请实施例提供的一种数据存储方法的交互示意图;
图5为本申请实施例提供的另一种数据存储方法的交互示意图;
图6为本申请实施例提供的另一种数据存储方法的交互示意图;
图7为本申请实施例提供的一种数据存储方法的流程图;
图8为本申请实施例提供的一种日志存储区域的示意图;
图9为本申请实施例提供的另一种数据存储方法的流程图;
图10为本申请实施例提供的一种读操作指令的处理流程图;
图11为本申请实施例提供的一种过滤器的示意图;
图12为本申请实施例提供的一种数据换入请求的处理流程图;
图13为本申请实施例提供的一种数据存储装置的结构框图;
图14为本申请实施例提供的一种芯片的结构框图;
图15为本申请实施例提供的一种加速设备的结构框图。
为了使本申请实施例的目的、技术方案和优点更加清楚,下面将结合附图,对本申请实施例进行详细描述。本申请的实施方式部分使用的术语仅用于对本申请的具体实施例进行解释,而非旨在限定本申请。
在介绍本申请实施例提供的具体方案之前,对本申请中的部分用语进行解释说明,以便于本领域技术人员理解,并不对本申请中的用语进行限定。
(1)加速设备:用于卸载主机的部分功能,例如,可以将网络、存储或操作系统中不适合主机处理的数据处理功能卸载到加速设备,释放主机的算力。加速设备可以包括但不限于数据处理单元(data processing unit,DPU)、基础设施处理器(infrastructure processing unit,IPU)、系统级芯片(system on chip,SoC)、iNIC或smartNIC等具有卸载功能的计算单元。其中,iNIC或smartNIC可以理解为智能网卡。
(2)网络处理器(network processor,NP):主要用于处理与网络相关的业务,如数据传输等。NP支持可编程架构且具有远程直接数据存取(remote direct memory access,RDMA)能力。
(3)文件系统:用于管理和存储文件信息的软件系统,是对文件存储设备的空间进行组织和分配,负责文件存储并对存入的文件进行保护和检索的系统。文件系统的数据存储区域可以包括元数据存储区和数据存储区,其中,数据存储区是负责对文件或文件中的数据进行存储的区域,在文件数据区中,文件以文件数据块的形式进行存储。元数据存储区是支持文件系统架构的区域,用于存储文件的索引、有效性等元数据(metadata),以及文件系统本身的属性数据;也可以理解为,文件系统中的数据分为数据类型的数据和元数据类型的数据,数据是指文件中的实际数据,即文件的内容;而元数据指用来描述一个文件的特征的系统数据,诸如访问权限、文件拥有者以及文件数据块的分布信息等,分布信息可以包括索引节点(inode)等。如果需要操作文件系统中的一个文件必须首先得到它的元数据,才能定位到该文件的位置并且得到文件的内容或相关属性。
本申请实施例中“多个”是指两个或两个以上,鉴于此,本申请实施例中也可以将“多个”理解为“至少两个”。“至少一个”,可理解为一个或多个,例如理解为一个、两个或更多个。例如,包括至少一个,是指包括一个、两个或更多个,而且不限制包括的是哪几个,例如,包括A、B和C中的至少
一个,那么包括的可以是A、B、C、A和B、A和C、B和C、或A和B和C。“和/或”,描述关联对象的关联关系,表示可以存在三种关系,例如,A和/或B,可以表示:单独存在A,同时存在A和B,单独存在B这三种情况。另外,字符“/”,如无特殊说明,一般表示前后关联对象是一种“或”的关系。
除非有相反的说明,本申请实施例提及“第一”、“第二”等序数词用于对多个对象进行区分,不用于限定多个对象的顺序、时序、优先级或者重要程度。
数据存储系统可以应用于云存储或云计算等应用场景。在数据存储系统中,为了节约主机的算力,可以使用加速设备卸载主机的数据存取功能。示例性地,图1所示的数据存储系统包括主机100、加速设备200和存储设备300,加速设备200连接在主机100和存储设备300之间,主机100可以通过加速设备200访问存储设备300。主机100和加速设备200可以作为一个物理机的组件,物理机和存储设备300可以通过网络进行远程通信。
主机100可以理解为物理机的核心处理器,是物理机的运算核心和控制核心。物理机可以包括但不限于云计算集群中的物理服务器、计算设备集群中的计算设备或网络管理中心的服务器。主机100可以接收用户通过客户端输入的数据,并对数据进行处理。
为了对数据进行持久化存储,数据存储系统中的存储设备300可以设置于主机100的外部,通过网络与主机100交换数据。存储设备300可以包括存储控制器310和盘框320,存储控制器310可以用于对盘框320进行管理和控制。存储控制器310和盘框320可以通过总线连接,也可以通过网络进行远程通信。
加速设备200用于卸载主机100的部分功能。示例性地,加速设备200可以卸载主机100的数据存取功能。加速设备200卸载主机100的部分功能,主机100便可以专用于进行管理和控制等功能,从而可以提升主机100的性能。
如图1所示,相关技术中,加速设备200可以包括主处理器210和专用集成电路(application specific integrated circuit,ASIC)芯片220。主处理器210可以采用中央处理器(Central Processing unit,CPU)或其他通用处理器实现。ASIC芯片220中可以包括硬件队列221和网络引擎222,其中,网络引擎222可以理解为网络接口,网络引擎222可以采用远程内存访问(rdma over converged ethernet,RoCE)接口实现。
示例性地,图1所示的数据存储系统可以用于处理文件系统中的文件输入输出(input output,IO)操作。主机100可以通过队列通道将文件操作命令发送至加速设备200中的硬件队列221。加速设备200的主处理器210从硬件队列221中读取文件操作指令,并执行该文件IO操作,基于该文件IO操作生成数据存取指令,然后通过加速设备200中的网络引擎222将数据存取指令发送至存储控制器310,由存储控制器310与盘框320进行交互,执行数据存取指令。数据存取指令执行完成后,存储控制器310向加速设备200返回存取完成消息,加速设备200通过网络引擎222接收存储控制器310发送的存取完成消息,并传输至加速设备200的主处理器210,主处理器210对存取完成消息进行处理,再通过ASIC芯片220中的硬件队列221通知主机100文件操作指令执行完成。
在上述处理文件IO操作的过程中,加速设备200对文件IO操作的处理主要由主处理器210实现,会消耗主处理器210的计算资源,影响加速设备200的性能;并且,在加速设备200的主处理器210和ASIC芯片220之间存在四次交互,增加了文件IO操作的时延。
基于此,本申请实施例提供一种数据存储系统。示例性地,在一些应用场景中,如图2所示,该数据存储系统可以包括多个物理机,如物理机410、物理机420、物理机430等。每个物理机均可以包括主机和加速设备,加速设备可以通过网络远程与存储控制器和盘框进行通信。每个物理机均可以通过加速设备与存储控制器或盘框进行数据或信令的交互。图2中以2个存储控制器和2个盘框为例进行说明,如存储控制器510、存储控制器520、盘框530和盘框540。在实际使用中,存储控制器和盘框的数量可以多于2个,也可以少于2个。存储控制器和盘框可以位于一个存储设备中,也可以是分离设置,通过网络进行远程通信。存储控制器可以用于对盘框进行管理和控制。
图3以一个物理机与存储设备交互为例进行说明,如图3所示,物理机600可以包括主机610和与主机610连接的加速设备620。在一些实施例中,加速设备620可以直接插在主机610的主板上的卡槽中,通过PCIe总线与主机610交换数据或信令。需要说明的是,PCIe总线能够被替换成计算快速互联(compute express link,CXL)、通用串行总线(Universal Serial Bus,USB)协议或其他协议的总线。
存储设备700可以包括存储控制器710和与存储控制器710连接的多个盘框,如盘框720,盘框730等,存储控制器710可以用于对多个盘框进行管理和控制。图3中仅是以2个盘框为例,在实际使用中,盘框的数量可以多于2个,也可以少于2个。
其中,盘框可以采用固态硬盘(solid state disk,SSD),固态硬盘是用固态电子存储芯片阵列而制成的硬盘,可以包括控制单元和存储单元,控制单元可以接收网络处理器或存储控制器发送的指令,对存储单元进行管理和控制。存储单元可以采用闪存(flash)芯片或动态随机存取存储器(dynamic random access memory,DRAM)芯片。
如图3所示,加速设备620可以包括主处理器621、网络处理器622和内部存储器623。主处理器621可以是CPU,也可以是可编程逻辑器件(programmable logic device,PLD)芯片,上述PLD可以是复杂程序逻辑器件(complex programmable logical device,CPLD),现场可编程门阵列(field-programmable gate array,FPGA),通用阵列逻辑(generic array logic,GAL)或其任意组合。主处理器621用于运行加速设备620的操作系统及基于操作系统运行的软件程序。网络处理器622用于接收并处理主机610发送的文件IO操作。
内部存储器623中的缓存区域可以用于保存文件系统对应的数据快表和元数据快表,用于保存文件系统对应的数据快表的区域可以称为数据缓存,用于保存文件系统对应的元数据快表的区域可以称为元数据缓存。在一些实施例中,数据缓存中保存有数据项,数据项可以用于保存数据在盘框中的存放地址。元数据缓存可以保存文件系统的元数据,元数据可以采用树状结构,包括多级的父节点信息和子节点信息。
本申请实施例中,主机610可以将文件IO操作的操作指令发送至加速设备620,加速设备620通过接口接收主机610发送的操作指令,把操作指令转发给加速设备中的网络处理器622。若接收到的操作指令是写操作指令,则网络处理器622可以基于写操作指令生成预写操作日志。其中,写操作指令指携带有待写入数据的操作指令,可以包括但不限于修改数据的操作指令。预写操作日志也可以称为预写式日志(write-ahead logging,WAL)。生成预写操作日志之后,网络处理器622可以将预写操作日志保存至存储设备700中的日志存储区域,预写操作日志用于在通过存储设备700的存储控制器进行回放时,将写操作指令中携带的待写入数据保存至存储设备中的数据存储区域。其中,日志存储区域和数据存储区域可以保存在存储设备700的盘框中。日志存储区域用于保存预写操作日志,也可以称为WAL区域;数据存储区域用于保存文件系统的数据,也可以称为持久化日志(persistent logging,PLOG)区域。
示例性地,图3所示的盘框720中设有日志存储区域721和数据存储区域722。盘框730设有日志存储区域731和数据存储区域732。在一些实施例中,日志存储区域721和日志存储区域731中可以用于保存不同的预写操作日志。在另一些实施例中,为保证存储数据的可靠性,日志存储区域721和日志存储区域731可以互为主备,即日志存储区域721和日志存储区域731中可以保存相同的预写操作日志。在另一些实施例中,主备WAL区域也可以位于同一盘框内。
在本申请实施例中,加速设备通过网络处理器处理接收到的写操作指令,可以节约加速设备的主处理器的计算资源,提高加速设备的性能,并且无需处理器与网络处理器之间多次交互,从而可以缩短执行IO操作的时延。网络处理器基于接收到的写操作指令生成预写操作日志,将预写操作日志保存至存储设备中的日志存储区域,这两个步骤比较简单,执行速度较快,因此可以节约时间。对预写操作日志进行回放的复杂过程由存储控制器执行,并且可以在存储控制器中批量完成回放,不影响加速设备执行IO操作的时延。
本申请实施例还提供了一种可以应用于图2和图3所示的数据存储系统的数据存储方法。在一些实施例提供的数据存储方法中,网络处理器接收到的主机发送的文件IO操作的操作指令根据操作类型可以分为两类:执行频率较高且对处理器性能和时延需求较高的操作指令可以划分至第一类操作指令,例如文件的创建、修改、读IO、写IO等;执行频率不高,或者,对处理器性能和时延需求相对不高的操作指令可以划分至第二类操作指令,例如重命名(rename)、计数(mount),初始化等。针对不同操作指令的不同需求,可以采取不同的处理方式。
例如,主机在需要对文件系统进行操作时,可以向加速设备发送文件IO操作的操作指令。加速设备中的接口接收到主机发送的操作指令后,将操作指令转发给加速设备中的网络处理器。网络处理器接收到主机发送的操作指令后,可以按格式对操作指令进行语义解析,确定该操作指令对应的操作类型,根据操作指令对应的操作类型,对操作指令进行处理。
图4示例性地示出了一种数据存储方法中,针对第二类操作指令进行处理时,各个设备之间的交互流程图。如图4所示,该方法可以包括如下步骤:
S401,主机向加速设备发送操作指令。
S402,加速设备通过网络处理器确定操作指令的操作类型为第二类操作指令。
加速设备中的接口接收到主机发送的操作指令后,将操作指令转发给加速设备中的网络处理器。网络处理器可以通过语义解析,确定该操作指令对应的操作类型。例如,网络处理器通过语义解析,确定该操作指令为初始化操作指令,从而可以确定该操作指令为第二类操作指令。
S403,加速设备向存储控制器发送操作指令。
S404,存储控制器处理操作指令。
在一些实施例中,加速设备的网络处理器可以将第二类操作指令转发至存储设备中的存储控制器,由存储控制器进行处理。例如,如果接收到的第二类操作指令为初始化操作指令,则存储控制器执行初始化操作指令,对初始化操作指令所指示的盘框进行初始化。
在另一些实施例中,由于第二类IO操作执行频率较低,且对处理器性能要求相对不高,网络处理器可以将第二类操作指令转发至加速设备中的主处理器,由主处理器执行第二类操作指令。例如,网络处理器可以将接收到的初始化操作指令转发至加速设备中的主处理器处理,加速设备中的主处理器执行初始化操作指令,确定需要进行初始化的盘框,通过存储控制器对该盘框进行初始化。
图5示例性地示出了一种数据存储方法中,针对第一类操作指令中的读操作指令进行处理时,各个设备之间的交互流程图。如图5所示,该方法可以包括如下步骤:
S501,主机向加速设备发送操作指令。
S502,加速设备通过网络处理器确定操作指令的操作类型为读操作指令,获取读操作指令中携带的地址信息。
加速设备中的接口接收到主机发送的操作指令后,将操作指令转发给加速设备中的网络处理器。网络处理器通过语义解析,确定该操作指令对应的操作类型为读操作指令。其中,读操作指令指示根据该读操作指令携带的地址信息从存储设备的数据存储区域获取数据。
S503,加速设备通过网络处理器根据该地址信息,查找加速设备的缓存区域,确定缓存区域不包含该地址信息。
S504,加速设备向存储控制器发送该读操作指令。
其中,地址信息可以是要读取的数据的逻辑地址。加速设备中的网络处理器获取该读操作指令中携带的逻辑地址后,查询加速设备的缓存区域是否包含该逻辑地址,如果包含,则可以从缓存区域获取该逻辑地址对应的数据;如果不包含,则加速设备中的网络处理器可以将读操作指令转发至存储设备中的存储控制器,由存储控制器进行处理。存储控制器可以根据读操作指令中携带的地址信息,从盘框中的数据存储区域读取数据。在另一些实施例中,加速设备中的网络处理器也可以将该读操作指令发送到加速设备中的主处理器,由主处理器进行处理。加速设备中的主处理器可以根据读操作指令中携带的地址信息,从盘框中的数据存储区域读取数据。
在一些实施例中,读操作指令根据要读取的数据的数据类型,可以分为读数据操作指令和读元数据操作指令。读数据操作指令用于读取数据类型的数据;读元数据操作指令用于读取元数据类型的数据。针对读数据操作指令,网络处理器可以在获取逻辑地址后,查询缓存区域中的数据缓存是否包含该逻辑地址。针对读元数据操作指令,网络处理器可以在获取逻辑地址后,查询缓存区域中的元数据缓存是否包含该逻辑地址。
上述第二类操作指令和读操作指令可以统称为非修改类操作指令,非修改类操作指令可以包括除写操作指令之外的任一操作指令。相对于写操作指令而言,非修改类操作指令可以指不携带待写入数据的操作指令。考虑到非修改类操作指令对时延要求较低,且对处理器的计算资源消耗较少,基本不会影响加速设备的性能,因此,网络处理器接收到主机发送的非修改类操作指令之后,可以将非修改类操作指令发送至加速设备的主处理器进行处理,也可以将非修改类操作指令发送至存储设备的存储控制器进行处理。
图6示例性地示出了一种数据存储方法中,针对第一类操作指令中的写操作指令进行处理时,各个设备之间的交互流程图。如图6所示,该方法可以包括如下步骤:
S601,主机向加速设备发送操作指令。
S602,加速设备通过网络处理器确定操作指令的操作类型为写操作指令,根据写操作指令中携带的地址信息和待写入数据,生成预写操作日志。
加速设备中的接口接收到主机发送的操作指令后,将操作指令转发给加速设备中的网络处理器。网络处理器通过语义解析,确定该操作指令对应的操作类型为写操作指令。其中,写操作指令指示根据该写操作指令携带的地址信息向存储设备的数据存储区域写入数据或更新数据,地址信息可以是待写入数据对应的逻辑地址。
网络处理器可以获取写操作指令中携带的逻辑地址和待写入数据,生成包含该逻辑地址和待写入数据的预写操作日志。
在一些实施例中,写操作指令根据要写入的数据的数据类型,可以分为写数据操作指令和写元数据操作指令。写数据操作指令用于写入数据类型的数据;写元数据操作指令用于写入元数据类型的数据。
针对写数据操作指令,网络处理器在生成预写操作日志之前,可以获取写操作指令中携带的逻辑地址,查询加速设备的数据缓存中是否保存有该逻辑地址。在一种实施例中,如果数据缓存中保存有该逻辑地址,说明数据缓存中保存有该写数据操作指令的关联数据项,网络处理器可以清除该写数据操作指令的关联数据项,即清除该逻辑地址对应的关联数据项,无需将写数据操作指令中携带的待写入数据和逻辑地址保存在数据缓存,以避免数据缓存因大量写数据操作频繁被刷写,减少资源消耗。在另一种实施例中,如果数据缓存中保存有该逻辑地址,则网络处理器可以根据待写入数据更新该逻辑地址对应的关联数据项。如果数据缓存中未保存该逻辑地址,则网络处理器可以在数据缓存中新建数据项,用于保存写数据操作指令中的待写入数据和待写入数据对应的逻辑地址。
针对写元数据操作指令,网络处理器在生成预写操作日志之前,可以获取写元操作指令中携带的待写入数据和待写入数据对应的逻辑地址,将待写入数据和待写入数据对应的逻辑地址添加至加速设备的元数据缓存中,其中,待写入数据为元数据类型的数据。加速设备中的元数据缓存写满后,加速设备可以将元数据缓存中的内容保存至存储控制器中用于保存文件系统元数据的存储区域。
S603,加速设备将预写操作日志写入盘框。
S604,加速设备向主机发送操作执行完成通知。
加速设备可以通过网络处理器将生成的预写操作日志保存至盘框中的日志存储区域,并在确定预写操作日志保存完成后,向主机发送操作执行完成通知。示例性地,网络处理器可以通过RDMA通道,将预写操作日志写入盘框中。其中,RDMA通道基于RDMA协议实现,通过RDMA协议,网络处理器将预写操作日志传输至盘框,盘框将接收到的预写操作日志保存至日志存储区域,并在保存完成后,向网络处理器反馈ACK消息。ACK消息为确认消息,用于通知网络处理器,预写操作日志已经写入完成。网络处理器接收到盘框反馈的ACK消息,确定预写操作日志已经保存完成,则向主机发送操作执行完成通知。预写操作日志用于在通过存储控制器进行回放时,将写操作指令中携带的待写入数据保存至盘框中的数据存储区域。数据存储区域可以包括元数据区和文件数据区。如果待写入数据是元数据类型的数据,可以写入元数据区;如果待写入数据是数据类型的数据,可以写入文件数据区。
上述基于写操作指令生成预写操作日志,以及将预写操作日志写入存储设备的盘框的过程,均由加速设备的网络处理器执行,网络处理器执行完成后,直接向主机返回操作执行完成通知。该过程无需加速设备的网络处理器与处理器之间进行多次交互,可以减少网络处理器与处理器之间的交互次数,有效减少执行操作指令的时延,实现IO的快速访问路径,提升数据存储系统的整体性能。
日志存储区域可以称为WAL区域。在一些实施例中,为了保证存储的数据的准确性和有效性,盘框中可以设置有主WAL区域和备WAL区域,可以将预写操作日志同时保存至主WAL区域和备WAL区域。主WAL区域和备WAL区域可以位于同一盘框中,也可以位于不同的盘框中。
S605,存储控制器向盘框发送读取日志通知。
S606,盘框向存储控制器发送预写操作日志。
存储控制器进行日志回放时,可以从盘框获取预写操作日志。存储控制器向盘框发送读取日志通知,盘框基于接收到的读取日志通知向存储控制器发送预写操作日志。当存在主WAL区域和备WAL区域时,存储控制器可以指定从主WAL区域或从备WAL区域读取预写操作日志,回放完成后,可以同时清空主WAL区域和备WAL区域中保存的该预写操作日志。示例性地,如果存储控制器指定从主WAL区域读取预写操作日志,主WAL区域保存的日志错误,无法正常读取时,可以从备WAL区域中读取该预写操作日志。
S607,存储控制器对接收到的预写操作日志进行回放。
预写操作日志中包括写操作指令携带的待写入数据和待写入数据对应的逻辑地址。存储控制器通过对预写操作日志进行回放,可以根据写操作指令携带的逻辑地址,将写操作指令携带的待写入数据保存至盘框中的数据存储区域。示例性地,存储控制器根据写操作指令携带的逻辑地址,确定该逻辑地址在数据存储区域中对应的物理地址,基于确定的物理地址,将待写入数据写入到数据存储区域。
在上述实施例中,加速设备执行的数据存储方法,可以包括如图7所示的如下步骤:
S701,加速设备中的接口接收主机发送的写操作指令,把写操作指令转发给加速设备中的网络处理器。
S702,加速设备中的网络处理器基于写操作指令生成预写操作日志。
预写操作日志中包括写操作指令携带的待写入数据和待写入数据对应的地址信息。
S703,加速设备中的网络处理器将预写操作日志保存至存储设备中的日志存储区域。
预写操作日志用于在通过存储设备的存储控制器进行回放时,将写操作指令中携带的待写入数据保存至存储设备中的数据存储区域。
如果网络处理器确定预写操作日志保存完成,则向主机发送操作执行完成通知。
通过网络处理器处理接收到的写操作指令,可以节约加速设备的主处理器的计算资源,提高加速设备的性能。网络处理器基于接收到的写操作指令生成预写操作日志,将预写操作日志保存至存储设备中的日志存储区域,这两个步骤比较简单,执行速度较快,因此可以节约时间。对预写操作日志进行回放的复杂过程由存储控制器执行,并且可以在存储控制器中批量完成回放,不会增加加速设备执行IO操作的时延。
在一些实施例中,加速设备中的接口接收主机发送的读操作指令,可以通过加速设备中的网络处理器将读操作指令发送至存储控制器,由存储控制器根据读操作指令中携带的地址信息从盘框的数据存储区域读取所需的数据。读操作指令中携带的地址信息可以称为读操作地址信息。考虑到存储控制器在执行读操作指令时,读操作地址信息对应的待写入数据有可能还保存在预写操作日志中,即预写操作日志仍在日志存储区域,还未进行回放。例如,在第一时刻,主机向加速设备发送第一操作指令,第一操作指令为写操作指令,写操作指令指示向地址a写入第一数据,网络处理器接收到第一操作指令,基于第一操作指令生成第一预写操作日志,并将第一预写操作日志写入盘框中的日志存储区域。第一预写操作日志中包括第一数据和第一数据对应的地址a。紧接着,在第二时刻,主机向加速设备发送第二操作指令,第二操作指令为读操作指令,读操作指令指示读取地址a中的数据,而由于地址a对应的第一预写操作日志还在日志存储区域中,尚未进行回放,地址a对应的第一数据尚未写入到数据存储区域中。如果存储控制器此时执行读操作指令,可能无法读取到数据,或者,可能读取到错误的数据。
为了解决存储控制器可能会读取尚未写入到数据存储区域的数据的问题,本申请实施例将盘框中的日志存储区域划分为至少一个存储空间,存储空间用于保存基于写操作指令生成的预写操作日志,每个存储空间可以保存一个或多个预写操作日志,每个存储空间可以称为一个数据文件空间(segment)。每个存储空间对应设置一个过滤器,过滤器用于确定某个地址信息是否包含在对应的存储空间中保存的尚未回放的预写操作日志中。例如,如图8所示,盘框800中的日志存储区域810可以包括多个存储空间,图8中仅示出3个存储空间,分别是存储空间811、存储空间812和存储空间813。第一过滤器是存储空间811对应的过滤器,第二过滤器是存储空间812对应的过滤器,第三过滤器是存储空间813对应的过滤器。其中,过滤器可以采用布隆过滤器(bloom filter),也可以采用商过滤器或其他过滤器。
在一些实施例中,如图9所示,加速设备中的网络处理器向盘框中写入预写操作日志的过程,可以包括如下步骤:
S901,基于接收到的写操作指令生成预写操作日志。
针对接收到的写操作指令,网络处理器可以根据写操作指令携带的待写入数据和第一地址信息,生成预写操作日志。预写操作日志中包括写操作指令中携带的第一地址信息,其中,第一地址信息可以是待写入数据对应的逻辑地址。
S902,将预写操作日志保存至日志存储区域中的当前存储空间。
其中,当前存储空间可以是多个存储空间中的任一存储空间。网络处理器依次向日志存储区域中的每个存储空间内写入预写操作日志,例如,网络处理器先向图8中所示的存储空间811中写入预写操作日志,此时,存储空间811为当前存储空间。若写入30个预写操作日志后,存储空间811写满,则再
生成第31个预写操作日志后,网络处理器可以将第31个预写操作日志写入到存储空间812中,此时,存储空间812为当前存储空间。存储空间812写满后,再生成的预写操作日志,可以将新生成的预写操作日志写入存储空间813中,此时,存储空间813为当前存储空间。
S903,根据预写操作日志中的第一地址信息,更新当前存储空间对应的当前过滤器。
更新后的当前过滤器用于表征第一地址信息对应的预写操作日志保存在当前存储空间内,等待通过存储控制器进行回放。
示例性地,网络处理器可以通过设定的一组哈希(hash)函数对第一地址信息进行处理,确定一组hash值,hash值用于指示第一地址信息在过滤器中对应的位置,可以更新过滤器中对应位置的元素。得到的hash值的数量与hash函数的数量一致。如果hash函数有3个,则对第一地址信息进行处理后,可以得到3个hash值,根据3个hash值,确定第一地址信息在过滤器中对应的3个位置。例如,假设过滤器中包括0~63共64个元素,每个元素占用1bit,也可以理解为,过滤器中包括64个位置。初始状态下,过滤器中64个位置的元素均为0。假设对第一地址信息进行处理,得到的3个hash值分别是8、15、19,则说明第一地址信息在过滤器中分别对应位置8、位置15和位置19,可以将位置8、位置15和位置19上的元素均设置为1。反过来说,位置8、位置15和位置19上的元素均为1时,说明第一地址信息对应的预写操作日志保存在当前存储空间内,此时,第一地址信息对应的待写入数据尚未写入到数据存储区域中。
当前存储空间对应的当前过滤器可以保存在加速设备的内部存储器中。待当前存储空间写满后,网络处理器将当前过滤器的内容保存至盘框中。例如,盘框检测到当前存储空间写满时,可以向网络处理器发送存储空间已满的消息,网络处理器接收到盘框发送的存储空间已满的消息,确定当前存储空间已满,则将当前过滤器的内容保存至盘框中,盘框将该当前过滤器与相应的存储空间进行对应保存。示例性地,图8中的存储空间811和存储空间812已经写满,存储空间811对应的第一过滤器和存储空间812对应的第二过滤器已经保存在盘框中,而存储空间813尚未写满,即网络处理器还在向存储空间813中写入预写操作日志,存储空间813是当前存储空间,存储空间813对应的第三过滤器是当前过滤器,第三过滤器的内容还保存在加速设备的内部存储器中,而未保存在盘框中,因此盘框中的第三过滤器采用虚线框表示。
上述是网络处理器向盘框中写入预写操作日志的过程。对于盘框中保存的预写操作日志,存储控制器可以依次对每个存储空间中保存的预写操作日志进行回放,当某个存储空间中保存的预写操作日志全部回放完毕,存储控制器可以控制盘框清空该存储空间,并清空该存储空间对应的过滤器中保存的内容,将该存储空间对应的过滤器中的所有位置的元素重新置为0。
在一些实施例中,如图10所示,加速设备中网络处理器对主机发送的读操作指令的处理过程,可以包括如下步骤:
S1001,获取读操作指令中携带的第二地址信息。
网络处理器接收到主机发送的操作指令,可以通过语义解析确定操作指令对应的操作类型。若确定接收到的操作指令为读操作指令,可以获取读操作指令中携带的第二地址信息。其中,读操作指令用于指示根据第二地址信息从数据存储区域获取数据。其中,第二地址信息可以是需要读取的数据的逻辑地址。
S1002,确定缓存区域是否包含第二地址信息;如果是,则执行步骤S1003;如果否,则执行步骤S1004。
S1003,从缓存区域中读取第二地址信息对应的数据,并将读取的数据发送至主机。
网络处理器根据第二地址信息查找加速设备的缓存区域,确定缓存区域是否包含第二地址信息,如果加速设备的缓存区域包含第二地址信息,可以从缓存区域获取第二地址信息对应的数据,并发送至主机。
S1004,基于当前过滤器,确定第二地址信息对应的预写操作日志是否保存在当前存储空间内;如果是,则执行步骤S1006;如果否,则执行步骤S1005。
如果加速设备的缓存区域不包含第二地址信息,说明需要从盘框的数据存储区域读取数据,则网络处理器可以基于加速设备中保存的当前过滤器,确定第二地址信息对应的预写操作日志是否保存在当前存储空间内。
示例性地,网络处理器可以通过设定的一组hash函数对第二地址信息进行处理,得到一组hash值,
即第二地址信息对应的hash值。网络处理器可以在当前过滤器中,确定第二地址信息对应的hash值所指示的位置的元素是否为1。
考虑到采用一个hash函数对两个不同的地址信息进行处理时,可能会得到相同的hash值。例如,地址b对应的预写操作日志保存在当前存储空间内,地址c对应的预写操作日志没有保存在当前存储空间内,由于采用一个hash函数对地址b和地址c进行处理时,得到相同的hash值,该hash值对应的位置的元素为1,因此,基于该hash值可以得出地址c对应的预写操作日志保存在当前存储空间内,从而造成误判。为减少误判,可以设置多个hash函数,通过多个hash函数对一个地址信息进行处理,得到多个hash值,如果多个hash值所指示的位置的元素全部为1,则可以确定该地址信息对应的预写操作日志保存在当前存储空间内。如果多个hash值所指示的位置的元素中存在至少一个元素不是1,则可以确定该地址信息对应的预写操作日志没有保存在当前存储空间内。
假设对第二地址信息进行处理,得到的hash值包括3个,如果第二地址信息对应的3个hash值所指示的位置的元素全部为1,则可以确定第二地址信息对应的预写操作日志保存在当前存储空间内。如果第二地址信息对应的3个hash值所指示的位置的元素中,存在一个或多个元素不是1,则可以确定第二地址信息对应的预写操作日志未保存在当前存储空间内。
S1005,将读操作指令发送至存储设备的存储控制器。
网络处理器确定第二地址信息对应的预写操作日志未保存在当前存储空间内时,可以将读操作指令发送至存储设备的存储控制器,以使存储控制器基于盘框中保存的过滤器,确定第二地址信息对应的预写操作日志是否保存在盘框中的日志存储区域,等待通过存储控制器进行回放。
例如,假设加速设备中保存的当前过滤器为图8中的存储空间813对应的第三过滤器,存储空间813是当前存储空间。网络处理器确定第二地址信息对应的预写操作日志未保存在存储空间813内,则可以将读操作指令发送至存储控制器,存储控制器可以获取盘框中保存的过滤器,如图8所示的第一过滤器和第二过滤器。存储控制器可以基于第一过滤器,确定第二地址信息对应的预写操作日志是否保存在存储空间811内,并且,可以基于第二过滤器,确定第二地址信息对应的预写操作日志是否保存在存储空间812内。如果第二地址信息对应的预写操作日志未保存在存储空间811内,并且未保存在存储空间812内,则存储控制器可以执行该读操作指令,基于第二地址信息从数据存储区域读取数据。否则,存储控制器可以等待第二地址信息对应的预写操作日志回放完成后,再执行该读操作指令。
S1006,向存储设备的存储控制器发送延迟读通知。
延迟读通知中可以包括上述读操作指令。延迟读通知用于指示存储控制器等待第二地址信息对应的预写操作日志回放完成后,执行该读操作指令。
在一些实施例中,延迟读通知可以是添加延迟读标志的读操作指令。示例性地,网络处理器确定第二地址信息对应的预写操作日志保存在当前存储空间内,尚未进行回放,说明该读操作指令与该读操作指令之前的某个写操作指令存在冲突,或者说,存在依赖关系。此时,网络处理器可以向读操作指令添加延迟读标志,将添加延迟读标志的读操作指令发送至存储控制器。
存储控制器接收到网络处理器发送的延迟读通知,可以等待第二地址信息对应的预写操作日志回放完成后,再执行该读操作指令。
在一些实施例中,网络处理器接收到的每个操作指令均具有唯一的编号,每个操作指令的编号可以是按照网络处理器接收到各个操作指令的时间顺序递增的编号,或称为序号(sequence)。基于写操作指令生成的预写操作日志中可以包括对应的写操作指令的编号。存储控制器接收到的网络处理器发送的延迟读通知中包括读操作指令,也包括读操作指令的编号。存储控制器进行日志回放时,按照各个写操作指令的时间顺序依次进行回放。存储控制器可以确定正在回放的预写操作日志所包含的写操作指令的编号,该编号可以称为回放进度标识。存储控制器将回放进度标识与延迟读通知中的读操作指令的编号进行比较,如果回放进度标识小于读操作指令的编号,则说明第二地址信息对应的预写操作日志有可能还未进行回放,存储控制器继续等待。如果回放进度标识大于读操作指令的编号,则说明读操作指令之前的所有写操作指令对应的预写操作日志均已回放,第二地址信息对应的预写操作日志必定已经回放,则存储控制器可以执行该读操作指令,基于第二地址信息从数据存储区域读取数据。
在另一些实施例中,可以增大过滤器中每个位置的元素的长度,例如,每个元素的长度可以从1bit增大到4个字节(32bit)或8个字节(64bit)。网络处理器接收到的每个操作指令均具有唯一的编号,基于写操作指令生成的预写操作日志中可以包括对应的写操作指令的编号。在实际使用中,过滤器中每
个位置的元素的长度,可以根据实际需求确定。元素的长度越长,其所能保存的最大编号越大,该最大编号应大于或等于设定的周期时长内主机发送至网络处理器的操作指令的数量。下一个周期开始时,主机发送至网络处理器的操作指令的编号从0开始,重新排号。
在根据预写操作日志中的第一地址信息,更新当前过滤器时,可以根据第一地址信息,在当前过滤器中确定第一目标位置,并将写操作指令的编号保存至当前过滤器中的第一目标位置。示例性地,可以采用设定的一组hash函数对第一地址信息进行处理,得到一个或多个hash值,根据一个或多个hash值,在当前过滤器中确定一个或多个第一目标位置,并将第一地址信息对应的写操作指令的编号保存至每个第一目标位置。将第一地址信息对应的写操作指令的编号保存至第一目标位置,与将第一目标位置中的元素置为1相比,可以大幅度降低误判的概率。
例如,如图11所示,仍假设采用3个hash函数对第一地址信息进行处理,得到的3个hash值分别是8、15、19,则说明第一地址信息在当前过滤器中分别对应位置8、位置15和位置19,可以将该写操作指令的编号S11分别保存至位置8、位置15和位置19。当前过滤器中其他位置可能保存有其他指令的编号,也可能未保存任何指令的编号。如果一个位置未保存任何指令的编号,则该位置对应的元素可以是初始值0。在多个位置保存该写操作指令的编号S11,同样是为了避免误判,上文中已经进行说明,在此不再赘述。
网络处理器在确定第二地址信息对应的预写操作日志保存在当前存储空间内时,可以根据第二地址信息,从当前过滤器中获取第二地址信息对应的写操作指令的编号,该编号可以称为冲突操作编号。在向存储控制器发送的延迟执行通知中,可以包括该冲突操作编号。
存储控制器进行日志回放时,可以确定正在回放的预写操作日志所包含的写操作指令的编号,该编号可以称为回放进度标识。每完成一个预写操作日志的回放,存储控制器将回放进度标识与冲突操作标识进行比对,如果回放进度标识与冲突操作标识相同,则说明第二地址信息对应的预写操作日志已经回放完成,存储控制器可以执行该读操作指令,基于第二地址信息从数据存储区域读取数据。如果回放进度标识与冲突操作标识不同,则说明第二地址信息对应的预写操作日志还未回放,存储控制器继续等待。
上述实施例中,通过将日志存储区域划分为一个或多个存储空间,为每个存储空间对应设置一个过滤器,基于过滤器确定读操作指令携带的地址信息对应的预写操作日志是否位于存储空间尚未进行回放,进而可以避免存储控制器执行读操作指令时,无法读取到数据或读取到错误数据的问题。
为方便主机快速读取到所需的数据,更高效地利用加速设备中的缓存区域,存储控制器可以将主机频繁读取的热点数据换入加速设备中的缓存区域。存储控制器可以识别热点数据,在识别到热点数据时,可以从数据存储区域读取热点数据,将热点数据写入加速设备中的缓存区域,或者,可以从盘框获取数据存储区域中用于保存该热点数据的地址,将该热点数据的地址写入加速设备中的缓存区域。
存储控制器在执行数据换入的过程中,可能存在如下问题:在一些实施例中,在存储控制器确定数据A为热点数据,从数据存储区域获取数据A时,加速设备中的网络处理器可能已经将修改数据A的写操作指令对应的预写操作日志保存至日志存储区域,数据A即将被修改。此时,存储控制器无法得知数据A即将被修改,仍将数据A写入加速设备中的缓存区域,会造成将未修改的旧数据换入加速设备中的缓存区域。在另一些实施例中,在存储控制器确定数据A为热点数据,从数据存储区域获取保存该数据A的地址ADD1时,网络处理器可能已经将修改数据A的写操作指令对应的预写操作日志保存至日志存储区域。此时,存储控制器无法得知数据A即将被修改,仍将保存该数据A的地址ADD1写入加速设备中的缓存区域,而当对修改数据A的写操作指令对应的预写操作日志进行回放时,可能会为修改后的数据A配置新的地址ADD2,将修改后的数据A保存至新的地址ADD2,导致主机或加速设备根据数据缓存中保存的地址ADD1读取数据时,读取到的是未修改的旧数据,相当于将未修改的旧数据换入加速设备中的缓存区域。
为了解决存储控制器将未修改的旧数据换入缓存区域的问题,在一些实施例中,加速设备中的网络处理器接收到的每个写操作指令均具有唯一的编号,每个操作指令的编号可以是按照网络处理器接收到各个写操作指令的时间顺序递增的编号。基于写操作指令生成的预写操作日志中可以包括对应的写操作指令的编号。网络处理器在根据预写操作日志中的第一地址信息,更新当前过滤器时,可以根据第一地址信息,在当前过滤器中确定一个或多个第一目标位置,并将第一地址信息对应的写操作指令的编号保存至每个第一目标位置。具体的执行过程,上文中已经介绍,在此不再赘述。
在数据换入过程中,加速设备中的网络处理器可以执行如图12所示的如下步骤:
S1201,接收到存储设备的存储控制器发送的数据换入请求,获取数据换入请求中携带的第三地址信息和回放进度标识。
示例性地,存储控制器在确定数据存储区域保存的数据A是热点数据时,可以向加速设备的网络处理器发送数据换入请求,数据换入请求中可以包括数据A对应的第三地址信息,或者,数据换入请求中可以包括数据A和数据A对应的第三地址信息。数据换入请求用于指示网络处理器根据第三地址信息,将数据存储区域保存的数据保存至加速设备的缓存区域。其中,第三地址信息可以是数据A对应的逻辑地址,该逻辑地址可以是根据数据存储区域中用于保存的数据A的物理地址确定的。
数据换入请求中还可以包括回放进度标识,回放进度标识指存储控制器已完成回放的预写操作日志中包含的最大编号S_MAX,即存储控制器已完成回放的最后一个预写操作日志所包含的写操作指令的编号。
S1202,根据第三地址信息,从当前过滤器中获取最新操作标识。
其中,最新操作标识指在接收到该数据换入请求之前,最后一次接收到的第三地址信息对应的写操作指令的编号。
示例性地,网络处理器可以通过设定的一组hash函数对第三地址信息进行处理,得到至少一个hash值,即第三地址信息对应的hash值。网络处理器可以从当前过滤器中,分别获取第三地址信息对应的至少一个hash值所指示的至少一个第二目标位置所保存的写操作指令的编号,得到一个或多个最新操作标识。
S1203,判断最新操作标识是否大于回放进度标识;如果是,则执行步骤S1204;如果否,则执行步骤S1205。
S1204,忽略该数据换入请求。
S1205,执行该数据换入请求。
网络处理器可以根据回放进度标识和最新操作标识,确定是否执行该数据换入请求。
在一种实施例中,当步骤S1202中得到一个最新操作标识时,如果该最新操作标识大于数据换入请求中携带的回放进度标识S_MAX,说明第三地址信息对应的预写操作日志仍保存在当前过滤器对应的存储空间,尚未进行回放或正在进行回放,第三地址信息对应的数据A即将被修改或正在被修改,则网络处理器忽略该数据换入请求,不执行数据换入,以避免将旧数据或部分旧数据换入加速设备的缓存区域。
当步骤S1202中得到多个最新操作标识时,如果每个最新操作标识均大于数据换入请求中携带的回放进度标识S_MAX,则可以确定第三地址信息对应的最新操作标识大于回放进度标识S_MAX,说明第三地址信息对应的数据A即将被修改或正在被修改,网络处理器可以忽略该数据换入请求,不执行数据换入。上文中已经介绍,由于不同的地址信息经过hash运算后可能会得到相同的值,如果任意一个最新操作标识小于数据换入请求中携带的回放进度标识,那么,大于回放进度标识的最新操作标识就可能不是第三地址信息对应的写操作指令的编号,而有可能是其他地址信息对应的写操作指令的编号,因此,可以根据小于回放进度标识的最新操作标识,认为不存在第三地址信息对应的尚未回放的预写操作日志,网络处理器可以执行该数据换入操作,根据第三地址信息,将数据存储区域保存的数据A保存至加速设备的缓存区域。
在另一种实施例中,步骤S1202中,如果网络处理器得到第三地址信息对应的多个hash值,根据多个hash值,在当前过滤器中可以确定多个第二目标位置。网络处理器可以分别获取多个第二目标位置中的每个第二目标位置所保存的写操作指令的编号,得到多个写操作编号,将多个写操作编号中最小的写操作编号作为最新操作标识。如果最新操作标识小于回放进度标识,则执行该数据换入请求,如果最新操作标识大于回放进度标识,则忽略该数据换入请求,不执行数据换入。
上述实施例中,网络处理器利用单调递增的操作标识,通过将最新操作标识与回放进度标识进行比较,确定某个地址信息是否存在尚未进行回放的最新预写操作日志,如果存在尚未进行回放的最新预写操作日志,则不执行该地址信息对应的数据换入请求,以避免将即将被修改的旧数据换入到加速设备的缓存区域中,针对同一地址信息,可以使加速设备的缓存区域保存的数据与存储设备的数据存储区域保存的数据保持一致性。
与上述方法实施例基于相同的技术构思,本申请实施例中还提供一种数据存储装置。该数据存储装置可以设置加速设备的网络处理器内,加速设备可以连接在主机和存储设备之间。在一些实施例中,如
图13所示,该数据存储装置1300可以包括日志生成单元1301和日志保存单元1302。数据存储装置1300可以用于实现上述方法实施例的功能,因此可以实现上述方法实施例所具备的有益效果。
其中,日志生成单元1301,可以用于接收加速设备中的接口转发的主机的写操作指令,并基于写操作指令生成预写操作日志;日志保存单元1302,可以用于将预写操作日志保存至存储设备中的日志存储区域;预写操作日志用于在通过存储设备的存储控制器进行回放时,将写操作指令中携带的待写入数据保存至存储设备中的数据存储区域。
在一些实施例中,日志保存单元1302,还可以用于:若确定预写操作日志保存完成,则向主机发送操作执行完成通知。
在一些实施例中,日志保存单元1302,具体可以用于:将预写操作日志保存至日志存储区域中的第一存储空间;预写操作日志中包括写操作指令中携带的第一地址信息;第一存储空间为至少一个存储空间中的任一存储空间;根据预写操作日志中的第一地址信息,更新第一存储空间对应的第一过滤器;更新后的第一过滤器用于表征第一地址信息对应的预写操作日志保存在第一存储空间内,等待通过存储设备的存储控制器进行回放。
在一些实施例中,第一过滤器保存在加速设备中。数据存储装置1300还可以包括读指令执行单元,读指令执行单元与日志保存单元1302连接。读指令执行单元可以用于:接收到主机发送的读操作指令,获取读操作指令中携带的第二地址信息;读操作指令用于指示根据第二地址信息从数据存储区域获取数据;若基于第一过滤器,确定第二地址信息对应的预写操作日志保存在第一存储空间内,则向存储设备的存储控制器发送延迟读通知,延迟读通知用于指示存储控制器等待第二地址信息对应的预写操作日志回放完成后,执行读操作指令。
在一些实施例中,日志保存单元1302,还可以用于:若确定第一存储空间已满,则将第一过滤器保存至存储设备中。
在一些实施例中,读指令执行单元,还可以用于:若基于第一过滤器,确定第二地址信息对应的预写操作日志未保存在第一存储空间内,则将读操作指令发送至存储设备的存储控制器,以使存储控制器基于存储设备中的过滤器,确定第二地址信息对应的预写操作日志是否保存在日志存储区域,等待通过存储设备的存储控制器进行回放。
在一些实施例中,所述预写操作日志中包括所述写操作指令的编号;日志保存单元1302,具体可以用于:根据第一地址信息,在第一过滤器中确定至少一个第一目标位置,并将写操作指令的编号分别保存至至少一个第一目标位置中的每个第一目标位置。
在一些实施例中,所述写操作指令的编号为按照所述网络处理器接收到各个写操作指令的时间顺序递增的编号。数据存储装置1300还可以包括数据换入单元,数据换入单元与日志保存单元连接。数据换入单元可以用于:接收到存储设备的存储控制器发送的数据换入请求,获取数据换入请求中携带的第三地址信息和回放进度标识;回放进度标识指存储控制器已完成回放的预写操作日志中包含的最大编号;数据换入请求用于指示根据第三地址信息,将数据存储区域保存的数据保存至加速设备的缓存区域;根据第三地址信息,从第一过滤器中获取最新操作标识;最新操作标识指在接收到数据换入请求之前,最后一次接收到的第三地址信息对应的写操作指令的编号;若最新操作标识大于回放进度标识,则忽略数据换入请求。
在一些实施例中,数据换入单元,具体可以用于:根据第三地址信息,在第一过滤器中确定至少一个第二目标位置;分别获取至少一个第二目标位置中的每个第二目标位置所保存的写操作指令的编号,得到至少一个写操作编号;将至少一个写操作编号中,最小的写操作编号作为最新操作标识。
在一些实施例中,日志保存单元1302,具体可以用于:分别将预写操作日志保存至存储设备中的主日志存储区域和备日志存储区域。
与图7所示的方法实施例基于相同的技术构思,本申请实施例中还提供一种芯片,该芯片可以是计算芯片,该芯片可以应用于上述实施例的网络处理器中。该芯片可以用于实现图7所示的方法实施例的功能,因此可以实现上述方法实施例所具备的有益效果。
在一些实施例中,该芯片1400的结构可以如图14所示,包括处理器1401以及与处理器1401连接的供电电路1402。处理器1401和供电电路1402之间可以通过总线相互连接,处理器1401可以是数字信号处理器(digital signal processor,DSP)、ASIC、现场可编程门阵列(field programmable gate array,FPGA)或者其他可编程逻辑器件、分立门或者晶体管逻辑器件、分立硬件组件或其他特定的集成电路
等。总线可以是外设部件互联标准(peripheral component interconnect,PCI)总线或扩展工业标准结构(extended industry standard architecture,EISA)总线等。总线可以分为地址总线、数据总线、控制总线等。供电电路1402用于通过总线为处理器1401供电。
处理器1401可以与设置在芯片外部的存储器连接,或者与设置在芯片内部的存储器连接,运行存储在存储器中的软件程序以及模块,从而执行芯片1400的各种功能应用以及数据处理,如本申请实施例提供的数据存储方法。
在一些实施例中,处理器1401可以包括一个或多个处理单元,不同的处理单元可以是独立的器件,也可以集成在一个或多个处理器中。处理器1401中还可以包括控制器,控制器可以根据指令操作码和时序信号,产生操作控制信号,完成取指令和执行指令的控制。
与上述实施例基于相同的技术构思,本申请实施例中还提供一种加速设备,该加速设备可以连接在主机和存储设备之间。该加速设备可以用于实现上述方法实施例的功能,因此可以实现上述方法实施例所具备的有益效果。
在一些实施例中,该加速设备1500的结构可以如图15所示,包括接口1501以及与接口1501连接的网络处理器1502。接口1501和网络处理器1502可以通过总线相互连接,接口1501可以是总线接口、数据线接口或其他通信接口,用于与主机之间进行通信,接收主机发送的数据,并把接收的数据提供给网络处理器1502。网络处理器1502可以是支持可编程架构且具有远程直接数据存取能力的NP,网络处理器1502可以采用图14所示的芯片。总线可以是PCI总线或EISA总线等。总线可以分为地址总线、数据总线、控制总线等。
网络处理器1502可以与设置在加速设备外部的存储器连接,或者与设置在加速设备内部的存储器连接,运行存储在存储器中的软件程序以及模块,并与主机和存储设备交互。从而执行本申请实施例提供的数据存储方法。
在一些实施例中,网络处理器1502可以包括一个或多个处理单元,不同的处理单元可以是独立的器件,也可以集成在一个或多个处理器中。网络处理器1502中还可以包括控制器,控制器可以根据指令操作码和时序信号,产生操作控制信号,完成取指令和执行指令的控制。
可以理解的是,本申请实施例示意的结构并不构成对加速设备的具体限定。在本申请另一些实施例中,加速设备可以包括比图示更多或更少的部件,或者组合某些部件,或者拆分某些部件,或者不同的部件布置。图示的部件可以以硬件,软件或软件和硬件的组合实现。
与上述方法实施例基于相同的技术构思,本申请实施例中还提供一种数据存储系统,该数据存储系统的结构可以如图3所示,包括主机610、加速设备620和存储设备700。加速设备620连接在主机610和存储设备700之间,加速设备620可以包括网络处理器622,网络处理器622可以用于执行计算机程序,以实现上述方法实施例的功能。
本申请的实施例中的方法步骤可以通过硬件的方式来实现,也可以由处理器执行计算机程序或指令的方式来实现。计算机程序或指令可以构成计算机程序产品。
本申请实施例还提供一种计算机程序产品,包含有计算机可执行指令。在一种实施例中,该计算机可执行指令用于使计算机执行上述方法实施例中的功能。
计算机可执行指令可以被存放于计算机可读存储介质中,本申请实施例还提供一种计算机可读存储介质,所述计算机可读存储介质内存储有可执行指令。在一种实施例中,该计算机可执行指令用于使计算机执行上述方法实施例中的功能。
本申请实施例提供的计算机可读存储介质可以是随机存取存储器(random access memory,RAM)、闪存、只读存储器(read-only memory,ROM)、可编程只读存储器(programmableROM,PROM)、可擦除可编程只读存储器(erasable PROM,EPROM)、电可擦除可编程只读存储器(electrically ePROM,EEPROM)、寄存器、硬盘、移动硬盘、CD-ROM或者本领域熟知的任何其它形式的计算机可读存储介质。
计算机可执行指令可以存储在计算机可读存储介质中,或者从一个计算机可读存储介质向另一个计算机可读存储介质传输,例如,所述计算机程序或指令可以从一个网站站点、计算机、服务器或数据中心通过有线或无线方式向另一个网站站点、计算机、服务器或数据中心进行传输。所述计算机可读存储介质可以是计算机能够存取的任何可用介质或者是集成一个或多个可用介质的服务器、数据中心等数据存储设备。所述可用介质可以是磁性介质,例如,软盘、硬盘、磁带;也可以是光介质,例如,数字视
频光盘(digital video disc,DVD);还可以是半导体介质,例如,固态硬盘。
以上模块或单元的一个或多个可以软件、硬件或二者结合来实现。当以上任一模块或单元以软件实现的时候,所述软件以计算机程序指令的方式存在,并被存储在存储器中,处理器可以用于执行所述程序指令并实现以上方法流程。所述处理器可以包括但不限于以下至少一种:CPU、微处理器、数字信号处理器(digital signal processor,DSP)、微控制器(microcontroller unit,MCU)、或人工智能处理器等各类运行软件的计算设备,每种计算设备可包括一个或多个用于执行软件指令以进行运算或处理的核。该处理器可以内置于SoC、DPU或ASIC,也可是一个独立的半导体芯片。该处理器内处理用于执行软件指令以进行运算或处理的核外,还可进一步包括必要的硬件加速器,如FPGA、PLD或者实现专用逻辑运算的逻辑电路。
当以上模块或单元以硬件实现的时候,该硬件可以是CPU、微处理器、DSP、MCU、人工智能处理器、ASIC、SoC、FPGA、PLD、专用数字电路、硬件加速器或非集成的分立器件中的任一个或任一组合,其可以运行必要的软件或不依赖于软件以执行以上方法流程。
以上所述,仅为本申请的具体实施方式,但本申请的保护范围并不局限于此,任何熟悉本技术领域的技术人员在本申请揭露的技术范围内,可轻易想到各种等效的修改或替换,这些修改或替换都应涵盖在本申请的保护范围之内。因此,本申请的保护范围应以权利要求的保护范围为准。
Claims (16)
- 一种数据存储方法,其特征在于,应用于加速设备,所述加速设备连接在主机和存储设备之间;所述方法包括:所述加速设备中的接口接收所述主机发送的写操作指令,把写操作指令转发给所述加速设备中的网络处理器;所述网络处理器用于执行以下操作:基于所述写操作指令生成预写操作日志;将所述预写操作日志保存至所述存储设备中的日志存储区域;所述预写操作日志用于在通过所述存储设备的存储控制器进行回放时,将所述写操作指令中携带的待写入数据保存至所述存储设备中的数据存储区域。
- 根据权利要求1所述的方法,其特征在于,所述方法还包括:若确定所述预写操作日志保存完成,则向所述主机发送操作执行完成通知。
- 根据权利要求1或2所述的方法,其特征在于,所述方法还包括:若所述接口接收到主机发送的非修改类操作指令,将所述非修改类操作指令发送至所述加速设备中的主处理器,或所述存储设备的存储控制器进行处理;所述非修改类操作指令包括除所述写操作指令之外的任一操作指令。
- 根据权利要求1~3中任一项所述的方法,其特征在于,所述日志存储区域包括至少一个存储空间;所述至少一个存储空间中的每个存储空间对应设置有一个过滤器;所述预写操作日志中包括所述写操作指令中携带的第一地址信息;所述将所述预写操作日志保存至所述存储设备中的日志存储区域,包括:将所述预写操作日志保存至第一存储空间,并更新所述第一存储空间对应的第一过滤器;所述第一存储空间为所述至少一个存储空间中的任一存储空间;更新后的第一过滤器用于表征所述第一地址信息对应的预写操作日志保存在所述第一存储空间内。
- 根据权利要求4所述的方法,其特征在于,所述第一过滤器保存在所述加速设备中,所述方法还包括:接收到主机发送的读操作指令,获取所述读操作指令中携带的第二地址信息;若基于所述第一过滤器,确定所述第二地址信息对应的预写操作日志保存在所述第一存储空间内,则向所述存储设备的存储控制器发送延迟读通知,所述延迟读通知用于指示所述存储控制器等待所述第二地址信息对应的预写操作日志回放完成后,执行所述读操作指令。
- 根据权利要求5所述的方法,其特征在于,所述方法还包括:若确定所述第一存储空间已满,则将所述第一过滤器保存至所述存储设备中。
- 根据权利要求5或6所述的方法,其特征在于,所述方法还包括:若基于所述第一过滤器,确定所述第二地址信息对应的预写操作日志未保存在所述第一存储空间内,则将所述读操作指令发送至所述存储设备的存储控制器进行处理。
- 根据权利要求4~7中任一项所述的方法,其特征在于,所述预写操作日志中包括所述写操作指令的编号;所述写操作指令的编号为按照所述网络处理器接收到各个写操作指令的时间顺序递增的编号;所述更新所述第一存储空间对应的第一过滤器,包括:根据所述第一地址信息,在所述第一过滤器中确定至少一个第一目标位置,并将所述写操作指令的编号分别保存至所述至少一个第一目标位置中的每个第一目标位置。
- 根据权利要求8所述的方法,其特征在于,所述方法还包括:接收到所述存储设备的存储控制器发送的数据换入请求,获取数据换入请求中携带的第三地址信息和回放进度标识;所述回放进度标识指所述存储控制器已完成回放的预写操作日志中包含的最大编号;所述数据换入请求用于指示根据所述第三地址信息,将所述数据存储区域保存的数据保存至所述加速设备的缓存区域;从所述第一过滤器中获取所述第三地址信息对应的最新操作标识;所述最新操作标识指在接收到所述数据换入请求之前,最后一次接收到的所述第三地址信息对应的写操作指令的编号;若所述最新操作标识大于所述回放进度标识,则忽略所述数据换入请求。
- 一种数据存储装置,其特征在于,应用于加速设备中的网络处理器,所述加速设备连接在主机和存储设备之间;所述装置包括:日志生成单元,用于接收所述加速设备中的接口转发的所述主机的写操作指令,并基于所述写操作指令生成预写操作日志;日志保存单元,用于将所述预写操作日志保存至所述存储设备中的日志存储区域;所述预写操作日志用于在通过所述存储设备的存储控制器进行回放时,将所述写操作指令中携带的待写入数据保存至所述存储设备中的数据存储区域。
- 根据权利要求10所述的装置,其特征在于,所述日志保存单元,还用于:在确定所述预写操作日志保存完成时,向所述主机发送操作执行完成通知。
- 根据权利要求10或11所述的装置,其特征在于,所述日志存储区域包括至少一个存储空间;所述至少一个存储空间中的每个存储空间对应设置有一个过滤器;所述预写操作日志中包括所述写操作指令中携带的第一地址信息;所述日志保存单元,具体用于:将所述预写操作日志保存至第一存储空间,并更新所述第一存储空间对应的第一过滤器;所述第一存储空间为所述至少一个存储空间中的任一存储空间;更新后的第一过滤器用于表征所述第一地址信息对应的预写操作日志保存在所述第一存储空间内。
- 根据权利要求12所述的装置,其特征在于,所述第一过滤器保存在所述加速设备中,所述装置还包括:读指令执行单元,用于接收到主机发送的读操作指令,获取所述读操作指令中携带的第二地址信息;若基于所述第一过滤器,确定所述第二地址信息对应的预写操作日志保存在所述第一存储空间内,则向所述存储设备的存储控制器发送延迟读通知,所述延迟读通知用于指示所述存储控制器等待所述第二地址信息对应的预写操作日志回放完成后,执行所述读操作指令。
- 一种芯片,其特征在于,包括处理器和供电电路;所述供电电路用于为所述处理器供电,所述处理器用于执行计算机程序,对通过接口获得的数据执行如权利要求1~9中任一项所述网络处理器所执行的方法。
- 一种加速设备,其特征在于,所述加速设备连接在主机和存储设备之间,所述加速设备包括接口和网络处理器;所述接口用于接收数据,把接收的数据提供给所述网络处理器;所述网络处理器用于执行权利要求1~9中任一项所述网络处理器所执行的方法。
- 一种数据存储系统,其特征在于,包括主机、存储设备和权利要求15所述的加速设备。
Applications Claiming Priority (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN202211115045.4A CN117742568A (zh) | 2022-09-14 | 2022-09-14 | 数据存储方法、装置、系统、网络处理器及加速设备 |
CN202211115045.4 | 2022-09-14 |
Publications (2)
Publication Number | Publication Date |
---|---|
WO2024055679A1 true WO2024055679A1 (zh) | 2024-03-21 |
WO2024055679A9 WO2024055679A9 (zh) | 2024-05-23 |
Family
ID=90259735
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
PCT/CN2023/102954 WO2024055679A1 (zh) | 2022-09-14 | 2023-06-27 | 数据存储方法、装置、系统、芯片及加速设备 |
Country Status (2)
Country | Link |
---|---|
CN (1) | CN117742568A (zh) |
WO (1) | WO2024055679A1 (zh) |
Citations (3)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN105787129A (zh) * | 2016-03-29 | 2016-07-20 | 联想(北京)有限公司 | 一种数据存储方法和电子设备 |
US20190188097A1 (en) * | 2017-12-15 | 2019-06-20 | Vmware, Inc. | Mirrored write ahead logs for data storage system |
CN110647511A (zh) * | 2019-09-27 | 2020-01-03 | 掌阅科技股份有限公司 | 数据同步方法、计算设备及计算机存储介质 |
-
2022
- 2022-09-14 CN CN202211115045.4A patent/CN117742568A/zh active Pending
-
2023
- 2023-06-27 WO PCT/CN2023/102954 patent/WO2024055679A1/zh unknown
Patent Citations (3)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN105787129A (zh) * | 2016-03-29 | 2016-07-20 | 联想(北京)有限公司 | 一种数据存储方法和电子设备 |
US20190188097A1 (en) * | 2017-12-15 | 2019-06-20 | Vmware, Inc. | Mirrored write ahead logs for data storage system |
CN110647511A (zh) * | 2019-09-27 | 2020-01-03 | 掌阅科技股份有限公司 | 数据同步方法、计算设备及计算机存储介质 |
Also Published As
Publication number | Publication date |
---|---|
WO2024055679A9 (zh) | 2024-05-23 |
CN117742568A (zh) | 2024-03-22 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
US11550819B2 (en) | Synchronization cache seeding | |
CN110119304B (zh) | 一种中断处理方法、装置及服务器 | |
US20200201578A1 (en) | Method and Apparatus for Transmitting Data Processing Request | |
US20190347167A1 (en) | Primary Node-Standby Node Data Transmission Method, Control Node, and Database System | |
US20200150903A1 (en) | Method for executing hard disk operation command, hard disk, and storage medium | |
WO2020199760A1 (zh) | 数据存储方法、存储器和服务器 | |
US20220164316A1 (en) | Deduplication method and apparatus | |
CN111158602A (zh) | 数据分层存储方法、读取方法、存储主机及存储系统 | |
WO2024041022A1 (zh) | 数据库表变更方法、装置、设备和存储介质 | |
TW201303870A (zh) | 利用快閃記憶體介面的方法及裝置 | |
US20240220334A1 (en) | Data processing method in distributed system, and related system | |
JP6944576B2 (ja) | キャッシュデバイス、命令キャッシュ、命令処理システム、データ処理方法、データ処理装置、コンピュータ可読記憶媒体及びコンピュータプログラム | |
WO2024055679A1 (zh) | 数据存储方法、装置、系统、芯片及加速设备 | |
CN112732176B (zh) | 基于fpga的ssd访问方法及装置、存储系统及存储介质 | |
CN116594551A (zh) | 一种数据存储方法及装置 | |
CN115970295A (zh) | 请求处理方法、装置和电子设备 | |
WO2022001133A1 (zh) | 一种提升软拷贝读性能的方法、系统、终端及存储介质 | |
CN115904211A (zh) | 一种存储系统、数据处理方法及相关设备 | |
WO2023125285A1 (zh) | 一种数据库系统更新方法及相关装置 | |
WO2022222523A1 (zh) | 一种日志管理方法及装置 | |
US12050623B2 (en) | Synchronization cache seeding | |
WO2024055660A1 (zh) | 一种数据运算方法、装置及设备 | |
WO2023029417A1 (zh) | 一种数据存储方法及装置 | |
WO2022126534A1 (zh) | 数据处理方法及相关设备 | |
US20240103897A1 (en) | Diversified virtual memory |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
121 | Ep: the epo has been informed by wipo that ep was designated in this application |
Ref document number: 23864418 Country of ref document: EP Kind code of ref document: A1 |