WO2023116438A1 - Procédé et appareil d'accès à des données, et dispositif - Google Patents

Procédé et appareil d'accès à des données, et dispositif Download PDF

Info

Publication number
WO2023116438A1
WO2023116438A1 PCT/CN2022/137163 CN2022137163W WO2023116438A1 WO 2023116438 A1 WO2023116438 A1 WO 2023116438A1 CN 2022137163 W CN2022137163 W CN 2022137163W WO 2023116438 A1 WO2023116438 A1 WO 2023116438A1
Authority
WO
WIPO (PCT)
Prior art keywords
data access
access request
hard disk
priority
storage device
Prior art date
Application number
PCT/CN2022/137163
Other languages
English (en)
Chinese (zh)
Inventor
朱晓明
玄在雄
Original Assignee
华为技术有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 华为技术有限公司 filed Critical 华为技术有限公司
Publication of WO2023116438A1 publication Critical patent/WO2023116438A1/fr

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
    • G06F3/0601—Interfaces specially adapted for storage systems
    • G06F3/0602—Interfaces specially adapted for storage systems specifically adapted to achieve a particular effect
    • G06F3/0604—Improving or facilitating administration, e.g. storage management
    • G06F3/0607—Improving or facilitating administration, e.g. storage management by facilitating the process of upgrading existing storage systems, e.g. for improving compatibility between host and storage device
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
    • G06F3/0601—Interfaces specially adapted for storage systems
    • G06F3/0628—Interfaces specially adapted for storage systems making use of a particular technique
    • G06F3/0653—Monitoring storage devices or systems
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
    • G06F3/0601—Interfaces specially adapted for storage systems
    • G06F3/0628—Interfaces specially adapted for storage systems making use of a particular technique
    • G06F3/0655—Vertical data movement, i.e. input-output transfer; data movement between one or more hosts and one or more storage devices
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
    • G06F3/0601—Interfaces specially adapted for storage systems
    • G06F3/0668—Interfaces specially adapted for storage systems adopting a particular infrastructure
    • G06F3/067—Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS]
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00—Arrangements for program control, e.g. control units
    • G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/46—Multiprogramming arrangements
    • G06F9/48—Program initiating; Program switching, e.g. by interrupt

Definitions

  • the present application relates to the field of storage technologies, and in particular to a data access method, device and equipment.
  • the data access requests initiated by some important users can be prioritized. Before reaching the storage system, the data access requests sent by the user through the client device will be adjusted in order. In this way, some data access requests initiated by important users can reach the storage system first and be processed by the storage system first.
  • the storage system needs to bear different loads at different times, and the load here can indicate the data access requests that the storage device needs to process.
  • the storage system has a heavy load and needs to process a large number of data access requests, just adjusting the order in which the data access requests arrive at the storage system cannot fully guarantee the processing efficiency of these data access requests.
  • the heavy load of the storage system even if the data access requests initiated by important users reach the storage system first, these data access requests cannot be processed in time because there are no idle nodes inside the storage system.
  • the present application provides a data access method, device and equipment, so as to ensure accurate control of the processing sequence for processing data access requests.
  • the embodiment of the present application provides a data access method, the method is applied to a storage device in a storage system, in this method, the storage device can receive a setting request, and the setting request is used to set the data access request Processing priority (the processing priority may be simply referred to as priority in this embodiment of the application), the processing priority describes the processing order of the data access request.
  • the setting request may be sent by a device other than the storage system.
  • the storage device may configure a priority list based on the setting request, and the priority list records processing priorities of different data access requests. Subsequently, when the storage device receives the data access request, the storage device may process the received data access request according to the priority list.
  • the storage device internally saves in the priority list.
  • each data access request can be processed in strict accordance with the priority list in sequence.
  • the order in which requests arrive at the storage device is more accurate and can accurately control the processing order of data access requests.
  • the processing priority may be set based on one or more granularities, and this embodiment of the present application does not limit the setting granularity of the processing priority.
  • the setting granularity includes, but is not limited to: the user who initiates the data access request, the application program that initiates the data access request, and the client device that initiates the data access request.
  • the above-mentioned setting granularity is just an example. In practical applications, the setting granularity can be selected according to specific scenarios, and then the processing priority can be set.
  • the granularity of setting the processing priority is more diverse, which can ensure that the setting of the processing priority can be applied to different data access scenarios.
  • the data access request received by the storage device carries identification information
  • the identification information includes part or all of the following: identification information of an application program, identification information of a user, and identification information of a client device.
  • the information carried in the identification information is related to the setting granularity of the processing priority.
  • the data access request carries identification information, which can identify the priority of the data access request to a certain extent.
  • the storage device when the storage device processes the received data access request according to the processing priority, it may parse the received data access request and extract the identification information in the data access request. After the storage device acquires the identification information, it can determine the processing priority of the data access request from the priority list according to the identification information. The storage device adjusts the processing order of the data access requests according to the processing priority. In this way, the storage device can sequentially process data access requests according to the processing order.
  • the storage device can accurately determine the processing priority of the data access request, further ensuring that the storage device can accurately process the data access request according to the processing priority.
  • the storage device includes multiple hard disks, and when the storage device sequentially processes the data access requests according to the processing sequence, the storage device may monitor the states of the multiple hard disks.
  • the storage device can monitor the states of the multiple hard disks in real time, and select a target hard disk for the data access request according to the states of the multiple hard disks.
  • the storage device may send monitoring commands to multiple hard disks respectively, and the monitoring commands are used to request the status of the hard disks. After receiving the monitoring command, the hard disk may feed back the state of the hard disk to the storage device.
  • the storage device selects a target hard disk for the data access request according to the states of the multiple hard disks. Afterwards, the storage device places the command generated based on the data access request in the submission queue of the target hard disk.
  • the storage device can select a target hard disk for a data access request according to the state of the hard disk, so as to ensure that the data access request can be processed efficiently.
  • the storage device when it selects a target hard disk for a data access request according to the states of multiple hard disks, it can select a target hard disk that satisfies the state condition from among the multiple hard disks.
  • the state condition includes the following parts or All: The bandwidth of the hard disk is greater than the bandwidth threshold, the latency of the hard disk is smaller than the latency threshold, and the hard disk is normal.
  • the performance of the target hard disk selected based on the state condition is better, which ensures the processing efficiency of the data access request.
  • the storage device may insert the command generated based on the data access request into the front position of the submission queue of the target hard disk, Such as sorting at the position of the first N (N is a positive integer) or the position of the head of the team.
  • the command generated based on the data access request may also be placed in a submission queue with a priority greater than a threshold among multiple submission queues of the target hard disk.
  • the storage device when the storage device places the command generated based on the data access request in the submission queue of the target hard disk, it will try to ensure that the hard disk can obtain the command generated based on the data access request from the submission queue as soon as possible to ensure that the data access request be processed as quickly as possible.
  • the storage device further instructs the target hard disk to suspend background operations on the target hard disk.
  • the hard disk can centrally process the commands in the submission queue to ensure the processing efficiency of data access requests.
  • the storage device when the storage device places the command generated based on the data access request in the submission queue of the target hard disk, if the target hard disk is busy, the storage device can place the command generated based on the data access request on the target hard disk.
  • the backup disk of the target hard disk refers to the hard disk that stores the same data as the target hard disk.
  • the command generated based on the data access request is processed by the backup disk instead of the target hard disk, to a certain extent, it can ensure that the command generated based on the data access request is processed in time, and the processing efficiency of the data access request is improved.
  • the command generated by the storage device based on the data access request may include a priority identifier, where the priority identifier is used to indicate the priority of the command.
  • the target hard disk After the target hard disk obtains the command from the submission queue, it can identify the priority identifier and adjust the processing order of each command inside the hard disk. The processing order of the commands can also be adjusted inside the hard disk, which further ensures the processing efficiency of the commands.
  • the embodiment of the present application also provides a data access right device, the data access right device has the function of realizing the behavior in the method example of the first aspect above, and the beneficial effects can be referred to the description of the first aspect and will not be repeated here repeat.
  • the functions described above may be implemented by hardware, or may be implemented by executing corresponding software on the hardware.
  • the hardware or software includes one or more modules corresponding to the above functions.
  • the structure of the device includes a receiving module, a processing module, and optionally a sending module and an indicating module. These modules can perform the corresponding functions in the method example of the first aspect above. For details, see Method The detailed description in the example is not repeated here.
  • the embodiment of the present application also provides a computing device, the computing device has the function of implementing the behavior in the method example of the first aspect above, and the beneficial effects can be referred to the description of the first aspect, which will not be repeated here.
  • the structure of the computing device includes a processor, a network card, and a memory, and the processor or the network card is configured to support the computing device to perform corresponding functions in the method of the first aspect above.
  • the memory coupled to the processor, holds program instructions and data necessary for the computing device.
  • the structure of the computing device also includes a communication interface for communicating with other devices, such as receiving setting requests and data access requests, sending commands based on data access requests, or sending instructions to the hard disk.
  • the present application also provides a computer-readable storage medium, where instructions are stored in the computer-readable storage medium, and when the computer-readable storage medium is run on a computer, the computer executes the above-mentioned first aspect and each possibility of the first aspect.
  • the present application further provides a computer program product including instructions, which, when run on a computer, cause the computer to execute the method described in the above first aspect and each possible implementation manner of the first aspect.
  • the present application also provides a computer chip, the chip is connected to the memory, and the chip is used to read and execute the software program stored in the memory, and implement the above first aspect and each possibility of the first aspect.
  • Fig. 1 is the structural representation of the submission queue and completion queue of a kind of hard disk provided by the present application
  • Fig. 2 is a schematic structural diagram of a multi-pair submission queue and completion queue system of a hard disk provided by the present application;
  • FIG. 3 is a schematic structural diagram of a data access system provided by the present application.
  • FIG. 4 is a schematic diagram of a data access method provided by the present application.
  • FIG. 5 is a schematic diagram of a priority list provided by the present application.
  • FIG. 6 is a schematic structural diagram of a data access device provided by the present application.
  • the embodiment of the present application provides a data access method, device and device.
  • the connection between the storage device and the hard disk involved in the embodiment of the present application A brief description of the interactive mode.
  • the interaction mode between the storage device and the hard disk is described by taking the hard disk in the storage device as an example of a solid state disk supporting the NVMe protocol.
  • solid-state drives that support the NVMe protocol can be referred to as NVMe SSDs for short.
  • NVMe protocol storage devices and SSDs interact based on a series of paired submission queues and completion queues. These queues are created by the driver running on the storage device, and these queues are shared between the driver and the NVMe SSD. The queue itself can be located either in the shared memory of the storage device or in the memory of the NVMe SSD. After the driver configures the submission queue and completion queue, the driver and NVMe SSD can interact. See Figure 1, the interaction steps are as follows:
  • step 1 the driver places a new command on the submission queue.
  • the command may be a data write command for requesting to write data, or a data read command for requesting to read data.
  • Step 2 the driver notifies the NVMe controller in the NVMe SSD that there is a new command to be executed by writing the tail pointer pointing to the end of the submission queue into the doorbell register.
  • step 3 the NVMe controller obtains the command from the submission queue.
  • the NVMe controller obtains commands from the submission queue, it generally starts to extract commands from the head of the submission queue.
  • Step 4 the NVMe controller processes the command. For example, the NVMe controller writes data into the flash memory in the SSD according to the command, or reads data from the flash memory in the SSD according to the command.
  • Step 5 the NVMe controller puts an entry in the associated completion queue after completing the command. If the command is used to request data to be written, an entry placed into the completion queue will indicate the success or failure of the data write. If the command is used to request to read data, the entry put into the completion queue will carry the read data.
  • step 6 the NVMe controller uses an interrupt mechanism to prompt the driver program that the command has been completed.
  • Step 7 the driver extracts the entry from the completion queue
  • Step 8 the driver sends the updated head pointer to the NVMe controller by writing it into the doorbell register.
  • the submission queue and the completion queue mentioned in the above steps exist in pairs, that is, the submission queue and the completion queue are associated one by one.
  • the submission queue and the completion queue are used to implement data reading and writing.
  • management queues which are used to pass control commands to complete some management operations, such as creating queues, deleting queues, and updating firmware.
  • the NVMe protocol allows up to 65535 separate queues, each of which can have up to 65535 entries.
  • the number of queues can be determined based on the system configuration and expected load of the storage device. That is to say, multiple queue pairs are allowed to exist in the storage device, and a queue pair includes a submission queue and a completion queue.
  • the driver can set different priorities for different queue pairs.
  • the NVMe controller can extract commands from each submission queue according to the priority of the submission queues, and process the commands.
  • FIG. 2 is a schematic diagram of a queue pair mechanism. As shown in FIG. 2 , multiple queue pairs can be created in a storage device, and there is a one-to-one correspondence between the submission queue and the completion queue in each queue pair.
  • FIG. 3 is a schematic diagram of a data access system architecture provided by an embodiment of the present application.
  • the data access system includes an application server 100 and a storage device 200 .
  • Data interaction can be performed between the application server 100 and the storage device 200 .
  • the interaction between the application server 100 and the storage device 200 can be divided into two types. One is the interaction performed when the application server 100 sets the priority of the data access request of the storage device 200 , and the other is the interaction performed when the application server 100 accesses the data stored in the storage device 200 .
  • the priority of data access requests refers to the order in which the storage device 200 processes data access requests. Data access requests with high priority need to be processed first, and data access requests with low priority need to be delayed or processed according to the normal order. sequential processing.
  • the application server 100 can instruct the storage device 200 to set the priority of the data access request by sending a setting request to the storage device 200; ask.
  • the data access request includes a data write request for requesting to write data, and a data read request for requesting to read data.
  • the storage device 200 may set the priority of the data access request under the instruction of the application server 100, and process the data access request based on the priority.
  • the application server 100 may be a device running applications such as client software and deployed near the user side.
  • the application server 100 may also be called a client device 300 .
  • the user can trigger the application server 100 to generate a data access request or a setting request through an application program running on the application server 100 according to his own needs or the current scenario.
  • the application server 100 can also be an intermediate device between the client device 300 and the storage device 200, the intermediate device can forward the data access request from the client device 300 for accessing the data in the storage device 200, and can also Generate and send a data access request or setting request triggered by the user.
  • FIG. 3 only exemplarily shows the case where the application server 100 is an intermediate device.
  • the application server 100 may be a physical machine or a virtual machine.
  • Physical machines include, but are not limited to, desktop computers, servers, laptops, and mobile devices.
  • the storage device 200 is a device having both computing capability and storage capability, and the storage device 200 may be a server, a desktop computer, or the like.
  • the embodiment of the present application does not limit the location where the storage device 200 is deployed.
  • the storage device 200 may be deployed in the cloud, for example, in an edge data center, or in a central cloud data center.
  • the storage device 200 may receive a data access request or a setting request from the application server 100, process the data access request or a setting request, and perform corresponding operations.
  • the storage device 200 may set the priority of the data access request according to the setting request. And process data access requests based on the set priority.
  • the storage device 200 includes at least one processor 210 , a memory 220 , a network card 230 and a hard disk 240 .
  • the processor 210, the memory 220, the network card 230 and the hard disk 240 are connected through a bus.
  • the processor 210 and the memory 220 are used to provide computing resources.
  • the processor 210 is a central processing unit 210 (central processing unit, CPU).
  • FIG. 3 only shows two CPUs, and the embodiment of the present application does not limit the number of CPUs, which may be one or multiple.
  • the memory 220 refers to an internal memory directly exchanging data with the processor 210. It can read and write data at any time, and the speed is very fast. It serves as a memory for storing program instructions of an operating system or other running programs.
  • the memory 220 includes at least two types of memory, for example, the memory 220 can be a random access memory, such as a dynamic random access memory (DRAM), a storage class memory (storage class memory, SCM), or a read-only memory. Memory (read only memory, ROM).
  • the memory 220 may also include other random access memories, such as static random access memory (static random access memory, SRAM) and the like.
  • the memory 220 can also be a dual in-line memory module or a dual in-line memory module (DIMM).
  • the hard disk 240 is used to provide storage resources, such as storing data. It can be a magnetic disk or other types of storage media, such as a solid state disk 240 (SSD) or a shingled magnetic recording hard disk 240 .
  • SSD solid state disk 240
  • shingled magnetic recording hard disk 240 a solid state disk 240 (SSD) or a shingled magnetic recording hard disk 240 .
  • the network card 230 is used to communicate with the application server 100.
  • the network card 230 can receive messages from the application server 100.
  • the network card 230 has data processing functions, such as the network card 230 is an intelligent network card, the network card 230 can also perform some processing operations instead of the processor 210. .
  • the processor 210 can process the setting request and data access request inside the storage device 200, and the network card 230 only forwards the setting request and data access request to Processor 210.
  • the processor 210 invokes program instructions in the memory 220 to process the setting request.
  • the network card 230 may also analyze and process the configuration request and the data access request. In this processing manner, the configuration request and the data access request are not transmitted to the processor 210 , which can reduce the occupation of the processor 210 .
  • the network card 230 and the processor 210 may cooperate to process the request message. For example, the network card 230 may process the data access request, forward the data access request to the hard disk 240 to implement data access, and the processor 210 processes the setting request.
  • the storage device 200 can process data access requests according to priority, and the adjustment of the processing sequence of data access requests is more standardized and efficient, and can better guarantee the time delay of processing data access requests.
  • the following describes the data access method provided by the embodiment of the present application in conjunction with FIG. 4. As shown in FIG. 4, it is a data access method provided by the embodiment of the present application.
  • the method includes:
  • Step 401 the application server 100 sends a setting request to the storage device 200, and the setting request is used for requesting to set the priority of the data access request of the storage device 200.
  • the application program running on the application server 100 can be operated to trigger the application server 100 to generate a setting request, which is used to set the priority of the data access request of the storage device 200. class.
  • the application server 100 may send the setting request to the storage device 200 .
  • the priority of the data access request refers to the priority order that the storage device 200 needs to follow in processing the data access request.
  • the data access request may be generated by the client device 300 triggered by the user through an application program.
  • the video playing application can be deployed on different client devices 300 such as a user's mobile terminal, a smart home device (such as a TV), a desktop computer, and a notebook computer.
  • Applications deployed on different types of client devices 300 may be different.
  • Applications deployed on mobile terminals are different from applications deployed on laptops.
  • the user can enter the account number in the application program and log in to the video playback platform.
  • the user watches the video by clicking on the video view or text information on the video playback platform.
  • the client device 300 where the application is located will be triggered to generate a data access request.
  • the data access request requested The data of is the video that the user needs to watch. Considering the source of the data access request from different perspectives, the data access request may be considered to be initiated by the user, the application program, or the client device 300 .
  • the priority configuration granularity of the data access request can be configured according to the granularity of the user, application program, or client device 300 . That is to say, the priority of the data access request may also be set based on some or all of the user, the application program, and the client device 300 .
  • the application server 100 is allowed to set the priority of data access requests based on different granularities. Users set priorities for data access requests initiated by granular pairs.
  • the file management application program can be deployed on different client devices 300 such as a user's mobile terminal, desktop computer, and notebook computer. Applications deployed on different types of client devices 300 may be different.
  • the file management application deployed on the mobile terminal can be a personal-level file management software, which is suitable for users to manage personal and private files.
  • users who log in to the file management application through a mobile terminal are personal user.
  • the application program for file management of desktop computers deployed in a specific area may be enterprise-level file management software, which is suitable for internal personnel of the enterprise to manage internal files of the enterprise. For the enterprise, different personnel have different levels, and high-level enterprise personnel can manage files with higher confidentiality levels, or can manage a wider range of files.
  • the data access request can be considered to be initiated by the user (such as an enterprise-level user or an individual user), or it can be considered to be initiated by the application program (enterprise-level file management software or personal-level management software), and may also be initiated by users of different levels in the enterprise.
  • the user such as an enterprise-level user or an individual user
  • the application program enterprise-level file management software or personal-level management software
  • User priority can be understood as the priority configured for data access requests initiated by different users, such as high priority for important users and low priority for ordinary users.
  • Application priority can be understood as the priority configured for data access requests initiated by different applications, such as high priority for enterprise-level applications and low priority for personal-level applications.
  • Service priority can be understood as the priority configured for data access requests initiated by users of different levels. For example, high-level users are configured with high priority, low-level users are configured with low priority, and high-paying users are configured with high priority without additional payment. The configuration of fees has low priority.
  • the above configuration granularity of the priority is just an example, and the embodiment of the present application does not limit the specific expression form of the priority configuration granularity, and different priority configuration granularities may be used in different scenarios.
  • a user management module 110 is deployed in the application server 100 .
  • the user management module 110 can be understood as a software module running on the application server 100 .
  • the embodiment of the present application does not limit the presentation form of the software module.
  • the user management module 110 may be called in the form of an application programming interface (application programming interface, API), or in the form of an API of a communication protocol.
  • API application programming interface
  • the user management module 110 is used to implement priority setting.
  • the application server 100 may generate a setting request for setting priority by calling the user management module 110 , and send the setting request to the storage device 200 .
  • the user can configure the priority through the client device 300 , for example, the user can obtain high-priority permissions by paying.
  • the client device 300 may send the priority configured by the user to the application server 100 , or the application server 100 may collect the priority configured by the user on the client device 300 .
  • the application server 100 may call the user management module 110 to generate a setting request, and send the setting request to the storage device 200 .
  • the operation and maintenance personnel can also directly operate the application server 100 to configure the priority. For example, the operation and maintenance personnel can adjust the priority of different client devices 300 or the priority of different application programs.
  • the client device 300 may call the user management module 110 according to the configuration of the operation and maintenance personnel, generate a setting request, and send the setting request to the storage device 200 .
  • the application server 100 may be the client device 300, that is, the application server 100 is directly deployed on the user side, and can send a data access request when triggered by the user.
  • the application server 100 has a priority setting function.
  • the application server 100 implements the priority setting function, it can only set the priority for sending data access requests for applications deployed on the application server 100, and send data access requests for the application server 100.
  • the priority is set for the access request, or the priority is set for the data access request sent by the user who logs in the client software on the application server 100 .
  • the application server 100 may also set priorities for data access requests sent by various applications, set priorities for data access requests sent by different client devices 300 , or set priorities for data access requests sent by different users.
  • the application server 100 can be regarded as a priority management device, and the set priority is not limited to the data access request sent by itself, but can also be set for the data access right request initiated outside the application server 100. class.
  • the application server 100 is used as a priority management device as an example for description.
  • the application server 100 may only set the priority of some data access requests. For example, the application server 100 may set priorities for data access requests initiated by some applications, some client devices 300 or some users. The application server 100 can also set the priority of all data access requests. For example, the application server 100 may also set priorities for data access requests initiated by all applications, all client devices 300 or all users.
  • the application server 100 and the storage device 200 can use numerical values to represent the priority of different data access requests initiated by different applications.
  • the priorities are 1, 2, 3, 4.
  • the application server 100 and the storage device 200 can use English to represent different priorities of data access requests initiated by different users, and the priorities are Ex (extreme high priority), Ui (ultra high priority), Ge (general priority).
  • the embodiment of the present application does not limit the representation manner of the priority.
  • the setting request initiated by the application server 100 may carry an identifier for the application program, client device 300 or The user's information, and the priority of the application, client device 300 or user.
  • the information identifying the application, the client device 300 or the user may be simply referred to as identification information of the application, the client device 300 or the user.
  • the embodiment of the present application does not limit the specific type of the identification information, and any information that can uniquely identify the application program, the client device 300 or the user is applicable to the embodiment of the present application.
  • the setting request initiated by the application server 100 may carry the identification of the multiple applications, Information of multiple client devices 300 or multiple users, and priorities of the multiple applications, multiple client devices 300 or multiple users.
  • the information of the plurality of applications, the plurality of client devices 300 or the plurality of users, and the priorities of the plurality of applications, the plurality of client devices 300 or the plurality of users can be integrated into a priority list (table), so that It is parsed on the storage device 200.
  • the priority set by the application server 100 is not fixed, and in the embodiment of the present application, the application server 100 is allowed to dynamically adjust the priority of some or all data access requests.
  • the application server 100 may generate a setting request for setting priority, and the setting request includes but is not limited to:
  • a setting request for initializing the priority such as Init(priority table[user,app,level]), where user is used to indicate the user, app is used to indicate the application program, and level is used to indicate the priority, such as in numbers or English
  • the ID indicates priority.
  • a setting request for updating priority such as update(priority tabe[user,app,level]).
  • Step 402 the storage device 200 receives the setting request, and updates the priority list according to the setting request.
  • a priority list may be stored in the storage device 200, and the priority list may record the priority of the data access request. Based on the foregoing description, it can be seen that from the perspective of the source of the data access request, the priority of the data access request can be divided into priorities of different granularities, such as the priority of different users, the priority of different applications, and the priority of different client devices 300 class. The priority list can record the priorities of different granularities.
  • FIG. 5 it is an abstract schematic diagram of a priority list stored in the storage device 200 .
  • priorities are recorded at the granularity of user and application level. Based on the priority of the user and the application level, the storage device 200 may obtain the final overall priority of the data access request in a weighted summation manner.
  • the storage device 200 may update the priority list according to the information carried in the setting request. Such as deleting the user in the priority list, the priority of the client device 300 or the application program, modifying the user in the priority list, the priority of the client device 300 or the application program, or adding a new user or client in the priority list The priority of the end device 300 or application.
  • the priority list can be stored in the network card 230, so that the network card 230 can use the priority list to process the received data access request.
  • the priority list stored inside the network card 230 may be updated according to the setting request.
  • the priority list can be stored in the memory 220 .
  • the processor 210 may update the priority list in the memory 220 according to the setting request.
  • the priority list can be stored in the network card 230, so that the network card 230 can use
  • the priority list handles incoming data access requests.
  • the storage device 200 may also store the same priority list in the memory 220 . In this way, when the priority list in the network card 230 becomes invalid or damaged, the priority list stored in the memory 220 can be used to restore the priority list in the network card 230 .
  • the processor 210 may instruct the network card 230 to update the priority list in the network card 230 . If a priority list is also stored in the memory 220, the processor 210 may also request to update the priority list in the memory 220 according to the setting.
  • the storage device 200 may process the data access request based on the updated priority list. For the manner in which the storage device 200 processes the data access request, reference may be made to steps 403 to 404 .
  • Step 403 The application server 100 generates a data access request triggered by the user, and the data access request carries identification information, and the identification information includes part or all of the following: identification information of the application program, identification information of the user, and the client device 300 Identification information.
  • the application server 100 can detect the user's operation, identify the information of the user, and the information of the application program operated by the user.
  • the application server 100 When the application server 100 generates a data access request, it can be in the data Identification information is carried in the access request.
  • the application server 100 may retain the identification information carried in the data access request, and The data access request is sent to the storage device 200 , and the identification information includes application program identification information, user identification information, or client device 300 identification information.
  • the identification information carried by the application server 100 is related to the granularity of the priority of the data access request of the storage device 200.
  • the priority of the data access request of the storage device 200 is only set at the granularity of the user and the application program.
  • the application server 100 may only need to ensure that the data access request carries the identification information of the user and the identification information of the application program.
  • the priority of the data access request of the storage device 200 is only set at the granularity of the user, and the application server 100 may carry the identification information of the user in the data access request.
  • Data access requests include, but are not limited to:
  • a data read request for reading data such as Read IO(info[user,app,address,etc.]), where Read IO indicates that the data access request is a data read request, and info indicates that the data read request can
  • the information carried, the information carried in the data read request may include the information in square brackets, user is used to indicate the user, app is used to indicate the application program, and address is used to identify the address of the client device 300 .
  • a data write request for writing data such as Write IO(info[user,app,address,etc.]), where Read IO indicates that the data access request is a data write request.
  • info info[user,app,address,etc.]
  • Read IO indicates that the data access request is a data write request.
  • Step 404 the storage device 200 receives the data access request, parses the data access request, and processes the data access request based on the priority list.
  • the storage device 200 may determine the priority of the data access request according to the identification information carried in the data access request and the priority list stored inside the storage device 200 .
  • the storage device 200 may sequentially process each data access request according to the priority. For example, for a data access request with a higher priority, that is, the priority of the data access request is higher than the first threshold, the storage device 200 may preferentially process the data access request, preferentially select the hard disk 240 for processing the data access request, and instruct the hard disk 240 processes the data access request. It should be noted that the hard disk 240 does not directly process the data access request, but processes commands generated based on the data access request and placed in the submission queue of the hard disk 240 .
  • the storage device 200 may delay processing the data access request or The receive sequence processes the data access request normally.
  • the storage device 200 When the storage device 200 processes each data access request sequentially according to the order of priority, for any data access request, the storage device 200 can select the hard disk 240 for processing the data access request for the data access request, and will generate the data based on the data access request.
  • the command is placed in the submission queue of the hard disk 240 .
  • the storage device 200 may also adopt other methods to ensure the priority of data access requests.
  • the storage device 200 can monitor the status of each hard disk 240 in the storage device 200, and can obtain the status of each hard disk 240 (such as bandwidth, time delay, whether it works normally, etc.) in real time.
  • the storage device 200 may send a monitoring command to the hard disk 240, and the monitoring command is used to request the status of the hard disk 240.
  • the hard disk 240 recognizes the monitoring command and feeds back a monitoring response to the storage device 200.
  • the status of the hard disk 240 is carried in the monitoring response.
  • the storage device 200 may send a monitoring command for requesting the bandwidth of the hard disk 240 to the hard disk 240, and after receiving the monitoring command, the hard disk 240 may send the hard disk 240 The bandwidth carry is fed back to the storage device 200 in the monitoring response.
  • the storage device 200 can obtain the state of the hard disk 240 conveniently and quickly through the monitoring command, so as to ensure the real-time performance of the obtained state of the hard disk 240 .
  • the storage device 200 may preferentially process the data access request in some or all of the following manners:
  • Method 1 The storage device 200 selects the hard disk 240 whose status meets the status condition according to the status of each hard disk 240 in the storage device 200, and places the command generated based on the data access request in the submission queue of the hard disk 240.
  • the bandwidth of the hard disk 240 is greater than the bandwidth threshold, the delay of the hard disk 240 is less than the delay threshold, and the hard disk 240 is normal.
  • the storage device 200 may insert the command generated based on the data access request into the front position of the submission queue of the hard disk 240. For example, storage device 200 may insert the command at the head of the commit queue. For another example, the storage device 200 may insert the command at the end of the submission queue, but add an identifier to the command, where the identifier is used to indicate that the command is processed first.
  • Mode 3 the storage device 200 may place the command generated based on the data access request in the submission queue with a higher priority among the multiple submission queues of the hard disk 240 .
  • the storage device 200 may instruct the hard disk 240 to suspend some or all background operations of the hard disk 240 .
  • Background operations refer to operations other than data reading and writing in the hard disk 240 .
  • Background operations include but are not limited to garbage collection, background inspection, data rereading, wear leveling, etc.
  • the storage device 200 may place the command in the submission queue of the backup disk of the hard disk 240 .
  • the backup disk of the hard disk 240 refers to the hard disk 240 that stores the same data as the hard disk 240 .
  • the storage device 200 adds a priority identifier to the command generated based on the data access request, where the priority identifier is used to identify the priority of the command.
  • the priority identifier may be determined based on the priority of the data access request.
  • the priority identification is identifiable by the hard disk 240, that is to say, when the hard disk 240 obtains each command that needs to be processed from the submission queue, the hard disk 240 can identify the priority identification carried in the command, and according to the priority carried in the command Level IDs process the commands in the submission queue sequentially.
  • the storage device 200 may only add priority identifiers to some commands. For example, for a data access request with the highest priority or a priority greater than a set value, the storage device 200 may add a priority identifier to a command generated based on the data access request. Add the priority mark in . For example, for the data access requests of the first priority and the second priority, the storage device may add a priority identifier to the command generated based on the data access request of the first priority, so as to indicate that the priority of the command is First. The storage device may add a priority identifier to a command generated based on the data access request of the second priority, to identify that the priority of the command is second.
  • the processing sequence of the command can be advanced, and the command with the first priority is processed first, and then the command with the second priority is processed. Commands without priority flags are processed later.
  • the storage device 200 side may be pre-configured with command formats of different priorities, that is to say, different priorities have different command formats.
  • the storage device 200 may generate commands with corresponding priorities according to the priorities of the data access requests.
  • the storage device 200 places different commands in the submission queue of the hard disk 240, and the hard disk 240 obtains commands from the submission queue, identifies the priority of each command according to the format of the command, and processes each command according to the priority.
  • the embodiment of the present application also provides a data access device, the data access device is used to execute the method performed by the storage device in the method embodiment as shown in Figure 4 above, and the related features Reference may be made to the foregoing method embodiments, and details are not repeated here.
  • the data access device 600 includes a receiving module 601 and a processing module 602 .
  • the receiving module 601 is configured to receive a setting request, where the setting request is used to set the processing priority of the data access request, and the processing priority describes the processing order of the data access request.
  • the processing module 602 is configured to configure a priority list based on the setting request, and the priority list records the processing priorities of different data access requests; process the received data access requests according to the processing priorities.
  • the processing priority is set based on some or all of the following granularities: the user initiating the data access request, the application program initiating the data access request, and the client device initiating the data access request.
  • the data access request carries identification information
  • the identification information includes part or all of the following: identification information of an application program, identification information of a user, and identification information of a client device.
  • the processing module 602 may parse the received data access request and extract the identification information in the data access request.
  • the processing module 602 determines the processing priority of the data access request from the priority list according to the identification information.
  • the processing module 602 adjusts the processing sequence of the data access requests according to the processing priority; and processes the data access requests sequentially according to the processing sequence.
  • the storage device includes a plurality of hard disks, and the device further includes a sending module 603, and the sending module 603 can send a monitoring command to the plurality of hard disks respectively, and the monitoring command is used to request the status of the hard disk.
  • the receiving module 601 may acquire the states of the multiple hard disks fed back by the multiple hard disks.
  • the processing module 602 processes the data access requests sequentially according to the processing order, it can select a target hard disk for the data access request according to the status of multiple hard disks; and then place the command generated based on the data access request in the submission queue of the target hard disk.
  • the processing module 602 when the processing module 602 selects a target hard disk for a data access request according to the states of multiple hard disks, it can select a target hard disk that meets the state conditions according to the states of the multiple hard disks.
  • the state conditions include the following parts Or all: The bandwidth of the hard disk is greater than the bandwidth threshold, the latency of the hard disk is smaller than the latency threshold, and the hard disk is normal.
  • the processing module 602 may insert the command generated based on the data access request into the front row of the submission queue of the target hard disk.
  • Position such as inserting to the head of the queue, can also place the command generated based on the data access request in the submission queue with a priority greater than the threshold among the multiple submission queues of the target hard disk.
  • the device further includes an instruction module 604, where the instruction module 604 may instruct the target hard disk to suspend background operations on the target hard disk.
  • the processing module 602 when the processing module 602 places the command generated based on the data access request in the submission queue of the target hard disk, it may place the command generated based on the data access request in the submission queue of the backup disk of the target hard disk
  • the backup disk of the target hard disk refers to the hard disk that saves the same data as the target hard disk.
  • the command generated based on the data access request includes a priority identifier, where the priority identifier is used to indicate the priority of the command.
  • each functional module in the embodiment of the present application may be integrated into one processing module, each module may exist separately physically, or two or more modules may be integrated into one module.
  • the above-mentioned integrated modules can be implemented in the form of hardware or in the form of software function modules.
  • the above-mentioned embodiments may be implemented in whole or in part by software, hardware, firmware or other arbitrary combinations.
  • the above-described embodiments may be implemented in whole or in part in the form of computer program products.
  • the computer program product includes one or more computer instructions. When the computer program instructions are loaded or executed on the computer, the processes or functions according to the embodiments of the present invention will be generated in whole or in part.
  • the computer may be a general purpose computer, a special purpose computer, a computer network, or other programmable devices.
  • the computer instructions may be stored in or transmitted from one computer-readable storage medium to another computer-readable storage medium, for example, the computer instructions may be transmitted from a website, computer, server or data center Transmission to another website site, computer, server, or data center by wired (eg, coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (eg, infrared, wireless, microwave, etc.).
  • the computer-readable storage medium may be any available medium that can be accessed by a computer, or a data storage device such as a server or a data center that includes one or more sets of available media.
  • the available media may be magnetic media (eg, floppy disk, hard disk, magnetic tape), optical media (eg, DVD), or semiconductor media.
  • the semiconductor medium may be a solid state drive (SSD).
  • the embodiments of the present application may be provided as methods, systems, or computer program products. Accordingly, the present application may take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present application may take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) having computer-usable program code embodied therein.
  • computer-usable storage media including but not limited to disk storage, CD-ROM, optical storage, etc.
  • These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing apparatus to operate in a specific manner, such that the instructions stored in the computer-readable memory produce an article of manufacture comprising instruction means, the instructions
  • the device realizes the function specified in one or more procedures of the flowchart and/or one or more blocks of the block diagram.

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Human Computer Interaction (AREA)
  • Software Systems (AREA)
  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)

Abstract

La demande concerne un procédé et appareil d'accès à des données, et dispositif. Dans la présente demande, un dispositif de stockage reçoit une demande de réglage envoyée par un dispositif autre qu'un système de stockage ; la demande de réglage est utilisée pour régler des priorités de traitement de demandes d'accès à des données ; les priorités de traitement décrivent la séquence dans laquelle les demandes d'accès à des données sont traitées ; le dispositif de stockage configure une liste de priorités sur la base de la demande de réglage ; et lorsque le dispositif de stockage reçoit les demandes d'accès à des données, le dispositif de stockage traite les demandes d'accès à des données reçues selon la liste de priorités. Une liste de priorités est stockée dans un dispositif de stockage, et des demandes d'accès à des données sont traitées séquentiellement selon la liste de priorités lors du traitement des demandes d'accès à des données, de telle sorte que la séquence dans laquelle les demandes d'accès à des données sont traitées peut être commandée avec précision.
PCT/CN2022/137163 2021-12-21 2022-12-07 Procédé et appareil d'accès à des données, et dispositif WO2023116438A1 (fr)

Applications Claiming Priority (4)

Application Number Priority Date Filing Date Title
CN202111574333 2021-12-21
CN202111574333.1 2021-12-21
CN202210151597.4 2022-02-18
CN202210151597.4A CN116301568A (zh) 2021-12-21 2022-02-18 一种数据访问方法、装置以及设备

Publications (1)

Publication Number Publication Date
WO2023116438A1 true WO2023116438A1 (fr) 2023-06-29

Family

ID=86782029

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2022/137163 WO2023116438A1 (fr) 2021-12-21 2022-12-07 Procédé et appareil d'accès à des données, et dispositif

Country Status (2)

Country Link
CN (1) CN116301568A (fr)
WO (1) WO2023116438A1 (fr)

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20080215871A1 (en) * 2004-05-22 2008-09-04 Chan Kam-Fu Swapping "Fixed System" Hard Disk
CN103995674A (zh) * 2014-04-17 2014-08-20 华为技术有限公司 访问请求处理方法、装置和设备
CN109343862A (zh) * 2018-08-31 2019-02-15 北京三快在线科技有限公司 应用的资源数据的调度方法及装置
CN110532133A (zh) * 2019-07-16 2019-12-03 中国科学院国家天文台 一种低功耗大容量可扩充数据存储方法
CN112003915A (zh) * 2020-08-14 2020-11-27 苏州浪潮智能科技有限公司 一种文件的访问方法、装置、设备及可读介质

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20080215871A1 (en) * 2004-05-22 2008-09-04 Chan Kam-Fu Swapping "Fixed System" Hard Disk
CN103995674A (zh) * 2014-04-17 2014-08-20 华为技术有限公司 访问请求处理方法、装置和设备
CN109343862A (zh) * 2018-08-31 2019-02-15 北京三快在线科技有限公司 应用的资源数据的调度方法及装置
CN110532133A (zh) * 2019-07-16 2019-12-03 中国科学院国家天文台 一种低功耗大容量可扩充数据存储方法
CN112003915A (zh) * 2020-08-14 2020-11-27 苏州浪潮智能科技有限公司 一种文件的访问方法、装置、设备及可读介质

Also Published As

Publication number Publication date
CN116301568A (zh) 2023-06-23

Similar Documents

Publication Publication Date Title
CN109725829B (zh) 用于数据存储系统的端到端QoS解决方案的系统和方法
US9692846B2 (en) System, device and method for providing push service using feedback message
EP2284725A1 (fr) Client, serveur de courtage et procédé pour fournir le stockage de nuages
US11262916B2 (en) Distributed storage system, data processing method, and storage node
CN110046026B (zh) 云主机指定虚拟磁盘限速方法、计算设备及云平台
US20090019223A1 (en) Method and systems for providing remote strage via a removable memory device
US20170124021A1 (en) Methods for enabling direct memory access (dma) capable devices for remote dma (rdma) usage and devices thereof
US10552089B2 (en) Data processing for managing local and distributed storage systems by scheduling information corresponding to data write requests
WO2017088572A1 (fr) Procédé, dispositif et système de traitement de données
CN112130748A (zh) 一种数据访问方法、网卡及服务器
US11010094B2 (en) Task management method and host for electronic storage device
CN113032099B (zh) 云计算节点、文件管理方法及装置
US20170153909A1 (en) Methods and Devices for Acquiring Data Using Virtual Machine and Host Machine
CN111782318A (zh) 远程挂载本地磁盘到云桌面虚拟机中共享访问系统及方法
CN103944979A (zh) 一种消息推送方法、装置及系统
US10970236B2 (en) System and method for optimized input/output to an object storage system
EP4369171A1 (fr) Procédé et appareil de traitement d'une demande d'accès, et dispositif de stockage et support d'enregistrement
US10545667B1 (en) Dynamic data partitioning for stateless request routing
US11888690B2 (en) System and method for subscription limitation enforcement in distributed system
CN101699421B (zh) 串口共享的方法和服务端
CN113411363A (zh) 一种镜像文件的上传方法、相关设备及计算机存储介质
CN112148206A (zh) 一种数据读写方法、装置、电子设备及介质
US10776173B1 (en) Local placement of resource instances in a distributed system
US8938420B1 (en) Systems and methods for natural batching of I/O operations on a replication log
WO2023116438A1 (fr) Procédé et appareil d'accès à des données, et dispositif

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 22909753

Country of ref document: EP

Kind code of ref document: A1