WO2022016767A1 - Procédé et appareil de traitement de file d'attente de données - Google Patents

Procédé et appareil de traitement de file d'attente de données Download PDF

Info

Publication number
WO2022016767A1
WO2022016767A1 PCT/CN2020/133402 CN2020133402W WO2022016767A1 WO 2022016767 A1 WO2022016767 A1 WO 2022016767A1 CN 2020133402 W CN2020133402 W CN 2020133402W WO 2022016767 A1 WO2022016767 A1 WO 2022016767A1
Authority
WO
WIPO (PCT)
Prior art keywords
queue
data
processed
group
queues
Prior art date
Application number
PCT/CN2020/133402
Other languages
English (en)
Chinese (zh)
Inventor
陈国涛
Original Assignee
北京泽石科技有限公司
泽石科技(武汉)有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 北京泽石科技有限公司, 泽石科技(武汉)有限公司 filed Critical 北京泽石科技有限公司
Publication of WO2022016767A1 publication Critical patent/WO2022016767A1/fr

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00—Arrangements for program control, e.g. control units
    • G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/46—Multiprogramming arrangements
    • G06F9/54—Interprogram communication

Definitions

  • the present disclosure relates to the field of data processing, and in particular, to a data queue processing method and device.
  • multi-queue technology is one of the important ways to improve performance.
  • NVMe realizes the allocation of different queues according to tasks, scheduling priorities and CPU cores to complete high-performance storage functions.
  • the embodiments of the present disclosure provide a data queue processing method and device, so as to at least solve the technical problems of difficult scheduling and low efficiency in the multi-queue processing method in the related art.
  • a method for processing a data queue including: acquiring a queue group to be processed stored in a memory, wherein the queue group to be processed is a queue group including a data queue to be processed , the memory is set to receive data queue messages, and divide the data queues corresponding to the data queue messages into queue groups for storage; process the data queues in the queue groups to be processed, wherein the to-be-processed queue groups The number of data queues for a queue group does not exceed the maximum number of locally stored data queues.
  • the process before acquiring the queue group to be processed sent by the memory, the process includes: determining whether the number of received data queues exceeds the maximum number of locally stored data queues; when the number of received data queues does not exceed the local In the case of storing the maximum number of data queues, directly receive the data queue; in the case that the number of the received data queues exceeds the maximum number of locally stored data queues, receive the data queue through the memory; In the case where the storage receives the data queue, the step of acquiring the queue group to be processed stored in the storage is performed.
  • acquiring the queue group to be processed sent by the memory includes: determining whether the queue group is a queue group to be processed with tasks to be processed through a first icon mark, wherein the queue group includes the first A bitmap marker, the first bitmap marker of the queue group to be processed with tasks is different from the first bitmap marker of the queue group without tasks; the data queue of the queue group to be processed is obtained through the read interface, Wherein, the data structure of the queue group is the same as the data structure of the queue data stored locally.
  • processing the data queues in the queue group to be processed includes: determining the data queues to be processed in the queue group through a second bitmap mark, wherein the data queues include a second bitmap mark. Bitmap mark, the second bitmap mark of the data queue to be processed with task processing is different from the second bitmap mark of the data queue without tasks; determine whether the data queue to be processed is an order-preserving queue; If the data queue to be processed is an order-preserving queue, the data queue to be processed and the corresponding order-preserving data queue are processed according to the order of the order-preserving queue.
  • the method further includes: acquiring timestamps of data queues in the queue group; determining whether processing of the queue group times out according to the timestamps, and in the case of the queue group timing out, scheduling a memory
  • the cohort group was subjected to anti-starvation treatment.
  • the method further includes: when the data queue processing of the queue group is completed, sending processing attribute information of the data queue of the queue group to the memory, wherein the processing attribute information Including the processing time stamp of the data queue of the queue group, the memory determines whether the queue group is updated in the processing process according to the processing time stamp, and the data queue of the queue group is updated in the processing process. In this case, it is determined that there are tasks to be processed in the queue group, and the queue group is added to the storage queue group in sequence again.
  • a method for processing a data queue including: receiving a data queue message, and dividing the data queue corresponding to the data queue message into queue groups for storage; The group is sent to the processor, wherein the queue group to be processed is a queue group that includes data queues to be processed, and the number of data queues of the queue group to be processed does not exceed the number of data queues stored by the processor. The maximum number that the processor is set to process the queues in the queue group.
  • before receiving the data queue message it includes: determining whether the number of received data queues exceeds the maximum number of data queues stored by the processor; when the number of received data queues exceeds the processor In the case of storing the maximum number of data queues, perform the steps of receiving data queue messages.
  • a data queue processing apparatus including: an acquisition module configured to acquire a queue group to be processed stored in a memory, wherein the queue group to be processed includes a queue group to be processed The queue group of the processed data queue, the memory is set to receive the data queue message, and the data queue corresponding to the data queue message is divided into queue groups for storage; the processing module is set to the queue group to be processed.
  • the data queue is processed, wherein the number of data queues of the queue group to be processed does not exceed the maximum number of locally stored data queues.
  • a data queue processing device including: a storage module configured to receive data queue messages, and divide the data queues corresponding to the data queue messages into queue groups for storage;
  • the sending module is configured to send the queue group to be processed to the processor, wherein the queue group to be processed is a queue group including data queues to be processed, and the number of data queues of the queue group to be processed.
  • the processor is configured to process the queues in the queue group not exceeding the maximum number of queues for storing data by the processor.
  • a computer storage medium is also provided, where the computer storage medium includes a stored program, wherein when the program runs, a device where the computer storage medium is located is controlled to execute any one of the above The processing method of the data queue described in item.
  • a processor is further provided, and the processor is configured to run a program, wherein when the program runs, any one of the data queue processing methods described above is executed.
  • a queue group to be processed stored in a memory is acquired, wherein the queue group to be processed is a queue group including a data queue to be processed, and the memory is set to receive data queue messages, and correspond to the data queue messages.
  • the data queues are divided into queue groups for storage; the data queues in the queue group to be processed are processed, wherein the number of data queues in the queue group to be processed does not exceed the maximum number of locally stored data queues.
  • Multiple data queues are divided into queue groups for storage, and the processor processes the data queues in units of queue groups, which achieves the purpose of effectively and quickly processing multiple data queues, thus achieving the technical effect of improving the efficiency of multiple data queue processing. , and further solves the technical problems of the multi-queue processing method, difficult scheduling and low efficiency in the related art.
  • FIG. 1 is a flowchart of a method for processing a data queue according to an embodiment of the present disclosure
  • FIG. 2 is a flowchart of another method for processing a data queue according to an embodiment of the present disclosure
  • FIG. 3 is a schematic diagram of an NVMe queue according to an embodiment of the present disclosure.
  • FIG. 4 is a schematic diagram of the connection of a host and a controller according to an embodiment of the present disclosure
  • FIG. 5 is a schematic diagram of a queue and DDR according to an embodiment of the present disclosure.
  • FIG. 6 is a schematic diagram of a data structure in a DDR according to an embodiment of the present disclosure.
  • FIG. 8 is a schematic diagram of a data structure of sq_cq_info/10 of a queue according to an embodiment of the present disclosure
  • FIG. 10 is a schematic diagram of a queue replacement group according to an embodiment of the present disclosure.
  • 11 is a schematic diagram of a queue replacing a group when the number of queues is small according to an embodiment of the present disclosure
  • FIG. 12 is a schematic diagram of a processing apparatus for a data queue according to an embodiment of the present disclosure
  • FIG. 13 is a schematic diagram of another processing apparatus of a data queue according to an embodiment of the present disclosure.
  • a method embodiment of a method for processing a data queue is provided. It should be noted that the steps shown in the flowchart of the accompanying drawing can be executed in a computer system such as a set of computer-executable instructions, Also, although a logical order is shown in the flowcharts, in some cases the steps shown or described may be performed in an order different from that herein.
  • Fig. 1 is a flow chart of a method for processing a data queue according to an embodiment of the present disclosure, as shown in Fig. 1 , the method comprises the following steps:
  • Step S102 obtain the queue group to be processed stored in the memory, wherein, the queue group to be processed is a queue group including the data queue to be processed, the memory is set to receive the data queue message, and the data queue corresponding to the data queue message is arranged according to the first.
  • a predetermined sequence is divided into queue groups for storage;
  • Step S104 Process the data queues in the queue group to be processed, wherein the number of data queues in the queue group to be processed does not exceed the maximum number of locally stored data queues.
  • the queue group to be processed stored in the memory is acquired, wherein the queue group to be processed is a queue group including the data queue to be processed, the memory is set to receive the data queue message, and the data queue corresponding to the data queue message is set It is divided into queue groups for storage; the data queues in the queue group to be processed are processed, wherein the number of data queues in the queue group to be processed does not exceed the maximum number of locally stored data queues.
  • the queue is divided into queue groups for storage, and the processor processes the data queues in units of queue groups, so as to achieve the purpose of effectively and quickly processing multiple data queues, thereby achieving the technical effect of improving the efficiency of multiple data queue processing, and then solving the problem. It solves the technical problems of difficult scheduling and low efficiency in the multi-queue processing method in the related art.
  • the above-mentioned memory may be a double-rate synchronous dynamic random access memory (DDR) (Double Date Rate), which is arranged in the controller, and the execution body of the above-mentioned steps can be the queue module Queue in the above-mentioned controller, which is referred to as a queue for short.
  • DDR double-rate synchronous dynamic random access memory
  • the execution body of the above-mentioned steps can be the queue module Queue in the above-mentioned controller, which is referred to as a queue for short.
  • DDR double-rate synchronous dynamic random access memory
  • Double Date Rate Double Date Rate
  • the memory can store a large number of data queues, when the queue needs to process the data queue, the corresponding data queue is called from the memory.
  • the data queue in the memory has a large amount of data, which is inconvenient to store and read. Therefore, in this embodiment, the data queues in the memory are formed into queue groups in the first order, and the queue groups are stored in the second order.
  • the queue groups here can be received and processed by the queue at one time. Therefore, The number of data queues for a queue group to be processed does not exceed the maximum number of locally stored data queues.
  • the number of data queues in the above queue group is the maximum number of data queues that can be stored locally in the queue.
  • the migration with the maximum efficiency can be realized, the migration efficiency of the data queue can be improved, and the processing efficiency of the data queue can be further improved.
  • the above-mentioned data queues can be formed into queue groups according to the first order, and the above-mentioned first order can be to store the receiving order of the data queues.
  • the data queues are divided into queue groups. There is a precedence for the receiving order of the queue groups. Therefore, there will also be a precedence relationship between the formed queue groups.
  • the order of the queue groups is also the second order. The retrieval and migration of the data group, thereby improving the retrieval and migration efficiency of the data group, thereby improving the processing efficiency of the data queue.
  • before acquiring the queue group to be processed sent by the memory it includes: determining whether the number of received data queues exceeds the maximum number of locally stored data queues; when the number of received data queues does not exceed the locally stored data In the case of the maximum number of queues, the data queue is directly received; when the number of received data queues exceeds the maximum number of locally stored data queues, the data queues are received through the memory; in the case of receiving data queues through storage, the acquisition is performed
  • the memory stores the steps of the queue group to be processed.
  • the above-mentioned host sends the data queue to be processed to the processor or memory through the network connection through the doorbell. Specifically, if the number of data queues does not exceed the maximum number of data queues stored by the processor, it is directly sent to the processor; in the data queue If the number exceeds the maximum number of data queues stored by the processor, it is sent to the memory, and the processor obtains the data queues from the memory.
  • the processor processes the data queues in units of queue groups, the purpose of effectively and quickly processing multiple data queues is achieved, thereby improving the processing of multiple data queues.
  • the technical effect of high efficiency is further solved, and the technical problems of multi-queue processing method, difficult scheduling and low efficiency in the related art are solved.
  • Obtaining the queue group to be processed sent by the memory includes: determining whether the queue group is the queue group with the first order; if the queue group is the queue group with the first order, directly obtaining the queue group; if the queue group is not the queue group with the first order In this case, the queue group is scheduled by the scheduling module, so that the queue group is at the front of the queue group sequence, and the scheduled queue group is obtained.
  • the normal reading order is also according to the above second order.
  • the queue group at the front of the second order is not necessarily in the pending state. Therefore, in the queue group If it is not the queue group with the first order, the scheduling module will schedule the queue group, so that the queue group is at the top of the queue group order, and the scheduled queue group will be obtained.
  • the above-mentioned scheduling module can be set in the above-mentioned queue Queue, and can schedule and migrate the queue group at any time according to the requirements of the queue Queue.
  • acquiring the queue group to be processed sent from the memory includes: determining whether the queue group is a queue group to be processed with tasks to be processed by using a first icon mark, wherein the queue group includes the first icon mark , the first map mark of the queue group to be processed with tasks is different from the first map mark of the queue group without tasks; the data queue of the queue group to be processed is obtained through the read interface, wherein the data of the queue group is The structure is the same as the data structure of locally stored queue data.
  • processing the data queues in the queue group to be processed includes: determining the data queues to be processed in the queue group through a second bitmap mark, wherein the data queue includes a second bitmap mark and has tasks The second bitmap mark of the data queue to be processed is different from the second bitmap mark of the data queue without tasks; determine whether the data queue to be processed is an order-preserving queue; the data queue to be processed is an order-preserving queue. In this case, the data queue to be processed and the corresponding order-preserving data queue are processed according to the order of the order-preserving queue.
  • the above-mentioned first bitmap mark may be Qbitmap/12, and the second bitmap mark may be Qbitmap/8.
  • the head pointer and tail pointer of multiple data queues of the queue group stored in the memory can be determined; if the values of the head pointer and tail pointer of the data queue are not equal, determine the data The queue has tasks, and the queue group where the data queue is located has tasks.
  • the processor may update the second bitmap flag of the data queue, thereby determining the data queue to be processed in the queue group.
  • the memory can update the first map flag of the queue group where the data queue is located, thereby determining that the queue group is a queue group to be processed.
  • the above data queue can be a submission Queue or a Completion Queue, where the Completion Queue is the completion information of the submission Queue.
  • Both the submission queue and the completion queue include head and tail pointers to identify the start and end of data for the submission queue and completion queue. In the case that the values of the head pointer and the tail pointer of the data queue are not equal, it means that there is data in the data queue, and it is determined that the data queue has tasks, thereby determining that the queue group where the data queue is located has tasks.
  • some data queues are order-preserving queues, that is, data queues with order-preserving queues must be processed in an order-preserving order, for example, the first data queue and the second data queue Sequence preservation means that the first data queue is prior to the second data queue, so the memory operation must be performed according to the request of the first data queue, and then the second data queue can be processed on the basis of the operation.
  • the method further includes: acquiring the timestamp of the data queue in the queue group; determining whether the processing queue group has timed out according to the timestamp, and in the case of the queue group timeout, scheduling the queue group in the memory to perform anti-starvation processing .
  • the queue group in the memory is scheduled for processing to prevent starvation.
  • the method further includes: when the data queue processing of the queue group is completed, sending processing attribute information of the data queue of the queue group to the memory, wherein the processing attribute information Including the processing time stamp of the data queue of the queue group, the memory determines whether the queue group is updated in the processing process according to the processing time stamp, and the data queue of the queue group is updated in the processing process. In this case, it is determined that there are tasks to be processed in the queue group, and the queue group is added to the storage queue group in sequence again.
  • FIG. 2 is a flowchart of another method for processing a data queue according to an embodiment of the present disclosure. As shown in FIG. 2 , according to another aspect of the embodiment of the present disclosure, a method for processing a data queue is also provided, including: :
  • Step S202 receiving the data queue message, and dividing the data queue corresponding to the data queue message into a queue group for storage;
  • Step S204 sending the queue group to be processed to the processor, wherein the queue group to be processed is a queue group including the data queue to be processed, and the number of data queues of the queue group to be processed does not exceed the number of data queues stored by the processor.
  • the processor is set to process the queues in the queue group.
  • the processor processes the data queues in units of queue groups, so as to achieve the purpose of effectively and quickly processing multiple data queues, thereby achieving the technical effect of improving the efficiency of multiple data queue processing, and then solving related problems.
  • the multi-queue processing method in the technology has the technical problems of difficult scheduling and low efficiency.
  • the execution subject of the above steps may be the above-mentioned queue Queue.
  • before receiving the data queue message it includes: determining whether the number of received data queues exceeds the maximum number of data queues stored by the processor; when the number of received data queues exceeds the processor In the case of storing the maximum number of data queues, perform the steps of receiving data queue messages.
  • FIG. 3 is a schematic diagram of an NVMe queue according to an embodiment of the present disclosure.
  • the NVMe queue of this embodiment is shown in FIG. 3 , where the host is called Host, the central controller CPU, and the NVMe device is called controller (hard disk, controller) .
  • controller hard disk, controller
  • each NVMe Controller (hard disk, controller) allows up to 65535 input and output IO queues and one user Admin queue.
  • User Admin queues are created immediately after device initialization, including a send queue and a completion queue.
  • Other IO queues are generated by control commands sent in the user Admin queue.
  • the IO queue relationship specified by NVMe is relatively flexible, that is, one send queue can correspond to one completion queue, or several send queues can jointly correspond to one completion queue.
  • the interaction process of the data queue is as follows:
  • the host fills in the command to the position of the corresponding sq.tail (Submission Queue.tail, the tail of the submission queue) pointer, and adds 1 to the sq.tail pointer;
  • the controller determines the sq.head (Submission Queue.head, submission queue head) pointer and the sq.tail pointer, obtains the related tasks, and adds 1 to the sq.tail pointer;
  • Controller Controller completes related tasks
  • the controller Controller fills in the completed task and the completed task includes sq.head information, and completes the maintenance of the cq.tail (Completion Queue.tail, completion queue tail) pointer;
  • the controller sends related interrupts to the cpu;
  • Cpu processes related cq (Completion Queue, completion queue) tasks, and updates the cq.head (Completion Queue.head, completion queue head) pointer through doorbell;
  • This embodiment solves the problems of implementing multiple queues (1-64K), such as difficulty in scheduling multiple queues, and difficulty in scheduling queues with low priority, and on-chip resources can be used for fast access and high-frequency access.
  • the advanced on-chip storage solution facilitates the concurrency of multi-instruction commands and the order preservation of multi-instruction commands.
  • the on-chip CPU can easily access sq/cq information, and is easy to manage and report errors.
  • the tasks of a queue can be executed in order or out of order, and returned in order or out of order. Tasks between different queues are executed out of order. In order to increase concurrency, solve system performance delay and throughput problems.
  • FIG. 4 is a schematic diagram of the connection between a host and a controller according to an embodiment of the present disclosure.
  • the host updates SQyTDBL (Submission Queue y Tail Doorbell, the tail doorbell of the submission queue) and CQyHDBL (Completion) through the network connection interconnect/7 Queue y Head Doorbell, the head doorbell that completes the queue), the controller Controller will select according to the queue mode, these doorbell doorbells are updated to the queue queue or DDR (Double Date Rate, double-rate synchronous dynamic random access memory), and record the updated Timestamp timestamp.
  • SQyTDBL Submission Queue y Tail Doorbell, the tail doorbell of the submission queue
  • CQyHDBL Consumer Performance
  • the controller Controller will select according to the queue mode, these doorbell doorbells are updated to the queue queue or DDR (Double Date Rate, double-rate synchronous dynamic random access memory), and record the updated Timestamp timestamp.
  • DDR Double Date Rate, double-rate
  • Sq_cq_entry/9 will store the original content of the response sq_command (submit command) and cq_command (complete command), and connect it to the network connection interconnect/7 through slv/7, which is convenient for software to query CPU/18 or related accelerator ACC/19 to query the original command .
  • each data stream entry of sq_cq_entry/9 has a unique number for easy indexing.
  • the information of the submission queue/completion queue Sq/cq_info/10 will store the relevant information of the queue, including but not limited to QoS (Quality of Service, quality of service), queue base address, queue depth, queue interrupt, the corresponding relationship between sq and cq, queue Whether the physical addresses are consecutive, the status of the current queue, etc.
  • QoS Quality of Service, quality of service
  • the command desktop Order_table/11 will store the information of the command command that needs to be preserved, including but not limited to which command to preserve the order, the execution status, and so on.
  • the order-preserving queue has an order-preserving depth. If the order-preserving queue is full, it will affect the execution of the queue.
  • Sq/cq_info/14 and the doorbell sq/cq_dbl/15 of the submission queue/completion queue can be mapped to the on-chip related units 1/2/3/4/10. How to replace the on-chip resources can be set according to the grouping weight or aging, which is convenient for system scheduling .
  • FIG. 5 is a schematic diagram of a queue and a DDR according to an embodiment of the present disclosure.
  • the data structure of sram/1/2/3/4 in the queue queue includes but is not limited to combining 1/2/3/4 Split into one or more pieces of SRAM (Static Random Access Memory, static random access memory).
  • sqt means sq.tail
  • sqh means sq.head
  • cqh means cq.head
  • cqt means cq.tail.
  • sqt and cqh are maintained by host
  • sqh and cqt are maintained by hardware.
  • bitmap bitmap 8 indicates whether the related queue has tasks.
  • the resource is also the storage resource of the queue queue of the controller controller.
  • CAP.DSTRD is configured according to the minimum granularity of one write of DDR, that is, it is necessary to ensure that DDR does not generate strobe write, so as to ensure the efficiency of DDR.
  • the bit width of the network connection interconnect/7 is generally an integer multiple of 8B, the resources of DDR and Queue can be easily mapped. That is, the information of the next 512 queue can be easily taken out from the DDR to replace the information in the queue queue on the chip, and the scheduling of multiple data queues can be realized.
  • FIG. 6 is a schematic diagram of a data structure in a DDR according to an embodiment of the present disclosure.
  • the doorbell doorbell space of the DDR can be grouped according to the doorbell doorbell resources on the queue queue, for example, each group has 512 doorbell doorbells.
  • Each group of doorbells has a timestamp register, which records the moment when the current CPU is updated.
  • the bitmap qbitmap/12 corresponds to the grouping.
  • the queue group takes the group as the unit, and records the time when the replacement occurs.
  • FIG. 7 is a schematic diagram of sq_cq_entry/9 of a queue according to an embodiment of the present disclosure.
  • the depth of sq_cq_entry/9 should ensure the needs of ACC/19 and CPU/18, for example, the depth is 64.
  • the scheduling of tasks needs to support three kinds of rr/wrr/user customization.
  • the replacement mechanism of the queue group Group can be but is not limited to the following three:
  • the scheduling mechanisms within the queue group Group include but are not limited to the following three:
  • Each sq and cq pair has an instruction address command_id, which is used to mark the unique address of the instruction command. If cpu/18 or ACC/19 has a task, you can apply for the command address command_id, and issue a command command through slv/6.
  • Sq_cq_entry/9 obtains the corresponding sq_command submission queue instruction information through mst/5, and completes the cq_command completion queue instruction information filling according to sq/cq_info/10.
  • sq_command submits the queue command and cq_command completes the queue command content is consistent with the NVMe regulations.
  • FIG. 8 is a schematic diagram of a data structure of sq_cq_info/10 of a queue according to an embodiment of the present disclosure.
  • the data structure of sq_cq_info/10 includes but is not limited to the following data:
  • sq_base_addr base address of sq queue
  • priority/pc Whether the priority/physical address of the current queue is continuous
  • Timestamp the last processing time of the current queue
  • queue_enable queue enable
  • abort_command There are tasks to be canceled in the current queue, you need to query the admin queue;
  • current_caddr The physical address of the current command of the cq queue
  • ien interrupt enable
  • interrupt interrupt vector
  • FIG. 9 is a schematic diagram of order_table/11 of a queue according to an embodiment of the present disclosure.
  • the depth of order_table/11 should be set according to the number of order-preserving tasks required, for example, 8 order-preserving depths.
  • Each order-preserving queue needs to store information such as sq_num (submission queue number)/sqid (submission queue address)/command_id (submission queue command).
  • sq_num submission queue number
  • sqid submission queue address
  • command_id submission queue command
  • FIG. 10 is a schematic diagram of a queue replacing a group according to an embodiment of the present disclosure. As shown in FIG. 10 , the mechanism for replacing a group is:
  • the queue bitmap Qbitmap[12] informs the queue which queue group group has tasks through hardwired, among which, hardwired 1004 can set the cyclic multicycle;
  • Queue/1001 sends the address of the corresponding queue group group according to the polling result, and updates the related information of the queue group group;
  • the queue Queue takes the command command of the current queue group group to complete the task until the conditions for exiting the queue group group are met;
  • the queue Queue updates the queue group number group_num/1003 and load time load_time/1002 to DDR through 1008 through the write command;
  • DDR compares the current queue group group and timestamp. If it is found that the host host has been updated during the processing of the queue queue, the information of the queue bitmap qbitmap/12 cannot be changed, that is, the current queue group group has tasks to be done;
  • FIG. 11 is a schematic diagram of group replacement by queues when the number of queues is small according to an embodiment of the present disclosure.
  • the system-on-chip SoC Syetem on Chip, system-on-chip
  • the mux selection of the multiplexer the updated updata information of the host host is all updated to the SoC to realize the fast mode of the queue.
  • the above-mentioned system-on-chip SoC can be the system-on-chip of the controller.
  • the quality of service QoS scheduling of different queues can be performed
  • On-chip resources and DDR resources have the same data structure, which is convenient for direct replacement and easy to implement;
  • the timestamp mechanism that is, to record the time of task update on the host and the time of on-chip resource exchange, to ensure that the task is not lost;
  • This embodiment can implement an arbitrary number of queues (1 to 64K), and a scheduling mode defined by wrr/rr/user can be performed between different queues, and an active anti-starvation mechanism can effectively manage multiple queues.
  • Multitasking can achieve disorder and order preservation, and multitasking can work at the same time, providing excellent performance while taking into account the area and implementation difficulty.
  • the queue group group mechanism and the queue group group prevention mechanism for task loss can ensure the safe and stable operation of the system, and each task has the opportunity to participate in scheduling. Replacing according to the queue group group can effectively save resources and implementation difficulty, and at the same time obtain relatively good power consumption.
  • DDR provides relevant information (task, timestamp) according to the queue group group, which effectively saves implementation resources and avoids the difficulty of interaction between queue queues and DDR. It is convenient to implement and can also provide excellent performance.
  • the host host directly writes the doorbell doorbell information to DDR or queue queue, which can be selected according to needs. When there are few queues, relatively excellent performance and low power consumption are obtained.
  • FIG. 12 is a schematic diagram of an apparatus for processing a data queue according to an embodiment of the present disclosure. As shown in FIG. 12 , according to another aspect of the embodiment of the present disclosure, a processing apparatus for a data queue is further provided, including: an acquisition module 1202 and the processing module 1204, the device will be described in detail below.
  • the acquiring module 1202 is configured to acquire the queue group to be processed stored in the memory, wherein the queue group to be processed is a queue group including the data queue to be processed, and the memory is set to receive the data queue message, and the data corresponding to the data queue message
  • the queues are divided into queue groups for storage; the processing module 1204, connected to the above-mentioned acquisition module 1202, is set to process the data queues in the queue groups to be processed, wherein the number of data queues in the queue groups to be processed does not exceed the locally stored data queues maximum number of .
  • the acquisition module 1202 is used to acquire the queue group to be processed stored in the memory, wherein the queue group to be processed is a queue group including the data queue to be processed, the memory is set to receive data queue messages, and the data queue messages correspond to The data queues are divided into queue groups for storage; the processing module 1204 processes the data queues in the queue groups to be processed, wherein the number of data queues of the queue groups to be processed does not exceed the maximum number of locally stored data queues, by storing in The memory divides multiple data queues into queue groups for storage, and the processor processes the data queues in units of queue groups, achieving the purpose of effectively and quickly processing multiple data queues, thereby improving the efficiency of multiple data queue processing.
  • the technical effect is obtained, and the technical problems of the multi-queue processing method in the related art, the difficulty of scheduling and the low efficiency are solved.
  • FIG. 13 is a schematic diagram of another processing apparatus for a data queue according to an embodiment of the present disclosure. As shown in FIG. 13 , according to another aspect of the embodiment of the present disclosure, there is also provided a processing apparatus for a data queue, including: a storage The module 1302 and the sending module 1304 are described in detail below.
  • the storage module 1302 is configured to receive data queue messages, and divide the data queues corresponding to the data queue messages into queue groups for storage; the sending module 1304, connected to the above-mentioned storage module 1302, is configured to send the queue groups to be processed to the processor,
  • the queue group to be processed is a queue group that includes data queues to be processed, the number of data queues of the queue group to be processed does not exceed the maximum number of data queues stored by the processor, and the processor is set to perform processing on the queues in the queue group. deal with.
  • the storage module 1302 is used to receive the data queue messages, and the data queues corresponding to the data queue messages are divided into queue groups for storage; the sending module 1304 sends the queue groups to be processed to the processor, wherein the queue groups to be processed are: A queue group including data queues to be processed, the number of data queues of the queue group to be processed does not exceed the maximum number of data queues stored by the processor, and the processor is set to process the queues in the queue group.
  • a computer storage medium is also provided, where the computer storage medium includes a stored program, wherein when the program is run, a device where the computer storage medium is located is controlled to perform any one of the above data queue processing method.
  • a processor is also provided, and the processor is configured to run a program, wherein when the program runs, any one of the above-mentioned processing methods for data queues is executed.
  • the disclosed technical content may be implemented in other manners.
  • the device embodiments described above are only illustrative, for example, the division of the units may be a logical function division, and there may be other division methods in actual implementation, for example, multiple units or components may be combined or Integration into another system, or some features can be ignored, or not implemented.
  • the shown or discussed mutual coupling or direct coupling or communication connection may be through some interfaces, indirect coupling or communication connection of units or modules, and may be in electrical or other forms.
  • the units described as separate components may or may not be physically separated, and components shown as units may or may not be physical units, that is, may be located in one place, or may be distributed to multiple units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution in this embodiment.
  • each functional unit in each embodiment of the present disclosure may be integrated into one processing unit, or each unit may exist physically alone, or two or more units may be integrated into one unit.
  • the above-mentioned integrated units may be implemented in the form of hardware, or may be implemented in the form of software functional units.
  • the integrated unit if implemented in the form of a software functional unit and sold or used as an independent product, may be stored in a computer-readable storage medium.
  • a computer-readable storage medium including several instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present disclosure.
  • the aforementioned storage medium includes: U disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), mobile hard disk, magnetic disk or optical disk and other media that can store program codes .

Landscapes

  • Engineering & Computer Science (AREA)
  • Software Systems (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)

Abstract

L'invention concerne un procédé et un appareil de traitement de file d'attente de données. Ledit procédé consiste : à acquérir un groupe de files d'attente à traiter envoyé par une mémoire, le groupe de files d'attente à traiter étant un groupe de files d'attente comprenant des files d'attente de données à traiter et la mémoire étant configurée pour recevoir un message de file d'attente de données, pour générer un groupe de files d'attente à partir des files d'attente de données correspondant au message de file d'attente de données selon un premier ordre prédéterminé, et pour stocker, dans les cas où il y a une pluralité de groupes de files d'attente générés, la pluralité de groupes de files d'attente selon un second ordre prédéterminé ; et à traiter les files d'attente de données dans le groupe de files d'attente à traiter, le nombre de files d'attente de données dans le groupe de files d'attente à traiter ne dépassant pas le nombre maximal de files d'attente de données stockées localement. La présente invention résout les problèmes techniques de difficulté de planification et de faible efficacité d'un procédé de traitement de multiples files d'attente dans l'état de la technique.
PCT/CN2020/133402 2020-07-20 2020-12-02 Procédé et appareil de traitement de file d'attente de données WO2022016767A1 (fr)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202010700105 2020-07-20
CN202010700105.3 2020-07-20

Publications (1)

Publication Number Publication Date
WO2022016767A1 true WO2022016767A1 (fr) 2022-01-27

Family

ID=79729810

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2020/133402 WO2022016767A1 (fr) 2020-07-20 2020-12-02 Procédé et appareil de traitement de file d'attente de données

Country Status (1)

Country Link
WO (1) WO2022016767A1 (fr)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN115842790A (zh) * 2022-12-29 2023-03-24 三未信安科技股份有限公司 一种应用于PCle设备的业务数据流量控制方法

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101094176A (zh) * 2007-07-10 2007-12-26 杭州华三通信技术有限公司 多通道数据处理方法及装置
US20080107020A1 (en) * 2006-11-03 2008-05-08 Man Trinh Highly-scalable hardware-based traffic management within a network processor integrated circuit
CN102594663A (zh) * 2012-02-01 2012-07-18 中兴通讯股份有限公司 队列调度方法及装置
CN103353851A (zh) * 2013-07-01 2013-10-16 华为技术有限公司 一种管理任务的方法和设备
CN111897665A (zh) * 2020-08-04 2020-11-06 北京泽石科技有限公司 数据队列的处理方法及装置

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20080107020A1 (en) * 2006-11-03 2008-05-08 Man Trinh Highly-scalable hardware-based traffic management within a network processor integrated circuit
CN101094176A (zh) * 2007-07-10 2007-12-26 杭州华三通信技术有限公司 多通道数据处理方法及装置
CN102594663A (zh) * 2012-02-01 2012-07-18 中兴通讯股份有限公司 队列调度方法及装置
CN103353851A (zh) * 2013-07-01 2013-10-16 华为技术有限公司 一种管理任务的方法和设备
CN111897665A (zh) * 2020-08-04 2020-11-06 北京泽石科技有限公司 数据队列的处理方法及装置

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN115842790A (zh) * 2022-12-29 2023-03-24 三未信安科技股份有限公司 一种应用于PCle设备的业务数据流量控制方法
CN115842790B (zh) * 2022-12-29 2023-05-02 三未信安科技股份有限公司 一种应用于PCIe设备的业务数据流量控制方法

Similar Documents

Publication Publication Date Title
EP3358463B1 (fr) Procédé, dispositif et système de mise en oeuvre d'un traitement d'accélération matérielle
CN111897665A (zh) 数据队列的处理方法及装置
EP2506147B1 (fr) Optimisations d'EPOLL
CN111651377B (zh) 一种用于片内报文处理的弹性共享缓存器
TW200411403A (en) System and method for transferring data between virtual machines or other computer entities
KR20070030285A (ko) 데이터 전송 방법, 데이터 전송 시스템 및 컴퓨터 판독가능한 기록 매체
US9836516B2 (en) Parallel scanners for log based replication
AU2015402888B2 (en) Computer device and method for reading/writing data by computer device
WO2022237695A1 (fr) Procédé et dispositif de délestage de pile de protocoles à simultanéité élevée basés sur une mémoire à grande capacité côté hôte, et support
CN115185880B (zh) 一种数据存储方法及装置
US11875198B2 (en) Synchronization object issue detection using object type queues and associated monitor threads in a storage system
WO2024046188A1 (fr) Procédé et système de déchargement d'e/s dans un environnement en nuage, dispositif, et support de stockage
WO2024088268A1 (fr) Procédés de gestion d'événements rdma, et dispositif et support de stockage
WO2022016767A1 (fr) Procédé et appareil de traitement de file d'attente de données
WO2023201987A1 (fr) Procédé et appareil de traitement de requête, et dispositif et support
JPWO2019100984A5 (fr)
US11106513B2 (en) Message passing in a data processing system
EP4390646A1 (fr) Procédé de traitement de données dans un système distribué, et système associé
US20230393782A1 (en) Io request pipeline processing device, method and system, and storage medium
CN117573041B (zh) 一种改进vhost-scsi提升虚拟化存储性能的方法
WO2023072223A1 (fr) Procédé et appareil de traitement de données, et dispositif informatique
US20240220334A1 (en) Data processing method in distributed system, and related system
CN117909087B (zh) 一种数据处理方法、装置、中央处理器及电子设备
US20210406066A1 (en) End-to-end quality of service mechanism for storage system using prioritized thread queues
WO2023231768A1 (fr) Processeur multicœur et procédé de communication inter-cœurs associé

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 20946065

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 20946065

Country of ref document: EP

Kind code of ref document: A1