CN109426632A

CN109426632A - Memory pool access method and device

Info

Publication number: CN109426632A
Application number: CN201810102233.0A
Authority: CN
Inventors: 朱勇
Original assignee: New H3C Technologies Co Ltd
Current assignee: New H3C Technologies Co Ltd
Priority date: 2018-02-01
Filing date: 2018-02-01
Publication date: 2019-03-05
Anticipated expiration: 2038-02-01
Also published as: CN109426632B

Abstract

This application provides memory pool access methods and device.In the application, by the WR chain for generating the condition that meets for RDMA operation, can be realized will be used to indicate RDMA operation completion WR and the WR for realizing RDMA operation the same WR chain is set and be used to indicate RDMA operation completion WR be arranged it is last in WR chain, this means that the WR for being used to indicate RDMA operation completion lies in RDMA operation and executes, guarantee that opposite end business device can perceive RDMA operation completion in time in the process of RDMA operation, improves the treatment effeciency of RDMA operation.

Description

Memory pool access method and device

Technical field

This application involves data communication technologies, in particular to memory pool access method and device.

Background technique

It is directly accessed in (RDMA:Remote Direct Memory Access) application in long-distance inner, it is straight by memory Receiving asks that (DMA:Direct Memory Access) mode remotely accesses target memory, reduces memory copying and kernel or more Text switching bring expense, realizes the high-property transmission of data.

In RDMA application, common operation has RDMASEND (transmission) operation, RDMA operation.Wherein, RDMA SEND is grasped Act on the data for sending fritter (such as size is less than 4k)；And RDMA operation, primarily directed to continuous, bulk data.This In, RDMA operation can be RDMA READ (reading) operation or RDMA WRITE (writing) operation.

In RDMA in use, timely learning RDMA operation is completed, to handle RDMA operation relevant information as quickly as possible, it is Improve the key of RDMA operation treatment effeciency.

Summary of the invention

This application provides memory pool access methods and device, to improve the treatment effeciency of RDMA operation.

The technical scheme provided by the application includes:

A kind of memory pool access method, this method are applied to business device, comprising:

The work request WR chain for the condition that meets, the condition are as follows: the last one WR is used in WR chain are generated for RDMA operation Indicate that RDMA operation is completed；

When initiating the RDMA operation to opposite end business device, in sequence to described in opposite end business device transmission WR in WR chain.

A kind of internal storage access device, the device are applied to business device, comprising:

Processing module generates the work request WR chain for the condition that meets, institute for directly accessing RDMA operation for long-distance inner State condition are as follows: the last one WR is used to indicate RDMA operation completion in WR chain；

Business module is used for when initiating the RDMA operation to opposite end business device, in sequence to the opposite end industry Business equipment sends the WR in the WR chain.

As can be seen from the above technical solutions, in the application, by generating the WR chain for the condition that meets for RDMA operation, with reality The WR for being now used to indicate RDMA operation completion is arranged in the same WR chain and is used to indicate with the WR for realizing RDMA operation WR chain is arranged in finally, the WR for being used to indicate RDMA operation completion is made to lie in same operating process in the WR that RDMA operation is completed It is executed in the process of RDMA operation, ensure that opposite end business device in the process of RDMA operation you can learn that RDMA operation is complete At, realize opposite end business device perceive in time RDMA operation completion, and then can for complete RDMA operation execute phase Such as notice upper layer application processing RDMA operation relevant information should be operated, the treatment effeciency of RDMA operation is improved.

Detailed description of the invention

The drawings herein are incorporated into the specification and forms part of this specification, and shows the implementation for meeting the disclosure Example, and together with specification for explaining the principles of this disclosure.

Fig. 1 shows RDMA READ operation implementation flow chart；

Fig. 2 shows the structural schematic diagrams of WR；

Fig. 3 shows the structural schematic diagram of the corresponding WR chain of RDMA READ operation；

Fig. 4 is memory pool access method flow chart provided by the present application；

Fig. 5 is the RDMA READ operation implementation flow chart of the embodiment of the present application 1；

Fig. 6 is the RDMA WRITE operation implementation flow chart of the embodiment of the present application 1；

Fig. 7 is the RDMA READ operation implementation flow chart of the embodiment of the present application 2；

Fig. 8 is the RDMA WRITE operation implementation flow chart of the embodiment of the present application 3；

Fig. 9 is apparatus structure schematic diagram provided by the present application；

Figure 10 is the hardware structural diagram of Fig. 9 shown device provided by the present application.

Specific embodiment

It, often can be because of some time-consuming places for reducing RDMA operation unnecessary when executing RDMA operation in RDMA application Manage efficiency.Here RDMA operation can be RDMA READ operation or RDMA WRITE operation.Below with RDMA READ operation For describe:

RDMA READ operation implementation flow chart is shown referring to Fig. 1, Fig. 1.As shown in Figure 1, the process may include following step It is rapid:

Step 101, the first business device is initiated to the second business device for requesting the RDMA SEND of memory address to operate (being denoted as RDMA SEND operation 1a).

Here, the first business device, the second business device are only the names to carry out convenient for differentiation, are not intended to limit It is fixed.

As one embodiment, RDMA SEND operation 1a here carries pending RDMA READ operation to be read The Data Identification of the data (being denoted as data 100) taken, the purpose is to for obtaining the second business device sheet from the second business device Ground is used to record the memory address (being denoted as memory address 1b, address needed for being summarised as RDMA READ operation) of data 100.

Data Identification after the second business device receives RDMA SEND operation 1a, according to RDMA SEND operation 1a carrying Data 100 are found, if the second business device discovery data 100 have been recorded to local memory, the second business device will be local The memory address 1b for being used to record data 100 in memory, which is encapsulated in RDMA SEND operation (being denoted as RDMA SEND operation 2a), to be returned Back to the first business device；If the second business device discovery data 100 do not record also to local memory, the second business device is first Data 100 are recorded to local memory, the second business device will be used to record the memory address of data 100 later in local memory 1b is encapsulated in RDMA SEND operation (being denoted as RDMA SEND operation 2a) and returns to the first business device.

It should be noted that above-mentioned second business device will be used to record data in local memory as one embodiment 100 memory address 1b is encapsulated in RDMA SEND operation (being denoted as RDMA SEND operation 2a) and returns to the first business device tool Body can are as follows: memory address 1b is divided into multiple address blocks according to the data volume of data 100 by the second business device, here, eachly The size of location block equal can also differ, and the application does not limit specifically；Later, by each address block being divided into and each The corresponding data length of address block is encapsulated in RDMA SEND operation (being denoted as RDMA SEND operation 2a) and returns to the first business Equipment.For example: if the data volume of data 100 is 512k, memory address 1b is divided into 128 addresses by the second business device Block, and the corresponding data length of each address block (such as 4k) is set, each address block and each address block is corresponding Data length be encapsulated in RDMA SEND operation (be denoted as RDMA SEND operation 2a) and return to the first business device.

Step 102, the RDMA SEND that the first business device receives that the second business device returns operates 2a.Under executing later State step 103.

Step 103, the first business device generates RDMA READ according to the RDMA SEND operation 2a memory address 1b carried Operate corresponding work request (WR:Work Request) chain.

Based on the memory address 1b for being used to record data 100 in local memory is encapsulated in above in connection with the second business device The specific descriptions of the first business device are returned in RDMA SEND operation (being denoted as RDMA SEND operation 2a), then this step 103 In, the essence that the RDMA SEND operation 2a that the first business device receives is carried is the divided each address block of memory address 1b. Based on this, in this step 103, it is corresponding that RDMA READ operation is generated according to the RDMA SEND operation 2a memory address 1b carried WR chain specifically includes:

The address block and the corresponding data length generation RDMA of address block that 2a is carried successively are operated according to RDMA SEND The WR sequentially generated, is serially connected forms WR in sequence by the WR (its address block with based on is corresponding) of READ operation Chain.For example, if RDMA SEND operation 2a specifically carries 128 address blocks and the corresponding data of each address block Length is 4k, then in this step 102, first address block that 2a is carried can be operated according to RDMA SEND and generates first WR, the One WR is corresponding with the first address block, and, second address block that 2a is carried is operated according to RDMA SEND and generates second WR, second WR is corresponding with the second address block, and so on, ultimately generate 128 WR.Fig. 2 shows the structures of WR.Later, 128 WR of generation are composed in series into the corresponding WR chain of the i.e. above-mentioned RDMA READ operation of a WR chain in order, it is specific such as Fig. 3 It is shown.

WR shown in Fig. 2 specifically includes that action type field (Opcode), local terminal memory address field (Localbuf_ Addr_info), opposite end memory address field (Remote_addr_info), flag bit field (Flag), next WR field (Next).Here, it is applied to RDMA READ operation process shown in FIG. 1, Opcode is RDMA READ；Localbuf_addr_ Info includes the data that are read based on this WR in the starting memory address of local memory and the address of the data occupancy of the reading Length；Remote_addr_info includes the initial address of the corresponding address block of this WR and the address size of the address block. Flag is default value；Next is used to indicate next WR.

Based on the Next in WR, the above-mentioned WR that will be sequentially generated in sequence, which is serially connected, forms WR chain concretely: The Next of first WR is directed toward second WR, the Next of second WR is directed toward third WR, and so on, will finally own WR composes in series a WR chain.

It should be noted that the Flag of the last one WR is arranged to IB_SEND_ in the WR chain that this step 103 generates SIGNALED, the purpose is to trigger the first business device to generate CQE message (Opcode is RADM READ), so as to the first business Equipment (the RDMA application program of specifically the first business device) is determined by parsing CQE message (Opcode is RADM READ) The RDMA READ operation that this equipment is initiated is completed, referring specifically to following step 105.

Step 104, the first business device sends the business device of WR to second in WR chain according to serial mode sequence.

In this step 104, after the first business device sends the WR in WR chain to the second business device, the second business device meeting Receive from the first business device send WR, when the second business device receive from the first business device send WR, second The RDMA hardware of business device such as network interface card reads data according to the Remote_addr_info in the WR that receives, and by reading Data return to the first business device, and entire data read process at this time, and the RDMA application program of the second business device is not Perceive RDMA READ operation.

In addition, first business device is specific to the second business device according to serial mode sequence transmission WR in this step 104 Refer to: after one WR of the every transmission of the first business device, if receiving the second business device is directed to the data that the WR is returned, after supervention Next WR is sent, and so on, finally, after the first business device sends the last one WR in WR chain, if receiving the second industry Equipment of being engaged in is directed to the data that the last one WR is returned, then means that the RDMA READ operation that the first business device is initiated is completed. At this point, the first business device executes following step 105.

Step 105, the IB_SEND_ that the first business device is arranged based on the Flag of the last one WR sent SIGNALED generates CQE message (Opcode is RADM READ), is determined by parsing CQE message (Opcode is RADM READ) The RDMA READ operation that this equipment is initiated is completed, and executes step 106 later.

In the application, even if the first business device has sent the last one WR of WR chain, and the second business device is also received For the data that the last one WR is returned, the first business device (the RDMA application program of specifically the first business device) itself Also can't logic judgment its initiate RDMA READ operation whether complete, in order to ensure the first business device (specifically first The RDMA application program of business device) it can be returned receiving the last one WR of the second business device for the WR chain sent Data when know this equipment initiate RDMA READ operation complete, then as described above, the first business device can for WR chain most IB_SEND_SIGNALED is arranged in the Flag of the latter WR.In this way, as step 105 describes, when the first business device has sent WR The last one WR of chain, and also receive the second business device for the last one WR return data when, the first business device The IB_SEND_SIGNALED of Flag setting based on the last one WR sent generates CQE message (Opcode RADM READ), in this way, the first business device determines the RDMA that this equipment is initiated by parsing CQE message (Opcode is RADM READ) READ operation is completed.

In this application, when the RDMA READ operation that the first business device is initiated is completed, the second business device is (specific It is the RDMA application program of the second business device) entire RDMA READ operation completion is not perceived, in order to guarantee that the second business is set The standby RDMA READ operation for perceiving the initiation of the first business device is completed, to facilitate the second business device to grasp in RDMA READ Relevant treatment such as address release etc. is executed after the completion of making in time, then needs to cut from the original process for executing RDMA READ operation The process for executing RDMA SEND operation is shifted to, following step 106 is specifically shown in.

Step 106, the first business device initiates to be used to indicate what RDMA READ operation was completed again to the second business device RDMA SEND operation (is denoted as RDMA SEND operation 3a).

When the second business device receives RDMASEND operation 3a, then according to RDMASEND operation 3a confirmation RDMAREAD behaviour It completes, later, relevant treatment such as address release etc. can be executed for the RDMA READ operation completed in time.

So far, process shown in Fig. 1 is completed.

In process shown in Fig. 1, the number that the last one WR is returned is directed to when the first business device receives the second business device According to, then mean the first business device initiate RDMA READ operation complete.And it can be seen that from process shown in FIG. 1 After the completion of the RDMAREAD operation that one business device is initiated, (RDMA of specifically the second business device is applied the second business device Program) it is not timely learning, but 4 periods (△ t1 to △ t4) to be undergone just to know.Wherein, △ t1 is that the first business is set Standby processing RDMA READ is completed the time of relevant information such as above-mentioned steps 105, and △ t2 is the first business device from RDMA READ operation process switches to the time of RDMA SEND operating process；△ t3 is the transmission time of RDMA SEND operation 3a；△ T4 is the time that the second business device receives and identifies RDMA SEND operation 3a.

Wherein, △ t1 and △ t2 is the time that the first business device locally consumes, and the second business device is in the first industry It can just know that RDMA READ operation is completed after business equipment consumption local zone time △ t1 and △ t2, this is equivalent to the first business and sets Second business device that is added to of the standby time locally consumed knows the time that RDMA READ operation is completed, and delays the second business Equipment knows that RDMAREAD operation is completed, so that the second business device is unable to learn in time RDMAREAD operation and completes, and then leads It causes the second business device that can not cope with relevant treatment such as address release of RDMA READ operation etc. in time, reduces RDMA READ The treatment effeciency of operation.

Be above the i.e. opposite end business device of the RDMA READ operation problem is described so that RDMAREAD is operated as an example cannot Timely learning RDMA READ operation is completed, and the treatment effeciency of RDMA READ operation is reduced.Similarly, RDMA WRITE operation It will appear similar problems, i.e. opposite end business device is unable to timely learning RDMAWRITE operation completion, reduces RDMA WRITE operation Treatment effeciency.

To solve the above-mentioned problems, this application provides methods as shown in Figure 4.

Referring to fig. 4, Fig. 4 is memory pool access method flow chart provided by the present application.The process is applied to any business device.

As shown in figure 4, the process can comprise the following steps that

Step 401, the WR chain for the condition that meets, the condition are as follows: the last one WR is used in WR chain are generated for RDMA operation Indicate that RDMA operation is completed.

By step 401, by for realizing the WR of RDMA operation be used to indicate that RDMA operation is complete to opposite end business device At WR (being denoted as WR400) be arranged in same WR chain, it means that WR400 is a part of RDMA operation, and being equivalent to WR400 is It is executed together with RDMA operation.

Step 402, it when initiating the RDMA operation to opposite end business device, is sent in sequence to opposite end business device WR in WR chain.

In this application, WR400 and the WR setting for realizing RDMA operation in the same WR chain and are arranged in WR chain It is executed finally, lying in the WR400 and WR for realizing RDMA operation in the same operating process i.e. process of RDMA operation, Opposite end business device is informed without additionally initiating a RDMA SEND operating process different from RDMA operation process, is realized Opposite end business device perceives RDMA operation completion in time, and then can be handled in time RDMA operation relevant information, improves The treatment effeciency of RDMA operation.

So far, process shown in Fig. 4 is completed.

By process shown in Fig. 4, so that WR400 lies in same operating process i.e. with the WR for realizing RDMA operation It is executed in the process of RDMA operation, ensure that opposite end business device in the process of RDMA operation you can learn that RDMA operation is complete At this aspect can save the business device for initiating RDMA operation again from RDMA operation such as RDMA READ operation process It switches to RDMA SEND operating process and is informed with the message for completing RDMA operation and give opposite end business device, on the other hand also keep away The local elapsed time (such as above-mentioned △ t1 and △ t2) for exempting from the business device of initiation RDMA operation is superimposed to opposite end business and sets The standby time knowing RDMA access operation and completing, finally compared to process shown in above-mentioned Fig. 1, process shown in Fig. 4 can be by opposite end industry Business equipment knows the time advance that RDMA operation is completed, and improves the treatment effeciency of RDMA operation.

It is in one example, specific for the corresponding work request WR chain of RDMA operation generation in above-mentioned steps 401 can include: It generates according to from the first memory address needed for the RDMA operation that the opposite end business device has obtained for realizing described The WR of RDMA operation；It generates and is used to indicate the first specified WR that the RDMA operation is completed；It will be for realizing the RDMA operation WR, the first specified WR form the WR chain, wherein the first specified WR is described the last one WR of WR chain.

In one example, the first above-mentioned specified WR refers to: Opcode is WR (the abbreviation RDMA SEND of RDMA SEND WR).RDMA SEND WR is different from RDMA WRITE WR (WR that Opcode is RDMA WRITE) or RDMA READ WR (WR that Opcode is RDMA READ).It illustrates below by embodiment 1 and describes RDMA SEND WR:

Embodiment 1:

It is the RDMA READ operation implementation flow chart of the embodiment of the present application 1 referring to Fig. 5, Fig. 5.As shown in figure 5, the process It can comprise the following steps that

Step 501, business device 5_1 is initiated to business device 5_2 for requesting the RDMA SEND of memory address to operate (being denoted as RDMA SEND operation S11).

This step 501 is similar to above-mentioned step 101.

Step 502, the RDMA SEND that business device 5_1 receives that business device 5_2 is returned operates S12, according to RDMA The memory address (being denoted as M1) that SEND operation S12 is carried generates at least one and (is denoted as realizing the WR of RDMA READ operation RDMA READ WR)。

In this step 502, the Opcode for each RDMA READ WR that business device 5_1 is generated is RDMA READ.

Step 503, business device 5_1 generates the RDMA for completing to business device 5_2 instruction RDMA READ operation Each RDMA READ WR, RDMA SEND WR are formed WR chain by SEND WR, wherein RDMA SEND WR is the last of WR chain One WR.

The RDMA SEND WR generated in this step 503 is different from RDMA READ WR, and Opcode is RDMA SEND.

As one embodiment, in this step 503, by each RDMA READ WR, RDMA SEND WR composition WR chain tool Body can include: the Next of first RDMA READ WR is directed toward second RDMA READ WR, by second RDMA READ The Next of WR is directed toward third RDMA READ WR, and so on, the Next of the last one RDMA READ WR is concatenated into RDMA SEND WR ultimately forms a WR chain.

Finally realize that business device 5_1 is that RDMA READ operation generates the WR chain for meeting following condition by step 503: The last one WR is used to complete to the RDMA READ operation that business device 5_2 instruction business device 5_1 is initiated in WR chain.

Step 504, business device 5_1 sends the WR in WR chain to business device 5_2 in sequence.

As one embodiment, in this step 504, business device 5_1 is sent out according to serial mode sequence to business device 5_2 Send the WR in WR chain.

In this step 504, after business device 5_1 sends WR to business device 5_2, business device 5_2 will receive WR.

If the WR that business device 5_2 is received is RDMA READ WR, then the RDMA hardware such as network interface card of business device 5_2 Data are read according to the memory address carried of the Remote_addr_info in the RDMA READ WR received and return to business Equipment 5_1, so that business device 5_1 (is equivalent to according to the Localbuf_addr_info write-in data of the RDMA READ WR Data are read by RDMA READ operation), the RDMA application program of entire reading data course, business device 5_2 does not perceive.

If it is RDMASEND WR that business device 5_2, which receives WR, then the hardware of business device 5_2 such as network interface card can generate CQE message, the Opcode of CQE message is to be used to indicate the type (recv) for receiving RDMA SEND WR, and by the CQE of generation Message is sent to RDMA application program, and the RDMA application program of business device 5_2 can be received according to the parsing of CQE message at this time RDMA SEND WR, and then know that RDMA READ operation is completed by parsing RDMA SEND WR.At this point, business device 5_2 is obtained Know that RDMA READ operation is completed, that is, realizes and operated by the RDMAREAD that RDMA SEND WR initiates business device 5_1 The message of completion is indicated to business device 5_2.And in fact, being set according to serial mode sequence to business based on business device 5_1 Standby 5_2 sends the description of WR, then when business device 5_2 receives RDMA SEND WR, also mean really RDMA READ operation at Function is completed.In other words, in the present embodiment 1, once business device 5_2 receives RDMA SEND WR, then mean RDMA really READ operation is completed.In this way, then meaning that business device 5_2 will be obtained in RDMA READ operation process in the present embodiment 1 Know that RDMA READ operation is completed.

Meanwhile business device 5_1 can also execute following step 505 after sending RDMA SEND WR.

Step 505, business device 5_1 generates CQE message (Opcode is RADM SEND), and passes through parsing CQE message (Opcode is RADM SEND) determines that the RDMA READ operation that this equipment is initiated is completed.

So far, by step 505, business device 5_1 also knows that the RDMA READ operation that this equipment is initiated is completed, and realizes It indicates that this business device 5_1 is generated after RDMA SEND WR is sent completely by RDMA SEND WR and corresponding completes CQE Message (Opcode is RDMA SEND), to indicate that the RDMA READ operation that this equipment is initiated is completed.

So far, process shown in Fig. 5 is completed.

It can be seen that in the present embodiment 1 from process shown in Fig. 5, business device 5_2 is receiving the last one in WR chain The RDMA READ behaviour that business device 5_1 is initiated can be known after WR, that is, RDMA SEND WR according to the RDMA SEND WR received It completes, realizes business device 5_2 in the process of RDMA READ operation you can learn that RDMA READ operation is completed, by industry Business equipment 5_2 knows the time advance that RDMA READ operation is completed, and improves the treatment effeciency of RDMA READ operation.

Embodiment 1 is described by taking RDMA READ operation as an example above, describes this reality by taking RDMA WRITE operation as an example below Apply example 1:

It is the RDMA WRITE operation implementation flow chart of the embodiment of the present application 1 referring to Fig. 6, Fig. 6.As shown in fig. 6, the process It can comprise the following steps that

Step 601, business device 6_1 is initiated to business device 6_2 for requesting the RDMA SEND of memory address to operate (being denoted as RDMA SEND operation S21).

As one embodiment, RDMA SEND operation S21 here carries pending RDMA WRITE operation and is wanted The size of data L of the data (being denoted as data D600) of write-in, the purpose is to for obtaining business device 6_2 from business device 6_2 Memory address (is denoted as by the memory address that data D600 is written in RDMA WRITE operation locally for record traffic equipment 6_1 M2, address needed for being summarised as RDMA WRITE operation).

After business device 6_2 receives RDMA SEND operation S21, the data carried according to RDMA SEND operation S21 are big Small L, which is determined, is designed for the memory that data D600 is written by RDMA WRITE operation by record traffic equipment 6_1 in local memory Address (is denoted as memory address M2), and memory address M2 is encapsulated in RDMA SEND operation (being denoted as RDMA SEND operation S22) Return to business device 6_1.

As one embodiment, it is above-mentioned by memory address M2 be encapsulated in RDMA SEND operation (be denoted as RDMA SEND operation S22 business device 6_1 is returned in) concretely: M2 being divided into multiple address blocks according to size of data L, here, each address The size of block equal can also differ, and the application does not limit specifically；Later, by each address block being divided into and each Block corresponding data length in location is encapsulated in RDMA SEND operation (being denoted as RDMA SEND operation S22) and returns to business device 6_ 1.For example: if size of data L is 512k, memory address M2 is divided into 128 address blocks by business device 6_2, and is arranged The corresponding data length of each address block (such as 4k), each address block and the corresponding data of each address block is long Degree is encapsulated in RDMA SEND operation (being denoted as RDMA SEND operation S22) and returns to business device 6_1.

Step 602, the RDMA SEND that business device 6_1 receives that business device 6_2 is returned operates S22, according to RDMA The memory address M2 that SEND operation S22 is carried generates at least one and (is denoted as RDMA for realizing the WR of RDMA WRITE operation WRITE WR)。

(RDMA SEND is denoted as based on memory address M2 is encapsulated in RDMA SEND operation above in connection with business device 6_2 Operation S22) in return to the description of business device 6_1, then in this step 602, RDMA SEND behaviour that business device 6_1 is received The essence for making S22 carrying is the divided each address block of memory address M1.Based on this, in this step 602, according to RDMA The memory address M2 that SEND operation S22 is carried generates at least one RDMA WRITE WR and specifically includes:

The address block and the corresponding data length generation RDMA of address block that S22 is carried successively are operated according to RDMA SEND The Opcode of WRITE WR, each RDMA WRITE WR of the generation are RDMA WRITE.For example, if RDMA SEND Operation S22 specifically carries 128 address blocks and the corresponding data length of each address block is that 4k can then in this step 602 First RDMA WRITE that first address block that S22 is carried generates corresponding first address block is operated according to RDMA SEND WR operates second RDMA that second address block that S22 is carried generates corresponding second address block according to RDMA SEND WRITE WR, and so on, ultimately generate 128 RDMA WRITE WR.

Here, the structure of each RDMA WRITE WR of generation is similar to Figure 2, is applied to RDMA shown in fig. 6 WRITE operation process, the then following each field in each RDMA WRITE WR generated specifically:

Opcode:RDMA WRITE；

Localbuf_addr_info: including the data that are written based on this WR local memory starting memory address, with And the address size that the data of the write-in occupy；

Remote_addr_info: the address of initial address and the address block including the corresponding address block of this WR is long Degree；

Flag: default value；

Next: next WR is indicated.

Step 603, business device 6_1 generates the RDMA for completing to business device 6_2 instruction RDMA WRITE operation Each RDMA WRITE WR and RDMA SEND WR is formed WR chain by SEND WR, wherein RDMA SEND WR is WR chain The last one WR.

The RDMA SEND WR generated in this step 603 is different from RDMA WRITE WR, and Opcode is RDMA SEND.

As one embodiment, in this step 603, by each RDMA WRITE WR, RDMA SEND WR composition WR chain tool Body can include: the Next of first RDMA WRITE WR is directed toward second RDMA WRITE WR, by second RDMA The Next of WRITE WR is directed toward third RDMA WRITE WR, and so on, by the Next of the last one RDMA WRITE WR Concatenation RDMA SEND WR ultimately forms a WR chain.

Finally realize that business device 6_1 is that RDMA WRITE operation generates the WR chain for meeting following condition by step 603: The last one WR is used to complete to the RDMA WRITE operation that business device 6_2 instruction business device 6_1 is initiated in WR chain.

Step 604, business device 6_1 sends the WR in WR chain to business device 6_2 in sequence.

In this step 604, business device 6_1 sends the WR in WR chain according to serial mode sequence to business device 6_2.

After business device 6_1 sends the WR in WR chain to business device 6_2, business device 6_2 will receive WR.

If the WR that business device 6_2 is received is RDMA WRITE WR, then the RDMA hardware such as net of business device 6_2 Card can be written data according to the Remote_addr_info field for the RDMA WRITE WR that this is received and (be equivalent to and pass through RDMA WRITE operation writes data), data procedures are entirely write, the RDMA application program of business device 6_2 does not perceive.

If the WR that business device 6_2 is received is RDMA SEND WR, then the hardware of business device 6_2 such as network interface card can give birth to At CQE message (Opcode recv) and it is sent to RDMA application program, the RDMA application program of business device 6_2 can be according at this time The RDMA SEND WR received is parsed according to CQE message, and then knows that RDMA WRITE operation is complete by parsing RDMA SEND WR At, at this point, business device 6_2 learn RDMA WRITE operation complete, that is, realize and set business by RDMA SEND WR The message that the RDMA WRITE operation that standby 6_1 is initiated is completed is indicated to business device 6_2.And in fact, being based on business device 6_1 The description for sending WR to business device 6_2 according to serial mode sequence, then when business device 6_2 receives RDMA SEND WR, Really mean that RDMA WRITE operation successfully completes.In other words, in the present embodiment 1, once business device 6_2 receives RDMA SEND WR then means that RDMA WRITE operation is completed really.In this way, then meaning in the present embodiment 1 in RDMA WRITE Business device 6_2 will know that RDMA WRITE operation is completed in operating process.

Meanwhile business device 6_1 can also execute following step 605 after sending RDMA SEND WR.

Step 605, business device 6_1 generates CQE message (Opcode is RADM SEND), and passes through parsing CQE message (Opcode is RADM SEND) determines that the RDMA WRITE operation that this equipment is initiated is completed.

So far, by step 605, business device 6_1 also knows that the RDMA WRITE operation that this equipment is initiated is completed, and realizes It indicates that this business device 6_1 is generated after RDMA SEND WR is sent completely by RDMA SEND WR and corresponding completes CQE Message (Opcode is RDMA SEND), is completed with the RDMAWRITE operation for indicating that this equipment is initiated.So far, business device 6_1 Know that the RDMA WRITE operation that this equipment is initiated is completed

So far, process shown in Fig. 6 is completed.

It can be seen that in the present embodiment 1 from process shown in Fig. 6, business device 6_2 is receiving the last one in WR chain The RDMA WRITE behaviour that business device 6_1 is initiated can be known after WR, that is, RDMA SEND WR according to the RDMA SEND WR received It completes, realizes business device 6_2 in the process of RDMA WRITE operation you can learn that the completion of RDMA WRITE operation, is incited somebody to action Business device 6_2 knows the time advance that RDMA WRITE operation is completed, and improves the treatment effeciency of RDMA WRITE operation.

Embodiment 1 is described so that RDMAWRITE is operated as an example above.

So far, the description of embodiment 1 is completed.

In another example, the first above-mentioned specified WR refers to: Opcode is that the WR of RDMA WRITE (is denoted as WR700).It should be noted that limiting the first specified WR here is specifically Opcode as the WR of RDMA WRITE, it is therefore an objective to pass through The WR that Opcode is RDMA WRITE indicates that RDMA operation such as RDMA WRITE operation or RDMA READ operation are completed.Under Text is described in detail by the citing of embodiment 2.

Embodiment 2:

It is the RDMA READ operation implementation flow chart of the embodiment of the present application 2 referring to Fig. 7, Fig. 7.As shown in fig. 7, the process It can comprise the following steps that

Step 701, business device 7_1 is initiated to business device 7_2 for requesting the RDMA SEND of memory address to operate (being denoted as RDMA SEND operation S31).

As one embodiment, RDMA SEND operation S31 here carries pending RDMA READ operation and is wanted The Data Identification of the data (being denoted as data D700) of reading, the purpose is to for obtaining business device 7_2 locally for record number According to the memory address of D00.

Data mark after business device 7_2 receives RDMA SEND operation S31, according to RDMA SEND operation S31 carrying Knowledge finds data D700, if business device 7_2 discovery data D700 has been recorded to local memory, will be used in local memory Record the memory address (being denoted as M311) of data D700 and according to before with the negotiation of business device 7_1 from local memory It chooses one piece of free address (being denoted as M312) and is encapsulated in RDMA SEND operation (be denoted as RDMA SEND operation S32) and return to industry Be engaged in equipment 7_1；If business device 7_2 discovery data D700 is not recorded also to local memory, first data D700 is recorded to this Ground memory, later by the memory address (being denoted as M311) for being used to record data D700 in local memory and according to before and industry Business equipment 7_1 negotiation is chosen from local memory one piece of free address (being denoted as M312) be encapsulated in RDMA SEND operate (be denoted as RDMA SEND operate S32) in return to business device 7_1.

It should be noted that in the present embodiment 2, business device 7_2 is by local memory such as the description of above-mentioned steps 101 In for record data D700 memory address (being denoted as M311) and free address block (M312) be encapsulated in RDMA SEND behaviour Return to business device 7_1 concretely in making and (being denoted as RDMA SEND operation S32): business device 7_2 is according to data D700's Memory address M311 is divided into multiple address blocks by data volume, and the size of the address block of the division can be big with free address block M312 It is small equal, it can not also wait, later, business device 7_2 is long by each address block being divided into and the corresponding data of each address block Degree and free address block M312, which are encapsulated in, to be returned to business in RDMA SEND operation (be denoted as RDMA SEND operation S32) and sets Standby 7_1.

Step 702, the RDMA SEND that business device 7_1 receives that business device 7_2 is returned operates S32, according to RDMA The memory address M311 (divided address block) that SEND operation S32 is carried generates at least one for realizing RDMA READ behaviour The WR (being denoted as RDMA READ WR) of work.

In this step 702, the Opcode for each RDMA READ WR that business device 7_1 is generated is RDMA READ.

Step 703, business device 7_1 generates the WR700 for completing to business device 5_2 instruction RDMA READ operation, The Opcode of WR700 is RDMA WRITE, and each RDMA READ WR is formed WR chain by business device 7_1 together with WR700, In, WR700 is the last one WR of WR chain.

The WR700 that this step 703 generates is generated according to free address block M312, and WR700 carries M312.As one Embodiment, WR structure as shown in connection with fig. 2, here, the Opcode of WR700 is RDMA WRITE, and Remote_addr_info takes Band free address block M312, Localbuf_addr_info are the memory address that business device 7_1 locally records designated identification, Flag is IB_SEND_SIGNALED, and Next is sky.Here designated identification refers to that business device 7_1 and business device 7_2 is assisted The mark for being used to indicate RDMA operation completion of quotient.

As one embodiment, in this step 703, each RDMA READ WR, WR700 composition WR chain can specifically be wrapped It includes: the Next of first RDMA READ WR is directed toward second RDMA READ WR, by second RDMA READ WR's Next is directed toward third RDMA READ WR, and so on, the Next concatenation WR700 of the last one RDMA READ WR is final Form a WR chain.

Finally realize that business device 7_1 is that RDMA READ operation generates the WR chain for meeting following condition by step 703: The last one WR is used to complete to the RDMA READ operation that business device 7_2 instruction business device 7_1 is initiated in WR chain.

Step 704, business device 7_1 sends the WR in WR chain to business device 7_2 in sequence.

As one embodiment, in this step 704, business device 7_1 is sent out according to serial mode sequence to business device 7_2 Send the WR in WR chain.

In this step 704, after business device 7_1 sends the WR in WR chain to business device 7_2, business device 7_2 meeting Receive WR.

When business device 7_2 receives RDMA READ WR, executed according to the description of above-mentioned steps 504；

And when business device 7_2 receives WR700, then business device 7_2 can be from the Localbuf_addr_info of WR700 It reads designated identification and the free address block M312 indicated to the Remote_addr_info of WR700 is written.It should be noted that In the application, business device 7_2 is after sending free address block M312 to business device 7_1, and business device 7_2 is (specifically The RDMA application program of business device 7_2) whether meeting real-time monitoring free address block M312 is written designated identification, once it monitors Designated identification is written in free address block M312, and based on the above-mentioned description to designated identification, then business device 7_2 will be considered that RDMA READ operation is completed, at this time, it is meant that business device 7_2 knows that RDMA READ operation is completed, that is, realizing will by WR700 The message that the RDMA READ operation that business device 7_1 is initiated is completed is indicated to business device 7_2.And in fact, being set based on business Standby 7_1 sends the description of WR according to serial mode sequence to business device 7_2, then when business device 7_2 receives WR700, also really Mean that RDMA READ operation successfully completes in fact.In other words, in the present embodiment 2, once business device 7_2 receives WR700, then Really mean that RDMA READ operation is completed.In this way, then meaning the industry in RDMA READ operation process in the present embodiment 2 Business equipment 7_2 will know that RDMA READ operation is completed.

Meanwhile business device 7_1 can execute following step 705 after sending WR700.

Step 705, business device 7_1 generates CQE message (Opcode is RADM WRITE), and passes through parsing CQE message (Opcode is RADM WRITE) determines that the RDMA READ operation that this equipment is initiated is completed.

Here, because the Flag of WR700 is IB_SEND_SIGNALED, IB_SEND_SIGNALED, business device are based on 7_1 can be generated CQE message (Opcode be RADM WRITE) after sending WR700, and (Opcode is by parsing CQE message RADM WRITE)) determine that WR chain corresponding RDMA READ operation in the place WR700 is completed, that is, it realizes and this is indicated by WR700 Business device 7_1 generates corresponding completion CQE message (Opcode is RDMA SEND) after WR700 is sent completely, to indicate this The RDMA READ operation that equipment is initiated is completed.So far, the RDMA READ operation that business device 7_1 knows that this equipment is initiated is complete At.

So far, process shown in Fig. 7 is completed.

It can be seen that in the present embodiment 2 from process shown in Fig. 7, business device 7_2 (hardware such as network interface card) is receiving WR The free address block that can be sent according to the WR700 received in this business device 7_2 after the last one WR, that is, WR700 in chain Designated identification is written in M312, and the description based on designated identification, business device 7_2 can know business device 7_ according to designated identification The 1 RDMA READ operation initiated is completed, and realizes business device 7_2 in the process of RDMA READ operation you can learn that RDMA READ operation is completed, and business device 7_2 is known the time advance that RDMA READ operation is completed, improves RDMA READ operation Treatment effeciency.

Describe embodiment 2 by taking RDMA READ operation as an example above, RDMA WRITE operation similar to RDMA READ operation, Here it no longer describes one by one.

So far, the description of embodiment 2 is completed.

Embodiment 3:

Compared to above-described embodiment 1, embodiment 2, additionally generation is not needed in the present embodiment 3 and is used to indicate RDMA WRITE behaviour Make the second specified WR completed.

The present embodiment 3 is applicable in the case where RDMA WRITE operation.

As one embodiment, in the present embodiment 3, the WR chain for the condition that meets is generated in above-mentioned steps 401 for RDMA operation Including step a1 to step a5:

Step a1, from the opposite end business device obtain the RDMA WRITE operation needed for third memory address, with And it is idle and with continuous 4th memory address of third memory address；

Step a2 is generated according to the third memory address and (is denoted as realizing the WR of the RDMA WRITE operation RDMA WRITE WR)；Different RDMAWRITE WR carry different opposite end memory address, what all RDMAWRITE WR were carried Opposite end memory address forms the third memory address.

WR structure as shown in connection with fig. 2, then in this step a2, each RDMA WRITE WR passes through opposite end memory address word Section carries opposite end memory address.Specifically, each RDMA WRITE WR opposite end memory address field carry to end memory Location originates memory address by opposite end and address size (also referred to as opposite end memory address length) forms.Wherein, each The opposite end starting memory address that the opposite end memory address field of RDMAWRITE WR carries is different, but each RDMAWRITE WR The opposite end starting memory address and address size that opposite end memory address field carries are with combining composition third memory Location.

Step a3, finds the second specified WR from the RDMAWRITE WR of generation, the second specified WR carry to end memory Address is closest to the 4th memory address；

Step a4 carries out specified modification to the described second specified WR, to indicate the opposite end business device the described 4th Designated identification is written in memory address, and the designated identification is that this equipment and being used to indicate for opposite end business device negotiation are described The mark that RDMAWRITE operation is completed.

Here, specified modification includes: that the second specified WR opposite end memory address length carried is increased the 4th memory The address size that address occupies, carries the 5th memory address in the second specified WR, and the 5th memory address is this equipment use In the memory address of record designated identification.WR structure as shown in connection with fig. 2, then specifically, above-mentioned specified modification can include: by The address size that opposite end memory address field (Remote_addr_info) carries in two specified WR increases by the 4th memory address and accounts for Address size；Increase the local memory address field for carrying the 5th memory address newly in the second specified WR (Localbuf_addr_info), the 5th memory address (being denoted as M413) is the memory that local memory is used to record designated identification Location.

The RDMA WRITE WR of generation is formed the WR chain by step a5, wherein the second specified WR is the WR The last one WR of chain.

The present embodiment 3 is described in detail below:

Implementation flow chart is operated referring to the RDMAWRITE that Fig. 8, Fig. 8 are the embodiment of the present application 3.As shown in figure 8, the process It can comprise the following steps that

Step 801, business device 8_1 is initiated to business device 8_2 for requesting the RDMA SEND of memory address to operate (being denoted as RDMA SEND operation S41).

As one embodiment, RDMA SEND operation S41 here carries pending RDMA WRITE operation and is wanted The size of data of the data (being denoted as data D800) of write-in, the purpose is to for obtaining business device 8_2 sheet from business device 8_2 The memory address of data D800 is written for record traffic equipment 8_1 by RDMA WRITE operation for ground.

Size of data after business device 8_2 receives RDMASEND operation S41, according to RDMASEND operation S41 carrying It determines and is designed for the memory address that data D800 is written by RDMA WRITE operation by record traffic equipment 8_1 in local memory (as above-mentioned third memory address, be denoted as M411) will be used to record the memory address of data D800 later in local memory It (above-mentioned third memory address, be denoted as M411) and is chosen from local memory according to before with the negotiation of business device 8_1 One piece of free time and with the continuous address M411 (the 4th above-mentioned memory address, be denoted as M412) be encapsulated in RDMA SEND operation (note To return to business device 8_1 in RDMA SEND operation S42).

It should be noted that such as the description of above-mentioned steps 101, in the present embodiment 3, business device 8_2 by M411 and M412 is encapsulated in RDMA SEND operation S42 and returns to business device 8_1 concretely: business device 8_2 is according to data D800 Data volume M411 is divided into multiple address blocks, the size of the address block of the division can be equal in magnitude with M412, can not also wait, Later, business device 8_2 is by each address block being divided into and the corresponding data length of each address block and free address block M412 is encapsulated in RDMASEND operation S42 and returns to business device 8_1.

Step 802, the RDMA SEND that business device 8_1 receives that business device 8_2 is returned operates S42, according to RDMASEND The memory address M411 that operation S42 is carried generates at least one and (is denoted as RDMA for realizing the WR of RDMA WRITE operation WRITE WR)。

This step 802 is similar to above-mentioned step 602.

For example, if M411 is divided into 1 to 128 address block, the corresponding data length of each address block is 4k, then according to First WR is generated according to first address block, generates second WR according to second address block, and so on, eventually generate 128 WR, in this way, in this step 802, what the WR chain that business device 8_1 is generated was made of in order 128 WR.

Step 803, business device 8_1 finds the second specified WR from the RDMA WRITE WR of generation, and described second is specified The opposite end memory address that WR is carried is closest to M412.

It is continuous with M411 with M412, and for M412 is in after M411, if business device 8_1 according to M411 according to Address from front to back when being sequentially generated RDMA WRITE WR, the second specified WR that this step 803 is found is exactly the last one RDMA WRITE WR；Illustrate again, if business device 8_1 according to M411 according to address from back to front be sequentially generated RDMA When WRITE WR, the second specified WR that this step 803 is found is exactly first RDMA WRITE WR.Finally, this step 803 is looked for Opposite end memory address entrained by the specified WR of second arrived is closest to M412.

Step 804, business device 8_1 carries out such as above-mentioned specified modification to the second specified WR, to indicate business device 8_1 Designated identification is written in M412, designated identification is used to indicate RDMAWRITE for what business device 8_1 and business device 8_2 negotiated Operate the mark completed.

By taking the last one RDMA WRITE WR that business device 8_1 is generated is the second specified WR as an example, then had based on above-mentioned The description for closing specified modification, in this step 804, as shown in figure 8, the structure of the last one RDMA WRITE WR is compared to other RDMA WRITE WR makes following update:

Add additional a Localbuf_addr_info (be denoted as Localbuf_addr_info2, it is existing to be denoted as Localbuf_addr_info1), also, the Remote_addr_info address size that includes increases what above-mentioned M412 was occupied Address size.By the update, the memory address that business device 8_1 is indicated from Localbuf_addr_info2 can be indicated M413 reads designated identification and the designated identification of reading is written to M412, is specifically described below.

Step 805, the WR for realizing the RDMA WRITE operation of generation is formed into the WR chain, wherein described Second specified WR is described the last one WR of WR chain.

By taking the last one RDMA WRITE WR that business device 8_1 is generated is the second specified WR as an example, then a reality is used as Apply example, in this step 805, the WR for realizing the RDMA WRITE operation of generation, which is formed the WR chain, can specifically be wrapped It includes: the Next of first RDMA WRITE WR is directed toward second RDMA WRITE WR, by second RDMA WRITE WR's Next is directed toward third RDMA WRITE WR, and so on, finally concatenate all RDMA WRITE WR to form a WR Chain.

Finally realize that business device 8_1 is that RDMA WRITE operation generates the WR chain for meeting following condition by step 805: The last one WR is used to complete to the RDMA WRITE operation that business device 8_2 instruction business device 6_1 is initiated in WR chain.

Step 806, business device 8_1 sends the WR in WR chain to business device 8_2 in sequence.

In this step 806, business device 8_1 sends the WR in WR chain according to serial mode sequence to business device 8_2.

After business device 8_1 sends the WR in WR chain to business device 8_2, business device 8_2 will receive WR.Work as business When equipment 8_2 receives the last one WR of WR chain, based on as above to the description of the last one WR, then business device 8_2 can be in sky Designated identification is written in not busy address block M412.It should be noted that sending free address block in business device 8_2 in the application After M412, business device 8_2 (RDMA application program) can real-time monitoring free address block M412 whether be written designated identification, one Designated identification is written in denier free address block M412, and business device 8_2 is easy to monitor in time.Description based on designated identification, When business device 8_2 (RDMA application program) monitor free address block M412 be written designated identification, then mean RDMA WRITE operation is completed, at this point, business device 8_2 knows that RDMA WRITE operation is completed, that is, is realized business device 8_2 and is known The RDMA WRITE operation that business device 8_1 is initiated is completed.And in fact, based on business device 8_1 according to serial mode sequence The description of WR is sent to business device 8_2, then when business device 8_2 receives the last one WR of above-mentioned WR chain, also meaning really RDMA WRITE operation successfully complete.In other words, in the present embodiment 3, once business device 8_2 receives the last of above-mentioned WR chain One WR then means that RDMA WRITE operation is completed really.In this way, then meaning to grasp in RDMA WRITE in the present embodiment 3 Making business device 8_2 in process will know that RDMA WRITE operation is completed.

Meanwhile business device 8_1 executes following step 807 after sending the last one WR.

Step 807, business device 8_1 generates CQE message (Opcode is RADM WRITE), and passes through parsing CQE message (Opcode RADMWRITE) determines that the RDMA WRITE operation that this equipment is initiated is completed.

Here, because the Flag of the last one WR is IB_SEND_SIGNALED, it is based on IB_SEND_SIGNALED, then industry Equipment 8_1 be engaged in after sending the last one WR, CQE message (Opcode is RADM WRITE) can be generated, and disappear by parsing CQE Breath (Opcode is RADM WRITE)) determine that RDMA WRITE operation is completed, so far, business device 8_1 knows that this equipment is initiated RDMA WRITE operation complete.

So far, process shown in Fig. 8 is completed.

It can be seen that in the present embodiment 3 from process shown in Fig. 8, business device 8_1 passes through to for realizing RDMA Some RDMA WRITE WR (the second specified WR above) of WRITE operation carries out specified modification as above, then may be implemented Business device 8_2 is in the process of RDMA WRITE operation you can learn that the completion of RDMA WRITE operation, improves RDMA WRITE The treatment effeciency of operation.

So far, the description of embodiment 3 is completed.

Method provided by the present application is described above, device provided by the present application is described below:

Referring to Fig. 9, Fig. 9 is structure drawing of device provided by the present application.The device is applied to business device, comprising:

As one embodiment, the processing module is specifically used for:

It is used for according to from the generation of the first memory address needed for the RDMA operation that the opposite end business device has obtained Realize the WR of the RDMA operation；

It generates and is used to indicate the first specified WR that the RDMA operation is completed；

The WR chain will be formed for realizing the WR of the RDMA operation, the first specified WR, wherein described first refers to Determining WR is described the last one WR of WR chain.

As one embodiment, the first specified WR are as follows: Opcode is the WR that RDMA sends SEND type.

As one embodiment, the first specified WR includes the second memory address；Second memory address are as follows: from institute State the free address in addition to first memory address of opposite end business device acquisition；

The first specified WR is used to indicate the opposite end business device and designated identification is written in second memory address, The designated identification is this equipment and the mark for being used to indicate the RDMA operation and completing that the opposite end business device is negotiated.

As one embodiment, the RDMA operation is RDMA WRITE operation；The processing module is specifically used for:

From the opposite end business device obtain the RDMA WRITE operation needed for third memory address and free time And with continuous 4th memory address of third memory address；

The WR for realizing the RDMA WRITE operation is generated according to the third memory address；Different WR carry different Opposite end memory address, the opposite end memory address that all WR are carried forms the third memory address；

The second specified WR is found from the WR for realizing the RDMA WRITE operation of generation, the second specified WR is carried Opposite end memory address closest to the 4th memory address；

Specified modification is carried out to the described second specified WR, to indicate the opposite end business device in the 4th memory address Designated identification is written, the designated identification is that this equipment and being used to indicate for opposite end business device negotiation are described The mark that RDMAWRITE operation is completed；The specified modification includes: in the opposite end memory address information for carrying the second specified WR Address size increase the 4th memory address occupy address size；The 5th memory address is carried in the second specified WR, it is described 5th memory address is the memory address that this equipment is used to record designated identification；

The WR for realizing the RDMA WRITE operation of generation is formed into the WR chain, wherein described second is specified WR is described the last one WR of WR chain.

So far, device description shown in Fig. 9 is completed.

Referring to Figure 10, Figure 10 is the hardware structural diagram of Fig. 9 shown device provided by the present application.As shown in Figure 10, should Hardware configuration includes:

Processor 1001, the machine readable storage medium 1002 for being stored with machine-executable instruction.

Wherein, processor 1001 can be communicated with machine readable storage medium 1002 via system bus 603.Also, it handles Device 1001 passes through reading and executes machine-executable instruction corresponding with memory pool access method in machine readable storage medium 1002, Above-described memory pool access method can be performed in processor 1001.

Machine readable storage medium 1002 referred to herein can be any electronics, magnetism, optics or other physics and deposit Storage device may include or store information, such as executable instruction, data, etc..For example, machine readable storage medium may is that Random access memory (English: Radom Access Memory, referred to as: RAM), volatile memory, nonvolatile memory, Flash memory, memory driver (such as hard disk drive), solid state hard disk, any kind of storage dish (such as CD, dvd) or class As storage medium or their combination.

So far, hardware configuration description shown in Fig. 10 is completed.

In this application, a kind of machine readable storage medium including machine-executable instruction, such as Figure 10 are additionally provided In machine readable storage medium 1002, the machine-executable instruction can execute by the processor 1001 in internal storage access device To realize memory pool access method described above.

Specifically, executable by calling and executing machine corresponding with memory pool access method in machine readable storage medium The operation in the above memory pool access method can be performed in instruction, processor 1001.

The foregoing is merely the preferred embodiments of the application, not to limit the application, all essences in the application Within mind and principle, any modification, equivalent substitution, improvement and etc. done be should be included within the scope of the application protection.

Claims

1. a kind of memory pool access method, which is characterized in that this method is applied to business device, comprising:

The work request WR chain that RDMA operation generates the condition that meets, the condition are as follows: in WR chain most are directly accessed for long-distance inner The latter WR is used to indicate RDMA operation completion；

When initiating the RDMA operation to opposite end business device, the WR chain is sent to the opposite end business device in sequence In WR.

2. the method according to claim 1, wherein described directly access RDMA operation generation completely for long-distance inner The work request WR chain of sufficient condition includes:

It generates according to from the first memory address needed for the RDMA operation that the opposite end business device obtains for realizing institute State the WR of RDMA operation；

The WR chain will be formed for realizing the WR of the RDMA operation, the first specified WR, wherein the first specified WR For described the last one WR of WR chain.

3. according to the method described in claim 2, it is characterized in that,

The first specified WR are as follows: action type is the WR that RDMA sends SEND type.

4. according to the method described in claim 2, it is characterized in that,

The first specified WR are as follows: action type is the WR of RDMA WRITE type；

The first specified WR includes the second memory address；Second memory address are as follows: obtained from the opposite end business device The free address in addition to first memory address；

The first specified WR is used to indicate the opposite end business device and designated identification is written in second memory address, described Designated identification is this equipment and the mark for being used to indicate the RDMA operation and completing that the opposite end business device is negotiated.

5. the method according to claim 1, wherein the RDMA operation is RDMA WRITE operation；

It is described for long-distance inner directly access RDMA operation generation and meets the work request WR chain of condition include:

The WR for realizing the RDMA WRITE operation is generated according to the third memory address；Different WR carry different pairs End memory address, the opposite end memory address that all WR are carried form the third memory address；

The second specified WR, pair that the second specified WR is carried are found from the WR for realizing the RDMA WRITE operation of generation End memory address is closest to the 4th memory address；

Specified modification is carried out to the described second specified WR, to indicate that the opposite end business device is written in the 4th memory address Designated identification, the designated identification are that the RDMAWRITE that is used to indicate that this equipment is negotiated with the opposite end business device is grasped Make the mark completed；The specified modification includes: that the second specified WR opposite end memory address length carried is increased the described 4th The address size that memory address occupies；The 5th memory address is carried in the second specified WR, the 5th memory address is originally to set It is ready for use on the memory address of record designated identification；

Generation is formed into the WR chain for realizing the WR of the RDMA WRITE operation, wherein the second specified WR is Described the last one WR of WR chain.

6. a kind of internal storage access device, which is characterized in that the device is applied to business device, comprising:

Processing module generates the work request WR chain for the condition that meets, the item for directly accessing RDMA operation for long-distance inner Part are as follows: the last one WR is used to indicate RDMA operation completion in WR chain；

Business module, for being set to the opposite end business in sequence when initiating the RDMA operation to opposite end business device Preparation send the WR in the WR chain.

7. device according to claim 6, which is characterized in that the processing module is specifically used for:

8. device according to claim 7, which is characterized in that

9. device according to claim 7, which is characterized in that the first specified WR are as follows: action type RDMA The WR of WRITE type；

10. device according to claim 6, which is characterized in that the RDMA operation is RDMAWRITE operation；

The processing module is specifically used for:

Specified modification is carried out to the described second specified WR, to indicate that the opposite end business device is written in the 4th memory address Designated identification, the designated identification are that the RDMAWRITE that is used to indicate that this equipment is negotiated with the opposite end business device is grasped Make the mark completed；The specified modification includes: will be described in the opposite end memory address address size increase of the second specified WR carrying The address size that 4th memory address occupies；The 5th memory address is carried in the second specified WR, the 5th memory address is This equipment is used to record the memory address of designated identification；