CN110535811A - Remote memory management method and system, server-side, client, storage medium - Google Patents
Remote memory management method and system, server-side, client, storage medium Download PDFInfo
- Publication number
- CN110535811A CN110535811A CN201810515949.3A CN201810515949A CN110535811A CN 110535811 A CN110535811 A CN 110535811A CN 201810515949 A CN201810515949 A CN 201810515949A CN 110535811 A CN110535811 A CN 110535811A
- Authority
- CN
- China
- Prior art keywords
- client
- message
- message blocks
- server
- remote
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
- H04L67/133—Protocols for remote procedure calls [RPC]
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/50—Network services
- H04L67/60—Scheduling or organising the servicing of application requests, e.g. requests for application data transmissions using the analysis and optimisation of the required network resources
Landscapes
- Engineering & Computer Science (AREA)
- Computer Networks & Wireless Communication (AREA)
- Signal Processing (AREA)
- Computer And Data Communications (AREA)
Abstract
The present invention discloses a kind of remote memory management method and system, server-side, client, computer readable storage medium, this method comprises: one piece of memory of application is registered to network interface card as message area, and by the message area；It receives client and the message blocks application that RDMA atomic operation is sent is accessed by remote direct memory, be the client assignment messages block and the exclusive time for specifying the message blocks；The remote request that the client is sent by RDMA Write operation is received, the information of the remote request is stored into the message blocks；Handle and respond the remote request of the client.The present invention receives message blocks application that client is sent by RDMA operation and remote request is simultaneously handled by being registered to network interface card as message area and by message area in server-side application memory；It can be effectively reduced the EMS memory occupation in cross-node message process, while promoting remote request handling capacity.
Description
Technical field
The present invention relates to field of communication technology more particularly to a kind of remote memory management method and system, server-side, clients
End, computer readable storage medium.
Background technique
RPC (Remote Procedure Call, remote procedure call) efficiency under distributed environment determines that system is whole
I/O (Input/Output, input/output) performance of body, traditional distributed system is by being based on TCP/IP
(Transmission Control Protocol/Internet Protocol/Internet Protocol passes transport control protocol
View/Internet Protocol) remote procedure call carry out data transmission, message transmission memory management be integrated into network association
Stack is discussed, without considering in system designs.
In recent years, in order to promote the computational efficiency of distributed memory, RDMA (Remote Direct Memory Access,
Remote direct memory access) technology widely studied and is applied to storage system, to accelerate upper layer calculated performance.RDMA communication
In the process, transmission is semantic has difference, and memory remotely can be accessed directly, therefore in remote procedure call of the design based on RDMA
When, remote memory management is particularly important.
Currently, can be effectively reduced in cross-node message process there is no the memory management module for being directed to RDMA
EMS memory occupation.Meanwhile in parallel processing, the prior art still uses simple batch processing model, cannot make full use of network interface card
The high throughput characteristic having.
Summary of the invention
In view of this, the embodiment of the present invention be designed to provide a kind of remote memory management method and system, server-side,
Client, computer readable storage medium, with solve the EMS memory occupation that how to be effectively reduced in cross-node message process with
And the problem of promoting remote request handling capacity.
It is as follows that the embodiment of the present invention solves technical solution used by above-mentioned technical problem:
According to an aspect of an embodiment of the present invention, a kind of remote memory management method provided, this method is for servicing
End, which comprises
Apply for that one piece of memory is registered to network interface card as message area, and by the message area；
The message blocks application that client is sent by RDMA atomic operation is received, for the client assignment messages block and is referred to
The exclusive time of the fixed message blocks；
The remote request that the client is sent by RDMA Write operation is received, the information storage of the remote request is arrived
In the message blocks；Handle and respond the remote request of the client.
Other side according to an embodiment of the present invention, a kind of server-side provided, the server-side include: memory,
Processor and it is stored in the remote memory management program that can be run on the memory and on the processor, in the distal end
Deposit the step of above-mentioned remote memory management method is realized when management program is executed by the processor.
Other side according to an embodiment of the present invention, a kind of computer readable storage medium provided, the computer
It is stored with remote memory management program on readable storage medium storing program for executing, is realized when the remote memory management program is executed by processor
The step of remote memory management method stated.
Other side according to an embodiment of the present invention, a kind of remote memory management method provided, this method is for visitor
Family end, which comprises
Message blocks application is sent to server-side by RDMA atomic operation；Wherein, one piece of memory of the server-side application is made
For message area, and the message area is registered to network interface card；Receive the message blocks that the client is sent by RDMA atomic operation
Application is the client assignment messages block and the exclusive time for specifying the message blocks；
After message blocks application success, remote request is sent to the server-side by RDMA Write operation, and wait
Response to the server-side；Wherein, the server-side receives the remote request that the client is sent by RDMA Write operation,
The information of the remote request is stored into the message blocks；Handle and respond the remote request of the client.
Other side according to an embodiment of the present invention, a kind of client provided, the client include: memory,
Processor and it is stored in the remote memory management program that can be run on the memory and on the processor, in the distal end
Deposit the step of above-mentioned remote memory management method is realized when management program is executed by the processor.
Other side according to an embodiment of the present invention, a kind of computer readable storage medium provided, the computer
It is stored with remote memory management program on readable storage medium storing program for executing, is realized when the remote memory management program is executed by processor
The step of remote memory management method stated.
Other side according to an embodiment of the present invention, a kind of remote memory management system provided, which is characterized in that institute
The system of stating includes above-mentioned server-side and above-mentioned client.
The remote memory management method and system of the embodiment of the present invention, server-side, client, computer-readable storage medium
Matter receives client and passes through RDMA operation by being registered to network interface card as message area and by message area in server-side application memory
The message blocks application of transmission and remote request are simultaneously handled；The memory that can be effectively reduced in cross-node message process accounts for
With, while promoting remote request handling capacity.
Detailed description of the invention
Fig. 1 is the RDMA data transmission structure schematic diagram of the embodiment of the present invention；
Fig. 2 is the remote memory management method flow diagram of first embodiment of the invention；
Fig. 3 is the server-side structural schematic diagram of second embodiment of the invention；
Fig. 4 is the remote memory management method flow diagram of fourth embodiment of the invention；
Fig. 5 is the client terminal structure schematic diagram of fifth embodiment of the invention；
Fig. 6 is the remote memory schematic diagram of management system structure of seventh embodiment of the invention；
Fig. 7 is workspace and the preheating zone structural schematic diagram of the embodiment of the present invention；
Fig. 8 is the allocation field structural schematic diagram of the embodiment of the present invention.
The embodiments will be further described with reference to the accompanying drawings for the realization, the function and the advantages of the object of the present invention.
Specific embodiment
In order to be clearer and more clear technical problems, technical solutions and advantages to be solved, tie below
Drawings and examples are closed, the present invention will be described in further detail.It should be appreciated that specific embodiment described herein is only
To explain the present invention, it is not intended to limit the present invention.
The present embodiment in order to better understand below says the term of the present embodiment before illustrating the present embodiment
It is bright:
DMA (Direct Memory Access, direct memory access), DMA allow certain hardware devices independently direct
Read/write memory, without a large amount of participations of CPU (Central Processing Unit, central processing unit), which is used for
Alleviate CPU to the processing pressure of peripheral hardware, entire data transmission procedure only needs CPU most starting to carry out initial transmission operation, so
Entire transmission movement is given to dma controller afterwards to execute completion.
RDMA is a kind of novel network communications technology, it is able to achieve in the case where both sides' operating system does not participate in directly
Directly access remote memory, and realize it is high handle up, the characteristic of low latency.RDMA is by allowing network adapter directly to transmit data
The zero-copy that data transmission is realized to the memory of other side, to eliminate the direct participation of CPU and Cache (caching), and is reduced
The Context switches of redundancy.Support that the network protocol stack of RDMA technology includes Infiniband, RoCE (RDMA at present
Converged Ethernet) and iWARP, the above two provide hardware technology by Mellanox and support, both rear due to using
The data link layer of common Ethernet, therefore can be completely compatible with Ethernet.
Fig. 1 illustrates the detailed process of RDMA communication: first by local (node 1) CPU with MMIO (Memory mapping
I/O, memory-mapped I/O) mode to network interface card initiate communications command, after local network interface card detects new order, with the side of DMA
Formula reads data to be transmitted from memory, and data are packaged, and carries out data transmission in RDMA network.Other side (node 2) net
After clamping receives data, data are directly write to the corresponding address area of memory in a manner of DMA, and will complete accordingly
Information, which is written to, completes queue, and whole process is not related to other side CPU participation, and has bypassed the kernel of both sides, realizes data biography
Defeated zero-copy.
Communication two party needs before establishing communication by following steps: opening network card equipment；Create protected field, the protection
Domain can be with the object binding that creates in the stage below, and to ensure data transmission security, any cross-domain operation will cause communication
Mistake；Registers memory, the stage register the memory of communication, and specific method is to establish this section of memory User space address and interior
The mapping table of address is deposited, and mapping table is stored to network interface card caching, while generating the key pair (lkey and rkey) of the application heap, net
Corresponding key need to be carried when being stuck in memory locally or remotely accessible carries out identity validation；Creation CQ (Completion Queue,
Complete queue), sender will can complete accordingly information after message is sent successfully or after recipient's reception message success
It is put into and completes queue, user, which can be repeatedly detected, completes queue to verify whether message transmission is completed；Create QP (Queue
Pair, queue to), QP is made of Send Queue (transmit queue) and Receive Queue (receiving queue), and sender is wanting
The message of transmission is put into transmit queue, while recipient is put into receiving queue for request is received, and both sides carry out in this way
Network communication；QP state initialization after communication two party creates one-to-one QP, needs to carry out a series of state of shaking hands and turns
It changes, until being successfully established communication link；QP can establish different connection types, including RC (Reliable
Connection), UC (Unreliable Connection) and UD (Unreliable Datagram)；Under RC mode, QP is only
It can be carried out one-to-one reliable transmission, data packet has corresponding confirmation message feedback after sending successfully；Under UC mode, QP carries out one
It is transmitted to one, no confirmation message feedback；UD mode is then defined without one-to-one, is also fed back without confirmation message；Three of the above passes
Defeated mode has the characteristics that different while also different to the degree of support of intercommunication primitive.
Memory calculates the demand for referring to data and the high real-time processing in face of magnanimity, traditional to be situated between with disk for storage
The storage system of matter is difficult to cope with new challenge because of slow access speed, carries out in fact so that storage system is transferred to memory
When the novel process mode that handles.Memory storage system mainly includes two classes, is memory database system and memory file respectively
System.The memory file system of relatively mainstream includes Alluxio, IGFS etc. at present.Alluxio is mainly used for solving Spark meter
The existing problem of frame is calculated, data processing performance is accelerated, and realizes single part of storage and the Trusted recovery of data using lineage.
IGFS is a cache file system between Computational frame and HDFS (Hadoop distributed file system), is mentioned to upper layer
The interface for being compatible with HDFS has been supplied, but unlike that HDFS, IGFS do not have individual meta data server, but has been used
The mode of Hash carries out data distribution.
RPC is a kind of telecommunication agreement, it is able to achieve the program allowed Yun Hang on one computer and remotely calls separately
Function above one computer, and user is not necessarily to be concerned about the communication interactive strategy of bottom.Remote procedure call is widely answered
Used in distributed system field, it uses the model of client-server, and calling process is always initiated by client, specifically
Including the sequence of function number will be called, the information packages such as function parameter are called to be sent to server-side, then server-side receives request simultaneously
It executes, implementing result is returned to client after being finished by server-side.
First embodiment
As shown in Fig. 2, first embodiment of the invention provides a kind of remote memory management method, this method is used for server-side,
The described method includes:
Step S11: one piece of memory of application is registered to network interface card as message area, and by the message area.
In the present embodiment, by, as message area, and being registered to network interface card in server-side application memory, to support client
Long-distance inner direct read/write, client can be remotely accessed directly by RDMA primitive, realize message between nodes transmission.Wherein, disappear
Breath area is used to store the information of the remote request of client.
Step S12: client is received by remote direct memory and accesses the message blocks application that RDMA atomic operation is sent, is
The client assignment messages block and the exclusive time for specifying the message blocks.
In the present embodiment, message area includes multiple independent message blocks, and client passes through before sending remote request
RDMA atomic operation is from the server-side lease message block, and the client individually occupies it within system specified time.
It should be noted that server-side can be detected simultaneously by multiple requests from the same client, carry out at batch
Reason and transmission promote response and handle up.
Step S13: receiving the remote request that the client is sent by RDMA Write operation, by the letter of the remote request
Breath storage is into the message blocks；Handle and respond the remote request of the client.
In the present embodiment, client is sent far by RDMA Write operation to server-side after message blocks application success
Journey request, and wait the response of the server-side.Server-side is after receiving the remote request of client, by remote request
Information is stored into message blocks.Server-side opens service thread poll scanning message area simultaneously, monitors the information of remote request
Respective handling is made later.
In a kind of embodiment, the method also includes:
The message area is divided into workspace and preheating zone；
By the workspace and the preheating zone, message blocks application and remotely ask that client described in parallel processing is sent
It asks.
In this embodiment, the message area of server-side is divided into workspace and preheating zone, can be used for accelerating server-side
Message Processing.Specifically, when the service thread of server-side handles the new request in the workspace, the open application in preheating zone, visitor
Family end can apply for the message blocks into preheating zone in advance, and will newly request filling preheating zone in advance.
In this embodiment, server-side can open multiple service threads, the difference in service thread supervisory messages area
Region is used for parallel processing.
In this embodiment, described by the workspace and the preheating zone, client described in parallel processing is sent
Message blocks application and remote request after further include:
Obtain the switching signal of the workspace and the preheating zone；
According to the switching signal, the workspace is switched to preheating zone and the preheating zone is switched to workspace.
Specifically, when service thread obtains switching signal, the remaining Message Processing of workspace is finished first, it then will be pre-
Hot-zone is switched to workspace, at the same time, workspace before is switched to preheating zone, and receive client application.
In a kind of embodiment, the client that receives is disappeared by what remote direct memory access RDMA atomic operation was sent
Cease block application, for the client assignment messages block and specify the message blocks the exclusive time after further include:
If the quantity of the client is greater than the quantity of the message blocks, shorten the exclusive time of the message blocks；
If the exclusive of the message blocks has timed out the phase, the information that will expire feeds back to the client, so that described
Client empties occupied state information and retransmits message blocks application.
In this embodiment, by " application ", the administrative mechanism of " expiring " forces multiple client according to time series
Grouping, service thread handles different groups of client request in different time sections, to reach timesharing effect.
Specifically, when client terminal quantity is more than the quantity of server-side message blocks, server-side forces to shorten client application
The exclusive time of the message blocks arrived.Particularly, when client terminal quantity is less than message number of blocks, occupancy of the client to message blocks
Time is set as endless；In this case, all clients assign to one group, and message area is used enough, and there is no need to carry out point
When handle and response, each client can exclusively enjoy always corresponding message blocks.After occupancy of the client to message blocks expires, service
End can incidentally expire information in message response information, client parses expire information after, need solicitation message block again.
At switch operating area and preheating zone, the client for including in workspace expires event service thread, service
Before thread executes switching, first entire scan workspace responds remaining request, and increases in the return information of the remaining request
" expiring " field information then increases " expiring " solicited message newly, and be sent to client for other clients, is used for notification message
Block use expires；After the client receives " expiring " information, local occupied state information is emptied, waits the set time
Afterwards, apply again；Based on above method, when client request density is high, whenever service thread executes message area switching, remnants
Request ratio is higher, and server-side carrys out notification message block more easily by the remaining request of response and fails, and individually sends fail message
Ratio it is relatively fewer, therefore introduce additional networks expense it is less；When client request density is low, since current network is hard
Part occupancy ratio is not high, therefore the additional notifications request introduced has no effect on overall performance；To sum up, based on server-side active feedback
Message releasing mechanism is adapted to the different scene of degree of concurrence.
In order to preferably illustrate workspace and preheating zone, it is illustrated below in conjunction with Fig. 7:
As shown in fig. 7, the message area of server-side is divided into workspace and preheating zone, server-side can open multiple service lines
Journey, the different zones in service thread supervisory messages area are used for parallel processing.Wherein four-headed arrow indicates that workspace and preheating zone can
It switches over, can be seen that the client request that service thread handles in different time sections different groups from intermediate timeline, reach
Timesharing effect is arrived.
In another embodiment, one piece of memory of the application is registered to network interface card as message area, and by the message area
Later further include:
It is configured to decide the allocation field of the message blocks application result of the client in the stem of the message blocks；
Regularly update the value of the allocation field.
As illustratively, the structure of allocation field be can refer to shown in Fig. 8, and the Message Zone Index in figure can be indicated
Message area call number, Client Number are client numbers, i.e. the application quantity of message blocks.Message Zone Index
It is all 32bits as the length of Client Number.Message blocks application process approximately as:
Initializing allocation field is (0,0), wherein represent message area first half for first 0 as preheating zone, second 0 table
The application quantity for showing current message block is 0.
Client executing RDMA-FETCH_AND_ADD atomic operation remotely executes the behaviour of atom adding one to allocation field
Make, while obtaining the value (m, n) before allocation field, if n is less than the total quantity of message blocks, represent message blocks application at
Function, otherwise message blocks application fails, and the set time is waited to execute same operation.Message blocks are realized using based on RDMA atomic operation
Application and distribution, it is therefore intended that the expense that discharges to message blocks application of server-side is reduced, in order to ensure the client end response time
In zone of reasonableness, the time is generally set lower than one millisecond, therefore the frequent application that message blocks occur is discharged operation, once
Application release Protocol Design is excessively complicated, will introduce huge expense for server-side, and cause systematic entirety that can decline, and is based on
The application process server-side of RDMA primitive greatly reduces the CPU overhead of application process without participating in.
In this embodiment, server-side regularly updates the value of the allocation field, for guiding client dynamically Shen
It please the message blocks.
As illustratively, when groups of clients number does not change (for two groups), server-side periodically (0, a), (1,
B) switching in, wherein 0 represents message area first half, 1 represents message area latter half, and a is represented currently to first half message
The occupancy situation in area, b then represent the occupancy situation to the latter half of message area；Client passes through the application of RDMA atomic operation
Cheng Zhong, if data are (0, a before obtaining1), and a1<Bmax, wherein BmaxThe total quantity of expression message blocks, (0, a1) represent visitor
The message blocks to message area first half have been applied at family end, otherwise application failure, wait the set time, again application for execution stream
Journey.
The remote memory management method of the embodiment of the present invention, by server-side application memory as message area and by message
Area is registered to network interface card, receives the message blocks application and remote request that client is sent by RDMA operation and is handled；It can
The EMS memory occupation in cross-node message process is effectively reduced, while promoting remote request handling capacity.
Second embodiment
As shown in figure 3, second embodiment of the invention provides a kind of server-side, the server-side includes: memory 21, processing
Device 22 and it is stored in the remote memory management program that can be run on the memory 21 and on the processor 22, the distal end
When storage management is executed by the processor 22, the step of for realizing remote memory management method as described below:
Apply for that one piece of memory is registered to network interface card as message area, and by the message area；
It receives client and the message blocks application that RDMA atomic operation is sent is accessed by remote direct memory, be the client
End assignment messages block and the exclusive time for specifying the message blocks；
The remote request that the client is sent by RDMA Write operation is received, the information storage of the remote request is arrived
In the message blocks；Handle and respond the remote request of the client.
When the remote memory management program is executed by the processor 22, it is also used to realize remote memory as described below
The step of management method:
The message area is divided into workspace and preheating zone；
By the workspace and the preheating zone, message blocks application and remotely ask that client described in parallel processing is sent
It asks.
When the remote memory management program is executed by the processor 22, it is also used to realize remote memory as described below
The step of management method:
Obtain the switching signal of the workspace and the preheating zone；
According to the switching signal, the workspace is switched to preheating zone and the preheating zone is switched to workspace.
When the remote memory management program is executed by the processor 22, it is also used to realize remote memory as described below
The step of management method:
If the quantity of the client is greater than the quantity of the message blocks, shorten the exclusive time of the message blocks；
If the exclusive of the message blocks has timed out the phase, the information that will expire feeds back to the client, so that described
Client empties occupied state information and retransmits message blocks application.
When the remote memory management program is executed by the processor 22, it is also used to realize remote memory as described below
The step of management method:
It is configured to decide the allocation field of the message blocks application result of the client in the stem of the message blocks；
Regularly update the value of the allocation field.
The server-side of the embodiment of the present invention, by being registered to net as message area and by message area in server-side application memory
Card receives the message blocks application and remote request that client is sent by RDMA operation and is handled；Can be effectively reduced across
EMS memory occupation in node messages transmittance process, while promoting remote request handling capacity.
3rd embodiment
Third embodiment of the invention provides a kind of computer readable storage medium, deposits on the computer readable storage medium
Remote memory management program is contained, for realizing described in first embodiment when the remote memory management program is executed by processor
Remote memory management method the step of.
The computer readable storage medium of the embodiment of the present invention, by that as message area and will disappear in server-side application memory
Breath area is registered to network interface card, receives the message blocks application and remote request that client is sent by RDMA operation and is handled；Energy
The EMS memory occupation in cross-node message process is enough effectively reduced, while promoting remote request handling capacity.
Fourth embodiment
As shown in figure 4, fourth embodiment of the invention provides a kind of remote memory management method, this method is used for client,
The described method includes:
Step S31: message blocks application is sent to server-side by RDMA atomic operation；Wherein, the server-side application one
Block memory is registered to network interface card as message area, and by the message area；The client is received to send by RDMA atomic operation
Message blocks application, for the client assignment messages block and specify exclusive time of the message blocks.
In the present embodiment, by, as message area, and being registered to network interface card in server-side application memory, to support client
Long-distance inner direct read/write, client can be remotely accessed directly by RDMA primitive, realize message between nodes transmission.Wherein, disappear
Breath area is used to store the information of the remote request of client.
In the present embodiment, message area includes multiple independent message blocks, and client passes through before sending remote request
RDMA atomic operation is from the server-side lease message block, and the client individually occupies it within system specified time.
It should be noted that client can send multiple requests simultaneously, server-side can be detected simultaneously by from same
Multiple requests of client carry out batch processing and transmission, promote response and handle up.
Step S32: it after message blocks application success, is remotely asked by RDMA Write operation to server-side transmission
It asks, and waits the response of the server-side；Wherein, the server-side receive the client sent by RDMA Write operation it is remote
Journey request, the information of the remote request is stored into the message blocks；Handle and respond the remote request of the client.
In the present embodiment, client is sent far by RDMA Write operation to server-side after message blocks application success
Journey request, and wait the response of the server-side.Server-side is after receiving the remote request of client, by remote request
Information is stored into message blocks.Server-side opens service thread poll scanning message area simultaneously, monitors the information of remote request
Respective handling is made later.
In one embodiment, if the message area is divided into workspace and preheating zone by the server-side, pass through
RDMA atomic operation sends message blocks application to the preheating zone.
In this embodiment, the message area of server-side is divided into workspace and preheating zone, can be used for accelerating server-side
Message Processing.Specifically, when the service thread of server-side handles the new request in the workspace, the open application in preheating zone, visitor
Family end can apply for the message blocks into preheating zone in advance, and will newly request filling preheating zone in advance.
In this embodiment, server-side can open multiple service threads, the difference in service thread supervisory messages area
Region is used for parallel processing.
In this embodiment, if the workspace is switched to preheating zone and incited somebody to action by the server-side according to switching signal
The preheating zone is switched to workspace, then sends message blocks to the workspace for being switched to preheating zone by RDMA atomic operation
Application.
Specifically, when service thread obtains switching signal, the remaining Message Processing of workspace is finished first, it then will be pre-
Hot-zone is switched to workspace, at the same time, workspace before is switched to preheating zone, and receive client application.
In one embodiment, after message blocks application success, if receiving arriving for the server-side feedback
Phase information then empties occupied state information and sends message blocks application to the server-side again.
In this embodiment, by " application ", the administrative mechanism of " expiring " forces multiple client according to time series
Grouping, service thread handles different groups of client request in different time sections, to reach timesharing effect.
Specifically, when client terminal quantity is more than the quantity of server-side message blocks, server-side forces to shorten client application
The exclusive time of the message blocks arrived.Particularly, when client terminal quantity is less than message number of blocks, occupancy of the client to message blocks
Time is set as endless；In this case, all clients assign to one group, and message area is used enough, and there is no need to carry out point
When handle and response, each client can exclusively enjoy always corresponding message blocks.After occupancy of the client to message blocks expires, service
End can incidentally expire information in message response information, client parses expire information after, need solicitation message block again.
At switch operating area and preheating zone, the client for including in workspace expires event service thread, service
Before thread executes switching, first entire scan workspace responds remaining request, and increases in the return information of the remaining request
" expiring " field information then increases " expiring " solicited message newly, and be sent to client for other clients, is used for notification message
Block use expires；After the client receives " expiring " information, local occupied state information is emptied, waits the set time
Afterwards, apply again；Based on above method, when client request density is high, whenever service thread executes message area switching, remnants
Request ratio is higher, and server-side carrys out notification message block more easily by the remaining request of response and fails, and individually sends fail message
Ratio it is relatively fewer, therefore introduce additional networks expense it is less；When client request density is low, since current network is hard
Part occupancy ratio is not high, therefore the additional notifications request introduced has no effect on overall performance；To sum up, based on server-side active feedback
Message releasing mechanism is adapted to the different scene of degree of concurrence.
In order to preferably illustrate workspace and preheating zone, it is illustrated below in conjunction with Fig. 7:
As shown in fig. 7, the message area of server-side is divided into workspace and preheating zone, server-side can open multiple service lines
Journey, the different zones in service thread supervisory messages area are used for parallel processing.Wherein four-headed arrow indicates that workspace and preheating zone can
It switches over, can be seen that the client request that service thread handles in different time sections different groups from intermediate timeline, reach
Timesharing effect is arrived.
In another embodiment, the message blocks application success is judged by following steps:
The value (a, b) of allocation field is obtained, if b is less than the total quantity of the message blocks, judges the message blocks application
Success；Wherein the allocation field is that the server-side is configured to decide disappearing for the client in the stem of the message blocks
Block application result is ceased, a indicates the message area call number, and b indicates the application quantity of the message blocks.
As illustratively, the structure of allocation field be can refer to shown in Fig. 8, and the Message Zone Index in figure can be indicated
Message area call number, Client Number are client numbers, i.e. the application quantity of message blocks.Message Zone Index
It is all 32bits as the length of Client Number.
The remote memory management method of the embodiment of the present invention, by server-side application memory as message area and by message
Area is registered to network interface card, receives the message blocks application and remote request that client is sent by RDMA operation and is handled；It can
The EMS memory occupation in cross-node message process is effectively reduced, while promoting remote request handling capacity.
5th embodiment
As shown in figure 5, fifth embodiment of the invention provides a kind of client, the client includes: memory 31, processing
Device 32 and it is stored in the remote memory management program that can be run on the memory 31 and on the processor 32, the distal end
When storage management is executed by the processor 32, the step of for realizing remote memory management method as described below:
Message blocks application is sent to server-side by RDMA atomic operation；Wherein, one piece of memory of the server-side application is made
For message area, and the message area is registered to network interface card；Receive the message blocks that the client is sent by RDMA atomic operation
Application is the client assignment messages block and the exclusive time for specifying the message blocks；
After message blocks application success, remote request is sent to the server-side by RDMA Write operation, and wait
Response to the server-side；Wherein, the server-side receives the remote request that the client is sent by RDMA Write operation,
The information of the remote request is stored into the message blocks；Handle and respond the remote request of the client.
When the remote memory management program is executed by the processor 32, it is also used to realize remote memory as described below
The step of management method:
If the message area is divided into workspace and preheating zone by the server-side, by RDMA atomic operation to described
Preheating zone sends message blocks application.
When the remote memory management program is executed by the processor 32, it is also used to realize remote memory as described below
The step of management method:
If the workspace is switched to preheating zone and is switched to the preheating zone by the server-side according to switching signal
Workspace then sends message blocks application to the workspace for being switched to preheating zone by RDMA atomic operation.
When the remote memory management program is executed by the processor 32, it is also used to realize remote memory as described below
The step of management method:
After message blocks application success, if receiving the information that expires of the server-side feedback, occupancy is emptied
Status information simultaneously sends message blocks application to the server-side again.
When the remote memory management program is executed by the processor 32, it is also used to realize remote memory as described below
The step of management method:
The value (a, b) of allocation field is obtained, if b is less than the total quantity of the message blocks, judges the message blocks application
Success；Wherein the allocation field is that the server-side is configured to decide disappearing for the client in the stem of the message blocks
Block application result is ceased, a indicates the message area call number, and b indicates the application quantity of the message blocks.
The client of the embodiment of the present invention, by being registered to net as message area and by message area in server-side application memory
Card receives the message blocks application and remote request that client is sent by RDMA operation and is handled；Can be effectively reduced across
EMS memory occupation in node messages transmittance process, while promoting remote request handling capacity.
Sixth embodiment
Sixth embodiment of the invention provides a kind of computer readable storage medium, deposits on the computer readable storage medium
Remote memory management program is contained, for realizing described in fourth embodiment when the remote memory management program is executed by processor
Remote memory management method the step of.
The computer readable storage medium of the embodiment of the present invention, by that as message area and will disappear in server-side application memory
Breath area is registered to network interface card, receives the message blocks application and remote request that client is sent by RDMA operation and is handled；Energy
The EMS memory occupation in cross-node message process is enough effectively reduced, while promoting remote request handling capacity.
7th embodiment
As shown in fig. 6, seventh embodiment of the invention provides a kind of remote memory management system, the system comprises server-sides
41 and client 42, server-side 41 can refer to second embodiment, client 42 can refer to the 5th embodiment, and therefore not to repeat here.
The remote memory management system of the embodiment of the present invention, by server-side application memory as message area and by message
Area is registered to network interface card, receives the message blocks application and remote request that client is sent by RDMA operation and is handled；It can
The EMS memory occupation in cross-node message process is effectively reduced, while promoting remote request handling capacity.
It should be noted that above-mentioned apparatus embodiment and embodiment of the method belong to same design, specific implementation process is detailed
See embodiment of the method, and the technical characteristic in embodiment of the method is corresponding applicable in Installation practice, which is not described herein again.
Through the above description of the embodiments, those skilled in the art can be understood that above-described embodiment side
Method can be realized by means of software and necessary general hardware platform, naturally it is also possible to be realized by hardware, but very much
In the case of the former be more preferably embodiment.Based on this understanding, technical solution of the present invention is substantially in other words to existing
The part that technology contributes can be embodied in the form of software products, which is stored in a storage
In medium (such as ROM/RAM, magnetic disk, CD), including some instructions are used so that a terminal device (can be mobile phone, calculate
Machine, server, air conditioner or network equipment etc.) execute method described in each embodiment of the present invention.
Preferred embodiments of the present invention have been described above with reference to the accompanying drawings, not thereby limiting the scope of the invention.This
Without departing from the scope and spirit of the invention, there are many variations to implement the present invention by field technical staff, for example as one
The feature of a embodiment can be used for another embodiment and obtain another embodiment.It is all to use institute within technical concept of the invention
Any modifications, equivalent replacements, and improvements of work, should all be within interest field of the invention.
Claims (15)
1. a kind of remote memory management method, this method is used for server-side, which is characterized in that the described method includes:
Apply for that one piece of memory is registered to network interface card as message area, and by the message area；
It receives client and the message blocks application that RDMA atomic operation is sent is accessed by remote direct memory, for the client point
With message blocks and specify exclusive time of the message blocks；
The remote request that the client is sent by RDMA Write operation is received, the information of the remote request is stored to described
In message blocks；Handle and respond the remote request of the client.
2. the method according to claim 1, wherein the method also includes:
The message area is divided into workspace and preheating zone；
By the workspace and the preheating zone, message blocks application and remote request that client described in parallel processing is sent.
3. according to the method described in claim 2, it is characterized in that, described by the workspace and the preheating zone, parallel
After the message blocks application and remote request that handle the client transmission further include:
Obtain the switching signal of the workspace and the preheating zone；
According to the switching signal, the workspace is switched to preheating zone and the preheating zone is switched to workspace.
4. the method according to claim 1, wherein the reception client is accessed by remote direct memory
The message blocks application that RDMA atomic operation is sent is the client assignment messages block and the exclusive time for specifying the message blocks
Later further include:
If the quantity of the client is greater than the quantity of the message blocks, shorten the exclusive time of the message blocks；
If the exclusive of the message blocks has timed out the phase, the information that will expire feeds back to the client, so that the client
End empties occupied state information and retransmits message blocks application.
5. the method according to claim 1, wherein one piece of memory of the application is as message area, and will be described
Message area is registered to after network interface card further include:
It is configured to decide the allocation field of the message blocks application result of the client in the stem of the message blocks；
Regularly update the value of the allocation field.
6. a kind of server-side, which is characterized in that the server-side includes: memory, processor and is stored on the memory
And the remote memory management program that can be run on the processor, the remote memory management program are executed by the processor
The step of remote memory management method of the Shi Shixian as described in any one of claims 1 to 5.
7. a kind of computer readable storage medium, which is characterized in that be stored in distal end on the computer readable storage medium
Management program is deposited, is realized as described in any one of claims 1 to 5 when the remote memory management program is executed by processor
The step of remote memory management method.
8. a kind of remote memory management method, this method is used for client, which is characterized in that the described method includes:
Message blocks application is sent to server-side by RDMA atomic operation；Wherein, one piece of memory of the server-side application, which is used as, disappears
Area is ceased, and the message area is registered to network interface card；The message blocks application that the client is sent by RDMA atomic operation is received,
For the client assignment messages block and specify exclusive time of the message blocks；
After message blocks application success, remote request is sent to the server-side by RDMA Write operation, and wait institute
State the response of server-side；Wherein, the server-side receives the remote request that the client is sent by RDMA Write operation, by institute
The information for stating remote request is stored into the message blocks；Handle and respond the remote request of the client.
9. according to the method described in claim 8, it is characterized in that, if the message area is divided into workspace by the server-side
And preheating zone, then message blocks application is sent to the preheating zone by RDMA atomic operation.
10. according to the method described in claim 9, it is characterized in that, if the server-side according to switching signal, by the work
Area is switched to preheating zone and the preheating zone is switched to workspace, then passes through RDMA atomic operation to the institute for being switched to preheating zone
It states workspace and sends message blocks application.
11. according to the method described in claim 8, it is characterized in that, the message blocks application success after, if receiving institute
The information that expires for stating server-side feedback then empties occupied state information and sends message blocks application to the server-side again.
12. according to the method described in claim 8, it is characterized in that, judging the message blocks application success by following steps:
The value (a, b) of allocation field is obtained, if b is less than the total quantity of the message blocks, judges the message blocks application success；
Wherein the allocation field is that the server-side is configured to decide the message blocks of the client in the stem of the message blocks
Application result, a indicate the message area call number, and b indicates the application quantity of the message blocks.
13. a kind of client, which is characterized in that the client includes: memory, processor and is stored on the memory
And the remote memory management program that can be run on the processor, the remote memory management program are executed by the processor
The step of remote memory management method of the Shi Shixian as described in any one of claim 8 to 12.
14. a kind of computer readable storage medium, which is characterized in that be stored in distal end on the computer readable storage medium
Management program is deposited, is realized as described in any one of claim 8 to 12 when the remote memory management program is executed by processor
Remote memory management method the step of.
15. a kind of remote memory management system, which is characterized in that the system comprises server-side as claimed in claim 6 and power
Benefit require 13 described in client.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810515949.3A CN110535811B (en) | 2018-05-25 | 2018-05-25 | Remote memory management method and system, server, client and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810515949.3A CN110535811B (en) | 2018-05-25 | 2018-05-25 | Remote memory management method and system, server, client and storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN110535811A true CN110535811A (en) | 2019-12-03 |
| CN110535811B CN110535811B (en) | 2022-03-04 |
Family
ID=68657010
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201810515949.3A Active CN110535811B (en) | 2018-05-25 | 2018-05-25 | Remote memory management method and system, server, client and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110535811B (en) |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111404931A (en) * | 2020-03-13 | 2020-07-10 | 清华大学 | Remote data transmission method based on persistent memory |
| CN112596669A (en) * | 2020-11-25 | 2021-04-02 | 新华三云计算技术有限公司 | Data processing method and device based on distributed storage |
| CN113448634A (en) * | 2021-05-31 | 2021-09-28 | 山东英信计算机技术有限公司 | ROCE network card resource management method, device, equipment and readable medium |
| CN114153785A (en) * | 2021-11-29 | 2022-03-08 | 北京志凌海纳科技有限公司 | Memory management method and device based on remote direct memory access |
Citations (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103530167A (en) * | 2013-09-30 | 2014-01-22 | 华为技术有限公司 | Virtual machine memory data migration method and relevant device and cluster system |
| CN103929415A (en) * | 2014-03-21 | 2014-07-16 | 华为技术有限公司 | Method and device for reading and writing data under RDMA and network system |
| CN105353992A (en) * | 2015-12-10 | 2016-02-24 | 浪潮(北京)电子信息产业有限公司 | Energy-saving dispatching method for disks |
| CN106603409A (en) * | 2016-11-30 | 2017-04-26 | 中国科学院计算技术研究所 | Data processing system, method and apparatus |
| CN106657365A (en) * | 2016-12-30 | 2017-05-10 | 清华大学 | High concurrent data transmission method based on RDMA (Remote Direct Memory Access) |
| CN106873915A (en) * | 2017-02-22 | 2017-06-20 | 郑州云海信息技术有限公司 | A kind of data transmission method and device based on RDMA registers memory blocks |
| US20170262405A1 (en) * | 2016-03-08 | 2017-09-14 | International Business Machines Corporation | Remote direct memory access-based on static analysis of asynchronous blocks |
| US20170300830A1 (en) * | 2016-04-15 | 2017-10-19 | Nec Laboratories America, Inc. | System and method for communication efficient sparse-reduce |
| CN107479833A (en) * | 2017-08-21 | 2017-12-15 | 中国人民解放军国防科技大学 | Key value storage-oriented remote nonvolatile memory access and management method |
-
2018
- 2018-05-25 CN CN201810515949.3A patent/CN110535811B/en active Active
Patent Citations (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103530167A (en) * | 2013-09-30 | 2014-01-22 | 华为技术有限公司 | Virtual machine memory data migration method and relevant device and cluster system |
| CN103929415A (en) * | 2014-03-21 | 2014-07-16 | 华为技术有限公司 | Method and device for reading and writing data under RDMA and network system |
| CN105353992A (en) * | 2015-12-10 | 2016-02-24 | 浪潮(北京)电子信息产业有限公司 | Energy-saving dispatching method for disks |
| US20170262405A1 (en) * | 2016-03-08 | 2017-09-14 | International Business Machines Corporation | Remote direct memory access-based on static analysis of asynchronous blocks |
| US20170300830A1 (en) * | 2016-04-15 | 2017-10-19 | Nec Laboratories America, Inc. | System and method for communication efficient sparse-reduce |
| CN106603409A (en) * | 2016-11-30 | 2017-04-26 | 中国科学院计算技术研究所 | Data processing system, method and apparatus |
| CN106657365A (en) * | 2016-12-30 | 2017-05-10 | 清华大学 | High concurrent data transmission method based on RDMA (Remote Direct Memory Access) |
| CN106873915A (en) * | 2017-02-22 | 2017-06-20 | 郑州云海信息技术有限公司 | A kind of data transmission method and device based on RDMA registers memory blocks |
| CN107479833A (en) * | 2017-08-21 | 2017-12-15 | 中国人民解放军国防科技大学 | Key value storage-oriented remote nonvolatile memory access and management method |
Non-Patent Citations (2)
| Title |
|---|
| A. DEVULAPALLI: ""Distributed queue-based locking using advanced network features"", 《2005 INTERNATIONAL CONFERENCE ON PARALLEL PROCESSING (ICPP"05)》 * |
| 李亮: ""基于RDMA跨态通信协议的研究与实现"", 《中国优秀硕士学位论文全文数据库-信息科技辑》 * |
Cited By (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111404931A (en) * | 2020-03-13 | 2020-07-10 | 清华大学 | Remote data transmission method based on persistent memory |
| CN111404931B (en) * | 2020-03-13 | 2021-03-30 | 清华大学 | Remote data transmission method based on persistent memory |
| CN112596669A (en) * | 2020-11-25 | 2021-04-02 | 新华三云计算技术有限公司 | Data processing method and device based on distributed storage |
| CN113448634A (en) * | 2021-05-31 | 2021-09-28 | 山东英信计算机技术有限公司 | ROCE network card resource management method, device, equipment and readable medium |
| CN113448634B (en) * | 2021-05-31 | 2022-07-19 | 山东英信计算机技术有限公司 | ROCE network card resource management method, device, equipment and readable medium |
| CN114153785A (en) * | 2021-11-29 | 2022-03-08 | 北京志凌海纳科技有限公司 | Memory management method and device based on remote direct memory access |
Also Published As
| Publication number | Publication date |
|---|---|
| CN110535811B (en) | 2022-03-04 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110535811A (en) | Remote memory management method and system, server-side, client, storage medium | |
| AU711393B2 (en) | A method and apparatus for client managed flow control on a limited memory computer system | |
| CN102369688B (en) | Method for adjusting resources dynamically and scheduling device | |
| US7627627B2 (en) | Controlling command message flow in a network | |
| CN108268208A (en) | A kind of distributed memory file system based on RDMA | |
| JP5635117B2 (en) | Dynamically connected transport service | |
| CN108270732B (en) | A kind of Streaming Media processing method and system | |
| CN106657365A (en) | High concurrent data transmission method based on RDMA (Remote Direct Memory Access) | |
| CN101370035A (en) | Method and system for dynamic client/server network management using proxy servers | |
| CN107948094A (en) | A kind of high speed data frame Lothrus apterus is joined the team the device and method of processing | |
| KR20060126374A (en) | Improved distributed kernel operating system | |
| CN112631788B (en) | Data transmission method and data transmission server | |
| CN105847142B (en) | A kind of message transmitting method and device | |
| CN108234149A (en) | Network request management method and device | |
| WO2018023988A1 (en) | Network packet processing method, device, and network server | |
| CN1863202B (en) | Method for improving load balance apparatus and server processing performance | |
| CN107302505B (en) | Manage the method and device of caching | |
| WO2021012787A1 (en) | Message load balancing method | |
| CN111404986B (en) | Data transmission processing method, device and storage medium | |
| CN106375240B (en) | Ethernet message forwarding method and system between a kind of Multi-netmouth | |
| CN106161194A (en) | A kind of online customer service center based on XMPP instant message exchange system | |
| CN108200221A (en) | Rule synchronization method and device are converted in a kind of network address translation environment | |
| CN112351089B (en) | Data transmission method, system and device between virtual machine and accelerator | |
| CN102255716A (en) | Method and system for protocol state backup between equipment | |
| CN107707546A (en) | A kind of packet copy method and the new network adapter for supporting message copy function |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |