CN104516952B - A kind of memory partitioning dispositions method and device - Google Patents
A kind of memory partitioning dispositions method and device Download PDFInfo
- Publication number
- CN104516952B CN104516952B CN201410767519.2A CN201410767519A CN104516952B CN 104516952 B CN104516952 B CN 104516952B CN 201410767519 A CN201410767519 A CN 201410767519A CN 104516952 B CN104516952 B CN 104516952B
- Authority
- CN
- China
- Prior art keywords
- node
- focus subregion
- focus
- subregion
- memory partitioning
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/24—Querying
- G06F16/245—Query processing
- G06F16/2458—Special types of queries, e.g. statistical queries, fuzzy queries or distributed queries
- G06F16/2471—Distributed queries
Abstract
The embodiment of the invention discloses a kind of memory partitioning dispositions method, for rationally disposing the memory partitioning of distributed system, reduces the time delay expense of distributed system.Present invention method includes：Determine in preset time period, the focus subregion in distributed system, the distributed system includes multiple nodes, and memory partitioning is deployed with the node, and the focus subregion is that the memory partitioning that number is stood out is accessed in distributed system；Determine focus subregion by the number of each node visit；According to focus subregion by the number of each node visit, the optimal deployment node of focus subregion is determined；The optimal deployment node of focus subregion is informed into the node that focus subregion is currently located.The embodiment of the present invention additionally provides the memory partitioning deployment device of correlation.
Description
Technical field
The present invention relates to data processing field, more particularly to a kind of memory partitioning dispositions method and device.
Background technology
Distributed memory data grids (IMDG, In-memory Data Grid) are that internal memory provides on a kind of wide area network-oriented
The shared New Grid system in source.Its main target is in the case of physical memory deficiency, improves memory-intensive application
Or the systematic function of I/O intensive type application.
Distributed memory grid will as far as possible be broken up after deblocking using uniformity hash algorithm and be distributed to distribution
On the memory partitioning of all nodes of formula system, to ensure data balancing, but the internal memory point of distributed system is so allowed for
Area's deployment is more random, causes distributed system to generally require to carry out across a network data access.For example, entering on node A
Journey App1 and App2 is in accessed node B memory partitioning Data1, it is necessary to carries out across a network access.Across a network access when
It is larger to prolong expense, therefore, rationally disposes the position of memory partitioning, to reduce the number of distributed system across a network access, for
Reduce distributed system time delay, lifted distributed system performance for it is significant.
In order to reduce time delay expense, in the prior art using Near-cache technologies, the data of focus memory partitioning are deposited
In the buffer area for storing up the node where process so that process can directly read the data of the focus memory partitioning.Such as will
Node B memory partitioning Data1 is stored into node A buffer area, and such App1 and App2 can be directly from node A caching
Data1 is accessed in area, reduces the number of across a network access.
But Near-cache technologies are not provided with a kind of scheme of rationally deployment memory partitioning, only by focus internal memory
In the buffer area of node where the data Cun Chudao processes of subregion, this method is only capable of reducing the across a network access time of read operation
Number.When carrying out write operation, after each process writes data into local cache area, the data are stilled need from local cache area inter-network
The memory partitioning of other nodes of the write-in of network, the across a network access times of write operation can't be reduced.Therefore, it is more in write operation
Application scenarios under, because distributed system neither one rationally disposes the scheme of memory partitioning, the memory partitioning of system can only
Random placement, so distributed system stills need to carry out substantial amounts of across a network write operation, the time delay expense of system is big, it is impossible to meets
The requirement of practical application.
The content of the invention
The embodiments of the invention provide a kind of memory partitioning dispositions method, for reducing the number of across a network access, reduces
The time delay expense of system.
The first aspect of the embodiment of the present invention provides a kind of memory partitioning dispositions method, including：
Determine in preset time period, the focus subregion in distributed system, the distributed system includes multiple nodes, institute
State and memory partitioning is deployed with node, the focus subregion is that the internal memory point that number is stood out is accessed in distributed system
Area；
Determine the focus subregion by the number of each node visit；
According to the focus subregion by the number of each node visit, the optimal deployment node of the focus subregion is determined；
The optimal deployment node of the focus subregion is informed into the node that the focus subregion is currently located.
With reference to the embodiment of the present invention in a first aspect, in the first implementation of the first aspect of the embodiment of the present invention,
It is described according to the focus subregion by the number of each node visit, determining the optimal deployment node of the focus subregion includes：
The most node of the focus subregion number will be accessed it is defined as the optimal deployment node of the focus subregion.
With reference to the embodiment of the present invention in a first aspect, second of implementation of the first aspect of the embodiment of the present invention is also wrapped
Include：
It is determined that the upper limit for the memory partitioning number that each node can accommodate；
It is described according to the focus subregion by the number of each node visit, determine the optimal deployment section of the focus subregion
Point includes：
The memory partitioning number accommodated is not up in the node of the upper limit, accesses the most section of the focus subregion number
Point is defined as the optimal deployment node of the focus subregion.
The first or second of implementation, the present invention of first aspect, first aspect with reference to the embodiment of the present invention are real
It is described to determine the focus subregion by the number bag of each node visit in the third implementation for applying the first aspect of example
Include：
It is determined that the actual frequency of the access focus subregion of each node；
It is determined that the weights of each node；
The weights and the product of the actual frequency for accessing the focus subregion of each node are calculated, are obtained described
Focus subregion is by the number of each node visit.
The first or second of implementation, the present invention of first aspect, first aspect with reference to the embodiment of the present invention are real
In the 4th kind of implementation for applying the first aspect of example, in the determination preset time period, the focus subregion in distributed system
Including：
Determine in preset time period, the accessed number of M memory partitioning in distributed system, the M is positive integer；
By in the M memory partitioning, number highest top n memory partitioning is accessed, is defined as focus subregion, it is described
N is the positive integer no more than M.
The second aspect of the embodiment of the present invention provides a kind of memory partitioning deployment device, including：
Focus subregion determining module, for determining in preset time period, the focus subregion in distributed system, the distribution
Formula system includes multiple nodes, and memory partitioning is deployed with the node, and the focus subregion is to be accessed in distributed system
The memory partitioning that number is stood out；
Access times determining module, for determining the focus subregion by the number of each node visit；
Optimal node determining module, for by the number of each node visit, determining the heat according to the focus subregion
The optimal deployment node of point subregion；
Optimal node informs module, for the optimal deployment node of the focus subregion to be informed into the focus subregion is current
The node at place.
With reference to the second aspect of the embodiment of the present invention, in the first implementation of the second aspect of the embodiment of the present invention,
The optimal node determining module is specifically used for：
The most node of the focus subregion number will be accessed it is defined as the optimal deployment node of the focus subregion.
With reference to the second aspect of the embodiment of the present invention, in second of implementation of the second aspect of the embodiment of the present invention,
Described device also includes：
Subregion upper limit determining module, the upper limit for the memory partitioning number that can be accommodated for determining each node；
The optimal node determining module is specifically used for：The memory partitioning number accommodated is not up to the node of the upper limit
In, access the optimal deployment node that the most node of the focus subregion number is defined as the focus subregion.
The first or second of implementation, the present invention of second aspect, second aspect with reference to the embodiment of the present invention are real
In the third implementation for applying the second aspect of example, the access times determining module includes：
Number determining unit, the actual frequency of the access focus subregion for determining each node；
Weights determining unit, for determining the weights of each node；
Number computing unit, for the weights for calculating each node and access the focus subregion actual time
Several products, the focus subregion is obtained by the number of each node visit.
The first or second of implementation, the present invention of second aspect, second aspect with reference to the embodiment of the present invention are real
In the 4th kind of implementation for applying the second aspect of example, the focus subregion determining module includes：
Number determining unit, for determining in preset time period, accessed time of M memory partitioning in distributed system
Number, the M is positive integer；
Focus determining unit, for by the M memory partitioning, being accessed number highest top n memory partitioning, really
It is set to focus subregion, the N is the positive integer no more than M.
In method provided in an embodiment of the present invention, memory partitioning deployment device is determined in preset time period in distributed system
Focus subregion；Determine focus subregion by the number of each node visit；According to focus subregion by the number of each node visit
Determine the optimal deployment node of focus subregion；And the optimal deployment node of focus subregion is informed into the section that focus subregion is currently located
Point so that the node that focus subregion is currently located can be by focus zoned migration to optimal deployment node.The embodiment of the present invention
Optimal deployment node is determined by the number of each node visit according to focus subregion so that focus subregion can be deployed to access
On the more node of the focus subregion, so the node directly can carry out local IP access to the focus subregion.No matter access is somebody's turn to do
The operation of focus subregion is read operation or write operation, and method provided in an embodiment of the present invention can reduce across a network access
Number, the time delay expense of system is reduced, disclosure satisfy that application request.
Brief description of the drawings
Fig. 1 is memory partitioning dispositions method one embodiment flow chart in the embodiment of the present invention；
Fig. 2 is another embodiment flow chart of memory partitioning dispositions method in the embodiment of the present invention；
Fig. 3 is that memory partitioning disposes device one embodiment structure chart in the embodiment of the present invention；
Fig. 4 is another example structure figure of memory partitioning deployment device in the embodiment of the present invention；
Fig. 5 is another example structure figure of memory partitioning deployment device in the embodiment of the present invention；
Fig. 6 is another example structure figure of memory partitioning deployment device in the embodiment of the present invention；
Fig. 7 is another example structure figure of memory partitioning deployment device in the embodiment of the present invention.
Embodiment
The embodiments of the invention provide a kind of memory partitioning dispositions method, for rationally disposing the internal memory point of distributed system
Area, reduce the time delay expense of distributed system.The embodiment of the present invention additionally provides the memory partitioning deployment device of correlation, below will
It is described respectively.
The basic procedure of memory partitioning dispositions method provided in an embodiment of the present invention is referring to Fig. 1, mainly include：
101st, determine in preset time period, the focus subregion in distributed system；
Distributed system includes multiple nodes, and memory partitioning is deployed with node.Wherein, focus subregion is distributed system
In be accessed the memory partitioning stood out of number；
Wherein, preset time period can be the period being manually set, or the period of distributed system acquiescence,
Do not limit herein.Wherein, the method that memory partitioning deployment device determines the focus subregion of distributed system in preset time period
Have a lot, will specifically be described in detail in the embodiment below, do not limit herein.
102nd, determine focus subregion by the number of each node visit；
Memory partitioning deployment device determines that focus subregion is had a lot, such as by the number of each node visit, its determination method
Focus subregion is obtained at the node where focus subregion by the number of each node visit, can also be determined by other means
Focus subregion will be specifically described in detail in the embodiment below, not limited herein by the number of each node visit.
103rd, according to focus subregion by the number of each node visit, the optimal deployment node of focus subregion is determined；
Memory partitioning disposes device according to focus subregion by the number of node visit, determines the optimal deployment section of focus subregion
Point.Its specific determination method will be described in detail in the embodiment below, not limit herein.
104th, the optimal deployment node of focus subregion is informed into the node that focus subregion is currently located.
After memory partitioning deployment device determines the optimal deployment node of focus subregion, by the optimal deployment section of focus subregion
Point informs the node that focus subregion is currently located so that the node where focus subregion can be by focus zoned migration to optimal portion
Affix one's name on node.
In the method that the present embodiment provides, memory partitioning deployment device determines the heat in preset time period in distributed system
Point subregion；Determine focus subregion by the number of each node visit；Determined according to focus subregion by the number of each node visit
The optimal deployment node of focus subregion；And the optimal deployment node of focus subregion is informed into the node that focus subregion is currently located,
The node that focus subregion is currently located enable by focus zoned migration to optimal deployment node.The present embodiment is according to focus
Subregion is determined optimal deployment node by the number of each node visit so that focus subregion, which can be deployed to, accesses the focus point
On the more node in area, so the node directly can carry out local IP access to the focus subregion.No matter the focus subregion is accessed
Operation be read operation or write operation, the method that the present embodiment provides can reduce the number of across a network access, reduce system
The time delay expense of system, disclosure satisfy that application request.
Embodiment shown in Fig. 1 provides a kind of memory partitioning dispositions method, according to focus subregion by each node visit
Number determine optimal deployment node so that focus subregion, which can be deployed to, to be accessed on the more node of the focus subregion.It is excellent
Choosing, as another embodiment of the present invention, in step 103, memory partitioning deployment device can will access focus subregion
The most node of number is defined as the optimal deployment node of focus subregion, farthest to reduce the access of distributed system across a network
Number.
The number for the memory partitioning that one node can accommodate has the upper limit, is more highly preferred to, in order to avoid focus point
Multiple focus subregions all move to same node when area migrates, and beyond the carrying capacity of the node, the invention provides one kind
The memory partitioning dispositions method more refined, referring to Fig. 2, specifically including：
201st, determine in preset time period, the focus subregion in distributed system；
Distributed system includes multiple nodes, and memory partitioning is deployed with node.Wherein, focus subregion is distributed system
In be accessed the memory partitioning stood out of number.Wherein, preset time period can be the period being manually set, or
The period of distributed system acquiescence, do not limit herein.
Optionally, memory partitioning deployment device determines that the method for the focus subregion of distributed system in preset time period can be with
For：Determine in preset time period, the accessed number of M memory partitioning in distributed system, wherein M is positive integer；Then will
In the M memory partitioning, number highest top n memory partitioning is accessed, is defined as focus subregion, wherein N is no more than M's
Positive integer.
202nd, determine focus subregion by the number of each node visit；
Memory partitioning deployment device determines that focus subregion by the number of each node visit, is specifically as follows focus subregion quilt
The actual frequency of each node visit, or focus subregion is by the number of each node visit after addition of weights.It is optional
, focus subregion can be determined as follows by the number of each node visit after addition of weights：
Step A, the actual frequency of the access focus subregion of each node is determined；
Memory partitioning deployment device can obtain each node visit focus subregion at the node where focus subregion
Actual frequency, the actual frequency of each node visit focus subregion can also be obtained by other approach, is not limited herein.
Step B, the weights of each node are determined；
In general, the weights of most of node are 1 in distributed system, when these node visit focus subregions, then hot
The accessed number of point subregion can add 1.But in some special scenes, the weights of some nodes can not be 1, if such as
The time delay of some node visit focus subregion is larger, then the weights of the node can be set greater than to 1 numerical value, so can
Increase the probability that the node is confirmed to be the optimal deployment node of focus subregion.The weights of each node can be that user is set,
Can also be that memory partitioning disposes the spontaneous setting of device, memory partitioning deployment device can also determine each to save by other means
The weights of point, are not limited herein.
Step C, the product of actual frequency of the weights of each node with accessing focus subregion is calculated, obtains focus subregion quilt
The number of each node visit.
For each node, memory partitioning disposes device by the reality of the weights of the node and the node visit focus subregion
Number is multiplied, and obtained product is exactly the number of the node visit focus subregion.
203rd, the upper limit for the memory partitioning number that each node can accommodate is determined；
In distributed system, the number for the memory partitioning that a node can accommodate has the upper limit.It is interior in the present embodiment
The upper limit that subregion deployment device determines the memory partitioning number that each node can accommodate is deposited, the upper limit can be used to indicate that node
Accommodate the maximum capacity of memory partitioning, or the upper limit artificially set, do not limit herein.
204th, the memory partitioning number accommodated is not up in the node of the upper limit, accesses the most section of focus subregion number
Point is defined as the optimal deployment node of focus subregion.
During in order to avoid focus zoned migration, multiple focus subregions all move to same node, beyond the appearance of the node
Receive ability, therefore the present embodiment is only not up in the node of the upper limit in the memory partitioning number of receiving and determines optimal deployment node.
If the number for the memory partitioning that a node is accommodated has reached the upper limit, the node will not be confirmed to be focus subregion again most
Excellent deployment node, so follow-up focus subregion can be just migrated on the node again.
205th, the optimal deployment node of focus subregion is informed into the node that focus subregion is currently located.
After memory partitioning deployment device determines the optimal deployment node of focus subregion, by the optimal deployment section of focus subregion
Point informs the node that focus subregion is currently located so that the node where focus subregion can be by focus zoned migration to optimal portion
Affix one's name on node.
In the method that the present embodiment provides, memory partitioning deployment device determines the heat in preset time period in distributed system
Point subregion；Determine focus subregion by the number of each node visit；Determined according to focus subregion by the number of each node visit
The optimal deployment node of focus subregion；And the optimal deployment node of focus subregion is informed into the node that focus subregion is currently located,
The node that focus subregion is currently located enable by focus zoned migration to optimal deployment node.The present embodiment is according to focus
Subregion is determined optimal deployment node by the number of each node visit so that focus subregion, which can be deployed to, accesses the focus point
On the more node in area, so the node directly can carry out local IP access to the focus subregion.No matter the focus subregion is accessed
Operation be read operation or write operation, the method that the present embodiment provides can reduce the number of across a network access, reduce system
The time delay expense of system, disclosure satisfy that application request.The present embodiment is only not up in the memory partitioning number of receiving simultaneously
Optimal deployment node is determined in the node of limit.If the number for the memory partitioning that a node is accommodated has reached the upper limit, the section
Point will not be confirmed to be the optimal deployment node of focus subregion again, and so follow-up focus subregion can just not be migrated to the section again
On point, without departing from the carrying capacity of the node.
Optionally, the node of the upper limit is reached for the memory partitioning number accommodated, memory partitioning deployment device can incite somebody to action
Non-hot memory partitioning migrates out in these nodes so that the memory partitioning number that the node is accommodated is less than the upper limit, can
To continue to accommodate new focus subregion.If for example, it is determined that the memory partitioning Data1 on node A is focus in step 201
Subregion, it is determined that Data1 have accessed 100 times by node B in step 202, node B accesses Data1 number highest, step 203
In be determined that node B can accommodate 50 memory partitionings.If the memory partitioning accommodated in node B has reached 50, internal memory
Subregion deployment device can be by Data1 information advised nodes B so that node B will be not belonging to focus subregion in memory partitioning
Data2 is moved on node A, and then node A again moves to Data1 on node B.
Optionally, memory partitioning deployment device was determined in preset time period in step 201, M internal memory in distributed system
The accessed number of subregion, the reality for being specifically as follows M memory partitioning are accessed number, or addition of M after weights
The accessed number of individual memory partitioning.It addition of the determination method of the accessed number of M memory partitioning and step 202 after weights
In it is similar, do not repeat herein.
In distributed system, one or more processes can be disposed on each node.Particularly, the present invention can also basis
The number that focus subregion is accessed by each process is come with determining the move target of focus subregion so that focus subregion can be deployed to
Access on the memory partitioning position corresponding to the more process of the focus subregion.Such as in the embodiment shown in Fig. 1, step 102
The number that focus subregion is accessed by each process can also be determined；Accessed in step 103 according to focus subregion by each process
Number, the optimal deployment district location of focus subregion is determined, it can be to access the focus subregion that the foot, which has deployment district location,
The position of memory partitioning corresponding to the most process of number；The optimal deployment district location of focus subregion is informed into heat in step 104
The node that point subregion is currently located so that the node is by focus zoned migration to the optimal deployment district location.
For the ease of understanding above-described embodiment, will be retouched below by taking above-described embodiment concrete application scene as an example
State.
Memory partitioning Data1 is deployed with node A, memory partitioning Data2 is deployed with node B, is deployed with node C interior
Deposit subregion Data3.Memory partitioning deployment device determines that within 4 hours preset time periods Data1 accessed number is 2500
It is secondary；Data2 accessed number is 3100 times；Data3 accessed number 2100 times.Memory partitioning deployment device will be accessed
Number highest the first two memory partitioning, i.e. Data1 and Data2 confirm as focus subregion.
Memory partitioning deployment device determines that Data1 is accessed 1000 times by the process App1 on node A, by positioned at node
Process App2 on B is accessed 500 times；Data2 is accessed 800 times by the process App2 on node B, by node C
Process App3 is accessed 700 times；Data3 is accessed 300 times by the process App1 on node A, by the process on node B
App2 is accessed 600 times.
Memory partitioning deployment device determines that node A weights are 1, and node B weights are 3, and node C weights are 1.
Memory partitioning deployment device is calculated Data1 and accessed 1000 × 1=1000 times by App1, and 500 are accessed by App2
× 3=1500 times, therefore the node B where App2 is confirmed as to Data1 optimal deployment node；Memory partitioning disposes device meter
Calculation obtains Data2 and accessed 800 × 3=2400 times by App2, is accessed 700 × 1=700 times by App3, therefore by where App2
Node B confirms as Data2 optimal deployment node.
Memory partitioning deployment device advised nodes A：Node B is Data1 optimal deployment node so that node A is by Data1
Move on node B；And memory partitioning deployment device advised nodes B：Node B is Data2 optimal deployment node, therefore node
B need not migrate Data2.
The embodiment of the present invention additionally provides the memory partitioning deployment device of correlation, and for realizing the above method, it is tied substantially
Structure referring to Fig. 3, including：
Focus subregion determining module 301, for determining in preset time period, the focus subregion in distributed system, this point
Cloth system includes multiple nodes, and memory partitioning is deployed with node, and the focus subregion is that number is accessed in distributed system
The memory partitioning stood out；
Access times determining module 302, for determining focus subregion by the number of each node visit；
Optimal node determining module 303, for by the number of each node visit, determining focus subregion according to focus subregion
Optimal deployment node；
Optimal node informs module 304, for the optimal deployment node of focus subregion to be informed into focus subregion is currently located
Node.
In the memory partitioning deployment device that the present embodiment provides, focus subregion determining module 301 was determined in preset time period
Focus subregion in distributed system；Access times determining module 302 determines focus subregion by the number of each node visit；Most
Excellent node determining module 303 is determined the optimal deployment node of focus subregion according to focus subregion by the number of each node visit；
Optimal node informs that module 304 informs the optimal deployment node of focus subregion the node that focus subregion is currently located so that heat
The node that point subregion is currently located can be by focus zoned migration to optimal deployment node.The present embodiment is according to focus subregion quilt
The number of each node visit determines optimal deployment node so that focus subregion can be deployed to that to access the focus subregion more
Node on, so the node can directly to the focus subregion carry out local IP access.No matter the operation of the focus subregion is accessed
It is read operation or write operation, the memory partitioning deployment device that the present embodiment provides can reduce the number of across a network access,
The time delay expense of reduction system, disclosure satisfy that application request.
Wherein, another embodiment as the present invention, optimal node determining module are specifically used for：
The most node of access focus subregion number is defined as to the optimal deployment node of focus subregion, with maximum
Reduce the number that distributed system across a network accesses.
Embodiment shown in Fig. 3 gives the basic structure of memory partitioning deployment device provided by the invention, reality below
Apply example and provide a kind of memory partitioning deployment device more refined, referring to Fig. 4, including：
Focus subregion determining module 401, for determining in preset time period, the focus subregion in distributed system, this point
Cloth system includes multiple nodes, and memory partitioning is deployed with node, and the focus subregion is that number is accessed in distributed system
The memory partitioning stood out；
Access times determining module 402, for determining focus subregion by the number of each node visit；
Subregion upper limit determining module 403, the upper limit for the memory partitioning number that can be accommodated for determining each node；
Optimal node determining module 404, for the memory partitioning accommodated number to be not up in the node of the upper limit, access
The most node of focus subregion number is defined as the optimal deployment node of focus subregion；
Optimal node informs module 405, for the optimal deployment node of focus subregion to be informed into focus subregion is currently located
Node.
In the method that the present embodiment provides, focus subregion determining module 401 is determined in preset time period in distributed system
Focus subregion；Access times determining module 402 determines focus subregion by the number of each node visit；Optimal node determines mould
Block 404 is determined the optimal deployment node of focus subregion according to focus subregion by the number of each node visit；Optimal node is informed
Module 405 informs the optimal deployment node of focus subregion the node that focus subregion is currently located so that the current institute of focus subregion
Node can by focus zoned migration to it is optimal deployment node on.The present embodiment is according to focus subregion by each node visit
Number determine optimal deployment node so that focus subregion, which can be deployed to, to be accessed on the more node of the focus subregion, this
The sample node directly can carry out local IP access to the focus subregion.No matter access the operation of the focus subregion be read operation or
Write operation, the present embodiment provide memory partitioning deployment device can reduce across a network access number, reduce system when
Prolong expense, disclosure satisfy that application request.Optimal node determining module 404 is only divided in the internal memory of receiving in the present embodiment simultaneously
Area's number is not up in the node of the upper limit of the determination of subregion upper limit determining module 403 and determines optimal deployment node.An if node
The number of the memory partitioning accommodated has reached the upper limit, then the node will not be confirmed to be the optimal deployment section of focus subregion again
Point, so follow-up focus subregion can be just migrated on the node again, without departing from the carrying capacity of the node.
Embodiment shown in Fig. 3 gives the basic structure of memory partitioning deployment device provided by the invention, reality below
Applying example will refine to access times determining module therein, referring to Fig. 5, including：
Focus subregion determining module 501, for determining in preset time period, the focus subregion in distributed system, this point
Cloth system includes multiple nodes, and memory partitioning is deployed with node, and the focus subregion is that number is accessed in distributed system
The memory partitioning stood out；
Access times determining module 502, for determining focus subregion by the number of each node visit.In the present embodiment,
Access times determining module 502 includes：
Number determining unit 5021, the actual frequency of the access focus subregion for determining each node；
Weights determining unit 5022, for determining the weights of each node；
Number computing unit 5023, for calculating multiplying for actual frequency of the weights of each node with accessing focus subregion
Product, obtains focus subregion by the number of each node visit.
Optimal node determining module 503, for the memory partitioning accommodated number to be not up in the node of the upper limit, access
The most node of focus subregion number is defined as the optimal deployment node of focus subregion；
Optimal node informs module 504, for the optimal deployment node of focus subregion to be informed into focus subregion is currently located
Node.
In the method that the present embodiment provides, focus subregion determining module 501 is determined in preset time period in distributed system
Focus subregion；Number determining unit 5021 determines the actual frequency of the access focus subregion of each node；Weights determining unit
5022 determine the weights of each node；Number computing unit 5023 calculates reality of the weights of each node with accessing focus subregion
The product of number, focus subregion is obtained by the number access times of each node visit；Optimal node determining module 503 is according to heat
Point subregion is determined the optimal deployment node of focus subregion by the number of each node visit；Optimal node informs module 504 by heat
The optimal deployment node of point subregion informs the node that focus subregion is currently located so that the node that focus subregion is currently located can
By on focus zoned migration to optimal deployment node.The present embodiment is according to focus subregion after addition of weights by each node visit
Number determine optimal deployment node so that focus subregion, which can be deployed to, to be accessed on the more node of the focus subregion, this
The sample node directly can carry out local IP access to the focus subregion.No matter access the operation of the focus subregion be read operation or
Write operation, the present embodiment provide memory partitioning deployment device can reduce across a network access number, reduce system when
Prolong expense, disclosure satisfy that application request.Optimal node determining module 504 is only divided in the internal memory of receiving in the present embodiment simultaneously
Area's number is not up in the node of the upper limit of the determination of subregion upper limit determining module 503 and determines optimal deployment node.An if node
The number of the memory partitioning accommodated has reached the upper limit, then the node will not be confirmed to be the optimal deployment section of focus subregion again
Point, so follow-up focus subregion can be just migrated on the node again, without departing from the carrying capacity of the node.
Embodiment shown in Fig. 4 gives the basic structure of memory partitioning deployment device provided by the invention, reality below
Applying example will refine to focus subregion determining module therein, referring to Fig. 6, including：
Focus subregion determining module 601, for determining in preset time period, the focus subregion in distributed system, this point
Cloth system includes multiple nodes, and memory partitioning is deployed with node, and the focus subregion is that number is accessed in distributed system
The memory partitioning stood out.In the present embodiment, focus subregion determining module 601 includes：
Number determining unit 6011, for determining in preset time period, M memory partitioning is accessed in distributed system
Number, wherein M are positive integer；
Focus determining unit 6012, for by the M memory partitioning, being accessed number highest top n memory partitioning
It is defined as focus subregion, wherein N is the positive integer no more than M.
Access times determining module 602, for determining focus subregion by the number of each node visit；
Subregion upper limit determining module 603, the upper limit for the memory partitioning number that can be accommodated for determining each node；
Optimal node determining module 604, for the memory partitioning accommodated number to be not up in the node of the upper limit, access
The most node of focus subregion number is defined as the optimal deployment node of focus subregion；
Optimal node informs module 605, for the optimal deployment node of focus subregion to be informed into focus subregion is currently located
Node.
In the method that the present embodiment provides, number determining unit 6011 was determined in preset time period, M in distributed system
The accessed number of memory partitioning；Focus determining unit 6012 is accessed number highest top n by the M memory partitioning
Memory partitioning is defined as focus subregion；Access times determining module 602 determines focus subregion by the number of each node visit；Most
Excellent node determining module 604 is determined the optimal deployment node of focus subregion according to focus subregion by the number of each node visit；
Optimal node informs that module 605 informs the optimal deployment node of focus subregion the node that focus subregion is currently located so that heat
The node that point subregion is currently located can be by focus zoned migration to optimal deployment node.The present embodiment is according to focus subregion quilt
The number of each node visit determines optimal deployment node so that focus subregion can be deployed to that to access the focus subregion more
Node on, so the node can directly to the focus subregion carry out local IP access.No matter the operation of the focus subregion is accessed
It is read operation or write operation, the memory partitioning deployment device that the present embodiment provides can reduce the number of across a network access,
The time delay expense of reduction system, disclosure satisfy that application request.Optimal node determining module 604 only exists in the present embodiment simultaneously
The memory partitioning number of receiving, which is not up in the node of the upper limit of the determination of subregion upper limit determining module 603, determines optimal deployment section
Point.If the number for the memory partitioning that a node is accommodated has reached the upper limit, the node will not be confirmed to be focus subregion again
Optimal deployment node, so follow-up focus subregion can be just migrated on the node again, without departing from the appearance of the node
Receive ability.
For the ease of understanding above-described embodiment, will be retouched below by taking above-described embodiment concrete application scene as an example
State.
Memory partitioning Data1 is deployed with node A, memory partitioning Data2 is deployed with node B, is deployed with node C interior
Deposit subregion Data3.Focus subregion determining module 501 determines that within 4 hours preset time periods Data1 accessed number is
2500 times；Data2 accessed number is 3100 times；Data3 accessed number 2100 times.Focus subregion determining module 501
By accessed number highest the first two memory partitioning, i.e. Data1 and Data2 confirms as focus subregion.
Number determining unit 5021 determines that Data1 is accessed 1000 times by the process App1 on node A, by positioned at node
Process App2 on B is accessed 500 times；Data2 is accessed 800 times by the process App2 on node B, by node C
Process App3 is accessed 700 times；Data3 is accessed 300 times by the process App1 on node A, by the process on node B
App2 is accessed 600 times.
Weights determining unit 5022 determines that node A weights are 1, and node B weights are 3, and node C weights are 1.
Number computing unit 5023 is calculated Data1 and accessed 1000 × 1=1000 times by App1, and 500 are accessed by App2
× 3=1500 times, therefore optimal node determining module 503 confirms as the node B where App2 at Data1 optimal deployment section
Point；Number computing unit 5023 is calculated Data2 and accessed 800 × 3=2400 times by App2, and 700 × 1=is accessed by App3
700 times, therefore optimal node determining module 503 confirms as the node B where App2 Data2 optimal deployment node.
Optimal node informs the advised nodes A of module 504：Node B is Data1 optimal deployment node so that node A will
Data1 is moved on node B；And optimal node informs the advised nodes B of module 504：Node B is Data2 optimal deployment node,
Therefore node B need not migrate Data2.
The memory partitioning deployment device in the embodiment of the present invention is retouched from the angle of blocking functional entity above
State, the memory partitioning deployment device in the embodiment of the present invention is described from the angle of hardware handles below, referring to Fig. 7,
Memory partitioning deployment 700 another embodiment of device in the embodiment of the present invention includes：
Input unit 701, output device 702, processor 703 and (the wherein memory partitioning deployment device 700 of memory 704
In the quantity of processor 703 can be one or more, in Fig. 4 by taking a processor 703 as an example).In some realities of the present invention
To apply in example, input unit 701, output device 702, processor 703 and memory 704 can be connected by bus or other means,
Wherein, in Fig. 4 exemplified by being connected by bus.
Wherein, by the operational order for calling memory 704 to store, processor 703 is used to perform following steps：
Determine in preset time period, the focus subregion in distributed system, the distributed system includes multiple nodes, the section
Memory partitioning is deployed with point, the focus subregion is that the memory partitioning that number is stood out is accessed in distributed system；
Determine focus subregion by the number of each node visit；
According to focus subregion by the number of each node visit, the optimal deployment node of focus subregion is determined；
The optimal deployment node of focus subregion is informed into the node that focus subregion is currently located.
In some embodiments of the present invention, processor 703 also executes the following steps：
The most node of access focus subregion number is defined as to the optimal deployment node of focus subregion.
In some embodiments of the present invention, processor 703 also executes the following steps：
It is determined that the upper limit for the memory partitioning number that each node can accommodate；
The memory partitioning number accommodated is not up in the node of the upper limit, it is true to access the most node of focus subregion number
It is set to the optimal deployment node of focus subregion.
In some embodiments of the present invention, processor 703 also performs following operation：
It is determined that the actual frequency of the access focus subregion of each node；
It is determined that the weights of each node；
The product of actual frequency of the weights of each node with accessing focus subregion is calculated, obtains focus subregion by each section
The number that point accesses.
In some embodiments of the present invention, processor 703 also performs following operation：
Determine in preset time period, the accessed number of M memory partitioning, M are positive integer in distributed system；
By in the M memory partitioning, number highest top n memory partitioning is accessed, is defined as focus subregion, wherein N
For the positive integer no more than M.
It is apparent to those skilled in the art that for convenience and simplicity of description, the system of foregoing description,
The specific work process of device and unit, the corresponding process in preceding method embodiment is may be referred to, will not be repeated here.
In several embodiments provided herein, it should be understood that disclosed system, apparatus and method can be with
Realize by another way.For example, device embodiment described above is only schematical, for example, the unit
Division, only a kind of division of logic function, can there is other dividing mode, such as multiple units or component when actually realizing
Another system can be combined or be desirably integrated into, or some features can be ignored, or do not perform.It is another, it is shown or
The mutual coupling discussed or direct-coupling or communication connection can be the indirect couplings by some interfaces, device or unit
Close or communicate to connect, can be electrical, mechanical or other forms.
The unit illustrated as separating component can be or may not be physically separate, show as unit
The part shown can be or may not be physical location, you can with positioned at a place, or can also be distributed to multiple
On NE.Some or all of unit therein can be selected to realize the mesh of this embodiment scheme according to the actual needs
's.
In addition, each functional unit in each embodiment of the present invention can be integrated in a processing unit, can also
That unit is individually physically present, can also two or more units it is integrated in a unit.Above-mentioned integrated list
Member can both be realized in the form of hardware, can also be realized in the form of SFU software functional unit.
If the integrated unit is realized in the form of SFU software functional unit and is used as independent production marketing or use
When, it can be stored in a computer read/write memory medium.Based on such understanding, technical scheme is substantially
The part to be contributed in other words to prior art or all or part of the technical scheme can be in the form of software products
Embody, the computer software product is stored in a storage medium, including some instructions are causing a computer
Equipment (can be personal computer, server, or network equipment etc.) performs the complete of each embodiment methods described of the present invention
Portion or part steps.And foregoing storage medium includes：USB flash disk, mobile hard disk, read-only storage (ROM, Read-Only
Memory), random access memory (RAM, Random Access Memory), magnetic disc or CD etc. are various can store journey
The medium of sequence code.
Described above, the above embodiments are merely illustrative of the technical solutions of the present invention, rather than its limitations；Although with reference to before
Embodiment is stated the present invention is described in detail, it will be understood by those within the art that：It still can be to preceding
State the technical scheme described in each embodiment to modify, or equivalent substitution is carried out to which part technical characteristic；And these
Modification is replaced, and the essence of appropriate technical solution is departed from the spirit and scope of various embodiments of the present invention technical scheme.
Claims (8)
- A kind of 1. memory partitioning dispositions method, it is characterised in that including：Determine in preset time period, the focus subregion in distributed system, the distributed system includes multiple nodes, the section Memory partitioning is deployed with point, the focus subregion is that the memory partitioning that number is stood out is accessed in distributed system；Determine the focus subregion by the number of each node visit；According to the focus subregion by the number of each node visit, the optimal deployment node of the focus subregion is determined；The optimal deployment node of the focus subregion is informed into the node that the focus subregion is currently located；It is described according to the focus subregion by the number of each node visit, determine the optimal deployment node bag of the focus subregion Include：The most node of the focus subregion number will be accessed it is defined as the optimal deployment node of the focus subregion.
- 2. memory partitioning dispositions method according to claim 1, it is characterised in that methods described also includes：It is determined that the upper limit for the memory partitioning number that each node can accommodate；It is described according to the focus subregion by the number of each node visit, determine the optimal deployment node bag of the focus subregion Include：The memory partitioning number accommodated is not up in the node of the upper limit, it is true to access the most node of the focus subregion number It is set to the optimal deployment node of the focus subregion.
- 3. memory partitioning dispositions method according to any one of claim 1 to 2, it is characterised in that described in the determination Focus subregion is included by the number of each node visit：It is determined that the actual frequency of the access focus subregion of each node；It is determined that the weights of each node；The weights and the product of the actual frequency for accessing the focus subregion of each node are calculated, obtain the focus Subregion is by the number of each node visit.
- 4. memory partitioning dispositions method according to any one of claim 1 to 2, it is characterised in that the determination is preset In period, the focus subregion in distributed system includes：Determine in preset time period, the accessed number of M memory partitioning in distributed system, the M is positive integer；By in the M memory partitioning, number highest top n memory partitioning is accessed, is defined as focus subregion, the N is Positive integer no more than M.
- 5. a kind of memory partitioning disposes device, it is characterised in that including：Focus subregion determining module, for determining in preset time period, the focus subregion in distributed system, the distributed system System includes multiple nodes, and memory partitioning is deployed with the node, and the focus subregion is that number is accessed in distributed system The memory partitioning stood out；Access times determining module, for determining the focus subregion by the number of each node visit；Optimal node determining module, for by the number of each node visit, determining the focus point according to the focus subregion The optimal deployment node in area；Optimal node informs module, for the optimal deployment node of the focus subregion to be informed into the focus subregion is currently located Node；The optimal node determining module is specifically used for：The most node of the focus subregion number will be accessed it is defined as the optimal deployment node of the focus subregion.
- 6. memory partitioning according to claim 5 disposes device, it is characterised in that described device also includes：Subregion upper limit determining module, the upper limit for the memory partitioning number that can be accommodated for determining each node；The optimal node determining module is specifically used for：The memory partitioning number accommodated is not up in the node of the upper limit, visited Ask that the most node of the focus subregion number is defined as the optimal deployment node of the focus subregion.
- 7. the memory partitioning deployment device according to any one of claim 5 to 6, it is characterised in that the access times Determining module includes：Number determining unit, the actual frequency of the access focus subregion for determining each node；Weights determining unit, for determining the weights of each node；Number computing unit, for the weights for calculating each node and the actual frequency for accessing the focus subregion Product, the focus subregion is obtained by the number of each node visit.
- 8. the memory partitioning deployment device according to any one of claim 5 to 6, it is characterised in that the focus subregion Determining module includes：Number determining unit, for determining in preset time period, the accessed number of M memory partitioning, institute in distributed system It is positive integer to state M；Focus determining unit, for by the M memory partitioning, being accessed number highest top n memory partitioning, being defined as Focus subregion, the N are the positive integer no more than M.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201410767519.2A CN104516952B (en) | 2014-12-12 | 2014-12-12 | A kind of memory partitioning dispositions method and device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201410767519.2A CN104516952B (en) | 2014-12-12 | 2014-12-12 | A kind of memory partitioning dispositions method and device |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN104516952A CN104516952A (en) | 2015-04-15 |
| CN104516952B true CN104516952B (en) | 2018-02-13 |
Family
ID=52792251
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201410767519.2A Active CN104516952B (en) | 2014-12-12 | 2014-12-12 | A kind of memory partitioning dispositions method and device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN104516952B (en) |
Families Citing this family (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105743975B (en) * | 2016-01-28 | 2019-03-05 | 深圳先进技术研究院 | Caching laying method and system based on data access distribution |
| CN112076464B (en) * | 2020-09-04 | 2022-06-21 | 腾讯科技（深圳）有限公司 | Data request processing method and device, computer equipment and storage medium |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101582013A (en) * | 2009-06-10 | 2009-11-18 | 成都市华为赛门铁克科技有限公司 | Method, device and system for processing storage hotspots in distributed storage |
| WO2012167378A1 (en) * | 2011-06-06 | 2012-12-13 | International Business Machines Corporation | System and method of optimization of in-memory data grid placement |
| CN102244685B (en) * | 2011-08-11 | 2013-09-18 | 中国科学院软件研究所 | Distributed type dynamic cache expanding method and system for supporting load balancing |
| CN103336670A (en) * | 2013-06-04 | 2013-10-02 | 华为技术有限公司 | Method and device for automatic distribution of data blocks based on data temperature |
| CN103365717A (en) * | 2013-06-25 | 2013-10-23 | 华为技术有限公司 | Memory access method, device and system |
-
2014
- 2014-12-12 CN CN201410767519.2A patent/CN104516952B/en active Active
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101582013A (en) * | 2009-06-10 | 2009-11-18 | 成都市华为赛门铁克科技有限公司 | Method, device and system for processing storage hotspots in distributed storage |
| WO2012167378A1 (en) * | 2011-06-06 | 2012-12-13 | International Business Machines Corporation | System and method of optimization of in-memory data grid placement |
| CN102244685B (en) * | 2011-08-11 | 2013-09-18 | 中国科学院软件研究所 | Distributed type dynamic cache expanding method and system for supporting load balancing |
| CN103336670A (en) * | 2013-06-04 | 2013-10-02 | 华为技术有限公司 | Method and device for automatic distribution of data blocks based on data temperature |
| CN103365717A (en) * | 2013-06-25 | 2013-10-23 | 华为技术有限公司 | Memory access method, device and system |
Also Published As
| Publication number | Publication date |
|---|---|
| CN104516952A (en) | 2015-04-15 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Ebadi et al. | An energy‐aware method for data replication in the cloud environments using a tabu search and particle swarm optimization algorithm | |
| JP6371858B2 (en) | Atomic writing for multiple extent operations | |
| CN109274752A (en) | The access method and device, electronic equipment, storage medium of block chain data | |
| CN104899286B (en) | Distributed content is stored and is fetched | |
| US9134908B2 (en) | Logical volume space sharing | |
| CN100476742C (en) | Load balancing method based on object storage device | |
| CN107247619B (en) | Live migration of virtual machine method, apparatus, system, storage medium and equipment | |
| CN104021145B (en) | The method and apparatus that a kind of mixed service concurrently accesses | |
| CN106294352B (en) | A kind of document handling method, device and file system | |
| CN105893139A (en) | Method and device for providing storage service for tenant in cloud storage environment | |
| CN107885758A (en) | The data migration method and dummy node of a kind of dummy node | |
| US10915497B1 (en) | Multi-tier storage system with controllable relocation of files from file system tier to cloud-based object storage tier | |
| US20120159115A1 (en) | Software architecture for service of collective memory and method for providing service of collective memory using the same | |
| CN105183670B (en) | Data processing method and device for distributed cache system | |
| CN107179878A (en) | The method and apparatus of data storage based on optimizing application | |
| CN109213699A (en) | A kind of metadata management method, system, equipment and computer readable storage medium | |
| CN103067479A (en) | Network disk synchronized method and system based on file coldness and hotness | |
| CN109446114A (en) | A kind of spatial data caching method, device and storage medium | |
| CN107465729B (en) | Virtual node copy adjusting system and method based on access time ranking | |
| CN108595503A (en) | Document handling method and server | |
| CN107807797A (en) | The method, apparatus and server of data write-in | |
| CN104956341A (en) | Data migration method, device and processor | |
| CN104461706B (en) | A kind of method and multiprocessing device that shared global variable is shared | |
| CN104516952B (en) | A kind of memory partitioning dispositions method and device | |
| CN106331160A (en) | Data migration method and system |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant | ||
| TR01 | Transfer of patent right |
Effective date of registration: 20220216 Address after: 550025 Huawei cloud data center, jiaoxinggong Road, Qianzhong Avenue, Gui'an New District, Guiyang City, Guizhou Province Patentee after: Huawei Cloud Computing Technology Co.,Ltd. Address before: 518129 Bantian HUAWEI headquarters office building, Longgang District, Guangdong, Shenzhen Patentee before: HUAWEI TECHNOLOGIES Co.,Ltd. |
|
| TR01 | Transfer of patent right |