CN109857550A - Resource allocation method, device, equipment and storage medium based on machine learning - Google Patents
Resource allocation method, device, equipment and storage medium based on machine learning Download PDFInfo
- Publication number
- CN109857550A CN109857550A CN201910013393.2A CN201910013393A CN109857550A CN 109857550 A CN109857550 A CN 109857550A CN 201910013393 A CN201910013393 A CN 201910013393A CN 109857550 A CN109857550 A CN 109857550A
- Authority
- CN
- China
- Prior art keywords
- node
- resource
- information
- resource allocation
- application
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000013468 resource allocation Methods 0.000 title claims abstract description 114
- 238000000034 method Methods 0.000 title claims abstract description 54
- 238000010801 machine learning Methods 0.000 title claims abstract description 34
- 238000000605 extraction Methods 0.000 claims abstract description 15
- 238000004364 calculation method Methods 0.000 claims abstract description 6
- 238000012549 training Methods 0.000 claims description 31
- 238000004590 computer program Methods 0.000 claims description 17
- 230000006399 behavior Effects 0.000 claims description 8
- 230000005540 biological transmission Effects 0.000 claims description 5
- 238000012886 linear function Methods 0.000 claims description 4
- 238000004321 preservation Methods 0.000 claims 1
- 238000010586 diagram Methods 0.000 description 10
- 238000004458 analytical method Methods 0.000 description 4
- 230000000694 effects Effects 0.000 description 3
- 239000004744 fabric Substances 0.000 description 3
- 230000008569 process Effects 0.000 description 3
- 238000012545 processing Methods 0.000 description 3
- 238000004891 communication Methods 0.000 description 2
- 239000003550 marker Substances 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 230000004044 response Effects 0.000 description 2
- 239000002699 waste material Substances 0.000 description 2
- 238000004422 calculation algorithm Methods 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 238000010276 construction Methods 0.000 description 1
- 238000012217 deletion Methods 0.000 description 1
- 230000037430 deletion Effects 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 238000012417 linear regression Methods 0.000 description 1
- 238000013508 migration Methods 0.000 description 1
- 230000005012 migration Effects 0.000 description 1
- 239000000203 mixture Substances 0.000 description 1
- 238000002360 preparation method Methods 0.000 description 1
- 230000009467 reduction Effects 0.000 description 1
- 230000003068 static effect Effects 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- 239000011800 void material Substances 0.000 description 1
Abstract
This application involves field of cloud calculation, load allotment is realized in specifically used machine learning, and discloses a kind of resource allocation method based on machine learning, device, equipment and storage medium, which comprises acquires the current performance information of each node；From the current performance information extraction characteristic；The characteristic is input to preset resources model, to export resource allocation information；The resource allocation information is sent to each node, so that the corresponding resource allocation information is saved into corresponding configuration file and then completed resource distribution by each node, and then the utilization rate of resource is improved and reduces human cost.
Description
Technical field
This application involves field of communication technology more particularly to a kind of resource allocation method based on machine learning, device, set
Standby and storage medium.
Background technique
In distributed resource scheduling system, static scheduling is generallyd use, that is, the good resource distribution text of user's predefined
Part, or the resource SC service ceiling of each application program of limitation.This scheduling mode not only wastes a large amount of manpowers, but also is distributed
Formula resource scheduling system cannot be each application assigned resource according to the practical operation situation of application program, so that resource point
With unreasonable, node resource cannot be utilized to greatest extent, thus is resulted in waste of resources or part resource overload, when serious
Data migration problems will lead to calculating and the delay of task.It is therefore desirable to provide a kind of resource distribution based on machine learning
Method, to improve the utilization rate of resource and reduce human cost.
Summary of the invention
This application provides a kind of resource allocation method based on machine learning, device, equipment and storage mediums, to improve
The utilization rate of resource simultaneously reduces human cost.
In a first aspect, be used for this application provides a kind of resource allocation method based on machine learning in distributed system,
The described method includes:
Acquire the current performance information of each node；
From the current performance information extraction characteristic；
The characteristic is input to preset resources model, to export resource allocation information；
The resource allocation information is sent to each node, so that each node is by the corresponding resource
Configuration information saves into corresponding configuration file and then completes resource distribution.
Second aspect, present invention also provides a kind of device for allocating resources based on machine learning, are applied to distributed system
System, described device include:
Acquisition unit, for acquiring the current performance information of each node；
Extraction unit, for from the current performance information extraction data information, the data information to include characteristic；
Input-output unit, for the node diagnostic data to be input to preset resources model, to export money
Source configuration information；
Transmission unit, for sending the resource allocation information to each node, so that each node will
The corresponding resource allocation information saves into corresponding configuration file and then completes resource distribution.
The third aspect, present invention also provides a kind of computer equipments, are applied to distributed system, the computer equipment
Including memory and processor；The memory is for storing computer program；The processor, for executing the computer
Program simultaneously realizes such as above-mentioned resource allocation method when executing the computer program.
Fourth aspect, present invention also provides a kind of computer readable storage medium, the computer readable storage medium
It is stored with computer program, the computer program makes the processor realize such as above-mentioned resource distribution when being executed by processor
Method.
This application discloses a kind of resource allocation method of machine learning, device, equipment and storage mediums, pass through host node
The resource allocation information of each node in host node and child node is exported according to current performance information based on resources model,
And corresponding resource allocation information is sent in the configuration file of respective nodes, and then complete resource distribution, improve this point
The resource utilization of cloth system.In addition, the resource allocation method needs not rely on the configuration file manually finished writing in advance, but
The resource characteristic message reflection that dynamic has been changed to is into configuration file, when changing resource characteristic information, does not need artificial
Configuration file is modified, the dynamic, real-time and reliability of resource allocation is realized, significantly reduces the workload of engineer,
The time of engineer is saved, so that resource allocation is simpler, wide application of the crowd, and by engineer from cumbersome configuration file
In free, can more be absorbed in the exploitation of business.In addition, this method dexterously utilizes time series to resource allocation information
Predicted, record the resource characteristic information of each node in real time, resource distribution it is high-efficient.
Detailed description of the invention
Technical solution in ord to more clearly illustrate embodiments of the present application, below will be to needed in embodiment description
Attached drawing is briefly described, it should be apparent that, the accompanying drawings in the following description is some embodiments of the present application, general for this field
For logical technical staff, without creative efforts, it is also possible to obtain other drawings based on these drawings.
Fig. 1 is the step exemplary flow that resources model is established based on machine learning that embodiments herein one provides
Figure；
Fig. 2 is the sub-step schematic flow diagram for establishing resources model provided in Fig. 1；
Fig. 3 is the step exemplary flow for the resource allocation method based on machine learning that embodiments herein one provides
Figure；
Fig. 4 is the distributed system that the resource allocation method based on machine learning that embodiments herein provides is applicable in
Schematic block diagram；
Fig. 5 is the step exemplary flow for the resource allocation method based on machine learning that embodiments herein two provides
Figure；
Fig. 6 is the step exemplary flow for the resource allocation method based on machine learning that embodiments herein three provides
Figure；
Fig. 7 is that embodiments herein also provides a kind of schematic block diagram of device for allocating resources based on machine learning；
Fig. 8 is the schematic block diagram of the model training unit of device for allocating resources in Fig. 7；
Fig. 9 is a kind of structural representation block diagram for computer equipment that one embodiment of the application provides.
Specific embodiment
Below in conjunction with the attached drawing in the embodiment of the present application, technical solutions in the embodiments of the present application carries out clear, complete
Site preparation description, it is clear that described embodiment is some embodiments of the present application, instead of all the embodiments.Based on this Shen
Please in embodiment, every other implementation obtained by those of ordinary skill in the art without making creative efforts
Example, shall fall in the protection scope of this application.
Flow chart shown in the drawings only illustrates, it is not necessary to including all content and operation/step, also not
It is that must be executed by described sequence.For example, some operation/steps can also decompose, combine or partially merge, therefore practical
The sequence of execution is possible to change according to the actual situation.
Embodiments herein provide a kind of resource allocation method based on machine learning, device, computer equipment and
Storage medium.Resource allocation method based on machine learning can be used for distributed system, to improve utilization rate and the reduction of resource
Human cost.
With reference to the accompanying drawing, it elaborates to some embodiments of the application.In the absence of conflict, following
Feature in embodiment and embodiment can be combined with each other.
Referring to Fig. 1, Fig. 1 is embodiments herein offer the step of establishing resources model based on machine learning
Schematic flow diagram.The resources method for establishing model is to utilize machine learning, training resource prediction model.
As shown in Figure 1, the step of resources model should being established based on machine learning.The resources model is applied to divide
Cloth system, for the resource allocation information according to the characteristic output node of the node of distributed system, so that each node
Corresponding resource allocation information is saved into corresponding configuration file to and then is completed resource distribution, improves the utilization rate of resource,
Reduce human cost.
Wherein, which includes that (node is the void in computer equipment or computer equipment at least two nodes
Quasi- machine).The executing subject for establishing resources model of the present embodiment can be any one node of distributed system.
As shown in Figure 1, this establishes the step of resources model, the following contents is specifically included:
The historical performance information of S101, each node of acquisition, construct sample data.
In the present embodiment, historical performance information includes CPU information, IO information, memory information, number of threads, connection quantity
At least one of equal resource characteristics information.Can certainly include request resource size, response time and ambient temperature,
The running state informations such as humidity, air quality and the weather condition in coming few hours.
Specifically, when acquiring the historical performance information of each node, it can be to preset the unit time as the one of acquisition
A period acquires the performance information of each node.Wherein, the acquisition of the performance information can be continuously, for example, when last
At the end of the acquisition in period, the acquisition of next cycle is got started.It is of course also possible to be discontinuous, for example, by daily
Fixed time period is acquired as a cycle.The historical performance information of each node is each node before current period
The collected performance information of period institute.
Specifically, when according to marking off the period as unit using each minute time span in each consecutive days in advance
Between, determined the performance information of each node in a upper period.That is, every after one minute, then acquired each node at this point
Performance information in clock.
For example, possessing 24 hours each consecutive days, that is, 1440 minutes, then divide a consecutive days according to per minute
For 1440 parts of unit time, and it is every through after one minute when, each node in (that is, in a upper unit time) upper one minute of acquisition
Performance information.Such as, current time be 13 points 59 seconds 05 minute, then when the time 13: 0 06: next second, then the host node can be with
Each node is acquired in 13 points of performance informations within 0 06: 0 second 05 minute to 13:.
Wherein, the period institute of preset quantity of the historical performance information of each node for each node before current period
The combination of corresponding performance information.For example, current period be T, preset quantity 3, then the historical performance information of each node be
Performance information of each node in the T-1 period, the performance information in T-2 period and the present count before the performance information in T-3 period
The collected performance information of institute in the period of amount.
For example, using each minute of consecutive days as acquisition a cycle, if current period be 13 points 0 second to 13 05 minute
Point 0 second 06 minute, preset quantity 3, then the historical performance information of each node be each node 13 points 0 second to 13: 05 02 minute
Performance information collected in point 0 second, including 13 points 0 03: 0 second 02 minute to 13:, 13 points 04 0 second 03 minute to 13: 0
Second, the performance information in 13 points of 0 second 04 minute to 13 points 05 minute 0 second three periods.
Wherein, as shown in Fig. 2, the historical performance information for obtaining each node, constructs sample data, specifically include with
Lower sub-step S101a and S101b.
The historical performance information of S101a, each node of acquisition.
Specifically, the historical performance information of each node in host node and child node is remembered in the form of timestamp and data
Record the system log in corresponding node.Host node can acquire the historical performance letter of corresponding node from the system log of each node
Breath.
S101b, the historical performance information is marked, constructs sample data.
Wherein, the historical performance information is marked, constructs sample data, specifically: it is right by preset mark rule
Historical performance information is marked.More specifically, preset mark rule can be by the historical performance information according to resource
The type of feature carries out classification marker, such as will divide about " CPU information " in historical performance information in first category, by history
" IO information " point in performance information divides in second category, by " memory information " in historical performance information in third classification.Then
When predicting the resource allocation information of node cpu, first category is marked, the sample data of CPU is constructed.For predict its
The resource allocation information of his resource is referred to above-mentioned mark mode.Certainly, preset mark rule can also be according to actual needs
It is designed as other marking conventions.
S102, model training is carried out according to the sample data, obtains resources model.
Specifically, described carry out model training according to the sample data, resources model is obtained, specifically: it is based on
Linear regression algorithm carries out model training according to the sample data, obtains resources model.
This utilizes machine mould establishing resource prediction model to be applied to distributed system to above-described embodiment, for according to distribution
The resource allocation information of the characteristic output node of the node of formula system, so that each node protects corresponding resource allocation information
Resource distribution is deposited into corresponding configuration file and then completed, the utilization rate of resource is improved, reduces human cost.
Referring to Fig. 3, the step of Fig. 3 is the resource allocation method based on machine learning that embodiments herein one provides
Schematic flow diagram.The resource allocation method is applied to distributed system.The distributed system includes that (node is at least two nodes
Virtual machine in computer equipment or computer equipment).The executing subject for establishing resources model of the present embodiment can be
Any one node of distributed system.
Wherein, multiple applications can be disposed on each node in distributed system, each application can occupy inhomogeneity
The resource of type, such as: the resources such as CPU, memory, IO.Using to be mounted on any application on node, including but not limited to browse
Device, Email, instant message service, word processing, keyboard be virtual, widget (Widget), encryption, digital publishing rights pipe
Reason, speech recognition, speech reproduction, music etc..It should be noted that the operation of each application is all based under the application
The operation of each process.
As shown in figure 4, the node for executing following resource allocation methods is known as host node 301, Qi Tajie in the present embodiment
Point is known as child node 302.For example, as shown in Fig. 2, include three nodes in the figure, one of them is host node 301, in addition two
A is child node 302.
It should be noted that the form that above-mentioned Fig. 4 does not constitute distributed system limits, host node in distributed system
Contacting between 301 quantity, the quantity of child node 302 and host node 301 and child node 302 can carry out according to actual needs
Design.
As shown in figure 3, being somebody's turn to do the resource allocation method based on machine learning, executing subject is host node, naturally it is also possible to
For one of child node.This method specifically includes: step S201 to step S204.
The current performance information of S201, each node of acquisition.
In the present embodiment, the current performance information of each node is each node in host node and child node in current period
Interior performance information.
The current performance information includes that the resources such as CPU information, IO information, memory information, number of threads, connection quantity are special
At least one of reference breath.It can certainly include request resource size, response time and ambient temperature, humidity, air
The running state informations such as the weather condition in quality and coming few hours.
S202, from the current performance information extraction characteristic.
In the present embodiment, the characteristic includes resource characteristic information, such as CPU information and memory information etc..Wherein,
Characteristic is the characteristic in current period.
From the current performance information extraction characteristic, specifically: according to default extracting rule, from current performance information
Extract characteristic.More specifically, preset mark rule can be by the current performance information according to the class of resource characteristic
Type carries out classification marker, such as will divide about " CPU information " in current performance information in first category, by historical performance information
In " IO information " point second category, by " memory information " in historical performance information point in third classification.Then in host node
When extracting the characteristic of the CPU of each node in host node and child node, first category is extracted, the spy of CPU is obtained
Levy data.Said extracted mode is referred to for the characteristic of other resources.Certainly, presetting extracting rule can also basis
Actual needs is designed as other extracting rules.
S203, the characteristic is input to preset resources model, to export resource allocation information.
In the present embodiment, resource allocation information is the characteristic in the prediction in next period.For example, it is assumed that current period
For T, next period is T+1.Based on the resources model, host node according to the T period characteristic output host node and
Resource allocation information of each node in the T+1 period in child node.
In one embodiment, in order to make the resource allocation information of output with more reference value, the duration in each period is all
Equal, i.e., the described current period and next period are equal.
In one embodiment, described that the characteristic is input to preset resources model, matched with exporting resource
The trigger condition of confidence breath can include at least following two: after host node receives specific information (such as characteristic),
Trigger the prediction to resource allocation information；Alternatively, in specific trigger condition, (such as newly-increased node, deletion of node, deployment are answered
With, delete application), trigger prediction to resource allocation information；Alternatively, can also be carried out with clocked flip to resource allocation information pre-
It surveys.Host node can trigger resource distribution every some cycles, can also be when detecting above-mentioned typical emergency event, triggering money
Source configuration.
S204, the resource allocation information is sent to each node, so that each node is by corresponding institute
Resource allocation information is stated to save into corresponding configuration file and then complete resource distribution.
Wherein, corresponding configuration file can be the configuration file of node, be also possible to the configuration file of application.
Specifically, host node is sent to respective nodes after exporting resource allocation information, by the resource allocation information.It connects
The node for receiving the resource allocation information of host node transmission, the corresponding resource allocation information is saved to matching accordingly
It sets in file, to complete resource distribution.
In above-described embodiment, host node is based on resources model, according to current performance information, exports host node and son section
The resource allocation information of each node in point, and corresponding resource allocation information is sent in the configuration file of respective nodes,
And then resource distribution is completed, improve the resource utilization of the distributed system.In addition, the resource allocation method needs not rely on people
The configuration file that fortification are first finished writing, but the resource characteristic message reflection that dynamic has been changed to is changing into configuration file
When resource characteristic information, manual amendment's configuration file is not needed, realizes the dynamic, real-time and reliability of resource allocation,
The workload for significantly reducing engineer saves the time of engineer, so that resource allocation is simpler, wide application of the crowd,
And free engineer from cumbersome configuration file, it can more be absorbed in the exploitation of business.In addition, this method is dexterously
Resource allocation information is predicted using time series, records the resource characteristic information of each node, the effect of resource distribution in real time
Rate is high.
The step of 5, Fig. 5 is the resource allocation method based on machine learning of the offer of embodiments herein two is please referred to show
Meaning flow chart.The executing subject of the present embodiment is known as host node, other nodes are known as child node.Wherein, host node can be point
Any one node in cloth storage system.
As shown in figure 5, being somebody's turn to do the resource allocation method based on machine learning, specifically include: step S401 to step S404.
The current performance information of S401, each node of acquisition.
In the present embodiment, current performance information includes present node performance information.Present node performance information, which is included in, works as
At least one of the CPU information of the node in preceding period, the IO information of node, resource characteristics information such as memory information of node.
S402, from the current performance information extraction characteristic, the characteristic includes node diagnostic data.
In the present embodiment, the node diagnostic data can be the CPU information of node and memory information of node etc..Its
In, node diagnostic data are the characteristic of host node or child node itself in current period.
S403, the node diagnostic data are input to preset node resource prediction model, are matched with output node resource
Confidence breath.
In one embodiment, described that the node diagnostic data are input to the node resource prediction model, with output
Before node resource configuration information, further includes: training node resource prediction model.
Wherein, training node resource prediction model, specifically includes the following steps: acquiring the history joint behavior of each node
Information, structure node sample data；Model training is carried out according to the node sample data, obtains node resource prediction model.
Specifically, each node is in each period, there are many joint behavior information of resource.According to the every of each node
The node sample data of the history joint behavior information structuring of the kind resource resource, can establish one according to the node sample data
A node resource prediction model predicts the node resource configuration information of the resource.The history node of every kind of resource of each node
Performance information can regard a time series { H asp(t), t=1,2 ..., T }, node resource configuration information can be by preceding k
Corresponding node sample data prediction obtains.
Wherein, the node resource prediction model are as follows:
Wherein, HpIt (t) is node resource configuration information,For linear function, k is Embedded dimensions, and p is node resource class
Type.
Specifically, can be by model above, according to the acquisition of the history joint behavior information of every kind of resource of each node
Period selects suitable parameter k, the input according to the history joint behavior H-k data set of information structuring, as model training
Data set (i.e. node sample data).Specifically it is referred to table 1.
Table 1
| Input data | Output data |
| Hp(1),Hp(2),...,Hp(k) | Hp(k+1) |
| ...... | ...... |
| Hp(t-k),...,Hp(t-2),Hp(t-1) | Hp(t) |
| ...... | ...... |
| Hp(T-k),...,Hp(T-2),Hp(T-1) | Hp(T) |
S404, the node resource configuration information is sent to each node, so that each node will correspond to
The node resource configuration information save into corresponding configuration file so that complete resource distribution.
In the present embodiment, corresponding configuration file is the configuration file of node.
Specifically, host node is sent to phase after output node resource allocation information, by the node resource configuration information
Answer node.
In above-described embodiment, host node is based on node resource prediction model and exports main section according to present node performance information
The node resource configuration information of each node in point and child node, and corresponding node resource configuration information is sent to corresponding section
In the configuration file of point, and then resource distribution is completed, improves the resource utilization of the distributed system.In addition, the resource distribution
Method needs not rely on the manually configuration file finished writing in advance, but the resource characteristic message reflection that has been changed to of dynamic is to configuring
In file, when changing resource characteristic information, do not need manual amendment's configuration file, realize resource allocation dynamic,
Real-time and reliability significantly reduce the workload of engineer, save the time of engineer, so that resource allocation is simpler
It is single, wide application of the crowd, and engineer is freed from cumbersome configuration file, it can more be absorbed in the exploitation of business.This
Outside, this method dexterously predicts resource allocation information using time series, records the resource characteristic letter of each node in real time
Breath, resource distribution it is high-efficient.
The step of 6, Fig. 6 is the resource allocation method based on machine learning of the offer of embodiments herein three is please referred to show
Meaning flow chart.The method of the present embodiment on the basis of example 1, has further refined some steps so that distributed be
The resource allocation proposal of system is more perfect, such as has refined the way of output of resource allocation information.The executing subject of the present embodiment
Referred to as host node, other nodes are known as child node.Wherein, host node can be any one section in distributed memory system
Point.
As shown in fig. 6, being somebody's turn to do the resource allocation method based on machine learning, specifically include: step S501 to step S505.
The current performance information of S501, each node of acquisition.
In the present embodiment, current performance information includes present node performance information and current application performance information.Work as prosthomere
Point performance information include the CPU information in the node of current period, the IO information of node, node the resource characteristics such as memory information
At least one of information.Current application performance information include the CPU information of the application of current period, the IO information of application and
At least one of resource characteristics information such as memory information of application.
S502, from the current performance information extraction characteristic, the characteristic includes applying characteristic.
In the present embodiment, described using characteristic can be the CPU information and memory information of application etc. of application.Its
In, it is the application characteristic applied in host node or child node in current period using characteristic.
S503, the application characteristic is input to preset application resource prediction model, is matched with exporting application resource
Confidence breath.
It is in one embodiment, described that the application characteristic is input to the application resource prediction model by described,
Before exporting application resource configuration information, further includes: training application resource prediction model.
Wherein, training application resource prediction model, specifically includes the following steps: acquiring going through for each application of each node
History application performance information, Structural application sample data；Model training is carried out according to the application sample data, be applied resource
Prediction model.
Specifically, each node is in each period, there are many performance informations of resource.Each of each node is applied
Each period, the historical performance information for the every kind of resource applied according to every kind constructed the money also there are many performance information of resource
The application sample data in source can establish an application resource prediction model using sample data according to this to predict the resource
Application resource configuration information.The historical performance information of every kind of resource of every kind of application can regard a time series { X asr(t),t
=1,2 ..., T }, application resource configuration information can be obtained by the application sample data prediction of first k corresponding application.
Wherein, the application resource prediction model are as follows:
Wherein, XrIt (t) is node resource configuration information,For linear function, k is Embedded dimensions, and r is resource type, than
Such as CPU, memory.
Specifically, can be by model above, the collection period of the historical performance information for the every kind of resource applied according to every kind
Suitable parameter k is selected, constructs T-k data set according to the historical performance information, the input data set as model training is (i.e.
Using sample data).Specifically it is referred to table 2.
Table 2
| Input data | Output data |
| Xr(1),Xr(2),...,Xr(k) | Xr(k+1) |
| ...... | ...... |
| Xr(t-k),...,Xr(t-2),Xr(t-1) | Xr(t) |
| ...... | ...... |
| Xr(T-k),...,Xr(T-2),Xr(T-1) | Xr(T) |
S504, according to the corresponding application resource configuration information of each node, calculate the corresponding section of each node
Point resource allocation information.
In the present embodiment, described according to the corresponding application resource configuration information of each node, it calculates each described
The corresponding resource allocation information of node, specifically includes: based on configuration calculation formula, being provided according to the corresponding application of each node
Source configuration information calculates the corresponding resource allocation information of each node.
Wherein, the configuration calculation formula are as follows:
Wherein, CrFor resource allocation information；N indicates the application number in the node, and r is resource type, and t is current week
Phase.
For some node in host node and child node, in the T+1 period, applied often according to formula (3) by each
The application resource configuration information summation of kind resource, can calculate the node resource configuration information of every kind of resource of each node.Example
Such as, A node includes using a and applying b, then, then can be by the CPU of application a when calculating the node resource configuration information of A node
Configuration information is added with the CPU configuration information of application b, and the CPU configuration information of A node can be obtained.Other resources of node
Node resource configuration information is referred to above-mentioned calculation and is calculated, and details are not described herein.
S505, the resource allocation information of the node is sent to the corresponding node, so that each node will be right
The resource allocation information answered saves into corresponding configuration file and then completes resource distribution.
In the present embodiment, resource allocation information may include node resource configuration information, may also comprise application resource and matches
Confidence breath, or be combination.The i.e. described resource allocation information for sending the node to the corresponding node so that
Corresponding resource allocation information is saved into corresponding configuration file and then is completed resource distribution by each node, specifically may be used
Think following at least three kinds of schemes:
The first scheme sends the node resource configuration information of the node to the corresponding node, so that each institute
Node is stated to save into corresponding configuration file corresponding node resource configuration information and then complete resource distribution.
Second scheme sends the application resource configuration information applied in the node to the corresponding node, so that
Corresponding application resource configuration information is saved into corresponding configuration file and then completes resource distribution by each node.
The third scheme is the combination of the first scheme and second scheme.
In above-described embodiment, the application characteristic is input to preset application resource prediction model by host node, with
Export application resource configuration information；According to the corresponding application resource configuration information of each node, each node is calculated
Corresponding node resource configuration information；And corresponding node resource configuration information is sent in the configuration file of respective nodes,
And then resource distribution is completed, improve the resource utilization of the distributed system.In addition, the resource allocation method needs not rely on people
The configuration file that fortification are first finished writing, but the resource characteristic message reflection that dynamic has been changed to is changing into configuration file
When resource characteristic information, manual amendment's configuration file is not needed, realizes the dynamic, real-time and reliability of resource allocation,
The workload for significantly reducing engineer saves the time of engineer, so that resource allocation is simpler, wide application of the crowd,
And free engineer from cumbersome configuration file, it can more be absorbed in the exploitation of business.In addition, this method is dexterously
Resource allocation information is predicted using time series, records the resource characteristic information of each node, the effect of resource distribution in real time
Rate is high.
It should be noted that the resource allocation method that Fig. 5 and Fig. 6 is provided, can be used alone to improve resource distribution effect
Rate simultaneously reduces human cost；Certainly it can also be used together to improve Allocation Efficiency and reduce human cost.
Referring to Fig. 7, Fig. 7 is that embodiments herein also provides a kind of showing for device for allocating resources based on machine learning
Meaning property block diagram, the device for allocating resources is for executing any one of aforementioned resource allocation method based on machine learning.Wherein, the money
Source configuration device can be configured in server or terminal.
Wherein, server can be independent server, or server cluster.The terminal can be mobile phone, put down
The electronic equipments such as plate computer, laptop, desktop computer, personal digital assistant and wearable device.
As shown in fig. 7, device for allocating resources 600 includes: acquisition unit 610, extraction unit 620, input-output unit
630, transmission unit 640.
Acquisition unit 610, for acquiring the current performance information of each node.
Extraction unit 620 is used for from the current performance information extraction characteristic.
Input-output unit 630, for the characteristic to be input to preset resources model, to export resource
Configuration information.
Transmission unit 640, for sending the resource allocation information to each node, so that each node
The corresponding resource allocation information is saved into corresponding configuration file and then completes resource distribution.
In one embodiment, the device for allocating resources further includes model training unit 650, predicts mould for training resource
Type.
As shown in figure 8, in some embodiments, model training unit 650 includes nodal analysis method training unit 651, use
In training node resource prediction model.
Wherein, nodal analysis method training unit 651 includes the first data configuration unit 6511 and nodal analysis method training subelement
6512。
Specifically, the first data configuration unit 6511, for acquiring the history joint behavior information of each node, construction section
Point sample data；Nodal analysis method trains subelement 6512, for carrying out model training according to the node sample data, is saved
Point resources model.
In some embodiments, model training unit 650 includes application model training unit 652, for training application
Resources model.Input-output unit 630 includes applying output unit 631 and node computing unit 632.
Wherein, using output unit 631, mould is predicted for the application characteristic to be input to the application resource
Type, to export application resource configuration information；Node computing unit 632, for according to the corresponding application resource of each node
Configuration information calculates the corresponding node resource configuration information of each node.
Wherein, application model training unit 652 includes the second data configuration unit 6521 and application model training subelement
6522。
Specifically, the second data configuration unit 6521, the historical usage performance of each application for acquiring each node
Information, Structural application sample data；Application model trains subelement 6522, for carrying out model according to the application sample data
Training, be applied resources model.
In one embodiment, the device for allocating resources further includes trigger unit, for triggering that the characteristic is defeated
Enter to preset resources model, to export resource allocation information.
It should be noted that it is apparent to those skilled in the art that, for convenience of description and succinctly,
The device for allocating resources based on machine learning of foregoing description and the specific work process of each unit can be based on machine with reference to aforementioned
Corresponding process in the resource allocation method embodiment of device study, details are not described herein.
Above-mentioned device for allocating resources can be implemented as a kind of form of computer program, which can be such as
It is run in computer equipment shown in Fig. 9.
Referring to Fig. 9, Fig. 9 is a kind of schematic block diagram of computer equipment provided by the embodiments of the present application.The computer
Equipment can be server or terminal.
Wherein, server can be independent server, be also possible to the server cluster of multiple server compositions；Terminal
It can be the electronics such as smart phone, tablet computer, laptop, desktop computer, personal digital assistant and wearable device to set
It is standby etc..
Refering to Fig. 9, which includes processor, memory and the network interface connected by system bus,
In, memory may include non-volatile memory medium and built-in storage.
Non-volatile memory medium can storage program area and computer program.The computer program includes program instruction,
The program instruction is performed, and processor may make to execute a kind of resource allocation method based on machine learning.
Processor supports the operation of entire computer equipment for providing calculating and control ability.
Built-in storage provides environment for the operation of the computer program in non-volatile memory medium, the computer program quilt
When processor executes, processor may make to execute a kind of resource allocation method.
The network interface such as sends the task dispatching of distribution for carrying out network communication.It will be understood by those skilled in the art that
Structure shown in Fig. 9, only the block diagram of part-structure relevant to application scheme, is not constituted to application scheme institute
The restriction for the computer equipment being applied thereon, specific computer equipment may include than more or fewer portions as shown in the figure
Part perhaps combines certain components or with different component layouts.
It should be understood that processor can be central processing unit (Central Processing Unit, CPU), it should
Processor can also be other general processors, digital signal processor (Digital Signal Processor, DSP), specially
With integrated circuit (Application Specific Integrated Circuit, ASIC), ready-made programmable gate array
(Field-Programmable Gate Array, FPGA) either other programmable logic device, discrete gate or transistor are patrolled
Collect device, discrete hardware components etc..Wherein, general processor can be microprocessor or the processor be also possible to it is any often
The processor etc. of rule.
Wherein, the processor is for running computer program stored in memory, to realize following steps:
Acquire the current performance information of each node；From the current performance information extraction characteristic；By the feature
Data are input to preset resources model, to export resource allocation information；The resource allocation information is sent to each institute
Node is stated, so that each node saves the corresponding resource allocation information into corresponding configuration file and then complete
At resource distribution.
In one embodiment, the characteristic described is input to preset resources mould realizing by the processor
When type is to export resource allocation information, for realizing:
The node diagnostic data are input to the node resource prediction model, with output node resource allocation information.
In one embodiment, the resources model includes node resource prediction model；The characteristic includes
Node diagnostic data；The characteristic described is input to preset resources model to export realizing by the processor
When resource allocation information, for realizing:
The node diagnostic data are input to the node resource prediction model, with output node resource allocation information.
In one embodiment, the node diagnostic data described be input to the node and provided realizing by the processor
Source prediction model is also used to before output node resource allocation information:
Acquire the history joint behavior information of each node, structure node sample data；
Model training is carried out according to the node sample data, obtains node resource prediction model.
In one embodiment, the resources model includes application resource prediction model, and the characteristic includes
Using characteristic；The processor realize it is described the characteristic is input to preset resources model, with defeated
Out when resource allocation information, it is used for:
The application characteristic is input to the application resource prediction model, to export application resource configuration information；
According to the corresponding application resource configuration information of each node, the corresponding node resource of each node is calculated
Configuration information.
In one embodiment, the application characteristic described is input to application money realizing by the processor
Source prediction model is also used to before exporting application resource configuration information:
Acquire the historical usage performance information of each application of each node, Structural application sample data；
Model training is carried out according to the application sample data, be applied resources model.
A kind of computer readable storage medium is also provided in embodiments herein, the computer readable storage medium is deposited
Computer program is contained, includes program instruction in the computer program, the processor executes described program instruction, realizes this
Apply for the resource allocation method for any one machine learning that embodiment provides.
Wherein, the computer readable storage medium can be the storage inside of computer equipment described in previous embodiment
Unit, such as the hard disk or memory of the computer equipment.The computer readable storage medium is also possible to the computer
The plug-in type hard disk being equipped on the External memory equipment of equipment, such as the computer equipment, intelligent memory card (Smart
Media Card, SMC), secure digital (Secure Digital, SD) card, flash card (Flash Card) etc..
The above, the only specific embodiment of the application, but the protection scope of the application is not limited thereto, it is any
Those familiar with the art within the technical scope of the present application, can readily occur in various equivalent modifications or replace
It changes, these modifications or substitutions should all cover within the scope of protection of this application.Therefore, the protection scope of the application should be with right
It is required that protection scope subject to.
Claims (10)
1. a kind of resource allocation method based on machine learning is applied to distributed system characterized by comprising
Acquire the current performance information of each node；
From the current performance information extraction characteristic；
The characteristic is input to preset resources model, to export resource allocation information；
The resource allocation information is sent to each node, so that each node is by the corresponding resource distribution
Information preservation is into corresponding configuration file and then completes resource distribution.
2. resource allocation method according to claim 1, which is characterized in that the resources model includes node resource
Prediction model；The characteristic includes node diagnostic data；
It is described that the characteristic is input to preset resources model to export resource allocation information, comprising:
The node diagnostic data are input to the node resource prediction model, with output node resource allocation information.
3. resource allocation method according to claim 2, which is characterized in that described to be input to the node diagnostic data
The node resource prediction model, before output node resource allocation information, further includes:
Acquire the history joint behavior information of each node, structure node sample data；
Model training is carried out according to the node sample data, obtains node resource prediction model.
4. resource allocation method according to claim 3, which is characterized in that the node resource prediction model are as follows:
Wherein, HpIt (t) is node resource configuration information,For linear function, k is Embedded dimensions, and p is node resource type.
5. resource allocation method according to claim 1 or 2, which is characterized in that the resources model includes application
Resources model, the characteristic include applying characteristic；
It is described that the characteristic is input to preset resources model, to export resource allocation information, comprising:
The application characteristic is input to the application resource prediction model, to export application resource configuration information；
According to the corresponding application resource configuration information of each node, the corresponding node resource configuration of each node is calculated
Information.
6. resource allocation method according to claim 5, which is characterized in that described to be input to the application characteristic
The application resource prediction model, before exporting application resource configuration information, further includes:
Acquire the historical usage performance information of each application of each node, Structural application sample data；
Model training is carried out according to the application sample data, be applied resources model.
7. resource allocation method according to claim 6, which is characterized in that the application resource prediction model are as follows:
Wherein, XrIt (t) is application resource configuration information,For linear function, k is Embedded dimensions, and r is resource type；
It is described according to the corresponding application resource configuration information of each node, calculate the corresponding resource distribution of each node
Information, comprising:
Each node is calculated according to the corresponding application resource configuration information of each node based on configuration calculation formula
Corresponding node resource configuration information；Wherein, the configuration calculation formula are as follows:
Wherein, CrFor resource allocation information；N indicates the application number in the node, and r is resource type, and t is current period.
8. a kind of device for allocating resources based on machine learning is applied to distributed system characterized by comprising
Acquisition unit, for acquiring the current performance information of each node；
Extraction unit, for from the current performance information extraction data information, the data information to include characteristic；
Input-output unit is matched for the node diagnostic data to be input to preset resources model with exporting resource
Confidence breath；
Transmission unit, for sending the resource allocation information to each node, so that each node will correspond to
The resource allocation information save into corresponding configuration file so that complete resource distribution.
9. a kind of computer equipment is applied to distributed system, which is characterized in that the computer equipment includes memory and place
Manage device；
The memory is for storing computer program；
The processor, for executing the computer program and realization such as claim 1 when executing the computer program
To resource allocation method described in any one of 7.
10. a kind of computer readable storage medium, which is characterized in that the computer-readable recording medium storage has computer journey
Sequence, the computer program make the processor realize the money as described in any one of claims 1 to 7 when being executed by processor
Source configuration method.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910013393.2A CN109857550A (en) | 2019-01-07 | 2019-01-07 | Resource allocation method, device, equipment and storage medium based on machine learning |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910013393.2A CN109857550A (en) | 2019-01-07 | 2019-01-07 | Resource allocation method, device, equipment and storage medium based on machine learning |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN109857550A true CN109857550A (en) | 2019-06-07 |
Family
ID=66894069
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910013393.2A Pending CN109857550A (en) | 2019-01-07 | 2019-01-07 | Resource allocation method, device, equipment and storage medium based on machine learning |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN109857550A (en) |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111010735A (en) * | 2019-11-20 | 2020-04-14 | 京信通信系统（中国）有限公司 | Resource allocation method, device, base station and storage medium |
| CN111210053A (en) * | 2019-12-20 | 2020-05-29 | 中国银行股份有限公司 | Resource information configuration method and device and server |
| CN112268909A (en) * | 2020-10-20 | 2021-01-26 | 中国科学院合肥物质科学研究院 | Parameter regulation and control method and system for detecting surface defects and uniformity of battery pole piece |
| WO2021051537A1 (en) * | 2019-09-18 | 2021-03-25 | 平安科技（深圳）有限公司 | Deep learning resource scheduling method and appartus, and terminal device |
Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104572308A (en) * | 2015-02-10 | 2015-04-29 | 飞狐信息技术（天津）有限公司 | Computing resource distributing method, distributed type computing method and distributed type computing device |
| CN105512762A (en) * | 2015-12-04 | 2016-04-20 | 网易（杭州）网络有限公司 | Game numerical value launching estimation method and device based on correlation analysis |
| CN107291546A (en) * | 2016-03-30 | 2017-10-24 | 华为技术有限公司 | A kind of resource regulating method and device |
| CN107885762A (en) * | 2017-09-19 | 2018-04-06 | 北京百度网讯科技有限公司 | Intelligent big data system, the method and apparatus that intelligent big data service is provided |
| CN108428020A (en) * | 2018-05-15 | 2018-08-21 | 中国平安人寿保险股份有限公司 | Business personnel's wages prediction technique, device, computer equipment and storage medium |
| CN108446200A (en) * | 2018-02-07 | 2018-08-24 | 福建星瑞格软件有限公司 | Server intelligence O＆M method based on big data machine learning and computer equipment |
| CN109032914A (en) * | 2018-09-06 | 2018-12-18 | 掌阅科技股份有限公司 | Resource occupation data predication method, electronic equipment, storage medium |
-
2019
- 2019-01-07 CN CN201910013393.2A patent/CN109857550A/en active Pending
Patent Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104572308A (en) * | 2015-02-10 | 2015-04-29 | 飞狐信息技术（天津）有限公司 | Computing resource distributing method, distributed type computing method and distributed type computing device |
| CN105512762A (en) * | 2015-12-04 | 2016-04-20 | 网易（杭州）网络有限公司 | Game numerical value launching estimation method and device based on correlation analysis |
| CN107291546A (en) * | 2016-03-30 | 2017-10-24 | 华为技术有限公司 | A kind of resource regulating method and device |
| CN107885762A (en) * | 2017-09-19 | 2018-04-06 | 北京百度网讯科技有限公司 | Intelligent big data system, the method and apparatus that intelligent big data service is provided |
| CN108446200A (en) * | 2018-02-07 | 2018-08-24 | 福建星瑞格软件有限公司 | Server intelligence O＆M method based on big data machine learning and computer equipment |
| CN108428020A (en) * | 2018-05-15 | 2018-08-21 | 中国平安人寿保险股份有限公司 | Business personnel's wages prediction technique, device, computer equipment and storage medium |
| CN109032914A (en) * | 2018-09-06 | 2018-12-18 | 掌阅科技股份有限公司 | Resource occupation data predication method, electronic equipment, storage medium |
Cited By (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2021051537A1 (en) * | 2019-09-18 | 2021-03-25 | 平安科技（深圳）有限公司 | Deep learning resource scheduling method and appartus, and terminal device |
| CN111010735A (en) * | 2019-11-20 | 2020-04-14 | 京信通信系统（中国）有限公司 | Resource allocation method, device, base station and storage medium |
| CN111210053A (en) * | 2019-12-20 | 2020-05-29 | 中国银行股份有限公司 | Resource information configuration method and device and server |
| CN112268909A (en) * | 2020-10-20 | 2021-01-26 | 中国科学院合肥物质科学研究院 | Parameter regulation and control method and system for detecting surface defects and uniformity of battery pole piece |
| CN112268909B (en) * | 2020-10-20 | 2023-10-31 | 中国科学院合肥物质科学研究院 | Parameter regulation and control method and system for detecting surface defects and uniformity of battery pole piece |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109857550A (en) | Resource allocation method, device, equipment and storage medium based on machine learning | |
| US10684761B2 (en) | Graphical user interface creation using highly rated GUI elements | |
| CN111767143B (en) | Transaction data processing method, device, equipment and system | |
| US8838738B2 (en) | System and method for processing medical information through medical terminal | |
| US10523580B2 (en) | Automatic cloud provisioning based on related internet news and social network trends | |
| CN108027905A (en) | Reservation management terminal, reservation management method and reservation management program | |
| WO2017167121A1 (en) | Method and device for determining and applying association relationship between application programs | |
| CN110019087A (en) | Data processing method and its system | |
| US20140123142A1 (en) | System and method for providing data analysis service in cloud environment | |
| CN110020273B (en) | Method, device and system for generating thermodynamic diagram | |
| CN110321283A (en) | Interface testing case generation method, device, computer installation and storage medium | |
| CN111985831A (en) | Scheduling method and device of cloud computing resources, computer equipment and storage medium | |
| CN112182359A (en) | Feature management method and system of recommendation model | |
| CN109325034A (en) | Data processing method, device, computer equipment and storage medium | |
| CN109582827A (en) | Nothing buries point data acquisition method, device, computer equipment and storage medium | |
| CN114157679A (en) | Cloud-native-based distributed application monitoring method, device, equipment and medium | |
| CN113204425A (en) | Method and device for process management internal thread, electronic equipment and storage medium | |
| CN111045928B (en) | Interface data testing method, device, terminal and storage medium | |
| US10397171B2 (en) | Managing content disclosure on social networking sites | |
| CN116032614A (en) | Container network micro-isolation method, device, equipment and medium | |
| US20180054473A1 (en) | System, method and recording medium for creating a social media sensing post | |
| US20200227164A1 (en) | Entity condition analysis based on preloaded data | |
| CN110968398A (en) | Interactive system based on machine learning and big data processing | |
| CN110309848A (en) | The method that off-line data and stream data real time fusion calculate | |
| CN109522492A (en) | A kind of webpage content display control method, device, equipment and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |