CN108664320A - The method and apparatus of resource management - Google Patents
The method and apparatus of resource management Download PDFInfo
- Publication number
- CN108664320A CN108664320A CN201710195959.9A CN201710195959A CN108664320A CN 108664320 A CN108664320 A CN 108664320A CN 201710195959 A CN201710195959 A CN 201710195959A CN 108664320 A CN108664320 A CN 108664320A
- Authority
- CN
- China
- Prior art keywords
- task
- resource
- subset
- goal
- goal task
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/46—Multiprogramming arrangements
- G06F9/50—Allocation of resources, e.g. of the central processing unit [CPU]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/46—Multiprogramming arrangements
- G06F9/50—Allocation of resources, e.g. of the central processing unit [CPU]
- G06F9/5005—Allocation of resources, e.g. of the central processing unit [CPU] to service a request
- G06F9/5027—Allocation of resources, e.g. of the central processing unit [CPU] to service a request the resource being a machine, e.g. CPUs, Servers, Terminals
- G06F9/5055—Allocation of resources, e.g. of the central processing unit [CPU] to service a request the resource being a machine, e.g. CPUs, Servers, Terminals considering software capabilities, i.e. software resources associated or available to the machine
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F2209/00—Indexing scheme relating to G06F9/00
- G06F2209/50—Indexing scheme relating to G06F9/50
- G06F2209/503—Resource availability
Abstract
This application provides a kind of method and apparatus of resource management.This method includes：Detect the resource status of goal task, the goal task belongs to first task subset, the first task subset includes multiple tasks, the multiple tasks of the first task subset are interrelated, the first task subset and the second task subset belong to first task set, the second task subset includes at least one task, and each task at least one task in the second task subset is mutually unrelated；According to the resource status of the goal task, resource management is carried out.The embodiment of the present application can dynamic adjustresources, to improve resource utilization.
Description
Technical field
This application involves big data fields, and more particularly, to the method and apparatus of resource management.
Background technology
Data gradually become the key factor for pushing business model to change with innovation, are emphasized in the starting stage of data analysis
The acquisition of data, after entering developing stage, data be increasingly becoming corporate controller and decision-maker formulate decision it is important according to
According to evolving to the stage of ripeness, data become intra-company's raising efficiency, an important factor for promoting business skill, and start to occur
Special data processing platform (DPP), till now, data more embody its commercial value, form a variety of in data are
The Business Ecosystem of the heart.
Data analysis process can be because the skilled and business of data analyst be accustomed to, and different analysis personnel can use
Different metalanguages, such as applicational language (Java, R, Python), structured query language (Structured Query
Language, SQL) and graphic user interface (Graphical User Interface, GUI) dragging etc..
For example, the pipeline (Pipeline) of u s company (Databricks) supports Scala language in traditional scheme, holding
It when row resource allocation, is distributed using Greedy, that is, pre-defines the resource of needs, for example, central processing unit (Central
Processing Unit, CPU) or distribution information memory (Memory, MEM), then in executing task process, resource is only
It accounts for, and no matter what kind of the service logic of Pipeline is, task is fixed when resource allocation, causes resource profit
It is relatively low with rate.
Invention content
The embodiment of the present application provides a kind of method and apparatus of resource management, can improve resource utilization.
In a first aspect, providing a kind of resource status of detection goal task, which belongs to first task subset,
The first task subset includes multiple tasks, and the multiple tasks of the first task subset are interrelated, the first task subset and
Second task subset belongs to first task set, which includes at least one task, and the second task subset
In at least one task in each task it is mutually unrelated；According to the resource status of the goal task, resource pipe is carried out
Reason.
By detecting the resource status of goal task, the money of goal task whether is met according to the resource status of goal task
Source demand carries out resource management, that is to say, that can dynamic adjustresources, to improve resource utilization.
In some possible realization methods, the resource status according to the goal task, carrying out resource management includes：Root
According to the resource status of the goal task, determine whether the corresponding resource of the goal task meets the resource requirement of the goal task；
When the corresponding resource of the goal task is unsatisfactory for the resource requirement of the goal task, other tasks are obtained for the goal task
Resource.
System can carry out rational allocation to resource in this way, further improve resource utilization.
In some possible realization methods, the resource of the goal task should be unsatisfactory in the corresponding resource of the goal task
When demand, the resource that other tasks are obtained for the goal task includes：The second task correspondence is obtained from the second task subset
Resource, second task be the second task subset in the resource free time task；The corresponding resource of second task is made
For the resource of the goal task.
Goal task finds the task of resource free time in inadequate resource from the second task subset, and then makes full use of
The resource of the idle task, improves resource utilization.
In some possible realization methods, the resource of the goal task should be unsatisfactory in the corresponding resource of the goal task
When demand, the resource that other tasks are obtained for the goal task includes：It is unsatisfactory for the target in the corresponding resource of the goal task
The resource requirement of task, and in the second task subset third is obtained from the first task subset there is no when the second task
The corresponding resource of task, wherein second task is the task of the resource free time in the second task subset, which is
The task of resource free time in the first task subset；Using the corresponding resource of third task as the resource of the goal task.
For system according to the resource status of the goal task, it can be the goal task in goal task to carry out resource management
Inadequate resource, and it is idle that resource is obtained from first task subset there is no when the task of resource free time in the second task subset
The corresponding resource of third task, that is, when the corresponding idling-resource of stateless task is not present, then go to obtain stateful task
Resource avoids the execution for influencing stateful task.
In some possible realization methods, before the resource status of detection goal task, this method further includes：To this
The feature description of goal task is handled to obtain sparse matrix；The sparse matrix is instructed according to stochastic gradient descent algorithm
Get dense matrix；According to the dense matrix, resource is distributed for the goal task.
Applied in data analysis system, after the data analysis system is allocated resource, after distribution resource
The resource of task is adjusted into Mobile state, improves resource utilization.
In some possible realization methods, the task in task and the second task subset in the first task subset
Pass through the same server process.
System can be that goal task obtains the idling-resource for being currently located the task that server includes, and improve the utilization of resources
Rate.
In some possible realization methods, the task in task and the second task subset in the first task subset
Pass through different server process.
System can be the idling-resource that goal task obtains the task that other servers include, and improve resource utilization.
Second aspect, provides a kind of device of resource management, which includes for executing first aspect or first party
The module of method in the possible realization method of any one of face.
The third aspect provides a kind of device of resource management, including processor, memory and communication interface.Processor
It is connect with memory and communication interface.For storing instruction, processor is used for memory for executing the instruction, communication interface
It is communicated with other network elements under the control of processor.When the processor executes the instruction of memory storage, which makes
The processor executes the method in the arbitrary possible realization method of first aspect or first aspect.
Fourth aspect provides a kind of computer storage media, and have program stored therein code in the computer storage media, should
Program code is used to indicate the signal processing in any possible realization method for executing above-mentioned first aspect or first aspect
Method instruction.
Based on the above-mentioned technical proposal, by detecting the resource status of goal task, it is according to the resource status of goal task
It is no meet goal task resource requirement carry out resource management, that is to say, that can dynamic adjustresources, to improve resource
Utilization rate.
Description of the drawings
Fig. 1 is the flow diagram of the universal process of data analysis；
Fig. 2 is the flow diagram of the Pipeline technologies of Databricks；
Fig. 3 is the schematic diagram of the method for the resource management of the embodiment of the present application；
Fig. 4 is the schematic diagram of the data analysis system of the application one embodiment；
Fig. 5 is the schematic diagram of the method for the resource allocation of the application one embodiment；
Fig. 6 is the schematic diagram of the monitor task of the embodiment of the present application；
Fig. 7 is the schematic flow chart of the resource management of the embodiment of the present application；
Fig. 8 is the schematic block diagram of the device of the resource management of the embodiment of the present application；
Fig. 9 is the schematic diagram of the device of the resource management of the embodiment of the present application.
Specific implementation mode
Below in conjunction with attached drawing, the technical solution in the application is described.
Fig. 1 shows the structural schematic diagram of the universal process of data analysis.As shown in Figure 1, data science household custom
Language is programmed, and leads in data analysis process that there are multilingual mixing.Further, since data mining process is constantly repeatedly
Generation, therefore have multiple workflows (workflow) composition.
Fig. 2 shows the flow diagrams of the Pipeline technologies of Databricks.As shown in Fig. 2, the program is to be based on
Template (Core/Mllib/SQL) technology of the platform (Spark) of memory analysis is by the data pick-up of data, conversion and loading
(Extract Transform Load, ETL), training (Train) and inquiry (Query), process is unitized, from distributed document
Data are read in system (Distributed File System, HDFS), handles, finally preserves data.This process passes through
After the mechanism of Pipeline describes processing procedure, finally seek unity of action.Wherein, the Pipeline of Databricks is supported
Scala language, and when executing resource allocation, being distributed using Greedy, that is, pre-define the resource of needs, for example, CPU or
MEM, then in executing task process, resource is exclusive, and no matter what kind of the service logic of Pipeline is, resource
Task is fixed when distribution, and it is relatively low to cause resource utilization.
In addition, traditional resource scheduling system (for example, Mesos, Torque, Omega), resource allocation is to determine resource
Live load and usage amount, for example, the CPU core number of the quantity of server, each server, memory and bandwidth resources etc..Mesos
Resource allocation when based on fairness, that is, pre-define the resource (CPU＆MEM) of needs, in task implementation procedure, money
Source is exclusive, and task is fixed when resource allocation.
Fig. 3 shows the schematic flow chart of the method 300 of the resource management of the embodiment of the present application.The execution master of this method
Body can be the system of resource management, as shown in figure 3, this method 300 includes：
S301 detects the resource status of goal task, which belongs to first task subset, the first task subset
Multiple tasks including multiple tasks, the first task subset are interrelated, the first task subset and the second task subset category
In first task set, which includes at least one task, and in the second task subset this is at least one
Each task in task is mutually unrelated；
S302 carries out resource management according to the resource status of the goal task.
The embodiment of the present application can be applied to include first task set system in, it is first in first task set
The interrelated multiple tasks for subset of being engaged in can have direct or indirect contact between task.For example, multiple task is total
With one project of processing, if lacking some task, which is unable to complete process；Or multiple task has priority suitable
The completion of sequence, previous task influences subsequent task.And the second task subset may include at least one task, between task
There is no correlations.The system can detect the resource status of goal task, according to the resource status of goal task, to target
The resource of task is adjusted.
Optionally, system distributes more resources when the resource of goal task is unsatisfactory for demand for goal task, or
Obtain the resource of other tasks.Or when goal task has idle resource, system is no longer that goal task distributes resource,
Or the idle resource of the goal task is sent to other tasks and is used.System can rationally adjust resource in this way
Match, improves resource utilization.
It should be understood that the goal task can be handled the data of different mode.
It should be noted that the task that first task subset includes is properly termed as the task of " stateful ", the second task
The task of subset is properly termed as the task of " stateless ".
It should also be understood that the embodiment of the present application is illustrated so that goal task is stateful task as an example, the goal task
It can be stateless task, that is, belong to the second task subset, the application is to this without limiting.
Optionally, the embodiment of the present application can be applied in data analysis system, as shown in Figure 4.The data analysis system
Including web page browsing (Web Browser) module, pipeline engine (Pipeline Engine) module, pipeline scheduler
(Pipeline Scheduler) and big data platform (Big Data Platform).The web page browsing module can export use
Family is according to the various programming languages accustomed to using of oneself, for example, the programming mode (Code Mode) in Fig. 4, go-ahead mode
(Guider Mode) and pulling mode (Graphical User Interface Mode, GUI mode) etc. can export programming
Language (for example, Java or Scala), interpreted language (for example, R or Python etc.), script formula language construction query language
(Structured Query Language, SQL) etc.) or GUI pull etc. metalanguages.Pipeline engine modules include field spy
Attribute says (Domain Special Language, DSL) analysis module, optimization module, packetization module and resource distribution module.
Pipeline engine modules parse different programming languages, layout, and after optimization and packing, resource is distributed for each task.It
Afterwards, resource management is carried out to each task in the data analysis system by the method for the embodiment of the present application 300.
It should be noted that DSL analysis modules can unitize different programming languages.The data analysis system can
It is described by unified DSL with the process to data mining analysis, task layout optimization can be carried out, and answered at different strong points
With.
It optionally, can be as shown in Figure 5 to the resource allocation of different task in above-mentioned data analysis system.System needle
This is constructed to the DSL descriptions (for example, data prediction, Feature Engineering, model construction and model prediction) of data analysis process
The sparse matrix of four dimensions, and the sparse matrix is input to stochastic gradient descent (Stochastic Gradient
Descent, SGD) algorithm is trained, and dense matrix is obtained, is that the goal task distributes resource according to the dense matrix.
As shown in fig. 6, system can monitor the resource status of each task in real time.Optionally, according to the goal task
Resource status, it can be that the is obtained from the second task subset in the inadequate resource of the goal task to carry out resource management
The corresponding resource of two tasks, second task are the task of the resource free time in the second task subset.That is, target is appointed
Business finds the task of resource free time in inadequate resource from the second task subset, and then makes full use of the money of the idle task
Source improves resource utilization.
It is initially the resource that second task is distributed it should be noted that the resource of the second task can be system.
It should be understood that goal task can obtain whole resources of second task, can also be the portion for second task that obtains
Divide resource, the application is to this without limiting.
It should also be understood that the data analysis system can comprehensively 20-30 workload of off-line analysis appoint in processing difference
Affecting parameters of business, such as isomery, interference etc..
Optionally, for system according to the resource status of the goal task, it can be the mesh in goal task to carry out resource management
The inadequate resource of mark task, and when task (i.e. the second task) of resource free time being not present in the second task subset, from first
The corresponding resource of third task is obtained in subset of being engaged in.The third task, which can be that the resource in first task subset is idle, appoints
Business.
Optionally, system can also preferentially obtain idle resource, first task from the task in first task subset
There is no tasks when idle resource, then into the second task subset to obtain idling-resource for the task of concentration.System can be pre-
Priority is first set, and the task in the high task subset of priority obtains idle resource.System can pre-set first
The priority of task subset can also pre-set the second task subset higher than the priority or system of the second task subset
Priority is higher than the priority of first task subset.
Optionally, the task in first task subset can pass through the same server with the task in the second task subset
Processing.
Optionally, the task in first task subset can also pass through different services from the task in the second task subset
Device processing.
It should be understood that the multiple tasks in first task subset can be by the same server process, it can also be by not
Same server process.Similarly, the multiple tasks in the second task subset can be by the same server process, can also
By different server process, the application is to this without limiting.
Therefore, the method for the resource management of the embodiment of the present application, by detecting the resource status of goal task, according to target
The resource requirement whether resource status of task meets goal task carries out resource management, that is to say, that can dynamically adjust money
Source, to improve resource utilization.
Fig. 7 shows the schematic diagram of the method for the resource management of another embodiment of the application.As shown in Figure 7.
701, detect the resource status of the goal task in first server.The goal task can be stateless
Business can also be stateful task.
702, according to the resource status of goal task, determine whether the resource of the goal task meets the need of goal task
It asks.
If the resource of goal task meets the needs of goal task, 707 are thened follow the steps, carries out the processing of goal task.
703, if the resource of the goal task cannot meet the needs of goal task and obtain the stateless of first server
The resource of task.
If after the resource for obtaining the stateless task of first server, meets the resource requirement of goal task, then execute
Step 707, the processing of goal task is carried out.
704, if after getting the resource of stateless task of first server, being not met by the resource of goal task
Demand then obtains the resource of the stateful task of first server.
If after the resource for obtaining the stateful task of first server, meets the resource requirement of goal task, then execute
Step 707, the processing of goal task is carried out.
705, if after getting the resource of stateful task of first server, being not met by the resource of goal task
Demand then obtains the resource of the stateless task of second server.
If after the resource for obtaining the stateless task of second server, meets the resource requirement of goal task, then execute
Step 707, the processing of goal task is carried out.
706, if after getting the resource of stateless task of second server, being not met by the resource of goal task
Demand then obtains the resource of the stateful task of second server.
If after the resource for obtaining the stateful task of second server, meets the resource requirement of goal task, then execute
Step 707, the processing of goal task is carried out.
707, processing target task.
Therefore, the method for the resource management of the embodiment of the present application, by detecting the resource status of goal task, according to target
The resource requirement whether resource status of task meets goal task carries out resource management, that is to say, that can dynamically adjust money
Source, to improve resource utilization.
Fig. 8 shows the schematic diagram of the device 800 of the resource management of the application one embodiment.As shown in figure 8, the device
800 include：
Processing module 810, the resource status for detecting goal task, the goal task belong to first task subset, should
First task subset includes multiple tasks, and the multiple tasks of the first task subset are interrelated, the first task subset and
Two task subsets belong to first task set, which includes at least one task, and in the second task subset
At least one task in each task it is mutually unrelated；
The processing module 810 is additionally operable to the resource status according to the goal task, carries out resource management.
Optionally, which is specifically used for：According to the resource status of the goal task, the goal task is determined
Whether corresponding resource meets the resource requirement of the goal task；It is unsatisfactory for the goal task in the corresponding resource of the goal task
Resource requirement when, the resource of other tasks is obtained for the goal task.
Optionally, which is specifically used for：The corresponding money of the second task is obtained from the second task subset
Source, second task are the task of the resource free time in the second task subset；Using the corresponding resource of the second task as this
The resource of goal task.
Optionally, which is specifically used for：It is unsatisfactory for the goal task in the corresponding resource of the goal task
Resource requirement, and third task pair is obtained from the first task subset there is no when the second task in the second task subset
The resource answered, wherein second task be the second task subset in resource free time task, the third task for this first
The task of resource free time in task subset；Using the corresponding resource of third task as the resource of the goal task.
Optionally, which is additionally operable to：The feature description of the goal task is handled to obtain sparse square
Battle array；The sparse matrix is trained to obtain dense matrix according to stochastic gradient descent algorithm；According to the dense matrix, for the mesh
Mark task distributes resource.
Optionally, the task in the first task subset and the task in the second task subset pass through the same server
Processing.
Optionally, the task in the first task subset and the task in the second task subset pass through different servers
Processing.
Therefore, the device of the resource management of the embodiment of the present application, by detecting the resource status of goal task, according to target
The resource requirement whether resource status of task meets goal task carries out resource management, that is to say, that can dynamically adjust money
Source, to improve resource utilization.
Fig. 9 shows the structural schematic diagram of the device of the resource management of the embodiment of the present application.As shown in figure 9, the device packet
Include at least one processor 902 (such as with calculate and processing capacity general processor CPU, digital signal processor
(DSP), application-specific integrated circuit (ASIC), ready-made programmable gate array (FPGA) etc.), processor 902 is used for device Nei Gemo
Block and device are managed and dispatch.Processing module 810 in embodiment shown in Fig. 8 can be realized by processor 902.It should
Device further includes at least one transceiver 905 (receiver/transmitter), memory 906.The various components of the device pass through inside
Connecting path communicates, and transmits control and/or data-signal.
The method that above-mentioned the embodiment of the present application discloses can be applied to processor 902, or for executing in memory 906
The executable module of storage, such as computer program.Memory 906 may include high-speed random access memory (Random
Access Memory, RAM), it is also possible to further include non-labile memory (non-volatile memory), memory can
To include read-only memory and random access memory, and the signaling needed or data, program etc. are provided to processor.Storage
The a part of of device can also include non-volatile row random access memory (NVRAM).It (can be with by least one transceiver 905
Wired or wireless) realize and at least one other network element between communication connection.
In some embodiments, memory 906 stores program 9061, and processor 902 executes program 9061, for holding
The following operation of row：
The resource status of goal task is detected, which belongs to first task subset, which includes
The multiple tasks of multiple tasks, the first task subset are interrelated, and the first task subset and the second task subset belong to
One set of tasks, which includes at least one task, and at least one task in the second task subset
In each task it is mutually unrelated；
According to the resource status of the goal task, resource management is carried out.
It should be noted that the device can be specially the device 800 in embodiment shown in Fig. 8, and can be used for
Execute each step corresponding with the device and/or flow in Fig. 6 or shown in Fig. 7 embodiments of the method.
The resource status by detecting goal task is can be seen that from above technical scheme provided by the embodiments of the present application,
The resource requirement for whether meeting goal task according to the resource status of goal task carries out resource management, that is to say, that Neng Goudong
State adjustresources, to improve resource utilization.
The embodiment of the present application also provides a kind of computer storage media, which can store and be used to indicate
The program instruction of any of the above-described kind of method.
Optionally, which is specifically as follows memory 906.
It should be understood that the specific example in the application is intended merely to that those skilled in the art is helped to more fully understand the application
Embodiment, and the range of unrestricted the embodiment of the present application.
Those of ordinary skill in the art may realize that lists described in conjunction with the examples disclosed in the embodiments of the present disclosure
Member and algorithm steps can be realized with the combination of electronic hardware or computer software and electronic hardware.These functions are actually
It is implemented in hardware or software, depends on the specific application and design constraint of technical solution.Professional technician
Each specific application can be used different methods to achieve the described function, but this realization is it is not considered that exceed
Scope of the present application.
It is apparent to those skilled in the art that for convenience and simplicity of description, the system of foregoing description,
The specific work process of device and unit, can refer to corresponding processes in the foregoing method embodiment, and details are not described herein.
In several embodiments provided herein, it should be understood that disclosed systems, devices and methods, it can be with
It realizes by another way.For example, the apparatus embodiments described above are merely exemplary, for example, the unit
It divides, only a kind of division of logic function, formula that in actual implementation, there may be another division manner, such as multiple units or component
It can be combined or can be integrated into another system, or some features can be ignored or not executed.Another point, it is shown or
The mutual coupling, direct-coupling or communication connection discussed can be the indirect coupling by some interfaces, device or unit
It closes or communicates to connect, can be electrical, machinery or other forms.
The unit illustrated as separating component may or may not be physically separated, aobvious as unit
The component shown may or may not be physical unit, you can be located at a place, or may be distributed over multiple
In network element.Some or all of unit therein can be selected according to the actual needs to realize the mesh of this embodiment scheme
's.
In addition, each functional unit in each embodiment of the application can be integrated in a processing unit, it can also
It is that each unit physically exists alone, it can also be during two or more units be integrated in one unit.
It, can be with if the function is realized in the form of SFU software functional unit and when sold or used as an independent product
It is stored in a computer read/write memory medium.Based on this understanding, the technical solution of the application is substantially in other words
The part of the part that contributes to existing technology or the technical solution can be expressed in the form of software products, the meter
Calculation machine software product is stored in a storage medium, including some instructions are used so that a computer equipment (can be
People's computer, server or network equipment etc.) execute each embodiment the method for the application all or part of step.
And storage medium above-mentioned includes：USB flash disk, mobile hard disk, read-only memory (Read-Only Memory, ROM), arbitrary access are deposited
The various media that can store program code such as reservoir (Random Access Memory, RAM), magnetic disc or CD.
The above, the only specific implementation mode of the application, but the protection domain of the application is not limited thereto, it is any
Those familiar with the art can easily think of the change or the replacement in the technical scope that the application discloses, and should all contain
It covers within the protection domain of the application.Therefore, the protection domain of the application shall be subject to the protection scope of the claim.
Claims (14)
1. a kind of method of resource management, which is characterized in that including：
The resource status of goal task is detected, the goal task belongs to first task subset, and the first task subset includes
The multiple tasks of multiple tasks, the first task subset are interrelated, the first task subset and the second task subset category
In first task set, the second task subset includes at least one task, and in the second task subset it is described extremely
Each task in a few task is mutually unrelated；
According to the resource status of the goal task, resource management is carried out.
2. according to the method described in claim 1, it is characterized in that, the resource status according to the goal task, carries out
Resource management includes：
According to the resource status of the goal task, determine whether the corresponding resource of the goal task meets the goal task
Resource requirement；
When the corresponding resource of the goal task is unsatisfactory for the resource requirement of the goal task, obtained for the goal task
The resource of other tasks.
3. according to the method described in claim 2, it is characterized in that, described be unsatisfactory for institute in the corresponding resource of the goal task
When stating the resource requirement of goal task, the resource that other tasks are obtained for the goal task includes：
The corresponding resource of the second task is obtained from the second task subset, second task is the second task subset
In the resource free time task；
Using the corresponding resource of second task as the resource of the goal task.
4. according to the method described in claim 2, it is characterized in that, described be unsatisfactory for institute in the corresponding resource of the goal task
When stating the resource requirement of goal task, the resource that other tasks are obtained for the goal task includes：
It is unsatisfactory for the resource requirement of the goal task in the corresponding resource of the goal task, and in the second task subset
There is no when the second task, the corresponding resource of third task is obtained from the first task subset, wherein second task
For the task of the resource free time in the second task subset, the third task is the resource space in the first task subset
Not busy task；
Using the corresponding resource of the third task as the resource of the goal task.
5. method according to claim 1 to 4, which is characterized in that in the resource for detecting the goal task
Before state, the method further includes：
The feature description of the goal task is handled to obtain sparse matrix；
The sparse matrix is trained to obtain dense matrix according to stochastic gradient descent algorithm；
According to the dense matrix, resource is distributed for the goal task.
6. the method according to any one of claims 1 to 5, it is characterized in that, task in the first task subset
Pass through the same server process with the task in the second task subset.
7. the method according to any one of claims 1 to 5, it is characterized in that, task in the first task subset
Pass through different server process with the task in the second task subset.
8. a kind of device of resource management, which is characterized in that including：
Processing module, the resource status for detecting goal task, the goal task belong to first task subset, and described first
Task subset includes multiple tasks, and the multiple tasks of the first task subset are interrelated, the first task subset and
Two task subsets belong to first task set, and the second task subset includes at least one task, and second task is sub
Each task at least one task concentrated is mutually unrelated；
The processing module is additionally operable to the resource status according to the goal task, carries out resource management.
9. device according to claim 8, which is characterized in that the processing module is specifically used for：
According to the resource status of the goal task, determine whether the corresponding resource of the goal task meets the goal task
Resource requirement；
When the corresponding resource of the goal task is unsatisfactory for the resource requirement of the goal task, obtained for the goal task
The resource of other tasks.
10. device according to claim 9, which is characterized in that the processing module is specifically used for：
The corresponding resource of the second task is obtained from the second task subset, second task is the second task subset
In the resource free time task；
Using the corresponding resource of second task as the resource of the goal task.
11. device according to claim 9, which is characterized in that the processing module is specifically used for：
It is unsatisfactory for the resource requirement of the goal task in the corresponding resource of the goal task, and in the second task subset
There is no when the second task, the corresponding resource of third task is obtained from the first task subset, wherein second task
For the task of the resource free time in the second task subset, the third task is the resource space in the first task subset
Not busy task；
Using the corresponding resource of the third task as the resource of the goal task.
12. the device according to any one of claim 8 to 11, which is characterized in that the processing module is additionally operable to：
The feature description of the goal task is handled to obtain sparse matrix；
The sparse matrix is trained to obtain dense matrix according to stochastic gradient descent algorithm；
According to the dense matrix, resource is distributed for the goal task.
13. the device according to any one of claim 8 to 12, which is characterized in that appointing in the first task subset
Task in business and the second task subset passes through the same server process.
14. the device according to any one of claim 8 to 12, which is characterized in that appointing in the first task subset
Task in business and the second task subset passes through different server process.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710195959.9A CN108664320A (en) | 2017-03-29 | 2017-03-29 | The method and apparatus of resource management |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710195959.9A CN108664320A (en) | 2017-03-29 | 2017-03-29 | The method and apparatus of resource management |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN108664320A true CN108664320A (en) | 2018-10-16 |
Family
ID=63786510
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201710195959.9A Pending CN108664320A (en) | 2017-03-29 | 2017-03-29 | The method and apparatus of resource management |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN108664320A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111950849A (en) * | 2020-07-09 | 2020-11-17 | 华为技术有限公司 | Data processing method and data processing device |
-
2017
- 2017-03-29 CN CN201710195959.9A patent/CN108664320A/en active Pending
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111950849A (en) * | 2020-07-09 | 2020-11-17 | 华为技术有限公司 | Data processing method and data processing device |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US10275278B2 (en) | Stream processing task deployment using precompiled libraries | |
| JP6523354B2 (en) | State machine builder with improved interface and handling of state independent events | |
| CN104699541B (en) | Method, apparatus, data transfer components and the system of synchrodata | |
| US11036547B2 (en) | File operation task optimization | |
| CN105900064B (en) | The method and apparatus for dispatching data flow task | |
| TWI547817B (en) | Method, system and apparatus of planning resources for cluster computing architecture | |
| CN103309738B (en) | User job dispatching method and device | |
| CN105339897B (en) | Efficient priority perceives thread scheduling | |
| US8423950B2 (en) | Method and apparatus for optimizing performance and network traffic in distributed workflow processing | |
| CN108351805A (en) | Calculate the accelerator processing based on stream of figure | |
| CN108874538B (en) | Scheduling server, scheduling method and application method for scheduling quantum computer | |
| CN104243617B (en) | Towards the method for scheduling task and system of mixed load in a kind of isomeric group | |
| CN104050042B (en) | The resource allocation methods and device of ETL operations | |
| JP2014089744A (en) | Remote system administration using command line environment | |
| CN107430549A (en) | Based on united memory management | |
| CN106548261A (en) | A kind of Business Process Control method, device and electronic equipment | |
| JPWO2012144647A1 (en) | Virtual machine management apparatus, virtual machine management method, and program | |
| US20130254376A1 (en) | Dynamic control over tracing of messages received by a message broker | |
| CN105378668B (en) | The interruption of operating system management in multicomputer system guides | |
| CN106708917A (en) | Data processing method and device and OLAP system | |
| EP2920691B1 (en) | A network-independent programming model for online processing in distributed systems | |
| CN102868573A (en) | Method and device for Web service load cloud test | |
| CN110308984A (en) | It is a kind of for handle geographically distributed data across cluster computing system | |
| CN103780686A (en) | Method and system for customizing application approval procedure in cloud organization | |
| EP3118784A1 (en) | Method and system for enabling dynamic capacity planning |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| WD01 | Invention patent application deemed withdrawn after publication | ||
| WD01 | Invention patent application deemed withdrawn after publication |
Application publication date: 20181016 |