CN110278257A - A kind of method of mobilism configuration distributed type assemblies node label - Google Patents
A kind of method of mobilism configuration distributed type assemblies node label Download PDFInfo
- Publication number
- CN110278257A CN110278257A CN201910513146.9A CN201910513146A CN110278257A CN 110278257 A CN110278257 A CN 110278257A CN 201910513146 A CN201910513146 A CN 201910513146A CN 110278257 A CN110278257 A CN 110278257A
- Authority
- CN
- China
- Prior art keywords
- label
- node
- tags
- distributed type
- type assemblies
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
- H04L67/10—Protocols in which an application is distributed across nodes in the network
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
- H04L67/10—Protocols in which an application is distributed across nodes in the network
- H04L67/104—Peer-to-peer [P2P] networks
- H04L67/1044—Group management mechanisms
Landscapes
- Engineering & Computer Science (AREA)
- Computer Networks & Wireless Communication (AREA)
- Signal Processing (AREA)
- Physics & Mathematics (AREA)
- Computing Systems (AREA)
- Mathematical Physics (AREA)
- Theoretical Computer Science (AREA)
- Data Exchanges In Wide-Area Networks (AREA)
Abstract
The present invention provides a kind of methods of mobilism configuration distributed type assemblies node label, it is realized based on YARN NODE LABEL, it include: label setting steps, label needed for being added in the cluster for clustered node, adding tagged node is label node, and the node for adding same label belongs to same set of tags；Tag monitoring control step, the resource service condition of the node of regular monitoring difference set of tags, obtains the resource utilization of different set of tags；Label configuration step adds or removes the label of the set of tags interior joint, to realize label mobilism according to the resource utilization of the set of tags.The present invention can maximize reduction resources idle, promote resource utilization, while reducing because of multiple cluster bring maintenance works.
Description
Technical field
The present invention relates to computer network fields, are related to a kind of distributed system architecture clustered deploy(ment) side of label mobilism
Method is more specifically a kind of method of mobilism configuration distributed type assemblies node label.
Background technique
As shown in Figure 1, being the block schematic illustration based on 2.0 big data platform of HADOOP.YARN structure passes through
Resource Manager monitoring this entity of cluster resource controls entire cluster and manages application program to basic calculation resource
Distribution.Resource Manager monitors cluster resource and gives each resource part (calculating, memory, bandwidth etc.) is elaborately planned
Basic Node Manager (every node-agent of YARN).Resource Manager monitor cluster resource also with
Application Master (APP Mstr) distribute resource together, starts and monitor their base together with Node Manager
Plinth application program.In this structure, Application Master assumes responsibility for Task Tracker in pervious HADOOP1.0
Some roles, Resource Manager monitoring cluster resource assume responsibility for the role of Job Tracker.Application
Master (APP Mstr) manages each example of an application program run in YARN.Application Master
(APP Mstr) is responsible for coordinating the resource from Resource Manager monitoring cluster resource, and is supervised by Node Manager
Execution and resource depending on container use (resource allocation of CPU, memory etc.).
Scheduling of the YARN based on label, the basic thought of the strategy is: user can stamp mark for each Node Manager
Label, such as highmem, highdisk etc., using the essential attribute as NodeManager；Meanwhile user can be in scheduler
Queue several labels are set, the node resource comprising corresponding label can only be occupied to limit the queue, in this way, being submitted to some
Operation in queue can only operate on specific some nodes.By labelling, Hadoop can be divided into several subsets by user
Group, so that user can run to application program on the node for meeting certain feature.
It is currently based on the big data platform of 2.0 distributed system architecture of HADOOP, in the business for supporting multiple and different types
When calculating mode, usually there are two types of modes:
1, more cluster modes: business is placed by classification in different clusters by certain
2, single Cluster tab mode: handling all business with same set of cluster, using the Node Label function of YARN, leads to
It crosses and label corresponding with specific transactions is specified to nodes certain in cluster, Lai Shixian business monopolizes computing resource.
It influences each other between multiple business although more cluster modes can effectively be evaded:
A. O&M cost is improved, when such as changing a configuration, needs similarly to be operated on each cluster；
B. it needs in the different a large amount of data of inter-cluster synchronization, such as because business needs to be associated with multiple tables, but part table
It is distributed on different clusters, at this moment just needs in synchrodata to the same cluster；
C. because different clusters are there are segment difference when job run, lead to resources idle.If number is diurnally run in some operations, and
It leaves unused at night, which results in the wastings of resources of corresponding machine.
As shown in Fig. 2, being the NODE LABEL function of having used built in HADOOP, in the same cluster internal to different
Business task distributes independent node.
(the i.e. Data of storage that same server carries out while being usually a storage process and multiple calculation procedures
Node；Calculating is Container in Yarn, is started on demand, is closed after task).
The service of calculating has label, and storing does not have.Shared resource scheduling and storage are relative plan 1 (more cluster modes)
Sharpest edges avoid data synchronous, simplify cluster management.Label is to the process of responsible task computation (i.e. in Yarn
Container virtual portfolio)；There are exclusive and non-exclusive two kinds of selections.
Exclusive label is task to be assigned to this according to the label mapping of queue where task after task submission
In the server of set of tags, realizes and the absolute physical of computing resource is isolated.The queue will not be assigned in other set of tags, not
Being mapped in the queue of the label for task can not be also assigned in the set of tags.
Its shortcoming is that the server of the set of tags can only be idle, the wasting of resources when not having operation in some set of tags.
Task can be assigned to this according to the label mapping of queue where task after task submission by non-exclusive label
In the server of set of tags, the physical isolation to computing resource is realized.The queue will not be assigned in other set of tags, not mapped
The idling-resource of the label can be used in task in the queue (server not labelled) of label, is mapped to other labels
Queue in task can not also be assigned in the set of tags.
Its shortcoming is that when in some set of tags without operation, and when the queue that label is not configured does not have operation yet, the mark
The server of label group can only be idle.It can not be used by the task in other set of tags.When the queue that label (DEFAULT) is not configured
When job initiation (T time), the set of tags is without task, then DEFAULT operation will use the server.The label after a period of time
Queue task start (T+1 time) needs etc. to execute task until DEFAULT task terminates or be preempted interruption, subtract
The weak effect of resource isolation.
Summary of the invention
The present invention is made in view of the above problem of the existing technology, and its purpose is to provide a kind of dynamics
The method for changing configuration distributed type assemblies node label makes on the basis of YARN NODE LABEL according to the resource of cluster resource
With rate, dynamic addition and node label is removed, while meeting business practical calculatings demand, maximization reduction resources idle,
Promote resource utilization.And realize by monitoring cluster resource, according to the actual use situation of resource come dynamically add with
Label is removed, data synchronization is evaded and maximumlly reduces resources idle phenomenon, while being reduced because of multiple cluster bring O&Ms
Work.
To solve the above-mentioned problems, the present invention provides a kind of mobilism configuration distributed type assemblies node label method,
It is realized, is specifically included based on YARN NODE LABEL:
Label setting steps, label needed for adding in the cluster for clustered node, adding tagged node is label node,
The node for adding same label belongs to unified set of tags；
Tag monitoring control step, the resource service condition of the node of regular monitoring difference set of tags, obtains different set of tags
Resource utilization；
Label configuration step adds or removes the set of tags interior joint according to the resource utilization of the set of tags
Label, to realize label mobilism.
Further, in order to be physically separated to node, in label setting steps further include: node mapping step is given
The clustered node for needing to be physically separated specifies corresponding label, then label allocation and team in scheduling resource queue
The mapping relations of column.
Further, to be configured flexibly task to node, in the label configuration step, the label has label
Attribute realizes absolute physical isolation when the exclusive attribute of label, which is, is, the operation of the queue of the non-label mapping will not be by
It is assigned on the node of the label and executes；When the exclusive attribute of label is no, which preferentially executes the label mapping
The task of queue also allows to execute the task of other queues.
Further, the resource utilization of set of tags is called by using read method, from the queue array returned the result
In the resource utilization of each queue that gets.
Further, in order to solve the problems, such as that the node resource of the set of tags effectively occupies, the label configuration step
Further include: threshold decision step, by the resource utilization of the set of tags and knot removal threshold value or node add threshold value into
Row multilevel iudge, to determine whether that the label for carrying out node is deleted or added；When the resource of the set of tags uses
When rate is continuously less than knot removal threshold value, the label of a label node in the set of tags is deleted；When the set of tags
Resource utilization be continuously greater than node addition threshold value when, by it is non-setting label a node label is set.
In this step, in such a way that setting knot removal threshold value and node add threshold value, pass through the money with set of tags
Source utilization rate is compared, be known that the node of set of tags on earth how many belong to effective use, idle resource accounting is how much,
If idle node resource accounting is excessive, should not just incorporate it into set of tags.
Further, in order to clearly distinguish the node that label is set and the node that label is not set, a shifting is set
Except list, when the resource utilization of the set of tags is continuously less than knot removal threshold value, by a mark in the set of tags
The label for signing node is deleted, and the node for being deleted label moves in the removal list；When the resource utilization of the set of tags
When being continuously greater than node addition threshold value, the node removed in list is arranged the label, and by the node of the setting label
It is removed from the removal list.
Further, in order to avoid deleting all label node, so that it cannot real work, when actually specifying the mark
When the number of nodes of label is less than minimum node number, no longer the label of a label node in the set of tags is deleted, it is described most
Trifle points are to meet the quantity for the least node that appointment task must use.
Further, it when node all configured labels all in the distributed type assemblies, is marked without node for setting
Then no longer the label is arranged in one node of non-setting label by label.
Further, a potential node listing is set, all nodes that can be set to the label are contained.
Further, in order to effectively obtain the resource utilization of set of tags, and it is unlikely to lead to data redundancy, in the mark
It signs in monitoring step, supervision interval duration is defaulted as 10s.
Further, in order to effectively obtain the resource utilization of set of tags, and it is unlikely to lead to data redundancy, feature exists
In in the tag monitoring control step, the maximum read-around ratio of monitoring is defaulted as 6 times.
Detailed description of the invention
Fig. 1 is the schematic diagram of the YARN structure based on HADOOP2.0 big data platform in the prior art.
Fig. 2 is that list Hadoop cluster utilizes label function to distribute task schematic diagram in single cluster in the prior art.
Fig. 3 is the flow chart of the dynamic labels clustered deploy(ment) method of embodiment one.
Fig. 4 is the mapping mapping relations schematic diagram of label and queue in the label setting steps of embodiment one.
Fig. 5 is the node pool schematic diagram of the label configuration step of embodiment one.
Fig. 6 is the removal list schematic diagram of the label configuration step of embodiment one.
Fig. 7 is that the label of embodiment one deletes the flow chart of step.
Fig. 8 is the flow chart of the label addition step of embodiment one.
Fig. 9 is the schematic diagram of another embodiment of the invention.
Specific embodiment
Exemplary embodiments of the present invention are illustrated below in conjunction with attached drawing, it should be understood that provide these embodiment party
Formula is used for the purpose of making those skilled in the art can better understand that realizing the present invention in turn, and not limit in any way
The scope of the present invention.
[embodiment one: dynamic labels clustered deploy(ment) method]
Fig. 3 is the flow chart of dynamic labels clustered deploy(ment) method, as shown in Figure 3, comprising:
Label setting steps S1, YARN starter node label function, label needed for adding in the cluster, establishes node pool；
Containment mapping node mapping step S11 is gone back in label starting step S1, the cluster section being physically separated to needs
Point specify corresponding label, then in the resource queue of YARN label allocation and queue mapping mapping relations.
It as indicated at 4, is the mapping relations schematic diagram of label and queue in label starting step of the invention, in Fig. 4, team
Mapping relations are established with corresponding label a (label_a) comprising two nodes (node_1/node_2) in column a (queue_a),
And two nodes (node_3/node_4) for including in queue b (queue_b) also establish with corresponding label b (label_b)
Mapping relations.
Here, as shown in figure 5, being the node pool schematic diagram of embodiment, node pool of the label mobilism cluster from tape label
Middle selection node carries out the removal of the label or adds again.In Fig. 5, a shared n node in label node pond, for
Having beaten the node of label (label_a), there are two (node_1/nod e_2).
Here, removal list is established for being removed the node of label in node pool, as shown in fig. 6, being the present embodiment
Label configuration step removal list schematic diagram, in Fig. 6, due to node_1/nod e_2 be present in remove list in, institute
To indicate that the label label_a of node_1/node_2 is removed.
Here, the label for node addition has attribute, when wherein the attribute exclusive of label is true (default),
Realize absolute physical isolation, the operation of the non-tag queue is not assigned on corresponding node and executes；When for false,
Node preferentially executes the task of corresponding tag queue, also allows to execute the operation of other queues.
Label configuration step S2, according to the resource utilization of clustered node, addition and removal clustered node such as node_1/
The label of node_2/node_3/node_4, to realize label mobilism.
It further include threshold decision step S21 in label configuration step S2, label deletes step S22, label adds step
S23。
Wherein, the resource utilization of clustered node and knot removal threshold value or node are added threshold by threshold decision step S21
Value is compared judgement, to determine whether that carrying out following labels deletes step S22 or label addition step S23.
Here, the resource utilization that threshold decision step S21 needs to obtain queue first is carried out, here for example using GET
(reading) method call YARN RESTFUL interface can get each from queue s (queue) array returned the result
The usedCapacity (utilization rate) of queue, the as resource utilization of the queue, RESTFUL interface are as follows: http://RM:
(RM is node where Resource Manger, PORT Resource to PORT/ws/v1/cluster/sch eduler
The port numbers of Manger).
Here, knot removal threshold value refers to when the resource utilization of a certain tag queue is less than the threshold value, from the label
Node pool in select a node, the label of this node is removed；Node addition threshold value refers to the money when a certain tag queue
When source utilization rate is greater than the threshold value, a node being removed is selected from the node pool of the label, node addition thus should
Label.
The label for being illustrated in figure 7 embodiment deletes the flow chart of step S22, in Fig. 6, when the resource of tag queue makes
Be less than knot removal threshold value with the continuous n times of rate (N be specified maximum read-around ratio), selected from the node pool of configuration one not
In the node for removing list, the label of the node is removed, the computing resource of the node is discharged, this node is then added to shifting
Except in list.The number of nodes of the label is actually specified to be less than minimum node number when node pool quantity subtracts removal list quantity
When, it no longer executes and removes judgement and operation.
Here, minimum node number refers to that queue executes the number of nodes that task at least needs, when the actual number of nodes of label is small
When being equal to minimum node number, no longer execution knot removal movement.If label_a has 4 nodes to use, if removing three
After node only surplus next node when, then will not be further continued for label delete step S22 deletion of node label.
It is illustrated in figure 8 the flow chart of the label addition step S23 of the present embodiment, in fig. 8, when the resource of tag queue
When the continuous n times of utilization rate (N is specified maximum read-around ratio) are greater than node addition threshold value, a section is selected in list from removing
Point, node adds the label so that the computing resource of this node is isolated thus, then removes the node from removal list.Work as shifting
Except when list is empty, that is, node pool in all nodes all the configured label when, no longer execute addition judgement and operation.
Tag monitoring control step S3 is used based on the resource service condition of YARN Restful interface regular monitoring tag queue
In determine whether carry out label configuration step S2.
Here, a length of default 10s when supervision interval is arranged in tag monitoring control step S3, and interval duration is submitted according to production cluster
The frequency of task substantially calculates, if time setting is too long, a job waiting time that cannot get resource can be allowed too long, if
Time setting is too short, will increase load.
Here, tag monitoring control step S3 is arranged maximum read-around ratio N and is defaulted as 6 times, and maximum read-around ratio collects according to production
Group submits the frequency of task substantially to calculate, be arranged it is too small will increase load, setting is excessive, can allow the job that cannot get resource
Waiting time is too long.
[embodiment two]
It is illustrated in figure 9 the schematic diagram of one embodiment of the invention, as shown in figure 9, supporting in same set of cluster multiple
Different demands of business, including data warehouse ETL operation, machine learning, JAVA class engine etc., resource of the different work to cluster
It is required that also far from each other.Classified for all kinds of operations (job) with queue, is submitted on corresponding operation (job), then match to queue
Label is set, stamps this label to specific node, such operation can obtain corresponding node resource.
The present embodiment can evade the data between more clusters and synchronize, such as the data requirements that current data is excavated, and need daily
More new data about 300G, synchronizing speed 30M/s need 3h.
Meanwhile the present embodiment can promote resource utilization, such as a business needs to apply that 4 nodes carry out every number of days
According to modeling, the modeling time is 2h, and after the completion of modeling, node is the free time, average daily cpu load only 5%；After enabling dynamic labels,
Label is removed after the completion of modeling and handles other types task, and average daily cpu load is promoted to 48%.
The HADOOP platform dynamic labels clustered deploy(ment) method of label mobilism of the invention can also for example have with
Lower variation:
Variation 1 can be consistent with configuration node maximum quantity and minimum number for the stream process class operation of no interruption
Label guarantees label without removal.
Variation 2, it is higher for computational valid time, and the batch processing job that runing time section is fixed, minimum number can be configured
Amount < maximum quantity discharges computing resource after the completion of same day batch processing.
Variation 3, the operation of less demanding for computational valid time can uniformly be placed in the node of no label and run.
Above, the present general inventive concept is described by way of flow chart, but it should be noted that in flow chart
The sequence of each step be not fixed, purport spirit of the invention can not departed from according to specific need
In the range of, the sequence of each step is suitably changed, and the process after change is still fallen in protection scope of the present invention.
Although each conception of species has already been described in detail, it will be appreciated by a person skilled in the art that for the various of those concepts
What modifications and substitutions can be achieved under the spirit disclosed by the invention integrally instructed.
Although in addition, describing the present invention under the background of system module and being illustrated in the form of functional module
It is bright, but it is to be understood that, unless otherwise indicated, one or more of the function and/or feature can be collected
At in single physical device and/or software module or one or more functions and/or feature can be filled in individual physics
Set or software module in be implemented.It will also be appreciated that the practical realization in relation to each module is discussed in detail for understanding
The present invention is unnecessary.More specifically, it is contemplated that the attribute of various functional modules, function in system disclosed herein
In the case where internal relations, it will understand that the practical realization of the module in the routine techniques of engineer.Therefore, this field skill
Art personnel can realize this illustrated in detail in the claims hair with ordinary skill in the case where being not necessarily to undue experimentation
It is bright.It will also be appreciated that disclosed specific concept is merely illustrative, it is not intended to limit the scope of the present invention, this
The range of invention is determined by the full scope of the appended claims and its equivalent program.
Claims (11)
1. a kind of method of mobilism configuration distributed type assemblies node label, comprising: label setting steps are in the cluster cluster
Label needed for node adds, adding tagged node is label node, and the node for adding same label belongs to same set of tags；
It is characterized by further comprising:
Tag monitoring control step, the resource service condition of regular monitoring difference label group node, the resource for obtaining different set of tags make
With rate；
Label configuration step adds or removes the label of the set of tags interior joint according to the resource utilization of the set of tags,
To realize label mobilism.
2. the method for mobilism configuration distributed type assemblies node label according to claim 1, which is characterized in that the mark
Sign setting steps further include:
Node maps mapping step, to needing the clustered node being physically separated to specify corresponding label, is then adjusting
Spend the mapping mapping relations of label allocation and queue in resource queue.
3. the method for mobilism configuration distributed type assemblies node label according to claim 2, which is characterized in that described
In label configuration step, the label have tag attributes, when the exclusive attribute of label be when, realize absolute physics every
From the operation of the queue of the non-label mapping is not assigned on the node of the label and executes；When the exclusive attribute of label is
When no, which preferentially executes the task of the label mapping queue, also allows to execute the task of other queues.
4. the method for mobilism configuration distributed type assemblies node label according to claim 2, which is characterized in that set of tags
Resource utilization called by using read method, the money of each queue got from the queue array returned the result
Source utilization rate.
5. the method for mobilism configuration distributed type assemblies node label according to claim 4, which is characterized in that the mark
Signing configuration step further includes,
The resource utilization of the set of tags is compared by threshold decision step with knot removal threshold value or node addition threshold value
Judgement, to determine whether that carrying out node label deletes step or node label addition step；
The node label deletes step, when the continuous n times of the resource utilization of the set of tags are less than knot removal threshold value,
The label of a label node in the set of tags is deleted；
The node label addition step is, when the continuous n times of the resource utilization of the set of tags are greater than node addition threshold value,
The label is arranged in one node of non-setting label；
The N is the natural number greater than zero.
6. the method for mobilism configuration distributed type assemblies node label according to claim 5, it is characterised in that: setting one
A removal list is deleted in step in the node label, and the node for being deleted label moves in the removal list；
In node label addition step, the node of final setting label is removed from the removal list.
7. the method for mobilism configuration distributed type assemblies node label according to claim 5, which is characterized in that work as reality
When the number of nodes of the label being specified to be less than minimum node number, no longer the label of a label node in the set of tags is deleted
It removes.
8. the method for mobilism configuration distributed type assemblies node label according to claim 5, which is characterized in that when described
In distributed type assemblies all nodes all configured label when, no longer by it is non-setting label a node label is set.
9. the method for mobilism configuration distributed type assemblies node label according to claim 1, it is characterised in that: setting one
A potential node listing contains all nodes that can be set to the label.
10. the method for mobilism configuration distributed type assemblies node label according to claim 1, which is characterized in that in institute
It states in tag monitoring control step, supervision interval duration is defaulted as 10s.
11. the method for mobilism configuration distributed type assemblies node label according to claim 1, which is characterized in that in institute
It states in tag monitoring control step, monitors maximum read-around ratio and be defaulted as 6 times.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910513146.9A CN110278257A (en) | 2019-06-13 | 2019-06-13 | A kind of method of mobilism configuration distributed type assemblies node label |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910513146.9A CN110278257A (en) | 2019-06-13 | 2019-06-13 | A kind of method of mobilism configuration distributed type assemblies node label |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN110278257A true CN110278257A (en) | 2019-09-24 |
Family
ID=67960762
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910513146.9A Pending CN110278257A (en) | 2019-06-13 | 2019-06-13 | A kind of method of mobilism configuration distributed type assemblies node label |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110278257A (en) |
Cited By (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110781000A (en) * | 2019-10-17 | 2020-02-11 | 中国建设银行股份有限公司 | Multi-application resource allocation method and device |
| CN110784347A (en) * | 2019-10-18 | 2020-02-11 | 北京浪潮数据技术有限公司 | Node management method, system, equipment and storage medium for container cluster |
| CN111124765A (en) * | 2019-12-06 | 2020-05-08 | 中盈优创资讯科技有限公司 | Big data cluster task scheduling method and system based on node labels |
| CN113344311A (en) * | 2020-03-03 | 2021-09-03 | 北京国双科技有限公司 | Task execution method and device, storage medium, processor and electronic equipment |
| CN113722054A (en) * | 2020-05-25 | 2021-11-30 | 中移(苏州)软件技术有限公司 | Resource management method and device and computer storage medium |
Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20070168919A1 (en) * | 2005-12-01 | 2007-07-19 | Cassatt Corporation | Automated deployment and configuration of applications in an autonomically controlled distributed computing system |
| CN105159775A (en) * | 2015-08-05 | 2015-12-16 | 浪潮(北京)电子信息产业有限公司 | Load balancer based management system and management method for cloud computing data center |
| CN106708622A (en) * | 2016-07-18 | 2017-05-24 | 腾讯科技（深圳）有限公司 | Cluster resource processing method and system, and resource processing cluster |
| CN107038069A (en) * | 2017-03-24 | 2017-08-11 | 北京工业大学 | Dynamic labels match DLMS dispatching methods under Hadoop platform |
| CN109343965A (en) * | 2018-10-31 | 2019-02-15 | 北京金山云网络技术有限公司 | Resource adjusting method, device, cloud platform and server |
| CN109408341A (en) * | 2018-11-13 | 2019-03-01 | 郑州云海信息技术有限公司 | Distributed memory system method for monitoring performance, device, equipment and readable storage medium storing program for executing |
| CN109766175A (en) * | 2018-12-28 | 2019-05-17 | 深圳晶泰科技有限公司 | Resource elastic telescopic system and its dispatching method towards high-performance calculation on cloud |
-
2019
- 2019-06-13 CN CN201910513146.9A patent/CN110278257A/en active Pending
Patent Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20070168919A1 (en) * | 2005-12-01 | 2007-07-19 | Cassatt Corporation | Automated deployment and configuration of applications in an autonomically controlled distributed computing system |
| CN105159775A (en) * | 2015-08-05 | 2015-12-16 | 浪潮(北京)电子信息产业有限公司 | Load balancer based management system and management method for cloud computing data center |
| CN106708622A (en) * | 2016-07-18 | 2017-05-24 | 腾讯科技（深圳）有限公司 | Cluster resource processing method and system, and resource processing cluster |
| CN107038069A (en) * | 2017-03-24 | 2017-08-11 | 北京工业大学 | Dynamic labels match DLMS dispatching methods under Hadoop platform |
| CN109343965A (en) * | 2018-10-31 | 2019-02-15 | 北京金山云网络技术有限公司 | Resource adjusting method, device, cloud platform and server |
| CN109408341A (en) * | 2018-11-13 | 2019-03-01 | 郑州云海信息技术有限公司 | Distributed memory system method for monitoring performance, device, equipment and readable storage medium storing program for executing |
| CN109766175A (en) * | 2018-12-28 | 2019-05-17 | 深圳晶泰科技有限公司 | Resource elastic telescopic system and its dispatching method towards high-performance calculation on cloud |
Non-Patent Citations (2)
| Title |
|---|
| WEIXIN_30384217: "yarn nodes label(yarn 划分子集群)", 《HTTPS://BLOG.CSDN.NET/WEIXIN_30384217/ARTICLE/DETAILS/97300059》 * |
| 陈良章: "任务感知YARN资源调度器的研究与实现", 《中国优秀硕士学位论文全文数据库信息科技辑》 * |
Cited By (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110781000A (en) * | 2019-10-17 | 2020-02-11 | 中国建设银行股份有限公司 | Multi-application resource allocation method and device |
| CN110781000B (en) * | 2019-10-17 | 2022-07-05 | 中国建设银行股份有限公司 | Multi-application resource allocation method and device |
| CN110784347A (en) * | 2019-10-18 | 2020-02-11 | 北京浪潮数据技术有限公司 | Node management method, system, equipment and storage medium for container cluster |
| CN111124765A (en) * | 2019-12-06 | 2020-05-08 | 中盈优创资讯科技有限公司 | Big data cluster task scheduling method and system based on node labels |
| CN113344311A (en) * | 2020-03-03 | 2021-09-03 | 北京国双科技有限公司 | Task execution method and device, storage medium, processor and electronic equipment |
| CN113722054A (en) * | 2020-05-25 | 2021-11-30 | 中移(苏州)软件技术有限公司 | Resource management method and device and computer storage medium |
| CN113722054B (en) * | 2020-05-25 | 2024-04-09 | 中移(苏州)软件技术有限公司 | Resource management method, device and computer storage medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110278257A (en) | A kind of method of mobilism configuration distributed type assemblies node label | |
| CN104317658B (en) | A kind of loaded self-adaptive method for scheduling task based on MapReduce | |
| CN103092698B (en) | Cloud computing application automatic deployment system and method | |
| CN103810023B (en) | The intelligent deployment method of Distributed Application and system in a kind of cloud platform | |
| CN108287756A (en) | A kind of method and device of processing task | |
| CN103473031B (en) | Collaborative concurrent type frog messaging bus, driving member composition model and component method for splitting | |
| CN109271450A (en) | Database synchronization method, device, server and storage medium | |
| CN105700948A (en) | Method and device for scheduling calculation task in cluster | |
| CN104679595B (en) | A kind of application oriented IaaS layers of dynamic resource allocation method | |
| CN105049218A (en) | PhiCloud cloud charging method and system | |
| CN114996018A (en) | Resource scheduling method, node, system, device and medium for heterogeneous computing | |
| CN102981893A (en) | Method and system for dispatching virtual machine | |
| CN105677763A (en) | Image quality evaluating system based on Hadoop | |
| CN107203255A (en) | Power-economizing method and device are migrated in a kind of network function virtualized environment | |
| CN116991558B (en) | Computing power resource scheduling method, multi-architecture cluster, device and storage medium | |
| CN104348852B (en) | A kind of method, apparatus and system for realizing telecommunication capability mass-sending | |
| CN110704465A (en) | Method, device and storage medium for processing service work order table | |
| CN108459905A (en) | Resource pool method for planning capacity and server | |
| CN114138488A (en) | Cloud-native implementation method and system based on elastic high-performance computing | |
| CN110007905A (en) | A kind of generation method and system of the software development scheme based on big data | |
| CN108595259A (en) | A kind of internal memory pool managing method based on global administration | |
| CN109542863A (en) | Mine machine system is dug in artificial intelligence storage | |
| CN116402318B (en) | Multi-stage computing power resource distribution method and device for power distribution network and network architecture | |
| CN109788013A (en) | Operation resource allocation methods, device and equipment in distributed system | |
| CN109213566A (en) | Virtual machine migration method, device and equipment |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20190924 |