CN111522637A - 一种基于成本效益的storm任务调度方法 - Google Patents
一种基于成本效益的storm任务调度方法 Download PDFInfo
- Publication number
- CN111522637A CN111522637A CN202010290723.5A CN202010290723A CN111522637A CN 111522637 A CN111522637 A CN 111522637A CN 202010290723 A CN202010290723 A CN 202010290723A CN 111522637 A CN111522637 A CN 111522637A
- Authority
- CN
- China
- Prior art keywords
- cost
- task
- storm
- resource
- database
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 230000008901 benefit Effects 0.000 title claims abstract description 34
- 238000000034 method Methods 0.000 title claims abstract description 34
- 238000004891 communication Methods 0.000 claims abstract description 42
- 238000012544 monitoring process Methods 0.000 claims abstract description 17
- 239000000523 sample Substances 0.000 claims abstract description 13
- 238000013515 script Methods 0.000 claims abstract description 13
- 238000005111 flow chemistry technique Methods 0.000 claims abstract description 10
- 238000012545 processing Methods 0.000 claims description 25
- 238000005265 energy consumption Methods 0.000 claims description 13
- 239000000126 substance Substances 0.000 claims description 5
- 238000004422 calculation algorithm Methods 0.000 description 5
- 238000004364 calculation method Methods 0.000 description 5
- 238000011156 evaluation Methods 0.000 description 4
- 229940126655 NDI-034858 Drugs 0.000 description 3
- 241000290929 Nimbus Species 0.000 description 3
- 238000005516 engineering process Methods 0.000 description 3
- 238000004458 analytical method Methods 0.000 description 2
- 238000010586 diagram Methods 0.000 description 2
- 238000013468 resource allocation Methods 0.000 description 2
- 230000003044 adaptive effect Effects 0.000 description 1
- 230000004075 alteration Effects 0.000 description 1
- 238000013459 approach Methods 0.000 description 1
- 230000001174 ascending effect Effects 0.000 description 1
- 238000007405 data analysis Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 238000009434 installation Methods 0.000 description 1
- 230000010354 integration Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000012946 outsourcing Methods 0.000 description 1
- 230000003068 static effect Effects 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- 238000012360 testing method Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/46—Multiprogramming arrangements
- G06F9/48—Program initiating; Program switching, e.g. by interrupt
- G06F9/4806—Task transfer initiation or dispatching
- G06F9/4843—Task transfer initiation or dispatching by program, e.g. task dispatcher, supervisor, operating system
- G06F9/4881—Scheduling strategies for dispatcher, e.g. round robin, multi-level priority queues
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/44—Arrangements for executing specific programs
- G06F9/455—Emulation; Interpretation; Software simulation, e.g. virtualisation or emulation of application or operating system execution engines
- G06F9/45533—Hypervisors; Virtual machine monitors
- G06F9/45558—Hypervisor-specific management and integration aspects
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/46—Multiprogramming arrangements
- G06F9/50—Allocation of resources, e.g. of the central processing unit [CPU]
- G06F9/5005—Allocation of resources, e.g. of the central processing unit [CPU] to service a request
- G06F9/5027—Allocation of resources, e.g. of the central processing unit [CPU] to service a request the resource being a machine, e.g. CPUs, Servers, Terminals
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/44—Arrangements for executing specific programs
- G06F9/455—Emulation; Interpretation; Software simulation, e.g. virtualisation or emulation of application or operating system execution engines
- G06F9/45533—Hypervisors; Virtual machine monitors
- G06F9/45558—Hypervisor-specific management and integration aspects
- G06F2009/45583—Memory management, e.g. access or allocation
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
- Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management
Abstract
Description
Claims (7)
Priority Applications (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN202010290723.5A CN111522637B (zh) | 2020-04-14 | 2020-04-14 | 一种基于成本效益的storm任务调度方法 |
Applications Claiming Priority (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN202010290723.5A CN111522637B (zh) | 2020-04-14 | 2020-04-14 | 一种基于成本效益的storm任务调度方法 |
Publications (2)
Publication Number | Publication Date |
---|---|
CN111522637A true CN111522637A (zh) | 2020-08-11 |
CN111522637B CN111522637B (zh) | 2024-03-29 |
Family
ID=71902845
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
CN202010290723.5A Active CN111522637B (zh) | 2020-04-14 | 2020-04-14 | 一种基于成本效益的storm任务调度方法 |
Country Status (1)
Country | Link |
---|---|
CN (1) | CN111522637B (zh) |
Cited By (1)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN112181620A (zh) * | 2020-09-27 | 2021-01-05 | 郑州轻工业大学 | 云环境下虚拟机服务能力感知的大数据工作流调度方法 |
Citations (11)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN105574082A (zh) * | 2015-12-08 | 2016-05-11 | 曙光信息产业(北京)有限公司 | 基于Storm的流处理方法及系统 |
CN105824618A (zh) * | 2016-03-10 | 2016-08-03 | 浪潮软件集团有限公司 | 一种关于Storm使用的实时消息处理方法 |
CN106021411A (zh) * | 2016-05-13 | 2016-10-12 | 大连理工大学 | 一种具有集群自适应性的Storm任务部署与配置平台 |
CN106502772A (zh) * | 2016-10-09 | 2017-03-15 | 国网浙江省电力公司信息通信分公司 | 基于分布式离线技术的电量数据批量高速处理方法及系统 |
WO2017125146A1 (en) * | 2016-01-20 | 2017-07-27 | Nec Europe Ltd. | Method and system for supporting stream processing framework functionality |
CN107038071A (zh) * | 2017-05-26 | 2017-08-11 | 重庆邮电大学 | 一种基于数据流预测的Storm任务伸缩调度算法 |
WO2017148503A1 (en) * | 2016-02-29 | 2017-09-08 | Nec Europe Ltd. | Method and stream processing system for managing data stream processing tasks of a predefined application topology |
CN108241530A (zh) * | 2016-12-23 | 2018-07-03 | 西北大学 | 一种基于Storm的流式计算二分图任务调度方法 |
CN108469988A (zh) * | 2018-02-28 | 2018-08-31 | 西北大学 | 一种基于异构Hadoop集群的任务调度方法 |
US10095547B1 (en) * | 2015-03-13 | 2018-10-09 | Twitter, Inc. | Stream processing at scale |
CN109614210A (zh) * | 2018-11-28 | 2019-04-12 | 重庆邮电大学 | 基于能耗感知的Storm大数据节能调度方法 |
-
2020
- 2020-04-14 CN CN202010290723.5A patent/CN111522637B/zh active Active
Patent Citations (11)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US10095547B1 (en) * | 2015-03-13 | 2018-10-09 | Twitter, Inc. | Stream processing at scale |
CN105574082A (zh) * | 2015-12-08 | 2016-05-11 | 曙光信息产业(北京)有限公司 | 基于Storm的流处理方法及系统 |
WO2017125146A1 (en) * | 2016-01-20 | 2017-07-27 | Nec Europe Ltd. | Method and system for supporting stream processing framework functionality |
WO2017148503A1 (en) * | 2016-02-29 | 2017-09-08 | Nec Europe Ltd. | Method and stream processing system for managing data stream processing tasks of a predefined application topology |
CN105824618A (zh) * | 2016-03-10 | 2016-08-03 | 浪潮软件集团有限公司 | 一种关于Storm使用的实时消息处理方法 |
CN106021411A (zh) * | 2016-05-13 | 2016-10-12 | 大连理工大学 | 一种具有集群自适应性的Storm任务部署与配置平台 |
CN106502772A (zh) * | 2016-10-09 | 2017-03-15 | 国网浙江省电力公司信息通信分公司 | 基于分布式离线技术的电量数据批量高速处理方法及系统 |
CN108241530A (zh) * | 2016-12-23 | 2018-07-03 | 西北大学 | 一种基于Storm的流式计算二分图任务调度方法 |
CN107038071A (zh) * | 2017-05-26 | 2017-08-11 | 重庆邮电大学 | 一种基于数据流预测的Storm任务伸缩调度算法 |
CN108469988A (zh) * | 2018-02-28 | 2018-08-31 | 西北大学 | 一种基于异构Hadoop集群的任务调度方法 |
CN109614210A (zh) * | 2018-11-28 | 2019-04-12 | 重庆邮电大学 | 基于能耗感知的Storm大数据节能调度方法 |
Non-Patent Citations (5)
Title |
---|
张楠等: "Storm流处理平台中负载均衡机制的实现", 《计算机与现代化》 * |
戴红希: "基于成本效益的Storm任务调度策略研究及应用", 《中国优秀硕士学位论文全文数据库(电子期刊)》 * |
杨亚南等: "跨地域分布数据中心高成本效益的任务调度", 《应用科学学报》 * |
蔡宇等: "实时流处理系统Storm的调度优化综述", 《计算机应用研究》 * |
陈庚: "基于成本效益和QoS状态感知的雾计算任务调度算法研究", 《中国优秀硕士学位论文全文数据库(电子期刊)》 * |
Cited By (2)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN112181620A (zh) * | 2020-09-27 | 2021-01-05 | 郑州轻工业大学 | 云环境下虚拟机服务能力感知的大数据工作流调度方法 |
CN112181620B (zh) * | 2020-09-27 | 2022-09-20 | 郑州轻工业大学 | 云环境下虚拟机服务能力感知的大数据工作流调度方法 |
Also Published As
Publication number | Publication date |
---|---|
CN111522637B (zh) | 2024-03-29 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
Singh et al. | Task scheduling in cloud computing | |
Salot | A survey of various scheduling algorithm in cloud computing environment | |
CN107038069B (zh) | Hadoop平台下动态标签匹配DLMS调度方法 | |
CN107222531B (zh) | 一种容器云资源调度方法 | |
Mittal et al. | An optimized task scheduling algorithm in cloud computing | |
Hwang et al. | Minimizing cost of virtual machines for deadline-constrained mapreduce applications in the cloud | |
CN111367630A (zh) | 一种基于云计算的多用户多优先级的分布式协同处理方法 | |
Sengupta et al. | Scheduling multi-tenant cloud workloads on accelerator-based systems | |
CN103701886A (zh) | 一种云计算环境下的服务及资源分层调度方法 | |
Chen et al. | Deep learning research and development platform: Characterizing and scheduling with qos guarantees on gpu clusters | |
CN106201681B (zh) | Hadoop平台下基于预释放资源列表的任务调度方法 | |
Kaur et al. | Meta-heuristic based framework for workflow load balancing in cloud environment | |
Shyam et al. | Resource allocation in cloud computing using agents | |
Maiti et al. | Internet of Things applications placement to minimize latency in multi-tier fog computing framework | |
Shen et al. | Goodbye to fixed bandwidth reservation: Job scheduling with elastic bandwidth reservation in clouds | |
Komarasamy et al. | A novel approach for Dynamic Load Balancing with effective Bin Packing and VM Reconfiguration in cloud | |
Walraven et al. | Adaptive performance isolation middleware for multi-tenant saas | |
Singh et al. | A comparative study of various scheduling algorithms in cloud computing | |
Ma et al. | Maximizing container-based network isolation in parallel computing clusters | |
CN111522637B (zh) | 一种基于成本效益的storm任务调度方法 | |
Markthub et al. | Using rcuda to reduce gpu resource-assignment fragmentation caused by job scheduler | |
Grigoras et al. | Elastic management of reconfigurable accelerators | |
CN115098240B (zh) | 一种多处理器应用调度方法和系统及存储介质 | |
CN110955526A (zh) | 一种用于在分布式异构环境下实现多gpu调度的方法和系统 | |
Gu et al. | Elastic model aggregation with parameter service |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
PB01 | Publication | ||
PB01 | Publication | ||
SE01 | Entry into force of request for substantive examination | ||
SE01 | Entry into force of request for substantive examination | ||
TA01 | Transfer of patent application right | ||
TA01 | Transfer of patent application right |
Effective date of registration: 20240123 Address after: 518000 B24 Huayuan street b23b, 428 Huayuan street, Longfa Road, Longyuan community, Longhua street, Longhua District, Shenzhen City, Guangdong Province Applicant after: Shenzhen Morning Intellectual Property Operations Co.,Ltd. Country or region after: China Address before: 710000 No. B49, Xinda Zhongchuang space, 26th Street, block C, No. 2 Trading Plaza, South China City, international port district, Xi'an, Shaanxi Province Applicant before: Xi'an Huaqi Zhongxin Technology Development Co.,Ltd. Country or region before: China Effective date of registration: 20240123 Address after: 710000 No. B49, Xinda Zhongchuang space, 26th Street, block C, No. 2 Trading Plaza, South China City, international port district, Xi'an, Shaanxi Province Applicant after: Xi'an Huaqi Zhongxin Technology Development Co.,Ltd. Country or region after: China Address before: 400065 Chongwen Road, Nanshan Street, Nanan District, Chongqing Applicant before: CHONGQING University OF POSTS AND TELECOMMUNICATIONS Country or region before: China |
|
GR01 | Patent grant | ||
GR01 | Patent grant |