WO2020147330A1 - Procédé et système de traitement de flux de données - Google Patents

Procédé et système de traitement de flux de données Download PDF

Info

Publication number
WO2020147330A1
WO2020147330A1 PCT/CN2019/106779 CN2019106779W WO2020147330A1 WO 2020147330 A1 WO2020147330 A1 WO 2020147330A1 CN 2019106779 W CN2019106779 W CN 2019106779W WO 2020147330 A1 WO2020147330 A1 WO 2020147330A1
Authority
WO
WIPO (PCT)
Prior art keywords
task
master node
node
worker
master
Prior art date
Application number
PCT/CN2019/106779
Other languages
English (en)
Chinese (zh)
Inventor
郭业俊
李�浩
王志强
孙迁
Original Assignee
苏宁云计算有限公司
苏宁易购集团股份有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 苏宁云计算有限公司, 苏宁易购集团股份有限公司 filed Critical 苏宁云计算有限公司
Priority to CA3168286A priority Critical patent/CA3168286A1/fr
Publication of WO2020147330A1 publication Critical patent/WO2020147330A1/fr

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00—Arrangements for program control, e.g. control units
    • G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/46—Multiprogramming arrangements
    • G06F9/50—Allocation of resources, e.g. of the central processing unit [CPU]

Definitions

  • the present invention relates to the field of big data processing, in particular to a data stream processing method and system.
  • the Flume service is usually started in each node of the cluster to realize the dump of the data of the data source source to the sink.
  • the inventor found that: due to the normal failure of the node equipment (such as deadlock, abnormal consumption, etc.), the system has no perception and requires manual processing, which affects the timeliness of failure processing; in addition, due to the failure of the same cluster
  • Each node uses the same configuration, but the amount of business data of each node is uneven, which easily leads to a large idle proportion of the Flume collection thread.
  • restarting is required The entire cluster, thereby affecting the normal execution of other businesses in the same cluster.
  • the present invention aims to solve at least one of the technical problems existing in the prior art or related technologies. To this end, the present invention provides a data stream processing method and system.
  • a data stream processing method includes:
  • the Zookeeper cluster determines one of several Master nodes as the master node
  • the master node provides an external interface to receive online requests for services and allocate tasks for the services;
  • the configuration information including information indicating the worker node that is scheduled to execute the task;
  • the Worker node If the Worker node detects that there is a task scheduled for itself in the ZooKeeper cluster, it starts the Flume service for execution.
  • the determination of one of the several Master nodes by the Zookeeper cluster as the master node includes:
  • the ZooKeeper cluster receives the master node election request initiated by the Master node based on a preset trigger event, and makes the Master node the master node after the election is successful, wherein the preset trigger event is one of the following events:
  • the Master node is started;
  • the current Master node as the master node fails.
  • the generating the configuration information of the task according to the current status information reported by each of the multiple Worker nodes includes:
  • the method further includes:
  • the master node adjusts the configuration information of the task according to the host operating status information and task execution status information reported by the multiple worker nodes;
  • the adjusted configuration information of the task indicates that the task in the idle state is to be scaled down, and the task in the stacked state is to be expanded;
  • the method further includes:
  • the master node receives the offline request for the service through the external interface
  • the offline information of the service and the offline information of the task assigned to the service are written into the ZooKeeper cluster, so that the Worker node that executes the task stops the Flume service.
  • a data stream processing system includes a Zookeeper cluster, a number of Master nodes and a number of Worker nodes, where:
  • the Zookeeper cluster is used to determine one of several Master nodes as the master node;
  • the master node is used to provide an external interface to receive online requests for services, and to allocate tasks for the services;
  • the master node is further configured to generate and write configuration information of the task in the ZooKeeper cluster according to the current status information reported by each of the multiple worker nodes.
  • the configuration information includes instructions to be scheduled to execute the task. Worker node information;
  • the Worker node is configured to start the Flume service for execution if it monitors that there is a task scheduled for itself in the ZooKeeper cluster.
  • ZooKeeper cluster is specifically used for:
  • the preset trigger event is one of the following events:
  • the Master node is started;
  • the current Master node as the master node fails.
  • master node is specifically used for:
  • master node is specifically used for:
  • the master node adjusts the configuration information of the task according to the host operating status information and task execution status information reported by the multiple worker nodes;
  • the adjusted configuration information of the task indicates that the task in the idle state is to be scaled down, and the task in the stacked state is to be expanded;
  • master node is specifically used for:
  • the offline information of the service and the offline information of the task assigned to the service are written into the ZooKeeper cluster, so that the Worker node that executes the task stops the Flume service.
  • the Zookeeper cluster determines one of several Master nodes as the master node, so that the Master node realizes the high availability mechanism through the Zookeeper cluster, ensuring that if one of the Master nodes has a problem, the other Master node will be in a short time
  • the internal quickly takes over external services, improves the availability of Flume services, and at the same time solves the problem that the timeliness of processing is affected by the failure of processing in time when the node equipment in the conventional fault occurs in the prior art.
  • the configuration information of the task is generated by the master node and the master node according to the host operating status information reported by multiple Worker nodes and written into the ZooKeeper cluster. If the Worker node monitors that there is a task scheduled for itself in the ZooKeeper cluster, then Start the Flume service for execution, thereby realizing the unified management of the configuration through the ZooKeeper cluster, avoiding the problem of a large idle proportion of the Flume collection thread, thus solving the problem of uneven resource use and waste, and improving the operation and maintenance Convenience.
  • FIG. 1 is a flowchart of a data stream processing method provided by Embodiment 1 of the present invention
  • Fig. 2 is a block diagram of a data stream processing system provided by the second embodiment of the present invention.
  • Zookeeper It is a sub-project of hadoop. It is a reliable coordination system for large-scale distributed systems. It provides functions including configuration maintenance, name service, distributed synchronization, group service, etc.
  • Flume It is a highly available, highly reliable and distributed technology for massive log collection, aggregation and transmission.
  • the embodiment of the present invention provides a data stream processing method, which can be executed by a data stream processing system.
  • the system adopts a distributed master-slave architecture and includes a Zookeeper cluster, a number of Master nodes, and a number of Worker nodes, where each Master node And each worker node needs to register the server node in the Zookeeper cluster first, so that each Master node and each Worker can be managed uniformly through the Zookeeper cluster; each Master node and each Worker node can be configured to correspond to a hardware server, Master node and Worker.
  • the specific number of nodes can be determined by users according to their own business application scenarios, and is not specifically limited here.
  • each Worker node starts the Flume service separately, uses the Flume service as a data dump tool, and is responsible for dumping the data of the data source source to the sink.
  • Fig. 1 is a flowchart of a data stream processing method provided by Embodiment 1 of the present invention. As shown in Fig. 1, the method may include the following steps:
  • the Zookeeper cluster determines one of several Master nodes as the master node.
  • the number of Master nodes can be two.
  • the other Master node is determined as the standby node.
  • the Master node can provide external services, provide external interfaces for business online, offline, view, modify, etc., and be responsible for task scheduling.
  • the ZooKeeper cluster receives the master node election request initiated by the Master node based on a preset trigger event, and makes the Master node the master node after the election is successful.
  • the preset trigger event may be the activation of the Master node.
  • the Master node After the Master node executes the start command, the Master node will initiate an election request to participate in the master node (ie Leader node) in the ZooKeeper cluster. If the ZooKeeper cluster determines that the Active Master node as the master node already exists, this If the master node election fails, the ZooKeeper cluster records the status of the Master node as Standby; if the ZooKeeper cluster determines that there is no Active Master node as the master node, the Master node election is successful, and the ZooKeeper cluster records the status of the Master node as Active state. The Master node in the Active state provides external services.
  • the master node ie Leader node
  • the preset trigger event may be used as the current Master node failure of the master node.
  • the ZooKeeper cluster obtains the failure information of the current Master node, and receives election requests from other Master nodes except the current Master node, at a preset time
  • a Master node is determined from other Master nodes as the master node, where the election process may be to elect the Master node with the best host performance among the other Master nodes, which is not limited in the embodiment of the present invention.
  • the Master node when a Master node is elected as the master node, the Master node will start multiple functional modules. Multiple functional modules can specifically include RestServer, RPCServer, and Scheduler. RestServer is responsible for the addition, deletion, and modification of services, and RPCServer Responsible for receiving the current status information reported by the Worker node, and the Scheduler is responsible for dynamically scheduling and assigning tasks.
  • RestServer is responsible for the addition, deletion, and modification of services
  • RPCServer Responsible for receiving the current status information reported by the Worker node
  • the Scheduler is responsible for dynamically scheduling and assigning tasks.
  • the Zookeeper cluster also stores working directories, including but not limited to:
  • one of the several Master nodes is determined by the Zookeeper cluster as the master node, so that the Master node realizes the high availability mechanism through the Zookeeper cluster, ensuring that if one of the Master nodes has a problem, the other Master node The node will quickly take over the external service in a short time, and at the same time solve the problem that the node equipment in the prior art is not handled in a timely manner when a normal failure occurs in the node device, which affects the timeliness of processing.
  • the master node provides an external interface to receive the online request of the service and assign tasks to the service.
  • the external interface is provided by the master node determined from a number of Master nodes.
  • the external interface can be used to receive the online request of the service (that is, Job), and the user can call the external interface to go online through the service application interface.
  • the online request of the service carries the configuration information of the service passed in in JSON format; the master node initializes the service parameters according to the configuration information of the service, and assigns tasks to the service.
  • the task (that is, Task) is the job in the Worker
  • the execution unit on the node is responsible for reading data from the source side and transferring it to the sink side.
  • the external interface provided by the master node through the external interface provided by the master node, it is convenient for the user to directly call the external interface of the master node to go online and offline tasks, which can shorten the operation time of business online and offline to less than 1 minute, thereby simplifying the business online and offline.
  • the configuration there is no need to manually modify the business configuration file, and there is no need to restart the cluster, just restart the business, thereby reducing the mutual influence between the businesses.
  • the master node generates task configuration information and writes it to the ZooKeeper cluster according to the host operating status information reported by each of the multiple worker nodes.
  • the configuration information includes information indicating the worker node that is scheduled to perform the task.
  • the host operating status information includes one or more of CPU usage, memory usage, disk read and write, and network uplink and downlink.
  • the process may include:
  • the target worker node According to the host operating status information reported by each of the multiple worker nodes, determine the target worker node with the best host operating status among the multiple worker nodes; generate configuration information indicating that tasks are scheduled to the target worker node.
  • each Worker node After each Worker node is started, it will start the Report thread, which is responsible for monitoring the running status of its own host, generating host running status information and reporting it to the Master node as the master node.
  • each Worker node monitors the status in the ZooKeeper cluster separately. If it monitors that there is a new task in the ZooKeeper cluster and the task is scheduled for itself, it will obtain the configuration information of the task from the ZooKeeper cluster, and Start the Flume service for execution, where each Worker node is deployed with the Flume service.
  • the configuration information of the task is generated by the master node and written into the ZooKeeper cluster. If the Worker node detects that there is a task scheduled for itself in the ZooKeeper cluster, it will start the Flume service for execution.
  • the unified management of the configuration by the cluster avoids the problem of a large idle proportion of Flume collection threads, thereby solving the problem of uneven use and waste of resources, and also improving the convenience of operation and maintenance.
  • the method provided in the embodiment of the present invention may further include:
  • the master node adjusts the configuration information of the task according to the host running status information and task execution status information reported by multiple worker nodes.
  • the configuration information of the adjusted task indicates that the task in the idle state should be scaled down, and the task in the stacked state should be scaled up;
  • the task execution status information includes task execution speed and task accumulation.
  • each Worker node will send the execution status information and host status information of the tasks on the machine to the master node;
  • the master node determines the task in the idle state and the task in the stacked state, and adds the task in the idle state (Idle Task) and the task in the stacked state (Busy Task) to the IdleTask queue and BusyTask queue respectively, and Automatically shrink IdleTask and actively expand BusyTask;
  • the master node determines that the worker node in the overload state of the host and the host load are in the idle state, and migrates the tasks on the worker node with the host load in the overload state to the worker node with the host load in the idle state Carry out execution.
  • the master node adjusts the configuration information of the task according to the host running status information and task execution status information reported by multiple worker nodes, which can ensure that the host load on the worker node is high and the cluster machine load
  • tasks on the Worker node with a high host load are migrated to other Worker nodes for execution, thereby realizing the worker node and task load balance between the clusters, and realizing the high availability of the Worker node, which improves The availability of Flume service also solves the problem of uneven use and waste of resources.
  • the method provided in the embodiment of the present invention may further include:
  • the master node receives the offline request for the service through the external interface
  • the offline information of the business and the offline information of the tasks assigned to the business are written into the ZooKeeper cluster, so that the worker nodes that perform the tasks stop the Flume service.
  • the master node provides an external interface to receive the offline request of the service (that is, Job), the master node marks the status of the offline service as offline, and then the offline information of the service and the task assigned to the service
  • the offline information is maintained in the ZooKeeper cluster; when the worker node performing the task monitors the offline information of the task from the ZooKeeper cluster, it will execute the stop command of the Task to stop the Flume service, if all tasks related to the business After the execution is stopped, the business is completed offline.
  • Fig. 2 is a block diagram of a data stream processing system provided by the second embodiment of the present invention.
  • the system includes a Zookeeper cluster 10, a number of Master nodes 20 and a number of Worker nodes 30.
  • the number of Master nodes 20 can be It is configured to two, including Master node 21 and Master node 22. When one of the Master nodes is used as the master node, the other Master node is used as the standby node.
  • the number of Worker nodes 30 can be configured to three, including Worker node 31 and Worker node. Node 32 and Worker node 33, where:
  • Zookeeper cluster used to determine one of several Master nodes as the master node
  • the master node is used to provide an external interface to receive online requests for services and allocate tasks for services;
  • the master node is also used to generate task configuration information and write it to the ZooKeeper cluster according to the current status information reported by each of the multiple worker nodes, the configuration information includes information indicating the worker node that is scheduled to perform the task;
  • Worker node is used to start the Flume service for execution if it detects that there is a task scheduled for itself in the ZooKeeper cluster.
  • ZooKeeper cluster is specifically used for:
  • the master node election request initiated by the Master node based on a preset trigger event, and make the Master node the master node after the election is successful, where the preset trigger event is one of the following events:
  • the Master node is started;
  • the current Master node as the master node fails.
  • master node is specifically used for:
  • master node is specifically used for:
  • the master node adjusts the configuration information of the task according to the host running status information and task execution status information reported by multiple Worker nodes;
  • the configuration information of the adjusted task indicates that the task in the idle state should be scaled down, and the task in the stacked state should be scaled up;
  • master node is specifically used for:
  • the offline information of the business and the offline information of the tasks assigned to the business are written into the ZooKeeper cluster, so that the worker nodes that perform the tasks stop the Flume service.

Landscapes

  • Engineering & Computer Science (AREA)
  • Software Systems (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Management, Administration, Business Operations System, And Electronic Commerce (AREA)
  • Hardware Redundancy (AREA)
  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)

Abstract

Cette invention concerne un procédé et un système de traitement de flux de données, appartement au domaine du traitement de mégadonnées. Le procédé comprend les étapes consistant à : déterminer par un cluster Zookeeper l'un de plusieurs nœuds maîtres comme étant un nœud maître (101) ; fournir, par le nœud maître, une interface externe pour recevoir une demande en ligne d'un service, et attribuer une tâche au service (102) ; générer, par le nœud maître, en fonction d'informations d'état actuel rapportées par chacun d'une pluralité de nœuds de travail, des informations de configuration de la tâche et inscrire celles-ci dans le cluster Zookeeper, les informations de configuration comprenant des informations pour indiquer un nœud de travail programmé pour exécuter la tâche (103) ; et, si le nœud de travail observe qu'une tâche planifiée pour celui-ci est présente dans le cluster Zookeeper, lancer un service Flume en vue de l'exécution (104). L'invention permet d'assurer une haute disponibilité des nœuds maîtres et des nœuds de travail, d'améliorer la disponibilité d'un service Flume et d'éviter les problèmes d'une utilisation non équilibrée et du gaspillage de ressources. De plus, les opérations en ligne et hors ligne d'un service peuvent être considérablement simplifiées, et l'influence mutuelle entre les services est réduite.
PCT/CN2019/106779 2019-01-18 2019-09-19 Procédé et système de traitement de flux de données WO2020147330A1 (fr)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CA3168286A CA3168286A1 (fr) 2019-01-18 2019-09-19 Procede et systeme de traitement de flux de donnees

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN201910048043.XA CN109857558A (zh) 2019-01-18 2019-01-18 一种数据流处理方法及系统
CN201910048043.X 2019-01-18

Publications (1)

Publication Number Publication Date
WO2020147330A1 true WO2020147330A1 (fr) 2020-07-23

Family

ID=66895175

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2019/106779 WO2020147330A1 (fr) 2019-01-18 2019-09-19 Procédé et système de traitement de flux de données

Country Status (3)

Country Link
CN (1) CN109857558A (fr)
CA (1) CA3168286A1 (fr)
WO (1) WO2020147330A1 (fr)

Cited By (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112052095A (zh) * 2020-09-11 2020-12-08 成都深思科技有限公司 一种分布式高可用的大数据挖掘任务调度系统
CN112416550A (zh) * 2020-11-19 2021-02-26 广州探途网络技术有限公司 爬虫调度管理平台通信方法和爬虫调度管理平台系统
CN113342508A (zh) * 2021-07-07 2021-09-03 湖南快乐阳光互动娱乐传媒有限公司 一种任务调度方法及装置
CN117076257A (zh) * 2023-09-14 2023-11-17 研华科技(中国)有限公司 一种基于服务器集群的管理方法、管理服务器及管理系统

Families Citing this family (14)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109857558A (zh) * 2019-01-18 2019-06-07 苏宁易购集团股份有限公司 一种数据流处理方法及系统
CN110262882A (zh) * 2019-06-17 2019-09-20 北京思特奇信息技术股份有限公司 一种分布式的通讯命令调度系统及方法
CN111078396B (zh) * 2019-11-22 2023-12-19 厦门安胜网络科技有限公司 一种基于多任务实例的分布式数据接入方法和系统
CN111447097A (zh) * 2020-04-20 2020-07-24 国网甘肃省电力公司信息通信公司 一种云平台资源调度管理方法及系统
CN113010307B (zh) * 2021-02-25 2024-04-05 库珀科技集团有限公司 一种多链区块链浏览器系统及其使用方法
CN113204418A (zh) * 2021-05-19 2021-08-03 中国建设银行股份有限公司 任务调度的方法、装置、电子设备和存储介质
CN113364864B (zh) * 2021-06-03 2022-09-30 上海微盟企业发展有限公司 一种服务器数据同步方法、系统及存储介质
CN113254010A (zh) * 2021-07-09 2021-08-13 广州光点信息科技有限公司 一种可视化dag工作流任务调度系统及其运行方法
CN113934782A (zh) * 2021-09-22 2022-01-14 易联众智鼎(厦门)科技有限公司 一种基于dag模型的数据etl系统及使用方法
CN114124959A (zh) * 2021-12-06 2022-03-01 天地伟业技术有限公司 云流媒体服务的数据处理装置及云流媒体集群
CN114697328A (zh) * 2022-03-25 2022-07-01 浪潮云信息技术股份公司 NiFi高可用集群模式的实现方法及系统
CN114884948A (zh) * 2022-05-05 2022-08-09 零氪科技（北京）有限公司 一种数据处理系统
CN115002122A (zh) * 2022-05-09 2022-09-02 中盈优创资讯科技有限公司 一种用于数据采集集群化管理方法及装置
CN114844799A (zh) * 2022-05-27 2022-08-02 深信服科技股份有限公司 一种集群管理方法、装置、主机设备及可读存储介质

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20030195962A1 (en) * 2002-04-10 2003-10-16 Satoshi Kikuchi Load balancing of servers
CN102521044A (zh) * 2011-12-30 2012-06-27 北京拓明科技有限公司 一种基于消息中间件的分布式任务调度方法及系统
CN105939389A (zh) * 2016-06-29 2016-09-14 乐视控股（北京）有限公司 负载均衡方法及装置
CN108228393A (zh) * 2017-12-14 2018-06-29 浙江航天恒嘉数据科技有限公司 一种可扩展的大数据高可用的实现方法
CN109857558A (zh) * 2019-01-18 2019-06-07 苏宁易购集团股份有限公司 一种数据流处理方法及系统

Family Cites Families (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN103595651B (zh) * 2013-10-15 2017-02-15 北京航空航天大学 基于分布式的数据流处理方法和系统
KR101858565B1 (ko) * 2016-02-19 2018-05-16 서영준 대용량 데이터 처리를 위한 분산 플랫폼 독립적 병렬 처리 방법 및 시스템
CN106375342A (zh) * 2016-10-21 2017-02-01 用友网络科技股份有限公司 一种基于zookeeper技术的系统集群方法及系统
CN108241534A (zh) * 2016-12-27 2018-07-03 阿里巴巴集团控股有限公司 一种任务处理、分配、管理、计算的方法以及装置
CN108304255A (zh) * 2017-12-29 2018-07-20 北京城市网邻信息技术有限公司 分布式任务调度方法及装置、电子设备及可读存储介质

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20030195962A1 (en) * 2002-04-10 2003-10-16 Satoshi Kikuchi Load balancing of servers
CN102521044A (zh) * 2011-12-30 2012-06-27 北京拓明科技有限公司 一种基于消息中间件的分布式任务调度方法及系统
CN105939389A (zh) * 2016-06-29 2016-09-14 乐视控股（北京）有限公司 负载均衡方法及装置
CN108228393A (zh) * 2017-12-14 2018-06-29 浙江航天恒嘉数据科技有限公司 一种可扩展的大数据高可用的实现方法
CN109857558A (zh) * 2019-01-18 2019-06-07 苏宁易购集团股份有限公司 一种数据流处理方法及系统

Cited By (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112052095A (zh) * 2020-09-11 2020-12-08 成都深思科技有限公司 一种分布式高可用的大数据挖掘任务调度系统
CN112052095B (zh) * 2020-09-11 2024-04-19 成都锋卫科技有限公司 一种分布式高可用的大数据挖掘任务调度系统
CN112416550A (zh) * 2020-11-19 2021-02-26 广州探途网络技术有限公司 爬虫调度管理平台通信方法和爬虫调度管理平台系统
CN112416550B (zh) * 2020-11-19 2024-04-05 广州探途网络技术有限公司 爬虫调度管理平台通信方法和爬虫调度管理平台系统
CN113342508A (zh) * 2021-07-07 2021-09-03 湖南快乐阳光互动娱乐传媒有限公司 一种任务调度方法及装置
CN117076257A (zh) * 2023-09-14 2023-11-17 研华科技(中国)有限公司 一种基于服务器集群的管理方法、管理服务器及管理系统
CN117076257B (zh) * 2023-09-14 2024-03-05 研华科技(中国)有限公司 一种基于服务器集群的管理方法、管理服务器及管理系统

Also Published As

Publication number Publication date
CA3168286A1 (fr) 2020-07-23
CN109857558A (zh) 2019-06-07

Similar Documents

Publication Publication Date Title
WO2020147330A1 (fr) Procédé et système de traitement de flux de données
US11226847B2 (en) Implementing an application manifest in a node-specific manner using an intent-based orchestrator
WO2020207266A1 (fr) Système de réseau, procédé de gestion d'instance, dispositif, et support d'informations
CN108845884B (zh) 物理资源分配方法、装置、计算机设备和存储介质
US10628273B2 (en) Node system, server apparatus, scaling control method, and program
EP3180695B1 (fr) Systèmes et procédés de mise à l'échelle automatique d'un système de mégadonnées
WO2022007552A1 (fr) Procédé de gestion de nœud de traitement, procédé de configuration et appareil associé
WO2018059423A1 (fr) Procédé de programmation de ressources réparties, nœud de programmation, et nœud d'accès
WO2012056596A1 (fr) Système informatique et procédé de commande de traitement
WO2017128507A1 (fr) Procédé et système de planification de ressource décentralisée
WO2013104217A1 (fr) Système et procédé de gestion basés sur une infrastructure infonuagique destinés à la maintenance et au déploiement d'un système d'applications
WO2022105138A1 (fr) Procédé, appareil, dispositif et support de planification de tâche décentralisée
CN113742031A (zh) 节点状态信息获取方法、装置、电子设备及可读存储介质
CN112437129B (zh) 集群的管理方法及集群的管理装置
WO2021143590A1 (fr) Système et procédé d'ordonnancement de construction distribuée d'images de conteneurs
CN110958311A (zh) 一种基于yarn的共享集群弹性伸缩系统及方法
CN113382077B (zh) 微服务调度方法、装置、计算机设备和存储介质
WO2021103646A1 (fr) Procédé et dispositif de déploiement de module
KR20110083084A (ko) 가상화를 이용한 서버 운영 장치 및 방법
WO2023115931A1 (fr) Procédé et appareil de réglage de paramètres de composant de mégadonnées, dispositif électronique et support de stockage
CN109117244B (zh) 一种虚拟机资源申请排队机制的实现方法
CN111427670A (zh) 任务调度方法和系统
CN112910937A (zh) 容器集群中的对象调度方法、装置、服务器和容器集群
CN114518955A (zh) 一种基于kubernetes的Flink云原生部署架构方法及系统
CN109992373B (zh) 资源调度方法、信息管理方法和装置及任务部署系统

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 19910871

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 19910871

Country of ref document: EP

Kind code of ref document: A1

122 Ep: pct application non-entry in european phase

Ref document number: 19910871

Country of ref document: EP

Kind code of ref document: A1

ENP Entry into the national phase

Ref document number: 3168286

Country of ref document: CA

32PN Ep: public notification in the ep bulletin as address of the adressee cannot be established

Free format text: NOTING OF LOSS OF RIGHTS PURSUANT TO RULE 112(1) EPC (EPO FORM 1205A DATED 160222)

122 Ep: pct application non-entry in european phase

Ref document number: 19910871

Country of ref document: EP

Kind code of ref document: A1