CN118101703A - Efficient streaming computing method and device for data hub system of Internet of things - Google Patents
Efficient streaming computing method and device for data hub system of Internet of things Download PDFInfo
- Publication number
- CN118101703A CN118101703A CN202410188711.XA CN202410188711A CN118101703A CN 118101703 A CN118101703 A CN 118101703A CN 202410188711 A CN202410188711 A CN 202410188711A CN 118101703 A CN118101703 A CN 118101703A
- Authority
- CN
- China
- Prior art keywords
- data
- internet
- things
- subtasks
- hub system
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000004364 calculation method Methods 0.000 title claims abstract description 30
- 238000012545 processing Methods 0.000 claims abstract description 38
- 238000000034 method Methods 0.000 claims abstract description 25
- CURLTUGMZLYLDI-UHFFFAOYSA-N Carbon dioxide Chemical compound O=C=O CURLTUGMZLYLDI-UHFFFAOYSA-N 0.000 claims description 8
- 238000000638 solvent extraction Methods 0.000 claims description 6
- 238000013523 data management Methods 0.000 claims description 5
- 229910002092 carbon dioxide Inorganic materials 0.000 claims description 4
- 239000001569 carbon dioxide Substances 0.000 claims description 4
- 230000006870 function Effects 0.000 claims description 4
- 230000003044 adaptive effect Effects 0.000 claims description 3
- 230000000903 blocking effect Effects 0.000 claims description 3
- 230000007613 environmental effect Effects 0.000 claims description 3
- 238000012800 visualization Methods 0.000 claims description 3
- 239000002699 waste material Substances 0.000 abstract description 3
- 238000010586 diagram Methods 0.000 description 5
- 238000004891 communication Methods 0.000 description 2
- 238000004590 computer program Methods 0.000 description 2
- 238000007726 management method Methods 0.000 description 2
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000003139 buffering effect Effects 0.000 description 1
- 238000006243 chemical reaction Methods 0.000 description 1
- 238000004140 cleaning Methods 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000005192 partition Methods 0.000 description 1
- 239000000779 smoke Substances 0.000 description 1
- XLYOFNOQVPJJNP-UHFFFAOYSA-N water Substances O XLYOFNOQVPJJNP-UHFFFAOYSA-N 0.000 description 1
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
- H04L67/12—Protocols specially adapted for proprietary or special-purpose networking environments, e.g. medical networks, sensor networks, networks in vehicles or remote metering networks
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
- H04L67/10—Protocols in which an application is distributed across nodes in the network
- H04L67/1097—Protocols in which an application is distributed across nodes in the network for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS]
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/50—Network services
- H04L67/60—Scheduling or organising the servicing of application requests, e.g. requests for application data transmissions using the analysis and optimisation of the required network resources
- H04L67/61—Scheduling or organising the servicing of application requests, e.g. requests for application data transmissions using the analysis and optimisation of the required network resources taking into account QoS or priority requirements
Landscapes
- Engineering & Computer Science (AREA)
- Computer Networks & Wireless Communication (AREA)
- Signal Processing (AREA)
- Health & Medical Sciences (AREA)
- Computing Systems (AREA)
- General Health & Medical Sciences (AREA)
- Medical Informatics (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The invention discloses a high-efficiency streaming computing method and device for an internet of things data hub system. The invention mainly comprises the following steps: constructing a data hub system oriented to an Internet of things scene, and realizing acquisition, processing, sharing and application of data; optimizing operator execution order by adopting an operator scheduler, and simultaneously maximizing the computing resource of the utilization device by balancing the load of the consumer threads; the self-adaption of parameters is realized, and performance bottleneck or resource waste of the system due to abrupt change of data input rate is avoided; and finally, the data processing efficiency of the Internet of things is improved, and data sharing is realized. The method and the system effectively solve the problem of data island in the scene of the Internet of things, and can effectively provide services for business parties.
Description
Technical Field
The invention relates to the field of big data processing, in particular to a high-efficiency streaming computing method and device for an internet of things data hub system.
Background
Streaming computing is a computing mode in big data processing, and is a method for processing streaming data in real time. In the context of the internet of things, data is often constantly generated and continuously arrived. The adoption of the streaming computing has important significance for processing and applying the data in the scene of the Internet of things. The currently commonly used streaming computing engines include Apache Storm, APACHE FLINK, SPARK STREAMING, etc.
Apache Storm is a real-time distributed stream calculation engine, has the characteristics of high throughput and low delay, but does not support state management. SPARK STREAMING is a stream calculation engine built on Spark framework, which adopts a micro batch model to perform stream calculation, has the characteristics of high throughput, high expansibility, fault tolerance and the like, and can be combined with other subframes of Spark, but has higher delay. APACHE FLINK is another distributed computing engine with both batch computing and streaming computing modes, but is currently being used primarily in streaming computing. APACHE FLINK, when used in streaming computing, understand data as an unbounded data stream. APACHE FLINK has the characteristics of high throughput, low delay, high expansibility, support of state management, flexible window operation and the like.
However, current stream computation engines often have difficulty meeting the requirements for data processing and sharing in an internet of things scenario. In order to realize efficient processing and application of the data of the Internet of things, the invention provides a method and a device for efficient streaming computing of a data hub system of the Internet of things.
Disclosure of Invention
The invention aims to overcome the defects of the prior art and provides a high-efficiency streaming computing method and device for an internet of things data hub system.
The aim of the invention is realized by the following technical scheme: a high-efficiency streaming computing method for an internet of things data hub system comprises the following steps:
(1) Collecting sensor data of a target area, and sending different data sources to corresponding stream computing engines;
(2) The streaming computing engine packages the sensor data into a plurality of subtasks, the subtasks are respectively added into task queues according to different priorities, scheduling and executing are carried out based on an operator scheduler, and a data processing result is sent to a corresponding node database;
(3) The data in each node database is copied to the hub database, and the data processing results are fused to realize data sharing;
(4) The hub database provides services based on the data management platform and the visualization platform.
Further, the sensor data includes environmental data of temperature, humidity and carbon dioxide concentration, and the collected data is sent to the flow computing engine by adopting an MQTT protocol.
Further, the operator scheduler includes:
Task queues: the subtasks submitted by the buffer operator are added into corresponding task queues, and the subtasks are realized by adopting a blocking queue;
Operator scheduling policy: for defining an execution order of the subtasks;
Consumer thread: the thread for executing the calculation task has the function of taking out a batch of subtasks from the task queue with highest priority and sequentially executing the subtasks according to a preset operator scheduling strategy, and after the execution of the subtasks is finished, the execution information of the subtasks is sent to the state monitor;
dynamic optimizer: the system is used for dynamically optimizing the task queue capacity, the number of consumer threads and the number of Worker processes according to the data input rate;
Status monitor: the method is used for recording state information, and comprises CPU utilization rate, memory utilization rate, length and capacity of a task queue and time-consuming data of each stage of subtasks.
Further, the operator scheduler adopts an adaptive operator scheduling strategy, and adopts a subtask strategy with the longest priority execution waiting time or adopts a subtask strategy with the highest priority execution load according to the data input rate.
Further, the number of the consumer threads is set to be the same as the number of the CPU cores, subtasks are uniformly distributed to the consumer threads, the consumer threads schedule operators according to different strategies, and the computing resources are utilized to the maximum extent by balancing the loads of the threads.
Further, the partitioning strategy of the node database comprises two types of partitioning according to geographic positions and partitioning according to data types.
Further, the node database and the hub database are deployed by MySQL.
Further, the data management platform is used for checking and managing data, and providing data service based on the application programming interface, so that the direct reading and publishing subscription of the data are realized.
In a second aspect, the present invention further provides a high-efficiency streaming computing device facing the data hub system of the internet of things, which includes a storage unit and one or more processing units, wherein executable codes are stored in the storage unit, and when the processing units execute the executable codes, the high-efficiency streaming computing method facing the data hub system of the internet of things is implemented.
In a third aspect, the present invention further provides a computer readable storage medium, on which a program is stored, which when executed by a processing unit, implements the method for efficient streaming computation for an internet of things data hub system
The invention has the beneficial effects that:
(1) The data hub system of the Internet of things is innovatively provided, the problem of data island in the scene of the Internet of things is effectively solved, data sharing is realized, and services can be effectively provided for business parties.
(2) The high-efficiency streaming computing method is provided, streaming computing efficiency is improved by maximally utilizing computing resources and optimizing operator execution sequences, and meanwhile, parameter self-adaption is realized, and performance bottleneck or resource waste of a system due to abrupt change of data input rate is avoided.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings that are required in the embodiments or the description of the prior art will be briefly described, and it is obvious that the drawings in the following description are only some embodiments of the present invention, and other drawings may be obtained according to these drawings without inventive effort for a person skilled in the art.
FIG. 1 is a data flow diagram in a data hub system for an Internet of things scenario;
FIG. 2 is an architecture diagram of a data hub architecture for an Internet of things scenario;
FIG. 3 is a diagram illustrating a design of an operator scheduler in a high-efficiency streaming computing method according to the present invention;
fig. 4 is a block diagram of the efficient streaming computing device facing the data hub system of the internet of things.
Detailed Description
The invention will be further described with reference to the drawings and examples in order to make the objects, technical solutions and some of the invention more apparent. It should be understood that the detailed description and specific examples, while indicating the invention, are intended for purposes of illustration only and are not intended to limit the scope of the invention.
In order to meet the requirements of data acquisition, processing, sharing and application in an internet of things scene, the invention provides a high-efficiency stream computing method for an internet of things data hub system, as shown in fig. 1 and 2, wherein the architecture of the data hub system comprises a device layer, a computing layer, a data layer and an application layer, the data layer consists of a plurality of node databases and a hub database, the node databases are connected by adopting a star-shaped structure, and the data of the node databases can be copied into the hub database in real time, and the method comprises the following specific steps:
(1) Data acquisition
The data acquisition is realized based on the MQTT protocol. Firstly, the equipment layer refers to a sensor in an internet of things scene, and the sensor comprises a smart electric meter, a smart water meter, a smart smoke sensor, a wireless temperature and humidity sensor and the like, and the sensor can acquire data in a target area, such as environmental data including temperature, humidity, carbon dioxide concentration and the like. The sensor would then send the collected data to the streaming computing engine using the MQTT protocol.
(2) Deployment of streaming computing engines
The computing layer comprises a plurality of streaming computing engines, and the data source of each sensor corresponds to one streaming computing engine. The novel streaming computing engine is adopted for processing the data and is mainly responsible for cleaning, conversion, calculation and the like of the data, so that the data information density can be improved, and excessive computing resources occupied by a large amount of data are avoided. After the accessed data is processed by the stream computation engine, the computation result is sent to the node database. The key configuration of the streaming computing engine is shown in the following table.
The streaming computing engine adopts a more efficient streaming computing method, so that the utilization rate of computer resources can be maximized, the streaming computing method is realized by improving the streaming computing engine Storm of an open source, after the streaming computing engine receives data sent by a sensor, processing logic of the data is packaged into a plurality of subtasks, the subtasks are submitted to an operator scheduler, and finally the operator scheduler performs scheduling and execution. As shown in fig. 3, the operator scheduler is specifically as follows:
Task queues: the method is mainly used for buffering subtasks submitted by operators and is realized by adopting a blocking queue. After receiving the data, the processing of the data is split into a plurality of sub-tasks, each of which is added to a corresponding task queue. If the queue is full, the task commit process will be blocked until there is room left in the queue.
Operator scheduling policy: for defining the execution order of the subtasks.
Consumer thread: the method is a thread for truly executing the calculation task, and has the effects that a batch of subtasks are taken out from a task queue with the highest priority level and are sequentially executed according to a preset operator scheduling strategy. After the execution of the subtasks is completed, the execution information of the subtasks is sent to the state monitor.
Dynamic optimizer: parameters such as task queue capacity, consumer thread number, worker process number and the like are dynamically optimized according to the system state, so that the system can have stable performance in a scene of abrupt change of the data input rate, and meanwhile, the system does not occupy excessive resources when the load is low.
Status monitor: the method is mainly used for recording state information of the system, and comprises data such as CPU utilization rate, memory utilization rate, length and capacity of a task queue, time consumption of each stage of subtasks and the like. The invention optimizes the execution sequence of operators by using an operator scheduler. The scheduler adopts an adaptive operator scheduling strategy. When the data input rate is low, the scheduler adopts a scheduling strategy pursuing low delay, namely, the subtasks with the longest waiting time are preferentially executed; when the data input rate is high, the scheduler adopts a scheduling strategy pursuing high throughput, namely, the operator with highest priority load.
The invention can maximize the utilization of the computing resources by only setting the number of the consumer threads to be the same as the number of the CPU cores. The reasons for this are two: first, the consumer thread can schedule operators according to different strategies, so that computing resources can be reasonably allocated, and the problem that the throughput of the whole streaming computing task is limited due to low throughput of a few operators is avoided. Second, the subtasks submitted by the operators are evenly distributed to the consumer threads, so that even if a "data skew" phenomenon occurs, the load of each consumer thread is approximately equal. The method may maximize the utilization of computing resources by balancing the load of threads.
The invention can adaptively adjust system parameters. When the data input rate is higher, the system can increase the task queue capacity, the number of consumer threads and the number of Worker processes, so that performance bottlenecks are avoided; when the data input rate is low, the system can reduce the task queue capacity, the number of threads of the consumer and the number of Worker processes, and reduce the waste of resources.
(3) Deployment of node databases
And processing the accessed data, and storing the calculation result in a node database. The data in the node database is copied to the hub database. The node database is deployed by MySQL.
The partition strategy of the node database is as follows:
1) The division is made according to geographic location. For example, the division is made according to the country, province, city, park, or the like in which the sensor is located;
2) The division is made according to the data type. For example, the temperature, the humidity and the carbon dioxide concentration are respectively sent to a node database according to the data types collected by the sensors.
(4) Deployment of pivot databases
The hub database copies data from the node database, fuses data processing results in the node database, realizes data sharing, and provides support for upper layer services. The pivot database is deployed by MySQL.
(5) Application of data hub system
The hub database can provide services to the outside based on an application layer, is designed by adopting a C/S architecture, is realized by adopting JavaScript and Vue at the front end, is realized by adopting Java and SpringBoot at the rear end, and mainly provides data services to realize data sharing. The invention not only provides a data management platform for checking and managing data, but also can provide data service based on an application programming interface, support direct reading and publishing and subscribing of the data, for example, a visualization platform can acquire the data through publishing and subscribing and visualize the data in real time.
The invention can solve the problem of information island by fusing data from different sources and providing a platform and an application programming interface, and realize the intercommunication and interconnection of data resources.
Corresponding to the embodiment of the efficient streaming computing method for the data hub system of the Internet of things, the invention further provides an embodiment of the efficient streaming computing device for the data hub system of the Internet of things.
Referring to fig. 4, the efficient streaming computing device for the data hub system of the internet of things provided by the embodiment of the invention includes a storage unit (memory) and one or more processing units (processors), wherein executable codes are stored in the storage unit, and when the processing units execute the executable codes, the processing units are used for implementing the efficient streaming computing method for the data hub system of the internet of things in the above embodiment, and the device also includes a communication unit, an input unit and an output unit, and mainly provides hardware support for the integrity of the data hub system, thereby providing efficient and reliable operation environment.
The embodiment of the high-efficiency streaming computing device for the data hub system of the Internet of things can be applied to any equipment with data processing capability, and the equipment with the data processing capability can be equipment or a device such as a computer. The apparatus embodiments may be implemented by software, or may be implemented by hardware or a combination of hardware and software. Taking software implementation as an example, the device in a logic sense is formed by reading corresponding computer program instructions in a storage unit (nonvolatile memory) into a memory by a processor of any device with data processing capability. From the hardware level, as shown in fig. 4, a hardware structure diagram of an apparatus with optional data processing capability, where the efficient streaming computing device for an internet of things data hub system provided by the present invention is located, is except for a processing unit, a storage unit, a communication unit, an input unit and an output unit shown in fig. 4, where the apparatus with optional data processing capability in the embodiment is generally according to an actual function of the apparatus with optional data processing capability, and may further include other hardware, which is not described herein.
The implementation process of the functions and roles of each unit in the above device is specifically shown in the implementation process of the corresponding steps in the above method, and will not be described herein again.
For the device embodiments, reference is made to the description of the method embodiments for the relevant points, since they essentially correspond to the method embodiments. The apparatus embodiments described above are merely illustrative, wherein the elements illustrated as separate elements may or may not be physically separate, and the elements shown as elements may or may not be physical elements, may be located in one place, or may be distributed over a plurality of network elements. Some or all of the modules may be selected according to actual needs to achieve the purposes of the present invention. Those of ordinary skill in the art will understand and implement the present invention without undue burden.
The embodiment of the invention also provides a computer readable storage medium, and a program is stored on the computer readable storage medium, and when the program is executed by a processor, the high-efficiency streaming computing method for the data hub system of the Internet of things in the embodiment is realized.
The computer readable storage medium may be an internal storage unit, such as a hard disk or a memory, of any of the data processing enabled devices described in any of the embodiments above. The computer readable storage medium may also be an external storage device of any device having data processing capabilities, such as a plug-in hard disk, a smart memory card (SMART MEDIA CARD, SMC), an SD card, a flash memory card (FLASH CARD), etc. provided on the device. Further, the computer readable storage medium may include both internal storage units and external storage devices of any device having data processing capabilities. The computer readable storage medium is used for storing the computer program and other programs and data required by the arbitrary data processing apparatus, and may also be used for temporarily storing data that has been output or is to be output.
The above-described embodiments are intended to illustrate the present invention, not to limit it, and any modifications and variations made thereto are within the spirit of the invention and the scope of the appended claims.
Claims (10)
1. The high-efficiency streaming computing method for the data hub system of the Internet of things is characterized by comprising the following steps of:
(1) Collecting sensor data of a target area, and sending different data sources to corresponding stream computing engines;
(2) The streaming computing engine packages the sensor data into a plurality of subtasks, the subtasks are respectively added into task queues according to different priorities, scheduling and executing are carried out based on an operator scheduler, and a data processing result is sent to a corresponding node database;
(3) The data in each node database is copied to the hub database, and the data processing results are fused to realize data sharing;
(4) The hub database provides services based on the data management platform and the visualization platform.
2. The efficient streaming computing method for the internet of things data hub system according to claim 1, wherein the sensor data comprise environmental data of temperature, humidity and carbon dioxide concentration, and the acquired data are sent to a streaming computing engine by adopting an MQTT protocol.
3. The efficient streaming computing method for the data hub system of the internet of things according to claim 1, wherein the operator scheduler comprises:
Task queues: the subtasks submitted by the buffer operator are added into corresponding task queues, and the subtasks are realized by adopting a blocking queue;
Operator scheduling policy: for defining an execution order of the subtasks;
Consumer thread: the thread for executing the calculation task has the function of taking out a batch of subtasks from the task queue with highest priority and sequentially executing the subtasks according to a preset operator scheduling strategy, and after the execution of the subtasks is finished, the execution information of the subtasks is sent to the state monitor;
dynamic optimizer: the system is used for dynamically optimizing the task queue capacity, the number of consumer threads and the number of Worker processes according to the data input rate;
Status monitor: the method is used for recording state information, and comprises CPU utilization rate, memory utilization rate, length and capacity of a task queue and time-consuming data of each stage of subtasks.
4. The method for efficient streaming computation of an internet of things data hub system according to claim 3, wherein the operator scheduler adopts an adaptive operator scheduling policy, and according to the data input rate, adopts a subtask policy with highest priority execution waiting time or adopts a subtask policy with highest priority execution load.
5. The method for efficient streaming computation of an internet of things data hub system according to claim 3, wherein the number of consumer threads is equal to the number of CPU cores, subtasks are uniformly distributed to consumer threads, the consumer threads schedule operators according to different policies, and computing resources are maximally utilized by balancing loads of the threads.
6. The method for efficient streaming computing of the data hub system oriented to the internet of things according to claim 1, wherein the partitioning strategy of the node database comprises two types of partitioning according to geographic positions and partitioning according to data types.
7. The efficient streaming computing method for the data hub system of the internet of things according to claim 1, wherein the node database and the hub database are deployed by MySQL.
8. The efficient streaming computing method for the data hub system of the internet of things according to claim 1, wherein the data management platform is used for checking and managing data, and providing data services based on an application programming interface, so that direct reading and publishing and subscribing of the data are realized.
9. An efficient streaming computing device facing an internet of things data hub system, comprising a storage unit and one or more processing units, wherein executable codes are stored in the storage unit, and the efficient streaming computing device facing the internet of things data hub system is characterized in that when the processing units execute the executable codes, the efficient streaming computing method facing the internet of things data hub system is realized according to any one of claims 1-8.
10. A computer readable storage medium, on which a program is stored, characterized in that the program, when being executed by a processing unit, implements a high-efficiency streaming computing method for an internet of things data hub system according to any of claims 1-8.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202410188711.XA CN118101703A (en) | 2024-02-20 | 2024-02-20 | Efficient streaming computing method and device for data hub system of Internet of things |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202410188711.XA CN118101703A (en) | 2024-02-20 | 2024-02-20 | Efficient streaming computing method and device for data hub system of Internet of things |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN118101703A true CN118101703A (en) | 2024-05-28 |
Family
ID=91147085
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202410188711.XA Pending CN118101703A (en) | 2024-02-20 | 2024-02-20 | Efficient streaming computing method and device for data hub system of Internet of things |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN118101703A (en) |
-
2024
- 2024-02-20 CN CN202410188711.XA patent/CN118101703A/en active Pending
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN112465129B (en) | On-chip heterogeneous artificial intelligent processor | |
| CN109144731A (en) | Data processing method, device, computer equipment and storage medium | |
| Castilhos et al. | Distributed resource management in NoC-based MPSoCs with dynamic cluster sizes | |
| WO2018072687A1 (en) | Resource scheduling method and apparatus, and filtered scheduler | |
| CN110308984B (en) | Cross-cluster computing system for processing geographically distributed data | |
| CN110489217A (en) | A kind of method for scheduling task and system | |
| Li et al. | Amoeba: Qos-awareness and reduced resource usage of microservices with serverless computing | |
| CN116360972A (en) | Resource management method, device and resource management platform | |
| CN103412786A (en) | High performance server architecture system and data processing method thereof | |
| US10521258B2 (en) | Managing test services in a distributed production service environment | |
| CN110069341A (en) | What binding function configured on demand has the dispatching method of dependence task in edge calculations | |
| CN117608840A (en) | Task processing method and system for comprehensive management of resources of intelligent monitoring system | |
| CN115421930B (en) | Task processing method, system, device, equipment and computer readable storage medium | |
| CN114679451B (en) | Service dispatching system and dispatching method for edge computing | |
| Petrov et al. | Adaptive performance model for dynamic scaling Apache Spark Streaming | |
| CN115460216A (en) | Calculation force resource scheduling method and device, calculation force resource scheduling equipment and system | |
| Liu et al. | Optimizing shuffle in wide-area data analytics | |
| CN115562846A (en) | Resource scheduling method and device and computing node | |
| Liu et al. | Siphon: Expediting {Inter-Datacenter} coflows in {Wide-Area} data analytics | |
| Sundar et al. | Communication augmented latest possible scheduling for cloud computing with delay constraint and task dependency | |
| CN112799811B (en) | High concurrency thread pool task scheduling method for edge gateway | |
| CN116302578B (en) | QoS (quality of service) constraint stream application delay ensuring method and system | |
| CN105740249B (en) | Processing method and system in parallel scheduling process of big data job | |
| CN117579701A (en) | Mobile edge network computing and unloading method and system | |
| CN112214299A (en) | Multi-core processor and task scheduling method and device thereof |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |