CN113673857A - Service sensing and resource scheduling system and method for data center station - Google Patents
Service sensing and resource scheduling system and method for data center station Download PDFInfo
- Publication number
- CN113673857A CN113673857A CN202110930126.9A CN202110930126A CN113673857A CN 113673857 A CN113673857 A CN 113673857A CN 202110930126 A CN202110930126 A CN 202110930126A CN 113673857 A CN113673857 A CN 113673857A
- Authority
- CN
- China
- Prior art keywords
- service
- task
- scheduling
- sequence
- data
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000000034 method Methods 0.000 title claims abstract description 26
- 238000005457 optimization Methods 0.000 claims abstract description 43
- 238000004458 analytical method Methods 0.000 claims abstract description 42
- 238000013145 classification model Methods 0.000 claims abstract description 12
- 238000007726 management method Methods 0.000 claims description 62
- 230000006870 function Effects 0.000 claims description 28
- 238000012549 training Methods 0.000 claims description 22
- 230000002068 genetic effect Effects 0.000 claims description 19
- 230000004044 response Effects 0.000 claims description 18
- 238000012544 monitoring process Methods 0.000 claims description 14
- 239000003016 pheromone Substances 0.000 claims description 12
- 238000012417 linear regression Methods 0.000 claims description 11
- 238000003860 storage Methods 0.000 claims description 10
- 238000005065 mining Methods 0.000 claims description 9
- 238000010801 machine learning Methods 0.000 claims description 7
- 230000008447 perception Effects 0.000 claims description 7
- 238000011161 development Methods 0.000 claims description 6
- 239000011159 matrix material Substances 0.000 claims description 6
- 238000012360 testing method Methods 0.000 claims description 5
- 230000000694 effects Effects 0.000 claims description 4
- 238000012423 maintenance Methods 0.000 claims description 4
- 230000000903 blocking effect Effects 0.000 claims description 3
- 125000004432 carbon atom Chemical group C* 0.000 claims description 3
- 238000013461 design Methods 0.000 claims description 3
- 238000009826 distribution Methods 0.000 claims description 3
- 238000000611 regression analysis Methods 0.000 claims description 3
- 238000013468 resource allocation Methods 0.000 claims description 3
- 238000012163 sequencing technique Methods 0.000 claims description 3
- 230000008014 freezing Effects 0.000 claims description 2
- 238000007710 freezing Methods 0.000 claims description 2
- 238000005516 engineering process Methods 0.000 description 9
- 238000013523 data management Methods 0.000 description 5
- 238000012545 processing Methods 0.000 description 4
- 230000007246 mechanism Effects 0.000 description 3
- 241000257303 Hymenoptera Species 0.000 description 2
- 238000009825 accumulation Methods 0.000 description 2
- 238000004364 calculation method Methods 0.000 description 2
- 238000010276 construction Methods 0.000 description 2
- 230000007547 defect Effects 0.000 description 2
- 238000010586 diagram Methods 0.000 description 2
- 238000004519 manufacturing process Methods 0.000 description 2
- 238000007405 data analysis Methods 0.000 description 1
- 230000001419 dependent effect Effects 0.000 description 1
- 238000004880 explosion Methods 0.000 description 1
- 230000004927 fusion Effects 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 230000009545 invasion Effects 0.000 description 1
- 238000011835 investigation Methods 0.000 description 1
- 230000035772 mutation Effects 0.000 description 1
- 230000008569 process Effects 0.000 description 1
- 238000007670 refining Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q10/00—Administration; Management
- G06Q10/06—Resources, workflows, human or project management; Enterprise or organisation planning; Enterprise or organisation modelling
- G06Q10/063—Operations research, analysis or management
- G06Q10/0631—Resource planning, allocation, distributing or scheduling for enterprises or organisations
- G06Q10/06313—Resource planning in a project environment
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/24—Classification techniques
- G06F18/243—Classification techniques relating to the number of classes
- G06F18/2431—Multiple classes
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N20/00—Machine learning
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/004—Artificial life, i.e. computing arrangements simulating life
- G06N3/006—Artificial life, i.e. computing arrangements simulating life based on simulated virtual individual or collective life forms, e.g. social simulations or particle swarm optimisation [PSO]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/12—Computing arrangements based on biological models using genetic models
- G06N3/126—Evolutionary algorithms, e.g. genetic algorithms or genetic programming
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q10/00—Administration; Management
- G06Q10/06—Resources, workflows, human or project management; Enterprise or organisation planning; Enterprise or organisation modelling
- G06Q10/063—Operations research, analysis or management
- G06Q10/0631—Resource planning, allocation, distributing or scheduling for enterprises or organisations
- G06Q10/06316—Sequencing of tasks or work
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q50/00—Information and communication technology [ICT] specially adapted for implementation of business processes of specific business sectors, e.g. utilities or tourism
- G06Q50/06—Energy or water supply
Landscapes
- Engineering & Computer Science (AREA)
- Business, Economics & Management (AREA)
- Human Resources & Organizations (AREA)
- Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Life Sciences & Earth Sciences (AREA)
- Economics (AREA)
- Health & Medical Sciences (AREA)
- Data Mining & Analysis (AREA)
- Strategic Management (AREA)
- Entrepreneurship & Innovation (AREA)
- General Engineering & Computer Science (AREA)
- Evolutionary Computation (AREA)
- Biophysics (AREA)
- Software Systems (AREA)
- Artificial Intelligence (AREA)
- Marketing (AREA)
- General Health & Medical Sciences (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Bioinformatics & Computational Biology (AREA)
- Mathematical Physics (AREA)
- Evolutionary Biology (AREA)
- Tourism & Hospitality (AREA)
- General Business, Economics & Management (AREA)
- Computing Systems (AREA)
- Molecular Biology (AREA)
- Development Economics (AREA)
- Biomedical Technology (AREA)
- Computational Linguistics (AREA)
- Quality & Reliability (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Operations Research (AREA)
- Game Theory and Decision Science (AREA)
- Educational Administration (AREA)
- Genetics & Genomics (AREA)
- Medical Informatics (AREA)
- Water Supply & Treatment (AREA)
- Public Health (AREA)
- Primary Health Care (AREA)
Abstract
The invention discloses a service sensing and resource scheduling system and method for a data center station. The system comprises a task management module, a task analysis module, a service management module and a scheduling optimization module; the method comprises the following steps: constructing a data service full life cycle model in a service management module; the service management module monitors and maintains the running state of the data service in real time by using the softmax multi-classification model; the task management module dynamically senses the relevant information of the task request of the user, and simultaneously blocks the relevant information and stores the information into the database and the zookeeper; the task analysis module performs quality analysis and request sequence prediction on the perceived user task request; and the scheduling optimization module selects a corresponding scheduling algorithm according to a scheduling strategy preferred by a user, and performs sequence optimization and release on the perceived task request. The invention can obviously improve the quality and efficiency of the data service management on the data center station and provide friendly and convenient data service.
Description
Technical Field
The invention relates to the technical field of data service management, in particular to a service sensing and resource scheduling system and method for a data center station.
Background
With the improvement of the informatization degree of the power enterprises and the explosion growth of the business volume, the data quantity and the variety of the related power systems are increased greatly compared with the past, and thus the massive data needs to be managed and controlled uniformly and efficiently. At present, the existing data service management technology and mechanism still have many problems and defects, for example, the construction of the integrated platform needs the data management technology as a support, the service level of the existing management system has reached a bottleneck, and a new management mechanism is urgently needed to break through the existing obstacles and the problem of difficult data tracing and the like, and needs to be mainly solved.
In order to improve the efficiency of foreground data analysis and the universality of application, the concept of data middlebox is proposed by the aid of the Aliibaba. The data center is a very intelligent data processing platform. The platform functions include the acceptance of technology, the construction and perfection of standard definition, the guidance of business development direction and the like. The data center service is a service servitization capacity framework of full-service and full-type data based on a full-service unified data center, mainly comprises multi-dimensional model data and unified services, is supported by a technical component, a data management tool and a data online development tool, and provides a series of data services for various applications of an analysis domain and a processing domain of the full-service unified data center. Based on enterprise multidimensional fusion model data, by means of enterprise unified analysis service and unified data access component, enterprise unified data service capability is constructed123]. The operator can refine the data according to the use requirement, thereby achieving good effect. In the course of data production, refining and processing, metadata, which is also called data in data, is associated with the data, and this data is very important in the actual production environment. If the control and management of the metadata are familiar, the source of the data in the system can be more definite, and the system can operate stably. If the metadata is managed effectively, each factor involved in the data generation process must be recorded, including information such as the life cycle of the data service. At present, the service unification and data sharing of the data center station and the likeTechniques may provide reference values for the data management model of a vast majority of large enterprises in the future. The technology of the current data center platform is advanced and mature, and is worthy of popularizing relevant business in many large-scale enterprises. On the other hand, as a technical support for many large enterprises, the data center has an indivisible relationship with many factors, and the data center is particularly important to develop in coordination with the factors.
The data center platform solves the problems that a traditional data center is easy to generate data islands, data uniformity is low, models are repeatedly built, the data management level is low and the like. The data center station is used as a carrier and a platform, and the management of data services in the data center station is particularly important. The data service management mechanism needs to provide a full life cycle monitoring and management function for the data service, so that related workers can use the Web management system to perform related operations on the data service under the authority of an administrator. The data service management component is a platform for running all data services and is a support for the data services to complete the life cycle of the data services.
The traditional data service management technology realized by a data center station has the defects of incapability of monitoring and analyzing services, coarse service scheduling granularity, low scheduling algorithm intelligence degree and the like, so that the data management service processing efficiency is low, and the full period of the services cannot be monitored fully and dynamically.
Disclosure of Invention
The invention aims to provide a data center station-oriented service sensing and resource scheduling system and method, which are used for monitoring and maintaining open services of the data center station, managing, analyzing and arranging task requests of users and the like, improving the quality and efficiency of data service management of the data center station, and providing friendly and convenient data services.
The technical solution for realizing the purpose of the invention is as follows: a service perception and resource scheduling system facing a data center platform comprises a task management module, a task analysis module, a service management module and a scheduling optimization module, wherein:
the service management module is used for establishing a full life cycle model for data services opened by a data center platform, establishing a softmax multi-classification model and dynamically monitoring and maintaining the data services;
the task management module senses relevant information of a task request of a user, blocks the task request and stores the relevant information into the database and the zookeeper;
the task analysis module is used for establishing a multiple linear regression model to perform quality analysis on the tasks and predicting the task sequence by using an Apriorall sequence mining algorithm;
and the scheduling optimization module is used for checking the scheduling sequence of the current task, selecting a corresponding scheduling algorithm according to a scheduling strategy preferred by a user, and optimizing the scheduling sequence of the current task so as to maximize the resource utilization.
A service perception and resource scheduling method facing a data center station comprises the following steps:
step 1, constructing a data service full life cycle model in a service management module;
step 2, the service management module monitors and maintains the running state of the data service in real time by using the softmax multi-classification model;
step 3, the task management module dynamically senses the relevant information of the task request of the user, and simultaneously blocks the relevant information and stores the relevant information into the database and the zookeeper;
step 4, the task analysis module carries out quality analysis and request sequence prediction on the perceived user task request;
and 5, selecting a corresponding scheduling algorithm by the scheduling optimization module according to a scheduling strategy preferred by the user, and performing sequence optimization and releasing on the perceived task request.
Compared with the prior art, the invention has the following remarkable advantages: (1) modeling the full life cycle of the data service, providing a machine learning method-based data service full life cycle management and monitoring analysis technology, and realizing high availability and high efficiency of the service; (2) the automatic data service scheduling technology and the optimization scheme based on the combination algorithm of the genetic algorithm and the ant colony algorithm, the priority scheduling algorithm and the high-response-ratio priority scheduling algorithm are provided, the purpose of efficient utilization of resources is achieved under different scheduling strategies, the quality and the efficiency of on-board data service management in data are remarkably improved, and friendly and convenient data services are provided.
Drawings
Fig. 1 is a flow chart of the system operation in the data center station oriented service awareness and resource scheduling system of the present invention.
FIG. 2 is a diagram of a data service full lifecycle model in the present invention.
FIG. 3 is a flow chart of an implementation of the Aprioralll algorithm in the present invention.
FIG. 4 is a diagram of a softmax multi-class model in the present invention.
Fig. 5 is a flow chart of the weighted high-response-ratio-first scheduling algorithm of the present invention.
FIG. 6 is a flow chart of the genetic-ant colony combining algorithm of the present invention.
Detailed Description
The invention relates to a service perception and resource scheduling system facing a data center station, which takes the data center station as a carrier and provides a new data service management model. Firstly, modeling the full life cycle of the data service, providing a machine learning method-based data service full life cycle management and monitoring analysis technology, and realizing high availability and high efficiency of the service; and secondly, a data service automatic scheduling technology and an optimization scheme based on a combination algorithm of a genetic algorithm and an ant colony algorithm, a priority scheduling algorithm and a high-response-ratio priority scheduling algorithm are provided, the purpose of high-efficiency utilization of resources is achieved under different scheduling strategies, and the quality and the efficiency of data service management on the data center station are obviously improved.
The invention relates to a service perception and resource scheduling system facing a data center, which comprises a task management module, a task analysis module, a service management module and a scheduling optimization module, wherein:
the service management module is used for establishing a full life cycle model for data services opened by a data center platform, establishing a softmax multi-classification model and dynamically monitoring and maintaining the data services;
the task management module senses relevant information of a task request of a user, blocks the task request and stores the relevant information into the database and the zookeeper;
the task analysis module is used for establishing a multiple linear regression model to perform quality analysis on the tasks and predicting the task sequence by using an Apriorall sequence mining algorithm;
and the scheduling optimization module is used for checking the scheduling sequence of the current task, selecting a corresponding scheduling algorithm according to a scheduling strategy preferred by a user, and optimizing the scheduling sequence of the current task so as to maximize the resource utilization.
Further, in the service management module, a full life cycle model is established for the data service opened by the data center station, including a generating state, an implementing state, a running state, a freezing state and a withdrawing state, wherein:
the generation state is an initial state of the service, and the developer sets basic information of the service at the stage;
the implementation state is a service early-stage state, and the operation and maintenance worker configures the parameters of the service at the stage;
the running state is a middle-stage state of the service, and the service is normally opened to work;
the frozen state is a later state of the service, the service is suspended to be opened to the outside at the stage, and the rest tasks are processed;
the revocation state is the termination state of the service, and provides reference for the subsequent service design development and implementation work of developers;
the full life cycle management of the data service includes that on one hand, the quality of the data service needs to be analyzed in real time, on the other hand, the current operation stage of the data service needs to be monitored in real time, and service resources are scheduled in a resource limited state in the later period, so that resource allocation is optimized.
Further, in the service management module, a softmax multi-classification model is established, and dynamic monitoring and maintenance are performed on the data service, specifically: and dynamically identifying the running state of the data service according to the current running parameters of the data service, including the number of responses made per second, the service state activity and the service deployment time, and monitoring and maintaining the running of the data service.
Further, the task management module specifically functions as: and dynamically sensing task request related information of a user in an annotation-based and http request non-invasive mode, wherein the task request related information comprises a host address, a port, a url and an interface type, and blocking the task request and storing the related information into a database and a zookeeper.
Further, the task analysis module is specifically used for: constructing a multiple linear regression model, training the model by using historical data such as response time, running time, priority and packet loss rate, analyzing the task quality in real time to obtain a quantized task quality value, analyzing whether data service needs to be improved or not, and further improving the service quality; and (3) excavating a high-frequency sequence from the historical task request sequence by using an Apriorall sequence excavating algorithm, and analyzing the current task request sequence, thereby predicting a task request which possibly occurs next, and performing lower-level resource scheduling in advance.
Further, the scheduling optimization module specifically functions as: and (3) checking the scheduling sequence of the current task by the user, selecting a corresponding scheduling algorithm from a weighted high-response-ratio priority scheduling algorithm and a genetic-ant colony combination algorithm according to a scheduling strategy preferred by the user, wherein the scheduling strategy comprises priority, time resource, computing resource and storage resource, and optimizing the scheduling sequence of the current task so as to maximize resource utilization.
The invention relates to a service sensing and resource scheduling method facing a data center station, which comprises the following steps:
step 1, constructing a data service full life cycle model in a service management module;
step 2, the service management module monitors and maintains the running state of the data service in real time by using the softmax multi-classification model;
step 3, the task management module dynamically senses the relevant information of the task request of the user, and simultaneously blocks the relevant information and stores the relevant information into the database and the zookeeper;
step 4, the task analysis module carries out quality analysis and request sequence prediction on the perceived user task request;
and 5, selecting a corresponding scheduling algorithm by the scheduling optimization module according to a scheduling strategy preferred by the user, and performing sequence optimization and releasing on the perceived task request.
Further, the step 2 of the service management module monitoring and maintaining the running state of the data service in real time by using the softmax multi-classification model specifically comprises:
the softmax model is:
in the formula, K is the number of categories of softmax multi-category, and K is 5, that is, 5 states corresponding to the service state full lifecycle management; x is training data, i is a training data number, and theta is a model parameter;
the cross entropy loss function J (theta) of the softmax multi-classifier is:
in the formula, m is the number of training data, y is a training data label column vector, j is an actual service state category corresponding to the training data, and k is the total number of service states; p is the probability that the test result is the same as the actual result, and is specifically expressed as:
minimizing a loss function by adopting a gradient descent optimization algorithm, and obtaining a model parameter theta by using a GradientDescementOptimizer optimizer in a TensorFlow library;
and identifying the running state of the current service according to the trained model and the given data service running state parameters.
Further, the task analysis module in step 4 performs quality analysis and request sequence prediction on the perceived user task request, specifically as follows:
(1) task quality analysis
Firstly, quantifying the task quality QoS, and quantifying the task quality to a number between 0.0 and 10.0, wherein [0.0-5.9) represents that the service quality is low, [6.0-8.0) represents that the service quality is good, and [8.0-10.0] represents that the service quality is excellent, and the quantified service quality is a label of life cycle data service;
adopting a linear regression analysis method in machine learning to establish a regression model between QoS and task operation characteristic parameters, wherein the regression model is shown as the following formula:
Qos＝θ0response_time+θ1cycling_time+θ2priority+θ3packet_loss+b
in the above formula, the first and second carbon atoms are,
is a column vector, i.e., a parameter of the regression model, where b is a bias parameter, [ theta ] is0,θ1,θ2,θ3]The priority is the priority of the task, and the packet _ loss is the packet loss rate of the task;
the optimization objective function of the regression model is shown as follows:
in the context of the objective function, the function,
training data matrix obtained for collection; y is a training data label column vector, namely quantized data service quality; min is a minimum function;
training the regression analysis model by adopting a gradient descent optimization method to obtain model parameters
Inputting the running characteristic parameters of the unlabeled data service into a regression model by adopting a GradientDescementOptimizer optimizer in a TensorFlow library aiming at the unlabeled data service, namely a test data set to obtain each unlabeled data serviceThe QoS predicted value of the data service further makes intelligent analysis on whether the data service needs to be optimized;
(2) task request sequence prediction
Mining a high-frequency task according to a user historical task request record by adopting a task prediction algorithm based on sequence mining; through the matching of the current sequence and the high-frequency sequence, the possible tasks of the next stage are further predicted, and the lower-stage resource scheduling is carried out according to the tasks discovered in advance, so that the data service of data service management is realized;
the task prediction algorithm adopts an Aprioralll algorithm, and firstly, sequences with different lengths and reaching the minimum support degree in historical task request sequences are mined and are called large sequences; then finding the largest sequence among the large sequences, the largest sequence representing a large sequence that is not contained in any other sequence; and after finding the maximum sequence set, sequencing the sequences in the maximum sequence set from high to low in support degree and length, matching the sequences of the current tasks with the sequences in the maximum sequence set, if the current task sequences are contained in one maximum sequence, successfully matching, and outputting the sequences behind the current task sequence in the maximum sequence successfully matched as the predicted task sequence.
Further, the scheduling optimization module in step 5 selects a corresponding scheduling algorithm according to a scheduling policy preferred by a user, and performs sequence optimization and release on the perceived task request, specifically as follows:
1) the method is characterized in that a scheduling strategy of task priority and time resources is preferred, a weighted high-response-ratio priority scheduling algorithm is correspondingly adopted, the algorithm combines a priority scheduling algorithm and a high-response-ratio priority scheduling algorithm, the priority is taken as a weight, a weighted response ratio concept is established, and the concept is defined as the product of the priority and the response ratio; optimizing the service sequence according to two characteristic parameters of the priority and the predicted running time of the task, selecting the task with the maximum weighted response ratio each time and scheduling and executing the task, thereby forming a task scheduling sequence of a scheduling optimization strategy emphasizing time resources and the priority;
2) the method prefers scheduling strategies of computing resources and storage resources, corresponds to a genetic-ant colony combined algorithm, integrates the genetic algorithm and the ant colony algorithm, executes the genetic algorithm, maps the result of the genetic algorithm to the initial pheromone distribution of the ant colony, obtains the optimal result by utilizing the characteristics of the ant colony algorithm, and finally obtains a task scheduling sequence of the scheduling optimization strategy emphasizing the computing resources and the storage resources.
The invention is described in further detail below with reference to the figures and the embodiments.
Examples
The service perception and resource scheduling system facing the data center station comprises a task management module, a task analysis module, a scheduling optimization module and a service management module, wherein:
under the service management module, firstly, the data service on the data center station is subjected to full life cycle modeling, and the individual states in the life cycle model are as follows:
generating state: the initial state of the service, the developer sets the basic information of the service at this stage;
the implementation state is as follows: configuring parameters of the service for the early state of the service, wherein the parameters are generally configured by an operation and maintenance worker;
the operation state is as follows: the middle-period state of the service, the service is normally opened to work, and is the 'vessel-containing' period in the life cycle;
a frozen state: the service is in a later state, and at the stage, the service is suspended from being opened to the outside and the rest tasks are processed and completed;
and (3) revocation state: the termination state of the service can provide a certain reference for the developer to open and implement the subsequent service design.
The full life cycle management of the data service needs to analyze the quality of the data service in real time and monitor the current operation stage of the data service in real time, so that the service resources are scheduled by adopting an evolutionary algorithm in a resource limited state at a later stage, the resource allocation is optimized, and quick and friendly data service support is realized.
Secondly, under the present module, a list of current data services and related information of each service can be viewed, and the current running state of each service can be identified.
According to the operation parameters (such as the number of responses made per second, the activity of the service state, the service deployment time and the like) which are extracted from the investigation and analysis stage and correspond to each state and characterize each state, the invention applies the softmax multi-classifier of the machine learning method to the data service state full life cycle management for identifying the data service state.
The softmax model is:
in the formula, K is the number of types of the softmax multi-category, and the corresponding K in the subject is 5, that is, the corresponding K corresponds to 5 states of the service state full lifecycle management; x is the training data and θ is the model parameter. The softmax multi-classifier also needs to give a cross entropy loss function thereof, and the formula is as follows:
wherein,
here, a gradient descent optimization algorithm is used to minimize the loss function, and a model parameter θ is derived using a gradientdescnoptimizing optimizer in the tensrflow library. And identifying the running state of the current service according to the trained model and the given data service running state parameters.
Under the task management module, a list of task requests sensed by the system and relevant information of each task request can be observed, and simple operation can be performed on the tasks. The module function adopts an annotation-based mode perception task in an open source micro service task scheduling framework, namely a sia-task scheduling framework of a trust company. The project was developed using a springboot framework. The related maven dependent package of the task fetcher is first imported into the project. And then, establishing a development specification rule of the project, and adding related annotations of a grabbing task on a method of a control layer. After the relevant dependence and annotation are introduced, the method can automatically acquire the information of the host IP address, the port, the url, the interface type and the like after the task carries out the http request. The general flow of the method comprises the steps of intercepting and blocking an http request, storing relevant information into a zookeeper and a database, and displaying the relevant information on a front-end page. And finally, after the tasks are subjected to information display, quality analysis, task sequence prediction, scheduling sequence optimization and the like, releasing according to the optimized sequence. The method has no invasion to the http method request and high safety.
Under the task analysis module, a list of current task requests can be viewed. A quality analysis of the tasks may be performed and a sequence of task requests that may occur next may be predicted from the sequence of current task requests. Wherein:
the task quality analysis is as follows: the quality of the task (QoS) needs to be quantified first. The task quality is quantified to a number between 0.0 and 10.0, where [0.0-5.9) represents low quality of service, [6.0-8.0) represents good quality of service, and [8.0-10.0] represents excellent quality of service. The quantified quality of service is a label for a lifecycle data service.
The module adopts a linear regression analysis method in machine learning to establish a regression model between QoS and task operation characteristic parameters, as shown in the following formula:
Qos＝θ0response_time+θ1cycling_time+θ2priority+θ3packet_loss+b
in the above formula, the first and second carbon atoms are,
is a column vector, i.e., a parameter of the regression model, where b is a bias parameter, [ theta ] is0,θ1,θ2,θ3]Is a weight parameter. The linear regression model also needs to give its optimal objective function as shown in the following equation:
in the regression model objective function,
for the acquired training data matrix, y is the training data label column vector, i.e. the quantized data quality of service. Training the regression analysis model by adopting a gradient descent optimization method to obtain model parameters
Here, the gradientdescntopizer optimizer in the TensorFlow library was used. For the unlabeled data services (i.e. the test data set), the operating characteristic parameters of the unlabeled data services are input into the regression model, so as to obtain the QoS predicted value of each unlabeled data service.
If the predicted data task quality is in the range of [0.0,5.9), the QoS of the data service is low, and the data service needs to be optimized.
By establishing a linear regression model in machine learning, the QoS of the data service can be predicted, whether the data service needs to be improved or not is further intelligently analyzed, the service quality is further improved, and therefore the efficient, stable and friendly data service is provided.
Task sequence prediction refers to finding a high-frequency sequence of task combinations according to historical task sequences. And matching the current sequence with the high-frequency sequence to predict the task. The project adopts a task prediction algorithm based on sequence mining, and mines high-frequency tasks according to historical task request records of users; the possible tasks in the next stage are further predicted according to the high-frequency tasks, subordinate resource scheduling can be performed according to the tasks discovered in advance, automation and convenience of data service management are achieved, and friendly and quick data services are provided.
The function of the module adopts an Aprioralll algorithm. Large sequences of different lengths are mined first for a minimum degree of support in the historical sequence of task requests. The minimum support is usually set manually. The largest sequence is then found among the large sequences, which means a large sequence that is not contained in any other sequence.
And after finding the maximum sequence set, sequencing the sequences in the maximum sequence set from high to low according to the support degree and the length, matching the sequences of the current task with the sequences in the maximum sequence set, if the current task sequence is contained in a certain maximum sequence, successfully matching, and outputting the sequence behind the current task sequence in the maximum sequence as a predicted task sequence.
Under the scheduling optimization module, the scheduling sequence of the current task can be checked, and a corresponding scheduling algorithm (a weighted high-response-ratio priority scheduling algorithm, a genetic-ant colony combination algorithm) can be selected according to a scheduling strategy (priority, time resource, computing resource and storage resource) preferred by a user to optimize the scheduling sequence of the current task so as to achieve the maximization of resource utilization.
Wherein the weighted high-response-ratio-first scheduling algorithm references the scheduling algorithm for jobs and tasks in the operating system. A priority scheduling algorithm (HPF) and a high response ratio priority scheduling algorithm (HRRN) are selected. The two are combined into a 'weighted high response ratio priority scheduling algorithm'. The basic idea of the HPF algorithm is to perform the highest priority task each time priority scheduling is performed. The basic idea of the HRRN algorithm is to perform the highest response ratio job for each priority scheduling. The response ratio is defined as the ratio of the turnaround time to the execution time. The weighted high-response-ratio priority scheduling algorithm integrates the first two algorithms, takes the priority as a weight, establishes a 'weighted response ratio' concept, and defines the concept as the product of the priority and the response ratio. The algorithm optimizes the service sequence according to two characteristic parameters of the priority and the predicted running time of the task, selects the task with the maximum weighted response ratio each time and schedules and executes the task first, and thus a task scheduling sequence of a scheduling optimization strategy emphasizing time resources and the priority is formed.
Wherein the genetic-ant colony combined algorithm integrates the advantages of the genetic algorithm and the ant colony algorithm. The former has high efficiency, and the latter has high solving precision. The two are combined skillfully, so that the time efficiency and the solving precision of the algorithm are improved simultaneously. The genetic algorithm is executed first, and then the results of the genetic algorithm are mapped to the initial pheromone distribution of the ant colony. On the basis, the optimal result is obtained by utilizing the characteristics of the ant colony algorithm. The algorithm details are as follows:
task basic data initialization: the CPU occupation (computing resources) and the memory occupation (storage resources) of the task are taken as two characteristic data of the task for scheduling.
Initializing parameters of a genetic algorithm: and taking the number of tasks needing scheduling optimization as the population scale. The algebra of population development, the probability of individual cross and mutation, and the like are set.
Population initialization of genetic algorithm: setting new and old populations, defining the number of tasks needing scheduling optimization as individual DNA length, and using genotype as task scheduling sequence. And initializing a fitness matrix, an accumulation matrix and an objective function at the same time. The target function is used as a function for calculating the individual fitness, and the specific constraint rule is as follows: if the sum of the cpu occupation ratios of the two tasks is less than 100%, the smaller the sum is, the more likely a new task is to be accommodated to be executed together, and the higher the probability that the two tasks are executed together is, the smaller the "distance" between the two tasks is; if the sum of the cpu occupancy of two tasks is greater than 100% and less than 200%, the closer the sum is to 100%, the more necessary it is to combine with the smaller tasks, the smaller the probability of executing together, the "greater the distance" of the two tasks, and the closer the sum is to 200%, the smaller the "distance" of the two tasks. The calculation mode of the storage resources is the same. And calculating the fitness of the individual in a constraint mode of the objective function.
Selection events for genetic algorithms: the individuals to be crossed are selected according to the calculated accumulation matrix.
Crossover events of genetic algorithms: for two individuals satisfying the crossover probability, crossover is performed in a randomly selected region.
Variant events of genetic algorithms: for individuals satisfying the variation probability, the sequence of the randomly selected DNA region is changed to the reverse order.
Initializing parameters of the ant colony algorithm: and taking the task needing to be scheduled as the city to be traversed by the ant colony. The distance between cities (tasks) is set according to the constraint rule of the objective function of the genetic algorithm.
Initializing ant colony algorithm individual information: the number of individuals in the ant colony, the city from which each individual departs, is determined.
The ant colony algorithm initializes pheromones with the results of the genetic algorithm: the pheromone between any two cities is set to be a certain fixed value, and then, the pheromone between the two tasks (cities) is multiplied by a weight (1.005) every time the genotype of the last generation of individuals of the genetic algorithm has two adjacent tasks, so that the higher the adjacent frequency of the two tasks (cities) is, the higher the pheromone is.
The selection mode of ants for the next city in the ant colony algorithm is as follows: the probability is calculated by pheromone and distance calculations and the selection is made in a roulette manner.
The ant colony algorithm calculates the current optimal path: in the ant colony of the present generation, after all ants traverse all cities (tasks), the distance traveled by each ant is calculated, and the most optimal path with the shortest distance is calculated.
The ant colony algorithm updates pheromone mode: the periant system (ant-cycle) model was used. The formula is as follows:
Tij(t+1)＝(1-r)×Tij(t)+Q/Lk
wherein r and Q are constants, r is 0.5, Q is 1.0, and L iskThe total length of the ant walking. (1-r). times.Tij(t) attenuating the pheromones on all paths, Q/LkFor newly adding pheromones, the longer the distance, the lower the pheromone concentration.
In summary, the invention first establishes a life cycle model for the services opened by the data center, and dynamically identifies and maintains the service operation state by using the softmax multi-classification model. And then the task management module senses the task request related information of the user in an annotation-based and http method request non-invasive mode, and simultaneously blocks and stores the task request related information into the zookeeper and the database. And then the task analysis module analyzes the task quality by using a multiple linear regression model, predicts the tasks possibly required in the next stage by analyzing the historical task request sequence and the current task request sequence by using a sequence mining algorithm, and can perform lower-stage resource scheduling in advance according to the task quality. And finally, the scheduling optimization module selects a corresponding scheduling algorithm (a weighted high-response-ratio priority scheduling algorithm, a genetic-ant colony combination algorithm) according to a scheduling strategy (priority, time resource, computing resource, storage resource and the like) preferred by a user, and optimizes and releases the current task scheduling sequence so as to maximize resource utilization. The invention can obviously improve the quality and efficiency of the data service management on the data center station and provide friendly, convenient and fast data service.
Claims (10)
1. A service perception and resource scheduling system facing a data center station is characterized by comprising a task management module, a task analysis module, a service management module and a scheduling optimization module, wherein:
the service management module is used for establishing a full life cycle model for data services opened by a data center platform, establishing a softmax multi-classification model and dynamically monitoring and maintaining the data services;
the task management module senses relevant information of a task request of a user, blocks the task request and stores the relevant information into the database and the zookeeper;
the task analysis module is used for establishing a multiple linear regression model to perform quality analysis on the tasks and predicting the task sequence by using an Apriorall sequence mining algorithm;
and the scheduling optimization module is used for checking the scheduling sequence of the current task, selecting a corresponding scheduling algorithm according to a scheduling strategy preferred by a user, and optimizing the scheduling sequence of the current task so as to maximize the resource utilization.
2. The system of claim 1, wherein the service management module is configured to establish a full lifecycle model for data services open to the data center, the full lifecycle model including a generation state, an implementation state, a running state, a freezing state, and a revocation state, wherein:
the generation state is an initial state of the service, and the developer sets basic information of the service at the stage;
the implementation state is a service early-stage state, and the operation and maintenance worker configures the parameters of the service at the stage;
the running state is a middle-stage state of the service, and the service is normally opened to work;
the frozen state is a later state of the service, the service is suspended to be opened to the outside at the stage, and the rest tasks are processed;
the revocation state is the termination state of the service, and provides reference for the subsequent service design development and implementation work of developers;
the full life cycle management of the data service includes that on one hand, the quality of the data service needs to be analyzed in real time, on the other hand, the current operation stage of the data service needs to be monitored in real time, and service resources are scheduled in a resource limited state in the later period, so that resource allocation is optimized.
3. The data center platform-oriented service awareness and resource scheduling system of claim 1, wherein in the service management module, a softmax multi-classification model is established for dynamically monitoring and maintaining the data service, and specifically: and dynamically identifying the running state of the data service according to the current running parameters of the data service, including the number of responses made per second, the service state activity and the service deployment time, and monitoring and maintaining the running of the data service.
4. The data center station-oriented service awareness and resource scheduling system of claim 1, wherein the task management module is specifically configured to: and dynamically sensing task request related information of a user in an annotation-based and http request non-invasive mode, wherein the task request related information comprises a host address, a port, a url and an interface type, and blocking the task request and storing the related information into a database and a zookeeper.
5. The data center station-oriented service awareness and resource scheduling system of claim 1, wherein the task analysis module is specifically configured to: constructing a multiple linear regression model, training the model by using historical data such as response time, running time, priority and packet loss rate, analyzing the task quality in real time to obtain a quantized task quality value, analyzing whether data service needs to be improved or not, and further improving the service quality; and (3) excavating a high-frequency sequence from the historical task request sequence by using an Apriorall sequence excavating algorithm, and analyzing the current task request sequence, thereby predicting a task request which possibly occurs next, and performing lower-level resource scheduling in advance.
6. The data center station-oriented service awareness and resource scheduling system of claim 1, wherein the scheduling optimization module specifically functions as: and (3) checking the scheduling sequence of the current task by the user, selecting a corresponding scheduling algorithm from a weighted high-response-ratio priority scheduling algorithm and a genetic-ant colony combination algorithm according to a scheduling strategy preferred by the user, wherein the scheduling strategy comprises priority, time resource, computing resource and storage resource, and optimizing the scheduling sequence of the current task so as to maximize resource utilization.
7. A service sensing and resource scheduling method for a data center station is characterized by comprising the following steps:
step 1, constructing a data service full life cycle model in a service management module;
step 2, the service management module monitors and maintains the running state of the data service in real time by using the softmax multi-classification model;
step 3, the task management module dynamically senses the relevant information of the task request of the user, and simultaneously blocks the relevant information and stores the relevant information into the database and the zookeeper;
step 4, the task analysis module carries out quality analysis and request sequence prediction on the perceived user task request;
and 5, selecting a corresponding scheduling algorithm by the scheduling optimization module according to a scheduling strategy preferred by the user, and performing sequence optimization and releasing on the perceived task request.
8. The data center-oriented service awareness and resource scheduling method according to claim 7, wherein the step 2 of the service management module monitoring and maintaining the running state of the data service in real time by using a softmax multi-classification model specifically comprises:
the softmax model is:
in the formula, K is the number of categories of softmax multi-category, and K is 5, that is, 5 states corresponding to the service state full lifecycle management; x is training data, i is a training data number, and theta is a model parameter;
the cross entropy loss function J (theta) of the softmax multi-classifier is:
in the formula, m is the number of training data, y is a training data label column vector, j is an actual service state category corresponding to the training data, and k is the total number of service states; p is the probability that the test result is the same as the actual result, and is specifically expressed as:
minimizing a loss function by adopting a gradient descent optimization algorithm, and obtaining a model parameter theta by using a GradientDescementOptimizer optimizer in a TensorFlow library;
and identifying the running state of the current service according to the trained model and the given data service running state parameters.
9. The method for service awareness and resource scheduling for a data center station according to claim 7, wherein the task analysis module performs quality analysis and request sequence prediction on the perceived user task request in step 4, specifically as follows:
(1) task quality analysis
Firstly, quantifying the task quality QoS, and quantifying the task quality to a number between 0.0 and 10.0, wherein [0.0-5.9) represents that the service quality is low, [6.0-8.0) represents that the service quality is good, and [8.0-10.0] represents that the service quality is excellent, and the quantified service quality is a label of life cycle data service;
adopting a linear regression analysis method in machine learning to establish a regression model between QoS and task operation characteristic parameters, wherein the regression model is shown as the following formula:
Qos＝θ0response_time+θ1cycling_time+θ2priority+θ3packet_loss+b
in the above formula, the first and second carbon atoms are,
is a column vector, i.e., a parameter of the regression model, where b is a bias parameter, [ theta ] is0,θ1,θ2,θ3]The priority is the priority of the task, and the packet _ loss is the packet loss rate of the task;
the optimization objective function of the regression model is shown as follows:
in the context of the objective function, the function,
training data matrix obtained for collection; y is a training data label column vector, namely quantized data service quality; min is a minimum function;
training the regression analysis model by adopting a gradient descent optimization method to obtain model parameters
These are tested against the unlabeled data service, i.e., the test data set, using the GradientDescementOptimizer optimizer in the TensorFlow libraryInputting the running characteristic parameters of the unmarked data services into a regression model to obtain a QoS predicted value of each unmarked data service, and further performing intelligent analysis on whether the data service needs to be optimized;
(2) task request sequence prediction
Mining a high-frequency task according to a user historical task request record by adopting a task prediction algorithm based on sequence mining; through the matching of the current sequence and the high-frequency sequence, the possible tasks of the next stage are further predicted, and the lower-stage resource scheduling is carried out according to the tasks discovered in advance, so that the data service of data service management is realized;
the task prediction algorithm adopts an Aprioralll algorithm, and firstly, sequences with different lengths and reaching the minimum support degree in historical task request sequences are mined and are called large sequences; then finding the largest sequence among the large sequences, the largest sequence representing a large sequence that is not contained in any other sequence; and after finding the maximum sequence set, sequencing the sequences in the maximum sequence set from high to low in support degree and length, matching the sequences of the current tasks with the sequences in the maximum sequence set, if the current task sequences are contained in one maximum sequence, successfully matching, and outputting the sequences behind the current task sequence in the maximum sequence successfully matched as the predicted task sequence.
10. The method for service awareness and resource scheduling to a data center station according to claim 7, wherein the scheduling optimization module in step 5 selects a corresponding scheduling algorithm according to a scheduling policy preferred by a user, performs sequence optimization on the perceived task request, and releases the sequence optimization, specifically as follows:
1) the method is characterized in that a scheduling strategy of task priority and time resources is preferred, a weighted high-response-ratio priority scheduling algorithm is correspondingly adopted, the algorithm combines a priority scheduling algorithm and a high-response-ratio priority scheduling algorithm, the priority is taken as a weight, a weighted response ratio concept is established, and the concept is defined as the product of the priority and the response ratio; optimizing the service sequence according to two characteristic parameters of the priority and the predicted running time of the task, selecting the task with the maximum weighted response ratio each time and scheduling and executing the task, thereby forming a task scheduling sequence of a scheduling optimization strategy emphasizing time resources and the priority;
2) the method prefers scheduling strategies of computing resources and storage resources, corresponds to a genetic-ant colony combined algorithm, integrates the genetic algorithm and the ant colony algorithm, executes the genetic algorithm, maps the result of the genetic algorithm to the initial pheromone distribution of the ant colony, obtains the optimal result by utilizing the characteristics of the ant colony algorithm, and finally obtains a task scheduling sequence of the scheduling optimization strategy emphasizing the computing resources and the storage resources.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110930126.9A CN113673857B (en) | 2021-08-13 | 2021-08-13 | Service awareness and resource scheduling system and method for data center |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110930126.9A CN113673857B (en) | 2021-08-13 | 2021-08-13 | Service awareness and resource scheduling system and method for data center |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN113673857A true CN113673857A (en) | 2021-11-19 |
| CN113673857B CN113673857B (en) | 2024-07-12 |
Family
ID=78542668
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110930126.9A Active CN113673857B (en) | 2021-08-13 | 2021-08-13 | Service awareness and resource scheduling system and method for data center |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113673857B (en) |
Cited By (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114648258A (en) * | 2022-05-23 | 2022-06-21 | 江苏金融租赁股份有限公司 | Task scheduling method and system based on financing lease service system |
| CN114757278A (en) * | 2022-04-15 | 2022-07-15 | 北京科杰科技有限公司 | Invalid task processing method and device, computer equipment and storage medium |
| CN115269612A (en) * | 2022-09-26 | 2022-11-01 | 公安部信息通信中心 | Cross-platform multidimensional data fusion system based on micro-service |
| CN115550236A (en) * | 2022-08-31 | 2022-12-30 | 国网江西省电力有限公司信息通信分公司 | Data protection method for routing optimization of security middlebox resource pool |
| CN115766418A (en) * | 2022-10-18 | 2023-03-07 | 中国电子科技集团公司第二十八研究所 | Agile adjustment method for service collaboration mode |
| CN117744878A (en) * | 2023-12-26 | 2024-03-22 | 国网吉林省电力有限公司经济技术研究院 | Multi-platform system construction method and system based on digital power system |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US6978303B1 (en) * | 1999-10-26 | 2005-12-20 | Iontal Limited | Monitoring of computer usage |
| JP2010160565A (en) * | 2009-01-06 | 2010-07-22 | Ricoh Co Ltd | Task scheduling apparatus, control method for task scheduling, and control program for task scheduling |
| US8209701B1 (en) * | 2007-09-27 | 2012-06-26 | Emc Corporation | Task management using multiple processing threads |
| US20180144251A1 (en) * | 2016-11-23 | 2018-05-24 | Institute For Information Industry | Server and cloud computing resource optimization method thereof for cloud big data computing architecture |
| CN110457159A (en) * | 2019-08-21 | 2019-11-15 | 深圳前海微众银行股份有限公司 | A kind of method, apparatus, calculating equipment and the storage medium of processing batch tasks |
| US20210097229A1 (en) * | 2018-05-18 | 2021-04-01 | Visa International Service Association | System, method, and computer program product for classifying service request messages |
-
2021
- 2021-08-13 CN CN202110930126.9A patent/CN113673857B/en active Active
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US6978303B1 (en) * | 1999-10-26 | 2005-12-20 | Iontal Limited | Monitoring of computer usage |
| US8209701B1 (en) * | 2007-09-27 | 2012-06-26 | Emc Corporation | Task management using multiple processing threads |
| JP2010160565A (en) * | 2009-01-06 | 2010-07-22 | Ricoh Co Ltd | Task scheduling apparatus, control method for task scheduling, and control program for task scheduling |
| US20180144251A1 (en) * | 2016-11-23 | 2018-05-24 | Institute For Information Industry | Server and cloud computing resource optimization method thereof for cloud big data computing architecture |
| US20210097229A1 (en) * | 2018-05-18 | 2021-04-01 | Visa International Service Association | System, method, and computer program product for classifying service request messages |
| CN110457159A (en) * | 2019-08-21 | 2019-11-15 | 深圳前海微众银行股份有限公司 | A kind of method, apparatus, calculating equipment and the storage medium of processing batch tasks |
Cited By (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114757278A (en) * | 2022-04-15 | 2022-07-15 | 北京科杰科技有限公司 | Invalid task processing method and device, computer equipment and storage medium |
| CN114648258A (en) * | 2022-05-23 | 2022-06-21 | 江苏金融租赁股份有限公司 | Task scheduling method and system based on financing lease service system |
| CN115550236A (en) * | 2022-08-31 | 2022-12-30 | 国网江西省电力有限公司信息通信分公司 | Data protection method for routing optimization of security middlebox resource pool |
| CN115550236B (en) * | 2022-08-31 | 2024-04-30 | 国网江西省电力有限公司信息通信分公司 | Data protection method oriented to security middle station resource pool route optimization |
| CN115269612A (en) * | 2022-09-26 | 2022-11-01 | 公安部信息通信中心 | Cross-platform multidimensional data fusion system based on micro-service |
| CN115269612B (en) * | 2022-09-26 | 2022-12-13 | 公安部信息通信中心 | Cross-platform multi-dimensional data fusion system based on micro-service |
| CN115766418A (en) * | 2022-10-18 | 2023-03-07 | 中国电子科技集团公司第二十八研究所 | Agile adjustment method for service collaboration mode |
| CN117744878A (en) * | 2023-12-26 | 2024-03-22 | 国网吉林省电力有限公司经济技术研究院 | Multi-platform system construction method and system based on digital power system |
Also Published As
| Publication number | Publication date |
|---|---|
| CN113673857B (en) | 2024-07-12 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN113673857A (en) | Service sensing and resource scheduling system and method for data center station | |
| US20220300812A1 (en) | Workflow optimization | |
| WO2021036936A1 (en) | Method and apparatus for allocating resources and tasks in distributed system, and system | |
| Pickardt et al. | Evolutionary generation of dispatching rule sets for complex dynamic scheduling problems | |
| US10528563B2 (en) | Predictive table pre-joins in large scale data management system using graph community detection | |
| US20200012962A1 (en) | Automated Machine Learning System | |
| Ayoubi et al. | An autonomous IoT service placement methodology in fog computing | |
| CN105005570B (en) | Magnanimity intelligent power data digging method and device based on cloud computing | |
| Shahzad et al. | Data mining based job dispatching using hybrid simulation-optimization approach for shop scheduling problem | |
| US8140682B2 (en) | System, method, and apparatus for server-storage-network optimization for application service level agreements | |
| US10223437B2 (en) | Adaptive data repartitioning and adaptive data replication | |
| CN113467944B (en) | Resource deployment device and method for complex software system | |
| CN101263458A (en) | Method and apparatus for a grid network throttle and load collector | |
| CN105379204A (en) | Methods and systems for selecting resources for data routing | |
| Huang et al. | A Simulation‐Based Approach of QoS‐Aware Service Selection in Mobile Edge Computing | |
| CN117076077A (en) | Planning and scheduling optimization method based on big data analysis | |
| CN111177640A (en) | Data center operation and maintenance work performance evaluation system | |
| Nguyen et al. | Zero-latency data warehousing (ZLDWH): the state-of-the-art and experimental implementation approaches | |
| Gu et al. | Characterizing job-task dependency in cloud workloads using graph learning | |
| Kim et al. | Design and implementation of I/O performance prediction scheme on HPC systems through large-scale log analysis | |
| KR101649913B1 (en) | Apparatus and method for managing a research and development project | |
| Suleiman et al. | A framework for characterizing very large cloud workload traces with unsupervised learning | |
| Draghici et al. | Prediction of distributed systems state based on monitoring data | |
| US12118049B2 (en) | Dynamically detecting user personas of network users for customized suggestions | |
| US20240291717A1 (en) | Predictive Analytics For Network Topology Changes |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |