CN115794354A - Cloud video transcoding task scheduling based on K-Means clustering and improved AOA algorithm - Google Patents
Cloud video transcoding task scheduling based on K-Means clustering and improved AOA algorithm Download PDFInfo
- Publication number
- CN115794354A CN115794354A CN202310035364.2A CN202310035364A CN115794354A CN 115794354 A CN115794354 A CN 115794354A CN 202310035364 A CN202310035364 A CN 202310035364A CN 115794354 A CN115794354 A CN 115794354A
- Authority
- CN
- China
- Prior art keywords
- task
- algorithm
- clustering
- tasks
- search
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000004422 calculation algorithm Methods 0.000 title claims abstract description 69
- 238000003064 k means clustering Methods 0.000 title claims abstract description 22
- 238000000034 method Methods 0.000 claims abstract description 18
- 230000003247 decreasing effect Effects 0.000 claims abstract description 17
- 238000005457 optimization Methods 0.000 claims abstract description 9
- 238000012545 processing Methods 0.000 claims abstract description 8
- 238000005312 nonlinear dynamic Methods 0.000 claims abstract description 5
- 238000007781 pre-processing Methods 0.000 claims description 26
- 230000001133 acceleration Effects 0.000 claims description 25
- 239000013598 vector Substances 0.000 claims description 21
- 238000013507 mapping Methods 0.000 claims description 15
- 230000006870 function Effects 0.000 claims description 9
- 238000012546 transfer Methods 0.000 claims description 9
- 230000008859 change Effects 0.000 claims description 6
- 238000007667 floating Methods 0.000 claims description 6
- 238000004364 calculation method Methods 0.000 claims description 5
- 230000008569 process Effects 0.000 claims description 5
- 230000009286 beneficial effect Effects 0.000 claims description 4
- 125000004432 carbon atom Chemical group C* 0.000 claims description 4
- 230000000739 chaotic effect Effects 0.000 claims description 4
- 238000011161 development Methods 0.000 claims description 4
- 238000005265 energy consumption Methods 0.000 claims description 4
- 239000003795 chemical substances by application Substances 0.000 claims description 3
- 230000007423 decrease Effects 0.000 claims description 3
- 238000013178 mathematical model Methods 0.000 claims description 3
- 238000005295 random walk Methods 0.000 claims description 3
- 230000009467 reduction Effects 0.000 claims description 3
- 230000007704 transition Effects 0.000 claims description 3
- 238000010977 unit operation Methods 0.000 claims description 3
- 238000012795 verification Methods 0.000 claims description 3
- 102100037812 Medium-wave-sensitive opsin 1 Human genes 0.000 description 4
- 101100508406 Caenorhabditis elegans ifa-1 gene Proteins 0.000 description 1
- 241000287127 Passeridae Species 0.000 description 1
- 238000010586 diagram Methods 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 239000012530 fluid Substances 0.000 description 1
- 238000011160 research Methods 0.000 description 1
Images
Classifications
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
- Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management
Landscapes
- Image Analysis (AREA)
Abstract
The invention discloses a cloud video transcoding task scheduling method based on K-Means clustering and an improved AOA algorithm, belongs to the technical field of video task scheduling strategies in a cloud computing environment, and comprises the following steps: step S1: clustering the tasks by using a K mean value clustering algorithm according to different video task characteristics; step S2: and based on a task scheduling strategy of an improved Archimedes optimization algorithm, searching an optimal scheduling scheme, and distributing the video task set to the virtual machine resources for transcoding scheduling processing. In the invention, the chaos reverse learning is added in the early stage of the task scheduling algorithm to improve the usability of the initial solution and the convergence speed, and then the nonlinear dynamic density decreasing factor is reconstructed, so that the algorithm can better balance the global search and the local search, and the final convergence result is better.
Description
Technical Field
The invention belongs to the technical field of video task scheduling strategies in cloud computing environments, and particularly relates to cloud video transcoding task scheduling based on K-Means clustering and an improved AOA algorithm.
Background
With the rapid development of the internet and the arrival of the information era, the data volume required to be processed by a computer is increased in a blowout mode, in order to better meet the requirements and service quality of users, how to improve the computing capacity, network bandwidth and resource utilization rate of the computer and reduce the energy consumption of a system become new research targets, a cloud computing concept is generated from the data volume, a complete solution is provided for the users, the cost can be reduced by cloud computing, the storage is increased, and the resource utilization rate and the adaptability are improved.
The video task is used as one of cloud computing tasks, various requirements of users enable video transcoding directions to be very diversified, and meanwhile, the users and cloud service providers have strict requirements on instantaneity, definition and service quality of the video transcoding task.
How to efficiently and reasonably allocate tasks to virtual machines in a resource pool is one of key technologies of cloud computing, also called task scheduling, a plurality of heuristic intelligent algorithms are applied to task scheduling, such as an artificial bee colony algorithm, a sparrow algorithm, an Archimedes optimization algorithm and the like, the Archimedes Optimization (AOA) algorithm is a novel meta-heuristic algorithm proposed by Hashim and the like in 2020, the algorithm enables individuals to reach a balanced state by simulating the relation of buoyancy force borne by objects completely or partially immersed in fluid when the objects collide, the individual density, the volume and the acceleration are continuously adjusted in an iteration process, so that the population is guided to converge to an optimal position by the individuals with an excellent fitness value, the purpose of optimizing is achieved, but the algorithm is updated around the optimal position in a global search stage because the individuals only depend on a random individual to lead the population to find the optimal solution to an optimal area in the global search stage, when the random individual is a poor solution, the population is easily guided to fall into a local optimal solution, the search stage is incapable of searching the optimal solution, the cloud computing the optimal solution is easy to solve the problem of improving the cloud scheduling in the cloud scheduling process.
Disclosure of Invention
The invention aims to: in order to solve the problems of long task scheduling time, low resource utilization rate, unbalanced load and the like of video task scheduling in a cloud computing environment, the cloud video transcoding task scheduling based on K-Means clustering and an improved AOA algorithm is provided.
In order to achieve the purpose, the invention adopts the following technical scheme:
a cloud video transcoding task scheduling method based on K-Means clustering and an improved AOA algorithm comprises the following steps:
step S1: in cloud computing, a user submits transcoding requirements, a system acquires a video task data set and divides the video task data set into different GOPs through a video divider, the GOPs represent different video tasks, and the video task set is usedFor presentation, for collection of virtual machinesIs shown in whichThe individual tasks are represented as:,a number indicating the number of the task,which represents the length of the task or tasks,representing the amount of computation required for the task,the memory required for the task is represented,expressing the bandwidth required by the tasks, clustering the tasks by using a K-means clustering algorithm according to different task characteristics, and clustering the tasks with similar requirements in the same cluster;
step S2: the task scheduling strategy based on the improved Archimedes optimization algorithm is characterized in that a chaos mapping reverse learning strategy is added in the early stage of the algorithm, a nonlinear dynamic density decreasing factor is reconstructed, global search and local search capabilities of the algorithm are balanced, an optimal scheduling scheme is searched, and a video task set is distributed to virtual machine resources for transcoding scheduling processing.
As a further description of the above technical solution:
the clustering preprocessing of the video tasks in the step S1 includes:
(1) Determining task samples to be clustered, and randomly selecting k tasks as initial clustering centers;
(2) Calculating the distance from each sample to a clustering center, and classifying the task samples to the closest cluster according to the distance;
(4) Repeating the iteration of (2) and (3);
outputting a task clustering calculation result:and outputting the mutually different task sets among the k similar clusters.
As a further description of the above technical solution:
the step S2 of allocating the tasks and the virtual machine resources by using the improved archimedes optimization algorithm includes:
mapping distribution between the tasks in each cluster of the cluster preprocessing and the virtual machines is carried out, for example, if one cluster is taken as an example, n subtask sets are distributed to m virtual machines, the size of an object population is NP, the position of each object is represented by a vector X, and the position of the ith object can be coded asWhereinEach dimension component represents a virtual machine assigned to the task, e.g.If the optimal solution is (1,5,4, …, m, …), it means that virtual machine 1 receives task 1, virtual machine 5 receives task 2, and virtual machine 4 receives task 3;
two n x m matrices Time and S are defined as follows:
whereinRepresenting the time it takes for virtual machine j to process task i,indicating whether the task i is executed on the virtual machine, if soA 1 indicates that task i is executed on virtual machine j, otherwise 0, and the execution time on virtual machine j is the sum of all the task times executed on it:。
as a further description of the above technical solution:
in the clustering preprocessing, the time for all the virtual machines to complete all the tasks is set to be FT, FT is the maximum value of the time for all the virtual machines to complete the tasks, and the method comprises the following steps:the energy consumption required by different virtual machines is also different, and the loss of the ith virtual machine in each unit operation is set asLoss of all virtual machines to complete all tasksConsumption of FC, thenThe load balance can be represented by the ratio of the task running time and the starting time of the virtual machine, when the task is completed, the shorter the idle time of the virtual machine is, the better the load balance capability is, the FZ is set, and then。
As a further description of the above technical solution:
in the clustering preprocessing, the fitness function is F,the smaller F is, the shorter the comprehensive time and cost for task completion is, the better the load balancing degree is, and the better the algorithm performance is;
each object position is represented by a vector X, then the ith object position can be encoded asWhereinAt the time of initializationIs a random integer between 1 and m;
at initializationAnd then, adding Logical mapping reverse learning, wherein the formula is as follows:
will be provided withEach of the dimensions of (0,1) maps to a region of (0,1)In between, a new vector is generatedThe value range of each dimension component in the vector A is (0,1), and then the vector A is used as an initial value to generate a new sequence,Is a chaotic parameter with the value range of,The closer to 4, the more evenly the population is distributed and eventually will beMapping back to (1,m), and then N of the above formulasGenerating corresponding N reverse initial solutions;
Andrespectively, the upper and lower limits of the search space, whereThe number of the carbon atoms is 1,for m, N will be generatedAnd N areSorting function fitness values of the positions, and taking the first N as initialized populations;
Initializing volume (vol) and density (den) of the ith object:,wherein rand is a number [0,1]N-dimensional vectors randomly generated;
acceleration of the initialization object (acc):,andrespectively, the upper and lower limits of the search space, whereThe number of the carbon atoms is 1,is m, i.e.;
In this step, the variables are initialized and the variable with the best fitness value is stored, i.e.,,And;
density and volume update of ith object for t +1 th iteration:
whereinAndis the best object-associated volume and density found so far, and rand is [0,1 ]]An n-dimensional random number in between.
As a further description of the above technical solution:
in the clustering preprocessing, objects collide with each other, after a period of time, the objects try to reach an equilibrium state, and a transfer operator TF transfers the search from global search to local search:
with followingThe transition operator is gradually increased with the lapse of time fromThe number of the steps is increased to 1,andrespectively representing the current iteration times and the maximum iteration times;
the density decrement factor also facilitates the search of the AOA from global to local, which decreases over time using the formula:
the decreasing density factor of the standard AOA is key to coordinating global and local searches, as can be seen from the equation,the method is characterized in that the number of iterations is reduced from e to 0 in a non-linear way, and in the early stage of the iteration, when the distance between the current optimal solution and the global optimal solution is far, the current optimal solution is reduced quickly and the value is smallIs not beneficial to realizing the search of the coverage of the algorithm in the solution space, and in the later period of iteration,the value is reduced too slowly, so that the local development capability of the algorithm is limited, the searching capability of the algorithm is limited, and in order to solve the problem, a density decreasing factor is reconstructed, and the mathematical model of the density decreasing factor is as follows:
wherein,indicating the start of an iterationValue of,indicating the end of an iterationThe value, i.e. when t =0,=e,=0,the smoothness of the control curve is verified whenIf the value is not less than 1.05, the experimental result is optimal, and when the global search is carried out in the middle period before iteration,the value is large, the nonlinear degressive is slow, the algorithm continuously searches unknown areas, the exploration capability is strong, and in the later stage of iteration,the value is small, the nonlinear decreasing trend is gradually increased, the local searching performance of the algorithm is gradually enhanced, and accurate searching is performed around the optimal solution as far as possible so as to balance the global searching capacity and the local searching capacity of the algorithm;
if it is usedEntering a global search phase, collisions between objects, randomly selecting an object (mr) and using the updated t +1 iterationsReplacing the acceleration of the object:
wherein,andrespectively the density, volume and acceleration of the object i,andis the acceleration, density and volume of a random object;
if it is usedEntering a local search stage, updating the acceleration of the object for t +1 times of iteration without collision between the objects:
As a further description of the above technical solution:
in the clustering pre-processing, the acceleration is normalized to calculate the percent change:
where u and l are normalized ranges set to 0.9 and 0.1, respectively, determining the percentage of step size that each agent will change, if object i is far from the global optimum, the acceleration value will be high, which means the target will be in the global search, otherwise it will be in the local search phase, which explains how the search is shifted from the global phase to the local phase;
if it is usedAnd in the global searching stage, calculating the position of the ith object in the t +1 th generation by using a formula:
as a further description of the above technical solution:
in the cluster preprocessing, ifA local searching stage, updating the position thereof by using a formula;
wherein,t is increased along with the number of iterations and is proportional to the transfer operatorDefinition, T is atThe range increases with time, and is verified whenThe experimental results are optimal, at a lower percentageThe larger difference between the optimal position and the current position than initially causes the step value of the random walk to be higher, but as the search progresses, the percentage is gradually increased to reduce the difference between the optimal position and the current position.
As a further description of the above technical solution:
in the clustering preprocessing, the moving direction F is changed by using a formula:
after verification, whenAnd (3) optimizing the experimental result, evaluating each object by using the objective function F, storing the optimal solution, and recording as the optimal solution、、And;
because the task scheduling is a discrete problem, the invention adopts natural number coding, the natural number coding can be changed into floating point numbers after being updated according to a position updating formula, the components of certain dimensionalities can exceed a specified value range, absolute values are sequentially taken from the floating point numbers, the absolute values are rounded downwards, and the remainder is taken, wherein the formula is as follows:
as a further description of the above technical solution:
in the clustering pretreatment, the iteration times are set, and the output is carried out when the maximum iteration times are reachedWill beDecoding into a mapping distribution scheme of the virtual machine and the task, and then handing over to the cloud environment for processing.
In summary, due to the adoption of the technical scheme, the invention has the beneficial effects that:
1. according to the invention, the cloud computing task scheduling method has stronger local search capability, chaotic reverse learning is added in the early stage of the algorithm to improve the availability of an initial solution, the convergence speed is improved, and the nonlinear dynamic density decreasing factor is reconstructed, so that the algorithm can balance global search and local search better, and the final convergence result is better.
2. Aiming at the problems of very diversified video transcoding requirements and different difficulty of video task transcoding, the invention aims to improve the load balance of the system, performs clustering pretreatment before all video tasks are scheduled in the system, clusters different types of tasks based on different task characteristics, and then allocates each cluster of tasks to a virtual machine for resource processing, thereby improving the load balance degree of the system.
Drawings
FIG. 1 is a diagram of a step of cloud video transcoding task scheduling based on K-Means clustering and an improved AOA algorithm according to the present invention;
fig. 2 is a flow chart of implementation of cloud video transcoding task scheduling based on K-Means clustering and an improved AOA algorithm.
Detailed Description
The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
Referring to fig. 1-2, the present invention provides a technical solution: a cloud video transcoding task scheduling method based on K-Means clustering and an improved AOA algorithm comprises the following steps:
step S1: in cloud computing, a user submits transcoding requirements, a system acquires a video task data set and divides the video task data set into different GOPs through a video divider, the GOPs represent different video tasks, and the video task set is usedFor representing, gathering of virtual machinesIs shown in whichThe tasks are represented as:,a number indicating the number of the task,which represents the length of the task or tasks,representing the amount of computation required for the task,the memory required for the task is represented,expressing the bandwidth required by the tasks, clustering the tasks by using a K-means clustering algorithm according to different task characteristics, and clustering the tasks with similar requirements in the same cluster;
step S2: the task scheduling strategy based on the improved Archimedes optimization algorithm is characterized in that a chaos mapping reverse learning strategy is added in the early stage of the algorithm, a nonlinear dynamic density decreasing factor is reconstructed, global search and local search capabilities of the algorithm are balanced, an optimal scheduling scheme is searched, and a video task set is distributed to virtual machine resources for transcoding scheduling processing.
Specifically, the clustering preprocessing on the video task in step S1 includes:
(1) Determining task samples to be clustered, and randomly selecting k tasks as initial clustering centers;
(2) Calculating the distance from each sample to a clustering center, and classifying the task samples to the closest cluster according to the distance;
(4) Repeating the iteration of (2) and (3);
outputting a task clustering calculation result:and outputting the mutually different task sets among the k similar clusters.
Specifically, the step S2 of allocating the tasks and the virtual machine resources by using the improved archimedes optimization algorithm includes:
mapping distribution between the tasks in each cluster of the cluster preprocessing and the virtual machines is carried out, for example, if one cluster is taken as an example, n subtask sets are distributed to m virtual machines, the size of an object population is NP, the position of each object is represented by a vector X, and the position of the ith object can be coded asWhereinEach dimension component represents a virtual machine allocated to the task, for example, if the optimal solution is (1,5,4, …, m, …), it represents that virtual machine 1 receives task 1, virtual machine 5 receives task 2, and virtual machine 4 receives task 3;
two n x m matrices Time and S are defined as follows:
whereinRepresenting the time it takes for virtual machine j to process task i,indicating whether the task i is executed on the virtual machine, ifA 1 indicates that task i is executed on virtual machine j, otherwise 0, and the execution time on virtual machine j is the sum of all the task times executed on it:。
specifically, in the clustering preprocessing, the time for all the virtual machines to complete all the tasks is set to be FT, where FT is the maximum value of the time for all the virtual machines to complete the tasks, and is:the energy consumption required by different virtual machines is also different, and the loss of the ith virtual machine in each unit operation is set asAnd the loss of all the virtual machines for completing all tasks is FC, thenThe load balance can be represented by the ratio of the task running time and the starting time of the virtual machine, when the task is completed, the shorter the idle time of the virtual machine is, the better the load balance capability is, the FZ is set, and then。
Specifically, in the clustering preprocessing, the fitness function is F,the smaller F is, the shorter the comprehensive time and cost for task completion is, the better the load balancing degree is, and the better the algorithm performance is;
each object position is represented by a vector X, and the ith object position can be encoded asWhereinAt the time of initializationIs a random integer between 1 and m;
at initializationAnd then, adding Logical mapping reverse learning, wherein the formula is as follows:
will be provided withEach dimension of (a) is mapped to a (0,1) interval, resulting in a new vectorThe value range of each dimension component in the vector A is (0,1), and then the vector A is used as an initial value to generate a new sequence,Is a chaotic parameter with the value range of,The closer to 4It is easy for the population to be evenly distributed, and finally, the population will be more evenly distributedMapping back to (1,m), and then N of the above formulasGenerating corresponding N reverse initial solutions;
Andrespectively, the upper and lower limits of the search space, whereIs a number of 1, and the number of the main chain is 1,for m, N will be generatedAnd N areSorting function fitness values of the positions, and taking the first N as initialized populations;
Initializing volume (vol) and density (den) of the ith object:,wherein rand is a number [0,1]Randomly generated n-dimensional vectors;
acceleration of the initialization object (acc):,andrespectively, the upper and lower limits of the search space, whereThe number of the carbon atoms is 1,is m, i.e.;
In this step, the variables are initialized and the variable with the best fitness value is stored, i.e.,,And;
density and volume update of ith object for t +1 th iteration:
whereinAndis the best object-associated volume and density found so far, and rand is [0,1 ]]An n-dimensional random number in between.
Specifically, in the clustering preprocessing, collision occurs between objects, after a period of time, the objects try to reach an equilibrium state, and a transfer operator TF transfers search from global search to local search:
the transition operator is gradually increased over time, fromThe number of the steps is increased to 1,andrespectively representing the current iteration times and the maximum iteration times;
the density decrement factor also facilitates the search of the AOA from global to local, which decreases over time using the formula:
the decreasing density factor of the standard AOA is key to coordinating global and local searches, as can be seen from the equation,the method is characterized in that the number of iterations is reduced from e to 0 in a non-linear way, and in the early stage of the iteration, when the distance between the current optimal solution and the global optimal solution is far, the current optimal solution is reduced quickly and the value is smallIs not beneficial to realizing the search of the coverage of the algorithm in the solution space, and in the later period of iteration,the local development capability of the algorithm is limited due to too slow value reduction, the searching capability of the algorithm is limited, and in order to solve the problem, a density decreasing factor is reconstructed, and the mathematical model of the density decreasing factor is as follows:
wherein,indicating the start of an iterationValue of,indicating the end of an iterationThe value, i.e. when t =0,=e,=0,the smoothness degree of the control curve is verified,when in useIf the value is not less than 1.05, the experimental result is optimal, and when the global search is carried out in the middle period before iteration,the value is large, the nonlinear degressive is slow, the algorithm continuously searches unknown areas, the exploration capability is strong, and in the later stage of iteration,the value is small, the nonlinear decreasing trend is gradually increased, the local searching performance of the algorithm is gradually enhanced, and accurate searching is performed around the optimal solution as far as possible so as to balance the global searching capacity and the local searching capacity of the algorithm;
if it is usedEntering a global search stage, collision occurs between the objects, one object (mr) is randomly selected and the object acceleration is iterated for t +1 times by using the update:
wherein,andrespectively the density, volume and acceleration of the object i,andis the acceleration, density and volume of a random object;
if it is usedEntering a local search stage, updating the acceleration of the object for t +1 times of iteration without collision between the objects:
Specifically, in the clustering pre-processing, the acceleration is normalized to calculate the change percentage:
where u and l are normalized ranges set to 0.9 and 0.1, respectively, determining the percentage of step size that each agent will change, if object i is far from the global optimum, the acceleration value will be high, which means the target will be in the global search, otherwise it will be in the local search phase, which explains how the search is shifted from the global phase to the local phase;
if it is notAnd in the global searching stage, calculating the position of the ith object in the t +1 th generation by using a formula:
specifically, in the cluster preprocessing, ifA local searching stage, updating the position of the local searching stage by using a formula;
wherein,t increases with the number of iterations, proportional to the transfer operator, usingDefinition, T is atThe range increases with time, and is verified whenThe experimental result is optimal, starting with a lower percentage, with a larger difference between the optimal position and the current position such that the step value of the random walk will be higher, but the percentage is gradually increased as the search progresses to reduce the difference between the optimal position and the current position.
Specifically, in the clustering preprocessing, the moving direction F is changed by using a formula:
after verification, whenAnd (3) optimizing the experimental result, evaluating each object by using the objective function F, storing the optimal solution, and recording as the optimal solution、、And;
because the task scheduling is a discrete problem, the invention adopts natural number coding, the natural number coding can be changed into floating point numbers after being updated according to a position updating formula, the components of certain dimensionalities can exceed a specified value range, absolute values are sequentially taken from the floating point numbers, the absolute values are rounded downwards, and the remainder is taken, wherein the formula is as follows:
specifically, in the clustering preprocessing, the iteration times are set, and output is performed when the maximum iteration times are reachedWill beDecoding into a mapping distribution scheme of the virtual machine and the task, and then handing over to the cloud environment for processing.
The above description is only for the preferred embodiment of the present invention, but the scope of the present invention is not limited thereto, and any person skilled in the art should be considered to be within the technical scope of the present invention, and the technical solutions and the inventive concepts thereof according to the present invention should be equivalent or changed within the scope of the present invention.
Claims (10)
1. A cloud video transcoding task scheduling method based on K-Means clustering and an improved AOA algorithm is characterized by comprising the following steps:
step S1: in cloud computing, a user submits transcoding requirements, a system acquires a video task data set, and the video task data set is divided into different GOPs (group of pictures) through a video divider, and the GOPs are used as substitutesVideo tasks, video task sets, with different tablesFor presentation, for collection of virtual machinesIs shown in whichThe tasks are represented as:,a number indicating the number of the task,which represents the length of the task or tasks,representing the amount of computation required for the task,the memory required by the task is represented,expressing the bandwidth required by the tasks, clustering the tasks by using a K-means clustering algorithm according to different task characteristics, and clustering the tasks with similar requirements in the same cluster;
step S2: the task scheduling strategy based on the improved Archimedes optimization algorithm is characterized in that a chaos mapping reverse learning strategy is added in the early stage of the algorithm, a nonlinear dynamic density decreasing factor is reconstructed, global search and local search capabilities of the algorithm are balanced, an optimal scheduling scheme is searched, and a video task set is distributed to virtual machine resources for transcoding scheduling processing.
2. The cloud video transcoding task scheduling based on K-Means clustering and improved AOA algorithm as claimed in claim 1, wherein the clustering preprocessing of the video tasks in step S1 comprises:
(1) Determining task samples to be clustered, and randomly selecting k tasks as initial clustering centers;
(2) Calculating the distance from each sample to a clustering center, and classifying the task samples into the closest cluster according to the distance;
(4) Repeating the iteration of (2) and (3);
3. The cloud video transcoding task scheduling method based on K-Means clustering and improved AOA algorithm as claimed in claim 2, wherein the step S2 of allocating tasks and virtual machine resources by using an improved archimedes optimization algorithm comprises:
mapping distribution between the tasks in each cluster of the cluster preprocessing and the virtual machines is carried out, for example, if one cluster is taken as an example, n subtask sets are distributed to m virtual machines, the size of an object population is NP, the position of each object is represented by a vector X, and the position of the ith object can be coded asWhereinEach dimension component represents a virtual machine allocated to the task, for example, if the optimal solution is (1,5,4, …, m, …), it represents that virtual machine 1 receives task 1, virtual machine 5 receives task 2, and virtual machine 4 receives task 3;
two n x m matrices Time and S are defined as follows:
whereinRepresenting the time it takes for virtual machine j to process task i,indicating whether the task i is executed on the virtual machine, if soA 1 indicates that task i is executed on virtual machine j, otherwise 0, and the execution time on virtual machine j is the sum of all the task times executed on it:。
4. the cloud video transcoding task scheduling method based on K-Means clustering and the improved AOA algorithm as claimed in claim 3, wherein in the clustering preprocessing, the time for all the virtual machines to complete all the tasks is set to be FT, and FT is the maximum value of the time for all the virtual machines to complete the tasks, and is as follows:the energy consumption required by different virtual machines is also different, and the loss of the ith virtual machine in each unit operation is set asAnd the loss of all the virtual machines for completing all tasks is FC, thenThe load balance can be embodied by the ratio of the task running time and the boot time of the virtual machine, when the task is completed, the shorter the idle time of the virtual machine is, the better the load balance capability is, the FZ is set, and then。
5. The cloud video transcoding task scheduling based on K-Means clustering and improved AOA algorithm as claimed in claim 4, wherein in the clustering pre-processing, the fitness function is F,the smaller F, the more comprehensive the task is completedThe inter-cost is short, the load balancing degree is better, and the algorithm performance is better;
each object position is represented by a vector X, then the ith object position can be encoded asWhereinAt the time of initializationIs a random integer between 1 and m;
at initializationAnd then, adding Logical mapping reverse learning, wherein the formula is as follows:
will be provided withEach dimension of (a) is mapped to a (0,1) interval, resulting in a new vectorThe value range of each dimension component in the vector A is (0,1), and then the vector A is used as an initial value to generate a new sequence,Is a chaotic parameter with the value range of,The closer to 4, the more evenly the population is distributed and eventually will beMapping back to (1,m), and then N of the above formulasGenerating corresponding N reverse initial solutions;
Andrespectively, the upper and lower limits of the search space, whereIs a number of 1, and the number of the main chain is 1,for m, N ofAnd N areSorting function fitness values of the positions, and taking the first N as initialized populations;
Initializing volume (vol) and density (den) of the ith object:,wherein rand is a number [0,1]N-dimensional vectors randomly generated;
initializing the acceleration (acc) of the object:,andrespectively, the upper and lower limits of the search space, whereThe number of the carbon atoms is 1,is m, i.e.;
In this step, the variables are initialized and the variable with the best fitness value is stored, i.e.,,And;
density and volume update of ith object for t +1 th iteration:
6. The task scheduling of cloud video transcoding based on K-Means clustering and improved AOA algorithm as claimed in claim 5, wherein in the clustering pre-processing, collision occurs between objects, after a period of time, the objects try to reach an equilibrium state, and the transfer operator TF transfers the search from global search to local search:
the transition operator is gradually increased over time, fromThe number of the steps is increased to 1,andrespectively representing the current iteration times and the maximum iteration times;
the density decrement factor also facilitates the search of the AOA from global to local, which decreases over time using the formula:
the decreasing density factor of the standard AOA is key to coordinating global and local searches, and as can be seen from the equation,the method is characterized in that the number of iterations is reduced from e to 0 in a non-linear way, and in the earlier stage of iteration, when the distance between the current optimal solution and the global optimal solution is far, the reduction is fast and the value is smallIs not beneficial to realizing the search of the coverage of the algorithm in the solution space, and in the later period of iteration,the local development capability of the algorithm is limited due to too slow value reduction, the searching capability of the algorithm is limited, and in order to solve the problem, a density decreasing factor is reconstructed, and the mathematical model of the density decreasing factor is as follows:
wherein,indicating the start of an iterationThe value of the sum of the values,indicating the end of an iterationThe value, i.e. when t =0,=e,=0,controlling the smoothness of the curve, and verifying whenIf the value is 1.05, the experimental result is optimal, and if the global search is performed in the early and middle stages of iteration,the value is larger, the nonlinear degressive is slower, the algorithm continuously searches unknown areas, has stronger exploration capacity, and in the later period of iteration,the value is small, the nonlinear decreasing trend is gradually increased, the local searching performance of the algorithm is gradually enhanced, and accurate searching is performed around the optimal solution as far as possible so as to balance the global searching capacity and the local searching capacity of the algorithm;
if it is notEntering a global search stage, collision occurs between the objects, one object (mr) is randomly selected and the object acceleration is iterated for t +1 times by using the update:
wherein,andrespectively the density, volume and acceleration of the object i,andis the acceleration, density and volume of a random object;
if it is notEntering a local search stage, wherein no collision exists between objects, and updating the acceleration of the iterative object for t +1 times:
7. The cloud video transcoding task scheduling based on K-Means clustering and improved AOA algorithm as claimed in claim 6, wherein in the clustering pre-processing, acceleration is normalized to calculate change percentage:
where u and l are normalized ranges set to 0.9 and 0.1, respectively, determining the percentage of step size that each agent will change, if object i is far from the global optimum, the acceleration value will be high, which means the target will be in the global search, otherwise it will be in the local search phase, which explains how the search is shifted from the global phase to the local phase;
if it is notAnd in the global searching stage, calculating the position of the ith object in the t +1 th generation by using a formula:
8. the cloud video transcoding task scheduling based on K-Means clustering and improved AOA algorithm as claimed in claim 7, wherein in the clustering pre-processing, if yes, the cloud video transcoding task schedulingA local searching stage, updating the position thereof by using a formula;
wherein,t is increased along with the number of iterations and is proportional to the transfer operatorDefinition, T is atThe range increases with time and, as verified,when in useThe experimental result is optimal, starting with a lower percentage, with a larger difference between the optimal position and the current position such that the step value of the random walk will be higher, but the percentage is gradually increased as the search progresses to reduce the difference between the optimal position and the current position.
9. The cloud video transcoding task scheduling method based on K-Means clustering and improved AOA algorithm as claimed in claim 8, wherein in the clustering pre-processing, the moving direction F is changed by using a formula:
after verification, whenAnd (3) optimizing the experimental result, evaluating each object by using the objective function F, storing the optimal solution, and recording as the optimal solution、、And;
because the task scheduling is a discrete problem, the invention adopts natural number coding, the natural number coding can be changed into floating point numbers after being updated according to a position updating formula, the components of certain dimensionalities can exceed a specified value range, absolute values are sequentially taken from the floating point numbers, the absolute values are rounded downwards, and the remainder is taken, wherein the formula is as follows:
10. the cloud video transcoding task scheduling method based on K-Means clustering and improved AOA algorithm as claimed in claim 9, wherein in the clustering preprocessing, iteration times are set, and output is performed when the maximum iteration times are reachedWill beDecoding into a mapping distribution scheme of the virtual machine and the task, and then handing over to the cloud environment for processing.
Priority Applications (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN202310035364.2A CN115794354A (en) | 2023-01-10 | 2023-01-10 | Cloud video transcoding task scheduling based on K-Means clustering and improved AOA algorithm |
Applications Claiming Priority (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN202310035364.2A CN115794354A (en) | 2023-01-10 | 2023-01-10 | Cloud video transcoding task scheduling based on K-Means clustering and improved AOA algorithm |
Publications (1)
Publication Number | Publication Date |
---|---|
CN115794354A true CN115794354A (en) | 2023-03-14 |
Family
ID=85428902
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
CN202310035364.2A Pending CN115794354A (en) | 2023-01-10 | 2023-01-10 | Cloud video transcoding task scheduling based on K-Means clustering and improved AOA algorithm |
Country Status (1)
Country | Link |
---|---|
CN (1) | CN115794354A (en) |
Cited By (3)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN117311993A (en) * | 2023-11-28 | 2023-12-29 | 华东交通大学 | Cloud computing load balancing method and system |
CN118093208A (en) * | 2024-04-26 | 2024-05-28 | 哈尔滨工业大学(深圳)(哈尔滨工业大学深圳科技创新研究院) | Deep learning task scheduling method for multi-user GPU cluster |
CN118093208B (en) * | 2024-04-26 | 2024-07-26 | 哈尔滨工业大学(深圳)(哈尔滨工业大学深圳科技创新研究院) | Deep learning task scheduling method for multi-user GPU cluster |
-
2023
- 2023-01-10 CN CN202310035364.2A patent/CN115794354A/en active Pending
Cited By (4)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN117311993A (en) * | 2023-11-28 | 2023-12-29 | 华东交通大学 | Cloud computing load balancing method and system |
CN117311993B (en) * | 2023-11-28 | 2024-03-08 | 华东交通大学 | Cloud computing load balancing method and system |
CN118093208A (en) * | 2024-04-26 | 2024-05-28 | 哈尔滨工业大学(深圳)(哈尔滨工业大学深圳科技创新研究院) | Deep learning task scheduling method for multi-user GPU cluster |
CN118093208B (en) * | 2024-04-26 | 2024-07-26 | 哈尔滨工业大学(深圳)(哈尔滨工业大学深圳科技创新研究院) | Deep learning task scheduling method for multi-user GPU cluster |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
CN110851272B (en) | Cloud task scheduling method based on phagocytic particle swarm genetic hybrid algorithm | |
CN110489229B (en) | Multi-target task scheduling method and system | |
CN111522666B (en) | Cloud robot edge computing unloading model and unloading method thereof | |
CN111124689B (en) | Container resource dynamic allocation method in cluster | |
CN109067834B (en) | Discrete particle swarm scheduling algorithm based on oscillation type inertia weight | |
CN112685138B (en) | Multi-workflow scheduling method based on multi-population hybrid intelligent optimization in cloud environment | |
CN113627871A (en) | Workflow scheduling method, system and storage medium based on multi-target particle swarm algorithm | |
CN114358286A (en) | Mobile equipment federal learning method and system | |
CN111966495B (en) | Data processing method and device | |
CN115794354A (en) | Cloud video transcoding task scheduling based on K-Means clustering and improved AOA algorithm | |
Luo et al. | An improved particle swarm optimization algorithm based on adaptive weight for task scheduling in cloud computing | |
Liu et al. | Task scheduling in cloud computing based on improved discrete particle swarm optimization | |
CN113255873A (en) | Clustering longicorn herd optimization method, system, computer equipment and storage medium | |
CN113391894A (en) | Optimization method of optimal hyper-task network based on RBP neural network | |
CN117032902A (en) | Cloud task scheduling method for improving discrete particle swarm algorithm based on load | |
CN113504998A (en) | Method, device and equipment for determining task scheduling scheme | |
CN115544033A (en) | Method, device, equipment and medium for updating weight checking vector library and data weight checking | |
CN114980216A (en) | Dependent task unloading system and method based on mobile edge calculation | |
CN110781003B (en) | Load balancing method for particle swarm fusion variation control | |
CN114035954A (en) | Scheduling system and task scheduling system based on DDQN algorithm | |
Liu et al. | Task scheduling model of edge computing for AI flow computing in Internet of Things | |
CN114444240A (en) | Delay and service life optimization method for cyber-physical system | |
CN111338757A (en) | Energy-optimized virtual machine deployment method and system | |
CN116828194A (en) | AVS video cloud transcoding task scheduling based on multi-strategy improved Archimedes algorithm | |
CN111813525A (en) | Heterogeneous system workflow scheduling method |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
PB01 | Publication | ||
PB01 | Publication | ||
SE01 | Entry into force of request for substantive examination | ||
SE01 | Entry into force of request for substantive examination |