CN114201306B - Multi-dimensional geographic space entity distribution method and system based on load balancing technology - Google Patents

Multi-dimensional geographic space entity distribution method and system based on load balancing technology Download PDF

Info

Publication number
CN114201306B
CN114201306B CN202210139643.9A CN202210139643A CN114201306B CN 114201306 B CN114201306 B CN 114201306B CN 202210139643 A CN202210139643 A CN 202210139643A CN 114201306 B CN114201306 B CN 114201306B
Authority
CN
China
Prior art keywords
task
node
resource
load
occupied
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN202210139643.9A
Other languages
Chinese (zh)
Other versions
CN114201306A (en
Inventor
邓凌云
李彦波
汤开文
江迎
李军
胡志超
向红梅
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Changsha Institute Of Planning And Surveying
Original Assignee
Changsha Institute Of Planning And Surveying
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Changsha Institute Of Planning And Surveying filed Critical Changsha Institute Of Planning And Surveying
Priority to CN202210139643.9A priority Critical patent/CN114201306B/en
Publication of CN114201306A publication Critical patent/CN114201306A/en
Application granted granted Critical
Publication of CN114201306B publication Critical patent/CN114201306B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00—Arrangements for program control, e.g. control units
    • G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/46—Multiprogramming arrangements
    • G06F9/50—Allocation of resources, e.g. of the central processing unit [CPU]
    • G06F9/5005—Allocation of resources, e.g. of the central processing unit [CPU] to service a request
    • G06F9/5027—Allocation of resources, e.g. of the central processing unit [CPU] to service a request the resource being a machine, e.g. CPUs, Servers, Terminals
    • G06F9/505—Allocation of resources, e.g. of the central processing unit [CPU] to service a request the resource being a machine, e.g. CPUs, Servers, Terminals considering the load
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/24—Querying
    • G06F16/245—Query processing
    • G06F16/2453—Query optimisation
    • G06F16/24532—Query optimisation of parallel queries
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/29—Geographical information databases
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00—Arrangements for program control, e.g. control units
    • G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/46—Multiprogramming arrangements
    • G06F9/50—Allocation of resources, e.g. of the central processing unit [CPU]
    • G06F9/5005—Allocation of resources, e.g. of the central processing unit [CPU] to service a request
    • G06F9/5027—Allocation of resources, e.g. of the central processing unit [CPU] to service a request the resource being a machine, e.g. CPUs, Servers, Terminals
    • G06F9/5033—Allocation of resources, e.g. of the central processing unit [CPU] to service a request the resource being a machine, e.g. CPUs, Servers, Terminals considering data affinity
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00—Arrangements for program control, e.g. control units
    • G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/46—Multiprogramming arrangements
    • G06F9/50—Allocation of resources, e.g. of the central processing unit [CPU]
    • G06F9/5005—Allocation of resources, e.g. of the central processing unit [CPU] to service a request
    • G06F9/5027—Allocation of resources, e.g. of the central processing unit [CPU] to service a request the resource being a machine, e.g. CPUs, Servers, Terminals
    • G06F9/5038—Allocation of resources, e.g. of the central processing unit [CPU] to service a request the resource being a machine, e.g. CPUs, Servers, Terminals considering the execution order of a plurality of tasks, e.g. taking priority or time dependency constraints into consideration
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00—Arrangements for program control, e.g. control units
    • G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/46—Multiprogramming arrangements
    • G06F9/50—Allocation of resources, e.g. of the central processing unit [CPU]
    • G06F9/5083—Techniques for rebalancing the load in a distributed system
    • G06F9/5088—Techniques for rebalancing the load in a distributed system involving task migration
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F2209/00—Indexing scheme relating to G06F9/00
    • G06F2209/50—Indexing scheme relating to G06F9/50
    • G06F2209/5022—Workload threshold
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F2216/00—Indexing scheme relating to additional aspects of information retrieval not explicitly covered by G06F16/00 and subgroups
    • G06F2216/03—Data mining

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Software Systems (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Databases & Information Systems (AREA)
  • Data Mining & Analysis (AREA)
  • Computational Linguistics (AREA)
  • Remote Sensing (AREA)
  • Information Transfer Between Computers (AREA)

Abstract

The invention discloses a multi-dimensional geographic space entity distribution method and a system based on a load balancing technology, wherein the method comprises the following steps: responding to a received task request command to obtain resource data required by the task according to the task request command, wherein the resource data comprises a task identifier, average occupied resources of the task, peak occupied resources of the task and a peak time occupation ratio; when judging that the first idle resource of the current node is larger than the resource occupied by the task peak value, the current node receives and processes the task; when judging that the first idle resource of the current node is larger than the product of the peak value occupied resource of the task and the peak value time occupied resource and is smaller than the average occupied resource of the task, carrying out data migration; or when the first idle resource of the current node is judged to be smaller than the product of the occupied resource of the task peak value and the occupied ratio of the peak time, the node is switched. The invention realizes reasonable allocation of resources and improves the utilization rate of resources by acquiring the resource data and selecting different task processing modes according to different conditions.

Description

Multi-dimensional geographic space entity distribution method and system based on load balancing technology
Technical Field
The invention belongs to the field of geographic information, and particularly relates to a multi-dimensional geographic space entity distribution method and system based on a load balancing technology.
Background
We in the digital age have not been satisfied with recording the real world on only two-dimensional digital imagery, and we prefer to see a more realistic, stereoscopic scene in front of them. With the rapid development of urbanization processes in various regions, the urban model is used as an important content of an urban spatial data framework, and the traditional mapping technology cannot meet the requirements of current development. The live-action three-dimensional technology is a brand-new application technology, and slowly becomes the first choice for building the live-action model by virtue of the advantages of intuition, large information amount, general data structure and high automation degree. However, the volume of entity data is huge, which brings huge challenges to data storage and management and severely restricts data processing and analysis efficiency.
Load balancing algorithm research is developed at home and abroad, and research focuses mainly on optimizing resource allocation in advance, namely resource scheduling under the conditions of uniform loading of newly added data and load inclination, wherein typical research comprises the following steps: the Capacity Scheduler job scheduling algorithm proposed by yahoo and the Hadoop load balancing method based on dynamic bandwidth allocation proposed by the south china science and technology college team, however, in the prior art, although the data processing capability is improved to a certain extent, in the using process, the load imbalance phenomenon often exists, and the improvement of the overall performance of the system is restricted.
Disclosure of Invention
In order to solve the above problems in the prior art, the present invention provides a method and a system for distributing multidimensional geospatial entities based on a load balancing technique. The technical problem to be solved by the invention is realized by the following technical scheme:
a multi-dimensional geographic space entity distribution method based on load balancing technology comprises the following steps:
responding to a received task request command to obtain resource data required by the task according to the task request command, wherein the resource data comprises a task identifier, average occupied resources of the task, peak occupied resources of the task and a peak time occupation ratio;
when judging that the first idle resource of the current node is larger than the resource occupied by the task peak value, the current node receives and processes the task; when the first idle resource of the current node is judged to be larger than the product of the peak value occupied resource of the task and the peak value time occupied ratio and smaller than the average occupied resource of the task, data migration is carried out; or when the first idle resource of the current node is judged to be smaller than the product of the occupied resource of the task peak value and the occupied ratio of the peak time, the node is switched; and the first idle resource of the current node is equal to the difference between the total resource of the current node and the resource occupied by the task peak value of the current node.
In one embodiment, the step of performing data migration comprises:
selecting a node to be migrated;
generating a migration state request frame according to the task request command so as to obtain a load identifier and a second idle resource of the node to be migrated;
and when the load identification of the node to be migrated is different from the load identification of the current node and the second idle resource is larger than the resource occupied by the task peak value, dividing the task into a geographic data processing task and a non-format data processing task, and distributing the geographic data processing task or the non-format data processing task to the corresponding node for processing according to the load identification, wherein the load identification comprises the geographic data identification and the non-format data identification.
In one embodiment, selecting a node to be migrated includes:
establishing a task queue set and a node load set;
sequentially acquiring a node with the minimum task queue number and a node with the minimum load value and adding the node into the corresponding task queue set or the corresponding node load set, acquiring an intersection of the task queue set and the node load set after each new element is added, and if the intersection is an empty set, continuously adding the node with the minimum task queue number and the node with the minimum load value which are not added into the set into the corresponding task queue set or the corresponding node load set; if the intersection is not empty and is unique, outputting a node corresponding to the intersection; and if the intersection is not empty and not unique, selecting an optimal node according to a preset algorithm for outputting.
In a specific embodiment, the selecting an optimal node according to a preset algorithm for output includes:
acquiring a task queue number set and a load value set of the nodes corresponding to the intersection;
and respectively carrying out normalization processing on the number of each task queue and the load value, respectively giving corresponding weight factors to the data after the normalization processing, then calculating to obtain a node saturation index, and selecting the node with the minimum node saturation index as an optimal node for outputting.
In one embodiment, the node saturation index calculation formula is:
k=α×(M-μ 1 )/ σ 1 +(1-α)×(N-μ 2 )/ σ 2 wherein M is the task queue number of the node, N is the load value of the node, mu1Is the task queue number set mean, σ1For task queue number set standard deviation, μ2Is the mean value of the load value set, σ2Is the standard deviation of the load value set and alpha is the weighting factor.
The invention also discloses a multi-dimensional geographic space entity distribution system based on the load balancing technology, which comprises the following steps:
the resource data acquisition module is used for responding to a received task request command to obtain resource data required by the task according to the task request command, wherein the resource data comprises a task identifier, average occupied resource of the task, peak occupied resource of the task and peak time occupation ratio;
the data processing module is used for enabling the current node to receive and process the task when judging that the first idle resource of the current node is larger than the resource occupied by the task peak value; and the data migration module is used for carrying out data migration when the first idle resource of the current node is judged to be larger than the product of the peak value occupied resource of the task and the peak value time occupied ratio and smaller than the average occupied resource of the task; or is used for switching the node when judging that the first idle resource of the current node is smaller than the product of the occupied resource of the task peak value and the occupied ratio of the peak time; and the first idle resource of the current node is equal to the difference between the total resource of the current node and the resource occupied by the task peak value of the current node.
In a specific embodiment, the data processing module specifically includes:
a migration node selection unit, configured to select a node to be migrated;
the migration state acquisition unit is used for generating a migration state request frame according to the task request command so as to acquire the load identifier and the second idle resource of the node to be migrated;
and the load migration unit is used for dividing the task into a geographic data processing task and a non-format data processing task and distributing the geographic data processing task or the non-format data processing task to the corresponding node for processing according to the load identification when the load identification of the node to be migrated is different from the load identification of the current node and the second idle resource is larger than the resource occupied by the task peak value, wherein the load identification comprises the geographic data identification and the non-format data identification.
In a specific embodiment, the migration node selecting unit specifically includes:
the set generation subunit is used for establishing a task queue set and a node load set;
the node selection subunit is used for sequentially acquiring a node with the minimum task queue number and a node with the minimum load value, adding the node into the corresponding task queue set or the corresponding node load set, acquiring an intersection of the task queue set and the node load set after each new element addition, and if the intersection is an empty set, continuously adding the node with the minimum task queue number and the node with the minimum load value which are not added into the set into the corresponding task queue set or the corresponding node load set; if the intersection is not empty and is only, outputting a node corresponding to the intersection; and if the intersection is not empty and not unique, selecting an optimal node according to a preset algorithm for outputting.
In a specific embodiment, the node selection subunit is specifically configured to obtain a task queue number set and a load value set of a node corresponding to the intersection; and respectively carrying out normalization processing on the number of each task queue and the load value, respectively giving corresponding weight factors to the data after the normalization processing, then calculating to obtain a node saturation index, and selecting the node with the minimum node saturation index as an optimal node for outputting.
The invention also discloses an electronic device, which comprises a processor, a communication interface, a memory and a communication bus, wherein the processor and the communication interface are used for finishing mutual communication by the memory through the communication bus;
a memory for storing a computer program;
the processor is used for realizing the steps of the method of the invention when executing the program stored in the memory.
The invention has the beneficial effects that:
1. the multi-dimensional geographic space entity distribution method based on the load balancing technology obtains the resource data and selects different task processing modes according to different conditions, thereby realizing reasonable distribution of resources and improving the utilization rate of the resources;
2. according to the multi-dimensional geographic space entity distribution method based on the load balancing technology, when data migration is carried out, tasks are divided into geographic data processing tasks or non-format data processing tasks, different types of nodes are selected to process the tasks, and the nodes with insufficient resource quantity only process the same type of tasks, so that the nodes can optimize processing logic according to task requirements of the nodes, the processing efficiency of the nodes is improved, and a chaotic processing mode is avoided.
3. According to the multi-dimensional geographic space entity distribution method based on the load balancing technology, the nodes for task migration are selected according to the task queues and the node load conditions, and calculation is performed through a reasonable algorithm, so that the optimal nodes can be screened out for task processing, and the task processing efficiency is improved.
The present invention will be described in further detail with reference to the accompanying drawings and examples.
Drawings
Fig. 1 is a schematic flowchart of a multidimensional geospatial entity distribution method based on a load balancing technique according to an embodiment of the present invention;
fig. 2 is a block diagram of a multidimensional geospatial entity distribution system based on load balancing technology according to an embodiment of the present invention;
fig. 3 is a block diagram of a data processing module of a multidimensional geospatial entity distribution system based on a load balancing technique according to an embodiment of the present invention;
fig. 4 is a block diagram of a migration node selection unit of a multidimensional geospatial entity distribution system based on a load balancing technique according to an embodiment of the present invention;
fig. 5 is a schematic diagram of an electronic device module according to an embodiment of the present invention.
Detailed Description
The present invention will be described in further detail with reference to specific examples, but the embodiments of the present invention are not limited thereto.
Example one
Referring to fig. 1, fig. 1 is a schematic flow chart of a method for distributing multidimensional geospatial entities based on a load balancing technique according to an embodiment of the present invention, including:
s1, responding to a received task request command to obtain resource data required by the task according to the task request command, wherein the resource data comprises a task identifier, average occupied resources of the task, peak occupied resources of the task and a peak time proportion;
in this example, the task request command is generally initiated by the service terminal actively, and when geographic data needs to be processed, the service terminal sends the task request command to request the server to process a task, where the geographic data is multidimensional data such as space-time large data. The server receives a task request command and then analyzes and judges the task to obtain resource data required by executing the task, namely a task identifier, a task average occupied resource, a task peak occupied resource and a peak time occupied ratio, wherein the task identifier is a unique identifier for distinguishing different tasks, all information related to the task execution needs to be carried with the task identifier for distinguishing when being sent, the task average occupied resource and the task peak occupied resource are determined according to task requirements and determined according to a task resource occupied model, and the prediction can be specifically carried out through a dynamic load balancing algorithm. For the peak ratio time, generally, if the occupied resource at a certain time is greater than 80% of the maximum occupied resource, the peak resource can be regarded as the peak resource, so as to obtain the peak ratio time.
For multi-dimensional data such as space-time big data, for example, data with space position coordinate information such as vector data, image data, three-dimensional model data, crystallization data, space planning data, etc., the data has a predetermined data format, so that when processing, only a corresponding software algorithm is needed, and the resource condition consumed by processing can be directly estimated through the data amount. The other kind of data has no direct calculation performance and needs to be processed to extract the data of the spatial position coordinate information. For example, the data carries location name address information, but the information is stored in a format that is not directly recognizable and processable, such as location or location identification in an image or video, building name, etc. Such information needs to be matched by spatial coordinate transformation or matched after being identified by a big data processing algorithm. The resources for such data are evaluated through a pre-established task resource occupation model.
S2, when judging that the first idle resource of the current node is larger than the resource occupied by the task peak value, the current node receives and processes the task; when the first idle resource of the current node is judged to be larger than the product of the peak value occupied resource of the task and the peak value time occupied ratio and smaller than the average occupied resource of the task, data migration is carried out; or when the first idle resource of the current node is judged to be smaller than the product of the occupied resource of the task peak value and the occupied ratio of the peak time, the node is switched; and the first idle resource of the current node is equal to the difference between the total resource of the current node and the resource occupied by the task peak value of the current node.
Generally, a computer node or a server node may run multiple tasks in parallel at the same time, and the tasks are not necessarily processed at the same time, but are time-delayed, so that the resource condition of each node at each time is the superposition of multiple tasks in different states, and therefore it is difficult to determine or predict the resource utilization condition of the node through physical derivation, and in order to avoid resource blockage caused by inaccurate prediction, the occupied resources are directly calculated by the sum of peak resource consumption of each task, so that the problem of resource shortage in subsequent execution caused by too low calculation can be avoided to the greatest extent. It should be noted that the peak value of the task executed by the current node is mainly determined by the resource occupied by the task peak value in the resource data, but since the data is theoretical data, when performing the calculation, the actual situation needs to be considered at the same time, if the actual peak occupied resource is greater than the resource occupied by the task peak value in the resource data when the task is executed, the actual peak occupied resource is used for performing the calculation, otherwise, the task peak occupied resource in the resource data is used for performing the calculation.
That is, when it is determined that the first idle resource of the current node is greater than the resource occupied by the task peak value, it indicates that the node resource is sufficient, and the task can be completely received and processed without slowing down the processing efficiency of the system due to the addition of one task; correspondingly, in this embodiment, the product of the resource occupied by the task peak value and the ratio of the peak value to the time represents the minimum requirement that the task can process, and if it is determined that the first idle resource of the current node is smaller than the product of the resource occupied by the task peak value and the ratio of the peak value to the time, it indicates that the node does not have the capability of processing the task, so that the node is directly switched, and the next node continues to perform the determination. And when the first idle resource of the current node is judged to be larger than the product of the peak value occupied resource and the peak value time occupied ratio of the task and smaller than the average occupied resource of the task, the node is indicated to have no capability of completely receiving the task but has a part of receiving capability, data migration operation is carried out, so that the node processes one part of the task, and the other node is selected to process the other part.
The multidimensional geospatial entity distribution method based on the load balancing technology of the embodiment obtains the resource data and selects different task processing modes according to different conditions, thereby realizing reasonable allocation of resources and improving the utilization rate of the resources.
In one embodiment, the step of performing data migration comprises:
selecting a node to be migrated; and the nodes to be migrated are nodes except the current nodes.
Generating a migration state request frame according to the task request command so as to obtain a load identifier and a second idle resource of the node to be migrated; it should be noted that the task request command is sent by the current node, and the task request command carries a task identifier, a task average occupied resource, a task peak occupied resource, and a peak time ratio, and also carries a load identifier of the current node, so as to distinguish the types of the two nodes processing tasks through the representation.
And when the load identification of the node to be migrated is different from the load identification of the current node and the second idle resource is larger than the resource occupied by the task peak value, dividing the task into a geographic data processing task and a non-format data processing task, and distributing the geographic data processing task or the non-format data processing task to the corresponding node for processing according to the load identification, wherein the load identification comprises the geographic data identification and the non-format data identification.
The geographic data of this embodiment is data with spatial position coordinate information, such as vector electronic map data, image data, digital elevation data, three-dimensional model data, street view data, and spatial planning data, which can be processed by direct calculation during processing, and the non-format data is data that has no direct calculation property and can be extracted after certain processing. Because the two data are processed in different manners, the two data can be divided by a task division manner, and preferably, the same processor can be used for processing only one task in a targeted manner, so that the optimization processing can be performed more efficiently. Preferably, the geographic data processing task is processed by the current node, and the non-format data processing task is processed by the node to be migrated, that is, the more direct geographic data processing task is processed locally.
For non-format data, a processor generally needs to have good processing capacity, and data mining needs to be performed on the non-format data through methods such as a regression model, a clustering model, a spatio-temporal frequent pattern, a spatio-temporal co-occurrence pattern and the like.
In the multidimensional geospatial entity distribution method based on the load balancing technology, when data migration is performed, tasks are divided into geographic data processing tasks or non-format data processing tasks, different types of nodes are selected to process the tasks, and the nodes with insufficient resource quantity only process the same type of tasks, so that the nodes can optimize processing logic according to task requirements, the processing efficiency of the nodes is improved, and a chaotic processing mode is avoided.
In one embodiment, selecting a node to be migrated includes:
establishing a task queue set and a node load set;
sequentially acquiring a node with the minimum task queue number and a node with the minimum load value and adding the node with the minimum task queue number and the node with the minimum load value into the corresponding task queue set or the corresponding node load set, acquiring an intersection of the task queue set and the node load set after elements are newly added every time, and if the intersection is an empty set, continuously adding the node with the minimum task queue number and the node with the minimum load value which are not added into the set into the corresponding task queue set or the corresponding node load set; if the intersection is not empty and is only, outputting a node corresponding to the intersection; and if the intersection is not empty and not unique, selecting an optimal node according to a preset algorithm for outputting.
In this embodiment, an optimal node is selected as a node to be migrated according to the number of task queues and the size of a load value, an element in a task queue set is a set of nodes with the minimum number of task queues, and an element in a node load set is a set of nodes with the minimum load value. While the set in the embodiment is continuously expanded, after the node with the minimum number of task queues is obtained, the node (possibly a plurality of nodes) is added into the task queue set, and after the node with the minimum load value is obtained, the node (possibly a plurality of nodes) is added into the task queue set, at this time, because the elements of the two sets are increased, performing intersection judgment once, if the intersection is an empty set, indicating that no coincident node exists in the two sets, and therefore considering that the optimal node is not found, then continuing to increase the nodes according to the condition of minimum task queue number and minimum load value, and judging again until the two sets are not empty, namely, if the coincident nodes exist, if the coincident nodes are unique, and if the coincident nodes are not unique, selecting an optimal node from the nodes as the node to be migrated.
Specifically, the selecting an optimal node according to a preset algorithm for outputting includes:
acquiring a task queue number set and a load value set of the nodes corresponding to the intersection; since some nodes are selected and all the nodes are selected by intersecting sets, that is, the nodes are selected from small to large in sequence, but the task queues and the load values are not completely consistent, a more detailed screening is required, and the task queues and the load values of the nodes are used as sets to select the optimal node.
And respectively carrying out normalization processing on the number of each task queue and the load value, respectively giving corresponding weight factors to the data after the normalization processing, then calculating to obtain a node saturation index, and selecting the node with the minimum node saturation index as an optimal node for outputting.
In one embodiment, the node saturation index calculation formula is:
k=α×(M-μ 1 )/ σ 1 +(1-α)×(N-μ 2 )/ σ 2 wherein, M is the number of task queues of a node, N is the load value of the node, μ 1 is the task queue number set mean value, σ 1 is the task queue number set standard deviation, μ 2 is the load value set mean value, σ 2 is the load value set standard deviation, α is a weight factor, whose value is (0-1), the weight factor is determined according to the actual situation, generally for a node with strong parallel computing capability, α may be appropriately larger, for example, 0.6-0.7, and generally 0.5.
According to the multi-dimensional geographic space entity distribution method based on the load balancing technology, the nodes for task migration are selected according to the task queues and the node load conditions, and calculation is performed through a reasonable algorithm, so that the optimal nodes can be screened out for task processing, and the task processing efficiency is improved.
Example two
Referring to fig. 2, the present embodiment provides a multi-dimensional geospatial entity distribution system based on load balancing technology, including:
a resource data obtaining module 21, configured to respond to a received task request command to obtain resource data required by the task according to the task request command, where the resource data includes a task identifier, a task average occupied resource, a task peak occupied resource, and a peak time occupied ratio;
the data processing module 22 is configured to, when it is determined that the first idle resource of the current node is greater than the resource occupied by the task peak value, enable the current node to receive and process the task; and the data migration module is used for carrying out data migration when the first idle resource of the current node is judged to be larger than the product of the peak value occupied resource of the task and the peak value time occupied ratio and smaller than the average occupied resource of the task; or is used for switching the node when judging that the first idle resource of the current node is smaller than the product of the occupied resource of the task peak value and the occupied ratio of the peak time; and the first idle resource of the current node is equal to the difference between the total resource of the current node and the resource occupied by the task peak value of the current node.
In one embodiment, referring to fig. 3, the data processing module specifically includes:
a migration node selection unit 221, configured to select a node to be migrated;
a migration status obtaining unit 222, configured to generate a migration status request frame according to the task request command, so as to obtain a load identifier and a second idle resource of the node to be migrated;
and the load migration unit 223 is configured to, when it is determined that the load identifier of the node to be migrated is different from the load identifier of the current node and the second idle resource is greater than the resource occupied by the task peak value, divide the task into a geographic data processing task and a non-format data processing task, and allocate the geographic data processing task or the non-format data processing task to a corresponding node for processing according to the load identifier, where the load identifier includes the geographic data identifier and the non-format data identifier.
In one embodiment, referring to fig. 4, the migration node selecting unit specifically includes:
a set generating subunit 2211, configured to establish a task queue set and a node load set;
a node selection subunit 2212, configured to sequentially obtain a node with the smallest number of task queues and a node with the smallest load value, add the node into the corresponding task queue set or the corresponding node load set, and after an element is newly added each time, obtain an intersection of the task queue set and the node load set, and if the intersection is an empty set, continue to add the node with the smallest number of task queues and the node with the smallest load value, which are not added into the set, into the corresponding task queue set or the corresponding node load set; if the intersection is not empty and is only, outputting a node corresponding to the intersection; and if the intersection is not empty and not unique, selecting an optimal node according to a preset algorithm for outputting.
In a specific embodiment, the node selection subunit is specifically configured to obtain a task queue number set and a load value set of a node corresponding to the intersection; and respectively carrying out normalization processing on the number of each task queue and the load value, respectively giving corresponding weight factors to the data after the normalization processing, then calculating to obtain a node saturation index, and selecting the node with the minimum node saturation index as an optimal node for outputting.
An embodiment of the present invention further provides an electronic device, as shown in fig. 5, including a processor 51, a communication interface 52, a memory 53 and a communication bus 54, where the processor 51, the communication interface 52, and the memory 53 complete mutual communication through the communication bus 54,
a memory 53 for storing a computer program;
the processor 51 is configured to implement the following steps when executing the program stored in the memory 53:
s1, responding to a received task request command to obtain resource data required by the task according to the task request command, wherein the resource data comprise a task identifier, task average occupied resources, task peak occupied resources and a peak time occupied ratio;
s2, when judging that the first idle resource of the current node is larger than the resource occupied by the task peak value, the current node receives and processes the task; when the first idle resource of the current node is judged to be larger than the product of the peak value occupied resource of the task and the peak value time occupied ratio and smaller than the average occupied resource of the task, data migration is carried out; or when the first idle resource of the current node is judged to be smaller than the product of the occupied resource of the task peak value and the occupied ratio of the peak time, the node is switched; and the first idle resource of the current node is equal to the difference between the total resource of the current node and the resource occupied by the task peak value of the current node.
The communication bus mentioned in the electronic device may be a Peripheral Component Interconnect (PCI) bus, an Extended Industry Standard Architecture (EISA) bus, or the like. The communication bus may be divided into an address bus, a data bus, a control bus, etc. For ease of illustration, only one thick line is shown, but this does not mean that there is only one bus or one type of bus.
The communication interface is used for communication between the electronic equipment and other equipment.
The Memory may include a Random Access Memory (RAM) or a Non-Volatile Memory (NVM), such as at least one disk Memory. Optionally, the memory may also be at least one memory device located remotely from the processor.
The Processor may be a general-purpose Processor, including a Central Processing Unit (CPU), a Network Processor (NP), and the like; but also Digital Signal Processors (DSPs), Application Specific Integrated Circuits (ASICs), Field Programmable Gate Arrays (FPGAs) or other Programmable logic devices, discrete Gate or transistor logic devices, discrete hardware components.
The method provided by the embodiment of the invention can be applied to electronic equipment. Specifically, the electronic device may be: desktop computers, portable computers, servers, etc. Without limitation, any electronic device that can implement the present invention is within the scope of the present invention.
For the apparatus/electronic device/storage medium embodiment, since it is substantially similar to the method embodiment, the description is relatively simple, and for the relevant points, reference may be made to part of the description of the method embodiment.
Furthermore, the terms "first", "second" and "first" are used for descriptive purposes only and are not to be construed as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature. In the description of the present invention, "a plurality" means two or more unless specifically defined otherwise.
In the description herein, references to the description of the term "one embodiment," "some embodiments," "an example," "a specific example," or "some examples," etc., mean that a particular feature, structure, material, or characteristic described in connection with the embodiment or example is included in at least one embodiment or example of the invention. In this specification, the schematic representations of the terms used above are not necessarily intended to refer to the same embodiment or example. Furthermore, the particular features, structures, materials, or characteristics described may be combined in any suitable manner in any one or more embodiments or examples. Furthermore, various embodiments or examples described in this specification can be combined and combined by those skilled in the art.
While the present application has been described in connection with various embodiments, other variations to the disclosed embodiments can be understood and effected by those skilled in the art in practicing the claimed application, from a review of the drawings, the disclosure, and the appended claims. In the claims, the word "comprising" does not exclude other elements or steps, and the word "a" or "an" does not exclude a plurality. A single processor or other unit may fulfill the functions of several items recited in the claims. The mere fact that certain measures are recited in mutually different dependent claims does not indicate that a combination of these measures cannot be used to advantage.
As will be appreciated by one skilled in the art, embodiments of the present application may be provided as a method, system (device), or computer program product. Accordingly, this application may take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware aspects that may all generally be referred to herein as a "module" or "system. Furthermore, the present application may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and the like) having computer-usable program code embodied therein. A computer program stored/distributed on a suitable medium supplied together with or as part of other hardware, may also take other distributed forms, such as via the Internet or other wired or wireless telecommunication systems.
The present application is described with reference to flowchart illustrations and/or block diagrams of methods, systems (devices) and computer program products according to embodiments of the application. It will be understood that each flow and/or block of the flow diagrams and/or block diagrams, and combinations of flows and/or blocks in the flow diagrams and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
The foregoing is a more detailed description of the invention in connection with specific preferred embodiments and it is not intended that the invention be limited to these specific details. For those skilled in the art to which the invention pertains, several simple deductions or substitutions can be made without departing from the spirit of the invention, and all shall be considered as belonging to the protection scope of the invention.

Claims (8)

1. A multi-dimensional geographic space entity distribution method based on load balancing technology is characterized by comprising the following steps:
responding to a received task request command to obtain resource data required by the task according to the task request command, wherein the resource data comprises a task identifier, average occupied resources of the task, peak occupied resources of the task and peak time proportion;
when judging that the first idle resource of the current node is larger than the resource occupied by the task peak value, the current node receives and processes the task; when the first idle resource of the current node is judged to be larger than the product of the peak value occupied resource of the task and the peak value time occupied ratio and smaller than the average occupied resource of the task, data migration is carried out; or when the first idle resource of the current node is judged to be smaller than the product of the occupied resource of the task peak value and the peak value time, the node is switched; the first idle resource of the current node is equal to the difference between the total resource of the current node and the resource occupied by the task peak value of the current node;
the step of performing data migration includes:
selecting a node to be migrated;
generating a migration state request frame according to the task request command so as to obtain a load identifier and a second idle resource of the node to be migrated;
and when the load identification of the node to be migrated is different from the load identification of the current node and the second idle resource is larger than the resource occupied by the task peak value, dividing the task into a geographic data processing task and a non-format data processing task, and distributing the geographic data processing task or the non-format data processing task to the corresponding node for processing according to the load identification, wherein the load identification comprises the geographic data identification and the non-format data identification.
2. The method of claim 1, wherein selecting the nodes to be migrated comprises:
establishing a task queue set and a node load set;
sequentially acquiring a node with the minimum task queue number and a node with the minimum load value and adding the node into the corresponding task queue set or the corresponding node load set, acquiring an intersection of the task queue set and the node load set after each new element is added, and if the intersection is an empty set, continuously adding the node with the minimum task queue number and the node with the minimum load value which are not added into the set into the corresponding task queue set or the corresponding node load set; if the intersection is not empty and is only, outputting a node corresponding to the intersection; and if the intersection is not empty and not unique, selecting an optimal node according to a preset algorithm for outputting.
3. The method for distributing the multidimensional geospatial entity based on the load balancing technology as claimed in claim 2, wherein the selecting an optimal node for output according to a preset algorithm comprises:
acquiring a task queue number set and a load value set of the nodes corresponding to the intersection;
and respectively carrying out normalization processing on the number of each task queue and the load value, respectively giving corresponding weight factors to the data after the normalization processing, then calculating to obtain a node saturation index, and selecting the node with the minimum node saturation index as an optimal node for outputting.
4. The method according to claim 3, wherein the node saturation index calculation formula is:
k=α×(M-μ 1 )/ σ 1 +(1-α)×(N-μ 2 )/ σ 2 wherein M is the task queue number of the node, N is the load value of the node, mu1Is the task queue number set mean, σ1For task queue number set standard deviation, μ2Is the mean value of the load value set, σ2Is the standard deviation of the load value set and alpha is the weighting factor.
5. A multi-dimensional geospatial entity distribution system based on load balancing techniques, comprising:
the resource data acquisition module is used for responding to a received task request command to obtain resource data required by the task according to the task request command, wherein the resource data comprises a task identifier, task average occupied resources, task peak occupied resources and a peak time occupied ratio;
the data processing module is used for enabling the current node to receive and process the task when judging that the first idle resource of the current node is larger than the resource occupied by the task peak value; and the data migration module is used for carrying out data migration when the first idle resource of the current node is judged to be larger than the product of the peak value occupied resource of the task and the peak value time occupied ratio and smaller than the average occupied resource of the task; or is used for switching the node when judging that the first idle resource of the current node is smaller than the product of the occupied resource of the task peak value and the occupied ratio of the peak time; the first idle resource of the current node is equal to the difference between the total resource of the current node and the resource occupied by the task peak value of the current node;
the data processing module specifically comprises:
a migration node selection unit, configured to select a node to be migrated;
the migration state acquisition unit is used for generating a migration state request frame according to the task request command so as to acquire the load identifier and the second idle resource of the node to be migrated;
and the load migration unit is used for dividing the task into a geographic data processing task and a non-format data processing task and distributing the geographic data processing task or the non-format data processing task to the corresponding node for processing according to the load identification when the load identification of the node to be migrated is different from the load identification of the current node and the second idle resource is larger than the resource occupied by the task peak value, wherein the load identification comprises the geographic data identification and the non-format data identification.
6. The system according to claim 5, wherein the migration node selection unit specifically comprises:
the set generation subunit is used for establishing a task queue set and a node load set;
the node selection subunit is used for sequentially acquiring a node with the minimum task queue number and a node with the minimum load value, adding the node into the corresponding task queue set or the corresponding node load set, acquiring an intersection of the task queue set and the node load set after each new element addition, and if the intersection is an empty set, continuously adding the node with the minimum task queue number and the node with the minimum load value which are not added into the set into the corresponding task queue set or the corresponding node load set; if the intersection is not empty and is only, outputting a node corresponding to the intersection; and if the intersection is not empty and not unique, selecting an optimal node according to a preset algorithm to output.
7. The system according to claim 6, wherein the node selection subunit is specifically configured to obtain a task queue set and a load value set of the nodes corresponding to the intersection; and respectively carrying out normalization processing on the number of each task queue and the load value, respectively giving corresponding weight factors to the data after the normalization processing, then calculating to obtain a node saturation index, and selecting the node with the minimum node saturation index as an optimal node for outputting.
8. An electronic device is characterized by comprising a processor, a communication interface, a memory and a communication bus, wherein the processor and the communication interface are used for realizing mutual communication by the memory through the communication bus;
a memory for storing a computer program;
a processor for implementing the method steps of any of claims 1 to 4 when executing a program stored in the memory.
CN202210139643.9A 2022-02-16 2022-02-16 Multi-dimensional geographic space entity distribution method and system based on load balancing technology Active CN114201306B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202210139643.9A CN114201306B (en) 2022-02-16 2022-02-16 Multi-dimensional geographic space entity distribution method and system based on load balancing technology

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202210139643.9A CN114201306B (en) 2022-02-16 2022-02-16 Multi-dimensional geographic space entity distribution method and system based on load balancing technology

Publications (2)

Publication Number Publication Date
CN114201306A CN114201306A (en) 2022-03-18
CN114201306B true CN114201306B (en) 2022-05-20

Family

ID=80659002

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202210139643.9A Active CN114201306B (en) 2022-02-16 2022-02-16 Multi-dimensional geographic space entity distribution method and system based on load balancing technology

Country Status (1)

Country Link
CN (1) CN114201306B (en)

Families Citing this family (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN118113443A (en) * 2024-04-29 2024-05-31 苏州元脑智能科技有限公司 Task scheduling method, system, program product, device and medium

Family Cites Families (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US7155722B1 (en) * 2001-07-10 2006-12-26 Cisco Technology, Inc. System and method for process load balancing in a multi-processor environment
CN103986766B (en) * 2014-05-19 2017-07-07 中国工商银行股份有限公司 Adaptive load balancing job task dispatching method and device
US9483187B2 (en) * 2014-09-30 2016-11-01 Nimble Storage, Inc. Quality of service implementation in a networked storage system with hierarchical schedulers
CN109522129A (en) * 2018-11-23 2019-03-26 快云信息科技有限公司 A kind of resource method for dynamically balancing, device and relevant device
WO2021226062A1 (en) * 2020-05-04 2021-11-11 Intel Corporation Intelligent transport system service dissemination
CN112256438B (en) * 2020-06-28 2021-06-25 腾讯科技（深圳）有限公司 Load balancing control method and device, storage medium and electronic equipment
CN112272203B (en) * 2020-09-18 2022-06-14 苏州浪潮智能科技有限公司 Cluster service node selection method, system, terminal and storage medium

Also Published As

Publication number Publication date
CN114201306A (en) 2022-03-18

Similar Documents

Publication Publication Date Title
CN1956457B (en) Method and apparatus for arranging mesh work in mesh computing system
CN115328663B (en) Method, device, equipment and storage medium for scheduling resources based on PaaS platform
CN112153700A (en) Network slice resource management method and equipment
CN103995735A (en) Device and method for scheduling working flow
CN107220108B (en) Method and system for realizing load balance of cloud data center
CN108270805B (en) Resource allocation method and device for data processing
CN102339233A (en) Cloud computing centralized management platform
CN115033340A (en) Host selection method and related device
CN114201306B (en) Multi-dimensional geographic space entity distribution method and system based on load balancing technology
CN114666335B (en) Distributed system load balancing device based on data distribution service DDS
CN108833592A (en) Cloud host schedules device optimization method, device, equipment and storage medium
CN107203256B (en) Energy-saving distribution method and device under network function virtualization scene
CN116048773B (en) Distributed collaborative task assignment method and system based on wave function collapse
Yassir et al. Graph-based model and algorithm for minimising big data movement in a cloud environment
Tang Load balancing optimization in cloud computing based on task scheduling
Cao et al. Online cost-rejection rate scheduling for resource requests in hybrid clouds
CN115981843A (en) Task scheduling method and device in cloud-edge cooperative power system and computer equipment
CN114860449A (en) Data processing method, device, equipment and storage medium
CN114090256A (en) Application delivery load management method and system based on cloud computing
CN114741160A (en) Dynamic virtual machine integration method and system based on balanced energy consumption and service quality
CN113377866A (en) Load balancing method and device for virtualized database proxy service
Lin et al. A multi-centric model of resource and capability management in cloud simulation
CN114490008A (en) Service scheduling method, device and system and edge computing node
CN111367653A (en) Stream computing task management method
Jiang et al. Energy optimization oriented three-way clustering algorithm for cloud tasks

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant