CN116185602A - Task balanced consumption method and system and electronic equipment - Google Patents

Task balanced consumption method and system and electronic equipment Download PDF

Info

Publication number
CN116185602A
CN116185602A CN202211519762.3A CN202211519762A CN116185602A CN 116185602 A CN116185602 A CN 116185602A CN 202211519762 A CN202211519762 A CN 202211519762A CN 116185602 A CN116185602 A CN 116185602A
Authority
CN
China
Prior art keywords
task
executed
node
service
service node
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202211519762.3A
Other languages
Chinese (zh)
Inventor
金磊
欧弘涛
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Shenzhen Yicang Tech Co ltd
Original Assignee
Shenzhen Yicang Tech Co ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Shenzhen Yicang Tech Co ltd filed Critical Shenzhen Yicang Tech Co ltd
Priority to CN202211519762.3A priority Critical patent/CN116185602A/en
Publication of CN116185602A publication Critical patent/CN116185602A/en
Pending legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00—Arrangements for program control, e.g. control units
    • G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/46—Multiprogramming arrangements
    • G06F9/50—Allocation of resources, e.g. of the central processing unit [CPU]
    • G06F9/5083—Techniques for rebalancing the load in a distributed system
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00—Arrangements for program control, e.g. control units
    • G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/46—Multiprogramming arrangements
    • G06F9/50—Allocation of resources, e.g. of the central processing unit [CPU]
    • G06F9/5005—Allocation of resources, e.g. of the central processing unit [CPU] to service a request
    • G06F9/5027—Allocation of resources, e.g. of the central processing unit [CPU] to service a request the resource being a machine, e.g. CPUs, Servers, Terminals
    • G06F9/505—Allocation of resources, e.g. of the central processing unit [CPU] to service a request the resource being a machine, e.g. CPUs, Servers, Terminals considering the load
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00—Arrangements for program control, e.g. control units
    • G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/46—Multiprogramming arrangements
    • G06F9/54—Interprogram communication
    • G06F9/546—Message passing systems or structures, e.g. queues
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F2209/00—Indexing scheme relating to G06F9/00
    • G06F2209/54—Indexing scheme relating to G06F9/54
    • G06F2209/548—Queue
    • Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
    • Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
    • Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management

Landscapes

  • Engineering & Computer Science (AREA)
  • Software Systems (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)

Abstract

The invention relates to the field of data processing, and discloses a task balance consumption method, which comprises the following steps: deploying service nodes in a pre-built registry and configuring node serial numbers; inquiring the task to be executed of each service node, and calculating the task distribution quantity of all the service nodes; inquiring the current load total amount of each service node, and calculating the current difference value between the current load total amount and the task distribution amount; according to the current difference value, calculating a hash slot corresponding to the task to be executed, and identifying the association relation between the hash slot and the node serial number; generating a mapping relation table of tasks to be executed and node serial numbers according to the association relation; and determining a task execution node corresponding to each task in the tasks to be executed from the service node based on the mapping relation table, and distributing the tasks to be executed to the task execution nodes in a message queue mode so as to process the tasks to be executed and obtain a task execution result. The invention can improve the execution efficiency of the task to be executed.

Description

Task balanced consumption method and system and electronic equipment
Technical Field
The present invention relates to the field of data processing, and in particular, to a task balanced consumption method, system, and electronic device.
Background
The task balancing is based on a load balancing strategy of task scheduling, and the relative balancing of loads among the servers is realized by distributing the tasks to the resource nodes in a balanced manner, so that the time of occupying resources by the tasks is reduced, and the system resources are fully utilized to realize the execution of the tasks.
The existing task balanced consumption method generally distributes each task to a corresponding service node for consumption processing in a multithreading distribution mode, but only downstream consumption service is balanced in the mode without considering the service capacity of upstream task consumption, so that when a task center is increased along with the continuous increase of services, the problems of task blockage, poor disaster tolerance, even the collapse of the whole service system and the like are caused, and the execution efficiency of the task to be executed is greatly reduced.
Disclosure of Invention
The invention provides a method, a system and electronic equipment for balanced consumption of tasks, and mainly aims to improve the execution efficiency of tasks to be executed.
In order to achieve the above object, the present invention provides a task balance consumption method, including:
deploying at least two service nodes in a pre-built registry, and configuring a node serial number of each service node;
Inquiring the task to be executed of each service node, and calculating the task distribution amount of each service node according to the task to be executed;
inquiring the current load total amount of each service node, and calculating the current difference value between the current load total amount of each service node and the task distribution amount;
according to the current difference value, calculating a hash slot corresponding to the task to be executed, and identifying an association relationship between the hash slot and the node serial number;
determining a mapping relation between a task to be executed and the node serial number according to the association relation to generate a mapping relation table of the task to be executed and the node serial number;
and determining a task execution node corresponding to each task in the tasks to be executed from the service node based on the mapping relation table, and distributing the tasks to be executed to the task execution nodes in a message queue mode so as to process the tasks to be executed and obtain task execution results.
Optionally, the deploying at least two service nodes in a pre-built registry includes:
determining a node server and a node environment when a task is executed in a pre-built registry;
And constructing a network topology position of the node server based on the node environment, and deploying service nodes when the task is executed based on the network topology position.
Optionally, the calculating, according to the current difference, a hash slot corresponding to the task to be executed includes:
deleting the service node corresponding to the task to be executed when the current difference value is larger than a preset threshold value to obtain a deleted node, and calculating a hash slot corresponding to the task to be executed by utilizing a hash slot algorithm according to the deleted node to obtain a hash slot;
and adding the service node corresponding to the task to be executed when the current difference value is not greater than the preset threshold value to obtain an added node, and calculating a hash slot corresponding to the task to be executed by utilizing a hash slot algorithm according to the added node to obtain a hash slot.
Optionally, the Ha Xicao algorithm includes:
s＝crc16(key)％h
wherein s represents a hash slot, crc16 represents a verification algorithm, key represents task information of a task to be executed, h represents a current load total amount of a corresponding deletion node or an addition node of the task to be executed, and% represents a modulo symbol.
Optionally, the identifying determines a mapping relationship between the task to be executed and the node serial number according to the association relationship, including:
Calculating the mapping relation between the task to be executed and the node serial number by using the following formula:
Figure BDA0003973297570000021
wherein, g is T-N representing the mapping relation between the task set T to be executed and the node serial number set N, T j Represents the j-th task to be executed in the task set to be executed, k represents the node serial number, if represents the conditional function, s j Representing the hash slot corresponding to the j-th task to be executed, q represents the hash slot value, and n i-1 Representing the current load total of the i-1 th service node corresponding to the node serial number, and sigma represents the summation symbol.
Optionally, the determining, based on the mapping table, a task execution node corresponding to each task in the tasks to be executed from the service node includes:
inquiring service nodes corresponding to each task in the tasks to be executed based on the mapping relation table;
according to the service node, inquiring a task execution node type of the service node, and matching each task in the tasks to be executed with the task execution node type to obtain a matching result;
when the matching result is successful, distributing each task in the tasks to be executed to a task execution node with matched types corresponding to the service node;
And when the matching result is unsuccessful, calculating the similarity between each task in the tasks to be executed and the key value information of the task execution node type, and taking the task execution node with the maximum similarity as the task execution node corresponding to each task in the tasks to be executed.
Optionally, the distributing the task to be executed to the task execution node by adopting a message queue mode includes:
configuring a distribution mode of the task to be executed between the message queue and the service node;
configuring a distribution path of the task to be executed between the message queue and the service node;
and distributing the task to be executed to the task execution node according to the distribution mode and the distribution path.
Optionally, the configuring a distribution manner of the task to be executed between the message queue and the service node includes:
respectively acquiring communication ports of the task to be executed in the message queue and the service node so as to respectively extract a message queue communication protocol and a service node communication protocol;
identifying communication difference parameters in the message queue communication protocol and the service node communication protocol, and converting the communication difference parameters into communication synchronization parameters by using a communication protocol unit;
And generating a synchronization protocol of the message queue and the service node according to the communication synchronization parameter, and taking the synchronization protocol as a distribution mode of the task to be executed between the message queue and the service node.
In order to solve the above problems, the present invention also provides a task balance consumption system, the system comprising:
the service node deployment module is used for deploying at least two service nodes in a pre-built registry and configuring the node serial number of each service node;
the task distribution amount calculating module is used for inquiring the task to be executed of each service node and calculating the task distribution amount of each service node according to the task to be executed;
the task difference calculation module is used for inquiring the current load total amount of each service node and calculating the current difference between the current load total amount of each service node and the task distribution amount;
the association relation identification module is used for calculating a hash slot position corresponding to the task to be executed according to the current difference value and identifying the association relation between the hash slot position and the node serial number;
the mapping relation table generation module is used for determining the mapping relation between the task to be executed and the node serial number according to the association relation so as to generate a mapping relation table of the task to be executed and the node serial number;
And the task processing module to be executed is used for determining a task execution node corresponding to each task in the tasks to be executed from the service node based on the mapping relation table, and distributing the tasks to be executed to the task execution nodes in a message queue mode so as to process the tasks to be executed and obtain task execution results.
In order to solve the above-mentioned problems, the present invention also provides an electronic apparatus including:
at least one processor; the method comprises the steps of,
a memory communicatively coupled to the at least one processor; wherein,,
the memory stores a computer program executable by the at least one processor to implement the task balance consumption method described above.
It can be seen that, in the embodiment of the present invention, by deploying at least two service nodes in a pre-built registry, a service object can be provided for subsequent task execution, and the service quality and the operation efficiency of the overlay network will be directly affected, and the node serial numbers of the service nodes are configured to distribute the tasks to corresponding node serial numbers for the subsequent tasks, so as to improve the task distribution efficiency, the service type and the task distribution number of the tasks to be executed of each service node can be determined by querying the task to be executed of each service node, so as to distribute the tasks to the corresponding task execution node for the subsequent tasks, and the task distribution amount of each service node can be calculated to compare the current load total amount with the current difference value of the task distribution amount for the subsequent tasks, so as to make a corresponding service node adjustment scheme; secondly, the embodiment of the invention can provide a calculation basis for subsequently calculating the current load total of all service nodes by inquiring the current load total of each service node, calculates the current difference value of the current load total of each service node and the task distribution amount to be used as a judgment object of the subsequent adding and deleting service nodes, and can provide a guarantee for subsequently establishing the corresponding relation between the task to be executed and the service nodes by performing condition judgment according to the current difference value, calculating the hash slot positions corresponding to the task to be executed, self-adaptively adjusting the number of the service nodes and distributing the hash slot positions of the task to be executed by performing condition judgment according to the current difference value and identifying the association relation between the hash slot positions and the node serial numbers; further, according to the embodiment of the invention, the mapping relation between the tasks to be executed and the node serial numbers is determined according to the association relation, so that a mapping relation table of the tasks to be executed and the node serial numbers is generated, so that an implementation rule is provided for distributing the tasks to be executed to the service nodes corresponding to the node serial numbers, based on the mapping relation table, the task execution node corresponding to each task in the tasks to be executed is determined from the service nodes, the task execution node corresponding to each task in the tasks to be executed can be distributed to the task execution node corresponding to the service node with the mapping relation in a one-to-one manner, so that task execution is realized, the tasks to be executed are distributed to the task execution nodes in a message queue mode, so that the task execution stability and execution efficiency of services are improved, task processing is performed on the tasks to be executed, and final task execution processing can be performed on the tasks to be executed to obtain a final task execution result, so that the intention is realized. Therefore, the task balance consumption method provided by the embodiment of the invention can improve the execution efficiency of the task to be executed.
Drawings
FIG. 1 is a flow chart of a task balance consumption method according to an embodiment of the present invention;
FIG. 2 is a schematic block diagram of a task balancing consumption system according to an embodiment of the present invention;
fig. 3 is a schematic diagram of an internal structure of an electronic device for implementing a task balancing consumption method according to an embodiment of the present invention;
the achievement of the objects, functional features and advantages of the present invention will be further described with reference to the accompanying drawings, in conjunction with the embodiments.
Detailed Description
It should be understood that the specific embodiments described herein are for purposes of illustration only and are not intended to limit the scope of the invention.
The embodiment of the invention provides a task balanced consumption method. The execution subject of the task balance consumption method includes, but is not limited to, at least one of a server, a terminal, and the like, which can be configured to execute the method provided by the embodiment of the invention. In other words, the task balancing consumption method may be performed by software or hardware installed in a terminal device or a server device, and the software may be a blockchain platform. The service end includes but is not limited to: a single server, a server cluster, a cloud server or a cloud server cluster, and the like. The server may be an independent server, or may be a cloud server that provides cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, content delivery networks (Content Delivery Network, CDN), and basic cloud computing services such as big data and artificial intelligence platforms.
Referring to fig. 1, a flow chart of a task balance consumption method according to an embodiment of the invention is shown. In the embodiment of the invention, the task balance consumption method comprises the following steps:
s1, at least two service nodes are deployed in a pre-built registration center, and node serial numbers of each service node are configured.
The embodiment of the invention can provide service objects for subsequent task execution by arranging at least two service nodes in the pre-built registration center, and can directly influence the service quality and the operation efficiency of the overlay network. The registry is an address book in the micro-service architecture, which records the mapping relation between the service and the service address, and in the distributed architecture, the service registers to the address book, and when the service needs to call other services, the service finds the address of other services to call, and the address book comprises the registry types such as Zookeeper, eureka, nacos, consul, etc. The service nodes are usually connected to the synthesis node or the task node, so long as the branches are executed, the branches are executed with defined frequency and are commonly used for checking and updating blackboard, and replace the traditional parallel nodes in other behavior tree systems, if the task to be executed is order payment of network platform second killing activity, the number of initially deployed service nodes needs to be dynamically adjusted according to the number of payment orders, a plurality of task nodes are connected under each service node for specifically executing the task to be executed, the task nodes can provide services of different types of tasks, in detail, if 100 tens of thousands of orders to be paid exist currently, 2 service nodes are initially deployed in a registry, but the total task load of the 2 service nodes is only 50 ten thousand, the current task to be executed cannot be satisfied, and then new service nodes need to be added to satisfy the task to be executed. The blackboard is used for storing and reading and writing all data needed by one AI instance, and has event notifications such as ValueChange and the like, which are usually used in a binding way with a behavior tree.
Further, as an embodiment of the present invention, the deploying at least two service nodes in a pre-built registry includes: determining a node server and a node environment when a task is executed in a pre-built registry; and constructing a network topology position of the node server based on the node environment, and deploying service nodes when the task is executed based on the network topology position.
The node environment is a built-in system bottom environment when the task is executed, and services such as a development language SDK, a command line tool and the like are preloaded. The network topology location refers to the distribution condition and connection state of devices between networks composed of computers, such as IP addresses, ports, calling modes, etc.
Further, according to the embodiment of the invention, the task can be distributed to the corresponding node serial number for the follow-up through the node serial number of the service node and the configuration, so that the task distribution efficiency is improved. Wherein the node serial number refers to a unique node identification code.
Further, as an embodiment of the present invention, the node serial number of the service node is configured and includes: acquiring node information of the service nodes, and calculating the node number of the service nodes; calculating a hash modular value of the service node by utilizing a hash modulo algorithm according to the node information and the node quantity; and determining the node serial number of the service node according to the hash module value.
Further, as an embodiment of the present invention, the hash modulo algorithm includes:
m＝Hash(key)％n
wherein m represents a Hash modulus, hash represents a Hash function, key represents node information of service nodes, n represents the number of nodes of the service nodes, and% represents a modulo symbol.
Further, as an embodiment of the present invention, the determining the node serial number of the service node according to the hash module value includes: generating a hash module value sequence according to the hash module value; sequentially configuring the repeated hash module values in the hash module value sequence as the nearest sequence numbers according to the sequence order until the hash module value sequence has no repeated sequence numbers; and determining the node serial number of the service node corresponding to the serial number of the hash module value sequence according to the hash module value sequence.
S2, inquiring the task to be executed of each service node, and calculating the task distribution amount of each service node according to the task to be executed.
According to the embodiment of the invention, the service type and the task distribution quantity of the tasks to be executed can be determined by inquiring the tasks to be executed of each service node so as to distribute the tasks to the corresponding task execution nodes for follow-up. The task execution node refers to a caller in a calling relation between micro services in a distributed system constructed by the micro services. The task to be executed refers to a consumption request task initiated by a client, such as a user initiating a ticket buying request to a 12306 ticketing system, and a commodity is ordered to wait for executing the task.
Further, as an embodiment of the present invention, the querying the task to be performed of each service node may be implemented by querying a service management center for the task to be performed to be distributed to the service node, where the task to be performed includes a task type and a task number. The service management center is used for effectively collecting, storing, processing and applying data among the micro service nodes by utilizing computer hardware and software technology, and sending various control instructions to the equipment according to service requirements so as to enable other equipment to execute the instructions and feed back data interaction of the results.
Further, according to the embodiment of the invention, the task distribution amount of each service node is calculated according to the task to be executed, and the current difference value between the current load total amount and the task distribution amount is compared later, so that a corresponding service node adjustment scheme is made. The task distribution amount refers to the number of tasks for sending the issued tasks to be processed to be executed to the service node to acquire a consumption result.
Further, as an embodiment of the present invention, the calculating the task distribution amount of each service node according to the task to be performed may be obtained by accumulating the number of tasks of each service node for the task to be performed.
S3, inquiring the current load total amount of each service node, and calculating the current difference value between the current load total amount of each service node and the task distribution amount;
according to the method and the device for inquiring the current load total of each service node, a calculation basis can be provided for the subsequent calculation of the current difference value between the task distribution quantity and the current load total of each service node.
Further, as an embodiment of the present invention, the querying the current load total of each service node may be obtained by querying in a pre-built registry.
Further, according to the embodiment of the invention, the current difference between the current load total of each service node and the task distribution amount can be used as a judging object of the subsequent adding and deleting service nodes.
Further, as an embodiment of the present invention, the calculating the current difference between the current load total of each service node and the task distribution amount may obtain the current difference between the current load total and the task distribution amount by subtracting the current load total and the task distribution amount.
S4, calculating a hash slot corresponding to the task to be executed according to the current difference value, and identifying the association relation between the hash slot and the node serial number.
According to the embodiment of the invention, the hash slot corresponding to the task to be executed is calculated according to the current difference value, the number of the service nodes can be adjusted in a self-adaptive mode by making condition judgment according to the current difference value, and the hash slot of the task to be executed is distributed.
Further, as an embodiment of the present invention, the calculating, according to the current difference, a hash slot corresponding to the task to be executed includes: deleting the service node corresponding to the task to be executed when the current difference value is larger than a preset threshold value to obtain a deleted node, and calculating a hash slot corresponding to the task to be executed by utilizing a hash slot algorithm according to the deleted node to obtain a hash slot; and adding the service node corresponding to the task to be executed when the current difference value is not greater than the preset threshold value to obtain an added node, and calculating a hash slot corresponding to the task to be executed by utilizing a hash slot algorithm according to the added node to obtain a hash slot. The preset threshold may be set to 0, or may be set according to an actual service scenario.
Further, in an alternative embodiment of the present invention, the Ha Xicao algorithm includes:
s＝crc16(key)％h
Wherein s represents a hash slot, crc16 represents a verification algorithm, key represents task information of a task to be executed, h represents a current load total amount of a corresponding deletion node or an addition node of the task to be executed, and% represents a modulo symbol.
Further, according to the method and the system for identifying the association relationship between the hash slot and the node serial number, the association relationship between the hash slot and the node serial number is identified, so that guarantee is provided for the subsequent establishment of the corresponding relationship between the task to be executed and the service node.
Further, in an optional embodiment of the present invention, the identifying the association relationship between the hash slot and the node serial number may be implemented by using the following formula, including:
Figure BDA0003973297570000081
wherein, g is T-N representing the mapping relation between the task set T to be executed and the node serial number set N, T j Represents the j-th task to be executed in the task set to be executed, k represents the node serial number, if represents the conditional function, s j Representing the hash slot corresponding to the j-th task to be executed, q represents the hash slot value, and n i-1 Representing the current load total of the i-1 th service node corresponding to the node serial number, and sigma represents the summation symbol.
S5, determining the mapping relation between the task to be executed and the node serial number according to the association relation so as to generate a mapping relation table of the task to be executed and the node serial number.
According to the embodiment of the invention, the mapping relation between the task to be executed and the node serial number is determined according to the association relation, so that a mapping relation table of the task to be executed and the node serial number is generated, and an implementation rule is provided for distributing the task to be executed to the service node corresponding to the node serial number.
Further, as an embodiment of the present invention, the determining, according to the association relationship, a mapping relationship between the task to be executed and the node serial number includes:
calculating the mapping relation between the task to be executed and the node serial number by using the following formula:
Figure BDA0003973297570000082
wherein, g is T-N representing the mapping relation between the task set T to be executed and the node serial number set N, T j Represents the j-th task to be executed in the task set to be executed, k represents the node serial number, s j Representing the j-th task pair to be performedThe corresponding hash slot, q represents the hash slot value, n i-1 Representing the current load total of the i-1 th service node corresponding to the node serial number, and sigma represents the summation symbol.
Further, as an embodiment of the present invention, the generating a mapping relationship table between the task to be executed and the node serial number may be generated according to a mapping relationship between the task to be executed and the node serial number and a data table structure. The data table is a data set of a two-dimensional table structure formed by fixed columns and arbitrary rows, wherein the columns in the data table are called fields, the rows are called records, and the fields are taken as basic storage units and calculation units.
S6, determining task execution nodes corresponding to each task in the tasks to be executed from the service nodes based on the mapping relation table, and distributing the tasks to be executed to the task execution nodes in a message queue mode so as to process the tasks to be executed and obtain task execution results.
According to the embodiment of the invention, the task execution node corresponding to each task in the tasks to be executed can be determined from the service node based on the mapping relation table, so that one-to-one load balancing of each task in the tasks to be executed can be distributed to the task execution node corresponding to the service node with the mapping relation, and the task execution can be realized.
Further, as an embodiment of the present invention, the determining, from the service node, a task execution node corresponding to each of the tasks to be executed based on the mapping relation table includes: inquiring service nodes corresponding to each task in the tasks to be executed based on the mapping relation table; according to the service node, inquiring a task execution node type of the service node, and matching each task in the tasks to be executed with the task execution node type to obtain a matching result; when the matching result is successful, distributing each task in the tasks to be executed to a task execution node with matched types corresponding to the service node; and when the matching result is unsuccessful, calculating the similarity between each task in the tasks to be executed and the key value information of the task execution node type, and taking the task execution node with the maximum similarity as the task execution node corresponding to each task in the tasks to be executed. The task execution node type is a node type set according to a specific service scenario, for example, the task execution node type may be a payment collection type, a query service type, an entry service type, etc., and generally, the configuration performance, the load capacity, the computing processing capacity, etc. of the hardware device for matching are different for different node types.
Further, in the embodiment of the invention, the task to be executed is distributed to the task execution nodes in a message queue mode so as to realize the load-balanced distribution of the task to be executed to each task execution node, thereby improving the stability and the execution efficiency of the service. The message queue refers to a container for storing the message in the transmission process of the message, is an important component in a distributed system, mainly solves the problems of application coupling, asynchronous processing, flow cutting and the like, and comprises a RabbitMQ, rocketMQ, activeMQ, kafka, zeroMQ, metaMq message queue, wherein the message queue is needed under the scene that the concurrence quantity needs to be controlled, such as investigation and registration, commodity purchase and the like.
Further, as an embodiment of the present invention, the distributing the task to be executed to the task execution node by using a message queue includes: configuring a distribution mode of the task to be executed between the message queue and the service node; configuring a distribution path of the task to be executed between the message queue and the service node; and distributing the task to be executed to the task execution node according to the distribution mode and the distribution path.
Further, as an embodiment of the present invention, the configuring a distribution manner of the task to be performed between the message queue and the service node includes: respectively acquiring communication ports of the task to be executed in the message queue and the service node so as to respectively extract a message queue communication protocol and a service node communication protocol; identifying communication difference parameters in the message queue communication protocol and the service node communication protocol, and converting the communication difference parameters into communication synchronization parameters by using a communication protocol unit; and generating a synchronization protocol of the message queue and the service node according to the communication synchronization parameter, and taking the synchronization protocol as a distribution mode of the task to be executed between the message queue and the service node.
The communication ports refer to interfaces connected between communication devices, such as RJ-45 ports, serial ports and the like. The port parameters are generalized digital metrics describing the overall characteristics of the communication port, such as data format, synchronization pattern, baud rate, transfer step, error detection pattern, and control character definition. The communication protocol refers to rules and conventions that must be followed by the two parties to complete a communication or service. The communication difference parameter refers to different communication parameters adopted by the message queue and the service node, such as an RJ-45 port parameter is adopted for the message queue, a Serial port parameter is adopted for the service node, and the communication protocol unit is a converter for converting the communication difference parameter so that the corresponding message queue and the service node have the same communication parameter, and the converter comprises a communication protocol converter, such as a Serial port protocol converter.
Further, as an embodiment of the present invention, the configuring the distribution path of the task to be performed between the message queue and the service node includes: constructing a physical transmission medium between the message queue and the service node; constructing a logic transmission rule between the message queue and the service node; and constructing a communication link of the task to be executed between the message queue and the service node according to the physical transmission medium and the logic transmission rule, and taking the communication link as a distribution path of the task to be executed between the message queue and the service node.
Further, in the embodiment of the invention, the task execution result is obtained by performing task processing on the task to be executed, and the task to be executed can be subjected to final task execution processing to obtain a final task execution result.
Further, as an embodiment of the present invention, performing task processing on the task to be executed based on the final task execution node to obtain a task execution result, including: and acquiring a data request of a task to be executed, and performing task processing on the data request according to the business logic of the final task execution node to obtain a task execution result. The business logic refers to rules and procedures that one entity unit should possess in order to provide services to another entity unit.
It can be seen that, in the embodiment of the present invention, by deploying at least two service nodes in a pre-built registry, a service object can be provided for subsequent task execution, and the service quality and the operation efficiency of the overlay network will be directly affected, and the node serial numbers of the service nodes are configured to distribute the tasks to corresponding node serial numbers for the subsequent tasks, so as to improve the task distribution efficiency, the service type and the task distribution number of the tasks to be executed of each service node can be determined by querying the task to be executed of each service node, so as to distribute the tasks to the corresponding task execution node for the subsequent tasks, and the task distribution amount of each service node can be calculated to compare the current load total amount with the current difference value of the task distribution amount for the subsequent tasks, so as to make a corresponding service node adjustment scheme; secondly, the embodiment of the invention can provide a calculation basis for subsequently calculating the current load total of all service nodes by inquiring the current load total of each service node, calculates the current difference value of the current load total of each service node and the task distribution amount to be used as a judgment object of the subsequent adding and deleting service nodes, and can provide a guarantee for subsequently establishing the corresponding relation between the task to be executed and the service nodes by performing condition judgment according to the current difference value, calculating the hash slot positions corresponding to the task to be executed, self-adaptively adjusting the number of the service nodes and distributing the hash slot positions of the task to be executed by performing condition judgment according to the current difference value and identifying the association relation between the hash slot positions and the node serial numbers; further, in the embodiment of the present invention, a mapping relationship between a task to be executed and the node serial number is determined according to the association relationship, so as to generate a mapping relationship table of the task to be executed and the node serial number, so as to provide an implementation rule for subsequently distributing the task to be executed to a service node corresponding to the node serial number, based on the mapping relationship table, a task execution node corresponding to each task in the task to be executed is determined from the service node, then each task in the task to be executed can be distributed to a task execution node corresponding to the service node with a mapping relationship in a one-to-one manner, so as to implement task execution, based on the task execution node, the task to be executed is distributed to the task execution node in a message queue manner, so as to implement load-balanced distribution of the task to be executed to each task execution node, further improve service stability and execution efficiency, perform task processing on the task to be executed, and finally perform task execution processing on the task to be executed to obtain a task execution result, so as to obtain a final task execution intention and implement the problem. Therefore, the task balance consumption method provided by the embodiment of the invention can improve the execution efficiency of the task to be executed.
As shown in fig. 2, a functional block diagram of the task balance consuming system of the present invention is shown.
The task balanced consumer system 100 of the present invention may be installed in an electronic device. The task balance consumption system may include a service node deployment module 101, a task distribution amount calculation module 102, a mapping relation table construction module 103, and a task processing module to be executed 104 according to the implemented functions. The module according to the invention, which may also be referred to as a unit, refers to a series of computer program segments, which are stored in the memory of the electronic device, capable of being executed by the processor of the electronic device and of performing a fixed function.
In the present embodiment, the functions concerning the respective modules/units are as follows:
a service node deployment module 101, configured to deploy at least two service nodes in a pre-built registry, and configure a node serial number of each service node;
the task distribution amount calculation module 102 is configured to query a task to be executed of each service node, and calculate a task distribution amount of each service node according to the task to be executed;
a task difference calculation module 103, configured to query a current load total amount of each service node, and calculate a current difference between the current load total amount of each service node and the task distribution amount;
The association relationship identifying module 104 is configured to calculate a hash slot corresponding to the task to be executed according to the current difference value, and identify an association relationship between the hash slot and the node serial number;
the mapping relation table generating module 105 is configured to determine a mapping relation between a task to be executed and the node serial number according to the association relation, so as to generate a mapping relation table between the task to be executed and the node serial number;
and the task to be executed processing module 106 is configured to determine, from the service node, a task execution node corresponding to each task in the tasks to be executed based on the mapping relation table, and distribute the tasks to be executed to the task execution nodes in a message queue manner, so as to process the tasks to be executed, and obtain a task execution result.
In detail, the modules in the task balance consumption system 100 in the embodiment of the present invention use the same technical means as the task balance consumption method described in fig. 1 and can produce the same technical effects, which are not described herein.
Fig. 3 is a schematic structural diagram of an electronic device 1 for implementing a task balance consumption method according to the present invention.
The electronic device 1 may comprise a processor 10, a memory 11, a communication bus 12 and a communication interface 13, and may further comprise a computer program, such as a task execution program, stored in the memory 11 and executable on the processor 10.
The processor 10 may be formed by an integrated circuit in some embodiments, for example, a single packaged integrated circuit, or may be formed by a plurality of integrated circuits packaged with the same function or different functions, including one or more central processing units (Central Processing unit, CPU), a microprocessor, a digital processing chip, a graphics processor, a combination of various control chips, and so on. The processor 10 is a Control Unit (Control Unit) of the electronic device 1, connects respective components of the entire electronic device 1 using various interfaces and lines, executes or executes programs or modules (e.g., execution of task execution programs, etc.) stored in the memory 11, and invokes data stored in the memory 11 to execute various functions of the electronic device 1 and process data.
The memory 11 includes at least one type of readable storage medium including flash memory, a removable hard disk, a multimedia card, a card type memory (e.g., SD or DX memory, etc.), a magnetic memory, a magnetic disk, an optical disk, etc. The memory 11 may in some embodiments be an internal storage unit of the electronic device 1, such as a removable hard disk of the electronic device 1. The memory 11 may in other embodiments also be an external storage device of the electronic device 1, such as a plug-in mobile hard disk, a Smart Media Card (SMC), a Secure Digital (SD) Card, a Flash memory Card (Flash Card) or the like, which are provided on the electronic device 1. Further, the memory 11 may also include both an internal storage unit and an external storage device of the electronic device 1. The memory 11 may be used not only for storing application software installed in the electronic device 1 and various types of data, such as codes of task execution programs, but also for temporarily storing data that has been output or is to be output.
The communication bus 12 may be a peripheral component interconnect standard (peripheral component interconnect, PCI) bus, or an extended industry standard architecture (extended industry standard architecture, EISA) bus, among others. The bus may be classified as an address bus, a data bus, a control bus, etc. The bus is arranged to enable a connection communication between the memory 11 and at least one processor 10 etc.
The communication interface 13 is used for communication between the electronic device 1 and other devices, including a network interface and an employee interface. Optionally, the network interface may comprise a wired interface and/or a wireless interface (e.g. WI-FI interface, bluetooth interface, etc.), typically used to establish a communication connection between the electronic device 1 and other electronic devices 1. The employee interface may be a Display (Display), an input unit such as a Keyboard (Keyboard), or alternatively a standard wired interface, a wireless interface. Alternatively, in some embodiments, the display may be an LED display, a liquid crystal display, a touch-sensitive liquid crystal display, an OLED (Organic Light-Emitting Diode) touch, or the like. The display may also be referred to as a display screen or display unit, as appropriate, for displaying information processed in the electronic device 1 and for displaying a visual staff interface.
Fig. 3 shows only an electronic device 1 with components, it being understood by a person skilled in the art that the structure shown in fig. 3 does not constitute a limitation of the electronic device 1, and may comprise fewer or more components than shown, or may combine certain components, or may be arranged in different components.
For example, although not shown, the electronic device 1 may further include a power source (such as a battery) for supplying power to the respective components, and preferably, the power source may be logically connected to the at least one processor 10 through a power management system, so as to perform functions of charge management, discharge management, and power consumption management through the power management system. The power supply may also include one or more of any of a direct current or alternating current power supply, a recharging system, a power failure detection circuit, a power converter or inverter, a power status indicator, and the like. The electronic device 1 may further include various sensors, bluetooth modules, wi-Fi modules, etc., which will not be described herein.
It should be understood that the embodiments described are for illustrative purposes only and are not limited in scope by this configuration.
The task execution program stored in the memory 11 in the electronic device 1 is a combination of a plurality of computer programs, which when run in the processor 10, can realize:
Deploying at least two service nodes in a pre-built registry, and configuring a node serial number of each service node;
inquiring the task to be executed of each service node, and calculating the task distribution amount of each service node according to the task to be executed;
inquiring the current load total amount of each service node, and calculating the current difference value between the current load total amount of each service node and the task distribution amount;
according to the current difference value, calculating a hash slot corresponding to the task to be executed, and identifying an association relationship between the hash slot and the node serial number;
determining a mapping relation between a task to be executed and the node serial number according to the association relation to generate a mapping relation table of the task to be executed and the node serial number;
and determining a task execution node corresponding to each task in the tasks to be executed from the service node based on the mapping relation table, and distributing the tasks to be executed to the task execution nodes in a message queue mode so as to process the tasks to be executed and obtain task execution results.
In particular, the specific implementation method of the processor 10 on the computer program may refer to the description of the relevant steps in the corresponding embodiment of fig. 1, which is not repeated herein.
Further, the integrated modules/units of the electronic device 1 may be stored in a non-volatile computer readable storage medium if implemented in the form of software functional units and sold or used as a stand alone product. The computer readable storage medium may be volatile or nonvolatile. For example, the computer readable medium may include: any entity or system capable of carrying the computer program code, a recording medium, a U disk, a removable hard disk, a magnetic disk, an optical disk, a computer Memory, a Read-Only Memory (ROM).
The present invention also provides a computer readable storage medium storing a computer program which, when executed by a processor of an electronic device 1, may implement:
deploying at least two service nodes in a pre-built registry, and configuring a node serial number of each service node;
inquiring the task to be executed of each service node, and calculating the task distribution amount of each service node according to the task to be executed;
inquiring the current load total amount of each service node, and calculating the current difference value between the current load total amount of each service node and the task distribution amount;
According to the current difference value, calculating a hash slot corresponding to the task to be executed, and identifying an association relationship between the hash slot and the node serial number;
determining a mapping relation between a task to be executed and the node serial number according to the association relation to generate a mapping relation table of the task to be executed and the node serial number;
and determining a task execution node corresponding to each task in the tasks to be executed from the service node based on the mapping relation table, and distributing the tasks to be executed to the task execution nodes in a message queue mode so as to process the tasks to be executed and obtain task execution results.
In the several embodiments provided by the present invention, it should be understood that the disclosed apparatus, system and method may be implemented in other manners. For example, the system embodiments described above are merely illustrative, e.g., the division of the modules is merely a logical function division, and other manners of division may be implemented in practice.
The modules described as separate components may or may not be physically separate, and components shown as modules may or may not be physical units, may be located in one place, or may be distributed over multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment.
In addition, each functional module in the embodiments of the present invention may be integrated in one processing unit, or each unit may exist alone physically, or two or more units may be integrated in one unit. The integrated units can be realized in a form of hardware or a form of hardware and a form of software functional modules.
It will be evident to those skilled in the art that the invention is not limited to the details of the foregoing illustrative embodiments, and that the present invention may be embodied in other specific forms without departing from the spirit or essential characteristics thereof.
The present embodiments are, therefore, to be considered in all respects as illustrative and not restrictive, the scope of the invention being indicated by the appended claims rather than by the foregoing description, and all changes which come within the meaning and range of equivalency of the claims are therefore intended to be embraced therein. Any reference signs in the claims shall not be construed as limiting the claim concerned.
The embodiment of the invention can acquire and process the related data based on the artificial intelligence technology. Among these, artificial intelligence (Artificial Intelligence, AI) is the theory, method, technique and application system that uses a digital computer or a digital computer-controlled machine to simulate, extend and extend human intelligence, sense the environment, acquire knowledge and use knowledge to obtain optimal results.
Furthermore, it is evident that the word "comprising" does not exclude other elements or steps, and that the singular does not exclude a plurality. Multiple units or systems as set forth in the system claims may also be implemented by means of one unit or system in software or hardware. The terms second, etc. are used to denote a name, but not any particular order.
Finally, it should be noted that the above-mentioned embodiments are merely for illustrating the technical solution of the present invention and not for limiting the same, and although the present invention has been described in detail with reference to the preferred embodiments, it should be understood by those skilled in the art that modifications and equivalents may be made to the technical solution of the present invention without departing from the spirit and scope of the technical solution of the present invention.

Claims (10)

1. A method of task balanced consumption, the method comprising:
deploying at least two service nodes in a pre-built registry, and configuring a node serial number of each service node;
inquiring the task to be executed of each service node, and calculating the task distribution amount of each service node according to the task to be executed;
inquiring the current load total amount of each service node, and calculating the current difference value between the current load total amount of each service node and the task distribution amount;
According to the current difference value, calculating a hash slot corresponding to the task to be executed, and identifying an association relationship between the hash slot and the node serial number;
determining a mapping relation between a task to be executed and the node serial number according to the association relation to generate a mapping relation table of the task to be executed and the node serial number;
and determining a task execution node corresponding to each task in the tasks to be executed from the service node based on the mapping relation table, and distributing the tasks to be executed to the task execution nodes in a message queue mode so as to process the tasks to be executed and obtain task execution results.
2. The task balanced consumption method according to claim 1, wherein the deploying at least two service nodes in a pre-built registry comprises:
determining a node server and a node environment when a task is executed in a pre-built registry;
and constructing a network topology position of the node server based on the node environment, and deploying service nodes when the task is executed based on the network topology position.
3. The task balance consumption method of claim 1, wherein the calculating the hash slot corresponding to the task to be executed according to the current difference value includes:
Deleting the service node corresponding to the task to be executed when the current difference value is larger than a preset threshold value to obtain a deleted node, and calculating a hash slot corresponding to the task to be executed by utilizing a hash slot algorithm according to the deleted node to obtain a hash slot;
and adding the service node corresponding to the task to be executed when the current difference value is not greater than the preset threshold value to obtain an added node, and calculating a hash slot corresponding to the task to be executed by utilizing a hash slot algorithm according to the added node to obtain a hash slot.
4. The task balanced consumption method according to claim 3, wherein the Ha Xicao algorithm comprises:
s＝crc16(key)％h
wherein s represents a hash slot, crc16 represents a verification algorithm, key represents task information of a task to be executed, h represents a current load total amount of a corresponding deletion node or an addition node of the task to be executed, and% represents a modulo symbol.
5. The task balancing consumption method according to claim 1, wherein the determining, according to the association relationship, a mapping relationship between the task to be executed and the node serial number includes:
calculating the mapping relation between the task to be executed and the node serial number by using the following formula:
Figure FDA0003973297560000021
and n i-1 ＝0and k≥0
Wherein, g is T-N representing the mapping relation between the task set T to be executed and the node serial number set N, T j Represents the j-th task to be executed in the task set to be executed, k representsNode sequence number, if, represents a conditional function, s j Representing a hash slot table corresponding to the j-th task to be executed, q represents a hash slot value, n i-1 Representing the current load total of the i-1 th service node corresponding to the node serial number, and sigma represents the summation symbol.
6. The task balance consumption method of claim 1, wherein determining, from the service node, a task execution node corresponding to each of the tasks to be executed based on the mapping relation table, includes:
inquiring service nodes corresponding to each task in the tasks to be executed based on the mapping relation table;
according to the service node, inquiring a task execution node type of the service node, and matching each task in the tasks to be executed with the task execution node type to obtain a matching result;
when the matching result is successful, distributing each task in the tasks to be executed to a task execution node with matched types corresponding to the service node;
and when the matching result is unsuccessful, calculating the similarity between each task in the tasks to be executed and the key value information of the task execution node type, and taking the task execution node with the maximum similarity as the task execution node corresponding to each task in the tasks to be executed.
7. The task balance consuming method of claim 1, wherein the distributing the task to be performed to the task execution node by means of a message queue comprises:
configuring a distribution mode of the task to be executed between the message queue and the service node;
configuring a distribution path of the task to be executed between the message queue and the service node;
and distributing the task to be executed to the task execution node according to the distribution mode and the distribution path.
8. The task balance consuming method of claim 7, wherein the configuring the distribution manner of the task to be performed between the message queue and the service node comprises:
respectively acquiring communication ports of the task to be executed in the message queue and the service node so as to respectively extract a message queue communication protocol and a service node communication protocol;
identifying communication difference parameters in the message queue communication protocol and the service node communication protocol, and converting the communication difference parameters into communication synchronization parameters by using a communication protocol unit;
and generating a synchronization protocol of the message queue and the service node according to the communication synchronization parameter, and taking the synchronization protocol as a distribution mode of the task to be executed between the message queue and the service node.
9. A task balanced consumption system, the system comprising:
the service node deployment module is used for deploying at least two service nodes in a pre-built registry and configuring the node serial number of each service node;
the task distribution amount calculating module is used for inquiring the task to be executed of each service node and calculating the task distribution amount of each service node according to the task to be executed;
the task difference calculation module is used for inquiring the current load total amount of each service node and calculating the current difference between the current load total amount of each service node and the task distribution amount;
the association relation identification module is used for calculating a hash slot position corresponding to the task to be executed according to the current difference value and identifying the association relation between the hash slot position and the node serial number;
the mapping relation table generation module is used for determining the mapping relation between the task to be executed and the node serial number according to the association relation so as to generate a mapping relation table of the task to be executed and the node serial number;
and the task processing module to be executed is used for determining a task execution node corresponding to each task in the tasks to be executed from the service node based on the mapping relation table, and distributing the tasks to be executed to the task execution nodes in a message queue mode so as to process the tasks to be executed and obtain task execution results.
10. An electronic device, the electronic device comprising:
at least one processor; the method comprises the steps of,
a memory communicatively coupled to the at least one processor; wherein,,
the memory stores a computer program executable by the at least one processor to enable the at least one processor to perform the task balance consumption method according to any one of claims 1 to 8.
CN202211519762.3A 2022-11-30 2022-11-30 Task balanced consumption method and system and electronic equipment Pending CN116185602A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202211519762.3A CN116185602A (en) 2022-11-30 2022-11-30 Task balanced consumption method and system and electronic equipment

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202211519762.3A CN116185602A (en) 2022-11-30 2022-11-30 Task balanced consumption method and system and electronic equipment

Publications (1)

Publication Number Publication Date
CN116185602A true CN116185602A (en) 2023-05-30

Family

ID=86449678

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202211519762.3A Pending CN116185602A (en) 2022-11-30 2022-11-30 Task balanced consumption method and system and electronic equipment

Country Status (1)

Country Link
CN (1) CN116185602A (en)

Similar Documents

Publication Publication Date Title
CN108282527B (en) Generate the distributed system and method for Service Instance
US9020949B2 (en) Method and system for centralized issue tracking
CN102710793B (en) Network printing system based on cloud computing and data storage method thereof
CN105940377A (en) Methods, systems, and computer readable media for a cloud-based virtualization orchestrator
CN107222530A (en) Service asynchronous exchange method, equipment, system and computer-readable recording medium
CN110659134A (en) Data processing method and device applied to artificial intelligence platform
CN109614227A (en) Task resource concocting method, device, electronic equipment and computer-readable medium
CN109462508A (en) Node deployment method, device and storage medium
CN110415042A (en) A kind of discount coupon generates system, method and coupon server
CN109413125A (en) The method and apparatus of dynamic regulation distributed system resource
CN110263581A (en) Contract signs method, system, terminal device and storage medium
CN112035350B (en) Test method and device for block chain system and computer equipment
CN110910143A (en) Identity identification generation method, device, related node and medium
CN107888663A (en) A kind of method of distribution of document, equipment and computer-readable medium
CN109582439A (en) DCN dispositions method, device, equipment and computer readable storage medium
CN111669466B (en) Intelligent outbound system configuration method, device and computer readable storage medium
CN111951112A (en) Intelligent contract execution method based on block chain, terminal equipment and storage medium
US20230267430A1 (en) Data processing method and device, and computer-readable storage medium
CN116932147A (en) Streaming job processing method and device, electronic equipment and medium
CN111858585A (en) Block chain strategy processing device, computer readable storage medium and terminal equipment
CN116185602A (en) Task balanced consumption method and system and electronic equipment
CN115328457A (en) Method and device for realizing form page based on parameter configuration
CN114820132A (en) Order distribution method and device, electronic equipment and storage medium
CN107545004A (en) The method, apparatus and search engine system of a kind of search
CN111353766A (en) Service process processing system and method of distributed service system

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication