CN117041264A - Block chain resource management system and method based on data processing - Google Patents
Block chain resource management system and method based on data processing Download PDFInfo
- Publication number
- CN117041264A CN117041264A CN202311289317.7A CN202311289317A CN117041264A CN 117041264 A CN117041264 A CN 117041264A CN 202311289317 A CN202311289317 A CN 202311289317A CN 117041264 A CN117041264 A CN 117041264A
- Authority
- CN
- China
- Prior art keywords
- servers
- candidate
- node
- block chain
- newly added
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000012545 processing Methods 0.000 title claims abstract description 23
- 238000000034 method Methods 0.000 title claims abstract description 21
- 238000013508 migration Methods 0.000 claims abstract description 146
- 230000005012 migration Effects 0.000 claims abstract description 145
- 238000007726 management method Methods 0.000 claims description 78
- 238000012216 screening Methods 0.000 claims description 63
- 238000011156 evaluation Methods 0.000 claims description 18
- 238000009434 installation Methods 0.000 claims description 11
- 238000007405 data analysis Methods 0.000 claims description 3
- 238000001914 filtration Methods 0.000 claims description 3
- 238000000513 principal component analysis Methods 0.000 claims description 2
- 230000000694 effects Effects 0.000 abstract 1
- 238000010586 diagram Methods 0.000 description 7
- 238000013528 artificial neural network Methods 0.000 description 5
- 230000005540 biological transmission Effects 0.000 description 4
- 238000005516 engineering process Methods 0.000 description 3
- 230000006870 function Effects 0.000 description 3
- 238000010606 normalization Methods 0.000 description 3
- 238000012847 principal component analysis method Methods 0.000 description 3
- 230000008569 process Effects 0.000 description 3
- 230000000306 recurrent effect Effects 0.000 description 3
- 239000011159 matrix material Substances 0.000 description 2
- 230000015654 memory Effects 0.000 description 2
- 230000004044 response Effects 0.000 description 2
- 238000004458 analytical method Methods 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000008901 benefit Effects 0.000 description 1
- 230000002457 bidirectional effect Effects 0.000 description 1
- 239000002131 composite material Substances 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 238000010801 machine learning Methods 0.000 description 1
- 230000006403 short-term memory Effects 0.000 description 1
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
- H04L67/10—Protocols in which an application is distributed across nodes in the network
- H04L67/104—Peer-to-peer [P2P] networks
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L41/00—Arrangements for maintenance, administration or management of data switching networks, e.g. of packet switching networks
- H04L41/08—Configuration management of networks or network elements
- H04L41/0803—Configuration setting
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L41/00—Arrangements for maintenance, administration or management of data switching networks, e.g. of packet switching networks
- H04L41/08—Configuration management of networks or network elements
- H04L41/0894—Policy-based network configuration management
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L41/00—Arrangements for maintenance, administration or management of data switching networks, e.g. of packet switching networks
- H04L41/14—Network analysis or design
- H04L41/142—Network analysis or design using statistical or mathematical methods
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L63/00—Network architectures or network communication protocols for network security
- H04L63/04—Network architectures or network communication protocols for network security for providing a confidential data exchange among entities communicating through data packet networks
- H04L63/0428—Network architectures or network communication protocols for network security for providing a confidential data exchange among entities communicating through data packet networks wherein the data content is protected, e.g. by encrypting or encapsulating the payload
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
- Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management
Landscapes
- Engineering & Computer Science (AREA)
- Computer Networks & Wireless Communication (AREA)
- Signal Processing (AREA)
- Computer Security & Cryptography (AREA)
- Mathematical Physics (AREA)
- General Physics & Mathematics (AREA)
- Mathematical Analysis (AREA)
- Mathematical Optimization (AREA)
- Algebra (AREA)
- Probability & Statistics with Applications (AREA)
- Pure & Applied Mathematics (AREA)
- Physics & Mathematics (AREA)
- Computer Hardware Design (AREA)
- Computing Systems (AREA)
- General Engineering & Computer Science (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The invention relates to the field of data processing, in particular to a block chain resource management system and a method based on data processing, which are applied to a computer cluster and are provided with management nodes and at least one block chain node, wherein the method comprises the following steps: the management node receives the newly added block chain link point request, acquires the related data of a plurality of servers and determines candidate servers; when the preset deployment condition is not met, determining a node migration scheme, and migrating at least one established blockchain node; after the node migration is completed, acquiring the related data of the plurality of servers again, and determining a target server based on the acquired related data of the plurality of servers again; the target server deploys the newly added block chain nodes based on the newly added block chain link point request; the problem of block chain resource scheduling can be solved, and the effect of block chain resource management is improved.
Description
Technical Field
The present invention relates to the field of data processing, and in particular, to a system and method for managing blockchain resources based on data processing.
Background
The block chain distributed account book technology is a distributed technology and is characterized by decentralization, transparent disclosure and non-falsification. Specifically, each piece of data in the blockchain network is broadcast to all blockchain nodes, each of which can store the same data through a corresponding storage resource.
Blockchain as-a-service (BaaS, blockchain as a Service) is a service provided by a third party that creates and manages a cloud-based network for the company that built the blockchain application. In the BaaS model, enterprises and organizations can access BaaS created and developed on the cloud. BaaS applications are developed, hosted, and deployed on the cloud. This application is the same as any other locally hosted blockchain application with smart contracts and other related blockchain functionality. An advantage of BaaS model applications is that enterprises do not have to worry about the management and installation of any type of infrastructure (e.g., servers), but instead rely on cloud-based service providers to do all of these IT-related tasks. In fact, the application of blockchain technology has far more than just been in cryptocurrency, and has expanded to address various secure transactions (automotive, agricultural, etc.) and the like.
In the prior art, when a blockchain is used as a service to manage the resources of the blockchain, management of blockchain nodes is involved. How to select which server a blockchain node is deployed on is a matter of system scheduling. The existing containerized or micro-service architecture has scheduling capability, but the scheduling strategy belongs to a general algorithm, and is not customized for the deployment scene of the blockchain, so that the defects in terms of scheduling resources exist.
Accordingly, there is a need for a system and method for managing blockchain resources based on data processing to improve the effectiveness of blockchain resource management.
Disclosure of Invention
The invention provides a block chain resource management method based on data processing, which is applied to a computer cluster, wherein the computer cluster comprises a plurality of servers, the computer cluster is used for providing block chain instant service, and a management node and at least one block chain node are deployed on the computer cluster, and the method comprises the following steps: the management node receives a newly added block chain link point request; the management node obtains the related data of the plurality of servers based on the newly added block link point request; the management node performs data analysis based on the newly added block link point request and related data of the plurality of servers, and determines candidate servers matched with the newly added block link point request in the plurality of servers; when the candidate server matched with the newly added block chain link point request does not meet the preset deployment condition, the management node determines a node migration scheme based on the related data of the plurality of servers, and migrates at least one established block chain node based on the node migration scheme; after the node migration is completed, acquiring the related data of the plurality of servers again, and determining a target server based on the acquired related data of the plurality of servers again; when the candidate server matched with the newly added block chain link point request meets a preset deployment condition, the management node determines a target server from the candidate servers; the target server deploys a newly added blockchain node based on the newly added blockchain link point request.
Further, the related data of the plurality of servers at least comprises residual resource data, performance parameter data and state data of each server; the management node determines a candidate server matched with the newly added block link point request in the plurality of servers based on the newly added block link point request and related data of the plurality of servers, and the candidate server comprises: determining node resource requirements corresponding to the newly added block link point requests based on the types of the newly added block link nodes corresponding to the newly added block link point requests; and determining a candidate server matched with the newly added block link point request based on the residual resource data of the plurality of servers and the node resource requirement corresponding to the newly added block link point request.
Further, the preset deployment condition at least includes a candidate server number requirement, a candidate server performance requirement, and a candidate server state requirement, wherein the candidate server performance requirement is determined by determining the newly added block link point request based on the newly added block link point request; the management node judges whether a candidate server matched with the newly added block link point request meets a preset deployment condition or not, and the method comprises the following steps: based on the state data of all candidate servers which are matched with the newly added block chain link point request, performing first screening, and determining candidate servers after the first screening; when the number of the candidate servers screened for the first time does not meet the number of the servers, judging that the candidate servers matched with the newly added block chain link point request do not meet a preset deployment condition; when the number of the candidate servers after the first screening meets the number requirement of the servers, performing the second screening based on the performance parameter data of all candidate servers matched with the newly added block chain link point request, and determining the candidate servers after the second screening; when the number of the candidate servers screened for the second time does not meet the number requirement of the servers, judging that the candidate servers matched with the newly added block chain link point request do not meet a preset deployment condition; and when the number of the candidate servers screened for the second time meets the number requirement of the servers, judging that the candidate servers matched with the newly added block chain link point request meet a preset deployment condition.
Further, the management node determines a node migration scheme based on the related data of the plurality of servers, including: generating a plurality of candidate migration schemes based on constraint condition sets through a Monte Carlo model, wherein the candidate migration schemes comprise at least one block chain node to be migrated and a target migration server corresponding to each block chain node to be migrated; for each candidate migration scheme, determining a matching score corresponding to the candidate migration scheme based on a migration evaluation system, wherein the migration evaluation system comprises a plurality of migration indexes and weights corresponding to each migration index; and determining a target migration scheme from the plurality of candidate migration schemes based on the matching scores corresponding to each of the candidate migration schemes.
Further, the migration index at least comprises a migration frequency index, a number index of candidate servers after migration, a load balancing index and/or a performance index of candidate servers after migration; the weight corresponding to the migration index is determined based on a principal component analysis method.
Still further, the managing node migrates at least one established blockchain node based on the node migration scheme, comprising: and for each block chain node to be migrated, newly adding a migration block chain node on a target migration server corresponding to the block chain link point to be migrated, carrying out data synchronization on the block chain node to be migrated and the corresponding migration block chain node, adding the block chain node to be migrated into a newly added block chain network after the data synchronization, exiting the original block chain network by the block chain node to be migrated, and deleting the block chain node to be migrated by the server where the block chain link point to be migrated is located.
Still further, the managing node determines a target server from the candidate servers, including: predicting newly-increased state data of each candidate server after the second screening after newly-increasing the new block chain link point request corresponding to the newly-increased block chain node based on the state data of the candidate server after the second screening at a plurality of historical time points through a state prediction model; and determining the target server based on the newly added state data, the residual resource data and the performance parameter data of the candidate server after the second screening.
Further, the management node determines the target server based on the newly added state data, the remaining resource data and the performance parameter data of the candidate server after the second filtering, including: determining a priority score corresponding to each candidate server after the second screening based on a screening evaluation system, the newly-added state data, the residual resource data and the performance parameter data of the candidate server after the second screening; and determining the target server based on the priority score corresponding to each candidate server after the second screening.
Still further, the target server deploys a newly added blockchain node based on the newly added blockchain link point request, including: the target server acquires an encrypted installation package based on the related data of the target server from the management node; the target server acquires parameter data encrypted based on the related data of the target server from the management node; the target server decrypts the encrypted installation package and installs node software; after decrypting the encrypted parameter data, the target server configures node parameters; and the target server starts the node software, synchronizes the data on the blockchain network and completes the deployment of the newly added blockchain node.
The invention provides a block chain resource management system based on data processing, which is applied to a computer cluster, wherein the computer cluster comprises a plurality of servers, the computer cluster is used for providing block chain instant service, and a management node and at least one block chain node are deployed on the computer cluster, and the system comprises: the request receiving module is deployed on the management node and is used for receiving the newly-added block chain link point request; the data acquisition module is deployed on the management node and is used for acquiring the related data of the plurality of servers based on the newly added block link point request; the deployment scheduling module is configured to determine a candidate server matching the new block link point request in the multiple servers based on the new block link point request and related data of the multiple servers, determine a node migration scheme based on the related data of the multiple servers when the candidate server matching the new block link point request does not meet a preset deployment condition, migrate at least one established block chain node based on the node migration scheme, acquire related data of the multiple servers again after completing node migration, determine a target server based on the related data of the multiple servers again, and determine the target server from the candidate servers when the candidate server matching the new block link point request meets a preset deployment condition, wherein the target server is configured to deploy the new block chain node based on the new block link point request.
Compared with the prior art, the block chain resource management system and method based on data processing provided by the specification have the following beneficial effects:
1. the method comprises the steps of managing a plurality of servers of a computer cluster through setting a management node, automatically determining a target server for deploying the new block chain node based on the new block chain node request and related data of the plurality of servers after receiving the new block chain node request, and realizing more flexible management on the block chain node through setting preset deployment conditions so that the target server is more matched with deployment requirements of the new block chain node.
2. Through twice screening, whether the current computer cluster needs node scheduling or not can be accurately judged, so that a server with more matched performance is determined for the newly added blockchain node.
3. Through the Monte Carlo model, a plurality of candidate migration schemes can be rapidly determined, further, a migration evaluation system is established, and based on the migration evaluation system, the matching scores corresponding to the candidate migration schemes are determined, so that the target migration scheme with higher resource utilization rate and matching with the newly added blockchain node is determined.
4. By encrypting the installation package and the parameter information of the node, data leakage in the data transmission process is effectively avoided, and the safety of data transmission is improved.
Drawings
The present specification will be further elucidated by way of example embodiments, which will be described in detail by means of the accompanying drawings. These embodiments are not limiting, and in these embodiments, like numerals represent like structures, and a brief description of each of the drawings of the present invention is as follows:
FIG. 1 is a block diagram of a blockchain resource management system based on data processing in accordance with some embodiments of the present description;
FIG. 2 is a flow diagram of a method for data processing based blockchain resource management in accordance with some embodiments of the present description;
FIG. 3 is a flow chart for determining whether a candidate server satisfies a preset deployment condition according to some embodiments of the present disclosure;
FIG. 4 is a flow diagram illustrating a determination of a node migration scheme according to some embodiments of the present description;
FIG. 5 is a flow diagram illustrating deployment of newly added blockchain nodes in accordance with some embodiments of the present description.
Detailed Description
FIG. 1 is a block diagram of a system for managing blockchain resources based on data processing according to some embodiments of the present disclosure, where the system for managing blockchain resources based on data processing may be applied to a computer cluster including a plurality of servers, the computer cluster being configured to provide blockchain services, and the computer cluster having a management node and at least one blockchain node disposed thereon, and where the modules of the system for managing blockchain resources based on data processing may include a request receiving module, a data acquisition module, and a deployment scheduling module disposed on the management node of the computer cluster, as shown in FIG. 1.
The request receiving module may be configured to receive a newly added block link point request.
The data acquisition module can be used for acquiring related data of a plurality of servers based on the newly added block link point request.
The deployment scheduling module may be configured to determine, based on the newly added block link point request and related data of the plurality of servers, a candidate server of the plurality of servers that matches the newly added block link point request, determine, when the candidate server that matches the newly added block link point request does not meet a preset deployment condition, a node migration scheme based on the related data of the plurality of servers, migrate at least one established block chain node based on the node migration scheme, acquire, after the node migration is completed, related data of the plurality of servers again, determine, based on the acquired related data of the plurality of servers again, a target server, and determine, when the candidate server that matches the newly added block link point request meets the preset deployment condition, the management node from the candidate server, where the target server is configured to deploy the newly added block chain node based on the newly added block link point request.
For more description of the request receiving module, the data obtaining module, and the deployment scheduling module, refer to fig. 2 and related description thereof, and are not repeated here.
FIG. 2 is a flow chart of a method for data processing based blockchain resource management, as shown in FIG. 2, according to some embodiments of the present disclosure, which may include the following.
In step 210, the management node receives a newly added block link point request.
Specifically, the management node may obtain the newly added block link point request from an external data source (e.g., a user side) or automatically generate the newly added block link point request.
Step 220, the management node obtains relevant data of a plurality of servers based on the newly added block link point request.
In some embodiments, the data associated with the plurality of servers includes at least remaining resource data, performance parameter data, and status data for each server. The remaining resource information of the server may include CPU remaining, memory remaining, disk remaining, and the like, the performance parameter data of the server may include CPU frequency, memory size, speed of the hard disk, size of the hard disk, and the like, and the status data of the server may include temperature data, process CPI, time consumption of a target link (e.g., a key function of IPVS, and the like), service response time, and the like.
In step 230, the management node performs data analysis based on the newly added blockchain node request and the related data of the plurality of servers, and determines a candidate server matching the newly added blockchain node request from the plurality of servers.
The candidate server may be a server that may be used for newly added block link point deployment.
In some embodiments, the management node determines a candidate server of the plurality of servers that matches the newly added blockchain node request based on the newly added blockchain node request and related data of the plurality of servers, comprising:
determining node resource requirements corresponding to the newly added block link point requests based on the types of the newly added block chain nodes (such as full nodes, super nodes, light nodes and the like) corresponding to the newly added block link point requests, wherein it can be understood that the different types of block chain nodes with different corresponding computer resource requirements can comprise type information of the newly added block chain nodes;
and determining candidate servers matched with the newly added block link point requests based on the residual resource data of the plurality of servers and the node resource requirements corresponding to the newly added block link point requests.
In some embodiments, the preset deployment conditions include at least a candidate server quantity requirement, a candidate server performance requirement, and a candidate server state requirement, wherein the candidate server performance requirement is determined based on the newly added block link point request.
Fig. 3 is a schematic flow chart of determining whether a candidate server meets a preset deployment condition according to some embodiments of the present disclosure, as shown in fig. 3, in some embodiments, the management node determines whether the candidate server matching the newly added block link point request meets the preset deployment condition, including:
based on the state data of all candidate servers which are matched with the newly added block chain link point request, performing first screening, determining the candidate servers after the first screening, specifically, judging whether the candidate servers are in a fault state or a resource race state based on the state data of the candidate servers, and regarding more description of the fault state and the resource race state, referring to the following related description, which is not repeated herein, taking the candidate servers in a normal state as the candidate servers after the first screening;
when the number of the candidate servers after the first screening does not meet the number requirement of the servers, judging that the candidate servers matched with the newly added block chain link point request do not meet the preset deployment condition, for example, when the number of the candidate servers after the first screening is smaller than the number threshold of the preset servers, judging that the candidate servers matched with the newly added block chain link point request do not meet the preset deployment condition;
when the number of the candidate servers after the first screening meets the number requirement of the servers, performing second screening on the basis of the performance parameter data of all candidate servers which are matched with the newly added block chain link point request, determining the candidate servers after the second screening, and particularly, when the performance parameters of the candidate servers after the first screening meet the preset performance parameter requirement, taking the candidate servers after the first screening as the candidate servers after the second screening;
when the number of the candidate servers after the second screening does not meet the number requirement of the servers, judging that the candidate servers matched with the newly added block chain link point request do not meet the preset deployment condition, specifically, when the number of the candidate servers after the second screening is smaller than the number threshold of the preset servers, judging that the candidate servers matched with the newly added block chain link point request do not meet the preset deployment condition;
when the number of the candidate servers after the second screening meets the number requirement of the servers, judging that the candidate servers matched with the newly added block chain link point request meet the preset deployment condition, and particularly, when the number of the candidate servers after the second screening is larger than or equal to the preset server number threshold, judging that the candidate servers matched with the newly added block chain link point request meet the preset deployment condition.
It can be appreciated that by twice screening, it can be more accurately determined whether the current computer cluster needs to perform node scheduling, thereby determining a server with more matched performance for the newly added blockchain node.
Step 240, when the candidate server matching the newly added block link point request does not meet the preset deployment condition, the management node determines a node migration scheme based on the related data of the plurality of servers, and migrates at least one established block chain node based on the node migration scheme.
FIG. 4 is a flow diagram of determining a node migration scheme, as shown in FIG. 4, according to some embodiments of the present disclosure, in some embodiments, a management node determines a node migration scheme based on data associated with a plurality of servers, including:
generating a plurality of candidate migration schemes based on constraint condition sets through a Monte Carlo model, wherein the candidate migration schemes comprise at least one block chain node to be migrated and target migration servers corresponding to each block chain node to be migrated, and the constraint condition sets at least comprise the maximum number constraint of the block chain nodes to be migrated, the minimum number threshold value of the candidate servers after migration and the like;
for each candidate migration scheme, determining a matching score corresponding to the candidate migration scheme based on a migration evaluation system, wherein the migration evaluation system comprises a plurality of migration indexes and weights corresponding to each migration index;
the target migration scheme is determined from the plurality of candidate migration schemes based on the matching score corresponding to each candidate migration scheme.
In some embodiments, the migration metrics include at least a migration times metric, a number of candidate servers after migration metrics, a load balancing metric, and/or a candidate server performance metric after migration. The weight corresponding to the migration index is determined based on principal component analysis. When the principal component analysis method calculates the weight of analysis items, the principal component analysis method needs to calculate by using information such as a load coefficient and the like, and is divided into three steps: first: calculating a linear combination coefficient matrix, wherein the formula is as follows: loading matrix/Sqrt (feature root), i.e. the load factor divided by the square root of the corresponding feature root; second,: calculating a comprehensive score coefficient, wherein the formula is as follows: accumulated (linear combination coefficient x variance interpretation rate)/accumulated variance interpretation rate, i.e. the linear combination coefficient is multiplied by the variance interpretation rate and accumulated, and divided by the accumulated variance interpretation rate, to obtain a composite score coefficient, and third: and calculating weight, and carrying out summation normalization processing on the comprehensive score coefficient to obtain each index weight value.
In some embodiments, for each candidate migration scheme, the management node may determine the scores of the selected migration scheme at the plurality of migration indexes, and determine the matching scores corresponding to the candidate migration schemes based on the scores of the selected migration scheme at the plurality of migration indexes and the weights corresponding to each migration index. It can be understood that the more migration times the candidate migration scheme corresponds to, the lower the score of the candidate migration scheme in the migration time index. The more the number indexes of the candidate servers after migration are, the higher the score of the number indexes of the candidate servers after migration is, and the candidate servers after migration can determine the candidate servers matched with the newly added block chain link point request in the plurality of servers after migration based on the newly added block chain link point request and related data of the plurality of servers after migration. The larger the load balancing rate of the candidate migration scheme after migration is, the higher the score of the candidate migration scheme in the load balancing index is. The higher the average performance of the candidate server after migration, the higher the score of the candidate server performance index of the candidate migration scheme after migration.
In some embodiments, the management node may calculate the post-migration load balancing rate corresponding to the candidate migration scheme based on the following formula:
wherein,for the load balancing rate after migration corresponding to the j-th candidate migration scheme,load for the jth server under the j-th candidate migration scheme,/for the jth server>Load mean of server under j-th candidate migration scheme,/for the j-th candidate migration scheme>And L is the total number of servers for the preset parameters.
In some embodiments, the management node may calculate the average performance of the migrated candidate servers corresponding to the candidate migration scheme based on the following formula:
wherein,the average performance of the migrated candidate servers corresponding to the j-th candidate migration scheme,performance score of the q candidate server after migration corresponding to the j candidate migration scheme +.>And determining based on the performance parameter data of the Q candidate servers after migration corresponding to the j candidate migration scheme, wherein Q is the total number of the candidate servers after migration corresponding to the j candidate migration scheme.
Illustratively, the management node may determine the matching score corresponding to the candidate migration scheme based on the migration evaluation system by the following formula:
wherein,matching score for j-th candidate migration scheme,/->Weight corresponding to the ith migration index, < +.>And (5) scoring the normalized j candidate migration scheme at the i migration index.
In some embodiments, the management node may take the candidate migration scheme with the largest matching score as the target migration scheme.
It can be appreciated that multiple candidate migration schemes can be rapidly determined through the Monte Carlo model, further, a migration evaluation system is established, and a matching score corresponding to the candidate migration scheme is determined based on the migration evaluation system, so that a target migration scheme with higher resource utilization rate and matching with a new blockchain node is determined.
In some embodiments, the managing node migrates at least one established blockchain node based on a node migration scheme, comprising: and for each block chain node to be migrated, newly adding a migration block chain node on a target migration server corresponding to the block chain link point to be migrated, carrying out data synchronization on the block chain node to be migrated and the corresponding migration block chain node, adding the migration block chain link point into a block chain network after the data synchronization, exiting the block chain network by the block chain link point to be migrated, and deleting the block chain node to be migrated by the server where the block chain link point to be migrated is positioned.
Step 250, after the node migration is completed, acquiring the related data of the plurality of servers again, and determining the target server based on the acquired related data of the plurality of servers again.
The target server may be a server for deploying the newly added blockchain node corresponding to the newly added blockchain node request.
In some embodiments, after the node migration is completed, step 230 may be performed again to redetermine the candidate servers that match the newly added block link point request, thereby determining the target server.
In step 260, when the candidate server matching the newly added block link point request meets the preset deployment condition, the management node determines the target server from the candidate servers.
In some embodiments, the management node determines the target server from the candidate servers, comprising:
for each candidate server after the second screening, predicting newly-added state data (such as temperature data, process CPI, target link (such as key function of IPVS, etc.) after the newly-added block chain node requests corresponding newly-added block chain node (such as time consuming and service response time) of the candidate server after the second screening based on state data (such as temperature data, process CPI, target link (such as key function of IPVS, etc.) of the candidate server at a plurality of historical time points through a state prediction model, wherein the state prediction model can be a machine learning model such as an artificial neural network (Artificial Neural Network, ANN) model, a recurrent neural network (Recurrent Neural Networks, RNN) model, a Long Short-Term Memory (LSTM) model, a Bidirectional Recurrent Neural Network (BRNN) model, etc.;
and determining the target server based on the newly added state data, the residual resource data and the performance parameter data of the candidate server after the second screening.
In some embodiments, the management node may calculate remaining resource data after deploying the newly added block chain link point request corresponding to the newly added block chain node based on the current remaining resource data.
In some embodiments, the management node determines the target server based on the newly added state data, the remaining resource data, and the performance parameter data of the candidate server after the second filtering, including:
determining a priority score corresponding to each candidate server after the second screening based on the screening evaluation system, the newly added state data, the residual resource data and the performance parameter data of the candidate server after the second screening;
and determining the target server based on the priority score corresponding to each candidate server screened for the second time.
In some embodiments, the screening evaluation system may include a plurality of screening evaluation indicators, where the plurality of screening evaluation indicators may include at least a status indicator, a remaining resource indicator, and a performance parameter indicator. The management node may calculate a score of the candidate server after the second screening on the status index based on the newly-added status data of the candidate server after the second screening predicted by the status prediction model, calculate a score of the candidate server after the second screening on the residual resource index based on the residual resource data of the candidate server after the second screening after the candidate server after the second screening is deployed with the newly-added blockchain link point request corresponding to the newly-added blockchain node, and calculate a score of the candidate server after the second screening on the performance parameter index based on the performance parameter data of the candidate server after the second screening.
Illustratively, the management node may determine the priority score corresponding to each candidate server after the second screening based on the screening evaluation system, the newly added state data, the remaining resource data, and the performance parameter data of the candidate server after the second screening by the following formula:
wherein,for the f candidate server after the second screening, the priority score is ++>、/>AndAre all preset weights, are->Score of candidate server in state index after f second screening after normalization, +.>Score of candidate server after f second screening after normalization in residual resource index, +.>Candidate server after second screening for normalized fScore at performance parameter index.
In some embodiments, the management node may target the candidate server with the greatest priority score.
In step 270, the target server deploys the newly added blockchain node based on the newly added blockchain link point request.
FIG. 5 is a flow diagram of deploying a newly added blockchain node, as shown in FIG. 5, according to some embodiments of the present disclosure, in some embodiments, the target server deploys the newly added blockchain node based on the newly added blockchain node request, including:
the target server acquires an installation package encrypted based on the related information of the target server from the management node;
the target server acquires parameter information encrypted based on the related information of the target server from the management node;
after decrypting the encrypted installation package, the target server installs node software;
after decrypting the encrypted parameter information, the target server configures node parameters, wherein the node parameters at least comprise information such as identity, IP address, port number and the like of the newly added blockchain node;
and the target server starts node software, synchronizes data on the blockchain network and completes deployment connection of newly added blockchain nodes.
Specifically, the management node may encrypt the installation package and/or encrypt the parameter information based on the unique device identifier and the public key corresponding to the target server, and the target server may decrypt the encrypted installation package and/or decrypt the encrypted parameter information based on the unique device identifier and the private key corresponding to the target server.
It can be understood that by encrypting the installation package and the parameter information of the node, data leakage in the data transmission process is effectively avoided, and the safety of data transmission is improved.
Finally, it is important to state that the specific embodiments described in this specification are merely illustrative of the principles of some of the embodiments of the invention. On this basis, other simple structural changes are also considered to be within the scope of the invention. Accordingly, the invention is not to be taken as limited by the specific examples of the invention, and alternatives to the specific embodiments of the invention may be seen as consistent with the invention. Accordingly, the embodiments of the present specification are not limited to the specific embodiments explicitly described and illustrated herein.
Claims (10)
1. The block chain resource management method based on data processing is applied to a computer cluster, the computer cluster comprises a plurality of servers, the computer cluster is used for providing block chain instant service, and a management node and at least one block chain node are deployed on the computer cluster, and the method is characterized by comprising the following steps:
the management node receives a newly added block chain link point request;
the management node obtains the related data of the plurality of servers based on the newly added block link point request;
the management node performs data analysis based on the newly added block link point request and related data of the plurality of servers, and determines candidate servers matched with the newly added block link point request in the plurality of servers;
when the candidate server matched with the newly added block chain link point request does not meet the preset deployment condition, the management node determines a node migration scheme based on the related data of the plurality of servers, and migrates at least one established block chain node based on the node migration scheme;
after the node migration is completed, acquiring the related data of the plurality of servers again, and determining a target server based on the acquired related data of the plurality of servers again;
when the candidate server matched with the newly added block chain link point request meets a preset deployment condition, the management node determines a target server from the candidate servers;
the target server deploys a newly added blockchain node based on the newly added blockchain link point request.
2. The method for managing blockchain resources based on data processing as in claim 1, wherein the related data of the plurality of servers at least includes remaining resource data, performance parameter data and status data of each of the servers;
the management node determines a candidate server matched with the newly added block link point request in the plurality of servers based on the newly added block link point request and related data of the plurality of servers, and the candidate server comprises:
determining node resource requirements corresponding to the newly added block link point requests based on the types of the newly added block link nodes corresponding to the newly added block link point requests;
and determining a candidate server matched with the newly added block link point request based on the residual resource data of the plurality of servers and the node resource requirement corresponding to the newly added block link point request.
3. The method of claim 2, wherein the predetermined deployment conditions include at least a candidate server number requirement, a candidate server performance requirement, and a candidate server state requirement, and determining a candidate server performance requirement based on the newly added block link point request;
the management node judges whether a candidate server matched with the newly added block link point request meets a preset deployment condition or not, and the method comprises the following steps:
based on the state data of all candidate servers which are matched with the newly added block chain link point request, performing first screening, and determining candidate servers after the first screening;
when the number of the candidate servers screened for the first time does not meet the number of servers, judging that the candidate servers matched with the newly added block chain link point request do not meet a preset deployment condition;
when the number of the candidate servers after the first screening meets the number requirement of the servers, performing the second screening based on the performance parameter data of all candidate servers matched with the newly added block chain link point request, and determining the candidate servers after the second screening;
when the number of the candidate servers screened for the second time does not meet the number requirement of the servers, judging that the candidate servers matched with the newly added block chain link point request do not meet a preset deployment condition;
and when the number of the candidate servers screened for the second time meets the number requirement of the servers, judging that the candidate servers matched with the newly added block chain link point request meet a preset deployment condition.
4. A blockchain resource management method based on data processing as in any of claims 1-3 wherein the management node determines a node migration scheme based on the relevant data of the plurality of servers, comprising:
generating a plurality of candidate migration schemes based on constraint condition sets through a Monte Carlo model, wherein the candidate migration schemes comprise at least one block chain node to be migrated and a target migration server corresponding to each block chain node to be migrated;
for each candidate migration scheme, determining a matching score corresponding to the candidate migration scheme based on a migration evaluation system, wherein the migration evaluation system comprises a plurality of migration indexes and weights corresponding to each migration index;
and determining a target migration scheme from the plurality of candidate migration schemes based on the matching scores corresponding to each of the candidate migration schemes.
5. The blockchain resource management method based on data processing as in claim 4, wherein the migration index at least includes a migration number index, a number of candidate servers after migration index, a load balancing index and/or a candidate server performance index after migration;
the weight corresponding to the migration index is determined based on principal component analysis.
6. The method of claim 4, wherein the managing node migrates at least one established blockchain node based on the node migration scheme, comprising:
for each block chain node to be migrated, newly adding a migration block chain node on a target migration server corresponding to the block chain link point to be migrated, performing data synchronization on the block chain node to be migrated and the corresponding migration block chain node, and adding the block chain link point to be migrated into a newly added block chain network after the data synchronization; the block chain node to be migrated exits the original block chain network, and the server where the block chain link point to be migrated is located deletes the block chain node to be migrated.
7. A blockchain resource management method based on data processing as in claim 3 wherein the management node determines a target server from the candidate servers, comprising:
predicting newly-increased state data of each candidate server after the second screening after the corresponding newly-increased block chain node is requested at the newly-increased block chain node according to the state data of the candidate server after the second screening at a plurality of historical time points through a state prediction model;
and determining the target server based on the newly added state data, the residual resource data and the performance parameter data of the candidate server after the second screening.
8. The method of claim 7, wherein the determining, by the management node, the target server based on the newly added state data, the remaining resource data, and the performance parameter data of the candidate server after the second filtering, comprises:
determining a priority score corresponding to each candidate server after the second screening based on the screening evaluation system, the newly added state data, the residual resource data and the performance parameter data of the candidate server after the second screening;
and determining the target server based on the priority score corresponding to each candidate server screened for the second time.
9. A blockchain resource management method based on data processing as in any of claims 1-3 wherein the target server deploys a newly added blockchain node based on the newly added blockchain node request, comprising:
the target server acquires an encrypted installation package based on the related information of the target server from the management node;
the target server acquires parameter information encrypted based on the related information of the target server from the management node;
the target server decrypts the encrypted installation package and installs node software;
after decrypting the encrypted parameter information, the target server configures node parameters;
and the target server starts the node software, synchronizes the data on the blockchain network and completes the deployment of newly added blockchain nodes.
10. A blockchain resource management system of a blockchain resource management method as in any of claims 1-9 applied to a computer cluster, the computer cluster including a plurality of servers, the computer cluster being configured to provide blockchain instant services, the computer cluster having disposed thereon a management node and at least one blockchain node, comprising:
the request receiving module is deployed on the management node and is used for receiving the newly-added block chain link point request;
the data acquisition module is deployed on the management node and is used for acquiring the related data of the plurality of servers based on the newly added block link point request;
the deployment scheduling module is deployed on the management node and is used for determining a candidate server matched with the newly-added block chain link point request in the plurality of servers based on the newly-added block chain link point request and related data of the plurality of servers, when the candidate server matched with the newly-added block chain link point request does not meet a preset deployment condition, the management node determines a node migration scheme based on the related data of the plurality of servers, migrates at least one established block chain node based on the node migration scheme, acquires related data of the plurality of servers again after node migration is completed, determines a target server based on the related data of the plurality of servers again, and determines the target server from the candidate servers when the candidate server matched with the newly-added block chain link point request meets the preset deployment condition; the target server is configured to deploy a newly added blockchain node based on the newly added blockchain link point request.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311289317.7A CN117041264B (en) | 2023-10-08 | 2023-10-08 | Block chain resource management system and method based on data processing |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311289317.7A CN117041264B (en) | 2023-10-08 | 2023-10-08 | Block chain resource management system and method based on data processing |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN117041264A true CN117041264A (en) | 2023-11-10 |
| CN117041264B CN117041264B (en) | 2024-01-12 |
Family
ID=88645185
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202311289317.7A Active CN117041264B (en) | 2023-10-08 | 2023-10-08 | Block chain resource management system and method based on data processing |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN117041264B (en) |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110795203A (en) * | 2018-08-03 | 2020-02-14 | 阿里巴巴集团控股有限公司 | Resource scheduling method, device and system and computing equipment |
| WO2020073859A1 (en) * | 2018-10-12 | 2020-04-16 | 阿里巴巴集团控股有限公司 | Blockchain node service deployment method, apparatus and system, and computing device and medium |
| CN111641715A (en) * | 2020-05-29 | 2020-09-08 | 深圳壹账通智能科技有限公司 | Data processing method, device, equipment and medium based on block chain |
| CN115866059A (en) * | 2023-01-13 | 2023-03-28 | 北京微芯区块链与边缘计算研究院 | Block chain link point scheduling method and device |
| CN115952008A (en) * | 2023-03-15 | 2023-04-11 | 之江实验室 | Unified scheduling method and device for server cluster resources |
| CN116107704A (en) * | 2023-04-17 | 2023-05-12 | 北京邮电大学 | Block chain BaaS and multi-alliance deployment method, data storage access method and device |
-
2023
- 2023-10-08 CN CN202311289317.7A patent/CN117041264B/en active Active
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110795203A (en) * | 2018-08-03 | 2020-02-14 | 阿里巴巴集团控股有限公司 | Resource scheduling method, device and system and computing equipment |
| WO2020073859A1 (en) * | 2018-10-12 | 2020-04-16 | 阿里巴巴集团控股有限公司 | Blockchain node service deployment method, apparatus and system, and computing device and medium |
| CN111641715A (en) * | 2020-05-29 | 2020-09-08 | 深圳壹账通智能科技有限公司 | Data processing method, device, equipment and medium based on block chain |
| CN115866059A (en) * | 2023-01-13 | 2023-03-28 | 北京微芯区块链与边缘计算研究院 | Block chain link point scheduling method and device |
| CN115952008A (en) * | 2023-03-15 | 2023-04-11 | 之江实验室 | Unified scheduling method and device for server cluster resources |
| CN116107704A (en) * | 2023-04-17 | 2023-05-12 | 北京邮电大学 | Block chain BaaS and multi-alliance deployment method, data storage access method and device |
Also Published As
| Publication number | Publication date |
|---|---|
| CN117041264B (en) | 2024-01-12 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Dabbagh et al. | Energy-efficient resource allocation and provisioning framework for cloud data centers | |
| US7478361B2 (en) | Method and system for managing application deployment | |
| US9811445B2 (en) | Methods and systems for the use of synthetic users to performance test cloud applications | |
| Edwin et al. | An efficient and improved multi-objective optimized replication management with dynamic and cost aware strategies in cloud computing data center | |
| Prachitmutita et al. | Auto-scaling microservices on IaaS under SLA with cost-effective framework | |
| US20150032817A1 (en) | System and method using software defined continuity (sdc) and application defined continuity (adc) for achieving business continuity and application continuity on massively scalable entities like entire datacenters, entire clouds etc. in a computing system environment | |
| TWI725744B (en) | Method for establishing system resource prediction and resource management model through multi-layer correlations | |
| Javadpour et al. | Improving load balancing for data-duplication in big data cloud computing networks | |
| CN113515364B (en) | Data migration method and device, computer equipment and storage medium | |
| CN116134448A (en) | Joint machine learning using locality sensitive hashing | |
| WO2016127664A1 (en) | Access control method and system | |
| Forti et al. | Declarative continuous reasoning in the cloud-IoT continuum | |
| Jayapandian | Cloud dynamic scheduling for multimedia data encryption using tabu search algorithm | |
| Vashisht et al. | Efficient dynamic replication algorithm using agent for data grid | |
| US11567835B2 (en) | Data protection and recovery | |
| CN117041264B (en) | Block chain resource management system and method based on data processing | |
| Pham-Nguyen et al. | Dynamic resource provisioning on fog landscapes | |
| CN115550373B (en) | Combined test task environment load balancing modeling method based on cloud platform management and control | |
| US12058179B2 (en) | Computing power network system | |
| Kirthica et al. | Unified framework for data management in multi-cloud environment | |
| CN114978913B (en) | Cross-domain deployment method and system for service function chains based on cut chains | |
| WO2023209414A1 (en) | Methods and apparatus for computing resource allocation | |
| CN115225645A (en) | Service updating method, device, system and storage medium | |
| Zaman et al. | Cost-effective data replication mechanism modelling for cloud storage | |
| Monniot et al. | Supporting dynamic allocation of heterogeneous storage resources on HPC systems |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |