CN112822235A - Data sharing system and method of heterogeneous cloud computing system - Google Patents
Data sharing system and method of heterogeneous cloud computing system Download PDFInfo
- Publication number
- CN112822235A CN112822235A CN202011603113.2A CN202011603113A CN112822235A CN 112822235 A CN112822235 A CN 112822235A CN 202011603113 A CN202011603113 A CN 202011603113A CN 112822235 A CN112822235 A CN 112822235A
- Authority
- CN
- China
- Prior art keywords
- data
- cloud server
- virtual
- target
- control
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
- H04L67/10—Protocols in which an application is distributed across nodes in the network
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
- H04L67/10—Protocols in which an application is distributed across nodes in the network
- H04L67/1001—Protocols in which an application is distributed across nodes in the network for accessing one among a plurality of replicated servers
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
- H04L67/10—Protocols in which an application is distributed across nodes in the network
- H04L67/104—Peer-to-peer [P2P] networks
- H04L67/1074—Peer-to-peer [P2P] networks for supporting data block transmission mechanisms
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04W—WIRELESS COMMUNICATION NETWORKS
- H04W24/00—Supervisory, monitoring or testing arrangements
- H04W24/02—Arrangements for optimising operational condition
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04W—WIRELESS COMMUNICATION NETWORKS
- H04W28/00—Network traffic management; Network resource management
- H04W28/02—Traffic management, e.g. flow control or congestion control
- H04W28/08—Load balancing or load distribution
Landscapes
- Engineering & Computer Science (AREA)
- Computer Networks & Wireless Communication (AREA)
- Signal Processing (AREA)
- Data Exchanges In Wide-Area Networks (AREA)
Abstract
The invention belongs to the technical field of cloud computing, and particularly relates to a data sharing system and method of a heterogeneous cloud computing system. The system comprises: heterogeneous cloud server groups and virtual base stations; the heterogeneous cloud server group comprises three heterogeneous cloud servers, which are respectively as follows: the system comprises a first cloud server, a second cloud server and a third cloud server; the virtual base station is configured for scheduling and sharing data in the heterogeneous cloud server group and is respectively in signal connection with the first cloud server, the second cloud server and the third cloud server; the virtual base station includes: a virtual plane, a control plane, and a number of applications; the virtual plane comprises a plurality of virtual machines which are in signal connection to form a block chain network; and is connected with only one cloud server; each cloud server is connected with at least one virtual machine. The data resources of a plurality of heterogeneous cloud servers are scheduled, distributed and shared through the virtual base station.
Description
Technical Field
The invention belongs to the technical field of cloud computing, and particularly relates to a data sharing system and method of a heterogeneous cloud computing system.
Background
With the continuous development of the information age, the information exchange of different departments and different regions is gradually increased, and the development of the computer network technology provides guarantee for information transmission. When large amounts of spatial data appear on a network, how can we effectively utilize them in the face of a wide variety of data formats? This is really a problem of data sharing and data transformation. In short, data sharing is to enable users who use different computers and different software in different places to read data of others and perform various operations, operations and analyses.
Cloud computing is a technology combining distributed processing, parallel processing, grid computing and the like. The core idea of cloud computing is to uniformly manage and schedule a large number of computing resources connected by a network to form a computing resource pool for users to serve as required.
By using the cloud computing service, the service provider server can reduce the operation cost of enterprises and provide reliable resource access service for users. There have been more and more service provider servers selecting cloud computing services to provide relevant service services to users.
The patent numbers are: CN201010527248.5A discloses a method for secure sharing of cloud computing resources, comprising: a user sends a resource access request to a cloud computing service provider server, wherein the resource access request carries relevant information of a resource to be accessed; the cloud computing service provider server acquires service provider server information to which the resource to be accessed belongs according to the relevant information of the resource to be accessed, and sends an authentication request to the service provider server to which the resource to be accessed belongs; the authentication request carries identification information of the user; the service provider server performs identity authentication on the user according to the identification information of the user, and sends resource access control information to the user or the cloud computing service provider server after the authentication is passed; and the cloud computing service provider server authenticates the resource access control information and provides the resource to be accessed to the user after the authentication is passed. The invention also discloses a cloud computing resource safety sharing device. The invention improves the efficiency and the safety of shared resource access. Although the sharing of cloud computing resources is realized, the resource access control based on authentication is essential, and no technical solution for data sharing of cloud computing exists.
The patent numbers are: CN201410564493.1A discloses a cloud computing data sharing system, comprising: the data module manages the input and output of the network sharing data; the cloud computing resource pool comprises a plurality of multicast nodes used for storing written or read shared data; the synchronization module is used for processing the sequence of reading and writing the network shared data in the multicast nodes, performing read-write transaction management when the read-write is blocked, realizing message queues and ensuring the data consistency of the multicast nodes; the network module is used for transmitting the network sharing data between the cloud computing resource pool and the synchronization module, analyzing the network sharing data and forming an instruction set and a data stream; and the control module controls the data module, the synchronization module and the network module, so that repeated transmission of network shared data in a network can be effectively reduced, the network throughput is greatly improved, and the complexity of the system is reduced. But there is no solution for cloud computing data balancing.
Disclosure of Invention
In view of this, the present invention mainly aims to provide a data sharing system and method for a heterogeneous cloud computing system, where data resources of a plurality of heterogeneous cloud servers are scheduled, allocated, and shared by a virtual base station, so as to implement sharing of the data resources of the cloud servers, and in the data sharing process, data balancing and data resource allocation are performed on the data resources of the cloud servers, so that system space is maximally utilized, and resource utilization rate is improved.
In order to achieve the purpose, the technical scheme of the invention is realized as follows:
a data sharing system for a heterogeneous cloud computing system, the system comprising: heterogeneous cloud server groups and virtual base stations; the heterogeneous cloud server group comprises three heterogeneous cloud servers, which are respectively as follows: the system comprises a first cloud server, a second cloud server and a third cloud server; the virtual base station is configured for scheduling and sharing data in the heterogeneous cloud server group and is respectively in signal connection with the first cloud server, the second cloud server and the third cloud server; the virtual base station includes: a virtual plane, a control plane, and a number of applications; the virtual plane comprises a plurality of virtual machines which are in signal connection to form a block chain network, and each virtual machine is connected with one cloud server in the heterogeneous cloud server group and is only connected with one cloud server; each cloud server is connected with at least one virtual machine; the control plane comprises a plurality of control functions, each control function is independent with each other and is in signal connection with each virtual machine; the types of control functions include at least: backhaul control, link control, measurement, access, resource awareness, resource allocation, and routing; the application is provided for the virtual base station administrator to control the operation of the virtual base station.
Further, the resource in the control function senses the change of the data resource in the blockchain network formed by the virtual machine in real time, if the change of the data resource is sensed to exist in the blockchain network of the virtual machine, the resource allocation in the control function starts to extract the changed data resource, the backhaul control in the control function opens up another shared data resource pool, the changed data resource is stored in the data resource pool in a mirror image mode, the measurement in the control function measures the data stored in the data resource pool in real time to acquire the data resource information in the current data resource pool, the link control in the control function receives the data resource request from each cloud server in the heterogeneous cloud server group in real time, and under the control of the route in the control function, the target data resource is acquired based on the data resource information in the current data resource pool, and returning and sending the acquired data resources to the corresponding cloud server.
Further, the virtual machines receive data from the heterogeneous cloud server, simultaneously exchange information of data resources stored by the virtual machines in real time, and simultaneously perform balanced operation on the data and perform balanced mapping on the data to the plurality of virtual machines in the virtual machine block chain network; the virtual machine includes: the acquiring unit is configured to acquire the load degrees and the P weight values of P weight factors on K other virtual machines, wherein K and P are integers greater than 1; the priority calculation unit is used for carrying out weighted average according to the load degrees of the P weight factors and the P weight values to obtain the priority of each virtual machine; the standard fragment number calculating unit is used for obtaining a standard fragment number according to the priority degree associated with each virtual machine; the balance judging unit is used for judging whether the data distribution meets the data balance condition according to the standard fragment number corresponding to each of the K virtual machines; and the balancing unit is used for carrying out data balancing processing on the K virtual machines if the judgment result of the balancing judgment unit is negative.
Further, the information of the data resources stored by each virtual machine is exchanged in real time, and meanwhile, the method for performing the equalization operation on the data executes the following steps: selecting a target interval from the data, extracting the perception equilibrium characteristics of the target interval as template characteristics, and carrying out initialization positioning on a data characteristic operator; establishing two link tables which are a first link table and a second link table respectively; in the first link table, performing state coordinate route mapping on a data characteristic operator, and marking a prediction region at the position of the route mapped data characteristic operator; in the second link table, extracting perception balance characteristics for each route mapping area, and calculating the similarity between the route mapping area and a target interval; updating the data feature operator weight according to the position contribution far from and near the target and the perception equilibrium feature similarity, determining the position of the estimated target, and obtaining a new target interval; obtaining a next data characteristic operator in the link table according to the size of the data characteristic operator and the resampling estimation value; after the similarity between each route mapping area and a target interval is obtained, the weight of the data feature operator is updated by combining the contribution of the positions far away from and near the target; and after the weight of the prediction data characteristic operator is updated, calculating the optimal prediction position of the target as the position of the estimated target, namely a new target interval.
Further, the method for updating the weight includes: in order to reduce the weight of the data characteristic operator at the non-target position, the weight of the data characteristic operator is adjusted in a self-adaptive manner according to the position contribution far away from the target and the calculated perception equilibrium similarity:
wherein the content of the first and second substances,
refers to the weight of the ith data characteristic operator at the t position,
refers to the similarity of the ith data characteristic operator at the t position,
refers to the coordinate position abscissa of the ith data characteristic operator at the t position,
the coordinate position ordinate of the ith data characteristic operator at the t position is shown, and W and H are the half width and half height of a target interval; x is the number of0The coordinate position abscissa of the 1 st data characteristic operator at the 0 position is referred to; y is0Refers to the ordinate of the coordinate position of the 1 st data feature operator at the 0 position.
A method of data sharing for a heterogeneous cloud computing system, the method performing the steps of: step 1: three heterogeneous cloud servers: the first cloud server, the second cloud server and the third cloud server form a heterogeneous cloud server group; step 2: the virtual base station schedules and shares data in the heterogeneous cloud server group.
Further, the virtual base station includes: a virtual plane, a control plane, and a number of applications; the virtual plane comprises a plurality of virtual machines which are in signal connection to form a block chain network, and each virtual machine is connected with one cloud server in the heterogeneous cloud server group and is only connected with one cloud server; each cloud server is connected with at least one virtual machine; the control plane comprises a plurality of control functions, each control function is independent with each other and is in signal connection with each virtual machine; the types of control functions include at least: backhaul control, link control, measurement, access, resource awareness, resource allocation, and routing; the application is provided for the virtual base station administrator to control the operation of the virtual base station.
Further, the resource in the control function senses the change of the data resource in the blockchain network formed by the virtual machine in real time, if the change of the data resource is sensed to exist in the blockchain network of the virtual machine, the resource allocation in the control function starts to extract the changed data resource, the backhaul control in the control function opens up another shared data resource pool, the changed data resource is stored in the data resource pool in a mirror image mode, the measurement in the control function measures the data stored in the data resource pool in real time to acquire the data resource information in the current data resource pool, the link control in the control function receives the data resource request from each cloud server in the heterogeneous cloud server group in real time, and under the control of the route in the control function, the target data resource is acquired based on the data resource information in the current data resource pool, and returning and sending the acquired data resources to the corresponding cloud server.
Further, the virtual machines receive data from the heterogeneous cloud server, simultaneously exchange information of data resources stored by the virtual machines in real time, and simultaneously perform balanced operation on the data and perform balanced mapping on the data to the plurality of virtual machines in the virtual machine block chain network; the virtual machine includes: the acquiring unit is configured to acquire the load degrees and the P weight values of P weight factors on K other virtual machines, wherein K and P are integers greater than 1; the priority calculation unit is used for carrying out weighted average according to the load degrees of the P weight factors and the P weight values to obtain the priority of each virtual machine; the standard fragment number calculating unit is used for obtaining a standard fragment number according to the priority degree associated with each virtual machine; the balance judging unit is used for judging whether the data distribution meets the data balance condition according to the standard fragment number corresponding to each of the K virtual machines; and the balancing unit is used for carrying out data balancing processing on the K virtual machines if the judgment result of the balancing judgment unit is negative.
Further, the information of the data resources stored by each virtual machine is exchanged in real time, and meanwhile, the method for performing the equalization operation on the data executes the following steps: selecting a target interval from the data, extracting the perception equilibrium characteristics of the target interval as template characteristics, and carrying out initialization positioning on a data characteristic operator; establishing two link tables which are a first link table and a second link table respectively; in the first link table, performing state coordinate route mapping on a data characteristic operator, and marking a prediction region at the position of the route mapped data characteristic operator; in the second link table, extracting perception balance characteristics for each route mapping area, and calculating the similarity between the route mapping area and a target interval; updating the data feature operator weight according to the position contribution far from and near the target and the perception equilibrium feature similarity, determining the position of the estimated target, and obtaining a new target interval; obtaining a next data characteristic operator in the link table according to the size of the data characteristic operator and the resampling estimation value; after the similarity between each route mapping area and a target interval is obtained, the weight of the data feature operator is updated by combining the contribution of the positions far away from and near the target; and after the weight of the prediction data characteristic operator is updated, calculating the optimal prediction position of the target as the position of the estimated target, namely a new target interval.
The data sharing system and method of the heterogeneous cloud computing system have the following beneficial effects: the data resources of the heterogeneous cloud servers are scheduled, distributed and shared through the virtual base station, the data resources of the cloud servers are shared, data equalization and data resource distribution are carried out on the data resources of the cloud servers in the data sharing process, the system space is utilized to the maximum extent, and the resource utilization rate is improved. The method is mainly realized by the following steps: 1. according to the invention, the data sharing is realized independently of the cloud server through the establishment of the virtual base station, and the data sharing can be realized under the condition of ensuring the working efficiency of the cloud server; meanwhile, the virtual base station is also divided into a virtual plane and a control plane, the virtual plane is responsible for storing resources of the cloud server, and the control plane is used for sharing and scheduling control over data of the virtual plane, so that the data sharing and the data storage are separated, and the efficiency is further improved; 2. the data balance calculation of the virtual machine is realized by the virtual machine, after the data is subjected to balance calculation, the data is uniformly mapped to a plurality of virtual machines in a virtual machine block chain network, so that the resource utilization rate of the virtual machine is maximized, and the data balance can ensure that the resources of each virtual machine can be fully utilized(ii) a 3. The invention relates to a data equalization algorithm, wherein in the data equalization process, a weight updating method comprises the following steps: in order to reduce the weight of the data characteristic operator at the non-target position, the weight of the data characteristic operator is adjusted in a self-adaptive manner according to the position contribution far away from the target and the calculated perception equilibrium similarity:
compared with the existing algorithm, the algorithm can obviously improve the efficiency of data equalization.
Drawings
Fig. 1 is a schematic system structure diagram of a data sharing system of a heterogeneous cloud computing system according to an embodiment of the present invention;
fig. 2 is a schematic method flow diagram of a data sharing method of a heterogeneous cloud computing system according to an embodiment of the present invention.
Fig. 3 is a schematic flowchart illustrating a balancing operation of a data sharing system and method of a heterogeneous cloud computing system according to an embodiment of the present invention;
fig. 4 is a schematic diagram of an experimental effect of system operation efficiency of a data sharing system and method of a heterogeneous cloud computing system according to an embodiment of the present invention and a schematic diagram of a comparative experimental effect in the prior art;
fig. 5 is a schematic diagram of an experimental effect of a system resource vacancy rate of a data sharing system and method of a heterogeneous cloud computing system according to an embodiment of the present invention and a schematic diagram of a comparative experimental effect in the prior art.
1-experimental curve of the invention, 2-experimental curve of the prior art, 3-experimental curve without data sharing.
Detailed Description
The method of the present invention will be described in further detail below with reference to the accompanying drawings and embodiments of the invention.
Example 1
As shown in fig. 1, a data sharing system of a heterogeneous cloud computing system, the system comprising: heterogeneous cloud server groups and virtual base stations; the heterogeneous cloud server group comprises three heterogeneous cloud servers, which are respectively as follows: the system comprises a first cloud server, a second cloud server and a third cloud server; the virtual base station is configured for scheduling and sharing data in the heterogeneous cloud server group and is respectively in signal connection with the first cloud server, the second cloud server and the third cloud server; the virtual base station includes: a virtual plane, a control plane, and a number of applications; the virtual plane comprises a plurality of virtual machines which are in signal connection to form a block chain network, and each virtual machine is connected with one cloud server in the heterogeneous cloud server group and is only connected with one cloud server; each cloud server is connected with at least one virtual machine; the control plane comprises a plurality of control functions, each control function is independent with each other and is in signal connection with each virtual machine; the types of control functions include at least: backhaul control, link control, measurement, access, resource awareness, resource allocation, and routing; the application is provided for the virtual base station administrator to control the operation of the virtual base station.
By adopting the technical scheme, the data resources of the heterogeneous cloud servers are scheduled, distributed and shared through the virtual base station, the data resources of the cloud servers are shared, data balance and data resource distribution are carried out on the data resources of the cloud servers in the data sharing process, the system space is utilized to the maximum extent, and the resource utilization rate is improved. The method is mainly realized by the following steps: 1. according to the invention, the data sharing is realized independently of the cloud server through the establishment of the virtual base station, and the data sharing can be realized under the condition of ensuring the working efficiency of the cloud server; meanwhile, the virtual base station is also divided into a virtual plane and a control plane, the virtual plane is responsible for storing resources of the cloud server, and the control plane is used for sharing and scheduling control over data of the virtual plane, so that the data sharing and the data storage are separated, and the efficiency is further improved; 2. the data balance calculation of the virtual machine is realized by the virtual machine which performs balance calculation on the data and then is mapped to a plurality of virtual machines in the virtual machine block chain network in a balanced manner, so that the resource benefit of the virtual machine is ensuredThe utilization rate is maximized, and the data balance can ensure that the resources of each virtual machine can be fully utilized; the invention relates to a data equalization algorithm, wherein in the data equalization process, a weight updating method comprises the following steps: in order to reduce the weight of the data characteristic operator at the non-target position, the weight of the data characteristic operator is adjusted in a self-adaptive manner according to the position contribution far away from the target and the calculated perception equilibrium similarity:
compared with the existing algorithm, the algorithm can obviously improve the efficiency of data equalization.
Example 2
On the basis of the previous embodiment, the resource in the control function senses the change of data resources in a block chain network formed by the virtual machine in real time, if the change of the data resources is sensed to exist in the virtual machine block chain network, the resource allocation in the control function starts to extract the changed data resources, the backhaul control in the control function opens up another shared data resource pool, the changed data resources are stored in the data resource pool in a mirror image mode, the measurement in the control function measures the data stored in the data resource pool in real time to acquire the data resource information in the current data resource pool, the link control in the control function receives the data resource request from each cloud server in the heterogeneous cloud server group in real time, and based on the data resource information in the current data resource pool under the control of the route in the control function, and acquiring the data resource of the target, and returning and sending the acquired data resource to the corresponding cloud server.
Specifically, the service provider server provides a service to the user using cloud computing. The service provider server provides the service resource for the resource owner to use, and the resource owner has the use and sharing authority for the service resource. The resource owner shares the traffic resource to other users. Currently, the sharing scheme is adopted in which a resource owner needs to set resource sharing permission in a service provider server to allow other users to access the shared resource in order to share the shared resource to others. Other users need to log in to the service provider server to see the shared resource when they want to obtain the shared resource.
This approach has a number of disadvantages. First, the manner in which users flexibly use shared resources is limited. The user can obtain the resources stored by the corresponding service provider server in the cloud computing service provider server only by logging in the service site provided by the service provider server. Second, the service provider server is required to have a large service providing capability. The service provider server needs to provide a service equivalent to one resource relay station for a plurality of users, which increases the load pressure of the service provider server, and in a cloud environment, the service provider server wants to realize simple deployment and cost reduction by using the cloud, which is contrary to the original intention of the cloud environment setting, and the burden of the service provider server is undoubtedly increased.
Example 3
On the basis of the previous embodiment, the virtual machines receive data from heterogeneous cloud servers, simultaneously exchange information of data resources stored by the virtual machines in real time, and perform balanced operation on the data and then perform balanced mapping on the data to the multiple virtual machines in the virtual machine block chain network; the virtual machine includes: the acquiring unit is configured to acquire the load degrees and the P weight values of P weight factors on K other virtual machines, wherein K and P are integers greater than 1; the priority calculation unit is used for carrying out weighted average according to the load degrees of the P weight factors and the P weight values to obtain the priority of each virtual machine; the standard fragment number calculating unit is used for obtaining a standard fragment number according to the priority degree associated with each virtual machine; the balance judging unit is used for judging whether the data distribution meets the data balance condition according to the standard fragment number corresponding to each of the K virtual machines; and the balancing unit is used for carrying out data balancing processing on the K virtual machines if the judgment result of the balancing judgment unit is negative.
Specifically, in a distributed database system based on a consistent hash algorithm, a physical node may virtualize a plurality of virtual nodes, and then map the plurality of virtual nodes onto a ring through the hash algorithm, so that the physical node may increase a hash value range mapped on the ring. When a physical node is added or deleted in the distributed database system, the number of virtual nodes associated with each physical node is adjusted to enable the number of virtual nodes associated with each physical node to tend to be equal, and all physical nodes of the distributed database system achieve balance of the virtual nodes. However, under the condition that the data distribution of the virtual nodes has large fluctuation, in order to ensure that the service can be normally performed, surplus of resources of the physical nodes needs to be considered, and the physical nodes are deployed according to the requirement of the maximum data volume, so that the cost is increased; or the surplus of the resources of the physical node is not considered, the physical node may be overloaded, and the service fails.
Example 4
On the basis of the previous embodiment, the information of the data resources stored by each virtual machine is exchanged in real time, and meanwhile, the method for performing the equalization operation on the data executes the following steps: selecting a target interval from the data, extracting the perception equilibrium characteristics of the target interval as template characteristics, and carrying out initialization positioning on a data characteristic operator; establishing two link tables which are a first link table and a second link table respectively; in the first link table, performing state coordinate route mapping on a data characteristic operator, and marking a prediction region at the position of the route mapped data characteristic operator; in the second link table, extracting perception balance characteristics for each route mapping area, and calculating the similarity between the route mapping area and a target interval; updating the data feature operator weight according to the position contribution far from and near the target and the perception equilibrium feature similarity, determining the position of the estimated target, and obtaining a new target interval; obtaining a next data characteristic operator in the link table according to the size of the data characteristic operator and the resampling estimation value; after the similarity between each route mapping area and a target interval is obtained, the weight of the data feature operator is updated by combining the contribution of the positions far away from and near the target; and after the weight of the prediction data characteristic operator is updated, calculating the optimal prediction position of the target as the position of the estimated target, namely a new target interval.
In particular, distributed file systems have become a new trend in the development of computer technology. Distributed storage systems that are rapidly emerging are facing the problem of distributing PB-level data among thousands of storage devices. In such systems, there is a need to distribute data and load to fully utilize the available resources and maximize the performance of the system, while accommodating the growth of the system and managing the failure of hardware devices. Most systems simply write data, and a common problem is that once written, the data is hardly moved any more. The system can also become unstable when the storage devices are expanding because newly added storage devices are either empty or full of new data. Storage devices and system resources are only fully utilized if all of the remaining available resources are fully utilized based on the workload of the system. A robust solution is to distribute all data randomly over the available storage devices. This results in a probability-balanced distribution and confusion of old and new data. When a new storage device is added, random samples of the original data will be moved to the new storage device to keep the system balanced. The key advantage of this approach is that all devices will load equally and the system will perform well under any potential workload.
Example 5
On the basis of the above embodiment, the method for updating the weights includes: in order to reduce the weight of the data characteristic operator at the non-target position, the weight of the data characteristic operator is adjusted in a self-adaptive manner according to the position contribution far away from the target and the calculated perception equilibrium similarity:
wherein the content of the first and second substances,
refers to the weight of the ith data characteristic operator at the t position,
refers to the similarity of the ith data characteristic operator at the t position,
refers to the coordinate position abscissa of the ith data characteristic operator at the t position,
the coordinate position ordinate of the ith data characteristic operator at the t position is shown, and W and H are the half width and half height of a target interval; x is the number of0The coordinate position abscissa of the 1 st data characteristic operator at the 0 position is referred to; y is0Refers to the ordinate of the coordinate position of the 1 st data feature operator at the 0 position.
Example 6
A method of data sharing for a heterogeneous cloud computing system, the method performing the steps of: step 1: three heterogeneous cloud servers: the first cloud server, the second cloud server and the third cloud server form a heterogeneous cloud server group; step 2: the virtual base station schedules and shares data in the heterogeneous cloud server group.
Specifically, the data resources of the heterogeneous cloud servers are scheduled, allocated and shared through the virtual base station, so that the data resources of the cloud servers are shared, data balance and data resource allocation are performed on the data resources of the cloud servers in the data sharing process, the system space is utilized to the maximum extent, and the resource utilization rate is improved. The method is mainly realized by the following steps: 1. according to the invention, the data sharing is realized independently of the cloud server through the establishment of the virtual base station, and the data sharing can be realized under the condition of ensuring the working efficiency of the cloud server; meanwhile, the virtual base station is also divided into a virtual plane and a control plane, the virtual plane is responsible for storing resources of the cloud server, the control plane is used for sharing and scheduling control over data of the virtual plane, and the data sharing and the data storage are separated, so that the improvement is further promotedEfficiency; the data balance calculation of the virtual machines, after the balance calculation is carried out on the data by the virtual machines, the data are mapped to a plurality of virtual machines in a virtual machine block chain network in a balanced mode, the resource utilization rate of the virtual machines is maximized, and the data balance can ensure that the resources of all the virtual machines can be fully utilized; 3. the invention relates to a data equalization algorithm, wherein in the data equalization process, a weight updating method comprises the following steps: in order to reduce the weight of the data characteristic operator at the non-target position, the weight of the data characteristic operator is adjusted in a self-adaptive manner according to the position contribution far away from the target and the calculated perception equilibrium similarity:
compared with the existing algorithm, the algorithm can obviously improve the efficiency of data equalization.
Example 7
On the basis of the previous embodiment, the virtual base station includes: a virtual plane, a control plane, and a number of applications; the virtual plane comprises a plurality of virtual machines which are in signal connection to form a block chain network, and each virtual machine is connected with one cloud server in the heterogeneous cloud server group and is only connected with one cloud server; each cloud server is connected with at least one virtual machine; the control plane comprises a plurality of control functions, each control function is independent with each other and is in signal connection with each virtual machine; the types of control functions include at least: backhaul control, link control, measurement, access, resource awareness, resource allocation, and routing; the application is provided for the virtual base station administrator to control the operation of the virtual base station.
Example 8
On the basis of the previous embodiment, the resource in the control function senses the change of data resources in a block chain network formed by the virtual machine in real time, if the change of the data resources is sensed to exist in the virtual machine block chain network, the resource allocation in the control function starts to extract the changed data resources, the backhaul control in the control function opens up another shared data resource pool, the changed data resources are stored in the data resource pool in a mirror image mode, the measurement in the control function measures the data stored in the data resource pool in real time to acquire the data resource information in the current data resource pool, the link control in the control function receives the data resource request from each cloud server in the heterogeneous cloud server group in real time, and based on the data resource information in the current data resource pool under the control of the route in the control function, and acquiring the data resource of the target, and returning and sending the acquired data resource to the corresponding cloud server.
Example 9
On the basis of the previous embodiment, the virtual machines receive data from heterogeneous cloud servers, simultaneously exchange information of data resources stored by the virtual machines in real time, and perform balanced operation on the data and then perform balanced mapping on the data to the multiple virtual machines in the virtual machine block chain network; the virtual machine includes: the acquiring unit is configured to acquire the load degrees and the P weight values of P weight factors on K other virtual machines, wherein K and P are integers greater than 1; the priority calculation unit is used for carrying out weighted average according to the load degrees of the P weight factors and the P weight values to obtain the priority of each virtual machine; the standard fragment number calculating unit is used for obtaining a standard fragment number according to the priority degree associated with each virtual machine; the balance judging unit is used for judging whether the data distribution meets the data balance condition according to the standard fragment number corresponding to each of the K virtual machines; and the balancing unit is used for carrying out data balancing processing on the K virtual machines if the judgment result of the balancing judgment unit is negative.
Example 10
On the basis of the previous embodiment, the information of the data resources stored by each virtual machine is exchanged in real time, and meanwhile, the method for performing the equalization operation on the data executes the following steps: selecting a target interval from the data, extracting the perception equilibrium characteristics of the target interval as template characteristics, and carrying out initialization positioning on a data characteristic operator; establishing two link tables which are a first link table and a second link table respectively; in the first link table, performing state coordinate route mapping on a data characteristic operator, and marking a prediction region at the position of the route mapped data characteristic operator; in the second link table, extracting perception balance characteristics for each route mapping area, and calculating the similarity between the route mapping area and a target interval; updating the data feature operator weight according to the position contribution far from and near the target and the perception equilibrium feature similarity, determining the position of the estimated target, and obtaining a new target interval; obtaining a next data characteristic operator in the link table according to the size of the data characteristic operator and the resampling estimation value; after the similarity between each route mapping area and a target interval is obtained, the weight of the data feature operator is updated by combining the contribution of the positions far away from and near the target; and after the weight of the prediction data characteristic operator is updated, calculating the optimal prediction position of the target as the position of the estimated target, namely a new target interval.
It can be clearly understood by those skilled in the art that, for convenience and brevity of description, the specific working process and related description of the system described above may refer to the corresponding process in the foregoing method embodiments, and will not be described herein again.
It should be noted that, the system provided in the foregoing embodiment is only illustrated by dividing the functional units, and in practical applications, the functions may be distributed by different functional units according to needs, that is, the units or steps in the embodiments of the present invention are further decomposed or combined, for example, the units in the foregoing embodiment may be combined into one unit, or may be further decomposed into multiple sub-units, so as to complete all or the functions of the units described above. The names of the units and steps involved in the embodiments of the present invention are only for distinguishing the units or steps, and are not to be construed as unduly limiting the present invention.
It can be clearly understood by those skilled in the art that, for convenience and brevity of description, the specific working processes and related descriptions of the storage device and the processing device described above may refer to the corresponding processes in the foregoing method embodiments, and are not described herein again.
Those of skill in the art would appreciate that the various illustrative elements, method steps, described in connection with the embodiments disclosed herein may be implemented as electronic hardware, computer software, or combinations of both, and that programs corresponding to the elements, method steps may be located in Random Access Memory (RAM), memory, Read Only Memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art. To clearly illustrate this interchangeability of electronic hardware and software, various illustrative components and steps have been described above generally in terms of their functionality. Whether such functionality is implemented as electronic hardware or software depends upon the particular application and design constraints imposed on the solution. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present invention.
The terms "first," "second," and the like, are used for distinguishing between similar elements and not necessarily for describing a particular sequential or chronological order.
The terms "comprises," "comprising," or any other similar term are intended to cover a non-exclusive inclusion, such that a process, method, article, or unit/apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or unit/apparatus.
So far, the technical solutions of the present invention have been described in connection with the preferred embodiments shown in the drawings, but it is easily understood by those skilled in the art that the scope of the present invention is obviously not limited to these specific embodiments. Equivalent changes or substitutions of related technical features can be made by those skilled in the art without departing from the principle of the invention, and the technical scheme after the changes or substitutions can fall into the protection scope of the invention.
The above description is only a preferred embodiment of the present invention, and is not intended to limit the scope of the present invention.
Claims (10)
1. A data sharing system for a heterogeneous cloud computing system, the system comprising: heterogeneous cloud server groups and virtual base stations; the heterogeneous cloud server group comprises three heterogeneous cloud servers, which are respectively as follows: the system comprises a first cloud server, a second cloud server and a third cloud server; the virtual base station is configured for scheduling and sharing data in the heterogeneous cloud server group and is respectively in signal connection with the first cloud server, the second cloud server and the third cloud server; the virtual base station includes: a virtual plane, a control plane, and a number of applications; the virtual plane comprises a plurality of virtual machines which are in signal connection to form a block chain network, and each virtual machine is connected with one cloud server in the heterogeneous cloud server group and is only connected with one cloud server; each cloud server is connected with at least one virtual machine; the control plane comprises a plurality of control functions, each control function is independent with each other and is in signal connection with each virtual machine; the types of control functions include at least: backhaul control, link control, measurement, access, resource awareness, resource allocation, and routing; the application is provided for the virtual base station administrator to control the operation of the virtual base station.
2. The system of claim 1, wherein the resources in the control function sense changes in data resources in a blockchain network of virtual machines in real time, if changes in data resources are sensed to exist in the blockchain network of virtual machines, resource allocation in the control function begins to extract the changed data resources, backhaul control in the control function opens up another shared data resource pool, the changed data resources are mirrored and stored in the data resource pool, measurements in the control function measure data stored in the data resource pool in real time, data resource information in a current data resource pool is obtained, link control in the control function receives data resource requests from each cloud server in the heterogeneous cloud server group in real time, based on data resource information in the current data resource pool under control of routing in the control function, and acquiring the data resource of the target, and returning and sending the acquired data resource to the corresponding cloud server.
3. The system of claim 2, wherein the virtual machines receive data from heterogeneous cloud servers, and simultaneously exchange information of data resources stored by the virtual machines in real time, and after performing equalization operation on the data, the data is mapped to the plurality of virtual machines in the virtual machine block chain network in an equalization manner; the virtual machine includes: the acquiring unit is configured to acquire the load degrees and the P weight values of P weight factors on K other virtual machines, wherein K and P are integers greater than 1; the priority calculation unit is used for carrying out weighted average according to the load degrees of the P weight factors and the P weight values to obtain the priority of each virtual machine; the standard fragment number calculating unit is used for obtaining a standard fragment number according to the priority degree associated with each virtual machine; the balance judging unit is used for judging whether the data distribution meets the data balance condition according to the standard fragment number corresponding to each of the K virtual machines; and the balancing unit is used for carrying out data balancing processing on the K virtual machines if the judgment result of the balancing judgment unit is negative.
4. The system of claim 3, wherein the virtual machines exchange information of data resources stored in themselves in real time, and the method for performing the equalization operation on the data performs the following steps: selecting a target interval from the data, extracting the perception equilibrium characteristics of the target interval as template characteristics, and carrying out initialization positioning on a data characteristic operator; establishing two link tables which are a first link table and a second link table respectively; in the first link table, performing state coordinate route mapping on a data characteristic operator, and marking a prediction region at the position of the route mapped data characteristic operator; in the second link table, extracting perception balance characteristics for each route mapping area, and calculating the similarity between the route mapping area and a target interval; updating the data feature operator weight according to the position contribution far from and near the target and the perception equilibrium feature similarity, determining the position of the estimated target, and obtaining a new target interval; obtaining a next data characteristic operator in the link table according to the size of the data characteristic operator and the resampling estimation value; after the similarity between each route mapping area and a target interval is obtained, the weight of the data feature operator is updated by combining the contribution of the positions far away from and near the target; and after the weight of the prediction data characteristic operator is updated, calculating the optimal prediction position of the target as the position of the estimated target, namely a new target interval.
5. The system of claim 4, wherein the update weight method is: in order to reduce the weight of the data characteristic operator at the non-target position, the weight of the data characteristic operator is adjusted in a self-adaptive manner according to the position contribution far away from the target and the calculated perception equilibrium similarity:
wherein the content of the first and second substances,
refers to the weight of the ith data characteristic operator at the t position,
refers to the similarity of the ith data characteristic operator at the t position,
refers to the coordinate position abscissa of the ith data characteristic operator at the t position,
the coordinate position ordinate of the ith data characteristic operator at the t position is shown, and W and H are the half width and half height of a target interval; x is the number of0The coordinate position abscissa of the 1 st data characteristic operator at the 0 position is referred to; y is0Refers to the ordinate of the coordinate position of the 1 st data feature operator at the 0 position.
6. A data sharing method of a heterogeneous cloud computing system based on the system of any one of claims 1 to 5, wherein the method performs the following steps: step 1: three heterogeneous cloud servers: the first cloud server, the second cloud server and the third cloud server form a heterogeneous cloud server group; step 2: the virtual base station schedules and shares data in the heterogeneous cloud server group.
7. The method of claim 6, wherein the virtual base station comprises: a virtual plane, a control plane, and a number of applications; the virtual plane comprises a plurality of virtual machines which are in signal connection to form a block chain network, and each virtual machine is connected with one cloud server in the heterogeneous cloud server group and is only connected with one cloud server; each cloud server is connected with at least one virtual machine; the control plane comprises a plurality of control functions, each control function is independent with each other and is in signal connection with each virtual machine; the types of control functions include at least: backhaul control, link control, measurement, access, resource awareness, resource allocation, and routing; the application is provided for the virtual base station administrator to control the operation of the virtual base station.
8. The method of claim 7, wherein the resources in the control function sense changes in data resources in a blockchain network of the real-time aware virtual machines, and if a change in data resources is sensed in the blockchain network of the virtual machines, resource allocation in the control function starts to extract the changed data resources, backhaul control in the control function opens up another shared data resource pool, the changed data resources are mirrored and stored in the data resource pool, measurements in the control function measure data stored in the data resource pool in real-time, and data resource information in the current data resource pool is obtained, link control in the control function receives data resource requests from each cloud server in the heterogeneous cloud server group in real-time, and based on data resource information in the current data resource pool under control of routing in the control function, and acquiring the data resource of the target, and returning and sending the acquired data resource to the corresponding cloud server.
9. The method according to claim 8, wherein the virtual machines receive data from heterogeneous cloud servers, simultaneously, the virtual machines exchange information of data resources stored by the virtual machines in real time, and simultaneously, after equalization operation is performed on the data, the data are mapped to a plurality of virtual machines in a virtual machine block chain network in an equalization mode; the virtual machine includes: the acquiring unit is configured to acquire the load degrees and the P weight values of P weight factors on K other virtual machines, wherein K and P are integers greater than 1; the priority calculation unit is used for carrying out weighted average according to the load degrees of the P weight factors and the P weight values to obtain the priority of each virtual machine; the standard fragment number calculating unit is used for obtaining a standard fragment number according to the priority degree associated with each virtual machine; the balance judging unit is used for judging whether the data distribution meets the data balance condition according to the standard fragment number corresponding to each of the K virtual machines; and the balancing unit is used for carrying out data balancing processing on the K virtual machines if the judgment result of the balancing judgment unit is negative.
10. The method as claimed in claim 9, wherein the virtual machines exchange information of data resources stored in themselves in real time, and the method for performing the equalization operation on the data performs the following steps: selecting a target interval from the data, extracting the perception equilibrium characteristics of the target interval as template characteristics, and carrying out initialization positioning on a data characteristic operator; establishing two link tables which are a first link table and a second link table respectively; in the first link table, performing state coordinate route mapping on a data characteristic operator, and marking a prediction region at the position of the route mapped data characteristic operator; in the second link table, extracting perception balance characteristics for each route mapping area, and calculating the similarity between the route mapping area and a target interval; updating the data feature operator weight according to the position contribution far from and near the target and the perception equilibrium feature similarity, determining the position of the estimated target, and obtaining a new target interval; obtaining a next data characteristic operator in the link table according to the size of the data characteristic operator and the resampling estimation value; after the similarity between each route mapping area and a target interval is obtained, the weight of the data feature operator is updated by combining the contribution of the positions far away from and near the target; and after the weight of the prediction data characteristic operator is updated, calculating the optimal prediction position of the target as the position of the estimated target, namely a new target interval.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011603113.2A CN112822235A (en) | 2020-12-29 | 2020-12-29 | Data sharing system and method of heterogeneous cloud computing system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011603113.2A CN112822235A (en) | 2020-12-29 | 2020-12-29 | Data sharing system and method of heterogeneous cloud computing system |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN112822235A true CN112822235A (en) | 2021-05-18 |
Family
ID=75855981
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202011603113.2A Pending CN112822235A (en) | 2020-12-29 | 2020-12-29 | Data sharing system and method of heterogeneous cloud computing system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112822235A (en) |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113342611A (en) * | 2021-06-18 | 2021-09-03 | 山东产研信息与人工智能融合研究院有限公司 | Method and system for accurately positioning virtual computing resources |
| CN114296953A (en) * | 2021-12-13 | 2022-04-08 | 中信云网有限公司 | Multi-cloud heterogeneous system and task processing method |
| CN116361120A (en) * | 2023-05-31 | 2023-06-30 | 山东浪潮科学研究院有限公司 | Method, device, equipment and medium for managing and scheduling heterogeneous resources of database |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103488540A (en) * | 2013-09-30 | 2014-01-01 | 华南理工大学 | Energy consumption optimization resource scheduling method for heterogeneous cloud data center |
| WO2016093390A1 (en) * | 2014-12-10 | 2016-06-16 | 엘지전자 주식회사 | Method for generating signal for reducing interference in user-centralized virtual cell in cloud wireless connection network environment, and apparatus therefor |
| CN106899518A (en) * | 2017-02-27 | 2017-06-27 | 腾讯科技（深圳）有限公司 | A kind of method for processing resource and device based on Internet data center |
| CN110290075A (en) * | 2019-04-17 | 2019-09-27 | 李士锋 | A kind of method for managing resource and system of police cloud computing platform |
| CN111629040A (en) * | 2020-05-20 | 2020-09-04 | 中移雄安信息通信科技有限公司 | Cloud edge coordination method, device, system, equipment and medium |
| CN112100185A (en) * | 2020-11-03 | 2020-12-18 | 江西云本数字科技有限公司 | Indexing system and method for block chain data balance load |
-
2020
- 2020-12-29 CN CN202011603113.2A patent/CN112822235A/en active Pending
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103488540A (en) * | 2013-09-30 | 2014-01-01 | 华南理工大学 | Energy consumption optimization resource scheduling method for heterogeneous cloud data center |
| WO2016093390A1 (en) * | 2014-12-10 | 2016-06-16 | 엘지전자 주식회사 | Method for generating signal for reducing interference in user-centralized virtual cell in cloud wireless connection network environment, and apparatus therefor |
| CN106899518A (en) * | 2017-02-27 | 2017-06-27 | 腾讯科技（深圳）有限公司 | A kind of method for processing resource and device based on Internet data center |
| CN110290075A (en) * | 2019-04-17 | 2019-09-27 | 李士锋 | A kind of method for managing resource and system of police cloud computing platform |
| CN111629040A (en) * | 2020-05-20 | 2020-09-04 | 中移雄安信息通信科技有限公司 | Cloud edge coordination method, device, system, equipment and medium |
| CN112100185A (en) * | 2020-11-03 | 2020-12-18 | 江西云本数字科技有限公司 | Indexing system and method for block chain data balance load |
Cited By (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113342611A (en) * | 2021-06-18 | 2021-09-03 | 山东产研信息与人工智能融合研究院有限公司 | Method and system for accurately positioning virtual computing resources |
| CN113342611B (en) * | 2021-06-18 | 2023-07-28 | 山东产研信息与人工智能融合研究院有限公司 | Method and system for precisely positioning virtual computing resources |
| CN114296953A (en) * | 2021-12-13 | 2022-04-08 | 中信云网有限公司 | Multi-cloud heterogeneous system and task processing method |
| CN116361120A (en) * | 2023-05-31 | 2023-06-30 | 山东浪潮科学研究院有限公司 | Method, device, equipment and medium for managing and scheduling heterogeneous resources of database |
| CN116361120B (en) * | 2023-05-31 | 2023-08-15 | 山东浪潮科学研究院有限公司 | Method, device, equipment and medium for managing and scheduling heterogeneous resources of database |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN112822235A (en) | Data sharing system and method of heterogeneous cloud computing system | |
| CN109218355B (en) | Load balancing engine, client, distributed computing system and load balancing method | |
| US10733029B2 (en) | Movement of services across clusters | |
| CN109618002B (en) | Micro-service gateway optimization method, device and storage medium | |
| CN110601992B (en) | Data processing method and device of intelligent measurement terminal based on edge calculation | |
| CN107196982B (en) | User request processing method and device | |
| Rath | Resource provision and QoS support with added security for client side applications in cloud computing | |
| US20140372611A1 (en) | Assigning method, apparatus, and system | |
| CN109857542B (en) | Calculation resource adjusting method, system and device | |
| CN108399101A (en) | The methods, devices and systems of scheduling of resource | |
| CN110704167A (en) | Method, device, equipment and storage medium for creating virtual machine | |
| CN111614657A (en) | Mobile edge security service method and system based on mode selection | |
| GB2610144A (en) | Low trust privileged access management | |
| CN109726005A (en) | For managing method, server system and the computer program product of resource | |
| CN106131244A (en) | A kind of message transmitting method and device | |
| CN113608869A (en) | Task scheduling method and device, electronic equipment and computer storage medium | |
| CN113014611B (en) | Load balancing method and related equipment | |
| CN115118784A (en) | Computing resource scheduling method, device and system | |
| Saravanakumar et al. | An Efficient On-Demand Virtual Machine Migration in Cloud Using Common Deployment Model. | |
| CN113315825A (en) | Distributed request processing method, device, equipment and storage medium | |
| US20170195192A1 (en) | Automated deployment of cloud-hosted, distributed network monitoring agents | |
| CN113312339A (en) | Data migration method and device, computer equipment and storage medium | |
| CN108289086A (en) | Request processing method and device, server | |
| CN116703071A (en) | Resource sharing method, device and equipment and storage medium | |
| CN112738247B (en) | Cloud computing resource distribution system and method based on multi-layer space scheduling |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |