CN116991324B - Cloud hard disk data capacity expansion method and system based on storage virtualization - Google Patents

Cloud hard disk data capacity expansion method and system based on storage virtualization Download PDF

Info

Publication number
CN116991324B
CN116991324B CN202310993842.0A CN202310993842A CN116991324B CN 116991324 B CN116991324 B CN 116991324B CN 202310993842 A CN202310993842 A CN 202310993842A CN 116991324 B CN116991324 B CN 116991324B
Authority
CN
China
Prior art keywords
capacity
ranking
storage
container
similarity
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN202310993842.0A
Other languages
Chinese (zh)
Other versions
CN116991324A (en
Inventor
陈武
骆健
文曦
吴康林
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Shenzhen Yuncunbao Technology Co ltd
Original Assignee
Shenzhen Yuncunbao Technology Co ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Shenzhen Yuncunbao Technology Co ltd filed Critical Shenzhen Yuncunbao Technology Co ltd
Priority to CN202310993842.0A priority Critical patent/CN116991324B/en
Publication of CN116991324A publication Critical patent/CN116991324A/en
Application granted granted Critical
Publication of CN116991324B publication Critical patent/CN116991324B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
    • G06F3/0601—Interfaces specially adapted for storage systems
    • G06F3/0602—Interfaces specially adapted for storage systems specifically adapted to achieve a particular effect
    • G06F3/0614—Improving the reliability of storage systems
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
    • G06F3/0601—Interfaces specially adapted for storage systems
    • G06F3/0628—Interfaces specially adapted for storage systems making use of a particular technique
    • G06F3/0646—Horizontal data movement in storage systems, i.e. moving data in between storage devices or systems
    • G06F3/0647—Migration mechanisms
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
    • G06F3/0601—Interfaces specially adapted for storage systems
    • G06F3/0628—Interfaces specially adapted for storage systems making use of a particular technique
    • G06F3/0662—Virtualisation aspects
    • G06F3/0664—Virtualisation aspects at device level, e.g. emulation of a storage device or system
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
    • G06F3/0601—Interfaces specially adapted for storage systems
    • G06F3/0668—Interfaces specially adapted for storage systems adopting a particular infrastructure
    • G06F3/067—Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS]

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Human Computer Interaction (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)

Abstract

The application provides a cloud hard disk data capacity expansion method and a system based on storage virtualization, wherein the system and the method comprise the steps of establishing a storage virtualization platform, and establishing an abstract layer and a storage infrastructure; performing content identification on the capacity expansion request, sending the capacity expansion request to a storage infrastructure, calculating and obtaining similarity sorting according to the similarity, and obtaining capacity sorting to obtain capacity expansion container sorting; and the storage virtualization facility mounts the persistent volumes into the expanded containers according to the ranking order of the expanded containers, and performs persistent storage on the data. The capacity expansion request can be responded and processed quickly, and the workload of manual operation and the possibility of errors are reduced. By using the similarity sorting and the capacity sorting, a proper container can be effectively selected to meet the capacity expansion requirement, and the utilization efficiency of resources is improved. By mounting the persistent volume and data migration, the persistent storage and seamless migration of the data are ensured, and the reliability and usability of the data are ensured.

Description

Cloud hard disk data capacity expansion method and system based on storage virtualization
Technical Field
The invention provides a cloud hard disk data capacity expansion method and system based on storage virtualization, relates to the technical field of cloud hard disk capacity expansion, and particularly relates to the technical field of cloud hard disk capacity expansion based on storage virtualization.
Background
In the current technological development, the storage technology of the cloud hard disk provides many convenience for people in the field of information storage, but in the specific use process, many problems still exist, such as lack of a suitable and intelligent capacity expansion method for storage virtualization of the cloud hard disk, lack of planning and management of capacity, low capacity expansion efficiency and low reliability.
Disclosure of Invention
The invention provides a capacity expansion method for solving the problems of lack of proper storage virtualization of a cloud hard disk and high intelligent degree, lack of planning and management of capacity, low capacity expansion efficiency and low reliability:
The invention provides a cloud hard disk data capacity expansion method and a system based on storage virtualization, wherein the method comprises the following steps:
S1, establishing a storage virtualization platform, and establishing an abstract layer and a storage infrastructure in the storage virtualization platform;
S2, after receiving the capacity expansion request, the abstract layer performs content identification on the capacity expansion request and sends the capacity expansion request to a storage infrastructure, the storage infrastructure obtains similarity sorting through calculating the similarity, and the capacity sorting is obtained through sorting the capacity of each container;
s3, according to parameters in the similarity sorting and the capacity sorting, obtaining the sorting of the capacity-expanded containers;
and S4, the storage virtualization facility mounts the persistent volume into the container after capacity expansion according to the ranking order of the capacity expansion containers, performs persistent storage on the data, migrates the old data and updates the capacity value of the container through an abstract layer.
Further, the establishing a storage virtualization platform, establishing an abstraction layer and a storage infrastructure in the storage virtualization platform, includes:
S101, establishing a storage virtualization platform in a virtualization cluster, wherein the storage virtualization platform comprises an abstract layer and a storage infrastructure;
S102, the interface of the abstract layer is connected with the interface of the storage infrastructure, and the abstract layer is communicated with the storage infrastructure through the interface.
Further, after the abstract layer receives the capacity expansion request, performing content identification on the capacity expansion request and sending the capacity expansion request to a storage infrastructure, wherein the storage infrastructure obtains a similarity ranking by calculating the similarity, and obtains a capacity ranking by ranking the capacities of the containers, and the method comprises the following steps:
s201, after a cloud hard disk data capacity expansion system receives a capacity expansion request, the abstract layer identifies request content of the capacity expansion request, obtains identification content and notifies the identification content to the storage virtualization facility;
S202, checking the residual capacity of the cloud hard disk in the cloud hard disk data expansion system at preset time by the storage virtualization facility, calibrating the residual capacity of the cloud hard disk at preset time, respectively marking all containers of the POD of the cloud hard disk, calculating the similarity of the data content in all the containers of the POD of the cloud hard disk, sorting the similarity of the data content in all the containers according to the sequence from large to small to obtain a similarity sorting, and sorting the residual capacity quota in all the containers according to the capacity from large to small to obtain the capacity sorting.
Further, the obtaining the dilatation container ranking according to the parameters in the similarity ranking and the capacity ranking includes:
S301, searching containers corresponding to the first capacity in the capacity sorting by taking the capacity sorting as a reference, searching containers corresponding to the second capacity in the capacity sorting by using the first capacity in the similarity sorting as a first similarity sorting, and stopping searching until all containers have obtained the similarity sorting by using the second similarity sorting;
S302, setting a similarity threshold, calculating the difference value between each similarity ranking and the similarity threshold, sorting the difference value from small to large, calculating the sum of the difference value corresponding to each container and the capacity ranking, obtaining a summation result, ranking the summation result from small to large, obtaining summation ranking, obtaining dilatation container ranking according to the summation ranking, wherein each ranking in the dilatation container ranking is the container ranking corresponding to each ranking in the summation ranking.
Further, the storage virtualization facility mounts the persistent volumes into the expanded containers according to the ranking order of the expanded containers, performs persistent storage on the data, migrates old data, and updates the capacity values of the containers through an abstraction layer, including:
S401, after the storage virtualization facility receives the initial persistence request, searching for an available persistence volume, binding the persistence volume with the initial persistence volume statement, calculating the ranking of the volume expansion containers of the persistence volume according to the storage data resources of the persistence volume, expanding the containers according to the ranking sequence of the volume expansion containers, mounting the persistence volume into the expanded containers according to the ranking sequence of the volume expansion containers through a file system, and performing persistence storage on the data resources in the persistence volume.
S402, after the capacity expansion is completed, transferring the old data from the old data container to the expanded container; after the abstract layer detects that the data migration is completed, the capacity values of the old container and the expanded container are modified and updated.
Further, the system comprises:
The platform building module is used for building a storage virtualization platform, and building an abstract layer and a storage infrastructure in the storage virtualization platform;
the ordering module is used for carrying out content identification on the capacity expansion request and sending the capacity expansion request to the storage infrastructure after the abstract layer receives the capacity expansion request, the storage infrastructure obtains similarity ordering by calculating the similarity, and the capacity ordering is obtained by ordering the capacity of each container;
the capacity expansion sequencing module is used for obtaining the sequencing of the capacity expansion containers according to the parameters in the similarity sequencing and the capacity sequencing;
And the persistent storage module is used for mounting the persistent volumes into the expanded containers according to the ranking order of the expanded containers by the storage virtualization facility, performing persistent storage on the data, migrating the old data and updating the capacity values of the containers through the abstract layer.
Further, the platform establishment module includes:
the virtualization platform module is used for establishing a storage virtualization platform in the virtualization cluster, wherein the storage virtualization platform comprises an abstract layer and a storage infrastructure;
And the interface communication module is used for connecting an interface of the abstract layer with an interface of the storage infrastructure, and the abstract layer and the storage infrastructure communicate through the interface.
Further, the sorting module includes:
The content identification module is used for carrying out identification of request content on the capacity expansion request by the abstract layer after the capacity expansion request is received by the cloud hard disk data capacity expansion system, obtaining identification content and informing the identification content to the storage virtualization facility;
The ordering acquisition module is used for enabling the storage virtualization facility to check the residual capacity of the cloud hard disk in the cloud hard disk data expansion system at preset time, calibrating the residual capacity of the cloud hard disk at preset time, respectively marking all containers of the POD of the cloud hard disk, calculating the similarity of the data content in all containers of the POD of the cloud hard disk, ordering the similarity of the data content in all containers according to the order from large to small to obtain similarity ordering, and ordering the residual capacity quota in all containers according to the capacity from large to small to obtain capacity ordering.
Further, the capacity expansion ordering module includes:
The similarity ranking module is used for searching containers corresponding to the first capacity in the capacity ranking based on the capacity ranking, wherein the ranking in the similarity ranking is called a first similarity ranking, then searching containers corresponding to the second capacity in the capacity ranking, and stopping searching until all containers have obtained the similarity ranking;
the container ranking module is used for setting a similarity threshold, calculating the difference value between each similarity ranking and the similarity threshold, sequencing the difference value from small to large, calculating the sum of the difference value corresponding to each container and the capacity ranking, obtaining a summation result, ranking the summation result from small to large, obtaining summation ranking, obtaining dilatation container ranking according to the summation ranking, wherein each ranking in the dilatation container ranking is the container ranking corresponding to each ranking in the summation ranking.
Further, the persistent storage module includes:
And the capacity expansion module is used for searching available persistence volumes after the storage virtualization facility receives the initial persistence request, binding the persistence volumes with the initial persistence volume declarations, calculating the capacity expansion container rank of the persistence volumes according to the storage data resources of the persistence volumes, expanding the containers according to the capacity expansion container rank according to the ranking order, mounting the persistence volumes into the expanded containers through a file system according to the capacity expansion container rank order, and carrying out persistence storage on the data resources in the persistence volumes.
The migration updating module is used for migrating the old data from the old data container to the expanded container after the capacity expansion is completed; after the abstract layer detects that the data migration is completed, the capacity values of the old container and the expanded container are modified and updated.
The invention has the beneficial effects that:
The application provides a cloud hard disk data capacity expansion method and a cloud hard disk data capacity expansion system based on storage virtualization, which can abstract a storage infrastructure of a bottom layer by establishing a storage virtualization platform, provide a uniform interface and a management mechanism and simplify management and use of storage resources. After the abstract layer of the storage virtualization platform receives the capacity expansion request, the request is analyzed and parsed through a content recognition technology, so that the user requirements can be accurately understood. The storage infrastructure can compare the expansion request with the existing container by calculating the similarity, and find the container with the highest similarity with the request content. In addition, by sorting the capacities of the individual containers, the container with the most appropriate capacity can be determined. And according to the results of the similarity sorting and the capacity sorting, the sorting of one capacity expansion container can be obtained. I.e., determining in which order the containers should be expanded, it is the container that is most suitable for the persistence volume and persistence request that ensures that the first expansion is. The storage virtualization facility mounts the persistent volume into the expanded container according to the ordering of the expanded containers, ensuring that the new container can access and use the storage resources. At the same time, old data is required to be migrated so as to ensure the integrity and consistency of the data. Through the abstraction layer, the storage virtualization platform can update the capacity value of the container to reflect the new capacity of the expanded container. By the automated mechanism of the storage virtualization platform, capacity expansion requests can be responded and processed quickly, and the workload of manual operation and the possibility of errors are reduced. By using the similarity sorting and the capacity sorting, a proper container can be effectively selected to meet the capacity expansion requirement, and the utilization efficiency of resources is improved. By mounting the persistent volume and data migration, the persistent storage and seamless migration of the data are ensured, and the reliability and usability of the data are ensured. The storage virtualization platform provides an abstraction layer, and can dynamically adjust and expand storage resources according to the needs to adapt to different business demands and scale changes. The expandability, flexibility and reliability of the storage resources are enhanced, the management and operation flow is simplified, and the efficiency and usability of the storage system are improved. The application can save similar data to the same area to the maximum extent or provide enough storage space for the similar data.
Drawings
Fig. 1 is a schematic diagram of a cloud hard disk data expansion method based on storage virtualization.
Detailed Description
The preferred embodiments of the present invention will be described below with reference to the accompanying drawings, it being understood that the preferred embodiments described herein are for illustration and explanation of the present invention only, and are not intended to limit the present invention.
The invention provides a cloud hard disk data capacity expansion method and a system based on storage virtualization, wherein the method comprises the following steps:
S1, establishing a storage virtualization platform, and establishing an abstract layer and a storage infrastructure in the storage virtualization platform;
S2, after receiving the capacity expansion request, the abstract layer performs content identification on the capacity expansion request and sends the capacity expansion request to a storage infrastructure, the storage infrastructure obtains similarity sorting through calculating the similarity, and the capacity sorting is obtained through sorting the capacity of each container;
s3, according to parameters in the similarity sorting and the capacity sorting, obtaining the sorting of the capacity-expanded containers;
and S4, the storage virtualization facility mounts the persistent volume into the container after capacity expansion according to the ranking order of the capacity expansion containers, performs persistent storage on the data, migrates the old data and updates the capacity value of the container through an abstract layer.
The working principle of the technical scheme is as follows: firstly, establishing a storage virtualization platform, and establishing an abstract layer and a storage infrastructure in the storage virtualization platform; then after receiving the capacity expansion request, the abstract layer carries out content identification on the capacity expansion request and sends the capacity expansion request to a storage infrastructure, wherein the content identification comprises capacity expansion capacity of the capacity expansion request and capacity expansion mode of the request, the storage infrastructure obtains similarity sorting through calculating the similarity, and the capacity sorting is obtained through sorting the capacity of each container; obtaining the sequencing of the capacity expansion containers according to parameters in the similarity sequencing and the capacity sequencing; and finally, the storage virtualization facility mounts the persistent volumes into the expanded containers according to the ranking order of the expanded containers, performs persistent storage on the data, migrates the old data and updates the capacity values of the containers through an abstract layer.
The technical effects of the technical scheme are as follows: by establishing a storage virtualization platform, the underlying storage infrastructure can be abstracted, a unified interface and management mechanism are provided, and management and use of storage resources are simplified. After the abstract layer of the storage virtualization platform receives the capacity expansion request, the request is analyzed and parsed through a content recognition technology, so that the user requirements can be accurately understood. The storage infrastructure can compare the expansion request with the existing container by calculating the similarity, and find the container with the highest similarity with the request content. In addition, by sorting the capacities of the individual containers, the container with the most appropriate capacity can be determined. And according to the results of the similarity sorting and the capacity sorting, the sorting of one capacity expansion container can be obtained. I.e., determining in which order the containers should be expanded, it is the container that is most suitable for the persistence volume and persistence request that ensures that the first expansion is. The storage virtualization facility mounts the persistent volume into the expanded container according to the ordering of the expanded containers, ensuring that the new container can access and use the storage resources. At the same time, old data is required to be migrated so as to ensure the integrity and consistency of the data. Through the abstraction layer, the storage virtualization platform can update the capacity value of the container to reflect the new capacity of the expanded container. By the automated mechanism of the storage virtualization platform, capacity expansion requests can be responded and processed quickly, and the workload of manual operation and the possibility of errors are reduced. By using the similarity sorting and the capacity sorting, a proper container can be effectively selected to meet the capacity expansion requirement, and the utilization efficiency of resources is improved. By mounting the persistent volume and data migration, the persistent storage and seamless migration of the data are ensured, and the reliability and usability of the data are ensured. The storage virtualization platform provides an abstraction layer, and can dynamically adjust and expand storage resources according to the needs to adapt to different business demands and scale changes.
In one embodiment of the present invention, the establishing a storage virtualization platform, establishing an abstraction layer and a storage infrastructure in the storage virtualization platform, includes:
S101, establishing a storage virtualization platform in a virtualization cluster, wherein the storage virtualization platform comprises an abstract layer and a storage infrastructure;
S102, the interface of the abstract layer is connected with the interface of the storage infrastructure, and the abstract layer is communicated with the storage infrastructure through the interface.
The working principle of the technical scheme is as follows: firstly, a storage virtualization platform is established in a virtualization cluster, wherein the storage virtualization platform comprises an abstract layer and a storage infrastructure; and finally, the interface of the abstract layer is connected with the interface of the storage infrastructure, and the abstract layer and the storage infrastructure are communicated through the interface.
The technical effects of the technical scheme are as follows: the interface of the abstract layer is connected with the interface of the storage infrastructure, and unified management and control are realized through the interfaces connecting the two components. This means that an administrator or user can manage the storage infrastructure through the interface of the abstraction layer without having to interact directly with the underlying storage device, simplifying the management operations. By building a storage virtualization platform, storage resources can be abstracted from physical devices to form a logical storage pool. In this way, the storage capacity can be conveniently expanded in the virtualized cluster, new storage resources are added to the storage pool, and management and allocation are performed through the interface of the abstraction layer. An administrator can flexibly configure and adjust storage capacity, performance, access rights and the like through an interface of an abstract layer so as to meet the requirements of different applications and services. Through the abstraction layer of the storage virtualization platform, an administrator may perform snapshot and backup operations on storage volumes of a virtual machine or container without concern for underlying storage device details. This simplifies the flow of backup and restore operations and improves the reliability and speed of data recovery. The abstract layer of the storage virtualization platform can realize a data redundancy and fault isolation mechanism, and the availability and safety of storage data are ensured. When the storage infrastructure fails, the abstract layer can be automatically switched to the standby storage equipment or recovered by using redundant data, so that the continuity of the service is ensured. The storage virtualization platform is established and connected with the abstract layer and the storage infrastructure, a unified management interface and an operation interface can be provided, the expandability, flexibility and reliability of storage resources are enhanced, management and operation processes are simplified, and the efficiency and usability of the storage system are improved.
In one embodiment of the present invention, after the abstract layer receives the capacity expansion request, the abstract layer performs content identification on the capacity expansion request and sends the capacity expansion request to a storage infrastructure, the storage infrastructure obtains a similarity ranking by calculating a similarity, and obtains a capacity ranking by ranking the capacities of the containers, including:
s201, after a cloud hard disk data capacity expansion system receives a capacity expansion request, the abstract layer identifies request content of the capacity expansion request, obtains identification content and notifies the identification content to the storage virtualization facility;
S202, checking the residual capacity of the cloud hard disk in the cloud hard disk data expansion system at preset time by the storage virtualization facility, calibrating the residual capacity of the cloud hard disk at preset time, respectively marking all containers of the POD of the cloud hard disk, calculating the similarity of the data content in all the containers of the POD of the cloud hard disk, sorting the similarity of the data content in all the containers according to the sequence from large to small to obtain a similarity sorting, and sorting the residual capacity quota in all the containers according to the capacity from large to small to obtain the capacity sorting. When the data content does not exist in the container of the POD, after the similarity sorting is completed, the container without the data content is sorted from big to small according to the residual capacity and is sorted in the backward direction after the similarity sorting, and the container without the data content is used as the similarity sorting, is sorted according to the residual capacity, and does not influence the subsequent capacity sorting.
The calculation formula of the similarity of the data content is as follows:
Wherein X is the similarity of data content, J is the data capacity retrieval times, c 1 is the data capacity value in any one container, c 2 is the total data capacity value of all containers, f 1 is the number of repetitions of a single character in one container, f 2 is the average of the number of repetitions of a single character in all containers, J 1 is the number of repetitions of a single character in a persistence volume, and J 2 is the average of the number of repetitions of a single character in all containers.
The working principle of the technical scheme is as follows: after a cloud hard disk data capacity expansion system receives a capacity expansion request, the abstract layer identifies request contents of the capacity expansion request, obtains identification contents, and notifies the identification contents to the storage virtualization facility; the storage virtualization facility checks the residual capacity of the cloud hard disk in the cloud hard disk data expansion system at preset time, marks the residual capacity of the cloud hard disk at preset time, marks each container of the POD of the cloud hard disk, calculates the similarity of the data content in each container of the POD of the cloud hard disk, sorts the similarity of the data content in each container according to the sequence from large to small to obtain a similarity sorting, and sorts the residual capacity quota in each container according to the capacity from large to small to obtain the capacity sorting. The individual characters in the formula are set by the user. The preset time is set by the user according to the needs. The calculation or comparison of the similarity is based on the data resources of the persistent volume bound to the persistent declaration, i.e., the more similar the data content of each container is to the data resources of the persistent volume, the higher the similarity is.
The technical effects of the technical scheme are as follows: the abstraction layer identifies the expansion request, ensuring that the contents of the request are correctly understood. This may help determine the cloud hard disk or container that needs to be expanded and determine the corresponding process flow. The storage virtualization facility checks the residual capacity of the cloud hard disk within the preset time, calibrates the residual capacity, facilitates subsequent searching and tracing by calibrating the residual capacity, is favorable for knowing the condition of the available capacity, and serves as a basis for subsequent calculation and sequencing. For each container of the cloud hard disk, a similarity size of the data content is calculated in the storage virtualization facility. This helps determine which containers of data content are relevant to the expansion request and provides a basis for subsequent ordering. And sequencing the residual quotas of the containers according to the similarity from large to small according to the similarity of the data content in the containers. The remaining capacity in the container will then be ordered from large to small. Thus, the container most suitable for capacity expansion can be found, and the reasonable utilization of the residual capacity is ensured. According to the technical scheme, the expansion request can be identified and processed, and the expansion requests are ordered according to the residual capacity of the cloud hard disk and the container, the similarity of data content and quota, so that the optimal expansion target is selected, and the expansion efficiency and accuracy are improved. The similarity of the data can be quantitatively evaluated by calculating the similarity of the data content through information such as the data capacity retrieval times, the data capacity values in the containers, the total data capacity values of all the containers, the repeated number of single characters in the persistent volume, the repeated number of single characters in one container, the repeated number average value of single characters in all the containers and the like. This helps determine the degree of similarity between different containers or datasets, thereby providing basis for subsequent data processing and decision making. The storage capacity can be planned and managed by analyzing the capacity value of the data in the container and the total capacity value of the overall data. This helps to optimize the utilization of storage resources, ensure that the capacity of the container meets the requirements, and avoid over-allocation or resource waste. The quality of the data can be evaluated by calculating the index such as the number of repetitions of a single character in one container and the average of the number of repetitions of a single character in all containers. This helps to detect duplicates, deletions or erroneous data in the data set and take corresponding action for repair and data quality improvement. The method can realize similarity analysis, data screening and optimization, capacity planning and resource utilization optimization of data, and evaluation and improvement of data quality. The technical effects can improve the efficiency, the data quality and the resource utilization rate of the storage system, and promote the accuracy and the reliability of data management and decision making.
According to one embodiment of the present invention, the obtaining the expansion container ranking according to the parameters in the similarity ranking and the capacity ranking includes:
S301, searching containers corresponding to the first capacity in the capacity sorting by taking the capacity sorting as a reference, searching containers corresponding to the second capacity in the capacity sorting by using the first capacity in the similarity sorting as a first similarity sorting, and stopping searching until all containers have obtained the similarity sorting by using the second similarity sorting;
S302, setting a similarity threshold, calculating the difference value between each similarity ranking and the similarity threshold, sorting the difference value from small to large, calculating the sum of the difference value corresponding to each container and the capacity ranking, obtaining a summation result, ranking the summation result from small to large, obtaining summation ranking, obtaining dilatation container ranking according to the summation ranking, wherein each ranking in the dilatation container ranking is the container ranking corresponding to each ranking in the summation ranking.
The calculation formula of the similarity threshold value is as follows:
Wherein S is a similarity threshold, R is a number corresponding to the capacity rank, for example, the capacity rank is the first name, the number is one, and X1 is a mean value calculated by similarity of all data contents.
The working principle of the technical scheme is as follows: searching containers corresponding to the first capacity in the capacity sorting, namely a first similar ranking in the similarity sorting, then searching containers corresponding to the second capacity in the capacity sorting, namely a second similar ranking in the similarity sorting, and stopping searching until all the containers have obtained the similar ranking; setting a similarity threshold, calculating the difference value between each similarity ranking and the similarity threshold, sorting the difference value from small to large, calculating the sum of the difference value and the capacity ranking corresponding to each container, obtaining a summation result, ranking the summation result from small to large, obtaining summation ranking, and obtaining dilatation container ranking according to the summation ranking, wherein each ranking in the dilatation container ranking is the container ranking corresponding to each ranking in the summation ranking. The capacity expansion ranks and the summation ranks are in one-to-one correspondence.
The technical effects of the technical scheme are as follows: first, the containers are sorted in order of capacity from large to small. Then, in the similarity sorting, the similarity of the data content with the capacity expansion request is calculated for each container, and the containers are sorted in the order of the similarity from the higher degree to the lower degree. The first capacity-corresponding container in the search capacity rank is the first container. The rank of the container in the similarity ranking, i.e., the first similarity rank, is then looked up. Next, the containers corresponding to the second capacity in the capacity rank are continually searched in order, then their ranks are looked up in the similarity rank, and so on, until all containers have obtained a similarity rank. A similarity threshold is set, each similarity rank is compared to the similarity threshold, and their difference is calculated. The containers are then ordered in order of decreasing difference. For each container, the sum of its difference and capacity rank is calculated and the results summed. And then ranking the summation results from small to large to obtain summation ranking. And determining the dilatation container rank of the container according to the summation rank. Each rank herein corresponds to a container rank to which each rank in the summed ranks corresponds. The priority order of the capacity-expanding containers can be determined, and the priority order considers the capacity and the similarity of the data content. This helps to select the container that is most suitable for expansion and improves the performance and efficiency of the system. The formula can determine the threshold value of the similarity according to the specific ranking position and the similarity condition of the data by using the corresponding quantity of the capacity ranks. By setting the similarity threshold, data with similarity higher than the threshold can be screened out, so that data with highest similarity can be selected for processing. This helps to improve the matching accuracy of the data, avoid processing data that are dissimilar or have a low degree of similarity, and reduce unnecessary calculation amount and resource consumption. Based on the similarity threshold, association and matching of data may be achieved. By calculating the similarity threshold value by using the quantity corresponding to the capacity rank, various technical effects such as determination of the similarity threshold value, screening, selection, association, quality control and the like of the data can be realized. These effects can improve the efficiency and accuracy of data processing and analysis, supporting the reliability of data management and decision making.
In one embodiment of the present invention, the storage virtualization facility mounts the persistent volumes into the expanded containers according to the ranking order of the expanded containers, performs persistent storage on the data, migrates the old data, and updates the capacity values of the containers through an abstraction layer, including:
S401, after the storage virtualization facility receives the initial persistence request, searching for an available persistence volume, binding the persistence volume with the initial persistence volume statement, calculating the ranking of the volume expansion containers of the persistence volume according to the storage data resources of the persistence volume, expanding the containers according to the ranking sequence of the volume expansion containers, mounting the persistence volume into the expanded containers according to the ranking sequence of the volume expansion containers through a file system, and performing persistence storage on the data resources in the persistence volume.
S402, after the capacity expansion is completed, transferring the old data from the old data container to the expanded container; after the abstract layer detects that the data migration is completed, the capacity values of the old container and the expanded container are modified and updated.
The working principle of the technical scheme is as follows: after the storage virtualization facility receives the initial persistence request, searching for an available persistence volume, binding the persistence volume with the initial persistence volume statement, calculating the dilatation container rank of the persistence volume according to the storage data resource of the persistence volume, dilatating the container according to the ranking sequence of the dilatation containers, mounting the persistence volume into the dilatation container according to the ranking sequence of the dilatation containers through a file system, and performing persistence storage on the data resource in the persistence volume. Binding the searched available persistence volume with the capacity expansion request, wherein the persistence volume is called an initial persistence volume, and expanding the capacity of the cloud hard disk and the file system of the cloud hard disk according to the capacity expansion program which is bound by the initial persistence volume statement. When the initial persistence volume is bound with the initial persistence statement, the initial persistence volume can be mounted on a formulated path of a container in the POD for persistence storage, data can be bound in the persistence volume in a persistence mode, and after the capacity expansion is completed, old data is migrated from the container of the old data to the container after the expansion; after the abstract layer detects that the data migration is completed, the capacity values of the old container and the expanded container are modified and updated. And identifying the container of the extended file system according to the cloud hard disk identification in the initial persistent volume statement. Searching the capacity of each node in the cluster, deleting the damaged container in each node, and detecting the data state of each node; and grading the node states, acquiring new capacity nodes according to the state grading, mounting data to be distributed to a mounting catalog, and detecting whether network connection is successful or not, and distributing the data.
The technical effects of the technical scheme are as follows: when the storage virtualization facility receives an initial persistence request, it searches for available persistence volumes that can be used to store the data in the request. After finding the available persistent volume, it is bound to the initial persistent volume declaration, ensuring that the persistent volume is properly assigned to the corresponding request. And calculating the ranking of the capacity expansion containers according to the storage data resources of the persistent volumes and the ordering method based on the capacity and the data similarity. And performing expansion operation on the containers according to the ranking order of the expansion containers. The containers are expanded in sequence according to the ranking order, so that the containers can meet the storage requirement in the expansion process. And mounting the persistent volume into the expanded container through a file system. In this way, the container can access and use the data resources in the persistent volume. And in the container after capacity expansion, carrying out persistent storage on the data resources in the persistent volume. This ensures that data is still reliably stored and accessed after migration or redistribution between containers. After the expansion and persistent storage of the container is completed, the data originally stored in the old container will migrate from the old container to the expanded container. Once the data migration is complete, the abstraction layer will detect this event and modify and update the capacity values of the old and expanded containers. The technology can bind and expand the persistent volume with a proper container according to the requirements and the priorities, and ensures the effective utilization of storage resources and the persistent storage of data. Meanwhile, the data migration and capacity updating can ensure the accuracy and consistency of the system.
In one embodiment of the invention, the system comprises:
The platform building module is used for building a storage virtualization platform, and building an abstract layer and a storage infrastructure in the storage virtualization platform;
the ordering module is used for carrying out content identification on the capacity expansion request and sending the capacity expansion request to the storage infrastructure after the abstract layer receives the capacity expansion request, the storage infrastructure obtains similarity ordering by calculating the similarity, and the capacity ordering is obtained by ordering the capacity of each container;
the capacity expansion sequencing module is used for obtaining the sequencing of the capacity expansion containers according to the parameters in the similarity sequencing and the capacity sequencing;
And the persistent storage module is used for mounting the persistent volumes into the expanded containers according to the ranking order of the expanded containers by the storage virtualization facility, performing persistent storage on the data, migrating the old data and updating the capacity values of the containers through the abstract layer.
The working principle of the technical scheme is as follows: the platform building module is used for building a storage virtualization platform, and building an abstract layer and a storage infrastructure in the storage virtualization platform; the ordering module is used for carrying out content identification on the capacity expansion request and sending the capacity expansion request to the storage infrastructure after the abstract layer receives the capacity expansion request, the storage infrastructure obtains similarity ordering by calculating the similarity, and the capacity ordering is obtained by ordering the capacity of each container; the capacity expansion ordering module is used for obtaining the ordering of the capacity expansion containers according to parameters in the similarity ordering and the capacity ordering; and the persistent storage module is used for storing the persistent volume which is mounted in the expanded container by the virtualization facility according to the ranking order of the expanded container, performing persistent storage on the data, migrating the old data and updating the capacity value of the container through the abstract layer.
The technical effects of the technical scheme are as follows: the storage virtualization platform is established through the platform establishment module, so that the underlying storage infrastructure can be abstracted, a unified interface and a management mechanism are provided, and management and use of storage resources are simplified. After the abstract layer of the storage virtualization platform receives the capacity expansion request, the request is analyzed and parsed through a content recognition technology, so that the user requirements can be accurately understood. The storage infrastructure calculates the similarity through the sequencing module, and can compare the capacity expansion request with the existing container to find the container with the highest similarity with the requested content. In addition, by sorting the capacities of the individual containers, the container with the most appropriate capacity can be determined. The capacity expansion ordering module can obtain the ordering of one capacity expansion container according to the results of the similarity ordering and the capacity ordering. I.e., determining in which order the containers should be expanded, it is the container that is most suitable for the persistence volume and persistence request that ensures that the first expansion is. And the storage virtualization facility mounts the persistent volume into the expanded container according to the ordering of the expanded container in the persistent storage module, so that the new container can access and use the storage resource. At the same time, old data is required to be migrated so as to ensure the integrity and consistency of the data. Through the abstraction layer, the storage virtualization platform can update the capacity value of the container to reflect the new capacity of the expanded container. By the automated mechanism of the storage virtualization platform, capacity expansion requests can be responded and processed quickly, and the workload of manual operation and the possibility of errors are reduced. By using the similarity sorting and the capacity sorting, a proper container can be effectively selected to meet the capacity expansion requirement, and the utilization efficiency of resources is improved. By mounting the persistent volume and data migration, the persistent storage and seamless migration of the data are ensured, and the reliability and usability of the data are ensured. The storage virtualization platform provides an abstraction layer, and can dynamically adjust and expand storage resources according to the needs to adapt to different business demands and scale changes.
In one embodiment of the present invention, the platform establishment module includes:
the virtualization platform module is used for establishing a storage virtualization platform in the virtualization cluster, wherein the storage virtualization platform comprises an abstract layer and a storage infrastructure;
And the interface communication module is used for connecting an interface of the abstract layer with an interface of the storage infrastructure, and the abstract layer and the storage infrastructure communicate through the interface.
The working principle of the technical scheme is as follows: the virtualization platform module is used for establishing a storage virtualization platform in the virtualization cluster, wherein the storage virtualization platform comprises an abstract layer and a storage infrastructure; the interface communication module is used for connecting an interface of the abstract layer with an interface of the storage infrastructure, and the abstract layer is communicated with the storage infrastructure through the interface.
The technical effects of the technical scheme are as follows: the interface of the abstract layers of the virtualized platform module and the interface communication module is connected with the interface of the storage infrastructure, and unified management and control are realized through the interfaces connecting the two components. This means that an administrator or user can manage the storage infrastructure through the interface of the abstraction layer without having to interact directly with the underlying storage device, simplifying the management operations. By building a storage virtualization platform, storage resources can be abstracted from physical devices to form a logical storage pool. In this way, the storage capacity can be conveniently expanded in the virtualized cluster, new storage resources are added to the storage pool, and management and allocation are performed through the interface of the abstraction layer. An administrator can flexibly configure and adjust storage capacity, performance, access rights and the like through an interface of an abstract layer so as to meet the requirements of different applications and services. Through the abstraction layer of the storage virtualization platform, an administrator may perform snapshot and backup operations on storage volumes of a virtual machine or container without concern for underlying storage device details. This simplifies the flow of backup and restore operations and improves the reliability and speed of data recovery. The abstract layer of the storage virtualization platform can realize a data redundancy and fault isolation mechanism, and the availability and safety of storage data are ensured. When the storage infrastructure fails, the abstract layer can be automatically switched to the standby storage equipment or recovered by using redundant data, so that the continuity of the service is ensured. The storage virtualization platform is established and connected with the abstract layer and the storage infrastructure, a unified management interface and an operation interface can be provided, the expandability, flexibility and reliability of storage resources are enhanced, management and operation processes are simplified, and the efficiency and usability of the storage system are improved.
In one embodiment of the present invention, the sorting module includes:
The content identification module is used for carrying out identification of request content on the capacity expansion request by the abstract layer after the capacity expansion request is received by the cloud hard disk data capacity expansion system, obtaining identification content and informing the identification content to the storage virtualization facility;
The ordering acquisition module is used for enabling the storage virtualization facility to check the residual capacity of the cloud hard disk in the cloud hard disk data expansion system at preset time, calibrating the residual capacity of the cloud hard disk at preset time, respectively marking all containers of the POD of the cloud hard disk, calculating the similarity of the data content in all containers of the POD of the cloud hard disk, ordering the similarity of the data content in all containers according to the order from large to small to obtain similarity ordering, and ordering the residual capacity quota in all containers according to the capacity from large to small to obtain capacity ordering.
The working principle of the technical scheme is as follows: the content identification module is used for carrying out identification of request content on the capacity expansion request by the abstract layer after the capacity expansion request is received by the cloud hard disk data capacity expansion system, so as to obtain identification content, and notifying the identification content to the storage virtualization facility; the ordering acquisition module is used for enabling the storage virtualization facility to check the residual capacity of the cloud hard disk in the cloud hard disk data expansion system at preset time, calibrating the residual capacity of the cloud hard disk at preset time, respectively marking all containers of the POD of the cloud hard disk, calculating the similarity of the data content in all containers of the POD of the cloud hard disk, ordering the similarity of the data content in all containers according to the order from large to small to obtain similarity ordering, and ordering the residual capacity quota in all containers according to the capacity from large to small to obtain capacity ordering.
The technical effects of the technical scheme are as follows: and identifying the capacity expansion request in the content identification module by an abstract layer to ensure that the content of the request is correctly understood. This may help determine the cloud hard disk or container that needs to be expanded and determine the corresponding process flow. And storing the residual capacity of the cloud hard disk checked by the virtualization facility within preset time in the sequencing acquisition module, and calibrating the residual capacity. This helps to understand the available capacity and serves as a basis for subsequent calculations and ordering. For each container of the cloud hard disk, a similarity size of the data content is calculated in the storage virtualization facility. This helps determine which containers of data content are relevant to the expansion request and provides a basis for subsequent ordering. And sequencing the residual quotas of the containers according to the similarity from large to small according to the similarity of the data content in the containers. The remaining capacity in the container will then be ordered from large to small. Thus, the container most suitable for capacity expansion can be found, and the reasonable utilization of the residual capacity is ensured. According to the technical scheme, the expansion request can be identified and processed, and the expansion requests are ordered according to the residual capacity of the cloud hard disk and the container, the similarity of data content and quota, so that the optimal expansion target is selected, and the expansion efficiency and accuracy are improved.
In one embodiment of the present invention, the capacity expansion ordering module includes:
The similarity ranking module is used for searching containers corresponding to the first capacity in the capacity ranking based on the capacity ranking, wherein the ranking in the similarity ranking is called a first similarity ranking, then searching containers corresponding to the second capacity in the capacity ranking, and stopping searching until all containers have obtained the similarity ranking;
the container ranking module is used for setting a similarity threshold, calculating the difference value between each similarity ranking and the similarity threshold, sequencing the difference value from small to large, calculating the sum of the difference value corresponding to each container and the capacity ranking, obtaining a summation result, ranking the summation result from small to large, obtaining summation ranking, obtaining dilatation container ranking according to the summation ranking, wherein each ranking in the dilatation container ranking is the container ranking corresponding to each ranking in the summation ranking.
The working principle of the technical scheme is as follows: the similarity ranking module is used for searching containers corresponding to the first capacity in the capacity ranking based on the capacity ranking, wherein the ranking in the similarity ranking is called a first similarity ranking, then searching containers corresponding to the second capacity in the capacity ranking, and the ranking in the similarity ranking is called a second similarity ranking, and stopping searching until all containers have obtained the similarity ranking; the container ranking module is used for setting a similarity threshold, calculating a difference value between each similarity ranking and the similarity threshold, sorting the difference value from small to large, calculating a sum of the difference value corresponding to each container and the capacity ranking, obtaining a summation result, ranking the summation result from small to large, obtaining summation ranking, obtaining dilatation container ranking according to the summation ranking, wherein each ranking in the dilatation container ranking is the container ranking corresponding to each ranking in the summation ranking.
The technical effects of the technical scheme are as follows: the similarity ranking module sorts the containers in order of capacity from large to small. Then, in the similarity sorting, the similarity of the data content with the capacity expansion request is calculated for each container, and the containers are sorted in the order of the similarity from the higher degree to the lower degree. The first capacity-corresponding container in the search capacity rank is the first container. The rank of the container in the similarity ranking, i.e., the first similarity rank, is then looked up. Next, the containers corresponding to the second capacity in the capacity rank are continually searched in order, then their ranks are looked up in the similarity rank, and so on, until all containers have obtained a similarity rank. The container ranking module sets a similarity threshold, compares each similarity ranking to the similarity threshold, and calculates their difference. The containers are then ordered in order of decreasing difference. For each container, the sum of its difference and capacity rank is calculated and the results summed. And then ranking the summation results from small to large to obtain summation ranking. And determining the dilatation container rank of the container according to the summation rank. Each rank herein corresponds to a container rank to which each rank in the summed ranks corresponds. The priority order of the capacity-expanding containers can be determined, and the priority order considers the capacity and the similarity of the data content. This helps to select the container that is most suitable for expansion and improves the performance and efficiency of the system. And the consideration of capacity and content similarity is realized.
In one embodiment of the invention, the persistent storage module includes:
And the capacity expansion module is used for searching available persistence volumes after the storage virtualization facility receives the initial persistence request, binding the persistence volumes with the initial persistence volume declarations, calculating the capacity expansion container rank of the persistence volumes according to the storage data resources of the persistence volumes, expanding the containers according to the capacity expansion container rank according to the ranking order, mounting the persistence volumes into the expanded containers through a file system according to the capacity expansion container rank order, and carrying out persistence storage on the data resources in the persistence volumes.
The migration updating module is used for migrating the old data from the old data container to the expanded container after the capacity expansion is completed; after the abstract layer detects that the data migration is completed, the capacity values of the old container and the expanded container are modified and updated.
The working principle of the technical scheme is as follows: and the capacity expansion module is used for searching available persistence volumes after the storage virtualization facility receives the initial persistence request, binding the persistence volumes with the initial persistence volume declaration, calculating the capacity expansion container rank of the persistence volumes according to the storage data resources of the persistence volumes, expanding the containers according to the ranking sequence of the capacity expansion containers, mounting the persistence volumes into the expanded containers according to the ranking sequence of the capacity expansion containers through a file system, and performing persistence storage on the data resources in the persistence volumes. The migration updating module is used for migrating old data from the old data container to the expanded container after the capacity expansion is completed; after the abstract layer detects that the data migration is completed, the capacity values of the old container and the expanded container are modified and updated.
The technical effects of the technical scheme are as follows: when the storage virtualization facility receives an initial persistence request, it searches for available persistence volumes that can be used to store the data in the request. After finding the available persistent volume, it is bound to the initial persistent volume declaration, ensuring that the persistent volume is properly assigned to the corresponding request. And calculating the ranking of the capacity expansion containers according to the storage data resources of the persistent volumes and the ordering method based on the capacity and the data similarity. And performing expansion operation on the containers according to the ranking order of the expansion containers. The containers are expanded in sequence according to the ranking order, so that the containers can meet the storage requirement in the expansion process. And mounting the persistent volume into the expanded container through a file system. In this way, the container can access and use the data resources in the persistent volume. And in the container after capacity expansion, carrying out persistent storage on the data resources in the persistent volume. This ensures that data is still reliably stored and accessed after migration or redistribution between containers. After the expansion and persistent storage of the container is completed, the data originally stored in the old container will migrate from the old container to the expanded container. Once the data migration is complete, the abstraction layer will detect this event and modify and update the capacity values of the old and expanded containers. The technology can bind and expand the persistent volume with a proper container according to the requirements and the priorities, and ensures the effective utilization of storage resources and the persistent storage of data. Meanwhile, the data migration and capacity updating can ensure the accuracy and consistency of the system.
It will be apparent to those skilled in the art that various modifications and variations can be made to the present invention without departing from the spirit or scope of the invention. Thus, it is intended that the present invention also include such modifications and alterations insofar as they come within the scope of the appended claims or the equivalents thereof.

Claims (2)

1. The cloud hard disk data expansion method based on storage virtualization is characterized by comprising the following steps of:
S1, establishing a storage virtualization platform, and establishing an abstract layer and a storage infrastructure in the storage virtualization platform;
wherein the establishing a storage virtualization platform, establishing an abstraction layer and a storage infrastructure in the storage virtualization platform, comprises:
S101, establishing a storage virtualization platform in a virtualization cluster, wherein the storage virtualization platform comprises an abstract layer and a storage infrastructure;
s102, an interface of the abstract layer is connected with an interface of the storage infrastructure, and the abstract layer is communicated with the storage infrastructure through the interface;
S2, after receiving the capacity expansion request, the abstract layer performs content identification on the capacity expansion request and sends the capacity expansion request to a storage infrastructure, the storage infrastructure obtains similarity sorting through calculating the similarity, and the capacity sorting is obtained through sorting the capacity of each container;
After receiving the capacity expansion request, the abstraction layer performs content identification on the capacity expansion request and sends the capacity expansion request to a storage infrastructure, the storage infrastructure obtains similarity sorting through calculating the similarity, and obtains capacity sorting through sorting the capacity of each container, including:
s201, after a cloud hard disk data capacity expansion system receives a capacity expansion request, the abstract layer identifies request content of the capacity expansion request, obtains identification content and notifies the identification content to the storage virtualization facility;
S202, checking the residual capacity of a cloud hard disk in a cloud hard disk data expansion system at preset time by the storage virtualization facility, calibrating the residual capacity of the cloud hard disk at preset time, respectively marking all containers of the POD of the cloud hard disk, calculating the similarity of data content in all containers of the POD of the cloud hard disk, sorting the similarity of the data content of all containers according to the sequence from large to small to obtain a similarity sorting, and sorting the residual capacity quota in all containers according to the capacity from large to small to obtain a capacity sorting;
s3, according to parameters in the similarity sorting and the capacity sorting, obtaining the sorting of the capacity-expanded containers;
wherein, the obtaining the dilatation container ranking according to the parameters in the similarity ranking and the capacity ranking includes:
S301, searching containers corresponding to the first capacity in the capacity sorting by taking the capacity sorting as a reference, searching containers corresponding to the second capacity in the capacity sorting by using the first capacity in the similarity sorting as a first similarity sorting, and stopping searching until all containers have obtained the similarity sorting by using the second similarity sorting;
S302, setting a similarity threshold, calculating a difference value between each similarity ranking and the similarity threshold, sorting the difference value from small to large, calculating a sum of the difference value corresponding to each container and the capacity ranking, obtaining a summation result, ranking the summation result from small to large, obtaining summation ranking, obtaining dilatation container ranking according to the summation ranking, wherein each ranking in the dilatation container ranking is the container ranking corresponding to each ranking in the summation ranking;
S4, the storage virtualization facility mounts the persistent volume into the container after capacity expansion according to the ranking order of the capacity expansion containers, performs persistent storage on the data, migrates the old data and updates the capacity value of the container through an abstract layer;
The storage virtualization facility mounts the persistent volume into the container after capacity expansion according to the ranking order of the capacity expansion containers, performs persistent storage on the data, migrates old data and updates the capacity value of the container through an abstract layer, and comprises the following steps:
s401, after a storage virtualization facility receives an initial persistence request, searching for an available persistence volume, binding the persistence volume with an initial persistence volume statement, calculating the ranking of the dilation containers of the persistence volume according to storage data resources of the persistence volume, carrying out dilation on the containers according to the ranking sequence of the dilation containers, mounting the persistence volume into the dilated containers according to the ranking sequence of the dilation containers through a file system, and carrying out persistence storage on the data resources in the persistence volume;
s402, after the capacity expansion is completed, transferring the old data from the old data container to the expanded container; after the abstract layer detects that the data migration is completed, the capacity values of the old container and the expanded container are modified and updated.
2.A cloud hard disk data capacity expansion system based on storage virtualization, the system comprising:
The platform building module is used for building a storage virtualization platform, and building an abstract layer and a storage infrastructure in the storage virtualization platform;
Wherein, the platform establishment module includes:
the virtualization platform module is used for establishing a storage virtualization platform in the virtualization cluster, wherein the storage virtualization platform comprises an abstract layer and a storage infrastructure;
the interface communication module is used for connecting an interface of the abstract layer with an interface of the storage infrastructure, and the abstract layer is communicated with the storage infrastructure through the interface;
the ordering module is used for carrying out content identification on the capacity expansion request and sending the capacity expansion request to the storage infrastructure after the abstract layer receives the capacity expansion request, the storage infrastructure obtains similarity ordering by calculating the similarity, and the capacity ordering is obtained by ordering the capacity of each container;
Wherein, the sequencing module includes:
The content identification module is used for carrying out identification of request content on the capacity expansion request by the abstract layer after the capacity expansion request is received by the cloud hard disk data capacity expansion system, obtaining identification content and informing the identification content to the storage virtualization facility;
The ordering acquisition module is used for enabling the storage virtualization facility to check the residual capacity of the cloud hard disk in the cloud hard disk data expansion system at preset time, calibrating the residual capacity of the cloud hard disk at preset time, respectively marking all containers of the POD of the cloud hard disk, calculating the similarity of the data content in all containers of the POD of the cloud hard disk, ordering the similarity of the data content in all containers according to the order from large to small to obtain similarity ordering, and ordering the residual capacity quota in all containers according to the capacity from large to small to obtain capacity ordering;
the capacity expansion sequencing module is used for obtaining the sequencing of the capacity expansion containers according to the parameters in the similarity sequencing and the capacity sequencing;
Wherein, the dilatation sequencing module includes:
The similarity ranking module is used for searching containers corresponding to the first capacity in the capacity ranking based on the capacity ranking, wherein the ranking in the similarity ranking is called a first similarity ranking, then searching containers corresponding to the second capacity in the capacity ranking, and stopping searching until all containers have obtained the similarity ranking;
The container ranking module is used for setting a similarity threshold, calculating a difference value between each similarity ranking and the similarity threshold, sequencing the difference value from small to large, calculating a sum of the difference value corresponding to each container and the capacity ranking, obtaining a summation result, ranking the summation result from small to large, obtaining summation ranking, and obtaining dilatation container ranking according to the summation ranking, wherein each ranking in the dilatation container ranking is the container ranking corresponding to each ranking in the summation ranking;
The persistent storage module is used for storing and virtualizing the persistent volume to be mounted in the container after the expansion according to the ranking order of the expansion containers, performing persistent storage on the data, migrating the old data and updating the capacity value of the container through the abstract layer;
wherein the persistent storage module comprises:
The capacity expansion module is used for searching available persistence volumes after the storage virtualization facility receives an initial persistence request, binding the persistence volumes with the initial persistence volume declarations, calculating the capacity expansion container ranks of the persistence volumes according to storage data resources of the persistence volumes, expanding the containers according to the ranking sequence of the capacity expansion containers, mounting the persistence volumes into the expanded containers according to the ranking sequence of the capacity expansion containers through a file system, and performing persistence storage on the data resources in the persistence volumes;
The migration updating module is used for migrating the old data from the old data container to the expanded container after the capacity expansion is completed; after the abstract layer detects that the data migration is completed, the capacity values of the old container and the expanded container are modified and updated.
CN202310993842.0A 2023-08-08 2023-08-08 Cloud hard disk data capacity expansion method and system based on storage virtualization Active CN116991324B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202310993842.0A CN116991324B (en) 2023-08-08 2023-08-08 Cloud hard disk data capacity expansion method and system based on storage virtualization

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202310993842.0A CN116991324B (en) 2023-08-08 2023-08-08 Cloud hard disk data capacity expansion method and system based on storage virtualization

Publications (2)

Publication Number Publication Date
CN116991324A CN116991324A (en) 2023-11-03
CN116991324B true CN116991324B (en) 2024-05-14

Family

ID=88533650

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202310993842.0A Active CN116991324B (en) 2023-08-08 2023-08-08 Cloud hard disk data capacity expansion method and system based on storage virtualization

Country Status (1)

Country Link
CN (1) CN116991324B (en)

Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110209342A (en) * 2018-03-26 2019-09-06 腾讯科技（深圳）有限公司 Expansion method, device, equipment and the system of cloud hard disk
CN111966305A (en) * 2020-10-22 2020-11-20 腾讯科技（深圳）有限公司 Persistent volume allocation method and device, computer equipment and storage medium
CN112328170A (en) * 2020-10-22 2021-02-05 杭州朗澈科技有限公司 Cloud hard disk capacity expansion method and device, computer equipment and storage medium

Family Cites Families (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US11467775B2 (en) * 2019-10-15 2022-10-11 Hewlett Packard Enterprise Development Lp Virtual persistent volumes for containerized applications
US20210240369A1 (en) * 2020-01-31 2021-08-05 Hewlett Packard Enterprise Development Lp Virtual storage policies for virtual persistent volumes

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110209342A (en) * 2018-03-26 2019-09-06 腾讯科技（深圳）有限公司 Expansion method, device, equipment and the system of cloud hard disk
CN111966305A (en) * 2020-10-22 2020-11-20 腾讯科技（深圳）有限公司 Persistent volume allocation method and device, computer equipment and storage medium
CN112328170A (en) * 2020-10-22 2021-02-05 杭州朗澈科技有限公司 Cloud hard disk capacity expansion method and device, computer equipment and storage medium

Also Published As

Publication number Publication date
CN116991324A (en) 2023-11-03

Similar Documents

Publication Publication Date Title
US11080081B2 (en) Virtual machine and volume allocation in hyperconverged infrastructure environment and storage system
US11620187B2 (en) Distributing data on distributed storage systems
US8359440B2 (en) Management server device for managing virtual storage device, and method for managing virtual storage device
US11403269B2 (en) Versioning validation for data transfer between heterogeneous data stores
US8291159B2 (en) Monitoring and updating mapping of physical storage allocation of virtual machine without changing identifier of the storage volume assigned to virtual machine
US10831387B1 (en) Snapshot reservations in a distributed storage system
EP2810164B1 (en) Managing partitions in a scalable environment
CN101601014B (en) Methods and systems for load balancing of virtual machines in clustered processors using storage related load information
CN105429776B (en) Method and system for managing functions of virtual network
CN107590033B (en) Method, device and system for creating DOCKER container
AU2016229135A1 (en) Opportunistic resource migration to optimize resource placement
US8001324B2 (en) Information processing apparatus and informaiton processing method
US20150128150A1 (en) Data processing method and information processing apparatus
US10817380B2 (en) Implementing affinity and anti-affinity constraints in a bundled application
US20170371910A1 (en) Real-time shard rebalancing for versioned entity repository
CN112083889A (en) Data migration method, device, equipment and readable storage medium
US11199972B2 (en) Information processing system and volume allocation method
CN101617297B (en) Virtualization support in a multiprocessor storage area network
CN113300953A (en) Management method, system and related device for multipath failover group
US20240152286A1 (en) Fast restart of large memory systems
US20210132812A1 (en) Parallel upgrade of nodes in a storage system
CN105635285B (en) A kind of VM migration scheduling method based on state aware
CN116991324B (en) Cloud hard disk data capacity expansion method and system based on storage virtualization
US20190020716A1 (en) Method and system for recovering data in distributed computing system
CN114253936A (en) Capacity reduction method, device, equipment and medium for distributed database

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant