CN114647385A - Volume mapping management method, system and related device for distributed storage - Google Patents

Volume mapping management method, system and related device for distributed storage Download PDF

Info

Publication number
CN114647385A
CN114647385A CN202210331086.0A CN202210331086A CN114647385A CN 114647385 A CN114647385 A CN 114647385A CN 202210331086 A CN202210331086 A CN 202210331086A CN 114647385 A CN114647385 A CN 114647385A
Authority
CN
China
Prior art keywords
volume mapping
target
volume
grouping
node
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202210331086.0A
Other languages
Chinese (zh)
Inventor
许江永
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Suzhou Inspur Intelligent Technology Co Ltd
Original Assignee
Suzhou Inspur Intelligent Technology Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Suzhou Inspur Intelligent Technology Co Ltd filed Critical Suzhou Inspur Intelligent Technology Co Ltd
Priority to CN202210331086.0A priority Critical patent/CN114647385A/en
Publication of CN114647385A publication Critical patent/CN114647385A/en
Pending legal-status Critical Current

Links

Images

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
    • G06F3/0601Interfaces specially adapted for storage systems
    • G06F3/0602Interfaces specially adapted for storage systems specifically adapted to achieve a particular effect
    • G06F3/061Improving I/O performance
    • G06F3/0611Improving I/O performance in relation to response time
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
    • G06F3/0601Interfaces specially adapted for storage systems
    • G06F3/0628Interfaces specially adapted for storage systems making use of a particular technique
    • G06F3/0638Organizing or formatting or addressing of data
    • G06F3/0644Management of space entities, e.g. partitions, extents, pools
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
    • G06F3/0601Interfaces specially adapted for storage systems
    • G06F3/0668Interfaces specially adapted for storage systems adopting a particular infrastructure
    • G06F3/067Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS]
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04LTRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L67/00Network arrangements or protocols for supporting network services or applications
    • H04L67/01Protocols
    • H04L67/10Protocols in which an application is distributed across nodes in the network
    • H04L67/1097Protocols in which an application is distributed across nodes in the network for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS]

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Human Computer Interaction (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Computer Networks & Wireless Communication (AREA)
  • Signal Processing (AREA)
  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)

Abstract

The application provides a volume mapping management method for distributed storage, which comprises the following steps: receiving a volume mapping request; determining a target group corresponding to the volume mapping request; and selecting a corresponding node in the target grouping according to the set path number, and executing a mapping command corresponding to the volume mapping request. The method and the device adopt a grouping mode to manage the volume mapping, when a volume mapping request is received and volume mapping management is needed, the nodes are selected only in the selected grouping according to the set path number, and a mapping command is executed, so that the response time can be greatly shortened, and the volume mapping management efficiency is improved. The application also provides a volume mapping management system of distributed storage, a computer readable storage medium and a storage device, which have the beneficial effects.

Description

Volume mapping management method, system and related device for distributed storage
Technical Field
The present application relates to the field of distributed storage, and in particular, to a volume mapping management method and system for distributed storage, and a related device.
Background
With the popularization of large-scale clusters, hundreds of thousands of nodes in the cluster often make higher requirements on the performance and response time of the storage management system. The existing volume mapping management scheme is suitable for small-scale clusters, and the management of target, the management of iqn and the management of volume mapping are independently maintained, and the interfaces provided for third parties such as OpenStack are also independent, so that if a third party wants to mount a volume, multiple interfaces of a storage system need to be called, the response time is too long, and the use of clients is seriously influenced; on the other hand, in the volume mapping management of the storage system itself, since the command is executed in a full path, the command needs to be executed in each node in the cluster at the same time, and as the number of nodes increases and the concurrency increases, the command is often jammed.
Therefore, how to implement efficient volume mapping and management is a technical problem that needs to be solved urgently by those skilled in the art.
Disclosure of Invention
The application aims to provide a volume mapping management method and system for distributed storage. The computer-readable storage medium and the storage device can reduce volume mapping response time and improve volume mapping processing efficiency.
In order to solve the above technical problem, the present application provides a volume mapping management method for distributed storage, which has the following specific technical solutions:
receiving a volume mapping request;
determining a target group corresponding to the volume mapping request;
and selecting a corresponding node in the target grouping according to the set path number, and executing a mapping command corresponding to the volume mapping request.
Optionally, before determining the target group corresponding to the volume mapping request, the method further includes:
judging whether the number of nodes in the storage pool is greater than a preset value or not;
if not, the grouping is set as a group;
if yes, the node number and the path number under the storage pool are used as parameters, and a circular recursion algorithm is adopted to generate grouping information.
Optionally, before selecting a corresponding node according to the set number of paths in the target packet, the method further includes:
generating json configuration files by using the storage pool information, the node information and the path information in a key value pair mode;
then said selecting a corresponding node in said target packet according to the set number of paths comprises:
and calling the json configuration file in the target grouping according to the set path number to select a corresponding node.
Optionally, selecting a corresponding node in the target packet according to the set number of paths includes:
converting the three digits after the roll name from hexadecimal to decimal;
selecting a target to be bound by taking the balance of the target number;
and determining the grouping to be selected after the remainder is taken out of the grouping number to obtain the target node.
Optionally, after obtaining the target node, the method further includes:
and executing a volume mapping command by using the target and the target node, searching an IP (Internet protocol) through the node, and returning path information.
Optionally, the method further includes:
when a volume mounting request of the cloud computing management platform is received, a single volume mounting interface and corresponding path information are provided.
The application provides a volume mapping management system of distributed storage, including:
the request receiving module is used for receiving volume mapping requests;
the grouping module is used for determining a target grouping corresponding to the volume mapping request;
and the volume mapping module is used for selecting corresponding nodes in the target grouping according to the set path number and executing the mapping command corresponding to the volume mapping request.
Optionally, the method further includes:
the cluster configuration module is used for judging whether the number of nodes in the storage pool is greater than a preset value or not; if not, the grouping is set as a group; if yes, the node number and the path number under the storage pool are used as parameters, and a circular recursion algorithm is adopted to generate grouping information.
The present application also provides a computer-readable storage medium having stored thereon a computer program which, when being executed by a processor, carries out the steps of the method as set forth above.
The present application also provides a storage device comprising a memory and a processor, wherein the memory stores a computer program, and the processor implements the steps of the method described above when calling the computer program in the memory.
The application provides a volume mapping management method for distributed storage, which comprises the following steps: receiving a volume mapping request; determining a target group corresponding to the volume mapping request; and selecting a corresponding node in the target grouping according to the set path number, and executing a mapping command corresponding to the volume mapping request.
The method and the device adopt a grouping mode to manage the volume mapping, when a volume mapping request is received and volume mapping management is needed, the nodes are selected only in the selected grouping according to the set path number, and a mapping command is executed, so that the response time can be greatly shortened, and the volume mapping management efficiency is improved.
The present application further provides a volume mapping management system of distributed storage, a computer-readable storage medium, and a storage device, which have the above beneficial effects and are not described herein again.
Drawings
In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings needed to be used in the description of the embodiments or the prior art will be briefly introduced below, it is obvious that the drawings in the following description are only embodiments of the present application, and for those skilled in the art, other drawings can be obtained according to the provided drawings without creative efforts.
Fig. 1 is a flowchart of a volume mapping management method for distributed storage according to an embodiment of the present application;
fig. 2 is a flowchart of a volume mapping management method for distributed storage according to an embodiment of the present application.
Detailed Description
In order to make the objects, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application, and it is obvious that the described embodiments are some embodiments of the present application, but not all embodiments. All other embodiments obtained by a person of ordinary skill in the art based on the embodiments in the present application without making any creative effort belong to the protection scope of the present application.
Referring to fig. 1, fig. 1 is a flowchart of a volume mapping management method for distributed storage according to an embodiment of the present application, where the method includes:
s101: receiving a volume mapping request;
s102: determining a target group corresponding to the volume mapping request;
s103: and selecting a corresponding node in the target grouping according to the set path number, and executing a mapping command corresponding to the volume mapping request.
The following description is first made on a distributed storage structure provided in the present application, where the present application needs to perform grouping configuration management on nodes in a cluster in advance. Specifically, the grouping may be performed according to the following steps:
judging whether the number of nodes in the storage pool is greater than a preset value or not; if not, the grouping is set as a group; and if the number of the nodes is larger than the preset value, generating grouping information by adopting a circular recursion algorithm by taking the number of the nodes and the number of the paths under the storage pool as parameters. The preset value is not limited and can be set by a person skilled in the art. It is easy to understand that the preset value plays a role in determining the minimum node number no matter what value is.
In addition, a json profile may be generated from the storage pool information, the node information, and the path information in the form of a key-value pair, and then, when step S103 is executed, the json profile may be directly called in the target group according to the set number of paths to select a corresponding node, in other words, the json profile is used in the path selection algorithm.
After the storage management system deployment is completed through the WEB interface, a set of targets for the cluster can be automatically created, for example, named as target0-target 7. The storage pool information, including node information of the pool and path information, is automatically generated into a json configuration file in the form of key, value or key-value pair by grouping the storage pool information, for example, a group. The execution efficiency of volume mapping is improved through the use of two application scenarios, namely virtualization and non-virtualization.
In the distributed storage deployment, the deployment is divided into two scenes, namely fusion deployment and non-fusion deployment, an MON script is called through network setting, public IP and cluster IP can be simultaneously configured in a configuration file of a cluster, in the fusion scene, the two networks are the same, in order to adapt to the fusion scene, front _ network is configured in a management software configuration file (ism.conf) file, and when service IP is analyzed through subsequent return path information, scene judgment is also carried out according to the field. After the cluster is deployed, the system collects node network information in the cluster, generates node _ ip.txt including node names and ip information, and automatically generates a group of targets for the cluster to use, which may be named as target0-target7, for example. In brief, in a fusion deployment mode, obtaining front _ network in ismconf; under the non-fusion mode, public ip in icfs. conf is obtained, and the application can be compatible with two modes.
After the cluster deployment is completed, the storage pool needs to be created, for a large-scale cluster, the storage pool needs to be created in a grouping and dividing manner, and a group.
When the number of paths is 5 for example, when the number of nodes in the storage pool is less than or equal to 5, the generated packets are a group, when the number of nodes is greater than 5, in order to ensure the load balance of the nodes, a circular recursive algorithm is adopted to generate packet information, parameters are the number of nodes in the storage pool and the number of paths, and the number of paths is fixed to 5. And c, taking b as a and b, and taking the remainder of a and b as b to be called circularly until the remainder of a and b is 0, and then returning to a. The number of packets is the number of nodes/round robin recursion result. Such as: if the number of paths of 8 nodes in the storage pool is 5, the nodes need to be divided into 8 groups, and one possible grouping information is as follows:
[[0,1,2,3,4],[1,2,3,4,5],[2,3,4,5,6],[3,4,5,6,7],[4,5,6,7,0],[5,6,7,0,1],[6,7,0,1,2],[7,0,1,2,3]]
and storing in a two-dimensional array form. When the volume is created, the last three bits of the volume name are intercepted, and if the volume name is not a number, a random number is automatically generated to be used as the volume name. Then, the last three-digit hexadecimal system of the volume name is converted into a decimal system, and the target needing to be bound is selected by taking the balance of the converted decimal system and the number of the target; and obtaining an index value of the required group information by taking the rest of the group number, for example, image110, the last three bits are converted into decimal 272, and 272% 8 is 0, then target0 may be selected, the group information with the index value of 0 is selected as the first group information, each group in the group information is the index value of a node, and finally the node with the index value of 0-4 is selected as the node for creating the volume mapping. After the node information and the target information are selected, ansable executes a volume mapping command, and then the service ip is analyzed in combination with the node _ ip. Whether a front _ network field exists in the ssm.conf file or not is judged to distinguish fusion scenes from non-fusion scenes, the service IP is front _ network in the fusion scene, the service IP is public IP in the non-fusion scene, and the public IP can be obtained from icfs.conf configuration files. Finally, displaying volume mapping information by the non-virtualization scene, wherein the volume mapping information comprises the pool, the volume, the path number and detailed path information; under the virtual scene, an interface is called through OpenStack mounting volume, path information is returned, interaction time is greatly shortened, and efficiency is improved.
In addition, when a volume mount request of the cloud computing management platform is received, a single volume mount interface and corresponding path information are provided. For other third parties, such as OpenStack, only a single volume mounting interface can be provided, and through combination of the interfaces, when OpenStack is docked, interaction times are reduced, and response efficiency is greatly improved.
The volume mapping management method and the volume mapping management device adopt a grouping mode to manage volume mapping, when a volume mapping request is received and volume mapping management is needed, nodes are selected only in the selected grouping according to the set path number, and a mapping command is executed, so that the response time can be greatly shortened, and the volume mapping management efficiency is improved.
In the following, a volume mapping management system of distributed storage provided in an embodiment of the present application is introduced, and a volume mapping management system described below and a volume mapping management method of distributed storage described above may be referred to correspondingly.
Referring to fig. 2, fig. 2 is a flowchart of a volume mapping management method for distributed storage according to an embodiment of the present application, where the present application provides a volume mapping management system for distributed storage, including:
the request receiving module is used for receiving volume mapping requests;
the grouping module is used for determining a target grouping corresponding to the volume mapping request;
and the volume mapping module is used for selecting corresponding nodes in the target grouping according to the set path number and executing the mapping command corresponding to the volume mapping request.
Based on the above embodiment, as a preferred embodiment, the method further includes:
the cluster configuration module is used for judging whether the number of nodes in the storage pool is greater than a preset value or not; if not, the grouping is set as a group; if yes, the node number and the path number under the storage pool are used as parameters, and a circular recursion algorithm is adopted to generate grouping information.
Based on the above embodiment, as a preferred embodiment, the method further includes:
the configuration file generation module is used for generating a json configuration file from the storage pool information, the node information and the path information in a key value pair mode;
correspondingly, the volume mapping module is a module for calling the json configuration file to select a corresponding node in the target grouping according to the set path number.
Based on the above embodiment, as a preferred embodiment, the volume mapping module includes:
the node determining unit is used for converting the three digits after the volume name from hexadecimal to decimal; selecting a target to be bound by taking the balance of the target number; and determining the grouping to be selected after the remainder is taken out of the grouping number to obtain the target node.
Based on the above embodiment, as a preferred embodiment, the method further includes:
and the command response module is used for executing the volume mapping command by using the target and the target node, searching the IP through the node and returning the path information.
The present application also provides a computer readable storage medium having stored thereon a computer program which, when executed, may implement the steps provided by the above-described embodiments. The storage medium may include: various media capable of storing program codes, such as a usb disk, a removable hard disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a magnetic disk, or an optical disk.
The present application further provides a storage device, which may include a memory and a processor, where the memory stores a computer program, and the processor may implement the steps provided in the foregoing embodiments when calling the computer program in the memory. Of course, the storage device may also include various network interfaces, power supplies, and the like.
The embodiments are described in a progressive manner in the specification, each embodiment focuses on differences from other embodiments, and the same and similar parts among the embodiments are referred to each other. For the system provided by the embodiment, the description is relatively simple because the system corresponds to the method provided by the embodiment, and the relevant points can be referred to the method part for description.
The principles and embodiments of the present application are described herein using specific examples, which are only used to help understand the method and its core idea of the present application. It should be noted that, for those skilled in the art, it is possible to make several improvements and modifications to the present application without departing from the principle of the present application, and such improvements and modifications also fall within the scope of the claims of the present application.
It is further noted that, in the present specification, relational terms such as first and second, and the like are used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Also, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising an … …" does not exclude the presence of other identical elements in a process, method, article, or apparatus that comprises the element.

Claims (10)

1. A volume mapping management method for distributed storage is characterized by comprising the following steps:
receiving a volume mapping request;
determining a target group corresponding to the volume mapping request;
and selecting a corresponding node in the target grouping according to the set path number, and executing a mapping command corresponding to the volume mapping request.
2. The volume mapping management method according to claim 1, wherein before determining the target group corresponding to the volume mapping request, the method further comprises:
judging whether the number of nodes in the storage pool is greater than a preset value or not;
if not, the grouping is set as a group;
if yes, the node number and the path number under the storage pool are used as parameters, and a circular recursion algorithm is adopted to generate grouping information.
3. The volume mapping management method according to claim 1, further comprising, before selecting a corresponding node in the target group according to the set number of paths:
generating json configuration files by using the storage pool information, the node information and the path information in a key value pair mode;
then said selecting a corresponding node in said target packet according to the set number of paths includes:
and calling the json configuration file in the target grouping according to the set path number to select a corresponding node.
4. The volume mapping management method according to claim 1, wherein selecting a corresponding node in the target group according to the set number of paths includes:
converting the three digits after the volume name from hexadecimal to decimal;
selecting a target to be bound by taking the balance of the target number;
and determining the grouping to be selected after the remainder is taken out of the grouping number to obtain the target node.
5. The volume mapping management method according to claim 4, further comprising, after obtaining the target node:
and executing a volume mapping command by using the target and the target node, searching an IP (Internet protocol) through the node, and returning path information.
6. The volume mapping management method according to claim 1, further comprising:
when a volume mounting request of the cloud computing management platform is received, a single volume mounting interface and corresponding path information are provided.
7. A volume mapping management system for distributed storage, comprising:
the request receiving module is used for receiving volume mapping requests;
the grouping module is used for determining a target grouping corresponding to the volume mapping request;
and the volume mapping module is used for selecting corresponding nodes in the target grouping according to the set path number and executing the mapping command corresponding to the volume mapping request.
8. The volume mapping management system according to claim 7, further comprising:
the cluster configuration module is used for judging whether the number of nodes in the storage pool is greater than a preset value or not; if not, the grouping is set as a group; if yes, the node number and the path number under the storage pool are used as parameters, and a circular recursion algorithm is adopted to generate grouping information.
9. A computer-readable storage medium, on which a computer program is stored, which, when being executed by a processor, carries out the steps of the method for volume mapping management of distributed storage according to any of claims 1 to 6.
10. A storage device comprising a memory and a processor, wherein the memory stores a computer program, and the processor implements the steps of the volume mapping management method for distributed storage according to any one of claims 1 to 6 when calling the computer program in the memory.
CN202210331086.0A 2022-03-31 2022-03-31 Volume mapping management method, system and related device for distributed storage Pending CN114647385A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202210331086.0A CN114647385A (en) 2022-03-31 2022-03-31 Volume mapping management method, system and related device for distributed storage

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202210331086.0A CN114647385A (en) 2022-03-31 2022-03-31 Volume mapping management method, system and related device for distributed storage

Publications (1)

Publication Number Publication Date
CN114647385A true CN114647385A (en) 2022-06-21

Family

ID=81994781

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202210331086.0A Pending CN114647385A (en) 2022-03-31 2022-03-31 Volume mapping management method, system and related device for distributed storage

Country Status (1)

Country Link
CN (1) CN114647385A (en)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN115314257A (en) * 2022-07-12 2022-11-08 天翼云科技有限公司 Authentication method and device of file system, electronic equipment and computer storage medium

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN115314257A (en) * 2022-07-12 2022-11-08 天翼云科技有限公司 Authentication method and device of file system, electronic equipment and computer storage medium
CN115314257B (en) * 2022-07-12 2024-03-12 天翼云科技有限公司 File system authentication method and device, electronic equipment and computer storage medium

Similar Documents

Publication Publication Date Title
WO2021052132A1 (en) Network edge computing method and device, apparatus, and storage medium
CN107872402B (en) Global flow scheduling method and device and electronic equipment
CN111478820A (en) Network equipment configuration system and method for large-scale network environment of network target range
CN110795029B (en) Cloud hard disk management method, device, server and medium
CN103685583A (en) Method and system for resolving domain names
WO2014194869A1 (en) Request processing method, device and system
CN113949707A (en) OpenResty and K8S-based container cloud service discovery and load balancing method
CN112256495A (en) Data transmission method and device, computer equipment and storage medium
CN113645304B (en) Data service processing method and related equipment
CN113032419B (en) Multi-source data aggregation search method, device, equipment and storage medium
CN112822298A (en) Business service capacity expansion and reduction method, device, medium and electronic equipment
CN111885216B (en) DNS query method, device, equipment and storage medium
WO2008060050A1 (en) Apparatus and method for parsing domain profile in software communication architecture
CN110519348A (en) A kind of mostly service distributed type assemblies deployment system and method
JP2021149409A (en) Application development support system and application development support method
CN114647385A (en) Volume mapping management method, system and related device for distributed storage
CN112311907A (en) ARP request response method, load balancing equipment and related device
CN114911602A (en) Load balancing method, device, equipment and storage medium for server cluster
WO2016000303A1 (en) Resource allocation method and system, and computer storage medium
CN113301173A (en) Domain name updating system and method, message forwarding method and server
CN112035413B (en) Metadata information query method, device and storage medium
CN106487916B (en) Statistical method and device for connection number
US9960957B2 (en) Methods for prioritizing failover of logical interfaces (LIFs) during a node outage and devices thereof
CN113873052B (en) Domain name resolution method, device and equipment of Kubernetes cluster
WO2022174675A1 (en) Computing power information processing method, first network device, and system

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination