CN114138181A - Method, device, equipment and readable medium for placing, grouping and selecting owners in binding pool - Google Patents
Method, device, equipment and readable medium for placing, grouping and selecting owners in binding pool Download PDFInfo
- Publication number
- CN114138181A CN114138181A CN202111237021.1A CN202111237021A CN114138181A CN 114138181 A CN114138181 A CN 114138181A CN 202111237021 A CN202111237021 A CN 202111237021A CN 114138181 A CN114138181 A CN 114138181A
- Authority
- CN
- China
- Prior art keywords
- data structure
- pool
- values
- members
- node
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 43
- 230000004044 response Effects 0.000 claims abstract description 40
- 238000007792 addition Methods 0.000 claims description 16
- 238000004590 computer program Methods 0.000 claims description 11
- 238000012217 deletion Methods 0.000 claims description 4
- 230000037430 deletion Effects 0.000 claims description 4
- 238000010586 diagram Methods 0.000 description 7
- 230000006870 function Effects 0.000 description 7
- 230000003287 optical effect Effects 0.000 description 4
- 230000004048 modification Effects 0.000 description 3
- 238000012986 modification Methods 0.000 description 3
- 230000008569 process Effects 0.000 description 3
- 238000012545 processing Methods 0.000 description 3
- 230000002776 aggregation Effects 0.000 description 2
- 238000004220 aggregation Methods 0.000 description 2
- 238000013461 design Methods 0.000 description 2
- 230000000694 effects Effects 0.000 description 2
- 238000005516 engineering process Methods 0.000 description 2
- 239000000835 fiber Substances 0.000 description 2
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000004891 communication Methods 0.000 description 1
- 238000005192 partition Methods 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- 238000012546 transfer Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0668—Interfaces specially adapted for storage systems adopting a particular infrastructure
- G06F3/067—Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0602—Interfaces specially adapted for storage systems specifically adapted to achieve a particular effect
- G06F3/061—Improving I/O performance
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0628—Interfaces specially adapted for storage systems making use of a particular technique
- G06F3/0638—Organizing or formatting or addressing of data
- G06F3/0644—Management of space entities, e.g. partitions, extents, pools
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Human Computer Interaction (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The invention provides a method, a device, equipment and a readable medium for placing a group and selecting a master in a binding pool in distributed storage, wherein the method comprises the following steps: acquiring PG in a reference pool, acquiring corresponding PG in a binding pool according to the corresponding relation of PG, and putting the acquired PG into a variable structure; traversing the members of the PG in the reference pool, acquiring the IP of the node where the member is located, taking the acquired IP as a keyword, and storing the ID of the member and the ID of the PG in a data structure as a group of values; traversing the members of the corresponding PG in the binding pool, and judging whether the IP of the node where the member is located can be inquired in a data structure; in response to the fact that the IP of the node where the member is located can be inquired in the data structure, the ID of the member and the PG are added to the data structure of the value of the IP key as a group of values; and acquiring the number of values in each IP, and selecting the member in the structure with the maximum number of IP median values as the main of each PG. By using the scheme of the invention, the performance of the storage cluster can be improved, and the product competitiveness can be improved.
Description
Technical Field
The present invention relates to the field of computers, and more particularly, to a method, an apparatus, a device, and a readable medium for placing a group owner in a binding pool in a distributed storage.
Background
In the distributed storage system, the stored content is cut according to a fixed size, the block of data with the fixed size is called an object, PG is an aggregation of a plurality of objects, and is a logical concept, PG and the object are corresponded through a consistent hash algorithm, and one PG corresponds to a plurality of objects. Each PG corresponds to an OSD through a data distribution algorithm, one PG corresponds to a plurality of OSDs, the OSDs can be simply understood as physical disks, and a plurality of PGs can exist on one OSD, so that the content to be stored is placed on the disks. The storage pool is the aggregation of PG.
In the distributed storage scenario, at present, there is a scheme that OSD is deployed on a high-speed medium such as SSD in a partition manner, a cache pool is created by using the OSD, OSD is deployed on a low-speed medium HDD, a low-speed large-capacity storage pool is created by using the OSD, the two storage pools are bound, all data requests from clients pass through the cache pool, and then it is determined whether subsequent logic is issued to the cache pool for processing or forwarded to the low-speed pool for processing. The forwarding process is determined by the master in each PG member, and only the master can bear the read-write service request. The main in PG member means that one of the member OSD of PG is selected as main OSD to undertake business reading and writing, and the other OSD is slave OSD. The two bound storage pools have a corresponding relation of one PG, that is, a PG in a high-speed pool can only be forwarded to a PG in a low-speed pool corresponding to the PG when forwarding a request, and only the master of the PG has the function of forwarding processing. Therefore, when the pressure of the storage cluster reaches a certain degree, the problem of slow forwarding of the masters corresponding to the PGs is exposed, and one main reason of the slow forwarding is that forwarding requests need to pass through the network, when the masters corresponding to the PGs are not located on the same node, the requests need to pass through the network, and when the network reaches a bottleneck, the forwarding speed is limited, so that the performance of the storage cluster cannot be expected.
Disclosure of Invention
In view of this, embodiments of the present invention provide a method, an apparatus, a device, and a readable medium for placing group masters in binding pools in distributed storage, where by using the technical solution of the present invention, masters of PGs corresponding to two bound storage pools can be located on the same physical node as much as possible, thereby reducing network pressure, improving performance of storage clusters, and improving product competitiveness.
Based on the above object, an aspect of the embodiments of the present invention provides a method for placing group owners in a binding pool in a distributed storage, including the following steps:
acquiring PG in a reference pool, acquiring corresponding PG in a binding pool according to the corresponding relation of PG, and putting the acquired PG in the reference pool and the corresponding PG in the binding pool into a variable structure;
traversing the members of the PG in the reference pool, acquiring the IP of the node where the member is located, taking the acquired IP as a keyword, and storing the ID of the member and the ID of the PG in a data structure as a group of values;
traversing the members of the corresponding PG in the binding pool, and judging whether the IP of the node where the member is located can be inquired in a data structure;
in response to the fact that the IP of the node where the member is located can be inquired in the data structure, the ID of the member and the PG are added to the data structure of the value of the IP key as a group of values;
and acquiring the number of values in each IP, and selecting the member in the structure with the maximum number of IP median values as the main of each PG.
According to an embodiment of the present invention, further comprising:
adding the times of the member selection as the main in the data structure of each IP in response to the same number of the obtained values in each IP;
and selecting the member with the least number of additions as the main part of each PG.
According to an embodiment of the present invention, further comprising:
in response to the same value obtained by adding the times of the members as the main in the data structure of each IP, calculating the sum of the times of the OSD main on the node where each member is located;
and selecting the member with the least number of the primary times as the primary of the respective PG.
According to an embodiment of the present invention, further comprising:
deleting the PG from the variable structure in response to setting the selected member as the master of the respective PG;
judging whether the variable structure is empty or not;
in response to the variable structure not being empty, traversing the remaining members of the PG in the variable structure and adding the times of the members as the main;
and selecting the member with the least number of additions as the main part of each PG.
In another aspect of the embodiments of the present invention, there is also provided an apparatus for placing a group owner in a binding pool in a distributed storage, where the apparatus includes:
the acquisition module is configured to acquire the PG in the reference pool, acquire the corresponding PG in the binding pool according to the corresponding relation of the PG, and place the acquired PG in the reference pool and the corresponding PG in the binding pool into a variable structure;
the storage module is configured to traverse the members of the PG in the reference pool, acquire the IP of the node where the member is located, use the acquired IP as a keyword, and store the IDs of the members and the PG in a data structure as a group of values;
the judging module is configured to traverse the members of the corresponding PG in the binding pool and judge whether the IP of the node where the member is located can be inquired in the data structure;
the adding module is configured to respond that the IP of the node where the member is located can be inquired in a data structure, and add the ID of the member and the PG as a group of values to the data structure of the value of the IP keyword;
and the selecting module is configured to acquire the number of values in each IP and select the member in the structure with the largest number of IP median values as the main part of each PG.
According to one embodiment of the invention, the selection module is further configured to:
adding the times of the member selection as the main in the data structure of each IP in response to the same number of the obtained values in each IP;
and selecting the member with the least number of additions as the main part of each PG.
According to one embodiment of the invention, the selection module is further configured to:
in response to the same value obtained by adding the times of the members as the main in the data structure of each IP, calculating the sum of the times of the OSD main on the node where each member is located;
and selecting the member with the least number of the primary times as the primary of the respective PG.
According to an embodiment of the present invention, the system further comprises a deletion module configured to:
deleting the PG from the variable structure in response to setting the selected member as the master of the respective PG;
judging whether the variable structure is empty or not;
in response to the variable structure not being empty, traversing the remaining members of the PG in the variable structure and adding the times of the members as the main;
and selecting the member with the least number of additions as the main part of each PG.
In another aspect of an embodiment of the present invention, there is also provided a computer apparatus including:
at least one processor; and
a memory storing computer instructions executable on the processor, the instructions when executed by the processor implementing the steps of any of the methods described above.
In another aspect of the embodiments of the present invention, there is also provided a computer-readable storage medium storing a computer program, which when executed by a processor implements the steps of any one of the above-mentioned methods.
The invention has the following beneficial technical effects: the method for placing the group selection master in the binding pool in the distributed storage, provided by the embodiment of the invention, comprises the steps of obtaining PG in a reference pool, obtaining corresponding PG in the binding pool according to the corresponding relation of PG, and putting the obtained PG in the reference pool and the corresponding PG in the binding pool into a variable structure; traversing the members of the PG in the reference pool, acquiring the IP of the node where the member is located, taking the acquired IP as a keyword, and storing the ID of the member and the ID of the PG in a data structure as a group of values; traversing the members of the corresponding PG in the binding pool, and judging whether the IP of the node where the member is located can be inquired in a data structure; in response to the fact that the IP of the node where the member is located can be inquired in the data structure, the ID of the member and the PG are added to the data structure of the value of the IP key as a group of values; the technical scheme that the number of values in each IP is obtained, and the members in the structure with the largest number of IP median values are selected as the main PG of each IP can enable the main PG corresponding to the two bound storage pools to be located on the same physical node as much as possible, so that the network pressure is reduced, the performance of the storage cluster is improved, and the product competitiveness is improved.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below, and it is obvious that the drawings in the following description are only some embodiments of the present invention, and it is obvious for those skilled in the art that other embodiments can be obtained by using the drawings without creative efforts.
FIG. 1 is a schematic flow chart diagram of a method for placing group masters by a binding pool in a distributed storage according to one embodiment of the present invention;
FIG. 2 is a diagram illustrating an apparatus for placing group masters in a binding pool in a distributed storage according to an embodiment of the present invention;
FIG. 3 is a schematic diagram of a computer device according to one embodiment of the present invention;
fig. 4 is a schematic diagram of a computer-readable storage medium according to an embodiment of the present invention.
Detailed Description
In order to make the objects, technical solutions and advantages of the present invention more apparent, the following embodiments of the present invention are described in further detail with reference to the accompanying drawings.
In view of the foregoing, a first aspect of the embodiments of the present invention provides an embodiment of a method for placing group owners in a binding pool in a distributed storage. Fig. 1 shows a schematic flow diagram of the method.
As shown in fig. 1, the method may include the steps of:
s1, acquiring the PG in the reference pool, acquiring the corresponding PG in the binding pool according to the corresponding relation of the PG, and putting the acquired PG in the reference pool and the corresponding PG in the binding pool into a variable structure.
A storage pool with a small PG number is used as a reference pool, the other storage pool is called a binding pool of the reference pool, and the PG number of the storage pools is always an integral power of 2, so even if the PG numbers of the two storage pools are different, the PG numbers of the two storage pools are in a relation of integral multiple of 2. It can be seen that the following PG relationships are always true: with the storage pool1 with a small number of PGs as a reference, the PGs in the storage pool with a large number of PGs can be split according to the number of PGs of pool1, the number of PGs in each split share is equal to the number of PGs of pool1, and each share is in one-to-one correspondence with the PG of pool1, so that the correspondence relationship of the PGs can be obtained. Obtaining the first PG in the reference pool and the corresponding PG in the binding pool, and configuring the structures of these PG anti-theft variables to correspond to PG (group _ PGs): 1.0,2.0,2.1024 indicate that PG number 0 in the base pool1 corresponds to PG number 0 in the bonding pool 2 and PG number 1024 in the bonding pool 2.
S2 traversing the PG member in the reference pool, obtaining the IP of the node where the member is located, using the obtained IP as a key word, and storing the ID of the member and PG as a group of values in a data structure.
Each PG has a plurality of OSD members, each member has a node IP where it is located, the node IP and the corresponding OSD member in the members are counted, then the node IP is used as a key word, the IDs of the OSD members and the PG are stored in a data structure as a group of values, as shown in fig. 3, the value in IP127.0.0.1 represents OSD No. 0 corresponding to PG No. 0 in the reference pool1, the value in ip127.0.0.2 represents OSD No. 6 corresponding to PG No. 0 in the reference pool1, and so on.
S3, traversing the corresponding PG member in the binding pool, and judging whether the node IP where the member is located can be inquired in the data structure.
S4 appends the ID of the member and PG as a set of values to the data structure of the value of the IP key in response to the member being able to be queried in the data structure at the node IP.
Then, traversing the members of the PG of the binding pool corresponding to PG No. 0 in the reference pool1, and checking whether the node IP where the members are located can be queried in the data structure, as shown in fig. 4, the values added in IP127.0.0.1 are OSD No. 1 corresponding to PG No. 0 in the binding pool 2, OSD No. 2 corresponding to PG No. 1024 in the binding pool 2, and so on.
S5 obtains the number of values in each IP, and selects the member in the structure with the largest number of IP medians as the master of each PG.
As shown in fig. 4, IP127.0.0.1 has 3 values, and if the number is the largest, the OSD corresponding to the 3 values is selected as the master of each PG. And after the steps are completed, restarting the steps from the second PG in the reference pool until all the main settings of all the PGs are completed.
By the technical scheme, the PG corresponding to the two bound storage pools can be located on the same physical node as much as possible, so that the network pressure is reduced, the performance of the storage cluster is improved, and the product competitiveness is improved.
In a preferred embodiment of the present invention, the method further comprises:
adding the times of the member selection as the main in the data structure of each IP in response to the same number of the obtained values in each IP;
and selecting the member with the least number of additions as the main part of each PG. And if the number of the data in each IP in the data structure is the same, counting the number of the masters made by each member, and selecting the member with the least number of the masters to make the PG.
In a preferred embodiment of the present invention, the method further comprises:
in response to the same value obtained by adding the times of the members as the main in the data structure of each IP, calculating the sum of the times of the OSD main on the node where each member is located;
and selecting the member with the least number of the primary times as the primary of the respective PG. And if the number of the membership owner is the same, counting the sum of the number of the membership owner on the node, and selecting the member with the minimum sum of the number of the membership owner on the node as the PG owner.
In a preferred embodiment of the present invention, the method further comprises:
deleting the PG from the variable structure in response to setting the selected member as the master of the respective PG;
judging whether the variable structure is empty or not;
in response to the variable structure not being empty, traversing the remaining members of the PG in the variable structure and adding the times of the members as the main;
and selecting the member with the least number of additions as the main part of each PG. The variable structure is group _ PGs, after the selected main modification is completed, OSD main counting and node main counting are updated, the PGs are deleted from the group _ PGs, if the group _ PGs are not empty, the PG members are not elected, the PG members are traversed, the member with the minimum OSD main counting is selected as the main, if the same, the main counting of the node where the OSD is located is compared, the fewer elected members are updated, and the OSD main counting and the node main counting are updated.
It should be noted that, as will be understood by those skilled in the art, all or part of the processes in the methods of the above embodiments may be implemented by instructing relevant hardware through a computer program, and the above programs may be stored in a computer-readable storage medium, and when executed, the programs may include the processes of the embodiments of the methods as described above. The storage medium may be a magnetic disk, an optical disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), or the like. The embodiments of the computer program may achieve the same or similar effects as any of the above-described method embodiments.
Furthermore, the method disclosed according to an embodiment of the present invention may also be implemented as a computer program executed by a CPU, and the computer program may be stored in a computer-readable storage medium. The computer program, when executed by the CPU, performs the above-described functions defined in the method disclosed in the embodiments of the present invention.
In view of the above object, according to a second aspect of the embodiments of the present invention, there is provided an apparatus for placing group election owners in a binding pool in a distributed storage, as shown in fig. 2, the apparatus 200 includes:
the acquisition module is configured to acquire the PG in the reference pool, acquire the corresponding PG in the binding pool according to the corresponding relation of the PG, and place the acquired PG in the reference pool and the corresponding PG in the binding pool into a variable structure;
the storage module is configured to traverse the members of the PG in the reference pool, acquire the IP of the node where the member is located, use the acquired IP as a keyword, and store the IDs of the members and the PG in a data structure as a group of values;
the judging module is configured to traverse the members of the corresponding PG in the binding pool and judge whether the IP of the node where the member is located can be inquired in the data structure;
the adding module is configured to respond that the IP of the node where the member is located can be inquired in a data structure, and add the ID of the member and the PG as a group of values to the data structure of the value of the IP keyword;
and the selecting module is configured to acquire the number of values in each IP and select the member in the structure with the largest number of IP median values as the main part of each PG.
In a preferred embodiment of the invention, the selection module is further configured to:
adding the times of the member selection as the main in the data structure of each IP in response to the same number of the obtained values in each IP;
and selecting the member with the least number of additions as the main part of each PG.
In a preferred embodiment of the invention, the selection module is further configured to:
in response to the same value obtained by adding the times of the members as the main in the data structure of each IP, calculating the sum of the times of the OSD main on the node where each member is located;
and selecting the member with the least number of the primary times as the primary of the respective PG.
In a preferred embodiment of the present invention, the system further includes a deletion module, and the deletion module is configured to:
deleting the PG from the variable structure in response to setting the selected member as the master of the respective PG;
judging whether the variable structure is empty or not;
in response to the variable structure not being empty, traversing the remaining members of the PG in the variable structure and adding the times of the members as the main;
and selecting the member with the least number of additions as the main part of each PG.
In view of the above object, a third aspect of the embodiments of the present invention provides a computer device. Fig. 3 is a schematic diagram of an embodiment of a computer device provided by the present invention. As shown in fig. 3, an embodiment of the present invention includes the following means: at least one processor 21; and a memory 22, the memory 22 storing computer instructions 23 executable on the processor, the instructions when executed by the processor implementing the method of:
acquiring PG in a reference pool, acquiring corresponding PG in a binding pool according to the corresponding relation of PG, and putting the acquired PG in the reference pool and the corresponding PG in the binding pool into a variable structure;
traversing the members of the PG in the reference pool, acquiring the IP of the node where the member is located, taking the acquired IP as a keyword, and storing the ID of the member and the ID of the PG in a data structure as a group of values;
traversing the members of the corresponding PG in the binding pool, and judging whether the IP of the node where the member is located can be inquired in a data structure;
in response to the fact that the IP of the node where the member is located can be inquired in the data structure, the ID of the member and the PG are added to the data structure of the value of the IP key as a group of values;
and acquiring the number of values in each IP, and selecting the member in the structure with the maximum number of IP median values as the main of each PG.
In a preferred embodiment of the present invention, the method further comprises:
adding the times of the member selection as the main in the data structure of each IP in response to the same number of the obtained values in each IP;
and selecting the member with the least number of additions as the main part of each PG.
In a preferred embodiment of the present invention, the method further comprises:
in response to the same value obtained by adding the times of the members as the main in the data structure of each IP, calculating the sum of the times of the OSD main on the node where each member is located;
and selecting the member with the least number of the primary times as the primary of the respective PG.
In a preferred embodiment of the present invention, the method further comprises:
deleting the PG from the variable structure in response to setting the selected member as the master of the respective PG;
judging whether the variable structure is empty or not;
in response to the variable structure not being empty, traversing the remaining members of the PG in the variable structure and adding the times of the members as the main;
and selecting the member with the least number of additions as the main part of each PG.
In view of the above object, a fourth aspect of the embodiments of the present invention proposes a computer-readable storage medium. FIG. 4 is a schematic diagram illustrating an embodiment of a computer-readable storage medium provided by the present invention. As shown in fig. 4, the computer readable storage medium S31 stores a computer program S32 that when executed by a processor performs the method of:
acquiring PG in a reference pool, acquiring corresponding PG in a binding pool according to the corresponding relation of PG, and putting the acquired PG in the reference pool and the corresponding PG in the binding pool into a variable structure;
traversing the members of the PG in the reference pool, acquiring the IP of the node where the member is located, taking the acquired IP as a keyword, and storing the ID of the member and the ID of the PG in a data structure as a group of values;
traversing the members of the corresponding PG in the binding pool, and judging whether the IP of the node where the member is located can be inquired in a data structure;
in response to the fact that the IP of the node where the member is located can be inquired in the data structure, the ID of the member and the PG are added to the data structure of the value of the IP key as a group of values;
and acquiring the number of values in each IP, and selecting the member in the structure with the maximum number of IP median values as the main of each PG.
In a preferred embodiment of the present invention, the method further comprises:
adding the times of the member selection as the main in the data structure of each IP in response to the same number of the obtained values in each IP;
and selecting the member with the least number of additions as the main part of each PG.
In a preferred embodiment of the present invention, the method further comprises:
in response to the same value obtained by adding the times of the members as the main in the data structure of each IP, calculating the sum of the times of the OSD main on the node where each member is located;
and selecting the member with the least number of the primary times as the primary of the respective PG.
In a preferred embodiment of the present invention, the method further comprises:
deleting the PG from the variable structure in response to setting the selected member as the master of the respective PG;
judging whether the variable structure is empty or not;
in response to the variable structure not being empty, traversing the remaining members of the PG in the variable structure and adding the times of the members as the main;
and selecting the member with the least number of additions as the main part of each PG.
Furthermore, the methods disclosed according to embodiments of the present invention may also be implemented as a computer program executed by a processor, which may be stored in a computer-readable storage medium. Which when executed by a processor performs the above-described functions defined in the methods disclosed in embodiments of the invention.
Further, the above method steps and system elements may also be implemented using a controller and a computer readable storage medium for storing a computer program for causing the controller to implement the functions of the above steps or elements.
Those of skill would further appreciate that the various illustrative logical blocks, modules, circuits, and algorithm steps described in connection with the disclosure herein may be implemented as electronic hardware, computer software, or combinations of both. To clearly illustrate this interchangeability of hardware and software, various illustrative components, blocks, modules, circuits, and steps have been described above generally in terms of their functionality. Whether such functionality is implemented as software or hardware depends upon the particular application and design constraints imposed on the overall system. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the disclosed embodiments of the present invention.
In one or more exemplary designs, the functions may be implemented in hardware, software, firmware, or any combination thereof. If implemented in software, the functions may be stored on or transmitted over as one or more instructions or code on a computer-readable medium. Computer-readable media includes both computer storage media and communication media including any medium that facilitates transfer of a computer program from one place to another. A storage media may be any available media that can be accessed by a general purpose or special purpose computer. By way of example, and not limitation, such computer-readable media can comprise RAM, ROM, EEPROM, CD-ROM or other optical disk storage, magnetic disk storage or other magnetic storage devices, or any other medium that can be used to carry or store desired program code in the form of instructions or data structures and that can be accessed by a general-purpose or special-purpose computer, or a general-purpose or special-purpose processor. Also, any connection is properly termed a computer-readable medium. For example, if the software is transmitted from a website, server, or other remote source using a coaxial cable, fiber optic cable, twisted pair, Digital Subscriber Line (DSL), or wireless technologies such as infrared, radio, and microwave, then the coaxial cable, fiber optic cable, twisted pair, DSL, or wireless technologies such as infrared, radio, and microwave are included in the definition of medium. Disk and disc, as used herein, includes Compact Disc (CD), laser disc, optical disc, Digital Versatile Disc (DVD), floppy disk, blu-ray disc where disks usually reproduce data magnetically, while discs reproduce data optically with lasers. Combinations of the above should also be included within the scope of computer-readable media.
The foregoing is an exemplary embodiment of the present disclosure, but it should be noted that various changes and modifications could be made herein without departing from the scope of the present disclosure as defined by the appended claims. The functions, steps and/or actions of the method claims in accordance with the disclosed embodiments described herein need not be performed in any particular order. Furthermore, although elements of the disclosed embodiments of the invention may be described or claimed in the singular, the plural is contemplated unless limitation to the singular is explicitly stated.
It should be understood that, as used herein, the singular forms "a", "an" and "the" are intended to include the plural forms as well, unless the context clearly supports the exception. It should also be understood that "and/or" as used herein is meant to include any and all possible combinations of one or more of the associated listed items.
The numbers of the embodiments disclosed in the embodiments of the present invention are merely for description, and do not represent the merits of the embodiments.
It will be understood by those skilled in the art that all or part of the steps for implementing the above embodiments may be implemented by hardware, or may be implemented by a program instructing relevant hardware, and the program may be stored in a computer-readable storage medium, and the above-mentioned storage medium may be a read-only memory, a magnetic disk or an optical disk, etc.
Those of ordinary skill in the art will understand that: the discussion of any embodiment above is meant to be exemplary only, and is not intended to intimate that the scope of the disclosure, including the claims, of embodiments of the invention is limited to these examples; within the idea of an embodiment of the invention, also technical features in the above embodiment or in different embodiments may be combined and there are many other variations of the different aspects of the embodiments of the invention as described above, which are not provided in detail for the sake of brevity. Therefore, any omissions, modifications, substitutions, improvements, and the like that may be made without departing from the spirit and principles of the embodiments of the present invention are intended to be included within the scope of the embodiments of the present invention.
Claims (10)
1. A method for placing group and master selection in a binding pool in distributed storage is characterized by comprising the following steps:
acquiring PG in a reference pool, acquiring corresponding PG in a binding pool according to the corresponding relation of PG, and putting the acquired PG in the reference pool and the corresponding PG in the binding pool into a variable structure;
traversing the members of the PG in the reference pool, acquiring the IP of the node where the member is located, taking the acquired IP as a keyword, and storing the IDs of the member and the PG in a data structure as a group of values;
traversing the members of the corresponding PG in the binding pool, and judging whether the IP of the node where the member is located can be inquired in the data structure;
in response to the member's node IP being able to be queried in the data structure, appending the ID of the member and PG as a set of values to the data structure of the values of the IP key;
and acquiring the number of values in each IP, and selecting the member in the structure with the maximum number of IP median values as the main of each PG.
2. The method of claim 1, further comprising:
adding the times of the member selection as the main in the data structure of each IP in response to the same number of the obtained values in each IP;
and selecting the member with the least number of additions as the main part of each PG.
3. The method of claim 2, further comprising:
in response to the same value obtained by adding the times of the members as the main in the data structure of each IP, calculating the sum of the times of the OSD main on the node where each member is located;
and selecting the member with the least number of the primary times as the primary of the respective PG.
4. The method of claim 1, further comprising:
deleting the PG from the variable structure in response to setting the selected member as the master of the respective PG;
judging whether the variable structure is empty or not;
in response to the variable structure not being empty, traversing the members of the remaining PGs in the variable structure and adding the number of times that the members are selected as the master;
and selecting the member with the least number of additions as the main part of each PG.
5. An apparatus for placing group owners in a binding pool in a distributed storage, the apparatus comprising:
the acquisition module is configured to acquire the PG in the reference pool, acquire the corresponding PG in the binding pool according to the corresponding relation of the PG, and place the acquired PG in the reference pool and the corresponding PG in the binding pool into a variable structure;
the storage module is configured to traverse the members of the PG in the reference pool, acquire the IP of the node where the member is located, use the acquired IP as a keyword, and store the IDs of the member and the PG in a data structure as a group of values;
the judging module is configured to traverse the members of the corresponding PG in the binding pool and judge whether the IP of the node where the member is located can be inquired in the data structure;
an adding module, configured to add the IDs of the member and the PG as a set of values to a data structure of values of the IP keyword in response to the node IP where the member is located being able to be queried in the data structure;
and the selecting module is configured to acquire the number of values in each IP and select the member in the structure with the largest number of IP median values as the main part of each PG.
6. The apparatus of claim 5, wherein the selection module is further configured to:
adding the times of the member selection as the main in the data structure of each IP in response to the same number of the obtained values in each IP;
and selecting the member with the least number of additions as the main part of each PG.
7. The apparatus of claim 6, wherein the selection module is further configured to:
in response to the same value obtained by adding the times of the members as the main in the data structure of each IP, calculating the sum of the times of the OSD main on the node where each member is located;
and selecting the member with the least number of the primary times as the primary of the respective PG.
8. The apparatus of claim 5, further comprising a deletion module configured to:
deleting the PG from the variable structure in response to setting the selected member as the master of the respective PG;
judging whether the variable structure is empty or not;
in response to the variable structure not being empty, traversing the members of the remaining PGs in the variable structure and adding the number of times that the members are selected as the master;
and selecting the member with the least number of additions as the main part of each PG.
9. A computer device, comprising:
at least one processor; and
a memory storing computer instructions executable on the processor, the instructions when executed by the processor implementing the steps of the method of any one of claims 1 to 4.
10. A computer-readable storage medium, in which a computer program is stored which, when being executed by a processor, carries out the steps of the method according to any one of claims 1 to 4.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111237021.1A CN114138181A (en) | 2021-10-24 | 2021-10-24 | Method, device, equipment and readable medium for placing, grouping and selecting owners in binding pool |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111237021.1A CN114138181A (en) | 2021-10-24 | 2021-10-24 | Method, device, equipment and readable medium for placing, grouping and selecting owners in binding pool |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN114138181A true CN114138181A (en) | 2022-03-04 |
Family
ID=80394704
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202111237021.1A Pending CN114138181A (en) | 2021-10-24 | 2021-10-24 | Method, device, equipment and readable medium for placing, grouping and selecting owners in binding pool |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114138181A (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115202589A (en) * | 2022-09-14 | 2022-10-18 | 浪潮电子信息产业股份有限公司 | Placement group member selection method, device, equipment and readable storage medium |
| CN117061541A (en) * | 2023-10-11 | 2023-11-14 | 苏州元脑智能科技有限公司 | Storage double-pool binding method and device, electronic equipment and readable storage medium |
Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106503237A (en) * | 2016-11-04 | 2017-03-15 | 郑州云海信息技术有限公司 | A kind of PG member selections method, device, storage pool and distributed file system |
| CN110018799A (en) * | 2019-04-12 | 2019-07-16 | 苏州浪潮智能科技有限公司 | A kind of main determining method, apparatus of storage pool PG, equipment and readable storage medium storing program for executing |
| CN110222013A (en) * | 2019-06-11 | 2019-09-10 | 深信服科技股份有限公司 | A kind of method, system, equipment and storage medium that cluster storage capacity determines |
| CN111414136A (en) * | 2020-03-13 | 2020-07-14 | 苏州浪潮智能科技有限公司 | Method, system, device and medium for creating storage pool |
| CN111708486A (en) * | 2020-05-24 | 2020-09-25 | 苏州浪潮智能科技有限公司 | Method, system, equipment and medium for balanced optimization of main placement group |
| CN111752483A (en) * | 2020-05-28 | 2020-10-09 | 苏州浪潮智能科技有限公司 | Method and system for reducing reconstruction data by changing storage medium in storage cluster |
| CN111930317A (en) * | 2020-09-21 | 2020-11-13 | 柏科数据技术（深圳）股份有限公司 | Data distribution method, device, server and storage medium based on CEPH |
| US20210216245A1 (en) * | 2018-07-10 | 2021-07-15 | Here Data Technology | Method of distributed data redundancy storage using consistent hashing |
-
2021
- 2021-10-24 CN CN202111237021.1A patent/CN114138181A/en active Pending
Patent Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106503237A (en) * | 2016-11-04 | 2017-03-15 | 郑州云海信息技术有限公司 | A kind of PG member selections method, device, storage pool and distributed file system |
| US20210216245A1 (en) * | 2018-07-10 | 2021-07-15 | Here Data Technology | Method of distributed data redundancy storage using consistent hashing |
| CN110018799A (en) * | 2019-04-12 | 2019-07-16 | 苏州浪潮智能科技有限公司 | A kind of main determining method, apparatus of storage pool PG, equipment and readable storage medium storing program for executing |
| CN110222013A (en) * | 2019-06-11 | 2019-09-10 | 深信服科技股份有限公司 | A kind of method, system, equipment and storage medium that cluster storage capacity determines |
| CN111414136A (en) * | 2020-03-13 | 2020-07-14 | 苏州浪潮智能科技有限公司 | Method, system, device and medium for creating storage pool |
| CN111708486A (en) * | 2020-05-24 | 2020-09-25 | 苏州浪潮智能科技有限公司 | Method, system, equipment and medium for balanced optimization of main placement group |
| CN111752483A (en) * | 2020-05-28 | 2020-10-09 | 苏州浪潮智能科技有限公司 | Method and system for reducing reconstruction data by changing storage medium in storage cluster |
| CN111930317A (en) * | 2020-09-21 | 2020-11-13 | 柏科数据技术（深圳）股份有限公司 | Data distribution method, device, server and storage medium based on CEPH |
Cited By (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115202589A (en) * | 2022-09-14 | 2022-10-18 | 浪潮电子信息产业股份有限公司 | Placement group member selection method, device, equipment and readable storage medium |
| CN115202589B (en) * | 2022-09-14 | 2023-02-24 | 浪潮电子信息产业股份有限公司 | Placement group member selection method, device and equipment and readable storage medium |
| WO2024055529A1 (en) * | 2022-09-14 | 2024-03-21 | 浪潮电子信息产业股份有限公司 | Placement group member selection method and apparatus, device, and readable storage medium |
| CN117061541A (en) * | 2023-10-11 | 2023-11-14 | 苏州元脑智能科技有限公司 | Storage double-pool binding method and device, electronic equipment and readable storage medium |
| CN117061541B (en) * | 2023-10-11 | 2024-02-09 | 苏州元脑智能科技有限公司 | Storage double-pool binding method and device, electronic equipment and readable storage medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CA2822900C (en) | Filtering queried data on data stores | |
| CN107911249B (en) | Method, device and equipment for sending command line of network equipment | |
| CN114138181A (en) | Method, device, equipment and readable medium for placing, grouping and selecting owners in binding pool | |
| CN109684099B (en) | Message processing method and device | |
| WO2016066108A1 (en) | Route access method, route access system and user terminal | |
| CN103973810A (en) | Data processing method and device based on IP disk | |
| CN106815218A (en) | Data bank access method, device and Database Systems | |
| CN110888837B (en) | Object storage small file merging method and device | |
| US11811839B2 (en) | Managed distribution of data stream contents | |
| CN108460041A (en) | The treating method and apparatus of data | |
| CN103324533A (en) | distributed data processing method, device and system | |
| CN109032533B (en) | Data storage method, device and equipment | |
| CN110781156A (en) | Data node distribution method, equipment and medium | |
| US10700934B2 (en) | Communication control device, communication control method, and computer program product | |
| CN111083179A (en) | Internet of things cloud platform, and equipment interaction method and device based on same | |
| CN104598652B (en) | A kind of data base query method and device | |
| CN114064662B (en) | Method, device, electronic equipment and medium for querying data | |
| CN112003915A (en) | File access method, device, equipment and readable medium | |
| CN102724301B (en) | Cloud database system and method and equipment for reading and writing cloud data | |
| CN116756253B (en) | Data storage and query methods, devices, equipment and media of relational database | |
| CN115904211A (en) | Storage system, data processing method and related equipment | |
| CN111209304B (en) | Data processing method, device and system | |
| CN108984780B (en) | Method and device for managing disk data based on data structure supporting repeated key value tree | |
| CN104239307A (en) | User information storage method and system | |
| CN109151016B (en) | Flow forwarding method and device, service system, computing device and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| TA01 | Transfer of patent application right | ||
| TA01 | Transfer of patent application right |
Effective date of registration: 20240419 Address after: Room 1801, Floor 18, Jiyun Investment Building, No. 278, Xinyi Road, Zhengdong New District, Zhengzhou City, Henan Province, 450000 Applicant after: Zhengzhou Inspur Data Technology Co.,Ltd. Country or region after: China Address before: Room s311, building S05, Inspur Science Park, No. 1036, Inspur Road, Jinan area, China (Shandong) pilot Free Trade Zone, Jinan City, Shandong Province Applicant before: Ji'nan tide data Technology Co.,Ltd. Country or region before: China |