Disclosure of Invention
In view of this, embodiments of the present invention provide a method, an apparatus, a device, and a readable medium for placing group masters in binding pools in distributed storage, where by using the technical solution of the present invention, masters of PGs corresponding to two bound storage pools can be located on the same physical node as much as possible, thereby reducing network pressure, improving performance of storage clusters, and improving product competitiveness.
Based on the above object, an aspect of the embodiments of the present invention provides a method for placing group owners in a binding pool in a distributed storage, including the following steps:
acquiring PG in a reference pool, acquiring corresponding PG in a binding pool according to the corresponding relation of PG, and putting the acquired PG in the reference pool and the corresponding PG in the binding pool into a variable structure;
traversing the members of the PG in the reference pool, acquiring the IP of the node where the member is located, taking the acquired IP as a keyword, and storing the ID of the member and the ID of the PG in a data structure as a group of values;
traversing the members of the corresponding PG in the binding pool, and judging whether the IP of the node where the member is located can be inquired in a data structure;
in response to the fact that the IP of the node where the member is located can be inquired in the data structure, the ID of the member and the PG are added to the data structure of the value of the IP key as a group of values;
and acquiring the number of values in each IP, and selecting the member in the structure with the maximum number of IP median values as the main of each PG.
According to an embodiment of the present invention, further comprising:
adding the times of the member selection as the main in the data structure of each IP in response to the same number of the obtained values in each IP;
and selecting the member with the least number of additions as the main part of each PG.
According to an embodiment of the present invention, further comprising:
in response to the same value obtained by adding the times of the members as the main in the data structure of each IP, calculating the sum of the times of the OSD main on the node where each member is located;
and selecting the member with the least number of the primary times as the primary of the respective PG.
According to an embodiment of the present invention, further comprising:
deleting the PG from the variable structure in response to setting the selected member as the master of the respective PG;
judging whether the variable structure is empty or not;
in response to the variable structure not being empty, traversing the remaining members of the PG in the variable structure and adding the times of the members as the main;
and selecting the member with the least number of additions as the main part of each PG.
In another aspect of the embodiments of the present invention, there is also provided an apparatus for placing a group owner in a binding pool in a distributed storage, where the apparatus includes:
the acquisition module is configured to acquire the PG in the reference pool, acquire the corresponding PG in the binding pool according to the corresponding relation of the PG, and place the acquired PG in the reference pool and the corresponding PG in the binding pool into a variable structure;
the storage module is configured to traverse the members of the PG in the reference pool, acquire the IP of the node where the member is located, use the acquired IP as a keyword, and store the IDs of the members and the PG in a data structure as a group of values;
the judging module is configured to traverse the members of the corresponding PG in the binding pool and judge whether the IP of the node where the member is located can be inquired in the data structure;
the adding module is configured to respond that the IP of the node where the member is located can be inquired in a data structure, and add the ID of the member and the PG as a group of values to the data structure of the value of the IP keyword;
and the selecting module is configured to acquire the number of values in each IP and select the member in the structure with the largest number of IP median values as the main part of each PG.
According to one embodiment of the invention, the selection module is further configured to:
adding the times of the member selection as the main in the data structure of each IP in response to the same number of the obtained values in each IP;
and selecting the member with the least number of additions as the main part of each PG.
According to one embodiment of the invention, the selection module is further configured to:
in response to the same value obtained by adding the times of the members as the main in the data structure of each IP, calculating the sum of the times of the OSD main on the node where each member is located;
and selecting the member with the least number of the primary times as the primary of the respective PG.
According to an embodiment of the present invention, the system further comprises a deletion module configured to:
deleting the PG from the variable structure in response to setting the selected member as the master of the respective PG;
judging whether the variable structure is empty or not;
in response to the variable structure not being empty, traversing the remaining members of the PG in the variable structure and adding the times of the members as the main;
and selecting the member with the least number of additions as the main part of each PG.
In another aspect of an embodiment of the present invention, there is also provided a computer apparatus including:
at least one processor; and
a memory storing computer instructions executable on the processor, the instructions when executed by the processor implementing the steps of any of the methods described above.
In another aspect of the embodiments of the present invention, there is also provided a computer-readable storage medium storing a computer program, which when executed by a processor implements the steps of any one of the above-mentioned methods.
The invention has the following beneficial technical effects: the method for placing the group selection master in the binding pool in the distributed storage, provided by the embodiment of the invention, comprises the steps of obtaining PG in a reference pool, obtaining corresponding PG in the binding pool according to the corresponding relation of PG, and putting the obtained PG in the reference pool and the corresponding PG in the binding pool into a variable structure; traversing the members of the PG in the reference pool, acquiring the IP of the node where the member is located, taking the acquired IP as a keyword, and storing the ID of the member and the ID of the PG in a data structure as a group of values; traversing the members of the corresponding PG in the binding pool, and judging whether the IP of the node where the member is located can be inquired in a data structure; in response to the fact that the IP of the node where the member is located can be inquired in the data structure, the ID of the member and the PG are added to the data structure of the value of the IP key as a group of values; the technical scheme that the number of values in each IP is obtained, and the members in the structure with the largest number of IP median values are selected as the main PG of each IP can enable the main PG corresponding to the two bound storage pools to be located on the same physical node as much as possible, so that the network pressure is reduced, the performance of the storage cluster is improved, and the product competitiveness is improved.
Detailed Description
In order to make the objects, technical solutions and advantages of the present invention more apparent, the following embodiments of the present invention are described in further detail with reference to the accompanying drawings.
In view of the foregoing, a first aspect of the embodiments of the present invention provides an embodiment of a method for placing group owners in a binding pool in a distributed storage. Fig. 1 shows a schematic flow diagram of the method.
As shown in fig. 1, the method may include the steps of:
s1, acquiring the PG in the reference pool, acquiring the corresponding PG in the binding pool according to the corresponding relation of the PG, and putting the acquired PG in the reference pool and the corresponding PG in the binding pool into a variable structure.
A storage pool with a small PG number is used as a reference pool, the other storage pool is called a binding pool of the reference pool, and the PG number of the storage pools is always an integral power of 2, so even if the PG numbers of the two storage pools are different, the PG numbers of the two storage pools are in a relation of integral multiple of 2. It can be seen that the following PG relationships are always true: with the storage pool1 with a small number of PGs as a reference, the PGs in the storage pool with a large number of PGs can be split according to the number of PGs of pool1, the number of PGs in each split share is equal to the number of PGs of pool1, and each share is in one-to-one correspondence with the PG of pool1, so that the correspondence relationship of the PGs can be obtained. Obtaining the first PG in the reference pool and the corresponding PG in the binding pool, and configuring the structures of these PG anti-theft variables to correspond to PG (group _ PGs): 1.0,2.0,2.1024 indicate that PG number 0 in the base pool1 corresponds to PG number 0 in the bonding pool 2 and PG number 1024 in the bonding pool 2.
S2 traversing the PG member in the reference pool, obtaining the IP of the node where the member is located, using the obtained IP as a key word, and storing the ID of the member and PG as a group of values in a data structure.
Each PG has a plurality of OSD members, each member has a node IP where it is located, the node IP and the corresponding OSD member in the members are counted, then the node IP is used as a key word, the IDs of the OSD members and the PG are stored in a data structure as a group of values, as shown in fig. 3, the value in IP127.0.0.1 represents OSD No. 0 corresponding to PG No. 0 in the reference pool1, the value in ip127.0.0.2 represents OSD No. 6 corresponding to PG No. 0 in the reference pool1, and so on.
S3, traversing the corresponding PG member in the binding pool, and judging whether the node IP where the member is located can be inquired in the data structure.
S4 appends the ID of the member and PG as a set of values to the data structure of the value of the IP key in response to the member being able to be queried in the data structure at the node IP.
Then, traversing the members of the PG of the binding pool corresponding to PG No. 0 in the reference pool1, and checking whether the node IP where the members are located can be queried in the data structure, as shown in fig. 4, the values added in IP127.0.0.1 are OSD No. 1 corresponding to PG No. 0 in the binding pool 2, OSD No. 2 corresponding to PG No. 1024 in the binding pool 2, and so on.
S5 obtains the number of values in each IP, and selects the member in the structure with the largest number of IP medians as the master of each PG.
As shown in fig. 4, IP127.0.0.1 has 3 values, and if the number is the largest, the OSD corresponding to the 3 values is selected as the master of each PG. And after the steps are completed, restarting the steps from the second PG in the reference pool until all the main settings of all the PGs are completed.
By the technical scheme, the PG corresponding to the two bound storage pools can be located on the same physical node as much as possible, so that the network pressure is reduced, the performance of the storage cluster is improved, and the product competitiveness is improved.
In a preferred embodiment of the present invention, the method further comprises:
adding the times of the member selection as the main in the data structure of each IP in response to the same number of the obtained values in each IP;
and selecting the member with the least number of additions as the main part of each PG. And if the number of the data in each IP in the data structure is the same, counting the number of the masters made by each member, and selecting the member with the least number of the masters to make the PG.
In a preferred embodiment of the present invention, the method further comprises:
in response to the same value obtained by adding the times of the members as the main in the data structure of each IP, calculating the sum of the times of the OSD main on the node where each member is located;
and selecting the member with the least number of the primary times as the primary of the respective PG. And if the number of the membership owner is the same, counting the sum of the number of the membership owner on the node, and selecting the member with the minimum sum of the number of the membership owner on the node as the PG owner.
In a preferred embodiment of the present invention, the method further comprises:
deleting the PG from the variable structure in response to setting the selected member as the master of the respective PG;
judging whether the variable structure is empty or not;
in response to the variable structure not being empty, traversing the remaining members of the PG in the variable structure and adding the times of the members as the main;
and selecting the member with the least number of additions as the main part of each PG. The variable structure is group _ PGs, after the selected main modification is completed, OSD main counting and node main counting are updated, the PGs are deleted from the group _ PGs, if the group _ PGs are not empty, the PG members are not elected, the PG members are traversed, the member with the minimum OSD main counting is selected as the main, if the same, the main counting of the node where the OSD is located is compared, the fewer elected members are updated, and the OSD main counting and the node main counting are updated.
It should be noted that, as will be understood by those skilled in the art, all or part of the processes in the methods of the above embodiments may be implemented by instructing relevant hardware through a computer program, and the above programs may be stored in a computer-readable storage medium, and when executed, the programs may include the processes of the embodiments of the methods as described above. The storage medium may be a magnetic disk, an optical disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), or the like. The embodiments of the computer program may achieve the same or similar effects as any of the above-described method embodiments.
Furthermore, the method disclosed according to an embodiment of the present invention may also be implemented as a computer program executed by a CPU, and the computer program may be stored in a computer-readable storage medium. The computer program, when executed by the CPU, performs the above-described functions defined in the method disclosed in the embodiments of the present invention.
In view of the above object, according to a second aspect of the embodiments of the present invention, there is provided an apparatus for placing group election owners in a binding pool in a distributed storage, as shown in fig. 2, the apparatus 200 includes:
the acquisition module is configured to acquire the PG in the reference pool, acquire the corresponding PG in the binding pool according to the corresponding relation of the PG, and place the acquired PG in the reference pool and the corresponding PG in the binding pool into a variable structure;
the storage module is configured to traverse the members of the PG in the reference pool, acquire the IP of the node where the member is located, use the acquired IP as a keyword, and store the IDs of the members and the PG in a data structure as a group of values;
the judging module is configured to traverse the members of the corresponding PG in the binding pool and judge whether the IP of the node where the member is located can be inquired in the data structure;
the adding module is configured to respond that the IP of the node where the member is located can be inquired in a data structure, and add the ID of the member and the PG as a group of values to the data structure of the value of the IP keyword;
and the selecting module is configured to acquire the number of values in each IP and select the member in the structure with the largest number of IP median values as the main part of each PG.
In a preferred embodiment of the invention, the selection module is further configured to:
adding the times of the member selection as the main in the data structure of each IP in response to the same number of the obtained values in each IP;
and selecting the member with the least number of additions as the main part of each PG.
In a preferred embodiment of the invention, the selection module is further configured to:
in response to the same value obtained by adding the times of the members as the main in the data structure of each IP, calculating the sum of the times of the OSD main on the node where each member is located;
and selecting the member with the least number of the primary times as the primary of the respective PG.
In a preferred embodiment of the present invention, the system further includes a deletion module, and the deletion module is configured to:
deleting the PG from the variable structure in response to setting the selected member as the master of the respective PG;
judging whether the variable structure is empty or not;
in response to the variable structure not being empty, traversing the remaining members of the PG in the variable structure and adding the times of the members as the main;
and selecting the member with the least number of additions as the main part of each PG.
In view of the above object, a third aspect of the embodiments of the present invention provides a computer device. Fig. 3 is a schematic diagram of an embodiment of a computer device provided by the present invention. As shown in fig. 3, an embodiment of the present invention includes the following means: at least one processor 21; and a memory 22, the memory 22 storing computer instructions 23 executable on the processor, the instructions when executed by the processor implementing the method of:
acquiring PG in a reference pool, acquiring corresponding PG in a binding pool according to the corresponding relation of PG, and putting the acquired PG in the reference pool and the corresponding PG in the binding pool into a variable structure;
traversing the members of the PG in the reference pool, acquiring the IP of the node where the member is located, taking the acquired IP as a keyword, and storing the ID of the member and the ID of the PG in a data structure as a group of values;
traversing the members of the corresponding PG in the binding pool, and judging whether the IP of the node where the member is located can be inquired in a data structure;
in response to the fact that the IP of the node where the member is located can be inquired in the data structure, the ID of the member and the PG are added to the data structure of the value of the IP key as a group of values;
and acquiring the number of values in each IP, and selecting the member in the structure with the maximum number of IP median values as the main of each PG.
In a preferred embodiment of the present invention, the method further comprises:
adding the times of the member selection as the main in the data structure of each IP in response to the same number of the obtained values in each IP;
and selecting the member with the least number of additions as the main part of each PG.
In a preferred embodiment of the present invention, the method further comprises:
in response to the same value obtained by adding the times of the members as the main in the data structure of each IP, calculating the sum of the times of the OSD main on the node where each member is located;
and selecting the member with the least number of the primary times as the primary of the respective PG.
In a preferred embodiment of the present invention, the method further comprises:
deleting the PG from the variable structure in response to setting the selected member as the master of the respective PG;
judging whether the variable structure is empty or not;
in response to the variable structure not being empty, traversing the remaining members of the PG in the variable structure and adding the times of the members as the main;
and selecting the member with the least number of additions as the main part of each PG.
In view of the above object, a fourth aspect of the embodiments of the present invention proposes a computer-readable storage medium. FIG. 4 is a schematic diagram illustrating an embodiment of a computer-readable storage medium provided by the present invention. As shown in fig. 4, the computer readable storage medium S31 stores a computer program S32 that when executed by a processor performs the method of:
acquiring PG in a reference pool, acquiring corresponding PG in a binding pool according to the corresponding relation of PG, and putting the acquired PG in the reference pool and the corresponding PG in the binding pool into a variable structure;
traversing the members of the PG in the reference pool, acquiring the IP of the node where the member is located, taking the acquired IP as a keyword, and storing the ID of the member and the ID of the PG in a data structure as a group of values;
traversing the members of the corresponding PG in the binding pool, and judging whether the IP of the node where the member is located can be inquired in a data structure;
in response to the fact that the IP of the node where the member is located can be inquired in the data structure, the ID of the member and the PG are added to the data structure of the value of the IP key as a group of values;
and acquiring the number of values in each IP, and selecting the member in the structure with the maximum number of IP median values as the main of each PG.
In a preferred embodiment of the present invention, the method further comprises:
adding the times of the member selection as the main in the data structure of each IP in response to the same number of the obtained values in each IP;
and selecting the member with the least number of additions as the main part of each PG.
In a preferred embodiment of the present invention, the method further comprises:
in response to the same value obtained by adding the times of the members as the main in the data structure of each IP, calculating the sum of the times of the OSD main on the node where each member is located;
and selecting the member with the least number of the primary times as the primary of the respective PG.
In a preferred embodiment of the present invention, the method further comprises:
deleting the PG from the variable structure in response to setting the selected member as the master of the respective PG;
judging whether the variable structure is empty or not;
in response to the variable structure not being empty, traversing the remaining members of the PG in the variable structure and adding the times of the members as the main;
and selecting the member with the least number of additions as the main part of each PG.
Furthermore, the methods disclosed according to embodiments of the present invention may also be implemented as a computer program executed by a processor, which may be stored in a computer-readable storage medium. Which when executed by a processor performs the above-described functions defined in the methods disclosed in embodiments of the invention.
Further, the above method steps and system elements may also be implemented using a controller and a computer readable storage medium for storing a computer program for causing the controller to implement the functions of the above steps or elements.
Those of skill would further appreciate that the various illustrative logical blocks, modules, circuits, and algorithm steps described in connection with the disclosure herein may be implemented as electronic hardware, computer software, or combinations of both. To clearly illustrate this interchangeability of hardware and software, various illustrative components, blocks, modules, circuits, and steps have been described above generally in terms of their functionality. Whether such functionality is implemented as software or hardware depends upon the particular application and design constraints imposed on the overall system. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the disclosed embodiments of the present invention.
In one or more exemplary designs, the functions may be implemented in hardware, software, firmware, or any combination thereof. If implemented in software, the functions may be stored on or transmitted over as one or more instructions or code on a computer-readable medium. Computer-readable media includes both computer storage media and communication media including any medium that facilitates transfer of a computer program from one place to another. A storage media may be any available media that can be accessed by a general purpose or special purpose computer. By way of example, and not limitation, such computer-readable media can comprise RAM, ROM, EEPROM, CD-ROM or other optical disk storage, magnetic disk storage or other magnetic storage devices, or any other medium that can be used to carry or store desired program code in the form of instructions or data structures and that can be accessed by a general-purpose or special-purpose computer, or a general-purpose or special-purpose processor. Also, any connection is properly termed a computer-readable medium. For example, if the software is transmitted from a website, server, or other remote source using a coaxial cable, fiber optic cable, twisted pair, Digital Subscriber Line (DSL), or wireless technologies such as infrared, radio, and microwave, then the coaxial cable, fiber optic cable, twisted pair, DSL, or wireless technologies such as infrared, radio, and microwave are included in the definition of medium. Disk and disc, as used herein, includes Compact Disc (CD), laser disc, optical disc, Digital Versatile Disc (DVD), floppy disk, blu-ray disc where disks usually reproduce data magnetically, while discs reproduce data optically with lasers. Combinations of the above should also be included within the scope of computer-readable media.
The foregoing is an exemplary embodiment of the present disclosure, but it should be noted that various changes and modifications could be made herein without departing from the scope of the present disclosure as defined by the appended claims. The functions, steps and/or actions of the method claims in accordance with the disclosed embodiments described herein need not be performed in any particular order. Furthermore, although elements of the disclosed embodiments of the invention may be described or claimed in the singular, the plural is contemplated unless limitation to the singular is explicitly stated.
It should be understood that, as used herein, the singular forms "a", "an" and "the" are intended to include the plural forms as well, unless the context clearly supports the exception. It should also be understood that "and/or" as used herein is meant to include any and all possible combinations of one or more of the associated listed items.
The numbers of the embodiments disclosed in the embodiments of the present invention are merely for description, and do not represent the merits of the embodiments.
It will be understood by those skilled in the art that all or part of the steps for implementing the above embodiments may be implemented by hardware, or may be implemented by a program instructing relevant hardware, and the program may be stored in a computer-readable storage medium, and the above-mentioned storage medium may be a read-only memory, a magnetic disk or an optical disk, etc.
Those of ordinary skill in the art will understand that: the discussion of any embodiment above is meant to be exemplary only, and is not intended to intimate that the scope of the disclosure, including the claims, of embodiments of the invention is limited to these examples; within the idea of an embodiment of the invention, also technical features in the above embodiment or in different embodiments may be combined and there are many other variations of the different aspects of the embodiments of the invention as described above, which are not provided in detail for the sake of brevity. Therefore, any omissions, modifications, substitutions, improvements, and the like that may be made without departing from the spirit and principles of the embodiments of the present invention are intended to be included within the scope of the embodiments of the present invention.