CN108694209A - Object-based distributed index method and client - Google Patents

Object-based distributed index method and client Download PDF

Info

Publication number
CN108694209A
CN108694209A CN201710233466.XA CN201710233466A CN108694209A CN 108694209 A CN108694209 A CN 108694209A CN 201710233466 A CN201710233466 A CN 201710233466A CN 108694209 A CN108694209 A CN 108694209A
Authority
CN
China
Prior art keywords
key
osd
node
pointer
interface
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN201710233466.XA
Other languages
Chinese (zh)
Other versions
CN108694209B (en
Inventor
胡斐然
李梁玉
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Huawei Cloud Computing Technologies Co Ltd
Original Assignee
Huawei Technologies Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Huawei Technologies Co Ltd filed Critical Huawei Technologies Co Ltd
Priority to CN201710233466.XA priority Critical patent/CN108694209B/en
Publication of CN108694209A publication Critical patent/CN108694209A/en
Application granted granted Critical
Publication of CN108694209B publication Critical patent/CN108694209B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Landscapes

  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)

Abstract

The embodiment of the present invention discloses a kind of object-based distributed index method and client, this method include:Client obtains the content and title of object to be stored;The title that client treats storage object is normalized to obtain the first normalization result;Client obtains the key of free time OSD in OSD clusters by interface, and the key for normalizing result, title, content and free time OSD by first by interface is stored in idle OSD, to generate new node;Client normalizes the front nodal point of result by being stored with the posterior nodal point of the second normalization result more than the first normalization result in interface polls distributed index and being stored with less than the third of the first normalization result normalization result;Client constructs pointer in new node, sets the pointer of new node to the key of posterior nodal point, sets the pointer of front nodal point to the key of new node.The key of object is broken up in distributed index by pointer, positioning object is indexed by Querying Distributed, the difficulty of deployment and maintenance can be reduced.

Description

Object-based distributed index method and client
Technical field
This application involves information technology field, more particularly to a kind of object-based distributed index method and client.
Background technology
Object storage technology is used widely at this stage, and in object storage technology, OSD is arranged in storage system (Object-based Storage Device, object storage device) cluster, OSD clusters include multiple OSD, every OSD settings There are fixed memory space, object to be stored in the OSD of OSD clusters, there are one key (key) for every OSD correspondences, and client passes through Key can find corresponding OSD, and from OSD from reading object.
In the storage system of the prior art, storage system also sets up MDS (Metadata Server, Metadata Service Device), MDS stores the relationship between the network address of key OSD corresponding with key, and OSD clusters are provided with interface, and client exists After obtaining key, the network address of the corresponding OSD of key is inquired to MDS by interface, so as to be accessed according to network address Object in OSD.
Conventionally, as the memory space of OSD is fixed, therefore it is that required space is more than in object to be stored When the larger object of the memory space of OSD, OSD cannot store the object, and in order to solve the technical problem, the prior art is by root The OSD of corresponding number is set according to the size of object, object is divided into multiple portions and is respectively stored in the OSD of corresponding number.
Therefore, the MDS for object need to be further arranged in the storage system of the prior art, for the MDS storages pair of object As the correspondence between title and the key of corresponding OSD, when client needs reading object, also need according to object oriented in needle Which to inquiring for object, it is stored in OSD to obtain the object, and write OSD reading objects from this respectively.
However, in the prior art, client was with both having needed to be inquired the network to obtain the corresponding OSD of key to MDS Location, and need to be inquired the key of multiple OSD to obtain object to the MDS for object, when for the failure of object When, client cannot access object, therefore can increase the difficulty of deployment and maintenance for the presence of the MDS of object.
Invention content
To solve problem of the prior art, a kind of object-based distributed index method of offer of the embodiment of the present invention and visitor Family end can be positioned pair due to being broken up the key of object in distributed index by pointer by Querying Distributed index As the difficulty of deployment and maintenance can be reduced.
In a first aspect, the embodiment of the present invention provides a kind of object-based distributed index method, this method is applied to visitor Family end, client are interacted by interface and OSD (Object-based Storage Device, object storage device) cluster, OSD clusters include multiple OSD, and client selects OSD as node to construct distributed index by interface, in distributed index Each node in, be stored with the title of itself and normalize result, from OSD clusters key (cluster) and pointer, return One change result is generated according to title, and pointer is stored with the key of the next node in distributed index, and this method includes:
Client obtains the content and title of object to be stored;
The title that client treats storage object is normalized to obtain the first normalization result;
Client obtains the key of free time OSD in OSD clusters by interface, and result, name are normalized by first by interface The key of title, content and free time OSD is stored in idle OSD, to generate new node;
Client is by being stored with the second normalization knot more than the first normalization result in interface polls distributed index The posterior nodal point of fruit and the front nodal point for being stored with the third normalization result that result is normalized less than the first normalization result;
Client constructs pointer by interface in new node, sets the pointer of new node to the key of posterior nodal point, and will before The pointer of node is set as the key of new node.
To sum up, in embodiments of the present invention, client by interface from OSD cluster application OSD, due to will be right by pointer The key of elephant breaks up in distributed index, can position object by Querying Distributed index, can reduce the difficulty of deployment and maintenance Degree.
In an implementation of the embodiment of the present invention, distributed index includes Ingress node, the pointer of Ingress node It is stored with the key that the node that result is minimum value is normalized in distributed index, client passes through interface polls distributed index In be stored with more than first normalization result second normalization result posterior nodal point and be stored with less than first normalization result The step of front nodal point of the third normalization result of normalization result, specifically includes:
Client is by interface since the pointer inquiry point stored according to each node in distributed index Ingress node The normalization of node storage in cloth index until inquiring in distributed index as a result, be stored with more than the first normalization knot The posterior nodal point of the second normalization result of fruit normalizes knot with the third for normalizing result less than the first normalization result is stored with The front nodal point of fruit.
In another realization method of the embodiment of the present invention, client constructs pointer by interface in new node, will be new The pointer of node is set as the key of posterior nodal point, and the step of setting the pointer of front nodal point to the key of new node specifically includes:
Client constructs pointer by interface in new node, sets the pointer of new node to the key of posterior nodal point;
Client constructs daily record by interface in front nodal point, and the key of new node is stored in daily record;
Client by interface by the pointer of front nodal point be set as new node storage key fail when, from front nodal point The key of log read new node, the key for the new node that the pointer of front nodal point is set as reading;
Client is by interface by daily record zero setting.
In another realization method of the embodiment of the present invention, client obtains free time OSD in OSD clusters by interface Key, the key for normalizing result, title, content and free time OSD by first by interface is stored in idle OSD, to produce Raw new node specifically includes:
Object to be stored is divided into pre- by client when the size for determining object to be stored is more than the memory space of OSD Fixed number purpose part obtains the key of the idle OSD of predetermined number in OSD clusters by interface, by each section of object to be stored It is respectively stored in the idle OSD of idle OSD and predetermined number;
Any OSD is chosen as new node from the idle OSD of predetermined number, and result, name are normalized by first by interface The key of the idle OSD of title, content and predetermined number is stored in new node.
In another realization method of the embodiment of the present invention, distributed index is skip list, and pointer includes multiple pointers, often One pointer is directed toward the different layers of skip list.
In another realization method of the embodiment of the present invention, this method further includes:
Client obtains the title of object to be deleted;
Client is normalized title to obtain the 4th normalization result;
Client is by being stored with the 4th node to be deleted for normalizing result in interface polls distributed index;
Client sets the pointer of the front nodal point of node to be deleted to by interface the posterior nodal point of node to be deleted key;
Client sets node to be deleted to idle OSD by interface notification OSD clusters.
In another realization method of the embodiment of the present invention, client is by interface by the front nodal point of node to be deleted Pointer is set as the step of key of the posterior nodal point of node to be deleted and specifically includes:
Client constructs daily record by interface in front nodal point, and the key that node to be deleted stores is stored in daily record;
Client is when the key for setting the pointer of front nodal point to by interface posterior nodal point fails, from the daily record of front nodal point The key for reading node to be deleted reads the deutomerite of the pointer storage of node to be deleted according to the key of node to be deleted by interface The key, the key for the posterior nodal point that the pointer of front nodal point is set as reading of point;
Client is by interface by daily record zero setting.
Second aspect, the embodiment of the present invention further provide for a kind of client, and client passes through interface and OSD collection flock-mates Mutually, OSD clusters include multiple OSD, and client selects OSD as node to construct distributed index by interface, in distribution In each node of index, it is stored with the title of itself and normalizes result, certainly in the key and pointer of OSD clusters, normalizing Change result to be generated according to title, pointer is stored with the key of the next node in distributed index, and client includes:
Object acquisition module to be stored, content and title for obtaining object to be stored;
Module is normalized, the title for treating storage object is normalized to obtain the first normalization result;
New node generation module, the key for obtaining free time OSD in OSD clusters by interface, is returned by interface by first One key for changing result, title, content and free time OSD is stored in idle OSD, to generate new node;
Enquiry module, for being returned more than the second of the first normalization result by being stored in interface polls distributed index The posterior nodal point of one change result and the front nodal point for being stored with the third normalization result that result is normalized less than the first normalization result;
Pointer modified module sets the pointer of new node to posterior nodal point for constructing pointer in new node by interface Key, and set the pointer of front nodal point to the key of new node.
In another realization method of the embodiment of the present invention, distributed index includes Ingress node, the finger of Ingress node Needle is stored with the key that the node that result is minimum value is normalized in distributed index, and client further includes entry record module, is entered Mouth logging modle is used for the key of record entry node,
Enquiry module, specifically for by interface since being stored according to each node in distributed index Ingress node The normalization of node storage in pointer inquiry linked list until inquiring in distributed index as a result, be stored with more than the first normalizing Change the posterior nodal point of the second normalization result of result and is stored with the third normalizing for normalizing result less than the first normalization result Change the front nodal point of result.
In another realization method of the embodiment of the present invention, pointer modified module is specifically used for:
Pointer is constructed in new node by interface, sets the pointer of new node to the key of posterior nodal point;
Daily record is constructed in front nodal point by interface, the key of new node is stored in daily record;
By interface when the key for setting the pointer of front nodal point to new node storage fails, read from the daily record of front nodal point The key for taking new node, the key for the new node that the pointer of front nodal point is set as reading;
By interface by daily record zero setting.
In another realization method of the embodiment of the present invention, new node generation module is specifically used for:
When the size for determining object to be stored is more than the memory space of OSD, object to be stored is divided into predetermined number Part, by interface obtain OSD clusters in predetermined number idle OSD key, each section of object to be stored is deposited respectively Storage is in the idle OSD of idle OSD and predetermined number;
Any OSD is chosen as new node from the idle OSD of predetermined number, and result, name are normalized by first by interface The key of the idle OSD of title, content and predetermined number is stored in new node.
In another realization method of the embodiment of the present invention, distributed index is skip list, and pointer includes multiple pointers, often One pointer is directed toward the different layers of skip list.
In another realization method of the embodiment of the present invention, client further includes notification module,
Name acquiring module is additionally operable to obtain the title of object to be deleted;
Module is normalized, is additionally operable to that title is normalized and returns four change results to obtain first;
Enquiry module is additionally operable to the section to be deleted by being stored with the 4th normalization result in interface polls distributed index Point;
Pointer modified module is additionally operable to set the pointer of the front nodal point of node to be deleted to the posterior nodal point of node to be deleted Key;
Notification module sets node to be deleted to idle OSD for passing through interface notification OSD clusters.
In another realization method of the embodiment of the present invention, pointer modified module is specifically used for:
Daily record is constructed in front nodal point by interface, the key that node to be deleted stores is stored in daily record;
By interface in the key failures for setting the pointer of front nodal point to posterior nodal point, waited for from the log read of front nodal point The key of deletion of node reads the posterior nodal point of the pointer storage of node to be deleted according to the key of node to be deleted by interface Key, the key for the posterior nodal point that the pointer of front nodal point is set as reading;
By interface by daily record zero setting.
The third aspect, a kind of client of offer of the embodiment of the present invention, including processor, memory and bus, processor, Memory is connect with bus respectively, and memory has program stored therein instruction, wherein:
Processor executes the first program instruction to execute step:Obtain the content and title of object to be stored;
Processor executes the second program instruction to execute step:The title for treating storage object is normalized to obtain Take the first normalization result;
Processor executes third program instruction to execute step:The key of free time OSD in OSD clusters is obtained by interface, is led to It crosses interface and the key of the first normalization result, title, content and free time OSD is stored in idle OSD, to generate new save Point;
Processor executes fourth program instructions to execute step:By being stored in interface polls distributed index more than The posterior nodal point of second normalization result of one normalization result normalizes knot with the third for normalizing result less than first is stored with The front nodal point of fruit;
Processor executes the 5th program instruction to execute step:Pointer is constructed in new node by interface, by new node Pointer is set as the key of posterior nodal point, and sets the pointer of front nodal point to the key of new node.
Description of the drawings
In order to illustrate the technical solution of the embodiments of the present invention more clearly, will make below to required in the embodiment of the present invention Attached drawing is briefly described, it should be apparent that, drawings described below is only some embodiments of the present invention, for For those of ordinary skill in the art, without creative efforts, other are can also be obtained according to these attached drawings Attached drawing.
Fig. 1 is the structural schematic diagram of storage system according to the ... of the embodiment of the present invention;
Fig. 2 is the flow chart of object-based distributed index method according to the ... of the embodiment of the present invention;
Fig. 3 is the schematic diagram of distributed index according to the ... of the embodiment of the present invention;
Fig. 4 is another schematic diagram of distributed index according to the ... of the embodiment of the present invention;
Fig. 5 is another schematic diagram of storage system according to the ... of the embodiment of the present invention;
Fig. 6 is the flow chart of object-based distributed index method according to the ... of the embodiment of the present invention;
Fig. 7 is another schematic diagram of distributed index according to the ... of the embodiment of the present invention;
Fig. 8 is another schematic diagram of distributed index according to the ... of the embodiment of the present invention;
Fig. 9 is the apparatus structure schematic diagram of client according to the ... of the embodiment of the present invention;
Figure 10 is another apparatus structure schematic diagram of client according to the ... of the embodiment of the present invention.
Specific implementation mode
It is the structural schematic diagram of storage system according to the ... of the embodiment of the present invention referring firstly to Fig. 1, Fig. 1, as shown in Figure 1, The storage system of the embodiment of the present invention includes client and OSD clusters.
For example, the quantity of client can be multiple, as shown in Figure 1, client includes client 101,102 ... N, in other examples, the quantity of client can also be 1, and the embodiment of the present invention is not construed as limiting this.
OSD clusters 20 include interface (not being painted) and multiple OSD, each OSD both corresponds to one of physical disk admittedly Determine the memory space of size, the memory space of physical disk is divided into multiple OSD by OSD clusters 20, to store data.
OSD clusters 20 include MDS, and the every OSD of OSD are corresponding to be assigned a key, MDS have recorded key and memory space it Between correspondence the object that the corresponding OSD of key are located in particular network address for example can be inquired in MDS by key Manage the particular memory space of the specific physical disk of host carry.
Client 101,102 ... n access OSD clusters 20 by distributed index 201, and the setting of distributed index 201 exists In the OSD for asking OSD clusters 20.
It is worth noting that, in embodiments of the present invention, OSD clusters 20 provide interface, and client only need to be using key as defeated Enter parameter access interface, you can the corresponding OSD of key are accessed from interface.
Specifically, client is using key and reading order as input parameter access interface, you can directly read from interface The data stored in OSD;Client can will be waited for using key, writing commands and object to be stored as input parameter access interface Storage object is written into the corresponding memory spaces of key;Further, client can apply for the free time by interface to OSD clusters 20 OSD, for example, client to interface input application OSD orders, you can from interface obtain free time OSD key;And Client can by interface notification OSD clusters 20 will store object to be stored OSD recycle, for example, client to Interface inputs after key and revocation OSD orders, and OSD clusters 20 recycle OSD, and the key is inputted to interface to try in subsequent client When figure OSD corresponding to key accesses, the access failure notification of interface transmission will be received.
Therefore, OSD clusters 20 shield client the internal course of work, and client only need to be by interface to OSD collection Group 20 applies for OSD, obtains key in interface, and access to OSD in interface according to key.
Continuing with referring to Fig. 1, in fig. 1, it is assumed that client 101 from OSD clusters 20 apply OSD2, OSD9, OSD3, OSD5, OSD7, OSD1 and OSD8, wherein above-mentioned every OSD stores corresponding data, distributed index 201 include OSD2, OSD9, OSD3, OSD5, OSD7, OSD1 and OSD8, in above-mentioned every OSD, be stored with the title of itself and normalization as a result, From the key and pointer in OSD clusters 20, normalization result is generated according to title, and pointer is stored in distributed index 201 In next node key, for example, OSD2 is stored with itself title, the normalization result that is generated according to title and refers to Needle, the wherein pointer of OSD2 store the key of OSD9, and therefore, client 101, can after accessing OSD2 according to the key of OSD2 OSD9 is further accessed by interface according to the key of the pointer of OSD2 storage, and is accessed by the pointer of OSD9 storages OSD3 ..., client 101 in OSD by being arranged pointer so that OSD forms distributed index 201, therefore client 101 The key of the node of the entrance as distributed index 201 need to be only recorded, for example such as the key of OSD2, passes through distributed index 201 can inquire all nodes in institute's distributed index 201 from 201 entrance of distributed index.
In embodiments of the present invention, client 101 applies for OSD, and the OSD that will apply by interface from OSD clusters 20 The form of index 201 is managed in a distributed manner, so as to avoid two layers of MDS of setting, can reduce the complexity of deployment and O&M Degree.
In order to further clearly illustrate, refer to Fig. 2 below, Fig. 2 is object distribution formula rope according to the ... of the embodiment of the present invention Draw the flow chart of 201 methods, includes the following steps as shown in Fig. 2, object distribution formula indexes 201 methods:
Step S301:Client 101 obtains the content and title of object to be stored.
Wherein, above-mentioned object includes file and uncorrected data.
For example, client 101 can be by providing webpage so that user accesses, and it is to be stored right that user uploads in webpage As so that client 101 can obtain the content and title of object to be stored.
In this step, it is assumed that entitled " Xiao Ming " of object to be stored.
Step S302:The title that client 101 treats storage object is normalized to obtain the first normalization knot Fruit.
In embodiments of the present invention, normalized refers to converting the title of object to be stored to the normalizing that can compare Change result.
For example, client 101 can calculate the corresponding ASCII of title (American Standard Code for Information Interchange, ASCII) code, title is scaled to the normalization that can compare As a result, to realize normalized.
In other example, client 101 can also to title carry out Hash operation with obtain normalization as a result, from And realize normalized.
Assuming that after " Xiao Ming " is normalized, the first normalization result of acquisition is 10.
Step S303:Client 101 obtains the key of free time OSD in OSD clusters 20 by interface, by interface by first The key for normalizing result, title, content and free time OSD is stored in idle OSD, to generate new node.
In this step, client 101 can determine whether object to be stored size whether be more than OSD memory space, and When determining memory space of the size of object to be stored more than OSD, object to be stored is divided into the part of predetermined number, is passed through Interface obtains the key of the idle OSD of predetermined number in OSD clusters 20, and each section of object to be stored is respectively stored in the free time In the idle OSD of OSD and predetermined number;Any OSD is chosen as new node from the idle OSD of predetermined number, passes through interface The key for the idle OSD that first normalizes result, title, content and predetermined number is stored in new node.
Therefore, in the above manner, larger object can be stored in OSD by the embodiment of the present invention, to realize dilatation.
In other examples, storage of the size less than or equal to OSD that object to be stored is determined in client 101 is empty Between when, directly object to be stored is stored in acquired idle OSD without by Object Segmentation to be stored, only needing.
Step S304:Client 101 in interface polls distributed index 201 by being stored with more than the first normalization result Second normalization result posterior nodal point and be stored with less than first normalization result normalize result third normalize result Front nodal point.
For details, reference can be made to the schematic diagrames that Fig. 3, Fig. 3 are distributed index 201 according to the ... of the embodiment of the present invention, shown in Fig. 3 Distributed index 201 is single distributed index 201, in the distributed index 201, including 7 nodes, respectively OSD2, OSD9, OSD3, OSD5, OSD7, OSD1 and OSD8.
OSD2 is the Ingress node of distributed index 201, is stored with pointer, normalization result and other data, pointer It is stored with the key of OSD9, normalization result is infinite to bear,
OSD8 is the end node of distributed index 201, is stored with pointer, normalization result and other data, pointer It is stored with the key of OSD9, normalization result is just infinite.
Other nodes are stored with pointer, normalization result and other data respectively, and the key of pointer storage is as shown in Figure 3 For the key of next node, wherein other data include contents of object, object oriented, object length etc..
In this step, client 101 by interface since Ingress node according to each node in distributed index 201 The normalization of node storage in the pointer Querying Distributed index 201 of storage is as a result, until inquire in distributed index 201 It is stored with the posterior nodal point of the second normalization result more than the first normalization result and is stored with and return less than the first normalization result One changes the front nodal point of the third normalization result of result.
Specifically, the access entrance node OSD2 first of client 101, accesses OSD9 according to the pointer of OSD2, knows The normalization result of OSD9 is 5, and since 10 are more than 5, client 101 further accesses the OSD5 that the pointer of OSD9 is directed toward, Know that the normalization result of OSD5 is 9, since 10 are more than 9, client 101 further accesses the pointer direction of OSD5 OSD11, since 10 are less than 53, client 101, which inquires, to be stored in distributed index 201 more than the first normalization result (10) second normalizes the posterior nodal point OSD11 of result (53) and is stored with less than the first normalization result normalization result (10) Third normalization result (9) front nodal point OSD5.
Step S305:Client 101 constructs pointer by interface in new node, sets the pointer of new node to posterior nodal point Key, and set the pointer of front nodal point to the key of new node.
For example, reference can be made to Fig. 4, Fig. 4 are another schematic diagrames of distributed index 201 according to the ... of the embodiment of the present invention, Client 101 constructs pointer in OSD9 so that the key of the pointer storage OSD7 of OSD9 changes the pointer of OSD7 so that OSD7 Pointer storage OSD11 key so that OSD7 can be plugged into single distributed index 201.
It is worth noting that, in this step, prosthomere is changed due to generating new node and this step in above-mentioned steps S303 The pointer of point is two independent steps, can not ensure atomicity, if this step unsuccessfully can to the pointer modified of front nodal point OSD5 Cause client 101 to lose the record of new node OSD7, therefore in this step, needs further to construct day in front nodal point OSD5 The key of new node OSD7 is stored in daily record by will, is arranged by the pointer of front nodal point OSD5 by interface in client 101 When failing for the key of new node OSD7, client 101 can be from the key of the log read new node of front nodal point OSD7, by front nodal point The pointer of OSD5 is set as the key of the new node OSD7 read, and after being arranged successfully, client 101 is by interface by day Will zero setting.
Therefore, after by above step, client 101 forms new distributed index 201 in OSD clusters 20, tool Body is as shown in figure 5, Fig. 5 is another schematic diagram of storage system according to the ... of the embodiment of the present invention, in Figure 5, OSD2, OSD9, OSD3, OSD5, OSD11, OSD7, OSD1 and OSD8 form distributed index 201, wherein OSD11 is inserted by the above method In distributed index 201 shown in FIG. 1.
Therefore, in embodiments of the present invention, client 101 applies for OSD by interface from OSD clusters 20, and application is arrived OSD be inserted into distributed index 201, so as to avoid setting two layers of MDS, the complexity of deployment and O&M can be reduced.
The embodiment of the deletion of node in distributed index 201 will be introduced below, refer to Fig. 6, Fig. 6 is basis The object distribution formula of the embodiment of the present invention indexes the flow chart of 201 methods, as shown in fig. 6, object distribution formula indexes 201 method packets Include following steps:
Step S401:Client 101 obtains the title of object to be deleted.
For example, client 101 can be by providing webpage so that user accesses, and user inputs object to be deleted in webpage Title so that client 101 can obtain the title of object to be deleted.
In this step, it is assumed that entitled " Xiao Ming " of object to be deleted.
Step S402:Client 101 is normalized title to obtain the 4th normalization result.
In embodiments of the present invention, normalized refers to converting the title of object to be stored to the normalizing that can compare Change result.
For example, client 101 can calculate the corresponding ASCII of title (American Standard Code for Information Interchange, ASCII) code, title is scaled to the normalization that can compare As a result, to realize normalized.
In other example, client 101 can also to title carry out Hash operation with obtain normalization as a result, from And realize normalized.
In other example, client 101 can also carry out title dictionary sequence to realize normalized.
Assuming that after " Xiao Ming " is normalized, the 4th normalization result of acquisition is 10.
Step S403:Client 101 is by being stored with waiting for for the 4th normalization result in interface polls distributed index 201 Deletion of node.
Specifically, distributed index 201 includes Ingress node, Ingress node is stored with negative infinite normalization as a result, visitor Family end 101 is by interface since the pointer Querying Distributed stored according to each node in distributed index 201 Ingress node The normalization of node storage in index 201 until inquiring as a result, be stored with the first normalization result in distributed index 201 Node.
As described above, OSD2 is Ingress node, and client 101 accesses OSD2 first, knows the normalization knot of OSD2 Fruit is negative infinite, and OSD9 is accessed according to the pointer of OSD2, knows that the normalization result of OSD9 is 5, since 10 more than 5, visitor Family end 101 further accesses the OSD5 that the pointer of OSD9 is directed toward, and knows that the normalization result of OSD5 is 9, since 10 more than 9, because This client 101 further accesses the OSD7 that the pointer of OSD5 is directed toward, and the normalization of OSD7 is the result is that 10, therefore client 101 Using node OSD7 as node to be deleted.
Step S404:Client 101 sets the pointer of the front nodal point of node to be deleted to node to be deleted by interface Posterior nodal point key.
In this step, client 101 sets the pointer of the front nodal point OSD5 of OSD7 to by interface the key of OSD1.
Step S405:Client 101 sets node to be deleted to idle OSD by interface notification OSD clusters 20.
In this step, client 101 notifies OSD clusters 20 to recycle node to be deleted in the above manner, can effectively return OSD resources are received, idle OSD will not be caused occupied.
Further, in this step, client 101 can inquire whether node to be deleted also stores just like described in step S303 For store larger object OSD key, if so, client 101 will further be used by interface notification OSD clusters 20 It is set as idle OSD in the OSD of storage larger object.
Significantly, since this step is two independent with the previous step step S404 pointers for changing front nodal point Step can not ensure atomicity, and when even step S404 executes failure, client 101 can lose the key of node to be deleted, from And step S404 and step S405 is caused to execute failure.
For this purpose, in step s 404, client 101 constructs daily record by interface in front nodal point, and node to be deleted is stored Key be stored in daily record, and in client 101 when the key for setting the pointer of front nodal point to by interface posterior nodal point fails, From the key of the log read of front nodal point node to be deleted, new node is read by interface according to the key of node to be deleted The key, the key for the posterior nodal point that the pointer of front nodal point is set as reading, before ensureing modification of the posterior nodal point of pointer storage When the pointer failure of node, the key of node to be deleted will not be lost, it is notable that the pointer success of node before a modification Afterwards, the daily record of front nodal point is not necessarily to record the key of node to be deleted, and client 101 can be by interface by daily record zero setting at this time.
To sum up, in embodiments of the present invention, client 101 applies for OSD by interface from OSD clusters 20, and application is arrived OSD from distributed index 201 delete, so as to avoid setting two layers of MDS, the complexity of deployment and O&M can be reduced.
Distributed index 201 described above is single distributed index 201, it is notable that in the optional of the present invention In embodiment, distributed index 201 can also be skip list and binary tree, and wherein skip list is particularly suitable in the number of objects stored Larger situation.
Specific to be referred in combination with Fig. 7 and Fig. 8, wherein Fig. 7 is distributed index 201 according to the ... of the embodiment of the present invention Another schematic diagram, distributed index 201 shown in Fig. 7 are skip list, and pointer includes multiple pointers, and each pointer is directed toward the difference of skip list Layer.
For example, OSD2 stores the 1st layer of pointer, the 2nd layer of pointer, the 3rd layer of pointer and the 4th layer of pointer, each finger Needle stores the corresponding node of each layer, when finding OSD7 (normalization result 10) in the skip list that client 101 wants shown in Fig. 7, First search the node of the same layer of the top directions of Ingress node OSD2, the i.e. OSD8 that the 4th layer of pointer is directed toward, due to returning for OSD8 One change result is just infinite, therefore searches time node OSD1 of high-rise (the 3rd layer) same layer being directed toward, and normalization result is 78 Inconsistent with 10, therefore search the OSD3 that the 2nd layer of pointer is directed toward, the normalization result of OSD3 is 5, is less than 10, in being to look for The OSD1 of the 2nd layer of direction of OSD3 is more than 10 since the normalization result of OSD1 is 53, therefore searches the 1st layer of direction of OSD3 OSD5 further search for the OSD7 of the 1st layer of OSD5 direction when the normalization result 9 for inquiring OSD5 is less than 10, due to The normalization of OSD7 confirms that OSD7 is the node to be inquired the result is that 10.
The number of plies of pointer is more, then search speed is faster, is according to the ... of the embodiment of the present invention point for details, reference can be made to Fig. 8, Fig. 8 Another schematic diagram of cloth index 201, unlike Fig. 7, in fig. 8, OSD7 stores 4 layers of pointer and therefore works as client 101 when wanting to find OSD7 (normalization result 10) in skip list shown in Fig. 8, first searches the top directions of Ingress node OSD2 The OSD7 that the node of same layer, i.e. the 4th layer of pointer are directed toward confirms that OSD7 is institute since the normalization result of OSD87 is 10 The node to be inquired.
It is worth noting that, if each node is both provided with multilayer pointer, the lookup of distributed index 201 can be seriously affected Speed, therefore the number of plies of each node need to be limited, existing " throwing coin " algorithm is specifically can refer to determine the layer of each node Number for example for OSD2, determines whether to increase the 2nd layer of pointer by " throw coin " algorithm, if throw coin is Front then increases the 2nd layer of pointer, if it is reverse side to throw coin, only retains the 1st layer of pointer, and increase the 2nd layer of pointer it Afterwards, further by throw coin come determine whether increase the 3rd layer of pointer ..., when being inserted into new node, to upper type at Reason, so that multilayer pointer is evenly distributed, does not interfere with 201 search speed of distributed index.
Further, client 101 can also cache the node inquired, when the normalization result to be inquired In the presence of in the buffer, without being inquired to distributed index 201, caching is locally being read.
Frequent node is accessed in 101 inner buffer of each client oneself, to naturally occur multiple pairs of thermal center point Originally hot spot is shared, simultaneously because service node is directly accessed from inner buffer when accessing thermal center point, also reduces the pressure to OSD Power.
For example, caching using LRU (Least recently used, least recently used) mode tissue, due to Node is globally shared, then the problem of caching can be related to consistency, so this caching can be used for making in positioning node With after having found interdependent node in the buffer, needing the content for directly accessing storages of the OSD to this node again to confirm, Key Inconsistent or node is there is no caching is so deleted, if the inconsistent so update caching of other content.Also, caching can only Ordinary node is cached, the node in addition deletion of node affairs cannot cache, in order to avoid more complicated consistency is caused to be asked Topic.
For example, the present embodiments relate to data change in OSD can be used CAS (Compare and swap, than Compared with exchange) operation, usually, CAS operation require be passed to old value, such as:
bool compare_and_set(osd_key_t key,void*old_value,void*new_value);
Wherein key is metadata, indicates object's position in OSD, such as the position offset of write-in, data length is written Data in up till now OSD objects occur for length etc., old_value, old value, i.e. write-in;New_value, data to be written.
The present invention program does following improvement to CAS operation, using but be not limited to checksum (check value), version number etc. and replace For old value:
1, old value is substituted using checksum values, for example uses HASH32 (32 algorithm of Hash) or MD5 (Message- Digest Algorithm 5, Message-Digest Algorithm 5) etc., when change, first calculates the checksum values of Current Content, if with It is incoming identical, it is modified.
2, old value is substituted using version identifier, i.e., increases version number's mark in the data structure to be changed, every time It changes and successfully then updates version identifier (such as one from the normalization increased as a result, changing successfully+1 every time), when change is passed to old Version identifier, Current Content version identifier is modified when consistent with incoming.
In the above manner, the data volume transmitted needed for can effectively reducing.
The embodiment of the present invention further provides for a kind of client 101, and client 101 is interacted by interface with OSD clusters 20, OSD clusters 20 include multiple OSD, and client 101 selects OSD as node to construct distributed index 201 by interface, are being divided Cloth index 201 each node in, be stored with the title of itself and normalize result, from OSD clusters 20 key and Pointer, normalization result are generated according to title, and pointer is stored with the key of the next node in distributed index 201, specifically asks It is the apparatus structure schematic diagram of client according to the ... of the embodiment of the present invention referring to Fig. 9, Fig. 9, as shown in figure 9, client 101 is wrapped It includes:
Object acquisition module 501 to be stored, content and title for obtaining object to be stored;
Module 502 is normalized, the title for treating storage object is normalized to obtain the first normalization knot Fruit;
New node generation module 503, the key for obtaining free time OSD in OSD clusters 20 by interface will by interface First key for normalizing result, title, content and free time OSD is stored in idle OSD, to generate new node;
Enquiry module 504, for by being stored in interface polls distributed index 201 more than the first normalization result The posterior nodal point of second normalization result normalizes result with the third for normalizing result less than the first normalization result is stored with Front nodal point;
Pointer modified module 505 sets the pointer of new node to deutomerite for constructing pointer in new node by interface The key of point, and set the pointer of front nodal point to the key of new node.
Optionally, distributed index 201 includes Ingress node, and Ingress node is stored with negative infinite normalization as a result, client End 101 further includes entry record module, and entry record module is used for the key of record entry node,
Enquiry module 504, specifically for pass through interface since Ingress node according to each node in distributed index 201 The normalization of node storage in the pointer Querying Distributed index 201 of storage is as a result, until inquire in distributed index 201 It is stored with the posterior nodal point of the second normalization result more than the first normalization result and is stored with and return less than the first normalization result One changes the front nodal point of the third normalization result of result.
Optionally, pointer modified module 505, is specifically used for:
Pointer is constructed in new node by interface, sets the pointer of new node to the key of posterior nodal point;By interface preceding The key of new node is stored in daily record by joint structure daily record;It is setting the pointer of front nodal point to new node storage by interface Key failure when, from the key of the log read new node of front nodal point, the new node that the pointer of front nodal point is set as reading Key;By interface by daily record zero setting.
Optionally, new node generation module 503, is specifically used for:
When the size for determining object to be stored is more than the memory space of OSD, object to be stored is divided into predetermined number Part, the key of the idle OSD of predetermined number in OSD clusters 20 is obtained by interface, each section of object to be stored is distinguished It is stored in the idle OSD of idle OSD and predetermined number;Any OSD is chosen as new section from the idle OSD of predetermined number The key for the idle OSD that first normalizes result, title, content and predetermined number is stored in new node by point by interface.
Optionally, distributed index 201 is skip list, and pointer includes multiple pointers, and each pointer is directed toward the different layers of skip list.
Optionally, the client 101 of the embodiment of the present invention further comprises notification module 505, wherein:
Name acquiring module 501 is additionally operable to obtain the title of object to be deleted;
Module 502 is normalized, for title being normalized to obtain the first normalization result;
Enquiry module 503, for waiting deleting by being stored with the first normalization result in interface polls distributed index 201 Except node;
Pointer modified module 504, the deutomerite for setting the pointer of the front nodal point of node to be deleted to node to be deleted The key of point;
Notification module 505 sets node to be deleted to idle OSD for passing through interface notification OSD clusters 20.
Optionally, distributed index 201 includes Ingress node, and the pointer of Ingress node is stored in distributed index 201 The node that result is minimum value is normalized, client 101 further includes entry record module, and entry record module is used for record entry The key of node, enquiry module 603 are specifically used for:
By interface since the pointer Querying Distributed stored according to each node in distributed index 201 Ingress node The normalization of node storage in index 201 until inquiring as a result, be stored with the first normalization result in distributed index 201 Node.
Optionally, pointer modified module 604 is specifically used for:
Daily record is constructed in front nodal point by interface, the key that new node stores is stored in daily record;
It is new from the log read of front nodal point by interface in the key failures for setting the pointer of front nodal point to posterior nodal point The key of node reads the key of the posterior nodal point of the pointer storage of new node according to the key of new node, by front nodal point by interface Pointer is set as the key of the posterior nodal point read.
Optionally, pointer modified module 604 is additionally operable to daily record zero setting through interface.
Optionally, distributed index 201 is skip list, and pointer includes multiple pointers, and each pointer is directed toward the different layers of skip list.
Fig. 9 is referred to below, and Fig. 9 is another apparatus structure schematic diagram of client according to the ... of the embodiment of the present invention, such as Fig. 9 It is shown, client 101 include processor 701, memory 702 and bus 703, processor 701, memory 702 respectively with always Line 703 connects, and memory 702 has program stored therein instruction, and processor 701 is executed program instructions to complete above example introduction The function of client 101, wherein:
Processor 701 executes the first program instruction to execute step S301:Obtain the content and title of object to be stored;
Processor 701 executes the second program instruction to execute step S302:The title for treating storage object is normalized Processing is to obtain the first normalization result;
Processor 701 executes third program instruction to execute step S303:It is obtained by interface idle in OSD clusters 20 The key of OSD, the key for normalizing result, title, content and free time OSD by first by interface are stored in idle OSD, from And generate new node;
Processor 701 executes fourth program instructions to execute step S304:By being deposited in interface polls distributed index 201 It contains the posterior nodal point of the second normalization result more than the first normalization result and is stored with less than the first normalization result normalizing Change the front nodal point of the third normalization result of result;
Processor 701 executes the 5th program instruction to execute step S305:Pointer is constructed in new node by interface, it will be new The pointer of node is set as the key of posterior nodal point, and sets the pointer of front nodal point to the key of new node.
Optionally, distributed index 201 includes Ingress node, and the pointer of Ingress node storage is stored with distributed index The node that result is minimum value, the key of 702 record entry node of memory are normalized in 201, processor 701 executes quadruple pass Sequence instruction with by interface since the pointer Querying Distributed stored according to each node in distributed index 201 Ingress node The normalization of node storage in index 201 until inquiring in distributed index 201 as a result, be stored with more than the first normalization As a result the posterior nodal point of the second normalization result normalizes the third normalization of result with being stored with less than the first normalization result As a result front nodal point.
Optionally, processor 701 executes the 5th program instruction to construct pointer in new node by interface, by new node Pointer is set as the key of posterior nodal point;Daily record is constructed in front nodal point by interface, the key of new node is stored in daily record;By connecing Mouthful when the key for setting the pointer of front nodal point to new node storage fails, from the key of the log read new node of front nodal point, The key for the new node that the pointer of front nodal point is set as reading;By interface by daily record zero setting.
Optionally, processor 701 executes third program instruction with the depositing more than OSD in the size for determining object to be stored When storing up space, object to be stored is divided into the part of predetermined number, the free time of predetermined number in OSD clusters 20 is obtained by interface Each section of object to be stored is respectively stored in the idle OSD of idle OSD and predetermined number by the key of OSD;From predetermined The idle OSD of number chooses any OSD as new node, normalizes result, title, content by first by interface and makes a reservation for The key of the idle OSD of number is stored in new node.
Optionally, distributed index 201 is skip list, and pointer includes multiple pointers, and each pointer is directed toward the different layers of skip list.
Another apparatus structure schematic diagram that 10, Figure 10 is client according to the ... of the embodiment of the present invention is referred to below, is such as schemed Shown in 10, client 101 include processor 801, memory 802 and bus 803, processor 801, memory 802 respectively with Bus 803 connects, and memory 802 has program stored therein instruction, and processor 801 is executed program instructions to complete above example introduction Client 101 function, wherein:
Processor 801 executes the first program instruction to execute step S401:Obtain the title of object to be deleted;
Processor 801 executes the second program instruction to execute step S402:Title is normalized to obtain Four normalization results;
Processor 801 executes third program instruction to execute step S403:By being deposited in interface polls distributed index 201 Contain the node to be deleted of the first normalization result;
Processor 801 executes fourth program instructions to execute step S404:By interface by the front nodal point of node to be deleted Pointer be set as node to be deleted posterior nodal point key;
Processor 801 executes the 5th program instruction to execute step S405:It will be to be deleted by interface notification OSD clusters 20 Node is set as idle OSD.
Optionally, distributed index 201 includes Ingress node, and Ingress node is stored with negative infinite normalization as a result, storage Device 802 is additionally operable to the key of record entry node, and processor 801 executes third program instruction to be opened from Ingress node by interface The normalization begun according to the node storage in the pointer Querying Distributed index 201 of each node storage in distributed index 201 As a result, until inquiring the node for being stored with the first normalization result in distributed index 201.
Optionally, processor 801 executes fourth program instructions to construct daily record in front nodal point by interface, and new node is deposited The key of storage is stored in daily record;By interface in the key failures for setting the pointer of front nodal point to posterior nodal point, from front nodal point The key of log read new node reads the key of the posterior nodal point of the pointer storage of new node according to the key of new node by interface, The key for the posterior nodal point that the pointer of front nodal point is set as reading.
Optionally, processor 801 execute fourth program instructions with by interface by daily record zero setting.
Optionally, distributed index 201 is skip list, and pointer includes multiple pointers, and each pointer is directed toward the different layers of skip list.
To sum up, in embodiments of the present invention, client 101 applies for OSD by interface from OSD clusters 20, and application is arrived OSD index 201 form in a distributed manner and be managed, so as to avoid two layers of MDS of setting, deployment and O&M can be reduced Complexity.
It should be noted that any device embodiment described above is all only schematical, wherein described as separation The unit of part description may or may not be physically separated, the component shown as unit can be or It can not be physical unit, you can be located at a place, or may be distributed over multiple network units.It can be according to reality Border needs to select some or all of process therein to achieve the purpose of the solution of this embodiment.In addition, provided by the invention In device embodiment attached drawing, the connection relation between process indicates there is communication connection between them, specifically can be implemented as one Item or a plurality of communication bus or signal wire.Those of ordinary skill in the art are without creative efforts, you can with Understand and implements.
Through the above description of the embodiments, it is apparent to those skilled in the art that the present invention can borrow Help software that the mode of required common hardware is added to realize, naturally it is also possible to by specialized hardware include application-specific integrated circuit, specially It is realized with CPU, private memory, special components and parts etc..Under normal circumstances, all functions of being completed by computer program can It is easily realized with corresponding hardware, moreover, for realizing that the particular hardware structure of same function can also be a variety of more Sample, such as analog circuit, digital circuit or special circuit etc..But it is more for the purpose of the present invention in the case of software program it is real It is now more preferably embodiment.Based on this understanding, technical scheme of the present invention substantially in other words makes the prior art The part of contribution can be expressed in the form of software products, which is stored in the storage medium that can be read In, such as the floppy disk of computer, USB flash disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory Device (RAM, Random Access Memory), magnetic disc or CD etc., including some instructions are with so that a computer is set Standby (can be personal computer, server or the network equipment etc.) executes the method described in each embodiment of the present invention.
It is apparent to those skilled in the art that the specific work of the system of foregoing description, device or unit Make process, can refer to corresponding processes in the foregoing method embodiment, details are not described herein.
The above description is merely a specific embodiment, but scope of protection of the present invention is not limited thereto, any Those familiar with the art in the technical scope disclosed by the present invention, can easily think of the change or the replacement, and should all contain Lid is within protection scope of the present invention.Therefore, protection scope of the present invention should be based on the protection scope of the described claims.

Claims (15)

1. a kind of object-based distributed index method, which is characterized in that the method is applied to client, the client It is interacted by interface and OSD (Object-based Storage Device, object storage device) cluster, the OSD clusters packet Multiple OSD are included, the client selects OSD as node to construct distributed index by the interface, in the distribution In each node of index, it is stored with the title of itself and normalizes result, (collects from the key in the OSD clusters Group) and pointer, the normalization result generated according to the title, the pointer is stored in the distributed index The key of next node, the method includes:
The client obtains the content and title of object to be stored;
The client is normalized the title of the object to be stored to obtain the first normalization result;
The client obtains the key of free time OSD in the OSD clusters by the interface, by the interface by described One key for normalizing result, the title, the content and the free time OSD is stored in the OSD of the free time, to produce Raw new node;
The client in distributed index described in the interface polls by being stored with more than the first normalization result The posterior nodal point of second normalization result normalizes knot with the third for normalizing result less than the first normalization result is stored with The front nodal point of fruit;
The client constructs pointer by the interface in the new node, after the pointer of the new node is set as described The key of node, and set the pointer of the front nodal point to the key of the new node.
2. according to the method described in claim 1, it is characterized in that, the distributed index includes Ingress node, the entrance The pointer of node is stored with the key that the node that result is minimum value is normalized in the distributed index, and the client passes through After being stored with the second normalization result more than the first normalization result in distributed index described in the interface polls Node has with the step of third for normalizing result less than the first normalization result normalizes the front nodal point of result is stored with Body includes:
Since the client by the interface stored according to each node in the distributed index the Ingress node The normalization of node storage inquired in the distributed index of pointer as a result, until inquiring and being deposited in the distributed index It contains the posterior nodal point for the second normalization result for normalizing result more than described first and is stored with less than first normalization As a result the front nodal point of the third normalization result of normalization result.
3. according to the method described in claim 1, it is characterized in that, the client by the interface in the new node structure Pointer is made, sets the pointer of the new node to the key of the posterior nodal point, and the pointer of the front nodal point is set as described The step of key of new node, specifically includes:
The client constructs pointer by the interface in the new node, after the pointer of the new node is set as described The key of node;
The client constructs daily record by the interface in the front nodal point, and the key of the new node is stored in the day Will;
The client is failed by the interface in the key for setting the pointer of the front nodal point to the new node storage When, the key of new node described in the log read from the front nodal point, the pointer of the front nodal point is set as reading described in The key of new node;
The client is by the interface by the daily record zero setting.
4. according to the method described in claim 3, it is characterized in that, the client obtains the OSD collection by the interface The key of free time OSD in group normalizes result, the title, the content and the sky by the interface by described first The key of not busy OSD is stored in the OSD of the free time, is specifically included to generate new node:
The client waits depositing when the size for determining the object to be stored is more than the memory space of the OSD by described Storage object is divided into the part of predetermined number, and the idle OSD of predetermined number described in the OSD clusters is obtained by the interface Each section of the object to be stored is respectively stored in the idle OSD of the free time OSD and the predetermined number by key;
Any OSD is chosen as the new node from the idle OSD of the predetermined number, is returned described first by the interface One key for changing the idle OSD of result, the title, the content and the predetermined number is stored in the new node.
5. method according to any one of claims 1 to 4, which is characterized in that the distributed index is skip list, the finger Needle includes multiple pointers, and each pointer is directed toward the different layers of the skip list.
6. according to the method described in claim 1, it is characterized in that, the method further includes:
The client obtains the title of object to be deleted;
The client is normalized the title to obtain the 4th normalization result;
The client waits deleting by being stored with the 4th normalization result in distributed index described in the interface polls Except node;
The client sets the pointer of the front nodal point of the node to be deleted to the node to be deleted by the interface Posterior nodal point key;
The client sets the node to be deleted to idle OSD by OSD clusters described in the interface notification.
7. according to the method described in claim 6, it is characterized in that, the client by the interface by the section to be deleted The step of key for the posterior nodal point that the pointer of the front nodal point of point is set as the node to be deleted, specifically includes:
The client constructs daily record by the interface in the front nodal point, and the key of the node storage to be deleted is stored In the daily record;
The client by the interface by the pointer of the front nodal point be set as the posterior nodal point key fail when, from The key of node to be deleted described in the log read of the front nodal point, is read according to the key of the node to be deleted by the interface The key for taking the posterior nodal point that the pointer of the node to be deleted stores, the pointer of the front nodal point is set as reading described in The key of posterior nodal point;
The client is by the interface by the daily record zero setting.
8. a kind of client, which is characterized in that the client is interacted by interface with OSD clusters, and the OSD clusters include more A OSD, the client select OSD as node to construct distributed index by the interface, in the distributed index Each node in, be stored with the title of itself and normalize result, from the OSD clusters key and pointer, The normalization result is generated according to the title, and the pointer is stored with the next node in the distributed index Key, the client include:
Object acquisition module to be stored, content and title for obtaining object to be stored;
Module is normalized, is normalized for the title to the object to be stored to obtain the first normalization result;
New node generation module, the key for obtaining free time OSD in the OSD clusters by the interface, passes through the interface The key for described first being normalized result, the title, the content and the free time OSD is stored in the free time OSD, to generate new node;
Enquiry module, for by being stored in distributed index described in the interface polls more than the first normalization result Second normalization result posterior nodal point and be stored with less than it is described first normalization result normalize result third normalization As a result front nodal point;
The pointer of the new node is arranged for constructing pointer in the new node by the interface for pointer modified module For the key of the posterior nodal point, and set the pointer of the front nodal point to the key of the new node.
9. client according to claim 8, which is characterized in that the distributed index includes Ingress node, it is described enter The pointer of mouth node is stored with the key that the node that result is minimum value is normalized in the distributed index, and the client is also Including entry record module, the entry record module is used to record the key of the Ingress node,
The enquiry module, specifically for pass through the interface since the Ingress node according to every in the distributed index The normalization for the node storage that the pointer of one node storage is inquired in the chained list is as a result, until inquire the distributed index In be stored with the posterior nodal point and being stored with of the second normalization result more than the first normalization result and return less than described first One changes the front nodal point of the third normalization result of result normalization result.
10. client according to claim 8, which is characterized in that the pointer modified module is specifically used for:
Pointer is constructed in the new node by the interface, sets the pointer of the new node to the key of the posterior nodal point;
Daily record is constructed in the front nodal point by the interface, the key of the new node is stored in the daily record;
By the interface when the key for setting the pointer of the front nodal point to the new node storage fails, before described The key of new node described in the log read of node, the new node that the pointer of the front nodal point is set as reading key;
By the interface by the daily record zero setting.
11. client according to claim 10, which is characterized in that new node generation module is specifically used for:
When the size for determining the object to be stored is more than the memory space of the OSD, the object to be stored is divided into The part of predetermined number obtains the key of the idle OSD of predetermined number described in the OSD clusters by the interface, will be described Each section of object to be stored is respectively stored in the idle OSD of the free time OSD and the predetermined number;
Any OSD is chosen as the new node from the idle OSD of the predetermined number, is returned described first by the interface One key for changing the idle OSD of result, the title, the content and the predetermined number is stored in the new node.
12. according to claim 8 to 11 any one of them client, which is characterized in that the distributed index is skip list, institute It includes multiple pointers to state pointer, and each pointer is directed toward the different layers of the skip list.
13. client according to claim 8, which is characterized in that the client further includes notification module,
The name acquiring module is additionally operable to obtain the title of object to be deleted;
The normalization module is additionally operable to that the title is normalized and returns four change results to obtain first;
The enquiry module is additionally operable to by being stored with the 4th normalization knot in distributed index described in the interface polls The node to be deleted of fruit;
The pointer modified module is additionally operable to set the pointer of the front nodal point of the node to be deleted to the node to be deleted Posterior nodal point key;
The node to be deleted is set as idle by the notification module for passing through OSD clusters described in the interface notification OSD。
14. client according to claim 13, which is characterized in that the pointer modified module is specifically used for:
Daily record is constructed in the front nodal point by the interface, the key of the node storage to be deleted is stored in the daily record;
By the interface in the key failures for setting the pointer of the front nodal point to the posterior nodal point, from the front nodal point Log read described in node to be deleted key, according to the key of the node to be deleted by the interface read described in wait deleting Except node pointer store posterior nodal point key, the posterior nodal point that the pointer of the front nodal point is set as reading key;
By the interface by the daily record zero setting.
15. a kind of client, which is characterized in that including processor, memory and bus, the processor, the memory It being connect respectively with the bus, the memory has program stored therein instruction, wherein:
The processor executes the first program instruction to execute step:Obtain the content and title of object to be stored;
The processor executes the second program instruction to execute step:The title of the object to be stored is normalized To obtain the first normalization result;
The processor executes third program instruction to execute step:The key of free time OSD in OSD clusters is obtained by interface, is led to It crosses the interface and the key of the first normalization result, the title, the content and the free time OSD is stored in institute Idle OSD is stated, to generate new node;
The processor executes fourth program instructions to execute step:It is big by being stored in the interface polls distributed index The posterior nodal point of result is normalized in the second of the first normalization result and is stored with less than the first normalization result Third normalizes the front nodal point of result;
The processor executes the 5th program instruction to execute step:Pointer is constructed in the new node by the interface, it will The pointer of the new node is set as the key of the posterior nodal point, and sets the pointer of the front nodal point to the new node key。
CN201710233466.XA 2017-04-11 2017-04-11 Distributed index method based on object and client Active CN108694209B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201710233466.XA CN108694209B (en) 2017-04-11 2017-04-11 Distributed index method based on object and client

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201710233466.XA CN108694209B (en) 2017-04-11 2017-04-11 Distributed index method based on object and client

Publications (2)

Publication Number Publication Date
CN108694209A true CN108694209A (en) 2018-10-23
CN108694209B CN108694209B (en) 2021-11-19

Family

ID=63843322

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201710233466.XA Active CN108694209B (en) 2017-04-11 2017-04-11 Distributed index method based on object and client

Country Status (1)

Country Link
CN (1) CN108694209B (en)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109669822A (en) * 2018-11-28 2019-04-23 平安科技(深圳)有限公司 The creation method and computer readable storage medium of electronic device, spare memory pool

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN1197570A (en) * 1995-08-02 1998-10-28 汤姆森消费电子有限公司 Linked list structure onscreen display
CN1514976A (en) * 1998-07-24 2004-07-21 �ָ��� Distributed computer data base system and method for object searching
US20100161614A1 (en) * 2008-12-22 2010-06-24 Electronics And Telecommunications Research Institute Distributed index system and method based on multi-length signature files
CN101996217A (en) * 2009-08-24 2011-03-30 华为技术有限公司 Method for storing data and memory device thereof
CN102779185B (en) * 2012-06-29 2014-11-12 浙江大学 High-availability distribution type full-text index method
CN106095920A (en) * 2016-06-07 2016-11-09 四川大学 Distributed index method towards extensive High dimensional space data

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN1197570A (en) * 1995-08-02 1998-10-28 汤姆森消费电子有限公司 Linked list structure onscreen display
CN1514976A (en) * 1998-07-24 2004-07-21 �ָ��� Distributed computer data base system and method for object searching
US20100161614A1 (en) * 2008-12-22 2010-06-24 Electronics And Telecommunications Research Institute Distributed index system and method based on multi-length signature files
CN101996217A (en) * 2009-08-24 2011-03-30 华为技术有限公司 Method for storing data and memory device thereof
CN102779185B (en) * 2012-06-29 2014-11-12 浙江大学 High-availability distribution type full-text index method
CN106095920A (en) * 2016-06-07 2016-11-09 四川大学 Distributed index method towards extensive High dimensional space data

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
胡永奎: ""对象存储文件系统的设计与实现"", 《中国优秀硕士学位论文全文数据库 信息科技辑》 *

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109669822A (en) * 2018-11-28 2019-04-23 平安科技(深圳)有限公司 The creation method and computer readable storage medium of electronic device, spare memory pool
CN109669822B (en) * 2018-11-28 2023-06-06 平安科技(深圳)有限公司 Electronic device, method for creating backup storage pool, and computer-readable storage medium

Also Published As

Publication number Publication date
CN108694209B (en) 2021-11-19

Similar Documents

Publication Publication Date Title
CN103067433B (en) A kind of data migration method of distributed memory system, equipment and system
KR102195836B1 (en) Method for managing index
CN109376156B (en) Method for reading hybrid index with storage awareness
US9871727B2 (en) Routing lookup method and device and method for constructing B-tree structure
CN103106286B (en) Method and device for managing metadata
JP2018518733A (en) File operation method and apparatus
CN106713250B (en) Data access method and device based on distributed system
US11775480B2 (en) Method and system for deleting obsolete files from a file system
WO2015118865A1 (en) Information processing device, information processing system, and data access method
CN110147345A (en) A kind of key assignments storage system and its working method based on RDMA
JP2001101042A (en) System and method for managing data
WO2014190700A1 (en) Method of memory access, buffer scheduler and memory module
CN109408539A (en) Data manipulation method, device, server and storage medium
CN110059026A (en) A kind of catalogue processing method, device and storage system
CN108694209A (en) Object-based distributed index method and client
CN108804571B (en) Data storage method, device and equipment
CN109492020A (en) A kind of data cache method, device, electronic equipment and storage medium
CN106796588A (en) The update method and equipment of concordance list
KR102214697B1 (en) A computer program for providing space managrment for data storage in a database management system
CN111209304A (en) Data processing method, device and system
KR100785774B1 (en) Obeject based file system and method for inputting and outputting
JP2014174870A (en) Virtual file access system, virtual file access method and virtual file access program
CN116821058B (en) Metadata access method, device, equipment and storage medium
CN109617943B (en) Data caching method and device, storage medium and computer equipment
CN116644006B (en) Memory page management method, system, device, equipment and computer medium

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant
TR01 Transfer of patent right
TR01 Transfer of patent right

Effective date of registration: 20220221

Address after: 550025 Huawei cloud data center, jiaoxinggong Road, Qianzhong Avenue, Gui'an New District, Guiyang City, Guizhou Province

Patentee after: Huawei Cloud Computing Technology Co.,Ltd.

Address before: 518129 Bantian HUAWEI headquarters office building, Longgang District, Guangdong, Shenzhen

Patentee before: HUAWEI TECHNOLOGIES Co.,Ltd.