CN108694209A - Object-based distributed index method and client - Google Patents
Object-based distributed index method and client Download PDFInfo
- Publication number
- CN108694209A CN108694209A CN201710233466.XA CN201710233466A CN108694209A CN 108694209 A CN108694209 A CN 108694209A CN 201710233466 A CN201710233466 A CN 201710233466A CN 108694209 A CN108694209 A CN 108694209A
- Authority
- CN
- China
- Prior art keywords
- key
- osd
- node
- pointer
- interface
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Landscapes
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The embodiment of the present invention discloses a kind of object-based distributed index method and client, this method include:Client obtains the content and title of object to be stored;The title that client treats storage object is normalized to obtain the first normalization result;Client obtains the key of free time OSD in OSD clusters by interface, and the key for normalizing result, title, content and free time OSD by first by interface is stored in idle OSD, to generate new node;Client normalizes the front nodal point of result by being stored with the posterior nodal point of the second normalization result more than the first normalization result in interface polls distributed index and being stored with less than the third of the first normalization result normalization result;Client constructs pointer in new node, sets the pointer of new node to the key of posterior nodal point, sets the pointer of front nodal point to the key of new node.The key of object is broken up in distributed index by pointer, positioning object is indexed by Querying Distributed, the difficulty of deployment and maintenance can be reduced.
Description
Technical field
This application involves information technology field, more particularly to a kind of object-based distributed index method and client.
Background technology
Object storage technology is used widely at this stage, and in object storage technology, OSD is arranged in storage system
(Object-based Storage Device, object storage device) cluster, OSD clusters include multiple OSD, every OSD settings
There are fixed memory space, object to be stored in the OSD of OSD clusters, there are one key (key) for every OSD correspondences, and client passes through
Key can find corresponding OSD, and from OSD from reading object.
In the storage system of the prior art, storage system also sets up MDS (Metadata Server, Metadata Service
Device), MDS stores the relationship between the network address of key OSD corresponding with key, and OSD clusters are provided with interface, and client exists
After obtaining key, the network address of the corresponding OSD of key is inquired to MDS by interface, so as to be accessed according to network address
Object in OSD.
Conventionally, as the memory space of OSD is fixed, therefore it is that required space is more than in object to be stored
When the larger object of the memory space of OSD, OSD cannot store the object, and in order to solve the technical problem, the prior art is by root
The OSD of corresponding number is set according to the size of object, object is divided into multiple portions and is respectively stored in the OSD of corresponding number.
Therefore, the MDS for object need to be further arranged in the storage system of the prior art, for the MDS storages pair of object
As the correspondence between title and the key of corresponding OSD, when client needs reading object, also need according to object oriented in needle
Which to inquiring for object, it is stored in OSD to obtain the object, and write OSD reading objects from this respectively.
However, in the prior art, client was with both having needed to be inquired the network to obtain the corresponding OSD of key to MDS
Location, and need to be inquired the key of multiple OSD to obtain object to the MDS for object, when for the failure of object
When, client cannot access object, therefore can increase the difficulty of deployment and maintenance for the presence of the MDS of object.
Invention content
To solve problem of the prior art, a kind of object-based distributed index method of offer of the embodiment of the present invention and visitor
Family end can be positioned pair due to being broken up the key of object in distributed index by pointer by Querying Distributed index
As the difficulty of deployment and maintenance can be reduced.
In a first aspect, the embodiment of the present invention provides a kind of object-based distributed index method, this method is applied to visitor
Family end, client are interacted by interface and OSD (Object-based Storage Device, object storage device) cluster,
OSD clusters include multiple OSD, and client selects OSD as node to construct distributed index by interface, in distributed index
Each node in, be stored with the title of itself and normalize result, from OSD clusters key (cluster) and pointer, return
One change result is generated according to title, and pointer is stored with the key of the next node in distributed index, and this method includes:
Client obtains the content and title of object to be stored;
The title that client treats storage object is normalized to obtain the first normalization result;
Client obtains the key of free time OSD in OSD clusters by interface, and result, name are normalized by first by interface
The key of title, content and free time OSD is stored in idle OSD, to generate new node;
Client is by being stored with the second normalization knot more than the first normalization result in interface polls distributed index
The posterior nodal point of fruit and the front nodal point for being stored with the third normalization result that result is normalized less than the first normalization result;
Client constructs pointer by interface in new node, sets the pointer of new node to the key of posterior nodal point, and will before
The pointer of node is set as the key of new node.
To sum up, in embodiments of the present invention, client by interface from OSD cluster application OSD, due to will be right by pointer
The key of elephant breaks up in distributed index, can position object by Querying Distributed index, can reduce the difficulty of deployment and maintenance
Degree.
In an implementation of the embodiment of the present invention, distributed index includes Ingress node, the pointer of Ingress node
It is stored with the key that the node that result is minimum value is normalized in distributed index, client passes through interface polls distributed index
In be stored with more than first normalization result second normalization result posterior nodal point and be stored with less than first normalization result
The step of front nodal point of the third normalization result of normalization result, specifically includes:
Client is by interface since the pointer inquiry point stored according to each node in distributed index Ingress node
The normalization of node storage in cloth index until inquiring in distributed index as a result, be stored with more than the first normalization knot
The posterior nodal point of the second normalization result of fruit normalizes knot with the third for normalizing result less than the first normalization result is stored with
The front nodal point of fruit.
In another realization method of the embodiment of the present invention, client constructs pointer by interface in new node, will be new
The pointer of node is set as the key of posterior nodal point, and the step of setting the pointer of front nodal point to the key of new node specifically includes:
Client constructs pointer by interface in new node, sets the pointer of new node to the key of posterior nodal point;
Client constructs daily record by interface in front nodal point, and the key of new node is stored in daily record;
Client by interface by the pointer of front nodal point be set as new node storage key fail when, from front nodal point
The key of log read new node, the key for the new node that the pointer of front nodal point is set as reading;
Client is by interface by daily record zero setting.
In another realization method of the embodiment of the present invention, client obtains free time OSD in OSD clusters by interface
Key, the key for normalizing result, title, content and free time OSD by first by interface is stored in idle OSD, to produce
Raw new node specifically includes:
Object to be stored is divided into pre- by client when the size for determining object to be stored is more than the memory space of OSD
Fixed number purpose part obtains the key of the idle OSD of predetermined number in OSD clusters by interface, by each section of object to be stored
It is respectively stored in the idle OSD of idle OSD and predetermined number;
Any OSD is chosen as new node from the idle OSD of predetermined number, and result, name are normalized by first by interface
The key of the idle OSD of title, content and predetermined number is stored in new node.
In another realization method of the embodiment of the present invention, distributed index is skip list, and pointer includes multiple pointers, often
One pointer is directed toward the different layers of skip list.
In another realization method of the embodiment of the present invention, this method further includes:
Client obtains the title of object to be deleted;
Client is normalized title to obtain the 4th normalization result;
Client is by being stored with the 4th node to be deleted for normalizing result in interface polls distributed index;
Client sets the pointer of the front nodal point of node to be deleted to by interface the posterior nodal point of node to be deleted
key;
Client sets node to be deleted to idle OSD by interface notification OSD clusters.
In another realization method of the embodiment of the present invention, client is by interface by the front nodal point of node to be deleted
Pointer is set as the step of key of the posterior nodal point of node to be deleted and specifically includes:
Client constructs daily record by interface in front nodal point, and the key that node to be deleted stores is stored in daily record;
Client is when the key for setting the pointer of front nodal point to by interface posterior nodal point fails, from the daily record of front nodal point
The key for reading node to be deleted reads the deutomerite of the pointer storage of node to be deleted according to the key of node to be deleted by interface
The key, the key for the posterior nodal point that the pointer of front nodal point is set as reading of point;
Client is by interface by daily record zero setting.
Second aspect, the embodiment of the present invention further provide for a kind of client, and client passes through interface and OSD collection flock-mates
Mutually, OSD clusters include multiple OSD, and client selects OSD as node to construct distributed index by interface, in distribution
In each node of index, it is stored with the title of itself and normalizes result, certainly in the key and pointer of OSD clusters, normalizing
Change result to be generated according to title, pointer is stored with the key of the next node in distributed index, and client includes:
Object acquisition module to be stored, content and title for obtaining object to be stored;
Module is normalized, the title for treating storage object is normalized to obtain the first normalization result;
New node generation module, the key for obtaining free time OSD in OSD clusters by interface, is returned by interface by first
One key for changing result, title, content and free time OSD is stored in idle OSD, to generate new node;
Enquiry module, for being returned more than the second of the first normalization result by being stored in interface polls distributed index
The posterior nodal point of one change result and the front nodal point for being stored with the third normalization result that result is normalized less than the first normalization result;
Pointer modified module sets the pointer of new node to posterior nodal point for constructing pointer in new node by interface
Key, and set the pointer of front nodal point to the key of new node.
In another realization method of the embodiment of the present invention, distributed index includes Ingress node, the finger of Ingress node
Needle is stored with the key that the node that result is minimum value is normalized in distributed index, and client further includes entry record module, is entered
Mouth logging modle is used for the key of record entry node,
Enquiry module, specifically for by interface since being stored according to each node in distributed index Ingress node
The normalization of node storage in pointer inquiry linked list until inquiring in distributed index as a result, be stored with more than the first normalizing
Change the posterior nodal point of the second normalization result of result and is stored with the third normalizing for normalizing result less than the first normalization result
Change the front nodal point of result.
In another realization method of the embodiment of the present invention, pointer modified module is specifically used for:
Pointer is constructed in new node by interface, sets the pointer of new node to the key of posterior nodal point;
Daily record is constructed in front nodal point by interface, the key of new node is stored in daily record;
By interface when the key for setting the pointer of front nodal point to new node storage fails, read from the daily record of front nodal point
The key for taking new node, the key for the new node that the pointer of front nodal point is set as reading;
By interface by daily record zero setting.
In another realization method of the embodiment of the present invention, new node generation module is specifically used for:
When the size for determining object to be stored is more than the memory space of OSD, object to be stored is divided into predetermined number
Part, by interface obtain OSD clusters in predetermined number idle OSD key, each section of object to be stored is deposited respectively
Storage is in the idle OSD of idle OSD and predetermined number;
Any OSD is chosen as new node from the idle OSD of predetermined number, and result, name are normalized by first by interface
The key of the idle OSD of title, content and predetermined number is stored in new node.
In another realization method of the embodiment of the present invention, distributed index is skip list, and pointer includes multiple pointers, often
One pointer is directed toward the different layers of skip list.
In another realization method of the embodiment of the present invention, client further includes notification module,
Name acquiring module is additionally operable to obtain the title of object to be deleted;
Module is normalized, is additionally operable to that title is normalized and returns four change results to obtain first;
Enquiry module is additionally operable to the section to be deleted by being stored with the 4th normalization result in interface polls distributed index
Point;
Pointer modified module is additionally operable to set the pointer of the front nodal point of node to be deleted to the posterior nodal point of node to be deleted
Key;
Notification module sets node to be deleted to idle OSD for passing through interface notification OSD clusters.
In another realization method of the embodiment of the present invention, pointer modified module is specifically used for:
Daily record is constructed in front nodal point by interface, the key that node to be deleted stores is stored in daily record;
By interface in the key failures for setting the pointer of front nodal point to posterior nodal point, waited for from the log read of front nodal point
The key of deletion of node reads the posterior nodal point of the pointer storage of node to be deleted according to the key of node to be deleted by interface
Key, the key for the posterior nodal point that the pointer of front nodal point is set as reading;
By interface by daily record zero setting.
The third aspect, a kind of client of offer of the embodiment of the present invention, including processor, memory and bus, processor,
Memory is connect with bus respectively, and memory has program stored therein instruction, wherein:
Processor executes the first program instruction to execute step:Obtain the content and title of object to be stored;
Processor executes the second program instruction to execute step:The title for treating storage object is normalized to obtain
Take the first normalization result;
Processor executes third program instruction to execute step:The key of free time OSD in OSD clusters is obtained by interface, is led to
It crosses interface and the key of the first normalization result, title, content and free time OSD is stored in idle OSD, to generate new save
Point;
Processor executes fourth program instructions to execute step:By being stored in interface polls distributed index more than
The posterior nodal point of second normalization result of one normalization result normalizes knot with the third for normalizing result less than first is stored with
The front nodal point of fruit;
Processor executes the 5th program instruction to execute step:Pointer is constructed in new node by interface, by new node
Pointer is set as the key of posterior nodal point, and sets the pointer of front nodal point to the key of new node.
Description of the drawings
In order to illustrate the technical solution of the embodiments of the present invention more clearly, will make below to required in the embodiment of the present invention
Attached drawing is briefly described, it should be apparent that, drawings described below is only some embodiments of the present invention, for
For those of ordinary skill in the art, without creative efforts, other are can also be obtained according to these attached drawings
Attached drawing.
Fig. 1 is the structural schematic diagram of storage system according to the ... of the embodiment of the present invention;
Fig. 2 is the flow chart of object-based distributed index method according to the ... of the embodiment of the present invention;
Fig. 3 is the schematic diagram of distributed index according to the ... of the embodiment of the present invention;
Fig. 4 is another schematic diagram of distributed index according to the ... of the embodiment of the present invention;
Fig. 5 is another schematic diagram of storage system according to the ... of the embodiment of the present invention;
Fig. 6 is the flow chart of object-based distributed index method according to the ... of the embodiment of the present invention;
Fig. 7 is another schematic diagram of distributed index according to the ... of the embodiment of the present invention;
Fig. 8 is another schematic diagram of distributed index according to the ... of the embodiment of the present invention;
Fig. 9 is the apparatus structure schematic diagram of client according to the ... of the embodiment of the present invention;
Figure 10 is another apparatus structure schematic diagram of client according to the ... of the embodiment of the present invention.
Specific implementation mode
It is the structural schematic diagram of storage system according to the ... of the embodiment of the present invention referring firstly to Fig. 1, Fig. 1, as shown in Figure 1,
The storage system of the embodiment of the present invention includes client and OSD clusters.
For example, the quantity of client can be multiple, as shown in Figure 1, client includes client 101,102 ...
N, in other examples, the quantity of client can also be 1, and the embodiment of the present invention is not construed as limiting this.
OSD clusters 20 include interface (not being painted) and multiple OSD, each OSD both corresponds to one of physical disk admittedly
Determine the memory space of size, the memory space of physical disk is divided into multiple OSD by OSD clusters 20, to store data.
OSD clusters 20 include MDS, and the every OSD of OSD are corresponding to be assigned a key, MDS have recorded key and memory space it
Between correspondence the object that the corresponding OSD of key are located in particular network address for example can be inquired in MDS by key
Manage the particular memory space of the specific physical disk of host carry.
Client 101,102 ... n access OSD clusters 20 by distributed index 201, and the setting of distributed index 201 exists
In the OSD for asking OSD clusters 20.
It is worth noting that, in embodiments of the present invention, OSD clusters 20 provide interface, and client only need to be using key as defeated
Enter parameter access interface, you can the corresponding OSD of key are accessed from interface.
Specifically, client is using key and reading order as input parameter access interface, you can directly read from interface
The data stored in OSD;Client can will be waited for using key, writing commands and object to be stored as input parameter access interface
Storage object is written into the corresponding memory spaces of key;Further, client can apply for the free time by interface to OSD clusters 20
OSD, for example, client to interface input application OSD orders, you can from interface obtain free time OSD key;And
Client can by interface notification OSD clusters 20 will store object to be stored OSD recycle, for example, client to
Interface inputs after key and revocation OSD orders, and OSD clusters 20 recycle OSD, and the key is inputted to interface to try in subsequent client
When figure OSD corresponding to key accesses, the access failure notification of interface transmission will be received.
Therefore, OSD clusters 20 shield client the internal course of work, and client only need to be by interface to OSD collection
Group 20 applies for OSD, obtains key in interface, and access to OSD in interface according to key.
Continuing with referring to Fig. 1, in fig. 1, it is assumed that client 101 from OSD clusters 20 apply OSD2, OSD9, OSD3,
OSD5, OSD7, OSD1 and OSD8, wherein above-mentioned every OSD stores corresponding data, distributed index 201 include OSD2,
OSD9, OSD3, OSD5, OSD7, OSD1 and OSD8, in above-mentioned every OSD, be stored with the title of itself and normalization as a result,
From the key and pointer in OSD clusters 20, normalization result is generated according to title, and pointer is stored in distributed index 201
In next node key, for example, OSD2 is stored with itself title, the normalization result that is generated according to title and refers to
Needle, the wherein pointer of OSD2 store the key of OSD9, and therefore, client 101, can after accessing OSD2 according to the key of OSD2
OSD9 is further accessed by interface according to the key of the pointer of OSD2 storage, and is accessed by the pointer of OSD9 storages
OSD3 ..., client 101 in OSD by being arranged pointer so that OSD forms distributed index 201, therefore client 101
The key of the node of the entrance as distributed index 201 need to be only recorded, for example such as the key of OSD2, passes through distributed index
201 can inquire all nodes in institute's distributed index 201 from 201 entrance of distributed index.
In embodiments of the present invention, client 101 applies for OSD, and the OSD that will apply by interface from OSD clusters 20
The form of index 201 is managed in a distributed manner, so as to avoid two layers of MDS of setting, can reduce the complexity of deployment and O&M
Degree.
In order to further clearly illustrate, refer to Fig. 2 below, Fig. 2 is object distribution formula rope according to the ... of the embodiment of the present invention
Draw the flow chart of 201 methods, includes the following steps as shown in Fig. 2, object distribution formula indexes 201 methods:
Step S301:Client 101 obtains the content and title of object to be stored.
Wherein, above-mentioned object includes file and uncorrected data.
For example, client 101 can be by providing webpage so that user accesses, and it is to be stored right that user uploads in webpage
As so that client 101 can obtain the content and title of object to be stored.
In this step, it is assumed that entitled " Xiao Ming " of object to be stored.
Step S302:The title that client 101 treats storage object is normalized to obtain the first normalization knot
Fruit.
In embodiments of the present invention, normalized refers to converting the title of object to be stored to the normalizing that can compare
Change result.
For example, client 101 can calculate the corresponding ASCII of title (American Standard Code for
Information Interchange, ASCII) code, title is scaled to the normalization that can compare
As a result, to realize normalized.
In other example, client 101 can also to title carry out Hash operation with obtain normalization as a result, from
And realize normalized.
Assuming that after " Xiao Ming " is normalized, the first normalization result of acquisition is 10.
Step S303:Client 101 obtains the key of free time OSD in OSD clusters 20 by interface, by interface by first
The key for normalizing result, title, content and free time OSD is stored in idle OSD, to generate new node.
In this step, client 101 can determine whether object to be stored size whether be more than OSD memory space, and
When determining memory space of the size of object to be stored more than OSD, object to be stored is divided into the part of predetermined number, is passed through
Interface obtains the key of the idle OSD of predetermined number in OSD clusters 20, and each section of object to be stored is respectively stored in the free time
In the idle OSD of OSD and predetermined number;Any OSD is chosen as new node from the idle OSD of predetermined number, passes through interface
The key for the idle OSD that first normalizes result, title, content and predetermined number is stored in new node.
Therefore, in the above manner, larger object can be stored in OSD by the embodiment of the present invention, to realize dilatation.
In other examples, storage of the size less than or equal to OSD that object to be stored is determined in client 101 is empty
Between when, directly object to be stored is stored in acquired idle OSD without by Object Segmentation to be stored, only needing.
Step S304:Client 101 in interface polls distributed index 201 by being stored with more than the first normalization result
Second normalization result posterior nodal point and be stored with less than first normalization result normalize result third normalize result
Front nodal point.
For details, reference can be made to the schematic diagrames that Fig. 3, Fig. 3 are distributed index 201 according to the ... of the embodiment of the present invention, shown in Fig. 3
Distributed index 201 is single distributed index 201, in the distributed index 201, including 7 nodes, respectively OSD2,
OSD9, OSD3, OSD5, OSD7, OSD1 and OSD8.
OSD2 is the Ingress node of distributed index 201, is stored with pointer, normalization result and other data, pointer
It is stored with the key of OSD9, normalization result is infinite to bear,
OSD8 is the end node of distributed index 201, is stored with pointer, normalization result and other data, pointer
It is stored with the key of OSD9, normalization result is just infinite.
Other nodes are stored with pointer, normalization result and other data respectively, and the key of pointer storage is as shown in Figure 3
For the key of next node, wherein other data include contents of object, object oriented, object length etc..
In this step, client 101 by interface since Ingress node according to each node in distributed index 201
The normalization of node storage in the pointer Querying Distributed index 201 of storage is as a result, until inquire in distributed index 201
It is stored with the posterior nodal point of the second normalization result more than the first normalization result and is stored with and return less than the first normalization result
One changes the front nodal point of the third normalization result of result.
Specifically, the access entrance node OSD2 first of client 101, accesses OSD9 according to the pointer of OSD2, knows
The normalization result of OSD9 is 5, and since 10 are more than 5, client 101 further accesses the OSD5 that the pointer of OSD9 is directed toward,
Know that the normalization result of OSD5 is 9, since 10 are more than 9, client 101 further accesses the pointer direction of OSD5
OSD11, since 10 are less than 53, client 101, which inquires, to be stored in distributed index 201 more than the first normalization result
(10) second normalizes the posterior nodal point OSD11 of result (53) and is stored with less than the first normalization result normalization result (10)
Third normalization result (9) front nodal point OSD5.
Step S305:Client 101 constructs pointer by interface in new node, sets the pointer of new node to posterior nodal point
Key, and set the pointer of front nodal point to the key of new node.
For example, reference can be made to Fig. 4, Fig. 4 are another schematic diagrames of distributed index 201 according to the ... of the embodiment of the present invention,
Client 101 constructs pointer in OSD9 so that the key of the pointer storage OSD7 of OSD9 changes the pointer of OSD7 so that OSD7
Pointer storage OSD11 key so that OSD7 can be plugged into single distributed index 201.
It is worth noting that, in this step, prosthomere is changed due to generating new node and this step in above-mentioned steps S303
The pointer of point is two independent steps, can not ensure atomicity, if this step unsuccessfully can to the pointer modified of front nodal point OSD5
Cause client 101 to lose the record of new node OSD7, therefore in this step, needs further to construct day in front nodal point OSD5
The key of new node OSD7 is stored in daily record by will, is arranged by the pointer of front nodal point OSD5 by interface in client 101
When failing for the key of new node OSD7, client 101 can be from the key of the log read new node of front nodal point OSD7, by front nodal point
The pointer of OSD5 is set as the key of the new node OSD7 read, and after being arranged successfully, client 101 is by interface by day
Will zero setting.
Therefore, after by above step, client 101 forms new distributed index 201 in OSD clusters 20, tool
Body is as shown in figure 5, Fig. 5 is another schematic diagram of storage system according to the ... of the embodiment of the present invention, in Figure 5, OSD2, OSD9,
OSD3, OSD5, OSD11, OSD7, OSD1 and OSD8 form distributed index 201, wherein OSD11 is inserted by the above method
In distributed index 201 shown in FIG. 1.
Therefore, in embodiments of the present invention, client 101 applies for OSD by interface from OSD clusters 20, and application is arrived
OSD be inserted into distributed index 201, so as to avoid setting two layers of MDS, the complexity of deployment and O&M can be reduced.
The embodiment of the deletion of node in distributed index 201 will be introduced below, refer to Fig. 6, Fig. 6 is basis
The object distribution formula of the embodiment of the present invention indexes the flow chart of 201 methods, as shown in fig. 6, object distribution formula indexes 201 method packets
Include following steps:
Step S401:Client 101 obtains the title of object to be deleted.
For example, client 101 can be by providing webpage so that user accesses, and user inputs object to be deleted in webpage
Title so that client 101 can obtain the title of object to be deleted.
In this step, it is assumed that entitled " Xiao Ming " of object to be deleted.
Step S402:Client 101 is normalized title to obtain the 4th normalization result.
In embodiments of the present invention, normalized refers to converting the title of object to be stored to the normalizing that can compare
Change result.
For example, client 101 can calculate the corresponding ASCII of title (American Standard Code for
Information Interchange, ASCII) code, title is scaled to the normalization that can compare
As a result, to realize normalized.
In other example, client 101 can also to title carry out Hash operation with obtain normalization as a result, from
And realize normalized.
In other example, client 101 can also carry out title dictionary sequence to realize normalized.
Assuming that after " Xiao Ming " is normalized, the 4th normalization result of acquisition is 10.
Step S403:Client 101 is by being stored with waiting for for the 4th normalization result in interface polls distributed index 201
Deletion of node.
Specifically, distributed index 201 includes Ingress node, Ingress node is stored with negative infinite normalization as a result, visitor
Family end 101 is by interface since the pointer Querying Distributed stored according to each node in distributed index 201 Ingress node
The normalization of node storage in index 201 until inquiring as a result, be stored with the first normalization result in distributed index 201
Node.
As described above, OSD2 is Ingress node, and client 101 accesses OSD2 first, knows the normalization knot of OSD2
Fruit is negative infinite, and OSD9 is accessed according to the pointer of OSD2, knows that the normalization result of OSD9 is 5, since 10 more than 5, visitor
Family end 101 further accesses the OSD5 that the pointer of OSD9 is directed toward, and knows that the normalization result of OSD5 is 9, since 10 more than 9, because
This client 101 further accesses the OSD7 that the pointer of OSD5 is directed toward, and the normalization of OSD7 is the result is that 10, therefore client 101
Using node OSD7 as node to be deleted.
Step S404:Client 101 sets the pointer of the front nodal point of node to be deleted to node to be deleted by interface
Posterior nodal point key.
In this step, client 101 sets the pointer of the front nodal point OSD5 of OSD7 to by interface the key of OSD1.
Step S405:Client 101 sets node to be deleted to idle OSD by interface notification OSD clusters 20.
In this step, client 101 notifies OSD clusters 20 to recycle node to be deleted in the above manner, can effectively return
OSD resources are received, idle OSD will not be caused occupied.
Further, in this step, client 101 can inquire whether node to be deleted also stores just like described in step S303
For store larger object OSD key, if so, client 101 will further be used by interface notification OSD clusters 20
It is set as idle OSD in the OSD of storage larger object.
Significantly, since this step is two independent with the previous step step S404 pointers for changing front nodal point
Step can not ensure atomicity, and when even step S404 executes failure, client 101 can lose the key of node to be deleted, from
And step S404 and step S405 is caused to execute failure.
For this purpose, in step s 404, client 101 constructs daily record by interface in front nodal point, and node to be deleted is stored
Key be stored in daily record, and in client 101 when the key for setting the pointer of front nodal point to by interface posterior nodal point fails,
From the key of the log read of front nodal point node to be deleted, new node is read by interface according to the key of node to be deleted
The key, the key for the posterior nodal point that the pointer of front nodal point is set as reading, before ensureing modification of the posterior nodal point of pointer storage
When the pointer failure of node, the key of node to be deleted will not be lost, it is notable that the pointer success of node before a modification
Afterwards, the daily record of front nodal point is not necessarily to record the key of node to be deleted, and client 101 can be by interface by daily record zero setting at this time.
To sum up, in embodiments of the present invention, client 101 applies for OSD by interface from OSD clusters 20, and application is arrived
OSD from distributed index 201 delete, so as to avoid setting two layers of MDS, the complexity of deployment and O&M can be reduced.
Distributed index 201 described above is single distributed index 201, it is notable that in the optional of the present invention
In embodiment, distributed index 201 can also be skip list and binary tree, and wherein skip list is particularly suitable in the number of objects stored
Larger situation.
Specific to be referred in combination with Fig. 7 and Fig. 8, wherein Fig. 7 is distributed index 201 according to the ... of the embodiment of the present invention
Another schematic diagram, distributed index 201 shown in Fig. 7 are skip list, and pointer includes multiple pointers, and each pointer is directed toward the difference of skip list
Layer.
For example, OSD2 stores the 1st layer of pointer, the 2nd layer of pointer, the 3rd layer of pointer and the 4th layer of pointer, each finger
Needle stores the corresponding node of each layer, when finding OSD7 (normalization result 10) in the skip list that client 101 wants shown in Fig. 7,
First search the node of the same layer of the top directions of Ingress node OSD2, the i.e. OSD8 that the 4th layer of pointer is directed toward, due to returning for OSD8
One change result is just infinite, therefore searches time node OSD1 of high-rise (the 3rd layer) same layer being directed toward, and normalization result is 78
Inconsistent with 10, therefore search the OSD3 that the 2nd layer of pointer is directed toward, the normalization result of OSD3 is 5, is less than 10, in being to look for
The OSD1 of the 2nd layer of direction of OSD3 is more than 10 since the normalization result of OSD1 is 53, therefore searches the 1st layer of direction of OSD3
OSD5 further search for the OSD7 of the 1st layer of OSD5 direction when the normalization result 9 for inquiring OSD5 is less than 10, due to
The normalization of OSD7 confirms that OSD7 is the node to be inquired the result is that 10.
The number of plies of pointer is more, then search speed is faster, is according to the ... of the embodiment of the present invention point for details, reference can be made to Fig. 8, Fig. 8
Another schematic diagram of cloth index 201, unlike Fig. 7, in fig. 8, OSD7 stores 4 layers of pointer and therefore works as client
101 when wanting to find OSD7 (normalization result 10) in skip list shown in Fig. 8, first searches the top directions of Ingress node OSD2
The OSD7 that the node of same layer, i.e. the 4th layer of pointer are directed toward confirms that OSD7 is institute since the normalization result of OSD87 is 10
The node to be inquired.
It is worth noting that, if each node is both provided with multilayer pointer, the lookup of distributed index 201 can be seriously affected
Speed, therefore the number of plies of each node need to be limited, existing " throwing coin " algorithm is specifically can refer to determine the layer of each node
Number for example for OSD2, determines whether to increase the 2nd layer of pointer by " throw coin " algorithm, if throw coin is
Front then increases the 2nd layer of pointer, if it is reverse side to throw coin, only retains the 1st layer of pointer, and increase the 2nd layer of pointer it
Afterwards, further by throw coin come determine whether increase the 3rd layer of pointer ..., when being inserted into new node, to upper type at
Reason, so that multilayer pointer is evenly distributed, does not interfere with 201 search speed of distributed index.
Further, client 101 can also cache the node inquired, when the normalization result to be inquired
In the presence of in the buffer, without being inquired to distributed index 201, caching is locally being read.
Frequent node is accessed in 101 inner buffer of each client oneself, to naturally occur multiple pairs of thermal center point
Originally hot spot is shared, simultaneously because service node is directly accessed from inner buffer when accessing thermal center point, also reduces the pressure to OSD
Power.
For example, caching using LRU (Least recently used, least recently used) mode tissue, due to
Node is globally shared, then the problem of caching can be related to consistency, so this caching can be used for making in positioning node
With after having found interdependent node in the buffer, needing the content for directly accessing storages of the OSD to this node again to confirm, Key
Inconsistent or node is there is no caching is so deleted, if the inconsistent so update caching of other content.Also, caching can only
Ordinary node is cached, the node in addition deletion of node affairs cannot cache, in order to avoid more complicated consistency is caused to be asked
Topic.
For example, the present embodiments relate to data change in OSD can be used CAS (Compare and swap, than
Compared with exchange) operation, usually, CAS operation require be passed to old value, such as:
bool compare_and_set(osd_key_t key,void*old_value,void*new_value);
Wherein key is metadata, indicates object's position in OSD, such as the position offset of write-in, data length is written
Data in up till now OSD objects occur for length etc., old_value, old value, i.e. write-in;New_value, data to be written.
The present invention program does following improvement to CAS operation, using but be not limited to checksum (check value), version number etc. and replace
For old value:
1, old value is substituted using checksum values, for example uses HASH32 (32 algorithm of Hash) or MD5 (Message-
Digest Algorithm 5, Message-Digest Algorithm 5) etc., when change, first calculates the checksum values of Current Content, if with
It is incoming identical, it is modified.
2, old value is substituted using version identifier, i.e., increases version number's mark in the data structure to be changed, every time
It changes and successfully then updates version identifier (such as one from the normalization increased as a result, changing successfully+1 every time), when change is passed to old
Version identifier, Current Content version identifier is modified when consistent with incoming.
In the above manner, the data volume transmitted needed for can effectively reducing.
The embodiment of the present invention further provides for a kind of client 101, and client 101 is interacted by interface with OSD clusters 20,
OSD clusters 20 include multiple OSD, and client 101 selects OSD as node to construct distributed index 201 by interface, are being divided
Cloth index 201 each node in, be stored with the title of itself and normalize result, from OSD clusters 20 key and
Pointer, normalization result are generated according to title, and pointer is stored with the key of the next node in distributed index 201, specifically asks
It is the apparatus structure schematic diagram of client according to the ... of the embodiment of the present invention referring to Fig. 9, Fig. 9, as shown in figure 9, client 101 is wrapped
It includes:
Object acquisition module 501 to be stored, content and title for obtaining object to be stored;
Module 502 is normalized, the title for treating storage object is normalized to obtain the first normalization knot
Fruit;
New node generation module 503, the key for obtaining free time OSD in OSD clusters 20 by interface will by interface
First key for normalizing result, title, content and free time OSD is stored in idle OSD, to generate new node;
Enquiry module 504, for by being stored in interface polls distributed index 201 more than the first normalization result
The posterior nodal point of second normalization result normalizes result with the third for normalizing result less than the first normalization result is stored with
Front nodal point;
Pointer modified module 505 sets the pointer of new node to deutomerite for constructing pointer in new node by interface
The key of point, and set the pointer of front nodal point to the key of new node.
Optionally, distributed index 201 includes Ingress node, and Ingress node is stored with negative infinite normalization as a result, client
End 101 further includes entry record module, and entry record module is used for the key of record entry node,
Enquiry module 504, specifically for pass through interface since Ingress node according to each node in distributed index 201
The normalization of node storage in the pointer Querying Distributed index 201 of storage is as a result, until inquire in distributed index 201
It is stored with the posterior nodal point of the second normalization result more than the first normalization result and is stored with and return less than the first normalization result
One changes the front nodal point of the third normalization result of result.
Optionally, pointer modified module 505, is specifically used for:
Pointer is constructed in new node by interface, sets the pointer of new node to the key of posterior nodal point;By interface preceding
The key of new node is stored in daily record by joint structure daily record;It is setting the pointer of front nodal point to new node storage by interface
Key failure when, from the key of the log read new node of front nodal point, the new node that the pointer of front nodal point is set as reading
Key;By interface by daily record zero setting.
Optionally, new node generation module 503, is specifically used for:
When the size for determining object to be stored is more than the memory space of OSD, object to be stored is divided into predetermined number
Part, the key of the idle OSD of predetermined number in OSD clusters 20 is obtained by interface, each section of object to be stored is distinguished
It is stored in the idle OSD of idle OSD and predetermined number;Any OSD is chosen as new section from the idle OSD of predetermined number
The key for the idle OSD that first normalizes result, title, content and predetermined number is stored in new node by point by interface.
Optionally, distributed index 201 is skip list, and pointer includes multiple pointers, and each pointer is directed toward the different layers of skip list.
Optionally, the client 101 of the embodiment of the present invention further comprises notification module 505, wherein:
Name acquiring module 501 is additionally operable to obtain the title of object to be deleted;
Module 502 is normalized, for title being normalized to obtain the first normalization result;
Enquiry module 503, for waiting deleting by being stored with the first normalization result in interface polls distributed index 201
Except node;
Pointer modified module 504, the deutomerite for setting the pointer of the front nodal point of node to be deleted to node to be deleted
The key of point;
Notification module 505 sets node to be deleted to idle OSD for passing through interface notification OSD clusters 20.
Optionally, distributed index 201 includes Ingress node, and the pointer of Ingress node is stored in distributed index 201
The node that result is minimum value is normalized, client 101 further includes entry record module, and entry record module is used for record entry
The key of node, enquiry module 603 are specifically used for:
By interface since the pointer Querying Distributed stored according to each node in distributed index 201 Ingress node
The normalization of node storage in index 201 until inquiring as a result, be stored with the first normalization result in distributed index 201
Node.
Optionally, pointer modified module 604 is specifically used for:
Daily record is constructed in front nodal point by interface, the key that new node stores is stored in daily record;
It is new from the log read of front nodal point by interface in the key failures for setting the pointer of front nodal point to posterior nodal point
The key of node reads the key of the posterior nodal point of the pointer storage of new node according to the key of new node, by front nodal point by interface
Pointer is set as the key of the posterior nodal point read.
Optionally, pointer modified module 604 is additionally operable to daily record zero setting through interface.
Optionally, distributed index 201 is skip list, and pointer includes multiple pointers, and each pointer is directed toward the different layers of skip list.
Fig. 9 is referred to below, and Fig. 9 is another apparatus structure schematic diagram of client according to the ... of the embodiment of the present invention, such as Fig. 9
It is shown, client 101 include processor 701, memory 702 and bus 703, processor 701, memory 702 respectively with always
Line 703 connects, and memory 702 has program stored therein instruction, and processor 701 is executed program instructions to complete above example introduction
The function of client 101, wherein:
Processor 701 executes the first program instruction to execute step S301:Obtain the content and title of object to be stored;
Processor 701 executes the second program instruction to execute step S302:The title for treating storage object is normalized
Processing is to obtain the first normalization result;
Processor 701 executes third program instruction to execute step S303:It is obtained by interface idle in OSD clusters 20
The key of OSD, the key for normalizing result, title, content and free time OSD by first by interface are stored in idle OSD, from
And generate new node;
Processor 701 executes fourth program instructions to execute step S304:By being deposited in interface polls distributed index 201
It contains the posterior nodal point of the second normalization result more than the first normalization result and is stored with less than the first normalization result normalizing
Change the front nodal point of the third normalization result of result;
Processor 701 executes the 5th program instruction to execute step S305:Pointer is constructed in new node by interface, it will be new
The pointer of node is set as the key of posterior nodal point, and sets the pointer of front nodal point to the key of new node.
Optionally, distributed index 201 includes Ingress node, and the pointer of Ingress node storage is stored with distributed index
The node that result is minimum value, the key of 702 record entry node of memory are normalized in 201, processor 701 executes quadruple pass
Sequence instruction with by interface since the pointer Querying Distributed stored according to each node in distributed index 201 Ingress node
The normalization of node storage in index 201 until inquiring in distributed index 201 as a result, be stored with more than the first normalization
As a result the posterior nodal point of the second normalization result normalizes the third normalization of result with being stored with less than the first normalization result
As a result front nodal point.
Optionally, processor 701 executes the 5th program instruction to construct pointer in new node by interface, by new node
Pointer is set as the key of posterior nodal point;Daily record is constructed in front nodal point by interface, the key of new node is stored in daily record;By connecing
Mouthful when the key for setting the pointer of front nodal point to new node storage fails, from the key of the log read new node of front nodal point,
The key for the new node that the pointer of front nodal point is set as reading;By interface by daily record zero setting.
Optionally, processor 701 executes third program instruction with the depositing more than OSD in the size for determining object to be stored
When storing up space, object to be stored is divided into the part of predetermined number, the free time of predetermined number in OSD clusters 20 is obtained by interface
Each section of object to be stored is respectively stored in the idle OSD of idle OSD and predetermined number by the key of OSD;From predetermined
The idle OSD of number chooses any OSD as new node, normalizes result, title, content by first by interface and makes a reservation for
The key of the idle OSD of number is stored in new node.
Optionally, distributed index 201 is skip list, and pointer includes multiple pointers, and each pointer is directed toward the different layers of skip list.
Another apparatus structure schematic diagram that 10, Figure 10 is client according to the ... of the embodiment of the present invention is referred to below, is such as schemed
Shown in 10, client 101 include processor 801, memory 802 and bus 803, processor 801, memory 802 respectively with
Bus 803 connects, and memory 802 has program stored therein instruction, and processor 801 is executed program instructions to complete above example introduction
Client 101 function, wherein:
Processor 801 executes the first program instruction to execute step S401:Obtain the title of object to be deleted;
Processor 801 executes the second program instruction to execute step S402:Title is normalized to obtain
Four normalization results;
Processor 801 executes third program instruction to execute step S403:By being deposited in interface polls distributed index 201
Contain the node to be deleted of the first normalization result;
Processor 801 executes fourth program instructions to execute step S404:By interface by the front nodal point of node to be deleted
Pointer be set as node to be deleted posterior nodal point key;
Processor 801 executes the 5th program instruction to execute step S405:It will be to be deleted by interface notification OSD clusters 20
Node is set as idle OSD.
Optionally, distributed index 201 includes Ingress node, and Ingress node is stored with negative infinite normalization as a result, storage
Device 802 is additionally operable to the key of record entry node, and processor 801 executes third program instruction to be opened from Ingress node by interface
The normalization begun according to the node storage in the pointer Querying Distributed index 201 of each node storage in distributed index 201
As a result, until inquiring the node for being stored with the first normalization result in distributed index 201.
Optionally, processor 801 executes fourth program instructions to construct daily record in front nodal point by interface, and new node is deposited
The key of storage is stored in daily record;By interface in the key failures for setting the pointer of front nodal point to posterior nodal point, from front nodal point
The key of log read new node reads the key of the posterior nodal point of the pointer storage of new node according to the key of new node by interface,
The key for the posterior nodal point that the pointer of front nodal point is set as reading.
Optionally, processor 801 execute fourth program instructions with by interface by daily record zero setting.
Optionally, distributed index 201 is skip list, and pointer includes multiple pointers, and each pointer is directed toward the different layers of skip list.
To sum up, in embodiments of the present invention, client 101 applies for OSD by interface from OSD clusters 20, and application is arrived
OSD index 201 form in a distributed manner and be managed, so as to avoid two layers of MDS of setting, deployment and O&M can be reduced
Complexity.
It should be noted that any device embodiment described above is all only schematical, wherein described as separation
The unit of part description may or may not be physically separated, the component shown as unit can be or
It can not be physical unit, you can be located at a place, or may be distributed over multiple network units.It can be according to reality
Border needs to select some or all of process therein to achieve the purpose of the solution of this embodiment.In addition, provided by the invention
In device embodiment attached drawing, the connection relation between process indicates there is communication connection between them, specifically can be implemented as one
Item or a plurality of communication bus or signal wire.Those of ordinary skill in the art are without creative efforts, you can with
Understand and implements.
Through the above description of the embodiments, it is apparent to those skilled in the art that the present invention can borrow
Help software that the mode of required common hardware is added to realize, naturally it is also possible to by specialized hardware include application-specific integrated circuit, specially
It is realized with CPU, private memory, special components and parts etc..Under normal circumstances, all functions of being completed by computer program can
It is easily realized with corresponding hardware, moreover, for realizing that the particular hardware structure of same function can also be a variety of more
Sample, such as analog circuit, digital circuit or special circuit etc..But it is more for the purpose of the present invention in the case of software program it is real
It is now more preferably embodiment.Based on this understanding, technical scheme of the present invention substantially in other words makes the prior art
The part of contribution can be expressed in the form of software products, which is stored in the storage medium that can be read
In, such as the floppy disk of computer, USB flash disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory
Device (RAM, Random Access Memory), magnetic disc or CD etc., including some instructions are with so that a computer is set
Standby (can be personal computer, server or the network equipment etc.) executes the method described in each embodiment of the present invention.
It is apparent to those skilled in the art that the specific work of the system of foregoing description, device or unit
Make process, can refer to corresponding processes in the foregoing method embodiment, details are not described herein.
The above description is merely a specific embodiment, but scope of protection of the present invention is not limited thereto, any
Those familiar with the art in the technical scope disclosed by the present invention, can easily think of the change or the replacement, and should all contain
Lid is within protection scope of the present invention.Therefore, protection scope of the present invention should be based on the protection scope of the described claims.
Claims (15)
1. a kind of object-based distributed index method, which is characterized in that the method is applied to client, the client
It is interacted by interface and OSD (Object-based Storage Device, object storage device) cluster, the OSD clusters packet
Multiple OSD are included, the client selects OSD as node to construct distributed index by the interface, in the distribution
In each node of index, it is stored with the title of itself and normalizes result, (collects from the key in the OSD clusters
Group) and pointer, the normalization result generated according to the title, the pointer is stored in the distributed index
The key of next node, the method includes:
The client obtains the content and title of object to be stored;
The client is normalized the title of the object to be stored to obtain the first normalization result;
The client obtains the key of free time OSD in the OSD clusters by the interface, by the interface by described
One key for normalizing result, the title, the content and the free time OSD is stored in the OSD of the free time, to produce
Raw new node;
The client in distributed index described in the interface polls by being stored with more than the first normalization result
The posterior nodal point of second normalization result normalizes knot with the third for normalizing result less than the first normalization result is stored with
The front nodal point of fruit;
The client constructs pointer by the interface in the new node, after the pointer of the new node is set as described
The key of node, and set the pointer of the front nodal point to the key of the new node.
2. according to the method described in claim 1, it is characterized in that, the distributed index includes Ingress node, the entrance
The pointer of node is stored with the key that the node that result is minimum value is normalized in the distributed index, and the client passes through
After being stored with the second normalization result more than the first normalization result in distributed index described in the interface polls
Node has with the step of third for normalizing result less than the first normalization result normalizes the front nodal point of result is stored with
Body includes:
Since the client by the interface stored according to each node in the distributed index the Ingress node
The normalization of node storage inquired in the distributed index of pointer as a result, until inquiring and being deposited in the distributed index
It contains the posterior nodal point for the second normalization result for normalizing result more than described first and is stored with less than first normalization
As a result the front nodal point of the third normalization result of normalization result.
3. according to the method described in claim 1, it is characterized in that, the client by the interface in the new node structure
Pointer is made, sets the pointer of the new node to the key of the posterior nodal point, and the pointer of the front nodal point is set as described
The step of key of new node, specifically includes:
The client constructs pointer by the interface in the new node, after the pointer of the new node is set as described
The key of node;
The client constructs daily record by the interface in the front nodal point, and the key of the new node is stored in the day
Will;
The client is failed by the interface in the key for setting the pointer of the front nodal point to the new node storage
When, the key of new node described in the log read from the front nodal point, the pointer of the front nodal point is set as reading described in
The key of new node;
The client is by the interface by the daily record zero setting.
4. according to the method described in claim 3, it is characterized in that, the client obtains the OSD collection by the interface
The key of free time OSD in group normalizes result, the title, the content and the sky by the interface by described first
The key of not busy OSD is stored in the OSD of the free time, is specifically included to generate new node:
The client waits depositing when the size for determining the object to be stored is more than the memory space of the OSD by described
Storage object is divided into the part of predetermined number, and the idle OSD of predetermined number described in the OSD clusters is obtained by the interface
Each section of the object to be stored is respectively stored in the idle OSD of the free time OSD and the predetermined number by key;
Any OSD is chosen as the new node from the idle OSD of the predetermined number, is returned described first by the interface
One key for changing the idle OSD of result, the title, the content and the predetermined number is stored in the new node.
5. method according to any one of claims 1 to 4, which is characterized in that the distributed index is skip list, the finger
Needle includes multiple pointers, and each pointer is directed toward the different layers of the skip list.
6. according to the method described in claim 1, it is characterized in that, the method further includes:
The client obtains the title of object to be deleted;
The client is normalized the title to obtain the 4th normalization result;
The client waits deleting by being stored with the 4th normalization result in distributed index described in the interface polls
Except node;
The client sets the pointer of the front nodal point of the node to be deleted to the node to be deleted by the interface
Posterior nodal point key;
The client sets the node to be deleted to idle OSD by OSD clusters described in the interface notification.
7. according to the method described in claim 6, it is characterized in that, the client by the interface by the section to be deleted
The step of key for the posterior nodal point that the pointer of the front nodal point of point is set as the node to be deleted, specifically includes:
The client constructs daily record by the interface in the front nodal point, and the key of the node storage to be deleted is stored
In the daily record;
The client by the interface by the pointer of the front nodal point be set as the posterior nodal point key fail when, from
The key of node to be deleted described in the log read of the front nodal point, is read according to the key of the node to be deleted by the interface
The key for taking the posterior nodal point that the pointer of the node to be deleted stores, the pointer of the front nodal point is set as reading described in
The key of posterior nodal point;
The client is by the interface by the daily record zero setting.
8. a kind of client, which is characterized in that the client is interacted by interface with OSD clusters, and the OSD clusters include more
A OSD, the client select OSD as node to construct distributed index by the interface, in the distributed index
Each node in, be stored with the title of itself and normalize result, from the OSD clusters key and pointer,
The normalization result is generated according to the title, and the pointer is stored with the next node in the distributed index
Key, the client include:
Object acquisition module to be stored, content and title for obtaining object to be stored;
Module is normalized, is normalized for the title to the object to be stored to obtain the first normalization result;
New node generation module, the key for obtaining free time OSD in the OSD clusters by the interface, passes through the interface
The key for described first being normalized result, the title, the content and the free time OSD is stored in the free time
OSD, to generate new node;
Enquiry module, for by being stored in distributed index described in the interface polls more than the first normalization result
Second normalization result posterior nodal point and be stored with less than it is described first normalization result normalize result third normalization
As a result front nodal point;
The pointer of the new node is arranged for constructing pointer in the new node by the interface for pointer modified module
For the key of the posterior nodal point, and set the pointer of the front nodal point to the key of the new node.
9. client according to claim 8, which is characterized in that the distributed index includes Ingress node, it is described enter
The pointer of mouth node is stored with the key that the node that result is minimum value is normalized in the distributed index, and the client is also
Including entry record module, the entry record module is used to record the key of the Ingress node,
The enquiry module, specifically for pass through the interface since the Ingress node according to every in the distributed index
The normalization for the node storage that the pointer of one node storage is inquired in the chained list is as a result, until inquire the distributed index
In be stored with the posterior nodal point and being stored with of the second normalization result more than the first normalization result and return less than described first
One changes the front nodal point of the third normalization result of result normalization result.
10. client according to claim 8, which is characterized in that the pointer modified module is specifically used for:
Pointer is constructed in the new node by the interface, sets the pointer of the new node to the key of the posterior nodal point;
Daily record is constructed in the front nodal point by the interface, the key of the new node is stored in the daily record;
By the interface when the key for setting the pointer of the front nodal point to the new node storage fails, before described
The key of new node described in the log read of node, the new node that the pointer of the front nodal point is set as reading
key;
By the interface by the daily record zero setting.
11. client according to claim 10, which is characterized in that new node generation module is specifically used for:
When the size for determining the object to be stored is more than the memory space of the OSD, the object to be stored is divided into
The part of predetermined number obtains the key of the idle OSD of predetermined number described in the OSD clusters by the interface, will be described
Each section of object to be stored is respectively stored in the idle OSD of the free time OSD and the predetermined number;
Any OSD is chosen as the new node from the idle OSD of the predetermined number, is returned described first by the interface
One key for changing the idle OSD of result, the title, the content and the predetermined number is stored in the new node.
12. according to claim 8 to 11 any one of them client, which is characterized in that the distributed index is skip list, institute
It includes multiple pointers to state pointer, and each pointer is directed toward the different layers of the skip list.
13. client according to claim 8, which is characterized in that the client further includes notification module,
The name acquiring module is additionally operable to obtain the title of object to be deleted;
The normalization module is additionally operable to that the title is normalized and returns four change results to obtain first;
The enquiry module is additionally operable to by being stored with the 4th normalization knot in distributed index described in the interface polls
The node to be deleted of fruit;
The pointer modified module is additionally operable to set the pointer of the front nodal point of the node to be deleted to the node to be deleted
Posterior nodal point key;
The node to be deleted is set as idle by the notification module for passing through OSD clusters described in the interface notification
OSD。
14. client according to claim 13, which is characterized in that the pointer modified module is specifically used for:
Daily record is constructed in the front nodal point by the interface, the key of the node storage to be deleted is stored in the daily record;
By the interface in the key failures for setting the pointer of the front nodal point to the posterior nodal point, from the front nodal point
Log read described in node to be deleted key, according to the key of the node to be deleted by the interface read described in wait deleting
Except node pointer store posterior nodal point key, the posterior nodal point that the pointer of the front nodal point is set as reading
key;
By the interface by the daily record zero setting.
15. a kind of client, which is characterized in that including processor, memory and bus, the processor, the memory
It being connect respectively with the bus, the memory has program stored therein instruction, wherein:
The processor executes the first program instruction to execute step:Obtain the content and title of object to be stored;
The processor executes the second program instruction to execute step:The title of the object to be stored is normalized
To obtain the first normalization result;
The processor executes third program instruction to execute step:The key of free time OSD in OSD clusters is obtained by interface, is led to
It crosses the interface and the key of the first normalization result, the title, the content and the free time OSD is stored in institute
Idle OSD is stated, to generate new node;
The processor executes fourth program instructions to execute step:It is big by being stored in the interface polls distributed index
The posterior nodal point of result is normalized in the second of the first normalization result and is stored with less than the first normalization result
Third normalizes the front nodal point of result;
The processor executes the 5th program instruction to execute step:Pointer is constructed in the new node by the interface, it will
The pointer of the new node is set as the key of the posterior nodal point, and sets the pointer of the front nodal point to the new node
key。
Priority Applications (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN201710233466.XA CN108694209B (en) | 2017-04-11 | 2017-04-11 | Distributed index method based on object and client |
Applications Claiming Priority (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN201710233466.XA CN108694209B (en) | 2017-04-11 | 2017-04-11 | Distributed index method based on object and client |
Publications (2)
Publication Number | Publication Date |
---|---|
CN108694209A true CN108694209A (en) | 2018-10-23 |
CN108694209B CN108694209B (en) | 2021-11-19 |
Family
ID=63843322
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
CN201710233466.XA Active CN108694209B (en) | 2017-04-11 | 2017-04-11 | Distributed index method based on object and client |
Country Status (1)
Country | Link |
---|---|
CN (1) | CN108694209B (en) |
Cited By (1)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN109669822A (en) * | 2018-11-28 | 2019-04-23 | 平安科技(深圳)有限公司 | The creation method and computer readable storage medium of electronic device, spare memory pool |
Citations (6)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN1197570A (en) * | 1995-08-02 | 1998-10-28 | 汤姆森消费电子有限公司 | Linked list structure onscreen display |
CN1514976A (en) * | 1998-07-24 | 2004-07-21 | �ָ��� | Distributed computer data base system and method for object searching |
US20100161614A1 (en) * | 2008-12-22 | 2010-06-24 | Electronics And Telecommunications Research Institute | Distributed index system and method based on multi-length signature files |
CN101996217A (en) * | 2009-08-24 | 2011-03-30 | 华为技术有限公司 | Method for storing data and memory device thereof |
CN102779185B (en) * | 2012-06-29 | 2014-11-12 | 浙江大学 | High-availability distribution type full-text index method |
CN106095920A (en) * | 2016-06-07 | 2016-11-09 | 四川大学 | Distributed index method towards extensive High dimensional space data |
-
2017
- 2017-04-11 CN CN201710233466.XA patent/CN108694209B/en active Active
Patent Citations (6)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN1197570A (en) * | 1995-08-02 | 1998-10-28 | 汤姆森消费电子有限公司 | Linked list structure onscreen display |
CN1514976A (en) * | 1998-07-24 | 2004-07-21 | �ָ��� | Distributed computer data base system and method for object searching |
US20100161614A1 (en) * | 2008-12-22 | 2010-06-24 | Electronics And Telecommunications Research Institute | Distributed index system and method based on multi-length signature files |
CN101996217A (en) * | 2009-08-24 | 2011-03-30 | 华为技术有限公司 | Method for storing data and memory device thereof |
CN102779185B (en) * | 2012-06-29 | 2014-11-12 | 浙江大学 | High-availability distribution type full-text index method |
CN106095920A (en) * | 2016-06-07 | 2016-11-09 | 四川大学 | Distributed index method towards extensive High dimensional space data |
Non-Patent Citations (1)
Title |
---|
胡永奎: ""对象存储文件系统的设计与实现"", 《中国优秀硕士学位论文全文数据库 信息科技辑》 * |
Cited By (2)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN109669822A (en) * | 2018-11-28 | 2019-04-23 | 平安科技(深圳)有限公司 | The creation method and computer readable storage medium of electronic device, spare memory pool |
CN109669822B (en) * | 2018-11-28 | 2023-06-06 | 平安科技(深圳)有限公司 | Electronic device, method for creating backup storage pool, and computer-readable storage medium |
Also Published As
Publication number | Publication date |
---|---|
CN108694209B (en) | 2021-11-19 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
CN103067433B (en) | A kind of data migration method of distributed memory system, equipment and system | |
KR102195836B1 (en) | Method for managing index | |
CN109376156B (en) | Method for reading hybrid index with storage awareness | |
US9871727B2 (en) | Routing lookup method and device and method for constructing B-tree structure | |
CN103106286B (en) | Method and device for managing metadata | |
JP2018518733A (en) | File operation method and apparatus | |
CN106713250B (en) | Data access method and device based on distributed system | |
US11775480B2 (en) | Method and system for deleting obsolete files from a file system | |
WO2015118865A1 (en) | Information processing device, information processing system, and data access method | |
CN110147345A (en) | A kind of key assignments storage system and its working method based on RDMA | |
JP2001101042A (en) | System and method for managing data | |
WO2014190700A1 (en) | Method of memory access, buffer scheduler and memory module | |
CN109408539A (en) | Data manipulation method, device, server and storage medium | |
CN110059026A (en) | A kind of catalogue processing method, device and storage system | |
CN108694209A (en) | Object-based distributed index method and client | |
CN108804571B (en) | Data storage method, device and equipment | |
CN109492020A (en) | A kind of data cache method, device, electronic equipment and storage medium | |
CN106796588A (en) | The update method and equipment of concordance list | |
KR102214697B1 (en) | A computer program for providing space managrment for data storage in a database management system | |
CN111209304A (en) | Data processing method, device and system | |
KR100785774B1 (en) | Obeject based file system and method for inputting and outputting | |
JP2014174870A (en) | Virtual file access system, virtual file access method and virtual file access program | |
CN116821058B (en) | Metadata access method, device, equipment and storage medium | |
CN109617943B (en) | Data caching method and device, storage medium and computer equipment | |
CN116644006B (en) | Memory page management method, system, device, equipment and computer medium |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
PB01 | Publication | ||
PB01 | Publication | ||
SE01 | Entry into force of request for substantive examination | ||
SE01 | Entry into force of request for substantive examination | ||
GR01 | Patent grant | ||
GR01 | Patent grant | ||
TR01 | Transfer of patent right | ||
TR01 | Transfer of patent right |
Effective date of registration: 20220221 Address after: 550025 Huawei cloud data center, jiaoxinggong Road, Qianzhong Avenue, Gui'an New District, Guiyang City, Guizhou Province Patentee after: Huawei Cloud Computing Technology Co.,Ltd. Address before: 518129 Bantian HUAWEI headquarters office building, Longgang District, Guangdong, Shenzhen Patentee before: HUAWEI TECHNOLOGIES Co.,Ltd. |