CN109388610A - A kind of distributed meta data services migrating method and system of low latency - Google Patents
A kind of distributed meta data services migrating method and system of low latency Download PDFInfo
- Publication number
- CN109388610A CN109388610A CN201811002580.2A CN201811002580A CN109388610A CN 109388610 A CN109388610 A CN 109388610A CN 201811002580 A CN201811002580 A CN 201811002580A CN 109388610 A CN109388610 A CN 109388610A
- Authority
- CN
- China
- Prior art keywords
- meta data
- moves
- volume
- migrated
- sub
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Abstract
The present invention relates to a kind of distributed meta data services migrating method and systems of low latency, it include: to obtain distributed meta data service to move out end, it obtains distributed meta data service and moves into end, " metadata migration " is set by the state at end of moving out, and set " metadata reception " for the state for moving into end, move out end export be migrated sub- volume to move into end；The state that end modification of moving out is migrated sub- volume is " state transition ", and the state that end of moving out will be migrated file in sub- volume, which copies to, moves into end；Moving into end and being migrated the corresponding list item of sub- volume in the update of local meta data server mapping table is " local ", and it is " non-local " that end of moving out, which is migrated the corresponding list item of sub- volume in the update of local meta data server mapping table, moves out and local unloading is held to be migrated sub- volume.Thus the present invention can respond different types of distributed meta data service request in different phase after carrying out distributed meta data services migrating process stage by stage.
Description
Technical field
The present invention relates to computer field of storage, in particular to the distributed meta data services migrating method of a kind of low latency
And system.
Background technique
The essence of distributed meta data services migrating mechanism is switching and the transition process of collection of metadata administration authority
In metadata consistency guarantee.
When Metadata Service migration occurs, distributed meta data service provider (end of moving out) needs the member until itself
Data and status data are completely migrating to the moving into after end of distributed meta data service, can be to distributed meta data service
Requesting party provides distributed meta data service, and there are Grace Time delay issues for this method.Grace Time is to move into end etc.
The minimum time restored to all clients completion status.In the moving method, moves into end and have to wait all clients
Completion status could provide normal Metadata Service after restoring.The reason is that if some client is not completed
The recovery of lock status, and this when, other clients would potentially result in not if accessing to the corresponding metadata of lock status
Consistent share and access.PNFS is the high-performance enhancing of NFSv4, it is maintaining NFSv4 operating system and hardware platform independence
While provide direct memory access to parallel file system again.Grace Time is set as 90s in pNFS, this be
It can not put up in many application environments.
Ceph be one reliably, rebalancing method, the distributed memory system restored automatically automatically.For each canonical system
Metadata migration method, in terms of migrating the transparency, the Ceph transparency is good, and Storage Tank (storage pool) is transparent
Difference, using needing to interrupt.In terms of transition delay, we only consider to migrate transparent system.Son volume granularity version blue whale cluster text
There are Grace Time for part service system, and delay is huge, and Ceph is relatively fast, but causes to postpone there is also some shortcomings
It is high.Ceph uses two-phase commitment protocol to guarantee the consistency of transition process, needs moving into a large amount of record logs in end, phase
Work as time-consuming.
Based on the essence of distributed meta data services migrating, with reference to the principle of displacement of the above various canonical systems, in conjunction with grinding
Study carefully the design feature of the blue whale cluster file system prototype system of platform fine granularity version, the present invention is quasi- to realize that one is suitable for carefully
The blue whale cluster file system plateform system structure of granularity version, based on Metadata Service migration frame, point of low latency
Cloth Metadata Service migration mechanism.
Summary of the invention
In order to solve the above-mentioned technical problem, the present invention is in original blue whale cluster file system (BWFS) fine granularity version
On distributed meta data services migrating manufacturing basis, the migration volume in conjunction with distributed meta data services migrating each stage is proposed
State, to distributed meta data operate carry out classification processing method, solve distributed meta data services migrating and postponed
High problem provides a kind of distributed meta data services migrating mechanism of low latency.
Specifically, the invention discloses a kind of distributed meta data services migrating method of low latency, including:
Step 1 obtains distributed meta data service and moves out end, and as end of moving out, acquisition distributed meta data service is moved into
End sets the state at the end of moving out to " metadata migration " as end is moved into, and sets " member for the state for moving into end
Data receiver ", the end export of moving out are migrated sub- volume and move into end to this；
It is " state transition " that step 2, the end of moving out, which modify the state for being migrated sub- volume, which is migrated son for this
The state of file copies to this and moves into end in volume；
Step 3, this move into end local meta data server mapping table update this be migrated the corresponding list item of sub- volume be " this
Ground ", it is " non-local " which updates this to be migrated the corresponding list item of sub- volume in local meta data server mapping table, this is moved
Outlet locally unloads this and is migrated sub- volume.
The distributed meta data services migrating method of the low latency, wherein
The step 1 further include: after setting " metadata migration " for the state at the end of moving out, carry out this and be migrated sub- volume
The lower brush of metadata, this moves into end carry, and this is migrated sub- volume, this moves into end modification, and this is migrated the distributed information log of sub- volume
State is NORMAL, which is migrated after sub- volume moves into end to this, this moves into end modification, and this is migrated the shape of sub- volume
State is " local "；
The step 2 further include: the end of moving out replicate the state for being migrated file in sub- volume by remote procedure call
End is moved into this；
The step 3 further include: it is " not in local " that the state for being migrated sub- volume is modified at the end of moving out, the end release of moving out
This is migrated the state of file in sub- volume.
The distributed meta data services migrating method of the low latency, wherein when the step 1 is carrying out and receives distribution
It when metadata write request, returns to the first error code of service requester and this moves into the IP at end, service requester creation is long-range
Then service call client waits Data Migration to terminate；
After the step 2 is carrying out and receives the write request that the service requester retries, the service requester is returned to
Second error code, the service requester learn that metadata migration finishes, and the IP at end is moved into according to this, send this for the write request
Move into end.
The distributed meta data services migrating method of the low latency, wherein when the step 2 is carrying out and receives distribution
When metadata write request, returns to second error code of service requester and this moves into the IP at end, the service requester is according to this
The IP for moving into end sends this for the write request and moves into end.
The distributed meta data services migrating method of the low latency, wherein when the step 3 is carrying out and receives distribution
When metadata write request, returns to service requester third error code and move into the IP at end, which moves into according to this
The IP at end creates the rpc client for moving into end, sends this for write request redirection and moves into end.
The distributed meta data services migrating method of the low latency, wherein when the step 1 is carrying out and receives distribution
It when metadata read request, responds the read request and returns to the IP for moving into end, service requester receives after this moves into the IP at end, more
IP in new local meta data server mapping table where migration volume.
The distributed meta data services migrating method of the low latency, wherein when the step 2 is carrying out and receives distribution
When metadata read request, returns to the second error code of service requester and this moves into the IP at end, service requester is moved into according to this
The rpc client at end is moved into the IP creation at end, is sent this for read request redirection and is moved into end.
The distributed meta data services migrating method of the low latency, wherein when the step 3 is carrying out and receives distribution
When metadata read request, returns to service requester third error code and this moves into the IP at end.Service requester is moved into according to this
The IP at end creates the rpc client for moving into end, sends this for read request redirection and moves into end.
The invention also discloses a kind of distributed meta data services migrating system of low latency, including:
Metadata migration module is moved out end for obtaining distributed meta data service, as end of moving out, obtains distributed member
Data service moves into end, as end is moved into, sets the state at the end of moving out to " metadata migration ", and this is moved into the shape at end
State is set as " metadata reception ", which is migrated sub- volume and moves into end to this；
State transition module is migrated the state " state transition " of sub- volume for making the end of moving out modify this, this is moved out
The state for being migrated file in sub- volume is copied to this and moves into end by end；
Unload module, for make this move into end local mds_table update this be migrated the corresponding list item of sub- volume "
Ground ", it is " non-local " which updates this to be migrated the corresponding list item of sub- volume in local mds_table, this, which is moved out, holds local
It unloads this and is migrated sub- volume.
The distributed meta data services migrating system of the low latency, wherein
The metadata migration module further include: after setting " metadata migration " for the state at the end of moving out, carry out the quilt
The lower brush of the metadata of migration son volume, this moves into end carry, and this is migrated sub- volume, this moves into end modification, and this is migrated point of sub- volume
The state of cloth log is NORMAL, which is migrated after sub- volume moves into end to this, this moves into end modification, and this is moved
The state for moving son volume is " local "；
The state transition module further include: the state for being migrated file in sub- volume is passed through remote process tune by the end of moving out
End is moved into this is copied to；
The Unload module further include: it is " not in local ", the end of moving out that the state for being migrated sub- volume is modified at the end of moving out
Discharge the state for being migrated file in sub- volume；
Wherein, when the metadata migration module is currently running and receives distributed meta data write request, service is returned to
The first error code of requesting party moves into the IP at end with this, which creates remote service and call client, then waits number
Terminate according to migration；After the state transition module is currently running and receives the write request that the service requester retries, this is returned to
The second error code of service requester, the service requester learn that metadata migration finishes, and the IP at end is moved into according to this, this is write and is asked
It asks and is sent to this and moves into end；
When the state transition module is currently running and receives distributed meta data write request, the service requester is returned to
Second error code moves into the IP at end with this, which moves into the IP at end according to this, sends this for the write request and moves into
End；
When the Unload module is currently running and receives distributed meta data write request, it is wrong to return to service requester third
Error code and the IP for moving into end, the service requester create the rpc client for moving into end according to the IP for moving into end, should
Write request redirection is sent to this and moves into end；
When the metadata migration module is currently running and receives distributed meta data read request, responds the read request and return
It returns this and moves into the IP at end, service requester receives after this moves into the IP at end, updates this in local meta data server mapping table and moves
Move the IP where son volume；
When the state transition module is currently running and receives distributed meta data read request, service requester is returned to
Two error codes move into the IP at end with this, and service requester moves into the rpc client at end according to IP creation for moving into end, will
Read request redirection is sent to this and moves into end；
When the Unload module is currently running and receives distributed meta data read request, it is wrong to return to service requester third
Error code moves into the IP at end with this.Service requester creates the rpc client for moving into end according to the IP for moving into end, should
Read request redirection is sent to this and moves into end.
The present invention can be with as a result:
After carrying out distributed meta data services migrating process stage by stage, different types of point can be responded in different phase
The request of cloth Metadata Service avoids distributed meta data services migrating and postpones excessively high problem.
Detailed description of the invention
Fig. 1 is that distributed meta data reads operating lag figure；
Fig. 2 is distributed meta data write response delay figure；
Fig. 3 is that the stage distributed metadata of META-MIGRATING writes flow chart；
Fig. 4, which is that the stage distributed metadata of META-MIGRATING is another, writes flow chart；
Fig. 5 is that the stage distributed metadata of STATE-MIGRATING writes process；
Fig. 6 is that the stage distributed metadata of ABSENT writes flow chart；
Fig. 7 is that the stage distributed metadata of META-MIGRATING reads flow chart；
Fig. 8 is that the stage distributed metadata of STATE-MIGRATING reads flow chart；
Fig. 9 is that the stage distributed metadata of ABSENT reads flow chart.
Specific embodiment
To allow features described above and effect of the invention that can illustrate more clearly understandable, special embodiment below, and cooperate
Bright book attached drawing is described in detail below.
The present invention includes subdivision migration phase and response different type operates two parts stage by stage.
Segment migration phase:
Each stage of original distributed meta data services migrating process is transparent to distributed meta data service requester
, when distributed meta data service provider migrates, the access request of service requester can be blocked always until whole
A distributed meta data services migrating process finishes, and can just go distributed meta data service to move into end and carry out distributed meta data
Access request.
Therefore, it is necessary to each stages to distributed meta data service whole process to be further analyzed.
As shown in table 1 below, the first stage is named as META-MIGRATING by the present invention, and interval of definition is that [service is moved
The state of outlet is set as " metadata migration ", and the move out state at end of service is set as " state transition ").
In order to which the semanteme for the frame that keeps moving forward with original Metadata Service almost the same does not generate two in second stage
Justice, second stage is logically divided into two stages again by we:
STATE-MIGRATING, interval of definition are that [the move out state at end of service is set as that [" state transition ", service is moved
The state that outlet modification is migrated sub- volume fsx is " not in local ")；
ABSENT, interval of definition are that [service is moved out, and to be migrated the state of sub- volume fsx be " not in local " for end modification, is taken
Business, which is moved out, holds local unloading to be migrated sub- volume fsx].
Wherein moving out and holding and move into end is all meta data server.
1 transition process substage of table:
Original distributed meta data services migrating process is migrated sub- volume fsx to distribution after planning again
The case where metadata read-write requests respond illustrates 1, diagram 2.
Response different type operation stage by stage:
Definition to distributed meta data operation is: in more MDS frameworks, relating only to two MDS or more MDS
The metadata operation of (STATFS operation).More MDS frameworks: the framework is one a metadata volume and a data volume formatter
A file system, this is known as a file system volume.In group system, the granularity of each file system volume is smaller, with
Several file system volumes can be managed as each meta data server.
The case where TOUCH, RM two operations will not relate to two MDS.Other operations are had and are related to two MDS
Or the case where more MDS, therefore for the recipient of the request of second sub-operation of each operation, we can do with
Lower classification, as shown in table 2.Wherein TOUCH is the operation for creating file, and there are two its core sub-operations: 1. in parent directory
The directory entry of addition creation file.2. being newly created file application inode.
RM is the operation for deleting file, and there are two its core sub-operations: 1. delete deleted file in parent directory
Directory entry.2. discharging the inode of deleted file.
Distributed meta data read operation: STATFS.
Distributed meta data write operation: MKDIR, RMDIR, HARDLINK, RENAME.It is related that STATFS inquires file system
Information；MKDIR creates file；RMDIR Delete Folder；The hard link of HARDLINK creation source file；RENAME is original
File renaming.
Next a point situation discussion, META-MIGRATING, STATE- during distributed meta data services migrating are needed
The distributed meta data operation that MIGRATING, ABSENT three phases can respond.
The operation of each stage response of table 2
The situation analysis of distributed meta data write request and design
Service moves out end in the META-MIGRATING stage when distributed meta data write request.
When the request for receiving distributed meta data and writing (META_WRITE) type, service requester NFS4ERR_ is returned to
The IP of end MDS is moved into METAMIGRATING error code and service.Service requester creates rpc client after receiving new IP,
Then Data Migration is waited to terminate, as shown 3.Rpc client effect is for establishing RPC link with server-side.
RPC (Remote Procedure Call)-remote procedure call, it is that one kind passes through network from remote computer
Service is requested in program, the agreement without understanding underlying network technology.
Service move out metadata migration finish, into the STATE-MIGRATING stage, receive what service requester retried
After distributed meta data writes (META_WRITE) request, service requester NFS4ERR_METAABSENT error code is returned to.Clothes
Business requesting party learns that metadata migration finishes, and sends service for distributed meta data write request redirection and moves into end, as shown
4。
Service moves out end in the STATE-MIGRATING stage when distributed meta data write request.
When the request for receiving distributed meta data and writing (META_WRITE) type, service requester NFS4ERR_ is returned to
The IP of end MDS is moved into METAABSENT error code and service.Service requester creates the rpc that end is moved into service according to the IP of return
Client sends service for distributed meta data write request redirection and moves into end, and as shown 5.
Service moves out end in the ABSENT stage when distributed meta data write request.
When the request for receiving distributed meta data and writing (META_WRITE) type, service requester NFS4ERR_ is returned to
The IP of end MDS is moved into MIGRATECOMPLETE error code and service.Service requester creates service according to the IP of return and moves into end
Rpc client, send service for distributed meta data write request redirection and move into end, as shown 6.
The situation analysis of distributed meta data read request and design:
Service moves out end in the META-MIGRATING stage when distributed meta data read request.
Service is moved out when rectifying in the META-MIGRATING stage, is received distributed meta data and is read (META_READ) class
The request of type, respond request simultaneously return to the IP that end MDS is moved into service.Service requester receive service move into end MDS IP after, more
The IP of MDS where migration volume fsx in new local mds_table, as shown 7.
Service moves out end in the STATE-MIGRATING stage when distributed meta data read request.
Service, which is moved out, rectified in the STATE-MIGRATING stage, obtains distributed meta data and reads (META_READ) type
Request, return to service requester NFS4ERR_METAABSENT error code and the IP. for holding MDS moved into service
Service requester creates the rpc client that end is moved into service according to the IP of return, by distributed meta data read request
Redirection is sent to service and moves into end, and as shown 8.
Service moves out end in the ABSENT stage when distributed meta data read request.,
The metadata at end is moved out in service and status data is all had moved and finished, and is in ABSENT state, obtains distributed member
Data read the request of (META_READ) type, return to service requester NFS4ERR_MIGRATECOMPLETE error code kimonos
The IP of end MDS is moved into business.Service requester creates the rpc client that end is moved into service according to the IP of return, by distributed first number
Service, which is sent to, according to read request redirection moves into end, as shown 9.
The following are system embodiment corresponding with above method embodiment, present embodiment can be mutual with above embodiment
Cooperation is implemented.The relevant technical details mentioned in above embodiment are still effective in the present embodiment, in order to reduce repetition,
Which is not described herein again.Correspondingly, the relevant technical details mentioned in present embodiment are also applicable in above embodiment.
The invention also discloses a kind of distributed meta data services migrating system of low latency, including:
Metadata migration module is moved out end for obtaining distributed meta data service, as end of moving out, obtains distributed member
Data service moves into end, as end is moved into, sets the state at the end of moving out to " metadata migration ", and this is moved into the shape at end
State is set as " metadata reception ", which is migrated sub- volume and moves into end to this；
State transition module is migrated the state " state transition " of sub- volume for making the end of moving out modify this, this is moved out
The state for being migrated file in sub- volume is copied to this and moves into end by end；
Unload module, for make this move into end local mds_table update this be migrated the corresponding list item of sub- volume "
Ground ", it is " non-local " which updates this to be migrated the corresponding list item of sub- volume in local mds_table, this, which is moved out, holds local
It unloads this and is migrated sub- volume.
The distributed meta data services migrating system of the low latency, wherein
The metadata migration module further include: after setting " metadata migration " for the state at the end of moving out, carry out the quilt
The lower brush of the metadata of migration son volume, this moves into end carry, and this is migrated sub- volume, this moves into end modification, and this is migrated point of sub- volume
The state of cloth log is NORMAL, which is migrated after sub- volume moves into end to this, this moves into end modification, and this is moved
The state for moving son volume is " local "；
The state transition module further include: the state for being migrated file in sub- volume is passed through remote process tune by the end of moving out
End is moved into this is copied to；
The Unload module further include: it is " not in local ", the end of moving out that the state for being migrated sub- volume is modified at the end of moving out
Discharge the state for being migrated file in sub- volume；
Wherein, when the metadata migration module is currently running and receives distributed meta data write request, service is returned to
The first error code of requesting party moves into the IP at end with this, which creates remote service and call client, then waits number
Terminate according to migration；After the state transition module is currently running and receives the write request that the service requester retries, this is returned to
The second error code of service requester, the service requester learn that metadata migration finishes, and the IP at end is moved into according to this, this is write and is asked
It asks and is sent to this and moves into end；
When the state transition module is currently running and receives distributed meta data write request, the service requester is returned to
Second error code moves into the IP at end with this, which moves into the IP at end according to this, sends this for the write request and moves into
End；
When the Unload module is currently running and receives distributed meta data write request, it is wrong to return to service requester third
Error code and the IP for moving into end, the service requester create the rpc client for moving into end according to the IP for moving into end, should
Write request redirection is sent to this and moves into end；
When the metadata migration module is currently running and receives distributed meta data read request, responds the read request and return
It returns this and moves into the IP at end, service requester receives after this moves into the IP at end, updates this in local meta data server mapping table and moves
Move the IP where son volume；
When the state transition module is currently running and receives distributed meta data read request, service requester is returned to
Two error codes move into the IP at end with this, and service requester moves into the rpc client at end according to IP creation for moving into end, will
Read request redirection is sent to this and moves into end；
When the Unload module is currently running and receives distributed meta data read request, it is wrong to return to service requester third
Error code moves into the IP at end with this.Service requester creates the rpc client for moving into end according to the IP for moving into end, should
Read request redirection is sent to this and moves into end.
Inventive point of the present invention includes: 1, distributed meta data services migrating process segment mechanism, and distributed meta data is taken
Business transition process is subdivided into META-MIGRATING, STATE-MIGRATING, ABSENT three phases；2, to distributed first number
Classification mechanism is carried out according to operation, responds different types of distributed meta data service class stage by stage on the basis of segmenting migration phase
Type.
Although the present invention is disclosed with above-described embodiment, specific examples are only used to explain the present invention, is not used to limit
The present invention, any those skilled in the art of the present technique without departing from the spirit and scope of the invention, can make some change and complete
It is kind, therefore the scope of the present invention is subject to claims.
Claims (10)
1. a kind of distributed meta data services migrating method of low latency characterized by comprising
Step 1 obtains distributed meta data service and moves out end, as end of moving out, obtains distributed meta data service and moves into end, work
It to move into end, sets the state at the end of moving out to " metadata migration ", and sets the state for moving into end to that " metadata connects
Receive ", which is migrated sub- volume and moves into end to this；
It is " state transition " that step 2, the end of moving out, which modify the state for being migrated sub- volume, which is migrated this in sub- volume
The state of file copies to this and moves into end；
Step 3, this move into end local meta data server mapping table update this be migrated the corresponding list item of sub- volume be " local ",
It is " non-local ", the end of moving out that the end of moving out updates this to be migrated the corresponding list item of sub- volume in local meta data server mapping table
Locally unloading this is migrated sub- volume.
2. the distributed meta data services migrating method of low latency as described in claim 1, which is characterized in that
The step 1 further include: after setting " metadata migration " for the state at the end of moving out, carry out the first number for being migrated sub- volume
According to lower brush, this move into end carry this be migrated sub- volume, this move into end modification this be migrated the state of the distributed information log of sub- volume
For NORMAL, which is migrated after sub- volume moves into end to this, this, which is moved into end and modifies the state for being migrated sub- volume, is
" local "；
The step 2 further include: the state for being migrated file in sub- volume is copied to this by remote procedure call by the end of moving out
Move into end；
The step 3 further include: it is " not in local " that the state for being migrated sub- volume is modified at the end of moving out, which discharges the quilt
The state of file in migration son volume.
3. the distributed meta data services migrating method of low latency as claimed in claim 2, which is characterized in that when the step 1
When being carrying out and receiving distributed meta data write request, returns to the first error code of service requester and this moves into the IP at end,
The service requester creates remote service and calls client, then Data Migration is waited to terminate；
After the step 2 is carrying out and receives the write request that the service requester retries, the service requester second is returned to
Error code, the service requester learn that metadata migration finishes, and the IP at end is moved into according to this, send this for the write request and move into
End.
4. the distributed meta data services migrating method of low latency as claimed in claim 2, which is characterized in that when the step 2
When being carrying out and receiving distributed meta data write request, returns to second error code of service requester and this moves into end
IP, the service requester move into the IP at end according to this, send this for the write request and move into end.
5. the distributed meta data services migrating method of low latency as claimed in claim 2, which is characterized in that when the step 3
When being carrying out and receiving distributed meta data write request, returns to service requester third error code and move into the IP at end, it should
Service requester creates the rpc client for moving into end according to the IP for moving into end, sends this for write request redirection
Move into end.
6. the distributed meta data services migrating method of low latency as claimed in claim 2, which is characterized in that when the step 1
When being carrying out and receiving distributed meta data read request, responds the read request and return to the IP for moving into end, service requester
It receives after this moves into the IP at end, updates the IP in local meta data server mapping table where migration volume.
7. the distributed meta data services migrating method of low latency as claimed in claim 2, which is characterized in that when the step 2
When being carrying out and receiving distributed meta data read request, returns to the second error code of service requester and this moves into the IP at end,
Service requester moves into the rpc client at end according to the IP creation for moving into end, sends this for read request redirection and moves
Enter end.
8. the distributed meta data services migrating method of low latency as claimed in claim 2, which is characterized in that when the step 3
When being carrying out and receiving distributed meta data read request, returns to service requester third error code and this moves into the IP at end.
Service requester creates the rpc client for moving into end according to the IP for moving into end, sends this for read request redirection
Move into end.
9. a kind of distributed meta data services migrating system of low latency characterized by comprising
Metadata migration module is moved out end for obtaining distributed meta data service, as end of moving out, obtains distributed meta data
End is moved into service, as end is moved into, is set the state at the end of moving out to " metadata migration ", and the state for moving into end is set
It is set to " metadata reception ", which is migrated sub- volume and moves into end to this；
State transition module is migrated the state " state transition " of sub- volume for making the end of moving out modify this, which will
The state for being migrated file in sub- volume copies to this and moves into end；
Unload module updates this in local meta data server mapping table and is migrated the corresponding list item of sub- volume for making this move into end
For " local ", it is " non-that which updates this to be migrated the corresponding list item of sub- volume in local meta data server mapping table
Ground ", this is moved out to hold and local unload this and be migrated sub- volume.
10. the distributed meta data services migrating system of low latency as claimed in claim 9, which is characterized in that
The metadata migration module further include: after setting " metadata migration " for the state at the end of moving out, carry out this and be migrated
The lower brush of the metadata of son volume, this moves into end carry, and this is migrated sub- volume, this moves into end modification, and this is migrated the distribution of sub- volume
The state of log is NORMAL, which is migrated after sub- volume moves into end to this, this moves into end modification, and this is migrated son
The state of volume is " local "；
The state transition module further include: the end of moving out answer the state for being migrated file in sub- volume by remote procedure call
It makes this and moves into end；
The Unload module further include: it is " not in local " that the state for being migrated sub- volume is modified at the end of moving out, the end release of moving out
This is migrated the state of file in sub- volume；
Wherein, when the metadata migration module is currently running and receives distributed meta data write request, service request is returned to
The first error code of side moves into the IP at end with this, which creates remote service and call client, and then equal pending datas move
Shifting terminates；After the state transition module is currently running and receives the write request that the service requester retries, the service is returned to
The second error code of requesting party, the service requester learn that metadata migration finishes, and the IP at end is moved into according to this, which is sent out
It is sent to this and moves into end；
When the state transition module is currently running and receives distributed meta data write request, the service requester second is returned to
Error code moves into the IP at end with this, which moves into the IP at end according to this, sends this for the write request and moves into end；
When the Unload module is currently running and receives distributed meta data write request, service requester third error code is returned to
With the IP for moving into end, which creates the rpc client for moving into end according to the IP for moving into end, this is write and is asked
It asks redirection to be sent to this and moves into end；
When the metadata migration module is currently running and receives distributed meta data read request, responds the read request and return and be somebody's turn to do
Move into the IP at end, service requester receives after this moves into the IP at end, updates migration in local meta data server mapping table
IP where rolling up；
When the state transition module is currently running and receives distributed meta data read request, it is wrong to return to service requester second
Error code moves into the IP at end with this, and service requester moves into the rpc client at end according to IP creation for moving into end, by the reading
Request redirection is sent to this and moves into end；
When the Unload module is currently running and receives distributed meta data read request, service requester third error code is returned to
The IP at end is moved into this.Service requester creates the rpc client for moving into end according to the IP for moving into end, which is asked
It asks redirection to be sent to this and moves into end.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811002580.2A CN109388610A (en) | 2018-08-30 | 2018-08-30 | A kind of distributed meta data services migrating method and system of low latency |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811002580.2A CN109388610A (en) | 2018-08-30 | 2018-08-30 | A kind of distributed meta data services migrating method and system of low latency |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN109388610A true CN109388610A (en) | 2019-02-26 |
Family
ID=65418440
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201811002580.2A Pending CN109388610A (en) | 2018-08-30 | 2018-08-30 | A kind of distributed meta data services migrating method and system of low latency |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN109388610A (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111726372A (en) * | 2019-03-18 | 2020-09-29 | 阿里巴巴集团控股有限公司 | Thermal migration method, device, equipment and storage medium |
| CN111800441A (en) * | 2019-04-08 | 2020-10-20 | 阿里巴巴集团控股有限公司 | Data processing method, system, device, user side server, user side and management and control server |
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103793475A (en) * | 2014-01-06 | 2014-05-14 | 无锡城市云计算中心有限公司 | Distributed file system data migration method |
| CN106034080A (en) * | 2015-03-10 | 2016-10-19 | 中兴通讯股份有限公司 | Metadata migration method and metadata migration device in distributed system |
-
2018
- 2018-08-30 CN CN201811002580.2A patent/CN109388610A/en active Pending
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103793475A (en) * | 2014-01-06 | 2014-05-14 | 无锡城市云计算中心有限公司 | Distributed file system data migration method |
| CN106034080A (en) * | 2015-03-10 | 2016-10-19 | 中兴通讯股份有限公司 | Metadata migration method and metadata migration device in distributed system |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111726372A (en) * | 2019-03-18 | 2020-09-29 | 阿里巴巴集团控股有限公司 | Thermal migration method, device, equipment and storage medium |
| CN111726372B (en) * | 2019-03-18 | 2022-10-21 | 阿里巴巴集团控股有限公司 | Thermal migration method, device, equipment and storage medium |
| CN111800441A (en) * | 2019-04-08 | 2020-10-20 | 阿里巴巴集团控股有限公司 | Data processing method, system, device, user side server, user side and management and control server |
| CN111800441B (en) * | 2019-04-08 | 2022-10-11 | 阿里巴巴集团控股有限公司 | Data processing method, system, device, user side server, user side and management and control server |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| EP3596619B1 (en) | Methods, devices and systems for maintaining consistency of metadata and data across data centers | |
| JP6777673B2 (en) | In-place snapshot | |
| US11157517B2 (en) | Versioned hierarchical data structures in a distributed data store | |
| JP6553822B2 (en) | Dividing and moving ranges in distributed systems | |
| US7801850B2 (en) | System of and method for transparent management of data objects in containers across distributed heterogenous resources | |
| US10936547B2 (en) | Filesystem replication using a minimal filesystem metadata changelog | |
| CN100541441C (en) | A kind of snapshot creation of computer file system, deletion and amending method | |
| EP1696346B1 (en) | File system represented inside a database | |
| US20090112879A1 (en) | Simultanously accessing file objects through web services and file services | |
| US7698319B2 (en) | Database system management method, database system, database device, and backup program | |
| CN110799960A (en) | System and method for database tenant migration | |
| US10852996B2 (en) | System and method for provisioning slave storage including copying a master reference to slave storage and updating a slave reference | |
| US7054887B2 (en) | Method and system for object replication in a content management system | |
| CA2403044A1 (en) | Method and apparatus for storing changes to file attributes without having to store an additional copy of the file contents | |
| CN104462185A (en) | Digital library cloud storage system based on mixed structure | |
| EP1480130B1 (en) | Method and apparatus for moving data between storage devices | |
| US11151081B1 (en) | Data tiering service with cold tier indexing | |
| CN100498766C (en) | Mass file management system and method based on database | |
| CN109388610A (en) | A kind of distributed meta data services migrating method and system of low latency | |
| US20210349850A1 (en) | Managing snapshotting of a dataset using an ordered set of b+ trees | |
| CN109684270A (en) | Database filing method, apparatus, system, equipment and readable storage medium storing program for executing | |
| CN108536833A (en) | A kind of distributed, database and its construction method towards big data | |
| CN102360382B (en) | High-speed object-based parallel storage system directory replication method | |
| CN112115115B (en) | File moving method, equipment and storage medium based on data warehouse | |
| US10657105B2 (en) | Method and computer system for sharing objects |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| WD01 | Invention patent application deemed withdrawn after publication | ||
| WD01 | Invention patent application deemed withdrawn after publication |
Application publication date: 20190226 |