CN105991769A - Data migration method and device and edge node server - Google Patents

Data migration method and device and edge node server Download PDF

Info

Publication number
CN105991769A
CN105991769A CN201610224612.8A CN201610224612A CN105991769A CN 105991769 A CN105991769 A CN 105991769A CN 201610224612 A CN201610224612 A CN 201610224612A CN 105991769 A CN105991769 A CN 105991769A
Authority
CN
China
Prior art keywords
data migration
resource file
task
data
mapping relations
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN201610224612.8A
Other languages
Chinese (zh)
Other versions
CN105991769B (en
Inventor
葛明雷
吴林峰
康凯
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Shanghai Supreme Being Joins Information Technology Share Co Ltd
Original Assignee
Shanghai Supreme Being Joins Information Technology Share Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Shanghai Supreme Being Joins Information Technology Share Co Ltd filed Critical Shanghai Supreme Being Joins Information Technology Share Co Ltd
Priority to CN201610224612.8A priority Critical patent/CN105991769B/en
Publication of CN105991769A publication Critical patent/CN105991769A/en
Application granted granted Critical
Publication of CN105991769B publication Critical patent/CN105991769B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L67/00—Network arrangements or protocols for supporting network services or applications
    • H04L67/50—Network services
    • H04L67/56—Provisioning of proxy services
    • H04L67/563—Data redirection of data network streams
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L67/00—Network arrangements or protocols for supporting network services or applications
    • H04L67/01—Protocols
    • H04L67/10—Protocols in which an application is distributed across nodes in the network
    • H04L67/1001—Protocols in which an application is distributed across nodes in the network for accessing one among a plurality of replicated servers
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L67/00—Network arrangements or protocols for supporting network services or applications
    • H04L67/50—Network services
    • H04L67/56—Provisioning of proxy services
    • H04L67/568—Storing data temporarily at an intermediate stage, e.g. caching
    • H04L67/5682—Policies or rules for updating, deleting or replacing the stored data

Abstract

The invention provides a data migration method and device and an edge node server. The method comprises the steps that a data migration task in the unfinished state is acquired from a preset database and inserted in a data migration list; the data migration task in the unfinished state is acquired from the data migration list, and existence of a resource file corresponding to the data migration task in the unfinished state is judged; when the judgment result is yes, the modification time of the corresponding resource file is modified into modification time when the corresponding resource file is stored according to a preset second cache mapping relation; and when the judgment result is no, the edge node server in which the corresponding resource file is located is determined through a second load balancer according to the preset second cache mapping relation, and the corresponding resource file is grabbed from the determined edge node server and stored. According to the scheme, the data migration efficiency can be enhanced in edge clustering and capacity expanding and the resource can be saved.

Description

Data migration method and device, edge node server
Technical field
The present invention relates to content distributing network field, particularly relate to a kind of data migration method and device,
Edge joint server.
Background technology
Content distributing network (Content Delivery Network, CDN), by placing everywhere at network Node server, builds one layer of intelligent virtual network, by website on existing Internet basic Hold the network " edge " being published to closest to user, make user can obtain required content nearby, with slow Solve the situation of network congestion, improve the response speed of website.Solve due to the network bandwidth technically comprehensively Little, user's visit capacity is big, the unequal reason of network point distribution, causes user to access the response speed of website slow Problem.
The service cluster that edge cluster is made up of multiple stage fringe node and load equalizer.Wherein, edge Node is responsible for storing data and providing the data cached to client, and load equalizer is then according to default Load-balancing algorithm realizes the load balancing of edge node server.
Along with the increase of the data volume of caching in edge node server, increasing of visit capacity, in order to improve The response speed of website, needs to carry out edge cluster dilatation, i.e. increases new edge in edge cluster Node server.But, during edge cluster is carried out dilatation, original resource file and limit Caching mapping relations between edge node server change, it is therefore desirable to carry out cache resources file Migrate.
But, existing data migration method, when edge cluster dilatation, edge node server is direct From the memorizer of upper strata, capture data, not only add load and the bandwidth of upper strata memorizer, and data Migration time is longer, the problem that there is inefficiency and the wasting of resources.
Summary of the invention
The problem that the embodiment of the present invention solves is how to improve the efficiency of Data Migration when edge cluster dilatation, Save resource.
For solving the problems referred to above, embodiments provide a kind of data migration method, at edge When increasing fringe node service in cluster migrating resource file, described method includes: from default Data base is obtained from and is in the data migration task of unfinished state and is inserted in Data Migration list； Wherein, described data migration task includes that the resource file of correspondence is mapping pass according to the first default caching The information of URL corresponding during system's storage, described first caching mapping relations are to increase in described edge cluster Caching mapping relations between edge node server back edge node server and the resource file cached； From described data migration task list, obtain the data migration task being in unfinished state, and judge institute Whether the resource file stating the data migration task of unfinished state corresponding exists；When the resource determining correspondence In the presence of file, the modification time of the corresponding resource file captured is revised as the resource file of correspondence Modification time when storing according to the second default caching mapping relations；Wherein, described second caching reflects Relation of penetrating increases edge node server before edge node server by described edge cluster and is cached The caching mapping relations of all resource files；In the presence of determining that the resource file of correspondence is not, by the Two load equalizers are according to presetting the edge joint that the second caching mapping relations determine the resource file place of correspondence Point server, from determined by edge node server capture the resource file of correspondence and store, and will The modification time of the corresponding resource file captured is revised as the resource file of correspondence according to described second Modification time during caching mapping relations storage；Wherein, in described second load equalizer, storage has described Second caching mapping relations.
Alternatively, the described Data Migration being in unfinished state that is obtained from from default data base is appointed It is engaged in and is inserted in Data Migration list, including: obtain from described data base and be in unfinished state Data migration task；Data migration task based on described unfinished state, determines the resource file of correspondence Information；Solicited message is obtained to the URL of resource file determined by the first load-balanced server transmission, So that the information of the URL of resource file determined by described first load equalizer return；Wherein, institute State storage in the first load equalizer and have described first caching mapping relations；Resolve from the URL returned Go out the IP address of correspondence, and judge that parsed IP address is the most consistent with the IP address of self；When When the IP address of self determines consistent with the IP address parsed, the data of described unfinished state are moved Shifting task is added to described data migration task list.
Alternatively, described method also includes: by the data migration task in described data migration task list Status modifier for completing；Corresponding state is sent more newly requested to data migration server, so that institute State the data migration server status modifier by data migration task described in described data base for completing.
Alternatively, before being stored by the resource file captured, described method also includes: to institute The resource file captured carries out verifying and determine and is verified.
Alternatively, described method also includes: is obtained from from described data base and is in the number in completion status According to the task of migration, and it is inserted in data deletion task list as data deletion task；From described data Delete the data deletion task that obtains in task list, and determine the information of the resource file of correspondence；By One load equalizer determines the resource file of correspondence place when according to described first caching mapping relations storage The information of edge node server；In described first load equalizer, storage has described first caching to map Relation；When determine from as determined by edge node server time, from described data deletion task list Data deletion task described in middle deletion；When determine self be not determined by edge node server time, Corresponding resource file is deleted；State is sent more newly requested to data migration server, so that described The state of the data migration task of corresponding completion status is set to delete successfully by data migration server.
Alternatively, described being obtained from from data base is in the data migration task in completion status, and makees It is inserted in data deletion task list for data deletion task, including: from described data base at acquisition In the data migration task of completion status, and determine the resource literary composition corresponding with acquired data migration task The information of part；Determine that the resource file of correspondence is according to default second by described second load equalizer The information of the edge node server at place during caching mapping relations storage；When determine from as determined by During edge node server, acquired data migration task is inserted into described as data deletion task In data deletion task list.
The embodiment of the present invention additionally provides a kind of data migration device, for increasing edge in edge cluster During node serve migrating resource file, described device includes: the first acquiring unit, is suitable to from advance If data base in be obtained from be in unfinished state data migration task and be inserted into Data Migration row In table；Wherein, described data migration task includes that the resource file of correspondence is according to the first default caching The information of URL corresponding during mapping relations storage, described first caching mapping relations are described edge cluster Caching between middle increase edge node server back edge node server and the resource file cached reflects Penetrate relation；First judging unit, is suitable to obtain to be in from described data migration task list be not fully complete shape The data migration task of state, and judge the resource file that the data migration task of described unfinished state is corresponding Whether exist；Data migration unit, is suitable in the presence of the resource file determining correspondence, by captured right The modification time of the resource file answered is revised as the resource file of correspondence and is mapping according to the second default caching Modification time during relation storage；Wherein, described second caching mapping relations are that described edge cluster increases Edge node server before edge node server maps with the caching of all resource files cached Relation；In the presence of determining that corresponding resource file is not, by the second load equalizer according to presetting second Caching mapping relations determine the edge node server at resource file place of correspondence, from determined by edge Node server captures the resource file of correspondence and stores, and by the corresponding resource file that captured Modification time is revised as the amendment when according to described second caching mapping relations storage of the resource file of correspondence Time；Wherein, in described second load equalizer, storage has described second caching mapping relations.
Alternatively, described first acquiring unit is suitable to obtain from described data base be in unfinished state Data migration task；Data migration task based on described unfinished state, determines the resource file of correspondence Information；Solicited message is obtained to the URL of resource file determined by the first load-balanced server transmission, So that the information of the URL of resource file determined by described first load equalizer return；Wherein, institute State storage in the first load equalizer and have described first caching mapping relations；Resolve from the URL returned Go out the IP address of correspondence, and judge that parsed IP address is the most consistent with the IP address of self；When When the IP address of self determines consistent with the IP address parsed, the data of described unfinished state are moved Shifting task is added to described data migration task list.
Alternatively, described device also includes: state updating unit, by described data migration task list The status modifier of data migration task for completing；State updates request unit, is suitable to take to Data Migration It is more newly requested, so that described data migration server is by described data base that business device sends corresponding state The status modifier of described data migration task is for completing.
Alternatively, described data migration unit is suitable to verifying the resource file captured, and really When being surely verified, the resource file captured is stored.
Alternatively, described device also includes: second acquisition unit, is suitable to be obtained from from described data base It is in the data migration task in completion status, and is inserted into data deletion task as data deletion task In list；First determines unit, is suitable to obtain data deletion task from described data deletion task list, And determine the information of the resource file of correspondence；Second determines unit, is suitable to by the first load equalizer true Fixed corresponding resource file is the fringe node service at place when according to described first caching mapping relations storage The information of device；In described first load equalizer, storage has described first caching mapping relations；Data deletion Unit, be suitable to when determine from as determined by edge node server time, from described data deletion task List is deleted described data deletion task；When determine self be not determined by edge node server Time, corresponding resource file is deleted；To described more newly requested to data migration server transmission state, So that the state of the data migration task of corresponding completion status is set to by described data migration server Delete successfully.
Alternatively, described second acquisition unit is suitable to obtain the number being in completion status from described data base According to the task of migration, and determine the information of the resource file corresponding with acquired data migration task；Pass through Described second load equalizer determines that the resource file of correspondence is being deposited according to the second default caching mapping relations The information of the edge node server at place during storage；When determine from as determined by edge node server Time, acquired data migration task is inserted into described data deletion task row as data deletion task In table.
The embodiment of the present invention additionally provides a kind of edge node server, including above-mentioned data migration device.
Compared with prior art, technical scheme has the advantage that
Above-mentioned scheme, in edge cluster dilation process, sets up corresponding number for corresponding resource file According to the task of migration, obtained by edge node server acquired in data migration task the execution of self Data migration task, thus realize the money in edge cluster according to the first caching mapping relations after dilatation Source file is buffered in the edge node server of correspondence, owing to above-mentioned process only needs part resource File carries out migrating rather than again capturing from the memorizer of upper strata the resource file of correspondence, can save The time of Data Migration, it is possible to alleviate load and the bandwidth resources of upper strata memorizer, therefore, it can carry The efficiency of high Data Migration, and save resource.
Accompanying drawing explanation
Fig. 1 is the flow chart of a kind of data migration method in the embodiment of the present invention；
Fig. 2 is the composition structural representation of the edge cluster in the embodiment of the present invention；
Fig. 3 is the flow chart of the data access request processing method of the edge cluster shown in Fig. 2；
Fig. 4 is the flow chart of the acquisition methods of a kind of data migration task in the embodiment of the present invention；
Fig. 5 is the flow chart of the processing method of a kind of data migration task in the embodiment of the present invention；
Fig. 6 is the flow chart of the acquisition methods of a kind of data deletion task in the embodiment of the present invention；
Fig. 7 is the flow chart of the processing method of a kind of data deletion task in the embodiment of the present invention；
Fig. 8 is the structural representation of a kind of data migration device in the embodiment of the present invention.
Detailed description of the invention
As background technology is sayed, existing data migration method, when edge cluster dilatation, edge saves Point server directly captures data from the memorizer of upper strata, not only adds load and the band of upper strata memorizer Width, and the Data Migration time is longer, the problem that there is inefficiency and the wasting of resources.
For solving the above-mentioned problems in the prior art, the technical scheme that the embodiment of the present invention uses is passed through In advance the action of event with corresponding execution is associated, with when meeting corresponding trigger condition, automatically Obtain action group corresponding to described event and perform, the efficiency of server O&M can be improved, promoting user Experience.
Understandable, below in conjunction with the accompanying drawings for enabling the above-mentioned purpose of the present invention, feature and advantage to become apparent from The specific embodiment of the present invention is described in detail.
Fig. 1 shows the flow chart of a kind of data migration method in the embodiment of the present invention.Refer to Fig. 1 Shown in, in being embodied as, the data migration method in the embodiment of the present invention can comprise the following steps that
Step S101: be obtained from from default data base and be in the data migration task in unfinished state And be inserted in Data Migration list.
In being embodied as, described data migration task includes that the resource file of correspondence is according to default the The information of URL corresponding during one caching mapping relations storage, described first caching mapping relations are described limit Edge cluster increases between edge node server back edge node server and the resource file cached Caching mapping relations.
Step S102: obtain the data migration task being in unfinished state from data migration task list, And judge whether the resource file that the data migration task of described unfinished state is corresponding exists.
In being embodied as, described second caching mapping relations are that described edge cluster increases fringe node clothes The caching of all resource files that the edge node server before business device is cached with edge cluster maps and closes System.
Step S103: the resource file corresponding when the data migration task determining described unfinished state exists Time, the modification time of corresponding resource file is revised as the resource file of correspondence according to default second Modification time during caching mapping relations storage.
In being embodied as, when the resource file that the data migration task determining described unfinished state is corresponding In the presence of, show resource file edge joint at place when according to the first caching mapping relations storage of correspondence Point server and this node when the edge node server at place is equal when storing according to the second caching mapping relations Edge node server, now, the data that these several data migration task are corresponding are considered as migrating successfully, At this point it is possible to the modification time of the resource file of described correspondence is revised as the modification time preset.
Step S104: when determine correspondence resource file not in the presence of, by the second load equalizer according to Preset the second caching mapping relations and determine the edge node server at resource file place of correspondence, from really Fixed edge node server captures the resource file of correspondence and stores, and the corresponding money that will be captured The modification time of source file is revised as the resource file of correspondence according to described second caching mapping relations storage Time modification time.
In being embodied as, in described second load equalizer, storage has described second caching mapping relations, I.e. before edge cluster dilatation, cached mapping relations to edge collection by the second load equalizer according to described second Edge node server before group's dilatation is scheduling.
Above-mentioned scheme, in edge cluster dilation process, sets up corresponding number for corresponding resource file According to the task of migration, obtained by edge node server acquired in data migration task the execution of self Data migration task, thus realize the money in edge cluster according to the first caching mapping relations after dilatation Source file is buffered in the edge node server of correspondence, owing to above-mentioned process only needs part resource File carries out migrating rather than again capturing from the memorizer of upper strata the resource file of correspondence, can save The time of Data Migration, it is possible to alleviate load and the bandwidth resources of upper strata memorizer, therefore, it can carry The efficiency of high Data Migration, and save resource, promote the experience of user.
Below in conjunction with the Fig. 2 to Fig. 7 process side to a kind of data access request in the embodiment of the present invention Method is further described in detail.
For the ease of describing, the edge cluster in implementing the present invention below in conjunction with Fig. 2 is detailed Jie Continue.
Shown in Figure 2, in being embodied as, the edge cluster in the embodiment of the present invention can include Multiple edge node server, the first load equalizer 102 and the second load equalizer 103.
In being embodied as, the first load equalizer 102 stores and has the first caching mapping relations, second In load equalizer 103, storage has the second caching mapping relations.Wherein, the first caching mapping relations are institute State edge cluster and increase all of edge node server and caching after edge node server i.e. dilatation Caching mapping relations between all resource files, the second caching mapping relations are for increasing fringe node service All edge node server in the leading edge cluster of device namely dilatation and all resource files cached Between caching mapping relations.
Continuing with seeing Fig. 2, edge cluster has three edge of table node server 101a before dilatation, expands The edge node server newly increased after appearance is three edge of table node server 101b.Wherein, the first load Equalizer 102 is installed on the platform in six edge of table node servers in edge cluster, i.e. from three edge of table Edge node server 101a and three edge of table node server 101b chooses one and installs the first load balancing Device 102.First load equalizer 102 dispatches six edge of table node clothes according to described first caching mapping relations Business device returns requested resource file to client.Before second load equalizer 103 is installed only at dilatation Edge node server on, in i.e. three edge of table node server 101a, it is according to described Second caching mapping relations are dispatched three edge of table node server 101a and are returned requested resource to client File.
Meanwhile, before edge cluster dilatation, edge cluster uses corresponding with the second load equalizer the The resource file access request of client C is monitored by Two-port netwerk, and such as 8080 ports, to client The resource file access request sent is monitored；After edge cluster dilatation, newly-increased in edge cluster First load equalizer 102, and it is provided with the first port that the first load equalizer 102 is corresponding, as 80 ports, monitor with the resource file access request sending client.
It is to be herein pointed out before edge cluster dilatation completing Data Migration, i.e. after dilatation All edge node server cache correspondence in described edge cluster according to described first caching mapping relations Before resource file, when user is by the resource file of client request correspondence, the portion in edge cluster Divide resource file may migrate to the fringe node service of correspondence according to described first caching mapping relations On device, another part resource file then may be stored in still according to original second caching mapping relations In edge node server before dilatation.Now, when the resource literary composition that the first port snoop to client sends During part access request, the first load equalizer is by according to owning after the first caching mapping relations scheduling dilatation Edge node server, may result in and the access request of client is dispatched to do not have requested resource In the edge node server of file, cause to return requested resource file, impact to client The experience of user.
For solving above-mentioned problem, corresponding for edge node server all in edge cluster are provided with Data access request processing means, to realize being dispatched to be asked by the resource file access request of client Resource file place edge node server on, it is ensured that the resource cached in edge cluster normal Access, promote the experience of user, specifically refer to Fig. 3.
Data access request method shown in Figure 3, in being embodied as, in the embodiment of the present invention Can comprise the following steps that
Step S301: by the resource file access request of the first port snoop client.
In being embodied as, when user sends corresponding resource file by client, can first by The resource file access request that client is sent by the first port is monitored.Wherein, real in the present invention one Executing in example, described first port is 80 ports.
Step S302: determined according to described first caching mapping relations by the first default load equalizer The edge node server at requested resource file place.
In being embodied as, when the resource file access request sent by the first port snoop to client Time, the resource file access request of received client is sent to being provided with the first load equalizer Edge node server so that the edge node server being provided with the first load equalizer can be by The URL that requested resource file is corresponding is returned according to described first caching mapping relations The information of (Uniform Resource Loc101ator, URL).Data access request processing means is obtaining When getting the URL information that the first load equalizer returns, obtained URLs information can be resolved, To determine that requested resource file is according to the fringe node at place during described first caching mapping relations storage Server, and the resource file access request of client is sent to according to described first caching mapping relations Determined by edge node server.
Step S303: judge that requested resource file is caching what mapping relations determined according to described first Whether edge node server device exists；When judged result is for being, perform step S304, otherwise, Then can perform step S305.
In being embodied as, receiving according to edge node server determined by the first caching mapping relations During to the URL information of requested resource file, received URL can be cached with self The URL of all resource files contrast one by one, to determine whether there is requested resource file.
Step S304: by the edge node server determined according to described first caching mapping relations to client End returns requested resource file.
In being embodied as, determining according to edge node server determined by the first caching mapping relations In when there is requested resource file, by according to fringe node clothes determined by the first caching mapping relations Business device directly returns requested resource file to client.
Step S305: return the address information of the second load equalizer preset to described client, so that Obtain described client described resource file access request to be sent to described second by the second port preset Load equalizer.
In being embodied as, when requested resource file is according to described first caching mapping relations institute really In fixed edge node server not in the presence of, show that requested resource file not yet migrates to according to During one caching mapping relations storage in the edge node server at place.Now, map according to the first caching The edge node server that relation determines can return 302 to client, will described second load equalizer The information of URL return to client so that client by described resource file access request by the Two-port netwerk sends to described second load equalizer.Wherein, in an embodiment of the present invention, described second Port is 8080 ports.
Step S306: determine by the second load equalizer asked according to described second caching mapping relations The edge node server at resource file place.
In being embodied as, when by the resource file access request of the second port snoop to client, The resource file access request of client is sent to being provided with the second load by data access request processing means The edge node server of equalizer, so that the second load equalizer can reflect according to described second caching Relation of penetrating returns the information of URL corresponding to requested resource file to data request processing device.Data Access request processing means, can be to being obtained when getting the URL information that the second load equalizer returns Take URL information to resolve, to determine that requested resource file is mapping pass according to described second caching The information of the edge node server at place during system's storage.
Step S307: judge according in edge node server determined by described second caching mapping relations Whether there is requested resource file；When judged result is for being, step S308 can be performed；Otherwise, Then can perform step S309.
In being embodied as, receiving according to edge node server determined by the second caching mapping relations During to the URL information of requested resource file, received URL can be delayed with according to second The URL depositing all resource files that mapping relations are buffered in self contrasts, to determine according to second Whether edge node server determined by caching mapping relations exists requested resource file.
Step S308: cached edge node server determined by mapping relations to visitor by according to described second Family end returns requested resource file.
In being embodied as, determining according to edge node server determined by the second caching mapping relations In when there is requested resource file, by according to fringe node clothes determined by the second caching mapping relations Business device directly returns requested resource file to client end.
Step S309: cached the upper of edge node server determined by mapping relations by according to described second One node layer server returns requested resource file to client.
In being embodied as, when requested resource file is according to described second caching mapping relations institute really In fixed edge node server the most not in the presence of, show that client requested resource file is at edge Cluster does not exists.Now, can be to according to the second caching edge node server of determining of mapping relations Client returns corresponding return code 302, i.e. sends according to described second caching mapping relations institute to client The information of the URL of the last layer node server of the edge node server determined, so that client will Described resource file access request sends to described last layer node server i.e. upper strata storage server, Other edge clusters at requested resource file place are determined by upper strata storage server, and by being determined Other edge clusters return requested resource file to client, thus realize the success of resource file Access.
By above-mentioned introduction, the data access request processing method in the embodiment of the present invention is permissible During edge cluster dilatation, it is dispatched to exactly be asked by the resource file access request of client In the edge node server at the resource file place asked, visit such that it is able to provide the user normal data Ask service, promote the experience of user.
Edge cluster described in Fig. 2 is when carrying out Data Migration, and edge node server can use existing Method from upper strata storage server, directly capture corresponding resource file store.But, from upper Layer storage server directly captures data, by the problem of following for existence three aspects:
One, directly captures whole resource files from the memorizer of upper strata, so will expend more depositing Storage resource, needs to be equipped with corresponding bandwidth resources for upper strata storage server, there is the wave of bandwidth resources The problem taken.
Its two, from upper strata storage server, directly capture corresponding resource file, that is to say by script the most Resource file through all capturing captures from upper strata memorizer the most again, and this will expend the longer time, because of This, also exist the problem that data migration efficiency is low.
Its two, when edge node server is when capturing corresponding resource file and storing, need edge The service of cluster is closed and restarts, and to close the second load equalizer, and makes the edge after restarting The resource file access request that client is sent by cluster by the first load equalizer is dispatched.Above-mentioned During, the interruption of service will certainly be caused so that the resource file access request of client during this period Will be unable to scheduling, have impact on the experience of user.
To this end, embodiments provide a kind of data migration method, before carrying out Data Migration, generate With each resource file data migration task one to one in edge cluster be stored in default data In storehouse.Wherein, the data migration task that each resource file is corresponding includes that described resource file is according to presetting The first caching mapping relations storage time the information of URL, and the information of the execution status of task of correspondence. When carrying out Data Migration, each edge node server in edge cluster directly obtains from data base The data migration task of self correspondence, and by performing corresponding data migration task, capture corresponding money Source file, is carried out according to described first caching mapping relations realizing all resource files in edge cluster Storage, such that it is able to save the bandwidth of upper layer node server, improves data migration efficiency, and promotes use The experience at family, specifically refers to Fig. 4 to Fig. 7.
For the ease of describing, how will obtain data migration task, how to perform acquired data below Three aspects of the deletion of migration task and redundant data, do the data migration method in the embodiment of the present invention Further details of introduction.
Fig. 4 shows the flow chart of the acquisition methods of a kind of data migration task in the embodiment of the present invention. Shown in Figure 4, in being embodied as, the acquisition side of the data migration task in the embodiment of the present invention Method, can comprise the following steps that
Step S401: obtain the data migration task being in unfinished state from data base.
In being embodied as, the edge node server of this node can be according to the default time cycle to number Job enquiry request is sent according to storehouse, to obtain the data migration task being in unfinished state in data base, And be stored in the internal memory of self.Wherein, the described default time cycle can enter according to the actual needs Row is arranged, such as, be set to 3 hours, and the present invention does not limits at this.
Step S402: to the resource that the first load equalizer request is corresponding with acquired data migration task The information of the URL of file.
In being embodied as, when getting the data migration task of correspondence, can move from corresponding data Shifting task determines the information of the resource file of correspondence, and is determined to the first load equalizer transmission acquisition The GET request of URL of resource file, with resource literary composition determined by sending to the first load equalizer The URL request that part is corresponding.First load equalizer is when receiving described GET request, according to being deposited First caching mapping relations of storage be calculated determined by respective file corresponding after edge cluster dilatation The information of URL.
Step S403: parse the information of the IP address of correspondence from the URL information asked.
In being embodied as, edge node server is receiving the URL letter that the first load equalizer returns During breath, the URL information returned can be resolved, to determine that the resource file of correspondence is according to institute The information of the IP address of the node server at place when stating the first caching mapping relations storage.
Step S404: judge that the IP address parsed is the most consistent with the IP address of self；Work as judgement When result is for being, step S405 can be performed；Otherwise, then can restart to perform from step S401.
Step S405: acquired data migration task is added to the data migration task list of self.
In being embodied as, the edge node server of this node can by the IP address that parsed with from The IP address of body compares, and when determining that the two is consistent, illustrates that the resource file of correspondence is according to first Edge node server this node just at place during caching mapping relations storage, now, the limit of this node Described data migration task can be added to the Data Migration list of self by edge node server, with logical Crossing the data migration task performed in Data Migration list, from edge cluster, the fringe node before dilatation takes Business device captures the resource file of correspondence.
Otherwise, the edge node server of this node is determining the IP ground with self, the parsed IP address When location is inconsistent, resource file limit at place when according to the first caching mapping relations storage of correspondence is described Edge node server is not this node.Now, edge node server can be by described data migration task Delete.
In being embodied as, when obtaining this node by the data migration task acquisition methods shown in Fig. 4 Data migration task also adds after the data migration task list of self, can be by performing described number According to the data migration task migrated in task list, the resource file capturing correspondence stores, with reality Existing corresponding resource file, according to the storage of the first caching mapping relations, specifically refers to Fig. 5.
Fig. 5 show the present invention implement in the flow chart of a kind of data migration task processing method.Such as Fig. 5 Shown in, the processing method of the data migration task in the embodiment of the present invention, can comprise the following steps that
Step S501: obtain from the Data Migration list of self and be in the Data Migration of unfinished state and appoint Business.
In being embodied as, the edge node server of this node can arrange from Data Migration in a different manner The data migration task obtaining correspondence in table processes.Such as, the edge node server of this node can In the way of employing is single-threaded, from Data Migration list, once obtain a number being in unfinished state Process according to migration task；Or, it is also possible to adopt the mode of multithreading, obtain a plurality of place the most simultaneously Data migration task in unfinished state processes, with the raising efficiency of Data Migration.
Step S502: whether the resource file that data migration task acquired in judgement is corresponding exists at self； When judged result is for being, step S503 and S504 can be performed；Otherwise, then step S505 can be performed.
In being embodied as, the edge node server of this node can be not fully complete according to acquired being in The information of the data migration task of state, determines the information of the resource file of correspondence, i.e. corresponding resource literary composition The information of the URL of part, and by the URL being determined the resource file of correspondence and the resource self stored The URL of file compares, to determine that resource file that acquired data migration task is corresponding is at self Whether exist.
Step S503: be revised as the modification time of corresponding resource file reflecting according to the second default caching Penetrate modification time during relation storage.
In being embodied as, the edge node server of this node is determining acquired data migration task Corresponding resource file, in the presence of self, illustrate that data are storing time institute according to the first caching mapping relations Edge node server with according to second caching mapping relations storage time place edge node server It is the edge node server of this node.Therefore, will be without capturing from other edge node server This resource file, this resource file has been realized in storing according to the first default caching mapping relations, I.e. it is considered as Data Migration success.Now, at the resource file determining that acquired data migration task is corresponding In the presence of self, can directly perform to be revised as the modification time of this resource file taking from upper strata first Business device node i.e. father's memory node captures the modification time of this resource file, i.e. realizes according to the second caching Modification time during mapping relations storage.
Step S504: deleted from Data Migration list by described data migration task, takes to Data Migration It is more newly requested that business device sends task status, so that described data migration server is by correspondence in data base Data migration task has been updated to.
In being embodied as, when corresponding resource file is considered as migrating successful, the fringe node of this node Described data migration task is deleted from Data Migration list by server, and sends out to data migration server Task status is sent to update solicited message, so that described data migration server can will store in data base The task status of corresponding data migration task be set to successfully, to avoid identical data to migrate task Repeat, and be easy to the implementation status of the data migration task in data base is controlled.
Step S505: determine that the resource file of correspondence is according to described second caching by the second load equalizer The information of URL during mapping relations storage.
In being embodied as, the edge node server of this node is determining acquired data migration task Corresponding resource file self not in the presence of, illustrate that data are when storing according to the first caching mapping relations The edge node server at place and the fringe node service at place when storing according to the second caching mapping relations Device is different, accordingly, it would be desirable to the fringe node at place in time storing according to the second default caching mapping relations Server captures this resource file.
Now, the edge node server of this node can send this resource file to the second load equalizer The information of URL corresponding when according to the second caching mapping relations storage, to return from the second load equalizer The corresponding URL information returned parses the information of the IP address of the edge node server of correspondence, namely Determine this resource file edge node server at place when according to described second caching mapping relations storage The information of IP address.
Step S506: capture the resource literary composition of correspondence the edge node server that URL determined by from is corresponding Part.
In being embodied as, the corresponding URL that the second load equalizer returns is to open with corresponding IP address Head, this IP address is i.e. that this resource file is according to the fringe node at place during the second caching mapping relations storage The IP address of server.Therefore, the edge node server of this node is receiving the second load equalizer During the corresponding URL returned, by the URL returned is resolved, just can obtain this resource The IP address of the edge node server at place when file stores according to the second caching mapping relations, and from this The edge node server that IP address is corresponding captures this resource file and stores.
Step S507: the resource file captured is verified, and judge to verify whether by storage institute The resource file captured.
In being embodied as, when grabbing the resource file of correspondence, in order to ensure the resource literary composition captured During the reliability of part, first the resource file captured can be carried out by the edge node server of this node Checking, as performed MD5 checking, and when being verified, stores the resource file captured, Thus completing the Data Migration of this resource file, it is achieved this resource file is according to the first caching mapping relations Storage.
When being stored by the resource file captured, can then perform step S503 and S504, with It is revised as the modification time of this resource file from top service device node, capturing this resource file first Modification time, and delete by data migration task corresponding in data migration task list, pass through simultaneously The state of data migration task corresponding in data base is updated to successfully by data migration server.
By above-mentioned introduction, from data base, obtain self correspondence by edge node server Data migration task also performs, and just can realize all resource files in edge cluster before dilatation The migration of edge node server edge node server after dilatation.Adopt in this way, on the one hand Just the data of resource file can be realized without again capturing resource file from the memory node of upper strata Migrate, therefore, it can save the bandwidth of upper strata memory node；On the other hand, owing to only needing the portion of carrying out The migration of point resource file, and without from upper strata memory node by already all in edge cluster Resource file captures one time the most again, therefore, it can save the time of Data Migration, improves Data Migration Efficiency.Meanwhile, at the end of above-mentioned Data Migration, i.e. realize all resources literary composition in edge cluster When part is according to the first caching mapping relations storage, it is not necessary to edge cluster is restarted, therefore, it is possible to be User provides normal data access demand, promotes the experience of user.
In being embodied as, when carrying out Data Migration and completing, owing to the edge before edge cluster dilatation saves In point server, still storage has the data that part repeats, and the data that this part repeats are according to described second Caching mapping relations are stored in the edge node server of this node, according to the first caching mapping relations During storage, the resource file of this part is stored in the edge node server of other nodes.Therefore, in order to save Resource-saving, the data that can repeat this part are deleted, and to save storage resource, specifically refer to Fig. 6 and Fig. 7.
Fig. 6 shows the flow chart of the acquisition methods of a kind of data deletion task in the embodiment of the present invention. Shown in Figure 6, in being embodied as, the acquisition side of the data deletion task in the embodiment of the present invention Method can comprise the following steps that
Step S601: obtain the data migration task being in completion status from data base.
In being embodied as, edge node server can send to data base according to the default time cycle Job enquiry is asked, and to obtain the data migration task being in completion status in data base, and is stored in certainly In the internal memory of body.Wherein, the described default time cycle can be configured according to the actual needs, this Invention does not limits at this.
Step S602: to the resource that the second load equalizer request is corresponding with acquired data migration task The information of the URL of file.
In being embodied as, when getting the data migration task being in completion status in data base, can To determine the information of the resource file of correspondence from corresponding data migration task, and to the second load balancing Device sends the GET request of URL, with to resource literary composition corresponding determined by the second load equalizer request The information of the URL of part.Second load equalizer is when receiving described GET request, according to being stored The second caching mapping relations be calculated the information of URL of correspondence, and return to the edge joint of this node Point server.
Step S603: parse the information of the IP address of correspondence from the URL information asked.
In being embodied as, the edge node server of this node is receiving the second load equalizer return URL information time, the URL information that can be returned resolves, with determine correspondence resource file The information of the IP address of the node server at place when according to described second caching mapping relations storage.
Step S604: judge that the IP address parsed is the most consistent with the IP address of self；Work as judgement When result is for being, step S605 can be performed；Otherwise, then can restart to perform from step S601.
Step S605: acquired data migration task is inserted in the data deletion task list of self.
In being embodied as, the edge node server of this node can by the IP address that parsed with from The IP address of body compares；When determining that the two is consistent, illustrate that the resource file of correspondence is according to second During caching mapping relations storage, the edge node server at place is this node, and this resource file is according to the During one caching mapping relations storage, the edge node server at place is not this node, so that by this money Source file is from this knot removal.Now, this can be in the data of completion status by edge node server Migration task is added to the Data Migration list of self as data deletion task, with by performing data Data deletion task in delete list, deletes unnecessary resource file.
Otherwise, the edge node server of this node is determining the IP ground with self, the parsed IP address When location is inconsistent, resource file limit at place when according to the second caching mapping relations storage of correspondence is described Edge node server is not this node, and therefore, this data migration task being in completion status is stored in it In his node, it is therefore not necessary to perform data deletion, therefore this can be in by edge node server The data migration task of one-tenth state is deleted.
Fig. 7 shows the flow chart of a kind of data-erasure method in the embodiment of the present invention.Refer to Fig. 7 Shown in, in being embodied as, the data-erasure method in the embodiment of the present invention can comprise the following steps that
Step S701: obtain from the data deletion list of self and be in the data deletion of unfinished state and appoint Business.
In being embodied as, the edge node server of this node can be in a different manner from the data of self The data deletion task obtaining correspondence in delete list processes.Such as, edge node server is permissible Single-threaded mode, the once data deletion task that obtains from data deletion list is used to process, Can also adopt the mode of multithreading, once obtain a plurality of data deletion task simultaneously and process, with data The raising efficiency deleted, and then improve the efficiency of Data Migration.
Step S702: to the resource that the first load equalizer request is corresponding with acquired data deletion task The information of the URL of file.
In being embodied as, the edge node server of this node process obtain data deletion task time, The information of resource file of correspondence can be determined according to the information of acquired data deletion task, and to the The resource file of one load equalizer request correspondence is corresponding when according to the first caching mapping relations storage The information of URL.
Step S703: parse the information of the IP address of correspondence from the URL information asked.
In being embodied as, the corresponding URL that the first load equalizer returns is to open with corresponding IP address Head, this IP address is i.e. that this resource file is according to the fringe node at place during the first caching mapping relations storage The IP address of server.Therefore, the edge node server of this node is receiving the second load equalizer During the corresponding URL returned, by the URL returned is resolved, just can obtain this resource The IP address of the edge node server at place when file stores according to the first caching mapping relations, thus can To determine, parsing obtains this resource file according to the fringe node at place during the first caching mapping relations storage The IP address of server is the most consistent with the IP address of the edge node server of this node.
Step S704: judge that the IP address parsed is the most consistent with the IP address of self；Work as judgement When result is no, step S705 can be performed；Otherwise, then step S706 can directly be performed.
Step S705: corresponding resource file is deleted.
In being embodied as, determining that this resource file parsed is deposited according to the first caching mapping relations During storage, the IP address of the edge node server of the IP address of the edge node server at place and this node is not Time consistent, then show resource file place when according to described first caching mapping relations storage of correspondence Edge node server is the edge node server of other nodes, therefore, and the fringe node clothes of this node The corresponding resource file stored can be deleted from this node by business device.
Step S706: described data deletion task is deleted from data deletion list, and to Data Migration It is more newly requested that server sends task status, so that described data migration server is by corresponding in data base The state of the data migration task being in completion status be updated to delete successfully.
In being embodied as, determining that this resource file parsed is deposited according to the first caching mapping relations The IP address one of the edge node server of the IP address of the edge node server at place and this node during storage During cause, then show resource file limit at place when according to described first caching mapping relations storage of correspondence Edge node server is the edge node server of this node, needs to retain corresponding file.Now, i.e. It is considered as the success of data deletion tasks carrying, data deletion task can be deleted from data deletion list, Corresponding data deletion tasks carrying completes.
In being embodied as, determining that this resource file parsed is deposited according to the first caching mapping relations During storage, the IP address of the edge node server of the IP address of the edge node server at place and this node is not Time consistent, then show resource file place when according to described first caching mapping relations storage of correspondence Edge node server is not the edge node server of this node, at this point it is possible to delete corresponding resource File, it is possible to when resource file is deleted successfully, deletes data deletion task from data deletion list Removing, corresponding data deletion tasks carrying completes.
Meanwhile, the fringe node service of this node determine correspondence data deletion tasks carrying complete time, This data deletion task can be deleted from the data deletion list of self, and to data migration server Send task status more newly requested, so that described data migration server is by corresponding being in data base The state of the data migration task of completion status is updated to delete successfully, to avoid identical data to migrate task Repeat, and be easy to the implementation status of the data migration task in data base is controlled.
The data migration device that data migration method in the embodiment of the present invention is corresponding will be done further below Detailed introduction.
Fig. 8 shows the structure of a kind of data migration device in the embodiment of the present invention.Refer to Fig. 8 institute Showing, in being embodied as, the data migration device 800 in the embodiment of the present invention can include the first acquisition Unit the 801, first judging unit 802 and data migration unit 803, wherein:
First acquiring unit 801, is suitable to be obtained from from default data base be in the number in unfinished state According to migration task and be inserted in Data Migration list；Wherein, described data migration task includes correspondence The information of the URL that resource file is corresponding when according to default first caching mapping relations storage, described the One caching mapping relations be in described edge cluster increase edge node server back edge node server with Caching mapping relations between the resource file cached.
In being embodied as, described first acquiring unit 801 is suitable to obtain from described data base be in not The data migration task of completion status；Data migration task based on described unfinished state, determines correspondence The information of resource file；Obtain to the URL of resource file determined by the first load-balanced server transmission Take solicited message, so that the letter of the URL of resource file determined by described first load equalizer return Breath；Wherein, in described first load equalizer, storage has described first caching mapping relations；From being returned URL in parse the IP address of correspondence, and judge parsed IP address and self IP address The most consistent；When the IP address of self determines consistent with the IP address parsed, it is not fully complete described The data migration task of state adds to data migration task list.
First judging unit 802, is suitable to obtain the number being in unfinished state from data migration task list According to the task of migration, and judge whether the resource file that the data migration task of described unfinished state is corresponding is deposited ?；Wherein, described second caching mapping relations are before described edge cluster increases edge node server Edge node server and the caching mapping relations of all resource files cached.
Data migration unit 803, is suitable in the presence of the resource file determining correspondence, by corresponding resource literary composition The modification time of part is revised as the resource file of correspondence when storing according to the second default caching mapping relations Modification time；In the presence of determining that corresponding resource file is not, by the second load equalizer according in advance If the second caching mapping relations determine the edge node server at the resource file place of correspondence, from being determined Edge node server in capture the resource file of correspondence and store, and repairing corresponding resource file Changing time modification is that corresponding resource file when caching amendment when mapping relations store according to described second Between；Wherein, in described second load equalizer, storage has described second caching mapping relations.
In being embodied as, described data migration unit 803 is suitable to carrying out the resource file captured Checking, and when determining and be verified, crawl resource file is stored.
In being embodied as, the data migration device 800 in the embodiment of the present invention can also include state more New unit 804 and state update request unit 805, wherein:
State updating unit 804, by the status maintenance of the data migration task in described data migration task list Change into.
State updates request unit 805, is suitable to send corresponding state to data migration server more newly requested, So that the status modifier of data migration task described in described data base is by described data migration server Complete.
In being embodied as, the data migration device 800 in the embodiment of the present invention can also include that second obtains Take unit 806:
Second acquisition unit 806, is suitable to be obtained from the Data Migration being in completion status from data base and appoints Business, and be inserted in data deletion task list as data deletion task.
In being embodied as, described second acquisition unit 806 is suitable to obtain from described data base be in The data migration task of one-tenth state, and determine the resource file corresponding with acquired data migration task Information；Determine that the resource file of correspondence is according to the second default caching by described second load equalizer The information of the edge node server at place during mapping relations storage；When determine from as determined by edge During node server, acquired data migration task is inserted into data deletion as data deletion task In task list.
First determines unit 807, is suitable to obtain data deletion task from data deletion task list, and really The information of fixed corresponding resource file.
Second determines unit 808, is suitable to determine that the resource file of correspondence exists by described first load equalizer According to the information of the edge node server at place during described first caching mapping relations storage.
Data deletion unit 809, be suitable to when determine from as determined by edge node server time, from institute State and data deletion task list is deleted described data deletion task；Determined by determining and self not being During edge node server, corresponding resource file is deleted；Send to data migration server to described State is more newly requested, so that data migration server is by the data migration task of corresponding completion status State is set to delete successfully.
The embodiment of the present invention additionally provides a kind of edge node server, including above-mentioned data migration device.
Compared with prior art, technical scheme has the advantage that
Above-mentioned scheme, in edge cluster dilation process, sets up corresponding number for corresponding resource file According to the task of migration, obtained by edge node server acquired in data migration task the execution of self Data migration task, thus realize the money in edge cluster according to the first caching mapping relations after dilatation Source file is buffered in the edge node server of correspondence, owing to above-mentioned process only needs part resource File carries out migrating rather than again capturing from the memorizer of upper strata the resource file of correspondence, can save The time of Data Migration, it is possible to alleviate load and the bandwidth resources of upper strata memorizer, therefore, it can carry The efficiency of high Data Migration, and save resource.
One of ordinary skill in the art will appreciate that all or part of step in the various methods of above-described embodiment Suddenly the program that can be by completes to instruct relevant hardware, and this program can be stored in computer-readable In storage medium, storage medium may include that ROM, R101AM, disk or CD etc..
Having been described in detail the method and system of the embodiment of the present invention above, the present invention is not limited to this. Any those skilled in the art, without departing from the spirit and scope of the present invention, all can make various change with Amendment, therefore protection scope of the present invention should be as the criterion with claim limited range.

Claims (13)

1. a data migration method, enters resource file during for increasing fringe node service in edge cluster Row migrates, it is characterised in that including:
It is obtained from from default data base and is in the data migration task of unfinished state and is inserted into number According to migrating in list；Wherein, described data migration task includes that the resource file of correspondence is according to presetting The first caching mapping relations storage time corresponding URL information, described first caching mapping relations By described edge cluster increasing edge node server back edge node server and the money cached Caching mapping relations between source file；
From described data migration task list, obtain the data migration task being in unfinished state, and judge Whether the resource file that the data migration task of described unfinished state is corresponding exists；
In the presence of the resource file determining correspondence, the modification time of corresponding resource file is revised as correspondence Resource file modification time when according to default second caching mapping relations storage；Wherein, institute Stating the second caching mapping relations is the edge joint that described edge cluster increases before edge node server Point server and the caching mapping relations of all resource files cached；
In the presence of determining that corresponding resource file is not, by the second load equalizer according to described second caching Mapping relations determine the edge node server at resource file place of correspondence, from determined by edge joint Point server captures the resource file of correspondence and stores, and by the corresponding resource file that captured Modification time be revised as correspondence resource file according to described second caching mapping relations storage time Modification time；Wherein, in described second load equalizer, storage has described second caching mapping relations.
Data migration method the most according to claim 1, it is characterised in that described from default data base In be obtained from and be in the data migration task of unfinished state and be inserted in Data Migration list, bag Include:
The data migration task being in unfinished state is obtained from described data base；
Data migration task based on described unfinished state, determines the information of the resource file of correspondence；
Solicited message is obtained to the URL of resource file determined by the first load-balanced server transmission, with Make the information of the URL of resource file determined by described first load equalizer return；Wherein,
In described first load equalizer, storage has described first caching mapping relations；
From the URL returned, parse the IP address of correspondence, and judge parsed IP address and oneself The IP address of body is the most consistent；
When the IP address of self determines consistent with the IP address parsed, by described unfinished state Data migration task adds to described data migration task list.
Data migration method the most according to claim 1, it is characterised in that also include:
By the status modifier of the data migration task in described data migration task list for completing；
Corresponding state is sent more newly requested to data migration server, so that described data migration server By the status modifier of data migration task described in described data base for completing.
Data migration method the most according to claim 1, it is characterised in that at the resource literary composition that will be captured Before part stores, also include:
The resource file captured is verified and determined and is verified.
Data migration method the most according to claim 1, it is characterised in that also include:
It is obtained from from described data base and is in the data migration task in completion status, and as data deletion Task is inserted in data deletion task list；
Acquisition data deletion task from described data deletion task list, and determine the resource file of correspondence Information；
Determine that the resource file of correspondence is according to described first caching mapping relations by the first load equalizer The information of the edge node server at place during storage；In described first load equalizer, storage has described First caching mapping relations；
When determine from as determined by edge node server time, delete from described data deletion task list Except described data deletion task；When determine self be not determined by edge node server time, will Corresponding resource file is deleted；
State is sent more newly requested, so that described data migration server will to described data migration server The state of the data migration task of corresponding completion status is set to delete successfully.
Data migration method the most according to claim 5, it is characterised in that described from described data base It is obtained from and is in the data migration task in completion status, and be inserted into data as data deletion task Delete in task list, including:
From described data base, obtain the data migration task being in completion status, and determine and acquired number Information according to resource file corresponding to the task of migration；
Determined that by described second load equalizer the resource file of correspondence is mapping according to described second caching The information of the edge node server at place during relation storage；
When determine from as determined by edge node server time, using acquired data migration task as Data deletion task is inserted in described data deletion task list.
7. a data migration device, enters resource file during for increasing fringe node service in edge cluster Row migrates, it is characterised in that including:
First acquiring unit, is suitable to be obtained from the data being in unfinished state from default data base and moves Shifting task is also inserted in Data Migration list；Wherein, described data migration task includes the money of correspondence The information of the URL that source file is corresponding when storing according to the first default caching mapping relations, described First caching mapping relations are to increase edge node server back edge node clothes in described edge cluster Caching mapping relations between business device and the resource file cached；
First judging unit, is suitable to obtain the number being in unfinished state from described data migration task list According to the task of migration, and judge resource file that the data migration task of described unfinished state is corresponding whether Exist；
Data migration unit, is suitable in the presence of the resource file determining correspondence, by corresponding resource file Modification time is revised as the resource file of correspondence when storing according to the second default caching mapping relations Modification time, wherein, described second caching mapping relations are that described edge cluster increases fringe node Edge node server before server and the caching mapping relations of all resource files cached；
In the presence of determining that corresponding resource file is not, by the second load equalizer according to presetting the second caching Mapping relations determine the edge node server at resource file place of correspondence, from determined by edge joint Point server captures the resource file of correspondence and stores, and by the corresponding resource file that captured Modification time be revised as correspondence resource file according to described second caching mapping relations storage time Modification time；Wherein, in described second load equalizer, storage has described second caching mapping relations.
Data migration device the most according to claim 7, it is characterised in that described first acquiring unit is fitted In obtaining the data migration task being in unfinished state from described data base；It is not fully complete based on described The data migration task of state, determines the information of the resource file of correspondence；To the first load balancing service Determined by device transmission, the URL of resource file obtains solicited message, so that described first load is all The information of the URL of resource file determined by weighing apparatus return；Wherein, described first load equalizer Middle storage has described first caching mapping relations；The IP ground of correspondence is parsed from the URL returned Location, and judge that parsed IP address is the most consistent with the IP address of self；IP ground when self When location determines consistent with the IP address parsed, the data migration task of described unfinished state is added Add in described data migration task list.
Data migration device the most according to claim 7, it is characterised in that also include:
State updating unit, by the status modifier of the data migration task in described data migration task list be Complete；
State updates request unit, is suitable to send corresponding state to data migration server more newly requested, with Make described data migration server by the status modifier of data migration task described in described data base For completing.
Data migration device the most according to claim 7, it is characterised in that described data migration unit is fitted In the resource file captured is being verified, and determine the resource literary composition that will be captured when being verified Part stores.
11. data migration device according to claim 7, it is characterised in that also include:
Second acquisition unit, is suitable to be obtained from from data base and is in the data migration task in completion status, And be inserted in data deletion task list as data deletion task；
First determines unit, is suitable to obtain data deletion task from described data deletion task list, and really The information of fixed corresponding resource file；
Second determines unit, is suitable to determine that the resource file of correspondence is according to described by the first load equalizer The information of the edge node server at place during the first caching mapping relations storage；Described first load is all In weighing apparatus, storage has described first caching mapping relations；
Data deletion unit, be suitable to when determine from as determined by edge node server time, from described number Described data deletion task is deleted according to deleting in task list；When determine self be not determined by limit During edge node server, corresponding resource file is deleted；State is sent more to data migration server It is newly requested, so that described data migration server is by the data migration task of corresponding completion status State is set to delete successfully.
12. data migration device according to claim 11, it is characterised in that described second acquisition unit Be suitable to from described data base, obtain the data migration task being in completion status, and determine with acquired The information of resource file corresponding to data migration task；It is right to be determined by described second load equalizer The resource file answered is the fringe node clothes at place when storing according to the second default caching mapping relations The information of business device；When determine from as determined by edge node server time, by acquired data Migration task is inserted in described data deletion task list as data deletion task.
13. 1 kinds of edge node server, it is characterised in that include the data described in any one of claim 7-12 Moving apparatus.
CN201610224612.8A 2016-04-12 2016-04-12 Data migration method and device, edge node server Active CN105991769B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201610224612.8A CN105991769B (en) 2016-04-12 2016-04-12 Data migration method and device, edge node server

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201610224612.8A CN105991769B (en) 2016-04-12 2016-04-12 Data migration method and device, edge node server

Publications (2)

Publication Number Publication Date
CN105991769A true CN105991769A (en) 2016-10-05
CN105991769B CN105991769B (en) 2019-04-16

Family

ID=57043952

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201610224612.8A Active CN105991769B (en) 2016-04-12 2016-04-12 Data migration method and device, edge node server

Country Status (1)

Country Link
CN (1) CN105991769B (en)

Cited By (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN107786668A (en) * 2017-11-09 2018-03-09 成都知道创宇信息技术有限公司 A kind of weight caching web site method based on CDN
CN107948165A (en) * 2017-11-29 2018-04-20 成都东方盛行电子有限责任公司 A kind of safety based on proprietary protocol send broadcast system and method
CN108021585A (en) * 2016-10-28 2018-05-11 腾讯科技（深圳）有限公司 Distributed data storage method and device
CN110222030A (en) * 2019-05-13 2019-09-10 福建天泉教育科技有限公司 The method of Database Dynamic dilatation, storage medium
CN111625498A (en) * 2020-05-28 2020-09-04 浪潮电子信息产业股份有限公司 Data migration method, system, electronic equipment and storage medium
CN113127444A (en) * 2020-01-15 2021-07-16 中移(苏州)软件技术有限公司 Data migration method, device, server and storage medium
CN113515237A (en) * 2021-07-22 2021-10-19 中移（杭州）信息技术有限公司 Edge node capacity expansion method and device, edge node and program product
CN115858503A (en) * 2023-02-28 2023-03-28 江西师范大学 Heterogeneous database migration management method and system based on migration linked list

Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US7721044B1 (en) * 2005-10-20 2010-05-18 American Megatrends, Inc. Expanding the storage capacity of a virtualized data storage system
CN102521297A (en) * 2011-11-30 2012-06-27 北京人大金仓信息技术股份有限公司 Method for achieving system dynamic expansion in shared-nothing database cluster
CN102955845A (en) * 2012-10-23 2013-03-06 北京亿赞普网络技术有限公司 Data access method and device as well as distributed database system
CN103034664A (en) * 2011-10-10 2013-04-10 上海盛霄云计算技术有限公司 Method, system and device for controlling data migration of database
CN104484469A (en) * 2014-12-31 2015-04-01 天津南大通用数据技术股份有限公司 Expansion method for supporting non-halt of multi-hash map database cluster system
CN104580393A (en) * 2014-12-18 2015-04-29 北京蓝汛通信技术有限责任公司 Method and device for expanding server cluster system and server cluster system
CN104615657A (en) * 2014-12-31 2015-05-13 天津南大通用数据技术股份有限公司 Expanding and shrinking method for distributed cluster with nodes supporting multiple data fragments

Patent Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US7721044B1 (en) * 2005-10-20 2010-05-18 American Megatrends, Inc. Expanding the storage capacity of a virtualized data storage system
CN103034664A (en) * 2011-10-10 2013-04-10 上海盛霄云计算技术有限公司 Method, system and device for controlling data migration of database
CN102521297A (en) * 2011-11-30 2012-06-27 北京人大金仓信息技术股份有限公司 Method for achieving system dynamic expansion in shared-nothing database cluster
CN102955845A (en) * 2012-10-23 2013-03-06 北京亿赞普网络技术有限公司 Data access method and device as well as distributed database system
CN104580393A (en) * 2014-12-18 2015-04-29 北京蓝汛通信技术有限责任公司 Method and device for expanding server cluster system and server cluster system
CN104484469A (en) * 2014-12-31 2015-04-01 天津南大通用数据技术股份有限公司 Expansion method for supporting non-halt of multi-hash map database cluster system
CN104615657A (en) * 2014-12-31 2015-05-13 天津南大通用数据技术股份有限公司 Expanding and shrinking method for distributed cluster with nodes supporting multiple data fragments

Cited By (11)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN108021585A (en) * 2016-10-28 2018-05-11 腾讯科技（深圳）有限公司 Distributed data storage method and device
CN107786668A (en) * 2017-11-09 2018-03-09 成都知道创宇信息技术有限公司 A kind of weight caching web site method based on CDN
CN107948165A (en) * 2017-11-29 2018-04-20 成都东方盛行电子有限责任公司 A kind of safety based on proprietary protocol send broadcast system and method
CN107948165B (en) * 2017-11-29 2023-10-20 成都东方盛行电子有限责任公司 Secure broadcast system and method based on private protocol
CN110222030A (en) * 2019-05-13 2019-09-10 福建天泉教育科技有限公司 The method of Database Dynamic dilatation, storage medium
CN113127444A (en) * 2020-01-15 2021-07-16 中移(苏州)软件技术有限公司 Data migration method, device, server and storage medium
CN113127444B (en) * 2020-01-15 2023-04-14 中移(苏州)软件技术有限公司 Data migration method, device, server and storage medium
CN111625498A (en) * 2020-05-28 2020-09-04 浪潮电子信息产业股份有限公司 Data migration method, system, electronic equipment and storage medium
CN111625498B (en) * 2020-05-28 2022-06-07 浪潮电子信息产业股份有限公司 Data migration method, system, electronic equipment and storage medium
CN113515237A (en) * 2021-07-22 2021-10-19 中移（杭州）信息技术有限公司 Edge node capacity expansion method and device, edge node and program product
CN115858503A (en) * 2023-02-28 2023-03-28 江西师范大学 Heterogeneous database migration management method and system based on migration linked list

Also Published As

Publication number Publication date
CN105991769B (en) 2019-04-16

Similar Documents

Publication Publication Date Title
CN105991769A (en) Data migration method and device and edge node server
CN105763628B (en) Data access request processing method and processing device, edge node server and cluster
CN103812849B (en) A kind of local cache update method, system, client and server
US8495013B2 (en) Distributed storage system and method for storing objects based on locations
US20180041468A1 (en) Managing dynamic ip address assignments
US11943291B2 (en) Hosted file sync with stateless sync nodes
CN107395659A (en) A kind of method and device of service handling and common recognition
CN106095533B (en) Method of server expansion and device
CN106294352B (en) A kind of document handling method, device and file system
CN103391312B (en) Resource offline method for down loading and device
CN105653198A (en) Data processing method and device
CN102708173A (en) Method and system for processing user requests of accessing to web pages
CN108282522A (en) Data storage access method based on dynamic routing and system
CN103440345B (en) Distributed database extension method and distributed database extension system based on relational database
CN104537013A (en) Data processing method and device
CN104935653A (en) Bypass cache method for visiting hot spot resource and device
CN108881066A (en) A kind of method of route requests, access server and storage equipment
CN106777085A (en) A kind of data processing method, device and data query system
CN108989430A (en) Load-balancing method, device and storage medium
CN105208058A (en) Information exchange system based on web session sharing
US20160234129A1 (en) Communication system, queue management server, and communication method
CN105868234A (en) Update method and device of caching data
CN106021284A (en) Data query method, data monitoring method and device
CN104125303B (en) Reading and writing data requesting method, client and system
CN103647811B (en) A method and an apparatus for application's accessing backstage service

Legal Events

Date Code Title Description
C06 Publication
PB01 Publication
C10 Entry into substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant