CN102456049A - Data migration method and device, and object-oriented distributed file system - Google Patents

Data migration method and device, and object-oriented distributed file system Download PDF

Info

Publication number
CN102456049A
CN102456049A CN2010105273897A CN201010527389A CN102456049A CN 102456049 A CN102456049 A CN 102456049A CN 2010105273897 A CN2010105273897 A CN 2010105273897A CN 201010527389 A CN201010527389 A CN 201010527389A CN 102456049 A CN102456049 A CN 102456049A
Authority
CN
China
Prior art keywords
file
object
storage server
data
target
Prior art date
Application number
CN2010105273897A
Other languages
Chinese (zh)
Inventor
何晓斌
景志洁
王希阳
蒋金虎
魏巍
Original Assignee
无锡江南计算技术研究所
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 无锡江南计算技术研究所 filed Critical 无锡江南计算技术研究所
Priority to CN2010105273897A priority Critical patent/CN102456049A/en
Publication of CN102456049A publication Critical patent/CN102456049A/en

Links

Abstract

The invention discloses a data migration method and device, and an object-oriented distributed file system, wherein the data migration method comprises the following steps of: when a file object stored in a source object storage server needs to be migrated to a target object storage server, creating an empty file object in the target object storage server, and then establishing a data transmission channel between the file object in the source object storage server and the empty file object in the target object storage server. In this way, the file object data can be directly migrated to the empty file object of the target object storage server from the file object in the source object storage server without the involvement of a client; therefore, the problems of long transmission path and easy occurrence of data transmission errors during the existing data migration operation are solved.

Description

数据迁移方法、装置及面向对象的分布式文件系统 The data migration method, apparatus and object-based distributed file system

技术领域 FIELD

[0001] 本发明涉及数据处理技术领域,更具体地说,涉及一种数据迁移方法、装置及面向对象的分布式文件系统。 [0001] Technical Field The present invention relates to data processing, and more particularly, relates to a data migration method, apparatus and object-based distributed file system.

背景技术 Background technique

[0002] 随着超级计算机向着分布化和集群化方向发展,对存储的大容量、高可靠、高带宽的要求也越来越高,现提出一种新型的面向对象存储文件系统。 [0002] With the supercomputer cluster toward the distribution and direction of development, demand for high-capacity storage, high reliability, high bandwidth is also growing, now propose a new object-oriented storage file system.

[0003] 如图1所示,该文件系统包括客户端101、元数据服务器102和对象存储服务器103,其中,元数据服务器102用以存储文件系统中文件的元数据信息;存储服务器103接有数据存储设备,用以存储文件的数据信息,客户端101可以直接访问元数据服务器102和对象存储服务器103。 [0003] As shown, the file system 1 includes a client 101, server 102 and metadata server object storage 103, wherein the metadata server 102 for metadata storage file system information of the files; storage server 103 is connected with the data storage device for storing data information file, the client 101 may access the metadata server 102 and the object storage server 103 directly.

[0004] 然而,现有的面向对象存储文件系统存有数据管理效率低的问题,例如:执行数据迁移等操作时,客户端发起获取数据请求,数据由源对象存储服务器传输至客户端,再由客户端将其传输至目标对象存储服务器,这样,数据的传输路径较长,容易出现数据传输错误的问题。 [0004] However, the conventional object-oriented storage file data management system there is low efficiency, for example: performing data migration and other operations, the client initiates data acquisition request, the data transmitted from the source object storage server to the client, and then by the client it is transmitted to a target object storage server so that the longer the data transmission path, data transmission error prone to problems.

发明内容 SUMMARY

[0005] 有鉴于此,本发明提供一种数据迁移方法、装置及面向对象的分布式文件系统,以解决现有数据迁移操作时,传输路径长且易出现数据传输错误的问题。 [0005] Accordingly, the present invention provides a data migration method, apparatus and object-based distributed file system, to solve an existing data migration operation, the transmission path is long and prone to data transmission errors.

[0006] 为解决上述问题,现提出的方案如下: [0006] To solve the above problems, solutions are proposed as follows:

[0007] —种数据迁移方法,包括: [0007] - kind of data migration method, comprising:

[0008] 监控文件的文件状态; [0008] The file status monitor file;

[0009] 若所述文件状态发生变化,获取该文件变化后的属性值; [0009] If the file status attribute value is changed, the file acquired after the change;

[0010] 当判断该文件的属性值触发执行文件对象迁移操作的条件时,将当前存储有所述文件中文件对象的对象存储服务器作为源对象存储服务器,迁移后存储文件中文件对象的对象存储服务器作为目标对象存储服务器; [0010] When it is determined that the attribute value of the object file migration operation conditions triggers execution file, the current file is stored in the object storage server file object as the source object storage server, the migration target file stored in the storage file objects server as the target object storage server;

[0011] 在所述目标对象存储服务器中创建空文件对象; [0011] Create an empty file objects in the target object storage server;

[0012] 建立所述源对象存储服务器中文件对象和目标对象存储服务器空文件对象之间的数据传输通道,将文件对象数据由所述源对象存储服务器中文件对象传输至所述目标对象存储服务器中空文件对象。 [0012] establishing a data transmission path between the source object storage server stores a file server and target object null file object, the object data file by a file object storage server transfer the source object to the target object in a storage server hollow file object.

[0013] 优选地,所述监控文件状态为监控文件列表中未满足预设条件文件的状态,监控文件状态前还包括: [0013] Preferably, the monitoring list file state monitoring file satisfies a preset condition file is not a state, before the file status monitor further comprises:

[0014] 缓存文件的文件路径或者文件名称; File [0014] Cache file path or file name;

[0015] 按照文件满足预设条件程度的顺序排序所述文件路径或者文件名称,形成文件列表。 [0015] satisfies a preset condition in accordance with the degree of file sorted in the order of the file path or file name, the file list is formed.

[0016] 优选地,所述文件对象数据传输完毕后还包括: [0016] Preferably, after the object data file transfer is completed further comprises:

5[0017] 分别读取存储在所述源对象存储服务器中文件对象和目标对象存储服务器中文件对象的预设位置的数据; 5 [0017] are respectively read data in the source object storage server stores the file in the preset position and target objects in an object file storage server;

[0018] 比对所述数据是否相同,若相同则得到传输成功结果,否则得到传输失败结果。 [0018] is the same as the ratio of data transmission is successful if the same result is obtained, or transmission failure result obtained.

[0019] 优选地,所述数据迁移方法还包括:检测得到文件对象数据传输失败原因。 [0019] Preferably, the data migration method further comprising: detecting a file obtained object data transmission failure reason.

[0020] 优选地,所述文件对象数据传输完毕后,还包括: [0020] Preferably, after the object data file transfer is complete, further comprising:

[0021] 制作传输结果汇报表,该传输结果汇报表中含有文件对象数据传输结果信息或文件对象数据传输失败原因信息。 [0021] Production transmission for reporting the results, the cause of failure information transmission result reporting table contains the data transfer result information file object file or object data transfer.

[0022] 一种数据迁移装置,包括: [0022] A data migration apparatus, comprising:

[0023] 第一监控单元,用于监控文件的文件状态; [0023] The first monitoring means for monitoring the state of the file;

[0024] 第一文件属性值获取单元,用于若所述文件状态发生变化,获取该文件变化后的属性值; [0024] The first file attribute value acquisition unit configured to, if the document status changes, acquires the file attribute value change;

[0025] 第一判断单元,用于判断获取到的文件的属性值是否满足阈值; [0025] The first determination unit configured to determine the acquired file attribute value satisfies a threshold;

[0026] 第一对象存储服务器类别确定单元,用于当判断该文件的属性值满足阈值,需执行文件对象迁移操作时,将当前存储有所述文件中文件对象的对象存储服务器作为源对象存储服务器,迁移后存储文件中文件对象的对象存储服务器作为目标对象存储服务器; [0026] The first object storage server class determining unit configured to store the object server when the file determination attribute value satisfies a threshold, the object needs to perform file migration operations, the current file is stored in the file object as the source object storage server, after migration store files in the file object object storage server as the target object storage server;

[0027] 第一空文件对象创建单元,用于在确定的目标对象存储服务器中创建空文件对象; [0027] The first object is an empty file creating unit for creating an empty file objects in the target object in the determined storage server;

[0028] 第一文件对象迁移单元,用于建立所述源对象存储服务器中文件对象和目标对象存储服务器空文件对象之间的数据传输通道,将文件对象数据由所述源对象存储服务器中文件对象传输至所述目标对象存储服务器空文件对象。 [0028] The first file object migration unit for establishing a data transmission path between the source object storage server stores a file server and target object null file object, the object data file by said file storage server source object transmitting the object to the target object empty file storage server object.

[0029] 优选地,所述数据迁移装置还包括: [0029] Preferably, the data migration apparatus further comprising:

[0030] 第一文件数据读取单元,用于当文件对象数据传输完毕后,分别读取存储在所述源对象存储服务器中文件对象和目标对象存储服务器中文件对象预设位置的数据; After the [0030] first file data reading unit, configured to, when the file transfer is complete object data, respectively read files stored on the server and target objects stored in the object file in the preset position data of the source object storage server;

[0031] 第一传输结果判定单元,用于比对所述数据是否相同,若相同则得到传输成功结果,否则得到传输失败结果。 [0031] The first transmission result determination unit for comparing the data is the same, if the same result is obtained successful transmission, or transmission failure result obtained.

[0032] 优选地,所述数据迁移装置还包括:第一传输结果汇报表制作单元,用于制作文件对象数据传输结果汇报表。 [0032] Preferably, the data migration apparatus further comprising: a first production unit for reporting a transmission result, for the production of the file object data transmission for reporting the results.

[0033] 一种数据迁移方法,包括: [0033] A data migration method, comprising:

[0034] 当接收到客户端发送的执行文件对象迁移操作的控制指令时,将当前存储有所述文件中文件对象的对象存储服务器作为源对象存储服务器,迁移后存储文件中文件对象的对象存储服务器作为目标对象存储服务器; [0034] Upon receiving the control instruction execution target file migration operations sent by the client, a current file is stored in the object storage server file object as the source object storage server, the migration target file stored in the storage file objects server as the target object storage server;

[0035] 在所述目标对象存储服务器中创建空文件对象; [0035] Create an empty file objects in the target object storage server;

[0036] 建立所述源对象存储服务器中文件对象和目标对象存储服务器空文件对象之间的数据传输通道,将文件对象数据由所述源对象存储服务器中文件对象传输至所述目标对象存储服务器中空文件对象。 [0036] establishing a data transmission path between the source object storage server stores a file server and target object null file object, the object data file by a file object storage server transfer the source object to the target object in a storage server hollow file object.

[0037] 优选地,所述文件对象数据传输完毕后还包括: [0037] Preferably, after the object data file transfer is completed further comprises:

[0038] 分别读取存储在所述源对象存储服务器中文件对象和目标对象存储服务器中文件对象的预设位置的数据; [0038] The data are read in the source object storage server stores the file in the preset position and target objects in an object file storage server;

[0039] 比对所述数据是否相同,若相同则得到传输成功结果,否则得到传输失败结果。 [0039] is the same as the ratio of data transmission is successful if the same result is obtained, or transmission failure result obtained. [0040] 一种数据迁移装置,包括: [0040] A data migration apparatus, comprising:

[0041] 第二对象存储服务器类别确定单元,用于当接收到客户端发送的执行文件对象迁移操作的控制指令时,将当前存储有所述文件中文件对象的对象存储服务器作为源对象存储服务器,迁移后存储文件中文件对象的对象存储服务器作为目标对象存储服务器; When the [0041] second server object storage class determining unit, when receiving the control command sent by the client file execution object migration operation, the current object storage server stores said file in the file object as the source object storage server after the migration file is stored in the file object object storage server as the target object storage server;

[0042] 第二空文件对象创建单元,用于在所述目标对象存储服务器中创建空文件对象; [0042] The second object is an empty file creating unit for creating an empty file object in the target object storage server;

[0043] 第二文件对象迁移单元,用于建立所述源对象存储服务器中文件对象和目标对象存储服务器空文件对象之间的数据传输通道,将文件对象数据由所述源对象存储服务器中文件对象传输至所述目标对象存储服务器空文件对象。 [0043] The second object is a file migration means for establishing a data transmission path between the source object storage server stores a file server and target object null file object, the object data file by said file storage server source object transmitting the object to the target object empty file storage server object.

[0044] 优选地,所述数据迁移装置还包括: [0044] Preferably, the data migration apparatus further comprising:

[0045] 第二文件数据读取单元,用于当文件对象数据传输完毕后,分别读取存储在所述源对象存储服务器中文件对象和目标对象存储服务器中文件对象的预设位置的数据; [0045] The second file data reading unit, for the object data when the file transfer is complete, the data are read in the source object storage server stores the file in the preset position and target objects in an object file storage server;

[0046] 第二传输结果判定单元,用于比对所述数据是否相同,若相同则得到传输成功结果,否则得到传输失败结果。 [0046] The second determination result transmitting unit is the same than for the data transmission is successful if the same result is obtained, or transmission failure result obtained.

[0047] 一种面向对象的分布式文件系统,包括:客户端、元数据服务器和对象存储服务器,其中: [0047] An object-oriented distributed file system, comprising: a client, a metadata server and the server object is stored, wherein:

[0048] 所述客户端包括控制指令发送单元,该单元用于发送控制指令至所述元数据服务器; [0048] The client includes a control instruction transmitting unit, the unit for transmitting control commands to the metadata server;

[0049] 所述元数据服务器,包括: [0049] The metadata server, comprising:

[0050] 监控单元,用于监控文件的文件状态以及客户端的控制指令; [0050] The monitoring unit, and a client file status file for monitoring control command;

[0051] 文件属性值获取单元,用于若所述文件状态发生变化,获取该文件变化后的属性值; [0051] The file attribute value acquisition unit for file attribute value if a state change occurs, obtain the file after the change;

[0052] 判断单元,用于判断获取到的文件的属性值是够满足阈值; [0052] determination means for determining the value of the acquired file attribute is enough to meet the threshold;

[0053] 对象存储服务器类别确定单元,用于当判断该文件的属性值满足阈值,需执行文件对象迁移操作时,或者,当所述客户端的控制指令为执行文件对象迁移操作时,将当前存储有所述文件中文件对象的对象存储服务器作为源对象存储服务器,迁移后存储文件中文件对象的对象存储服务器作为目标对象存储服务器; [0053] The object storage server class determining unit configured to, when determining the property value of the file satisfies a threshold, perform file object migration operation, or when the control command to the client to perform the file object migration, currently stored there are file objects in the file storage server object as the source object storage server, the storage server stores the migration target file object as the target object file storage server;

[0054] 空文件对象创建单元,用于在所述目标对象存储服务器中创建空文件对象; [0054] null file object creation means for creating an empty file object in the target object storage server;

[0055] 文件对象迁移单元,用于建立所述源对象存储服务器中文件对象和目标对象存储服务器中空文件对象之间的数据传输通道,将文件对象数据由所述源对象存储服务器中文件对象传输至所述目标对象存储服务器中空文件对象。 [0055] Object file migration means for establishing a data transmission path between the source object storage server stores the target object and the file object file server hollow objects, the object data file transmitted by the server object is stored in the source file object to the target object file storage server of the hollow object.

[0056] 从上述的技术方案可以看出,本发明公开的数据迁移方法中,当需要将源对象存储服务器存储的文件对象迁移至目标对象存储服务器中时,在目标对象存储服务器中创建空文件对象,再建立所述源对象存储服务器中文件对象和目标对象存储服务器空文件对象之间的数据传输通道,这样,就可以将文件对象数据直接由源对象存储服务器中文件对象迁移至目标对象存储服务器空文件对象中,不需要客户端参与,解决了现有数据迁移操作时,传输路径长且易出现数据传输错误的问题。 When [0056] As can be seen from the above technical solution, the data migration method disclosed in the present invention, when it is necessary to migrate source object storage server stores the file object storage server to a target object, the target object to create an empty file storage server objects, and then establishes a data transmission path between the source object storage server stores a file server and target object null file object, so that it can migrate from the file object data directly in the source object storage server stores the target object to the file object when empty file server object, the client does not need to participate, to solve the existing data migration operation, the transmission path is long and prone to data transmission errors.

附图说明 BRIEF DESCRIPTION

[0057] 为了更清楚地说明本发明实施例或现有技术中的技术方案,下面将对实施例或现有技术描述中所需要使用的附图作简单地介绍,显而易见地,下面描述中的附图仅仅是本发明的一些实施例,对于本领域普通技术人员来讲,在不付出创造性劳动的前提下,还可以根据这些附图获得其他的附图。 [0057] In order to more clearly illustrate the technical solutions in the embodiments or the prior art embodiment of the present invention, briefly introduced hereinafter, embodiments are described below in the accompanying drawings or described in the prior art needed to be used in describing the embodiments the drawings are only some embodiments of the present invention, those of ordinary skill in the art is concerned, without creative efforts, can derive from these drawings other drawings.

[0058] 图1为现有技术中面向对象存储文件系统的结构示意图; [0058] FIG. 1 is a schematic structural diagram of a file system object oriented storage prior art;

[0059] 图2为本发明实施例公开的一种数据迁移方法的流程图; [0059] FIG 2 is a flowchart A data migration method according to the present embodiment of the disclosed invention;

[0060] 图3为本发明再一实施例公开的一种数据迁移方法的流程图; [0060] FIG. 3 is a flowchart of a method for data migration to another embodiment of the disclosed embodiment of the present invention;

[0061] 图4为本发明实施例公开的一种数据迁移装置的结构示意图; [0061] FIG. 4 configuration diagram of an embodiment of a data migration apparatus disclosed embodiment of the invention;

[0062] 图5为本发明再一实施例公开的一种数据迁移装置的结构示意图; [0062] Figure 5 configuration diagram of an embodiment of a data migration apparatus disclosed another embodiment of the present invention;

[0063] 图6为本发明再一实施例公开的一种数据迁移方法的流程图; [0063] FIG 6 is a flowchart of a method for data migration to another embodiment of the disclosed embodiment of the present invention;

[0064] 图7为本发明再一实施例公开的一种数据迁移装置的结构示意图; [0064] FIG 7 schematic diagram of a data structure of the migration apparatus embodiments disclosed another embodiment of the present invention;

[0065] 图8为本发明再一实施例公开的一种数据迁移装置的结构示意图; [0065] FIG. 8 configuration diagram of a data migration apparatus disclosed embodiment of another embodiment of the present invention;

[0066] 图9为本发明实施例公开的一种面向对象的分布式文件系统的结构示意图。 [0066] FIG. 9 is a schematic structure of an object-oriented distributed file system according to the disclosed embodiment of the present invention.

具体实施方式 Detailed ways

[0067] 下面将结合本发明实施例中的附图,对本发明实施例中的技术方案进行清楚、完整地描述,显然,所描述的实施例仅仅是本发明一部分实施例,而不是全部的实施例。 [0067] below in conjunction with the present invention in the accompanying drawings, technical solutions of embodiments of the present invention are clearly and completely described, obviously, the described embodiments are merely part of embodiments of the present invention, but not all embodiments example. 基于本发明中的实施例,本领域普通技术人员在没有作出创造性劳动前提下所获得的所有其他实施例,都属于本发明保护的范围。 Based on the embodiments of the present invention, those of ordinary skill in the art to make all other embodiments without creative work obtained by, it falls within the scope of the present invention.

[0068] 本发明实施例公开了一种数据迁移方法、装置及面向对象的分布式文件系统,以解决现有数据迁移操作时,传输路径长且易出现数据传输错误的问题。 When Example discloses a data migration method, apparatus and object-based distributed file system, to solve the conventional data migration operation [0068] of the present invention, the transmission path is long and prone to data transmission errors.

[0069] 下面先将本发明实施例中的一些相关概念做一下解释: [0069] Next, a number of related concepts in a first embodiment of the present invention do some explanation:

[0070] 本发明实施例公开的数据迁移方法,数据迁移分为两种,一种是数据剪切,即数据迁移完毕后,删除原始存储数据的文件;另一种是数据复制,即数据迁移完毕后,保留原始存储数据的文件。 The data migration method disclosed in the embodiments of the [0070] present invention, data migration is divided into two, one is cutting the data, i.e. the data migration is completed, deleting the original file data is stored; the other is a data replication, i.e., data migration after completion, retain the original file to store data.

[0071] 本发明实施例中说明的文件是本实施例公开的面向对象的分布式文件系统中保存的文件。 Examples of documents described embodiment [0071] of the present invention is stored in object-oriented distributed file system files embodiment disclosed embodiments.

[0072] 本发明实施例中说明的文件对象是组成文件的最小单元,各个文件对象可以存放于不同的对象存储服务器中,可以存放在相同的对象存储服务器中。 [0072] The file object described embodiment of the present invention is the minimum unit composed of files, each file can be stored in the object storage server different object, the object can be stored in the same storage server.

[0073] 如图2所示,本发明一实施例公开的一种数据迁移方法,包括: [0073] As shown in FIG 2, according to the present invention, eleven kinds of data migration method according to embodiments disclosed, comprising:

[0074] 步骤Sl 1、监控文件的文件状态; [0074] Step Sl 1, monitoring the state of the file;

[0075] 具体的,所述监控文件状态为监测该文件的基本属性,其基本属性包括文件时间属性、文件大小属性、文件被访问频率等。 [0075] Specifically, the monitor state to monitor the basic file attribute of the file, the file attributes include basic time attribute, the file size attribute, the file is accessed frequency.

[0076] 步骤S12、若所述文件状态发生变化,获取该文件变化后的属性值; [0076] step S12, if the file status attribute value is changed, the file acquired after the change;

[0077] 具体的,随着数据不断写入文件,文件逐渐变大,占用的存储空间逐渐增大,文件的大小属性随之而变化;或者,随时时间推移,文件的时间属性也会发生变化。 [0077] Specifically, as the data is written to a file, the file gradually increases, storage space is gradually increased, the size of the file attributes will vary; or, at any time lapse time will change the file attribute .

[0078] 步骤S13、当判断该文件的属性值满足阈值,需执行文件对象迁移操作时,将当前存储有所述文件中文件对象的对象存储服务器作为源对象存储服务器,迁移后存储文件中文件对象的对象存储服务器作为目标对象存储服务器; [0078] Step S13, the when determining the property value of the file satisfies a threshold, perform file object migration, currently stores the file in the file object object storage server as the source object storage server, after migration store file file object object storage server as the target object storage server;

[0079] 具体的,文件的属性值满足阈值可以为:文件大小超过阈值,文件存储文件存放时间超过阈值或者文件未被访问时间达到阈值。 [0079] Specifically, the file attribute value satisfies the threshold may be: file size exceeds the threshold value, file storage storage time exceeds a threshold or not the file access time reaches the threshold value. 其中,用户根据需要设定文件属性值的阈值, 例如,文件未被访问的时间为30天。 Wherein, the user needs to set the file attribute value threshold value, e.g., time of the file has not been accessed for 30 days.

[0080] 并且,文件对象迁移操作中包含有文件中需迁移的文件对象的源存储位置和目标存储位置,即确定需迁移文件对象的源对象存储服务器和目标对象存储服务器。 [0080] Further, the migration file object file contains the file object has to be relocated in the operation of source storage and target storage location location, i.e. to determine the source object and the target object storage server stores the server object file to be relocated.

[0081] 步骤S14、在所述目标对象存储服务器中创建空文件对象; [0081] step S14, creates an empty object in the target object file storage server;

[0082] 具体的,该空文件对象可以具有源文件对象的名称,且具有指向与其同名称、存储于源对象存储服务器内的文件对象的扩展属性,以便于标明该文件内的数据由源对象存储服务器内的文件对象迁移而来。 [0082] Specifically, the empty file object may have a name of the source file object, and having directed thereto with the same name, extended attribute file objects stored in the source object storage server in order to indicate the data in the file from the source object file objects within the storage server migration from.

[0083] 步骤S15、建立所述源对象存储服务器中文件对象和目标对象存储服务器空文件对象之间的数据传输通道,将文件对象数据由所述源对象存储服务器中文件对象传输至所述目标对象存储服务器中空文件对象。 [0083] Step S15, the data transmission path established between the source object storage server stores a file server and target object null file object, the object data file by a file object to transfer the source object to the target storage server object storage file server hollow object.

[0084] 本实施例公开的数据迁移方法中,当需要将源对象存储服务器存储的文件对象迁移至目标对象存储服务器中时,在目标对象存储服务器中创建空文件对象,再建立所述源对象存储服务器中文件对象和目标对象存储服务器空文件对象之间的数据传输通道,这样,就可以将文件对象数据直接由对象存储服务器中文件对象迁移至目标对象存储服务器空文件对象中,不需要客户端参与,解决了现有数据迁移操作时,传输路径长且易出现数据传输错误的问题。 [0084] The present embodiment of the data migration method disclosed in the embodiment, when it is necessary to migrate a file stored in the object storage server source object to the target object storage server, creating an empty file object in the target object storage server, and then establishing the source object storing data transmission channel between server and target object file storage server null file object, so that it can migrate directly from the file object data objects stored in the server file object to a target object is an empty file storage server object, the client does not need to when the end involved to solve the existing data migration operation, the transmission path is long and prone to data transmission errors.

[0085] 并且,由于预先设定了触发文件对象迁移的条件,当判断文件的属性值满足该预设条件时,就自动执行文件对象迁移操作,不仅能够解决现有技术中出现的数据传输路径长且易出现数据传输错误的问题,还提高了工作效率。 [0085] Further, since the preset trigger conditions migrated file object, the attribute value is determined when the predetermined condition is satisfied file, the migration file object automatically performs the operation, not only can solve the prior art data transmission path occurring long and prone to error data transmission problems, but also improve work efficiency.

[0086] 上述实施例中,由于文件数量较多,且其组成单元文件对象可能存储在不同的对象存储服务器中,如果只在每个文件的属性发生变化后逐一判断是否满足预设条件,那么效率会很低,为解决上述问题,所以本发明另一实施例还提供了一种数据迁移方法。 [0086] In the above embodiment, because the number of files and file objects constituting units of the object may be in different storage server, it is determined by one if a change occurs only in the memory of each file attribute meets a preset condition, low efficiency, to solve the above problems, it is a further embodiment of the present invention further provides a method for data migration.

[0087] 本实施例公开的数据迁移方法,如图3所示,包括: [0087] The data migration method disclosed embodiment, shown in Figure 3, comprising:

[0088] 步骤S21、缓存文件的文件路径或者文件名称; [0088] step S21, the file path or file name of the cache file;

[0089] 步骤S22、按照文件满足预设条件程度的顺序排序所述文件路径或者文件名称,形成文件列表; [0089] step S22, the file meets a preset condition according to the degree of the sequential ordering file path or file name, the file list is formed;

[0090] 步骤S23、监控所述文件列表中未满足预设条件文件的状态; [0090] step S23, the list of files to monitor the status of files in a preset condition is not satisfied;

[0091] 步骤S24、若所述文件状态发生变化,获取该文件变化后的属性值; [0091] Step S24, the file attribute value if the state changes, the file acquired after the change;

[0092] 步骤S25、当判断该文件的属性值满足阈值,需执行文件对象迁移操作时,将当前存储有所述文件中文件对象的对象存储服务器作为源对象存储服务器,迁移后存储文件中文件对象的对象存储服务器作为目标对象存储服务器; [0092] step S25, the when determining the property value of the file satisfies a threshold, perform file object migration, currently stores the file in the file object object storage server as the source object storage server, after migration store file file object object storage server as the target object storage server;

[0093] 步骤S26、在所述目标对象存储服务器中创建空文件对象; [0093] step S26, the creating an empty file object in the target object storage server;

[0094] 步骤S27、建立所述源对象存储服务器中文件对象和目标对象存储服务器空文件对象之间的数据传输通道,将文件对象数据由所述源对象存储服务器中文件对象传输至所述目标对象存储服务器中空文件对象。 [0094] step S27, the data transmission path established between the source object storage server stores a file server and target object null file object, the object data file by a file object transmission of the source object to the target storage server object storage file server hollow object.

[0095] 该实施例公开的数据迁移方法的步骤S22中,按照文件满足预设条件程度顺序排序文件具体可以为按照满足预设条件、不满足预设条件的顺序排列文件,也可以为按照不满足预设条件、满足预设条件的顺序排列文件,形成文件列表,除此之外,本实施例公开的步骤S24-S27的内容与上述实施例公开的步骤S11-S14公开的内容相同,请参见上述实施例公开的内容,此处不再赘述。 [0095] The data migration method disclosed in the examples step S22, the file satisfies predetermined extent according to the conditions specific sequential ordering file may be in accordance with the order file satisfies a preset condition, the preset condition is not satisfied, it may not be in accordance with satisfies a preset condition, the preset condition is satisfied order file, the file list is formed, in addition, the steps of step S24-S27 disclosed embodiments to the above disclosed embodiments of the present embodiment is the same as disclosed in S11-S14, please Referring to the embodiment disclosed above content, not further described herein.

[0096] 在本实施例公开的数据迁移方法中,只需监控上述的文件列表中未满足预设条件的文件的状态,不仅解决了现有技术中出现的数据传输路径长且易出现数据传输错误的问题,还缩短了数据处理时间,加快了数据处理速度。 [0096] In the present embodiment the data migration method disclosed embodiment, simply monitor the status of the file list in the above document preset condition is not satisfied, not only solved the prior art data transmission path occurring long and prone data transmission the wrong question, but also shorten the data processing time, speed up data processing speed.

[0097] 上述两个实施例中,由于在文件对象数据的迁移操作中,会有例如网络中端、版本更迭等因素可能导致数据迁移的不完整或不一致,因此,在文件对象数据迁移结束后,还需要对存储在目标对象存储服务器内的内容进行检测,具体操作如下: [0097] In the above two embodiments, since the object data file migration operation, the network end will e.g., version and other factors may cause the change of the data migration is incomplete or inconsistent, thus ending the data migration in file object , also needs content stored in the storage server within the target object is detected, as follows:

[0098] 分别读取存储在所述源对象存储服务器中文件对象和目标对象存储服务器中文件对象的预设位置的数据; [0098] In reading data, respectively, of the source object storage server stores the file in the preset position and target objects in an object file storage server;

[0099] 比对所述数据是否相同,若相同则得到传输成功结果,否则得到传输失败结果。 [0099] is the same as the ratio of data transmission is successful if the same result is obtained, or transmission failure result obtained.

[0100] 读取源对象存储服务器中文件对象和目标对象存储服务器中文件对象相同位置的数据,若文件对象数据传输成功,两个位置的数据应该相同,若不成功,两个位置的数据则会不同。 Data [0100] source object storage server reads the file server and target objects in the same storage location file object, if the object data file transmission is successful, the data of the two positions should be the same, if not successful, the data is two positions It will be different.

[0101] 并且,每次进行检测前,均可以随机设定几个位置,这里的位置并不固定,避免出现数据造假伪造的问题。 Before [0101] Then, each time the detection, can be set at random a few positions, where the location is not fixed, to avoid the problem of false data falsification occurred. 对于大文件对象,设定的位置还要多一点,对于小文件对象,设定的位置可以少一点。 For large file objects, the set position even a little more, for a small file object, you can set the position of a little less.

[0102] 当判断文件传输失败时,还包括检测得到文件输出失败的原因,具体操作如下: [0102] When it is determined that file transfer failed, further comprising detecting the failure reason to give an output file, as follows:

[0103] 文件对象迁移完成后,记录文件对象迁移过程中返回值、文件对象的传输时间以及文件对象的大小,其中所述返回值一般包括:(1)资源忙;(2)存储空间不够;(3)目标文件锁定;⑷连接出错;(5)被中断; After [0103] completion of the migration file object, object migration log file during the return value, the transmission time and file size of the object file object, wherein said return value generally includes: (1) resource is busy; (2) is not enough storage space; (3) target file locking; ⑷ connection error; (5) is interrupted;

[0104] 利用以上信息就可以得到文件传输失败的原因: [0104] With the above information can be obtained A file transfer failed:

[0105] (1)如果目标文件对象小于源文件对象,则报网络错; [0105] (1) if the target object is smaller than the source file object file, the packet network error;

[0106] (2)如果目标文件对象内容与源文件对象不同,则报传输链路错; [0106] (2) If the target file object contents of the object different from the source file, the packet transmission link fault;

[0107] (3)如果目标文件对象传输时间太久,偏离经验值太多,则报网络传输链路错;如果返回值为资源忙,则报目标文件对象所在存储设备错; [0107] (3) if the target object is a file transfer time is too long, too much deviation from empirical values, the packet network transmission link fault; if the return value of the resource is busy, the message storage device where the file object target fault;

[0108] (4)如果返回值为存储空间不够,则报目标文件对象所在的存储设备空间不足; [0108] (4) if the return is not enough storage space, the reported lack of storage space the target file object is located;

[0109] (5)如果目标文件对象被锁定,则报存在多个进程同时管理; [0109] (5) If the target file object is locked, it is reported that there are multiple processes at the same time management;

[0110] (6)如果返回值为连接出错,则报传输链路错; [0110] (6) if an error return value is connected, the packet transmission link fault;

[0111] (7)如果放回值为被中断,则报传输链路错。 [0111] (7) If the returned value is interrupted, the packet transmission link fault.

[0112] 当文件对象传输完毕后,还要制作传输结果汇报表,该表中汇总有本次文件对象迁移操作的执行结果,具体的,若传输成功,则在表中显示文件迁移对象操作执行成功,若传输失败,则将上述传输失败的原因写入该表中。 [0112] When the file transfer is complete the object, but also to produce a transmission result report table, which has summarized the results of this operation of the document object migration, particularly, if the transmission is successful, the display operation is performed in the object file migration table successful, if a transmission fails, then the reason for the above-described transmission failure is written in the table.

[0113] 制作完毕后,可以直接输出该表,具体、采用屏幕显示的方法,也可以将其存储,等用户执行调用操作时,再将其显示。 After the [0113] production is completed, the table can be directly output, specifically, a screen display method, may be stored, when the user performs a calling operation and the like, and then display it.

[0114] 本发明实施例还公开了一种数据迁移装置,如图4所示,包括:第一监控单元201、 第一文件属性值获取单元202、第一判断单元203、第一对象存储服务器类别确定单元204、 第一空文件对象创建单元205以及第一文件对象迁移单元206,其中: [0114] Example embodiments of the present invention further discloses a data migration apparatus, as shown in FIG 4 comprising: a first monitoring unit 201, a first file attribute value acquisition unit 202, a first determining unit 203, a first object storage server class determining unit 204, a first null file object file creating unit 205, and a first object migration unit 206, wherein:

[0115] 第一监控单元201,用于监控文件的文件状态; [0115] The first monitoring unit 201 for monitoring the state of the file;

10[0116] 第一文件属性值获取单元202,用于若所述文件状态发生变化,获取该文件变化后的属性值; 10 [0116] a first file attribute value acquisition unit 202, the file status attribute values ​​if a change occurs, obtain the file after the change;

[0117] 第一判断单元203,用于判断获取到的文件的属性值是否满足阈值; [0117] The first determination unit 203 for determining whether the acquired file attribute value satisfies a threshold;

[0118] 第一对象存储服务器类别确定单元204,用于当判断该文件的属性值满足阈值,需执行文件对象迁移操作时,将当前存储有所述文件中文件对象的对象存储服务器作为源对象存储服务器,迁移后存储文件中文件对象的对象存储服务器作为目标对象存储服务器; [0118] The first object storage server class determining unit 204, an object storage server for determining when the file attribute value satisfies a threshold, the object needs to perform file migration operations, the current file is stored in the file object as the source object storage server, the file migration store file objects object storage server as the target object storage server;

[0119] 第一空文件对象创建单元205,用于在确定的目标对象存储服务器中创建空文件对象; [0119] The first object is an empty file creating unit 205 for creating an empty file objects in the target object in the determined storage server;

[0120] 第一文件对象迁移单元206,用于建立所述源对象存储服务器中文件对象和目标对象存储服务器空文件对象之间的数据传输通道,将文件对象数据由所述源对象存储服务器中文件对象传输至所述目标对象存储服务器空文件对象。 [0120] The first object migration file unit 206, for establishing a data transmission path between the source object storage server stores a file server and target object null file object, the object data file by the source object storage server file object is transmitted to the target object empty file storage server object.

[0121] 当文件传输完毕后,对文件对象的传输效果进行检测,如图5所示,本发明再一实施例公开的数据迁移装置,还包括: [0121] When the file transfer is complete, the effect of the transmission file object is detected, shown in Figure 5, the data migration apparatus of another embodiment of the disclosed embodiment of the present invention, further comprising:

[0122] 第一文件数据读取单元207,用于当文件对象数据传输完毕后,分别读取存储在所述源对象存储服务器中文件对象和目标对象存储服务器中文件对象预设位置的数据; After the [0122] first file data reading unit 207, configured to, when the file transfer is complete object data, respectively read files stored on the server and target objects stored in the object file in the preset position data of the source object storage server;

[0123] 第一传输结果判定单元208,用于比对所述数据是否相同,若相同则得到传输成功结果,否则得到传输失败结果。 [0123] The first transmission result judgment unit 208 is the same than for the data transmission is successful if the same result is obtained, or transmission failure result obtained.

[0124] 第一传输结果汇报表制作单元209,用于制作传输结果汇报表,该传输结果汇报表内保存有文件对象传输结果,当文件对象传输结果为传输失败时,还保存有文件对象传输失败的原因。 [0124] report the results of a first transfer table generation unit 209, for the production of transmission for reporting the results, the transmission result reporting of file objects stored transmission result, when a file object is transmitted as a transmission failure result, further transfer file objects stored reason of failure.

[0125] 具体的,上述两个实施例公开的数据迁移装置的工作如上述三个实施例公开的数据迁移方法公开的内容,请参见上述数据迁移方法中公开的内容,此处不再赘述。 [0125] Specifically, the content of the work data migration apparatus disclosed in the above two embodiments of the data migration method as the above-described three embodiments disclosed embodiment disclosed, see the contents of the data migration method disclosed is not repeated here.

[0126] 当文件的属性值未满足触发执行文件对象迁移操作的条件时,是不对该文件对象执行文件迁移操作,但是,若用户需要迁移文件对象,可以通过客户端直接发送控制指令, 执行文件对象迁移操作。 [0126] When the attribute value of the file is not the object migration operation conditions triggers execution file satisfied, it is not migrate the file object to perform file operations, however, if the user needs to migrate a file object that can send control commands directly through the client, the execution file Object migration. 具体请参见以下实施例公开的数据迁移方法。 For details, see the data migration method disclosed embodiment the following embodiments.

[0127] 本发明实施例公开的一种数据迁移方法,如图6所示,包括: [0127] A data migration method according to the present invention, the disclosed embodiments, as shown in FIG 6, comprising:

[0128] 步骤S31、当接收到客户端发送的执行文件对象迁移操作的控制指令时,将当前存储有所述文件中文件对象的对象存储服务器作为源对象存储服务器,迁移后存储文件中文件对象的对象存储服务器作为目标对象存储服务器; [0128] step S31, the control instruction is received when the execution of the migration file object sent by the client, all objects currently stored in said file storage server file object as the source object storage server, file storage after the migration file object object storage server as the target object storage server;

[0129] 步骤S32、在所述目标对象存储服务器中创建空文件对象; [0129] step S32, creates an empty object in the target object file storage server;

[0130] 步骤S33、建立所述源对象存储服务器中文件对象和目标对象存储服务器空文件对象之间的数据传输通道,将文件对象数据由所述源对象存储服务器中文件对象传输至所述目标对象存储服务器中空文件对象。 [0130] step S33, the data transmission path established between the source object storage server stores a file server and target object null file object, the object data file by a file object transmission of the source object to the target storage server object storage file server hollow object.

[0131] 同样,本实施例公开的数据迁移方法,文件对象传输完毕后,需要对文件对象的传输效果进行检测,并且,当文件对象传输失败时,还要获取文件对象传输失败原因,最终制作传输结果汇报表。 [0131] Similarly, the data migration method disclosed embodiment of the present embodiment, the file transfer is complete the object, the need to effect transfer of the file object is detected, and, when the object is a file transfer failed, but also obtain a file object transmission failure reason, making the final transmission for reporting the results.

[0132] 具体的,文件对象传输效果的检测方法为: [0132] Specifically, the detection method of the file object transfer effect:

[0133] 分别读取存储在所述源对象存储服务器中文件对象和目标对象存储服务器中文件对象的预设位置的数据;[0134] 比对所述数据是否相同,若相同则得到传输成功结果,否则得到传输失败结果。 [0133] The read data stored in each preset position of the target object and the file object storage server file objects in the source object storage server; [0134] is the same as the ratio of data transmission is successful if the same result is obtained otherwise, get a transmission failure result.

[0135] 同样,具体检测过程如上述数据迁移方法实施例公开的内容,此处不再赘述。 [0135] Similarly, as the above-described specific detection process content data migration method disclosed in Example embodiments will not be repeated here.

[0136] 本发明另一实施例公开了一种数据迁移装置,如图7所示,包括: Another embodiment [0136] The present invention discloses a data migration apparatus, shown in Figure 7, comprising:

[0137] 第二对象存储服务器类别确定单元301,用于当接收到客户端发送的执行文件对象迁移操作的控制指令时,将当前存储有所述文件中文件对象的对象存储服务器作为源对象存储服务器,迁移后存储文件中文件对象的对象存储服务器作为目标对象存储服务器; When the [0137] second server object storage class determining unit 301, when receiving a file to execute a control command sent by the client object migration operation, the current file is stored in the object file object as the source object storage server stores server, after migration store files in the file object object storage server as the target object storage server;

[0138] 第二空文件对象创建单元302,用于在所述目标对象存储服务器中创建空文件对象; [0138] The second object is an empty file creating unit 302 for creating an empty file object in the target object storage server;

[0139] 第二文件对象迁移单元303,用于建立所述源对象存储服务器中文件对象和目标对象存储服务器空文件对象之间的数据传输通道,将文件对象数据由所述源对象存储服务器中文件对象传输至所述目标对象存储服务器空文件对象。 [0139] The second object is a file migration unit 303, for establishing a data transmission path between the source object storage server stores a file server and target object null file object, the object data file by the source object storage server file object is transmitted to the target object empty file storage server object.

[0140] 同样,如图8所示,本发明另一实施例还公开了一种数据迁移装置,除包含第二对象存储服务器类别确定单元301、第二空文件对象创建单元302以及第二文件对象迁移单元303外,还包括: [0140] Also, as shown in FIG. 8, another embodiment of the present invention further discloses a data migration apparatus, comprising in addition to the second object storage server class determining unit 301, a second null file object file creating unit 302 and a second migration unit 303 outside the object, further comprising:

[0141] 第二文件数据读取单元304,用于当文件对象数据传输完毕后,分别读取存储在所述源对象存储服务器中文件对象和目标对象存储服务器中文件对象的预设位置的数据; [0141] The second file data reading unit 304, configured to, when the file transfer is complete object data, read data stored in each of the source object storage server in a preset position and the target object file object file storage server object ;

[0142] 第二传输结果判定单元305,用于比对所述数据是否相同,若相同则得到传输成功结果,否则得到传输失败结果。 [0142] The second determination result transmitting unit 305, is the same than for the data transmission is successful if the same result is obtained, or transmission failure result obtained.

[0143] 上述两个实施例公开的数据迁移装置的数据迁移工作与上述实施例公开的内容相同,请参见上述实施例公开的内容,此处不再赘述。 [0143] Data Migration Data migration apparatus disclosed in the above two embodiments with the content disclosed embodiments above, see the contents of the above-described embodiments of the present disclosure is not repeated here.

[0144] 本发明实施例还公开了一种面向对象的分布式文件系统,如图9所示,包括:客户端401、元数据服务器402和对象存储服务器403,其中: [0144] Example embodiments of the present invention also discloses a distributed object-oriented file system, shown in Figure 9, comprising: a client 401, the metadata server 402 and the object storage server 403, wherein:

[0145] 客户端401包括控制指令发送单元11,该单元用于发送控制指令至所述元数据服务器; [0145] The client 401 includes a control command transmitting unit 11, the means for transmitting the control command to the meta data server;

[0146] 元数据服务器402,包括: [0146] The metadata server 402, comprising:

[0147] 监控单元21,用于监控该系统存储文件的文件状态以及客户端401的控制指令; [0147] The monitoring unit 21 for monitoring the state of the system to store the file, and a client 401, a control command;

[0148] 文件属性值获取单元22,用于若所述文件状态发生变化,获取该文件变化后的属性值; [0148] file attribute value acquisition unit 22, the file status attribute values ​​if a change occurs, obtain the file after the change;

[0149] 判断单元23,用于判断获取到的文件的属性值是否满足阈值; [0149] 23 determination unit, for determining the value of the acquired file attribute meets a threshold;

[0150] 对象存储服务器类别确定单元24,用于当判断该文件的属性值满足阈值,需执行文件对象迁移操作时,或者,当客户端401的控制指令为执行文件对象迁移操作时,将当前存储有所述文件中文件对象的对象存储服务器作为源对象存储服务器,迁移后存储文件中文件对象的对象存储服务器作为目标对象存储服务器; [0150] Object Storage Server class determining unit 24 for, when determining the property value of the file satisfies a threshold, perform file object migration operation, or when the control command 401 is a client to execute a file object migration operation, the current the file is stored in the server object storage file object as the source object storage server, the storage server stores the migration target file object as the target object file storage server;

[0151] 空文件对象创建单元25,用于在所述目标对象存储服务器中创建空文件对象; [0151] null file object creating unit 25 for creating an empty file object in the target object storage server;

[0152] 文件对象迁移单元26,用于建立所述源对象存储服务器中文件对象和目标对象存储服务器中空文件对象之间的数据传输通道,将文件对象数据由所述源对象存储服务器中文件对象传输至所述目标对象存储服务器中空文件对象。 [0152] Object file migration means 26 for establishing a data transmission path between the source object storage server stores the target object and the file object file server hollow objects, the object data file by the source object storage server file object transmitted to the target object file storage server of the hollow object.

[0153] 面向对象的分布式文件系统中,包含至少一个对象存储服务器,客户端可以直接访问所述对象存储服务器,发送控制指令,控制其执行操作,并不限于本实施例中说明的迁移对象存储服务器中文件对象操作。 [0153] for the distributed file system object, the object comprising at least one storage server, the client can directly access the server object storage, transmission control commands, which performs a control operation, is not limited to the embodiment described migration object of the present embodiment Object storage server file operations.

[0154] 本说明书中各个实施例采用递进的方式描述,每个实施例重点说明的都是与其他实施例的不同之处,各个实施例之间相同相似部分互相参见即可。 [0154] In the present specification, the various embodiments described in a progressive manner, differences from the embodiment and the other embodiments each of which emphasizes embodiment, the same or similar portions between the various embodiments refer to each other.

[0155] 对所公开的实施例的上述说明,使本领域专业技术人员能够实现或使用本发明。 [0155] The above description of the disclosed embodiments enables those skilled in the art to make or use the present invention. 对这些实施例的多种修改对本领域的专业技术人员来说将是显而易见的,本文中所定义的一般原理可以在不脱离本发明的精神或范围的情况下,在其它实施例中实现。 Various modifications to these professionals skilled in the art of the present embodiments will be apparent, and the generic principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the invention. 因此,本发明将不会被限制于本文所示的这些实施例,而是要符合与本文所公开的原理和新颖特点相一致的最宽的范围。 Accordingly, the present invention will not be limited to the embodiments shown herein but is to be accorded herein consistent with the principles and novel features disclosed widest scope.

Claims (13)

1. 一种数据迁移方法,其特征在于,包括: 监控文件的文件状态;若所述文件状态发生变化,获取该文件变化后的属性值;当判断该文件的属性值满足阈值,需执行文件对象迁移操作时,将当前存储有所述文件中文件对象的对象存储服务器作为源对象存储服务器,迁移后存储文件中文件对象的对象存储服务器作为目标对象存储服务器;在所述目标对象存储服务器中创建空文件对象;建立所述源对象存储服务器中文件对象和目标对象存储服务器空文件对象之间的数据传输通道,将文件对象数据由所述源对象存储服务器中文件对象传输至所述目标对象存储服务器中空文件对象。 A data migration method comprising: monitoring a status file; if the file status change, acquires the file attribute value change; when it is judged that the attribute value satisfies a threshold file, the file needs to be performed when the object migration operation, the current file is stored in the object storage server file object as the source object storage server, the storage server stores the migration target file object file storage server as the target object; in the target object storage server Create an empty file object; establishing a data transmission path between the source object storage server stores a file server and target object null file object, the object data file by a file object transmission of the source object storage server to the target object file storage server hollow object.
2.根据权利要求1所述的方法,其特征在于,所述监控文件状态为监控文件列表中未满足预设条件文件的状态,监控文件状态前还包括:缓存文件的文件路径或者文件名称;按照文件满足预设条件程度的顺序排序所述文件路径或者文件名称,形成文件列表。 2. The method according to claim 1, wherein the monitor monitors file list file state is preset condition is not satisfied in the state of the file, the file before the status monitor further comprises: a file path or file name of the cache file; file satisfies a preset condition according to the degree of the sequential ordering file path or file name, the file list is formed.
3.根据权利要求1所述的方法,其特征在于,所述文件对象数据传输完毕后还包括: 分别读取存储在所述源对象存储服务器中文件对象和目标对象存储服务器中文件对象的预设位置的数据;比对所述数据是否相同,若相同则得到传输成功结果,否则得到传输失败结果。 3. The method according to claim 1, wherein, after completion of the transfer object data file further comprises: reading each pre-file storage server and target objects in the object file stored in the source object storage server position data set; is the same as the ratio of data transmission is successful if the same result is obtained, or transmission failure result obtained.
4.根据权利要求3所述的方法,其特征在于,还包括:检测得到文件对象数据传输失败原因。 4. The method according to claim 3, characterized in that, further comprising: detecting a file obtained object data transmission failure reason.
5.根据权利要求4所述的方法,其特征在于,所述文件对象数据传输完毕后,还包括: 制作传输结果汇报表,该传输结果汇报表中含有文件对象数据传输结果信息或文件对象数据传输失败原因信息。 The method according to claim 4, characterized in that, after the object data file transfer is complete, further comprising: a transmission result reporting of production, transmission result reporting table contains the data transfer result information file object file or object data transmission failure cause information.
6. 一种数据迁移装置,其特征在于,包括: 第一监控单元,用于监控文件的文件状态;第一文件属性值获取单元,用于若所述文件状态发生变化,获取该文件变化后的属性值;第一判断单元,用于判断获取到的文件的属性值是否满足阈值; 第一对象存储服务器类别确定单元,用于当判断该文件的属性值满足阈值,需执行文件对象迁移操作时,将当前存储有所述文件中文件对象的对象存储服务器作为源对象存储服务器,迁移后存储文件中文件对象的对象存储服务器作为目标对象存储服务器; 第一空文件对象创建单元,用于在确定的目标对象存储服务器中创建空文件对象; 第一文件对象迁移单元,用于建立所述源对象存储服务器中文件对象和目标对象存储服务器空文件对象之间的数据传输通道,将文件对象数据由所述源对象存储服务器中文件对象传 A data migration apparatus, characterized by comprising: a first monitoring unit for monitoring the state of the file in the file; a first file attribute value acquisition unit for file later if a change condition occurs, obtain the file changes attribute value; a first determination unit configured to determine the acquired file attribute value meets a threshold value; a first object storage server class determining means for determining if the file attribute value satisfies a threshold, perform the migration file object when the current file is stored in the object storage server file object as the source object storage server, the server object is stored in the file storage after migration file object as the target object storage server; a first null file object creation means for audience the determined storage server to create an empty file objects; a first document object migration unit for establishing a data transmission path between the source object storage server stores a file server and target object null file object, the object data file storage server by the source object file object by 输至所述目标对象存储服务器空文件对象。 Output to the target object empty file storage server object.
7.根据权利要求6所述的装置,其特征在于,还包括:第一文件数据读取单元,用于当文件对象数据传输完毕后,分别读取存储在所述源对象存储服务器中文件对象和目标对象存储服务器中文件对象预设位置的数据;第一传输结果判定单元,用于比对所述数据是否相同,若相同则得到传输成功结果,否则得到传输失败结果。 7. The device according to claim 6, characterized in that, further comprising: a first data file reading unit for, when the object data file transfer is complete, read the file objects are stored in the source object storage server and audience data file object storage server predetermined position; a first determination result transmitting unit is the same than for the data transmission is successful if the same result is obtained, or transmission failure result obtained.
8.根据权利要求6所述的装置,其特征在于,还包括:第一传输结果汇报表制作单元, 用于制作文件对象数据传输结果汇报表。 8. The apparatus according to claim 6, characterized in that, further comprising: a first production unit for reporting a transmission result, for the production of the file object data transmission for reporting the results.
9. 一种数据迁移方法,其特征在于,包括:当接收到客户端发送的执行文件对象迁移操作的控制指令时,将当前存储有所述文件中文件对象的对象存储服务器作为源对象存储服务器,迁移后存储文件中文件对象的对象存储服务器作为目标对象存储服务器;在所述目标对象存储服务器中创建空文件对象;建立所述源对象存储服务器中文件对象和目标对象存储服务器空文件对象之间的数据传输通道,将文件对象数据由所述源对象存储服务器中文件对象传输至所述目标对象存储服务器中空文件对象。 A data migration method comprising: when receiving the document sent by the client performs a control object migration operation instruction, the current object storage server stores said file in the file object as the source object storage server after the migration file is stored in the file object object storage server as the target object storage server; create an empty file object in the target object storage server; establish the source object storage server files and target objects stored in the file object of server space data transmission channel between the object data file transmitted by the server object is stored in the source file object to the target object file storage server of the hollow object.
10.根据权利要求9所述的方法,其特征在于,所述文件对象数据传输完毕后还包括: 分别读取存储在所述源对象存储服务器中文件对象和目标对象存储服务器中文件对象的预设位置的数据;比对所述数据是否相同,若相同则得到传输成功结果,否则得到传输失败结果。 10. The method according to claim 9, wherein, after completion of the transfer object data file further comprises: reading each pre-file storage server and target objects in the object file stored in the source object storage server position data set; is the same as the ratio of data transmission is successful if the same result is obtained, or transmission failure result obtained.
11. 一种数据迁移装置,其特征在于,包括:第二对象存储服务器类别确定单元,用于当接收到客户端发送的执行文件对象迁移操作的控制指令时,将当前存储有所述文件中文件对象的对象存储服务器作为源对象存储服务器,迁移后存储文件中文件对象的对象存储服务器作为目标对象存储服务器; 第二空文件对象创建单元,用于在所述目标对象存储服务器中创建空文件对象; 第二文件对象迁移单元,用于建立所述源对象存储服务器中文件对象和目标对象存储服务器空文件对象之间的数据传输通道,将文件对象数据由所述源对象存储服务器中文件对象传输至所述目标对象存储服务器空文件对象。 A data migration apparatus, characterized by comprising: a second server object storage class determining unit, when receiving the control command file execution object migration operation sent by the client, the file will be stored in the current object storage server file object as the source object storage server, the migration file objects stored in the object file storage server as the target object storage server; a second empty file object creation means for creating an empty file in the target object storage server objects; second file object migration unit for establishing a data transmission path between the file server and target object storage space of the source object file storage server objects, the object data file by the source object storage server file object transmitted to the target object empty file storage server object.
12.根据权利要求11所述的装置,其特征在于,还包括:第二文件数据读取单元,用于当文件对象数据传输完毕后,分别读取存储在所述源对象存储服务器中文件对象和目标对象存储服务器中文件对象的预设位置的数据;第二传输结果判定单元,用于比对所述数据是否相同,若相同则得到传输成功结果,否则得到传输失败结果。 12. The apparatus as claimed in claim 11, characterized in that, further comprising: a second data file reading unit for, when the object data file transfer is complete, read the file objects are stored in the source object storage server and the predetermined position of the object storage server file object data; a second transmission result determining means for determining whether the ratio of the same data transmission is successful if the same result is obtained, or transmission failure result obtained.
13. 一种面向对象的分布式文件系统,其特征在于,包括:客户端、元数据服务器和对象存储服务器,其中:所述客户端包括控制指令发送单元,该单元用于发送控制指令至所述元数据服务器; 所述元数据服务器,包括:监控单元,用于监控文件的文件状态以及客户端的控制指令; 文件属性值获取单元,用于若所述文件状态发生变化,获取该文件变化后的属性值; 判断单元,用于判断获取到的文件的属性值是够满足阈值;对象存储服务器类别确定单元,用于当判断该文件的属性值满足阈值,需执行文件对象迁移操作时,或者,当所述客户端的控制指令为执行文件对象迁移操作时,将当前存储有所述文件中文件对象的对象存储服务器作为源对象存储服务器,迁移后存储文件中文件对象的对象存储服务器作为目标对象存储服务器;空文件对象创建单元 13. An object-oriented distributed file system comprising: a client, a metadata server and the server object is stored, wherein: the client comprises sending the control instruction unit which transmits a control command to the said metadata server; the metadata server, comprising: a monitoring unit, for controlling the state of the instruction file and the client monitoring file; file attribute value acquisition unit for file if the state changes, the file obtaining changes attribute value; determining means, an attribute value determination acquired file is enough to meet the threshold; object storage server class determining unit configured to, when determining the property value of the file satisfies a threshold, perform file object migration, or when the control instruction for the client to perform file migration target, the current file is stored in the object storage server file object as the source object storage server, the storage server stores the migration target file object file as the target object storage server; empty file object creation unit ,用于在所述目标对象存储服务器中创建空文件对象; 文件对象迁移单元,用于建立所述源对象存储服务器中文件对象和目标对象存储服务器中空文件对象之间的数据传输通道,将文件对象数据由所述源对象存储服务器中文件对象传输至所述目标对象存储服务器中空文件对象。 For creating an empty file object in the target object storage server; file object migration unit for establishing a data transmission path between the source object storage server stores a file server and target object hollow file object file object data transmission by said source object storage server file object to the target object file storage server of the hollow object.
CN2010105273897A 2010-10-28 2010-10-28 Data migration method and device, and object-oriented distributed file system CN102456049A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN2010105273897A CN102456049A (en) 2010-10-28 2010-10-28 Data migration method and device, and object-oriented distributed file system

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN2010105273897A CN102456049A (en) 2010-10-28 2010-10-28 Data migration method and device, and object-oriented distributed file system

Publications (1)

Publication Number Publication Date
CN102456049A true CN102456049A (en) 2012-05-16

Family

ID=46039246

Family Applications (1)

Application Number Title Priority Date Filing Date
CN2010105273897A CN102456049A (en) 2010-10-28 2010-10-28 Data migration method and device, and object-oriented distributed file system

Country Status (1)

Country Link
CN (1) CN102456049A (en)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN103078933A (en) * 2012-12-29 2013-05-01 深圳先进技术研究院 Method and device for determining data migration time

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN1570842A (en) * 2003-03-27 2005-01-26 株式会社日立制作所 Storage device
CN101079902A (en) * 2007-06-29 2007-11-28 清华大学 A great magnitude of data hierarchical storage method
CN101460934A (en) * 2006-05-29 2009-06-17 微软公司 Creating frequent application-consistent backups efficiently
CN101582787A (en) * 2008-05-16 2009-11-18 中兴通讯股份有限公司 Double-computer backup system and backup method
CN101621535A (en) * 2009-06-19 2010-01-06 许继集团有限公司;许继电气股份有限公司 Network communication method and device of real-time monitoring system

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN1570842A (en) * 2003-03-27 2005-01-26 株式会社日立制作所 Storage device
CN101460934A (en) * 2006-05-29 2009-06-17 微软公司 Creating frequent application-consistent backups efficiently
CN101079902A (en) * 2007-06-29 2007-11-28 清华大学 A great magnitude of data hierarchical storage method
CN101582787A (en) * 2008-05-16 2009-11-18 中兴通讯股份有限公司 Double-computer backup system and backup method
CN101621535A (en) * 2009-06-19 2010-01-06 许继集团有限公司;许继电气股份有限公司 Network communication method and device of real-time monitoring system

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN103078933A (en) * 2012-12-29 2013-05-01 深圳先进技术研究院 Method and device for determining data migration time
CN103078933B (en) * 2012-12-29 2015-12-02 深圳先进技术研究院 A method and apparatus for determining timing data migration

Similar Documents

Publication Publication Date Title
US9560093B2 (en) Higher efficiency storage replication using compression
US9684453B2 (en) Cluster federation and trust in a cloud environment
US9069827B1 (en) System and method for adjusting membership of a data replication group
US9396206B2 (en) Peer-to-peer redundant file server system and methods
CN103425734B (en) The controller stores a database encoded triplets, methods and systems
US9367252B2 (en) System and method for data replication using a single master failover protocol
JP2016027476A (en) System and method for implementing scalable data storage service
US9053167B1 (en) Storage device selection for database partition replicas
CN101414277A (en) Need-based increment recovery disaster-containing system and method based on virtual machine
CN101562543A (en) Cache data processing method and processing system and device thereof
CN101957863B (en) Data parallel processing method, device and system
US8719225B1 (en) System and method for log conflict detection and resolution in a data store
CN101183377A (en) High availability data-base cluster based on message middleware
US8930312B1 (en) System and method for splitting a replicated data partition
CN102880475A (en) Real-time event handling system and method based on cloud computing in computer software system
US8694578B2 (en) Swarm-based synchronization over a network of object stores
CN102790771B (en) A document transmission method and system
Zhang et al. BitVault: A highly reliable distributed data retention platform
CN102158540A (en) System and method for realizing distributed database
US8832234B1 (en) Distributed data storage controller
CN102523279B (en) A distributed file system and a file access method hotspot
US8843441B1 (en) System and method for maintaining a master replica for reads and writes in a data store
US9607001B2 (en) Automated failover of a metadata node in a distributed file system
WO2012045245A1 (en) Method and system for maintaining data consistency
US8151062B2 (en) Consistency models in a distributed store

Legal Events

Date Code Title Description
C06 Publication
C10 Entry into substantive examination
C12 Rejection of a patent application after its publication