CN101188569B - Method for constructing data quanta space in network and distributed file storage system - Google Patents

Method for constructing data quanta space in network and distributed file storage system Download PDF

Info

Publication number
CN101188569B
CN101188569B CN 200610114592 CN200610114592A CN101188569B CN 101188569 B CN101188569 B CN 101188569B CN 200610114592 CN200610114592 CN 200610114592 CN 200610114592 A CN200610114592 A CN 200610114592A CN 101188569 B CN101188569 B CN 101188569B
Authority
CN
Grant status
Grant
Patent type
Prior art keywords
node
user
file
service
network
Prior art date
Application number
CN 200610114592
Other languages
Chinese (zh)
Other versions
CN101188569A (en )
Inventor
饶大平
Original Assignee
饶大平
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Grant date

Links

Abstract

The invention relates to a method for constructing data quantum space in the network and a distributed file storage system. Through the distributed file storage system, user nodes use service nodes to choose and upload file blocks of the data files onto usable user nodes in the network so as to form distributed nodes; after the user nodes upload the file blcoks of data files or during the transmission of the file blocks of data files, the service nodes automatically adjust the usable nodes in the whole backbone node network to distribute loading tasks, and distribute the file blocks of the data files that the user requests to upload or transmit and store in the sharing space of a plurality of user nodes through copy, so as to form a plurality of sharing nodes for the user nodes download or transmission of files, thereby constructing and forming the quantum space for data storage and transmission in the network. The invention can realize stable, high efficient and opening enterprise-level distributed storage application.

Description

在网络上构建数据量子空间的方法及分布式文件存储系统技术领域 The method of constructing the quantization of the data space on the network and the distributed file system storage Field

[0001] 本发明涉及一种在网络上构建数据量子空间的方法及分布式文件存储系统,特别是涉及一种基于internet、应用企业级的分布式文件存储系统及在网络上构建数据量子空间的方法。 [0001] The present invention relates to a quantum of data space constructed on the network and a method of distributed file storage system, particularly to a distributed file storage system internet, enterprise applications based on the quantization of the data space and construct the network method. 背景技术 Background technique

[0002] “Peer-to-Peer”对等互联网络技术(点对点网络技术)让用户可以直接连接到其他用户的计算机,进行文件共享与交换,另外“Peer-to-Peer”在深度搜索、分布计算、协同工作等方面也大有用途。 [0002] "Peer-to-Peer" peering network technology (peer to peer network technology) allows users to connect directly to another user's computer for file sharing and exchange, in addition to "Peer-to-Peer" in depth search, distributed aspects of a computing, collaboration, also great use. 其实“Peer-to-Peer”并不是互联网出现后才被引申出来的一项全新网络技术,早在几年前互联网出现的时候“Peer-to-Peer”就已经应运而生,但是由于当时互联网发展的并不是像今天这样普及,而且在网络建设(如带宽、处理能力)和相关的硬件器材上并不能购满足“Peer-to-Peer”的需求,因而在几年的互联网发展过程中也衍生出来了许多中间产物。 In fact, "Peer-to-Peer" after the advent of the Internet is not arise out of a new network technology, the advent of the Internet several years ago when the "Peer-to-Peer" has emerged, but because at that time the Internet development is not so popular like today, but also in the construction of the network (such as bandwidth, processing power) and associated hardware equipment does not meet the shopping needs of "Peer-to-Peer", and therefore also in the development of the Internet course of several years derived from a number of intermediate products.

[0003] 可以看出“Peer-to-Peer” 一个显著的特点就是改变了互联网现有的这种以门户网站为中心的这种大型的网状结构,它重新给予了“非中心化”结构中网络用户应有的权力,也就是说网络应用的核心从中央服务器向网络边缘的终端设备扩散:服务器到服务器、 服务器到PC机、PC机到PC机,PC机到WAP手机……所有网络节点上的设备都可以建立“Peer-to-Peer”对话。 [0003] As can be seen "Peer-to-Peer" a significant feature of this is to change the existing Internet portals to the center of such a large network structure, it again gave a "decentralization" structure network users should have the power, that is to say the core network applications to a terminal edge of the network spread from the central server: server to server, server to the PC, PC-to-PC machine, PC machine to WAP mobile phones ...... all network device on the node can establish "Peer-to-Peer" dialogue. 这使人们在hternet上的共享行为被提到了一个更高的层次。 This allows people to share behavior on hternet be referred to a higher level.

[0004] 文件共享是“Peer-to-Peer”软件最实质的使用宗旨,在国内外众多的“Peer-to-Peef软件中所提供的功能也是多以此为主。这些软件提供了一个大范围文件共享的平台,其用户通过它可以与其它网友交换共享出来的音乐、视频、图书、图片、软件、游戏等功能。 [0004] File sharing is the "Peer-to-Peer" software most essential purpose of use, at home and abroad many of the "Peer-to-Peef software functionality is also provided in this multi-oriented. The software provides a large range of file-sharing platform, through which users can exchange their share out of the music, video, books, images, software, games and other functions with other users.

[0005] 如美国专利申请号为10/957,184,公开号为US2005/0216473的“Peer-to-Peer网络系统”就公开了实现个人用户之间的文件与空间的共享的技术方案。 [0005] The U.S. Patent Application No. 10 / 957,184, Publication No. US2005 / 0216473 the "Peer-to-Peer Network Systems" discloses a technical solution to share files with a space between the individual users. 如图1所示,示出了该申请的网络拓扑结构,该网络拓扑结构为“完全分布式”结构,所有节点直接通过网络NW连接,利用点对点的传输实现个人用户之间的文件与空间的共享、传输。 1, shows a network topology of the application, the network topology as "fully distributed" structure, all nodes directly connected via a network NW, using the transfer point implementation file space between the individual users sharing, transmission. 但是,上述现有技术的“Peer-to-Peer”网络系统没有统一的数据的均衡分布和冗余管理机制,且用户行为存在不可控性,例如,存在用户上传文件的可控性不强的问题,当用户不在线时,一般不能进行文件的上传/下载。 However, the above prior art "Peer-to-Peer" network system is not uniform and balanced distribution of redundant data management system, and there is not controllable user behavior, for example, there is controllability users to upload files is not strong problem, when the user is not online, generally can not file upload / download. 并且,上述现有技术的“Peer-to-Peer”网络系统在网络利用效率(尤其是查找效率)等方面也存在缺陷,网络利用效率低。 Further, the above-described prior art "Peer-to-Peer" networks (especially the searching efficiency), it is also deficient in the efficiency of network utilization, network utilization efficiency is low.

[0006] 另外,现有技术还存在一些“中心化的Peer-to-Peer网络”结构,对于这些结构的“Peer-to-Peer”网络系统中共享文件的“热点”节点来说,容易由于请求共享的节点过多, 导致传输速度的减慢和查找效率的降低,并且,也容易由于请求连接的节点过多,导致节点失效,而当节点失效时又会容易造成数据的丢失,导致数据不完整,因此,系统的稳定性能较差。 [0006] Further, the prior art also has some "center of the Peer-to-Peer Network" configuration, for the "hot" nodes "Peer-to-Peer" network systems of these structures for the shared files, it is easy since too many nodes to share request, and search results in slowing the speed of decrease in the efficiency of the transmission, and, since too many requests can be easily connected to a node, cause the node failure, node failure when would easily lead to loss of data, resulting in data incomplete, therefore, poor performance and stability of the system.

[0007] 特别的是,现有技术的“Peer-to-Peer”网络系统,实际上都是对文件的共享,并没有提出对大文件传输的概念;所以现有系统不能提供真正意义上的“大文件”传输。 [0007] In particular, the prior art "Peer-to-Peer" network system are actually sharing files, and no concept of large file transfer; so existing systems can not provide the true sense "large files" on transmission. 对于大文件的传输来说,现有的网络结构存在着数据传输的瓶颈。 For large file transfers, the presence of a conventional network structure for data transmission bottlenecks.

[0008] 发明内容 [0008] SUMMARY OF THE INVENTION

[0009] 本发明所要解决的技术问题在于提供一种分布式文件存储系统及在网络上构建数据量子空间的方法,解决现有技术的“Peer-to-Peer”网络系统中存在的网络“热点”效应及数据传输的瓶颈,为文件(尤其是大文件)的上传、分布存储、下载和传输等,提供一种稳定、高效、开放的企业级的分布式存储应用。 [0009] The present invention solves the technical problem is to provide a distributed file system and building data storage space on the network quantum method, the prior art to solve the "Peer-to-Peer" network system existing network "hot spots "bottleneck effect and data transmission, the file (especially large file) upload, store distribution, downloading and transmission, to provide a stable, efficient, open, enterprise-class distributed storage applications.

[0010] 为了实现上述目的,本发明提供了一种在网络上构建数据量子空间的方法,该方法包括:构建一分布式文件存储系统,该系统包括由若干个彼此网络连接的服务节点构成的骨干节点网,并以每个服务节点为中心构成一独立的边缘节点网,每一边缘节点网中包含有若干个用户节点,每个用户节点与其所属边缘节点网内的服务节点网络连接,且每个用户节点都具有一预定的共享空间;用户节点利用服务节点选择将数据文件的文件块上传发布到网络中可用的用户节点上,形成分布节点;在用户节点将数据文件的文件块上传后或在数据文件的文件块的传输过程中,服务节点自动调度整个骨干节点网络内的可用节点,分配承载任务,把用户请求上传或传输的数据文件的文件块通过复制,分发存储到多个用户节点的共享空间内,形成多个共享节点,供用户 [0010] To achieve the above object, the present invention provides a method of constructing a quantization of the data space on a network, the method comprising: constructing a distributed file storage system, the system comprising a service node connected to one another by a plurality of network configuration backbone network nodes, each service node and is configured as a separate center edge node network, each node of the edge network contains a plurality of user nodes, each service node within the network user node to which belongs edge-node connections, and after the user node uploads the data file file blocks; each user node having a predetermined shared space; user node using the service node to select the file upload file block data published to the user nodes in the network is available, the distribution node formation during transmission of data files or file blocks, automatic scheduling service node available nodes throughout the network backbone nodes, tasks assigned carrier, the user requests a file block data file uploaded or transmitted by copying, distributing to a plurality of users store nodes within the shared space, forming a plurality of shared nodes for the user 节点下载或传输文件用,从而在网络上构建形成数据存储和传输的量子空间。 Node downloaded or transferred files to construct a quantum space formed of data storage and transmission over a network.

[0011] 为了实现上述目的,本发明还提供了一种分布式文件存储系统,该系统包括由若干个彼此网络连接的服务节点构成的骨干节点网,并以每个服务节点为中心构成一独立的边缘节点网,每一边缘节点网中包含有若干个用户节点,每个用户节点与其所属边缘节点网内的服务节点网络连接,且每个用户节点都具有一预定的共享空间;其中,该用户节点作为数据文件的文件块的分布载体,在网络上共享空间,并使用其它用户节点的共享空间,实现数据文件的上传、分布存储、下载和传输;该服务节点用于支撑服务节点与用户节点之间的存储与查找,可控制完成数据文件的文件块的上传、分布存储、下载和传输。 [0011] To achieve the above object, the present invention also provides a distributed file storage system, the system comprising a network backbone nodes connected by a plurality of serving node network configuration to each other, and each service node in order to constitute an independent center the network edge nodes, each edge node of the network contains a plurality of user nodes, each service node within the network user node to which it belongs edge-node connection, the user node and each having a predetermined shared space; wherein the distribution vector file blocks as a data file of the user node, shared on the network space, and the use of other user nodes in the shared space, to upload, storage distribution, downloading and transferring data files; for supporting the service node serving the user node upload blocks between the storage node and find, to complete the data files can be controlled, distributed memory, download and transfer.

[0012] 根据本发明的一方面,本发明的分布式文件存储系统是一个基于internet的存储系统,主要针对大文件的网络存储与传输。 [0012] According to an aspect of the present invention, the distributed file storage system of the present invention is an internet-based storage system, the primary network for storage and transmission of large files. 系统整体采用了混合型的“Peer-to-Peer” 网络结构,既有利于系统的稳定性,减小网络“热点”的影响,又提高了查找效率,同时还方便了对文件的监控。 Overall system uses a hybrid of "Peer-to-Peer" network structure is conducive to the stability of the system, reducing the impact of network "hot spots", but also improves search efficiency, but also facilitate the monitoring of the file. 本发明的系统的路由算法采用了新型的棋盘映射(CBMP,Chessboard Map)算法,对节点(peer)的加入/离开(join/leave)效率,和资源查找(search)效率都有很出色的表现,同时对于数据安全、完整性都提供了充分的保证。 Routing algorithm system of the present invention uses a new type of checkerboard map (CBMP, Chessboard Map) algorithm, added to the node (peer) / Leave (join / leave) efficiency, and resource lookup (search) has a very good performance efficiency while for data security, integrity provides sufficient guarantees. 本发明的分布式文件存储系统还对第三方应用提供了足够的接口支持,可以绑定邮件,传输,视频等需要高效底层传输支持的应用服务。 Distributed file storage system of the present invention also provides a third-party application support enough interfaces, you can bind mail transmission, video and other application services require efficient underlying transport supports.

[0013] 根据本发明的另一方面,本发明的分布式文件存储系统针对的是企业级的用户, 用于门户或其他内容提供商,可以在资源能能够掌控的前提下,提供优质的下载服务;同时,可以针对不同的需求,提供不同类型的服务,如邮件,文件传输,文件共享等。 [0013] According to another aspect of the invention, a distributed file storage system of the present invention is directed to enterprise-level user for portal or other content providers, may, subject to the control of resources can be able to provide quality downloads service; the same time, for different needs can provide different types of services, such as mail, file transfer, file sharing and so on. 因此,本发明基于以上考虑,系统的实现是考虑了对于客户机与服务器的分离,在服务器端主要实现用户的管理,信息路由查找,网络性能的保证等;在客户端,只是一个简单的下载平台。 Accordingly, the present invention is based on the above considerations, realization of the system is considered isolated for the client and server, the main achievement of the user management server, information on a route lookup to ensure that network performance; in the client, just a simple download platform. 对此,本发明提出了一个“服务器层”的概念,该服务器层可以运行在高性能的机器中, 各服务节点之间都有联系,任何一台断掉了不影响整个系统运行。 In this regard, the present invention proposes a concept of "server layer", this layer may be running on the server machine performance, has links between the serving node, it broke any one does not affect the overall system operation. 并且,在用户层中,各用户节点中存在一个或多个虚拟的超级节点,为用户提供数据完整的保证,以一个“Pure Peer-to-Peer"的结构,实现了一个可靠性很高的“服务器集群”,以对等的身份为客户提供信息存储,路由查找等服务。 And, at the user level, each user node in the presence of one or more virtual super node, to provide a complete data guarantee, to a structure of "Pure Peer-to-Peer", to achieve a high reliability "server clusters", as a peer to provide customers with information storage, routing lookup services. 并且,在“服务器层”的支持下,客户端对文件下载、节点间的通信做了大部分优化。 And, with the support of "server layer," the client for file downloads, communications do most optimized between nodes. 对于客户上传文件,在很大程度上有了可控性,即提出的并不是一个完全对等的概念,而是一个分级的应用,不同的节点(普通用户,特权用户,超级用户)分别在共享空间,共享资源,上传/下载控制等方面有着不同的权限。 For customers to upload files, to a large extent with the concept of controllability, that is not put forward a totally equal, but a classification of applications, different nodes (common user, privileged user, super user), respectively shared space, shared resources, upload / download control, and so has different permissions.

[0014] 以下结合附图和具体实施例对本发明进行详细描述,但不作为对本发明的限定。 [0014] The following embodiments in conjunction with accompanying drawings and specific embodiments of the present invention will be described in detail, but do not limit the present invention. 附图说明 BRIEF DESCRIPTION

[0015] 图1为现有技术的“Peer-to-Peer”网络系统的网络结构示意图; [0015] Fig 1 a schematic view of a network structure "Peer-to-Peer" network system of the prior art;

[0016] 图2为本发明的一较佳的分布式文件存储系统的网络结构示意图; Schematic structural diagram of a preferred network of distributed file storage system [0016] FIG. 2 of the present invention;

[0017] 图3为本发明的另一较佳的分布式文件存储系统的网络结构示意图; Networking diagram [0017] FIG. 3 is another preferred distributed file storage system of the present invention;

[0018] 图4为本发明的服务节点的组成结构示意图; Composition Structure [0018] FIG. 4 serving node schematic diagram of the present invention;

[0019] 图5为本发明的用户节点的组成结构示意图; [0019] FIG. 5 composition structure of the present invention, the user node schematic;

[0020] 图6A为本发明的服务节点标识符的组成结构示意图; Composition Structure [0020] FIG. 6A serving node identifier of the present invention, a schematic view;

[0021] 图6B为本发明的用户节点标识符的组成结构示意图; [0021] FIG. 6B user nodes composition structure of the present invention, a schematic view identifier;

[0022] 图6C为本发明的文件标识符的组成结构示意图; Composition structure of file identifier [0022] FIG 6C is a schematic view of the present disclosure;

[0023] 图7为本发明的服务节点存储表、用户节点存储表与服务节点、用户节点之间的存储关系图; [0023] Figure 7 is stored the relation between the serving node memory table and user table stored in the node serving node, the user node to the invention;

[0024] 图8为基于本发明的分布式文件存储系统的数据文件上传的工作流程图; [0024] FIG 8 is a flowchart of the data files distributed file storage system of the present invention is based on the uploaded;

[0025] 图9为基于本发明的分布式文件存储系统的数据文件复制分布存储的结构示意图; [0025] FIG. 9 is a schematic view of the distribution of replicated data files stored in a distributed file storage system of the present invention is based;

[0026] 图10为基于本发明的分布式文件存储系统的数据文件下载的工作流程图; [0026] FIG. 10 is a data file stored in a distributed file system according to the present invention is based on the download operation flowchart;

[0027] 图IlA为本发明的分布式文件存储系统应用于无线网络的结构示意图,其中移动终端用户与系统网络中的用户节点绑定; [0027] FIG IlA distributed file storage system of the present invention is applied to a schematic structural diagram of a wireless network, wherein the mobile terminal user with the user node in a binding system network;

[0028] 图IlB为图IlA中移动终端通过3G网络与系统的服务节点通信的结构示意图; [0028] FIG IlB FIG schematic configuration IlA mobile terminal node of the communication service through the 3G network and the system;

[0029] 图12为本发明的分布式文件存储系统与邮件系统配合用于大附件邮件传输的结构示意图。 [0029] The distributed file system of FIG mail storage system 12 of the present invention with a schematic structural diagram of message transfer for large attachments. 具体实施方式 detailed description

[0030] 本发明的分布式文件存储系统的一较佳网络结构如图2所示,包括由服务节点组成的服务器层和由用户节点组成的用户层。 [0030] a preferred network of distributed file storage system of the present invention is shown in Figure 2, it includes a server and user levels of user nodes by the service nodes. 其中,该服务器层由若干个彼此网络连接的服务节点构成一骨干节点网,并以每个服务节点为中心构成一边缘节点网。 Wherein the server layer service node connected by a plurality of network nodes to each other constitute a backbone network, to each service node and edge nodes constituting a network center. 该用户层由若干个用户节点组成,每个用户节点分别通过其所属边缘节点网内的服务节点与骨干节点网连接,且每个用户节点都具有一预定的共享空间。 The user layer is composed of a plurality of user nodes, each user node and each service node within the backbone network node which belongs to the network nodes connected by an edge, and each user node having a predetermined shared space. 本发明的服务节点用于支撑服务节点与用户节点的存储与查找,可控制完成数据文件的上传、分布存储、下载和传输。 Serving node of the invention for supporting a service node and user node lookup store, control is completed can upload data files, storage distribution, downloading and transfer. 本发明中的用户节点作为分布文件的实际存储节点,其不仅在网络上共享空间,而且可使用其他节点共享的空间。 User node in the present invention the storage node as the actual file distribution, not only shared space on the network, and may use other nodes shared space. 这些用户节点通过后台进程完成与服务节点的连接,共享空间的管理等,主要提供本地文件上传、传输操作,并为其他扩展应用提供底层支持。 These user node is connected via a background process is complete, shared space management services node, mainly to provide local file upload, transfer operations, and to provide underlying support for the expansion of other applications. 例如,本实施例中,服务节点1〜4构成了该骨干节点网,并以每个服务节点为中心构成了一个边缘节点网,分别维护其分布区域A〜D下的用户节点1〜5,即用户节点1、2从属于服务节点1,用户节点3从属于服务节点2,用户节点4、5从属于服务节点3,而服务节点4为一个空节点,可表示该区域D内暂时没有用户节点存在,当有该区域内的用户节点加入时,由该服务节点4维护。 For example, in this embodiment, the service node 1 ~ 4 constituting the backbone network nodes, each service node and to constitute a network centering edge node, the user node are maintained in their distribution area A~D 1~5, 1, i.e., user node subordinate to serving node 1, node 3 belonging to the user service node 2, node 4, user 3 belonging to the service node, the service node 4 to an empty node, the user may not represent the temporary region D node exists, when there is the region of the user node joins, maintained by the service node 4.

[0031] 在本发明中,骨干节点网中的服务节点可以选择其运行方式:分散式或集中式。 [0031] In the present invention, the backbone network node serving node may choose its operating mode: centralized or distributed. 对于分散式服务节点,服务节点维护与自己相邻或相关的其他服务节点的信息,提供给其他服节节点或用户节点请求的路由计算;对于集中式服务节点,服务节点维护整个网络内服务节点的信息。 For distributed service node, the service node maintenance and other service information related to their neighboring nodes or other services provided to the route calculation section requesting user node or a node; For a centralized service node, the service node serving node maintains the entire network Information. 其中,分散式结构主要用于大型网络,集中式结构主要用于中小型网络。 Wherein the structure is mainly used for large distributed networks, centralized structure is mainly used for small and medium sized networks.

[0032] 在本发明中,以每个服务节点为中心构成独立的边缘节点网,整个骨干节点网络上维护一个分布式的棋盘映射表(CBMP,Chessboard Map),由每个服务节点所维护的、与自己边缘节点网相关的CBMP组成;每个用户节点维护与自己相邻节点的CBMP信息。 [0032] In the present invention, each service node to the center edge constitute an independent network node, maintaining a distributed mapping table board (CBMP, Chessboard Map) backbone nodes across the network, by each service node maintained associated with their network of CBMP edge node; each node maintains CBMP user information with their neighboring nodes. 路由算法在服务节点计算插入、查找路径,查找资源分布信息,并把信息返回给客户。 Routing algorithm to calculate the insertion in the service node, find the path, the distribution of resources to find information, and send the information back to the client.

[0033] 如图3所示,示出了本发明的分布式文件存储系统的另一较佳的实施例。 [0033] Figure 3 shows another distributed file storage system of the present preferred embodiment of the invention. 为了满足边缘节点网内用户节点之间的通信需求,也作为文件发布的第一级节点(在后述图9中文件复制分布存储过程中详细说明),较佳的,本发明不仅骨干节点网采用环形网络拓扑结构,而且边缘节点网内的用户节点也同样采用环形的网络拓扑结构。 In order to meet the needs of the communication between the network user node of the edge node, as the first stage node also published document (described in detail later in FIG. 9 distribution storage file copy process), preferably, the present invention is not a backbone node network using a ring topology network, a user node and the edge node in the network also uses network topology in a ring. 例如,在本实施例中, 该服务节点A〜D构成了环形的骨干节点网,分别以服务节点A、B、C、D为中心,其所属的用户节点Al〜A3、Bl〜B3、Cl〜C3、Dl〜D3也构成了环形的边缘节点网。 For example, in the present embodiment, the service node A~D constitute an annular network backbone nodes, each service node A, B, C, D as the center, Al~A3 user node to which it belongs, Bl~B3, Cl ~C3, Dl~D3 also constitutes an annular edge node network. 这样,如需在本边缘节点网内的用户节点之间传输文件,则可以直接通过同一边缘节点网内用户节点Al〜A3之间的直接通信,保证了用户节点之间数据传输的速度。 Thus, the user node To transfer files between edge nodes within the network, can be directly through the direct communication between the edge nodes in the same network user node Al~A3, to ensure that the speed of data transmission between user nodes.

[0034] 并且,较佳的,本发明还设置了一汇集服务节点,与骨干节点网的服务节点连接, 用于检测服务节点的连接情况与用户统计,并不参与实际的传输服务。 [0034] Further, preferably, the present invention is also provided with a collection of service nodes, backbone nodes connected with the network service node for connection to the service node detected user statistics, is not involved in the actual transport service.

[0035] 下面结合图4,详细说明本发明的服务节点的结构及其实现的功能。 [0035] below with reference to FIG. 4, a detailed functional configuration of the service node of the present invention and its implementation.

[0036] 如图4所示,本发明的服务节点主要由棋盘映射服务节点模块构成,支撑服务节点与用户节点的存储与查找。 [0036] As shown, the service node of the present invention is composed mainly of board mapping module 4 serving node, serving support node and the user node storing lookup. 该棋盘映射服务节点模块主要由协议(Protocol)模块、服务节点存储(SCB,Server Chessboard)模块、用户节点存储(CCB,Client Chessboard)模块、 监听模块(Listen)和其他辅助功能模块构成。 The board mapping service node module consists protocol (Protocol) module, a service node storage (SCB, Server Chessboard) module, the user node storage (CCB, Client Chessboard) module, a monitoring module (Listen) and other auxiliary functional modules. 其中,该协议模块包含有用于实现数据质量保证管理的质量控制(Qos)模块,用于实现节点分布、查找的节点管理算法(DHash)模块, 用于实现节点失效的控制和管理的错误控制(Failure Control)模块;该服务节点存储模块用于存储所有在线的服务节点信息,即存储有服务单元映射表(SCB);该用户节点存储模块用于存储登陆到本服务节点的所有用户节点信息,即存储有用户单元映射表(CCB); 该监听(Listen)模块用于监听节点的请求,并保持与其他节点的信息交换;。 Wherein the module includes protocol for implementing data management of quality assurance quality control (Qos) module, for implementing distributed nodes, the algorithm to find the node manager (DHash) modules, error control node failure for the control and management of the implementation ( Failure Control) module; the service node in the storage module for storing all online information service node, i.e. the mapping table stores a service unit (SCB of); the user node storing means for storing the node information of all users to log in to this service node, i.e., the subscriber unit stores a mapping table (the CCB); the listener (the listen) monitor means for requesting node, and maintaining information exchange with other nodes;. 并且,本发明的服务节点还设置有多个辅助功能模块,例如,接口层anterface)模块,主要是针对第三方软件的支持;用户界面层(shell)模块,负责本地服务节点的配置与操作并提供与汇集服务节点的通信功能;配置管理(Conf)模块,用于配置系统参数;连接管理(Connection Manage)模块,用于本服务节点与其他节点的连接管理;系统心跳(HeartBeat)模块,用于保持连接;底层连接(Communication)模块,用于提供网络连接接口给上层应用;日志管理(Log)模块,用于本服务节点的日志管理;错误管理(ErrManage)模块,用于进行错误管理。 And the service node of the present invention is also provided with a plurality of auxiliary function module, for example, an interface layer anterface) module, the main support for third-party software; user interface layer (the shell) module responsible for the local service node configuration and operation together provided with a communication function serving node; configuration management (Conf) module, configured to configure system parameters; connection management (connection manage) means for connection management of the service node and other nodes; heartbeat system (heartBeat) module, with to remain connected; underlying connection (Communication) module, for providing a network connection interface to the upper application; log management (the log) module for log management according to the present serving node; error management (ErrManage) means for error management.

[0037] 本发明的服务节点,可实现如下功能:[0038] (1)服务节点注册:服务节点启动后,需要配置自己的网络域和至少一个邻居节点。 Service node [0037] of the present invention can achieve the following functions: Register [0038] (1) the service node: node starts the service, the need to configure their network domain and at least one neighbor node. 根据上述信息,服务节点发出上线通知,通告网络自己的状态和标识符,并同步得到自己网络域内的可用服务单元映射表(SCB)。 Issued based on the information, on-line notification service node, the network advertise their status and identifier, and synchronizing the mapping table to obtain the available services unit of its own network domain (SCB). 最后打开侦听进程,接受用户节点的连接请求。 Finally, open the listening process, accept the connection request of the user node.

[0039] 该服务单元映射表(SCB)的存储结构可如下表所示: [0039] The map service unit (SCB) storage structure may be as follows:

[0040]SCBID ADDRESS CLIENTNUM AVAILABLENUM AREAServerl 61. 49. 22. 74 100 80 AServer2 61. 172. 171. 33 500 350 B [0040] SCBID ADDRESS CLIENTNUM AVAILABLENUM AREAServerl 61. 49. 22. 74 100 80 AServer2 61. 172. 171. 33 500 350 B

[0041] 其中,该服务节点存储表中可包括:服务节点标识符(SCBID)、服务节点地址(ADDRESS)、用户节点数(CLIENTNUM)、可用节点数(AVAILABLENUM)、分布区域(AREA)等信息。 [0041] wherein the serving node memory table may include: serving node identifier (SCBID), serving node address (ADDRESS), the user nodes (clientNum), the available nodes (AVAILABLENUM), distribution area (the AREA) and other information . 其中,该服务节点标识符(SCBID)如图6A所示,可由一个32位的标识符确定,前1 6 位代表该服务节点的信息,后16位保留,以提供服务节点新功能的扩充。 Wherein the serving node identifier (SCBID) shown in Figure 6A, a 32-bit identifier can be determined, the former 16 bits represent information of the service node, 16 bits reserved to provide the functionality of the expansion of the new serving node. 当然,可以理解的是,该服务节点也可仅由前16位代表该服务节点的信息确定。 Of course, it is appreciated that the service node may be only the first 16 bits of information representative of the determined serving node.

[0042] (2)用户节点注册:用户节点启动后,如果是首次安装,需要连接到默认服务节点,注册自己的用户节点ID。 [0042] (2) user node registration: the user restarted, if it is first installed, you need to connect to the default service node, a node register their user ID. 服务节点根据用户节点提供的注册信息和源IP地址,生成唯一的用户标识符(32-bit)返回,并指定用户节点的首选服务节点。 The user registration information service node and the source node provides the IP address, generates a unique user identifier (32-bit) returns, and specify the preferred serving node user node.

[0043] 该服务节点内还存有一用户单元映射表(CCB),其存储结构如下表所示: [0043] service node within that there is also a subscriber unit mapping table (CCB), which storage structure as follows:

[0044]CCBID ADDRESS SHARE (M) AVOID PRI STAT AREAClientl 10. 8. 1. 3 10 5 1 0 AClient2 192. 168. 1. 200 20 10 0 1 AClient3 192. 168. 1. 188 50 50 3 1 BClient4 202. 88. 23. 152 40 5 2 0 CClient5 61. 89. 120. 48 20 15 1 1 C ...... ...... [0044] CCBID ADDRESS SHARE (M) AVOID PRI STAT AREAClientl 10. 8. 1. 3 10 5 1 0 AClient2 192. 168. 1. 200 20 10 0 1 AClient3 192. 168. 1. 188 50 50 3 1 BClient4 202 . 88. 23. 152 40 5 2 0 CClient5 61. 89. 120. 48 20 15 1 1 C ...... ......

[0045] 其中,该用户节点信息存储表(CHT)内包括:用户节点标识符(CCBID)、用户节点地址(ADDRESS)、共享空间(SHARE(M))、可用节点空间(AVAID)、优先级(PRI)、用户节点在线状态(STAT)、所属区域(AREA)等信息。 [0045] wherein, the user node information stored in the table (CHT) comprising: a user node identifier (CCBID), the user node address (ADDRESS), shared space (SHARE (M)), the space available node (AVAID), priority information (PRI), the user node online status (STAT), belongs to the region (the aREA) and the like.

[0046] 该用户节点标识符(CCBID)如图6B所示,可为一32位标识符,前16位代表所属的边缘节点网,即其中的服务节点标识符的前16位,后16位代表该用户节点自己的节点信息。 [0046] The user node identifier (CCBID) shown in Figure 6B, may be a 32-bit identifier, the edge node 16 representative of the net belongs, i.e., where the serving node identifier of 16 bits, and 16 bits on behalf of the user node's own node information. 这样,每个独立的边缘节点网可以容纳65535个用户节点,整个系统可以存在65536个独立的边缘节点网,可支持的用户数量为4. 2亿左右。 Thus, each individual edge node network user node can accommodate 65,535, the entire system can exist independently 65536 network edge node, the number of users that can be supported is approximately 420 million.

[0047] 对于用户节点,都维护O(IogN)个其他节点的信息;对于服务节点,维护该用户网内的所有节点信息。 [0047] For the user node, maintains information O (IogN) other nodes; for service node, maintains information about all nodes within the network user.

[0048] (3)用户节点登陆:用户节点提供已经注册的ID和口令到首选服务节点,如果首选节点不可用,则计算二级服务节点,并尝试登陆。 [0048] (3) login user node: providing registered ID and password to the user preferred service node Node, if the preferred node is not available, two service nodes are calculated, and attempt to land. 服务节点接收到登陆请求后,转发给协议模块处理,生成用户本次会话的ID,向用户发出状态请求,要求用户返回当前可用状态, 并更新用户的在线状态(STAT)。 Serving node after receiving the log-in request, forwarded to the protocol processing module generates a user ID of this session, status request sent to the user, requiring the user to return the current status available, and update the user's online status (STAT).

[0049] (4)用户查找请求:查找分两种情况:服务节点内查找和服务节点间查找。 [0049] (4) a user search request: Find two cases: intra-service nodes and service node searches to find. 服务节点内查找,主要针对用户节点的上传(upload)操作,如果在用户节点定义的数据安全级别以内,在服务节点上能满足用户节点的需求,则只执行本服务节点内的查找;只有不能满足用户节点的节点需求时,才发起节点间的查找操作:服务节点间查找,主要针对文件传输操作,涉及到源用户节点和目的用户节点的服务节点。 Within the service node searches, mainly for uploading user node (upload) operation, if the level of data security within the user-defined nodes, the nodes in the service to meet the needs of the user node, only the lookup is performed within the Service Node; not only when a node needs to meet the user node, the search operation was launched between nodes: room service node searches, mainly for file transfer operations, related to the service node of the source node and the destination user user node.

[0050] 在查找时,主要是通过文件标识符(FCBID)进行查找。 [0050] When searching, mainly through the file to find identifiers (FCBID). 其中,如图6C所示,该文件标识符(FCBID)主要包括源用户节点ID、目的用户节点ID、文件物理信息、文件校验值等信息。 Wherein, as shown in FIG. 6C, the file identifier (FCBID) including the user node information source ID, the destination user node ID, physical information file, the file and the like check value.

[0051] 并且,在服务节点中,还包括一个文件存储表,用于支持服务节点的查找和传输, 其存储结构如下表所示: [0051] Further, in the service node, further comprising storing a file table, for supporting the transmission and find the service node, which stores configuration shown in the following table:

[0052] [0052]

Figure CN101188569BD00121

[0053] 其中,该表中可包括:文件标识符(FCBID)、文件作者(OWNER)、文件总长度(TOTALSIZE(K))、文件块数(BLOCKS)、优先级(PRI)、可用节点数(PEERUSED)、下一跳地址(ΝΕΧΤΗ0Ρ)等信息。 [0053] wherein, the table may include: a file identifier (FCBID), author of the file (OWNER), the total length of the file (TOTALSIZE (K)), file block number (the BLOCKS), priority (the PRI), number of nodes available (PEERUSED), the next hop address (ΝΕΧΤΗ0Ρ) and other information.

[0054] (5)服务节点失效:服务节点的非正常失效,对于正在传输的用户节点不会有明显影响,但把它设为首选服务器的用户节点,会登陆缓慢;所以通过服务节点间心跳机制来保证失效节点的及早通告。 [0054] (5) Service node failure: the failure of non-normal service node, there will be no significant impact on the user node is transmitted, but set it to the user's preferred server node, will be landing slowly; so through the inter-service node heartbeat mechanisms to ensure early announcement of the failed node.

[0055] (6)用户节点失效:用户节点非正常失效,包括用户网络故障(可恢复)和用户节点信息故障(不可恢复)两种情况。 [0055] (6) User node failure: Unusual Failure user node, including the user network failure (recoverable) and failure information of the user node (unrecoverable) in both cases. 对于恢复情况,标识用户节点状态为不可用;对于不可恢复情况,通知所有存储的信息源节点,做重新数据分布。 For recovery, the user identifies the node status is unavailable; not for recovery, to notify all the information stored in the source node, re-do data distribution.

[0056] (7)连接保持:分别通过服务节点间和服务节点与用户节点的心跳保持。 [0056] (7) holding: are held by the serving node and the serving node between the user node and the heartbeat.

[0057] 在本发明的服务单元(服务节点)中,存在以下数据结构:通用单元信息描述符NUID(Normal UnitInformation Description):用于存储单个节点的存在信息,包括时间、 区域等统计加权值;数据交换单元DEU(Data ExchangeUnit):用于与其他服务单元交换信息库,提交查找、存储返加信息等;本地节点信息库LPIB (Local PeerInformation Base): 用于存储本地子网中的所有NUID,提供一套存取接口。 [0057] In the service unit (serving node) of the present invention, the following data structure: the common unit information descriptor NUID (Normal UnitInformation Description): storing presence information for a single node, including statistical weight value of the time, area and the like; data exchange unit DEU (data ExchangeUnit): used to exchange information with other service units library, submit to find, store a returning information; local node repository LPIB (local PeerInformation Base): used for storing all NUID local subnet, providing a set of access interface.

[0058] 其中,服务单元的数据流程包括: [0058] wherein, the service data flow unit comprises:

[0059] 服务单元上线,初始化LPIB (加载已有LPIB或初始化一个空的LPIB); [0059] On-line service unit, initialize LPIB (or initializing a load prior LPIB empty LPIB);

[0060] 发起与其他服务单元的同步操作,由DEU交换得到最新的LPIB ; [0060] initiate synchronization operations with other service units, obtained by the latest LPIB DEU exchange;

[0061] 当有本地用户单元发出传输请求时,从LPIB得到一个NUID集合,并返回给用户单元; [0061] When a transmission request issued by the local user unit, derived from a NUID LPIB collection unit and returned to the user;

[0062] 当本地有新的数据单元上线时,根据系统调度,主动发起复制进程; [0062] When a new line with a local data unit, according to the scheduling system, initiate the copying process;

[0063] 当请求发生后(主动和被动),动态监听反馈信息,更新LPIB。 [0063] When the request occurs (active and passive), the dynamic feedback monitor, update LPIB.

[0064] 下面结合图5,详细说明本发明的用户节点的组成结构及其实现的功能。 [0064] below with reference to FIG. 5, described in detail the structure of functional components of the user node and achieved the present invention.

[0065] 如图5所示,在本发明中,该用户节点包括:用户登陆模块,用于用户登陆操作;用户注册模块,用于用户注册操作;文件发布模块,用于发布共享文件操作;本地文件管理模块,用于数据文件的上传/下载,以及文件块的验证;共享空间管理模块,用于统一管理用户节点共享给系统的磁盘空间,并动态分配共享空间的使用;传输模块,用于提供接口给上层结构,并通知数据文件的接收与发送;监听模块,用于监听服务节点或其他节点的连接请求;配置管理模块,用于本地系统参数的配置;日志管理模块,用于本地的日志管理;底层连接模块,用于提供网络连接接口给上层应用。 [0065] FIG. 5, in the present invention, the user node comprising: a user login module, a user login operation; user registration module configured to register a user operation; document publishing means for publishing documents sharing operation; local file management module, upload / download data for a file, and a verification file blocks; shared space management module, for the unified management of the system to the user node shared disk space, and the use of dynamic allocation of shared space; a transmission module, with to provide an interface to the upper structure, and notifies the receiving and sending data file; the monitoring module, listening for connection requests service node or other nodes; configuration management module configured to local system parameter; log management module for local log management; the underlying connection module, for providing a network connection interface to an upper layer application.

[0066] 在本发明中,由于服务器层的服务节点基本都是运行于同一个操作系统中,如LINUX操作系统,而用户层的用户节点中,有可能运行于不同的操作平台中,为了实现跨平台的操作,本发明较佳的在用户节点中设置了一个系统支持层模块,用于为用户节点运行于不同的操作平台提供支持。 [0066] In the present invention, since the service server node layer are basically running on the same operating system, such as LINUX operating system, the user and the user node layer, it is possible to run on different platforms in order to achieve cross-platform of the present invention preferably is provided in a system supporting user node layer module, for providing support for the user nodes run on different platforms. 另外,为了方便用户节点对第三方软件的支持,本发明的用户节点中还设置了一个接口层模块,用于为第三方软件的支持提供调用接口。 Further, the user node in order to facilitate support for third party software, the user node of the present invention is also provided an interface layer module, for providing a call interface to support third-party software.

[0067] 在用户节点,可实现以下功能: [0067] In the user node may implement the following functions:

[0068] (1)文件发布:制作发布索引文件,用于文件的上传、传输等操作。 [0068] (1) file distribution: production release of the index file for file uploading and transmission operations.

[0069] 该发布索引文件可至少包含有文件标识信息、文件分块信息。 [0069] The release index file may contain at least the document identification information, the file block information. 在本发明中,每个共享或上传的文件,由一个U8-bitS的文件标识符唯一确定,如图6C所示,该文件标识信息分别包括32位的源用户节点ID信息、目的用户节点ID信息、文件物理信息和文件校验值。 In the present invention, each shared or uploaded file by a file identifier U8-bitS uniquely determined, as shown in FIG. 6C, the file identification information of each user comprises a 32-bit source node ID, destination node ID of the user information, physical information file and file checksum values. 该文件标识符由源用户节点生成,随文件发布到存储节点中,由存储节点确认后,在服务节点中存储于文件存储表中,可供摘索。 The file identifier is generated by the source node user, with the storage node to the publish documents, after confirmation by the storage node, the serving node is stored in the file storage table, for picking the cable. 文件标识的分布算法与节点标识的分布算法相同,每个单独的文件块,都存储到与其标识相近的节点中;如果该节点状态为不可用,则查找它的后续节点存储。 Distribution Algorithm distribution algorithm and the same node identifier identifying the file, a separate file for each block, are stored close to its node identifier; if the node status is unavailable, nodes find its subsequent storage.

[0070] 其中,目的节点存在三个系统内的定义:1)ALL,表示所有节点都有使用权;2) NONE,表示只有源用户节点自己有使用权;3) AUTH,只有授权用户节点才能使用权。 [0070] wherein, in the presence of the three node definition system: 1) ALL, showing the right to use all nodes; 2) NONE, showing only the source user node that they have the right to use; 3) AUTH, only authorized users can node use rights. 系统默认的目的节点ID信息为ALL,即所有节点都可以使用该目的节点。 Default destination node ID information ALL, i.e., all nodes can use the destination node.

[0071] (2)文件操作(查找,下载,上传): [0071] (2) file operations (search, download, upload):

[0072] 共享节点的查找,用户节点只需向服务节点发出请求。 [0072] shared node search, the user node by issuing a request to the service node. 查找时只需要提供需查找的文件名或FCBID,其中,文件名查找,需要服务节点的检索,FCBID可以直接从服务节点定位文件。 Only need to provide the time needed to find to find the file name or FCBID, wherein the file name lookup, you need to retrieve a service node, FCBID can locate files directly from the service node.

[0073] 下载共享文件,首先得到索引文件,然后确定连接节点,交换文件状态,由传输模块负责文件的传输完成。 [0073] download the shared file, the index file is first obtained, and then determines that the connection nodes, the switching state of the document, performed by the transmission module is responsible for file transfer.

[0074] 上传文件,节点的分布,由服务节点按分布算法确定并返回,客户端根据节点信息分布文件到相关节点。 [0074] upload files, distributed nodes, determined by the service node distribution algorithm and press return, the client node file according to information distributed to the relevant node.

[0075] (3)共享使用流程: [0075] (3) shared process:

[0076] 一、用户节点间操作:数据传输,源用户节点向服务节点提供的所有目标用户节点发出连接请求;如果目标用户节点可用,则建立连接,开始传输相关数据块;如果目标用户节点不可用,则源用户节点向服务节点发出新的节点请求,根据返回节点传输其他数据。 [0076] First, the inter-user node operation: data transmission, all of the target user of the source user node provides the service node sends a connection request node; if the target user node is available, a connection is established, starts to transmit the relevant data block; if the target user node is not used, then the source node sends the user a new node requests service node, according to other data transmission return node. 在本发明中,这两项操作是同时进行的,并非一定要所有节点都可用才开始传输,而是有可用节点就传一部分,同时对于不可用节点,请求分配新的可用节点。 In the present invention, these two operations are carried out simultaneously, it is not necessarily available to all nodes start transmitting, but have to pass a portion of the available nodes, while for the unavailable node, a request to allocate a new node is available.

[0077] 二、用户节点与服务节点间操作:通告服务节点自己数据块的状态,以及共享空间的使用情况。 [0077] Second, between the user node and the service node Action: Notification Service node's own status data block, and the use of the shared space.

[0078] 三、用户节点主动操作:清除过期数据,向源用户节点发出数据过期询问,如果返回确认过期,则清除数据,并向服务器更新可用空间;如果源用户节点需要继续保留数据,则由源用户节点发起一个数据重新分布的过程。 [0078] Third, the user node active: Clear the obsolete data sent to the source node user data expires asked if the return confirmation of date, clear data, and to update the available space in the server; if the source node needs to retain user data, by source user node initiates a process of redistribution of the data.

[0079] 较佳的,为了保证数据的完整性,在本发明中,可通过如下三种方法实现: [0079] Preferably, in order to ensure data integrity, in the present invention may be implemented by the following three methods:

[0080] 一、节点冗余:保证数据文件分块的长度(Size)在一个合理的区间,节点冗余度在一个可靠范围。 [0080] First, the node redundancy: ensure that the length (Size) block of the data file in a reasonable range, the node redundancy in a reliable range.

[0081] 二、该用户层中一个或多个用户节点可为虚拟的超级节点,用于在一预定时间内保留特定用户数据的全部镜像信息,针对用户节点的不同级别和请求类型,选择性的使用这些虚拟的超级节点进行数据的传输。 [0081] Second, the one or more user-level nodes may be a virtual user super node for a specific user all image information retaining data within a predetermined time, different levels and types of requests for a user node, selectively the transmission uses these virtual supercomputer node data.

[0082] 三、棋盘映射策略,由服务单元自动完成,在保证数据完整性的同时提高磁盘利用率。 [0082] Third, the board mapping strategy, completed by the service unit automatically, ensuring data integrity while improving disk utilization.

[0083] 较佳的,为了保证数据传输的安全性,本发明还进行了数据的加密和用户口令保护操作。 [0083] Preferably, in order to guarantee the security of data transmission, the present invention also encrypted passwords and user data protection operation. 该数据加密操作是指用户可以将分布数据块以指定加密算法加密,可由用户在数据安全与上传速度间权衡后决定,该用户口令保护操作是将数据文件以用户私有口令加密。 The data encryption operation that the users can be distributed to the specified data block encryption algorithm, the user may be determined after weighing between upload speed and data security, the user password protection operation is a data file to the user's private encryption password.

[0084] 同样的,在用户节点中也存在如下存储结构,用于支持服务节点与用户节点之间的连接、文件的操作等: [0084] Similarly, there is also stored in the structure of the user node, the connection between the service node and the user node to support file operations like:

[0085] (1)共享空间使用信息表,其存储结构如下表所示: [0085] (1) using the shared space information table, which stores configuration shown in the following table:

[0086] [0086]

Figure CN101188569BD00141

[0087] 其中,该表中可包括:文件标识符(FCBID)、共享文件源(FROM)、共享文件块数(BLOCKS)、长度(SIZE (K))、开始位置(STARTP0S)、结束位值(ENDPOS)、可用时间(AVAILABLETIME)。 [0087] wherein, the table may include: a file identifier (FCBID), file sharing source (the FROM), shared file block number (the BLOCKS), the length, the end of the bit value, the start position (STARTP0S) ((K) SIZE) (eNDPOS), available time (AVAILABLETIME).

[0088] (2)网络连接信息表,其存储结构如下表所示: [0088] (2) the network connection information table, which stores configuration shown in the following table:

[0089] [0089]

Figure CN101188569BD00142

[0090] 其中,该表中可包括:网络名(ID)、网络地址(ADDRESS)、网络类型(TYPE)、网络分区(AREA)、升级时间(UPTIME)、操作类型(OPTYPE)等信息。 [0090] wherein, the table may include: a network name (ID), a network address (ADDRESS), the network type (the TYPE), network partition (the AREA), the upgrade time (UPTIME), type of operation (OPTYPE) and other information.

[0091] 由上述可知,整个骨干节点网维护一个分布式的棋盘映射(CBMP)表,该表由每个服务节点所维护的、与自己边缘节点网相关的映射表组成,该棋盘映射表包括用户单元映射表(CCB)、服务单元映射表(SCB)、文件存储表(FCB)等,分别存储有用户层信息、服务器层信息、及文件信息,用于支持整个网络存储系统的连接。 [0091] From the foregoing, the entire backbone network node maintains a distributed mapping board (CBMP) a table maintained by each service node, the edge-node associated with their mapping tables, the board mapping table comprising subscriber unit mapping table (the CCB), the service unit mapping table (SCB of), the file storage table (the FCB), etc., are stored user information layer, the server layer information, and file information, for supporting the whole network storage system is connected.

[0092] 本发明还提供了用户信息查询,提供给第三方或汇集服务器使用,辅助完成用户数据的统计、用户计费管理等操作。 [0092] The present invention further provides a user information query, or to a third party server uses pooled, complete user assistance data statistics, user billing management operations. 其基本结构由LDAP(LightweightDirectory Access Protocol)实现,分布于相关的服务单元。 Its basic structure is implemented by LDAP (LightweightDirectory Access Protocol), located in the associated service unit.

[0093] 并且,在本发明中,系统共涉及以下几种文件格式:配置文件、摘要文件、共享库文件、摘要库文件、信息库文件。 Profile, summary files, shared libraries, library file summary information library: [0093] In the present invention, the following common system relates to file formats. 下面分别叙述: The following are described:

[0094] ①配置文件:由XML格式存储,分别用于用户单元和服务单元,提供基本的节点信息; [0094] ① profile: storing the XML format, for each subscriber unit and the service unit to provide basic node information;

[0095] ②摘要文件:由XML格式存储,并做对称加密处理,用于描述存储文件的摘要信息和分布信息,由用户单元生成,同时存储于用户单元和服务单元,可以独立发送,是文件下载和文件发送的首要条件; [0095] ② summary file: XML format by the storage, processing and make symmetric encryption, digest information and for storing distribution information description file, is generated by the subscriber unit, the subscriber unit and simultaneously stored in the service unit can be transmitted independently a file download files sent and the first condition;

[0096] ③共享库文件:由二进制格式存储,生成于用户单元,用来存储共享空间数据;对于用户单元,此文件是不可访问内容,只有用户单元软件才能在需要的时候,访问基本的相关内容。 [0096] ③ shared library files: a binary format, generating a user unit for storing shared data space; user unit, the content file is inaccessible, the subscriber unit only when necessary software to access related basic content. 文件内的索引块,只能由服务单元指令访问,用来定位和查找数据单元。 Index block within the file, can only be accessed by the service instruction unit, to locate and find the data unit.

[0097] ④摘要库文件:以XML文件存储,生成于服务单元,只能由服务单元访问,用来向用户单元提供数据文件摘要文件。 [0097] ④ Summary library files: The XML files are stored, generated in the service unit, can only be accessed by the service unit to provide data file summary file to the user unit.

[0098] ⑤信息库文件:以LDAP目录存储,生成于所有相关的服务单元,存储用户单元和服务单元的信息,可以由授权用户通过管理接口访问。 [0098] ⑤ file repositories: LDAP directory to store all relevant information generated in the service unit, storage unit and user services unit by an authorized user can access the management interface through.

[0099] 在本发明中,为了进一步保证数据传输的速率,较佳的采用了下述路由方法来进行路由的选择: [0099] In the present invention, in order to further ensure the rate of data transmission, preferably using the following routing method to route selection:

[0100] (1)采用分布区域:用户节点连接服务节点,以不跨服务商、地域的节点为优先。 [0100] (1) The distribution area: user node connected to the service node, so as not to cross the service providers, local node priority.

[0101] 针对国内互联网的实际状况,在跨运营商的网络间传输速度会有较大损失,所以服务节点原则上只接受同一运营商用户的连接,以保证用户节点间数据传输的速度;只有当服务节点失效,没有其他服务节点可选的情况,才考虑用户节点的跨运营商连接。 [0101] For the actual situation of the domestic Internet, among inter-carrier network transmission speed will be a great loss, so operators only accept the same user connected to the principle of serving node, in order to ensure the speed of data transmission between the user node; only when the case of the service node fails, there is no other alternative service node, the user node before considering the inter-operator connection.

[0102] 基于这个因素,服务节点或用户节点在计算自己的SCBID或CCBID时,会首先考虑一个“IP-运营商”的对照关系表。 [0102] Based on this factor, the service node or a user node when calculating their SCBID or CCBID, will first consider a "IP- operators' control table. 该“IP-运营商”对照关系表,由一台服务器完成,定期采集域名分配组织(ICANN)公布的运营商数据,整理后发布给其他服务节点。 The "IP- operators' control table, done by a server, regularly collecting organization for Assigned Names (ICANN) released data carriers, issued to other service nodes finishing.

[0103] 该IP-ISP对应表如下表所示: [0103] The IP-ISP correspondence table shown in the following table:

[0104] [0104]

Figure CN101188569BD00151

[0105] 其中,用户节点在注册、登陆时,服务节点根据上述关系表,决定允许用户登陆,或把用户的登陆请求转发到更适合匹配关系的服务节点。 [0105] wherein, the user node registration, landing, service node according to the above table, the decision to allow the user login, the user or the login request is forwarded to the serving node is more suitable for matching relationship.

[0106] 由于本发明在同一服务节点内,组成一个边缘节点网,网内的用户节点之间可以直接通信。 [0106] Since the present invention is within the same server node, composed of a network edge node, the user can communicate directly between nodes within the network. 因此,对于不同的数据安全级别,在跨网选择上,提供不同的策略。 Therefore, for different levels of data security in the inter-network options, provides a different strategy. 对于数据安全级别要求较低的同一子网内的文件传输,可直接在两个用户节点之间通信,进行数据的直接传输,从而大大增加了数据的传输速率。 For a lower level of data security requirements file transfer within the same subnet, the communication may be directly between the two user nodes, data is directly transmitted, thereby greatly increasing the data transmission rate.

[0107] (2)采用优先级:服务器层统计、计算每个用户节点的效率和可用度,排列用户节点在系统中的优先级。 [0107] (2) The priority: Statistics Server layer, each computing node efficiency and user availability, user nodes are arranged in a priority system. 对于在线时间长、共享空间足够、可用度和可靠度高的用户节点,设置其优先级为较高的优先级,这样既可保证共享文件的分布更广,其他用户节点可随时通过这些共享节点来下载需要的文件。 For a long time line, shared space sufficiently, high reliability and availability of the user node, set the priority of a higher priority, so can ensure a wider distribution of the shared file, the other nodes at any time by the user shared node to download the required files.

[0108] (3)节点选择:在用户节点请求存储空间时,服务节点选择不同的匹配法(最佳匹配,最差匹配,首次匹配)。 [0108] (3) node selection: when the user node requests memory space, the service node to select a different matching (the best match, worst match, the first match).

[0109] 最佳匹配:查找所有节点提供的可用空间块,和需要存储的文件块比较,相差最小的块,认为是最佳匹配结果。 [0109] Best match: Find all available space node provides blocks, and comparison file blocks to be stored, the smallest difference block that is the best match. 最佳匹配的目的是保证有足够大的空间块提供给其他用户使用。 The best-fit is to ensure a large enough space available to block other users.

[0110] 最差匹配:查找所有节点提供的可用空间块,把最大的块提供给文件块使用,认为是最差匹配结果。 [0110] Worst match: find free space and block all the nodes provided the biggest block to block file use, considered to be the worst matches. 最差匹配的目的是保证剩余的空间块都足够大,不至于失去存储价值。 The purpose is to ensure that the worst match of the remaining chunks of space is large enough, and will not lose stored value.

[0111] 首次匹配:从可用的空间列表中,选取首先查找到的可用块给用户,认为是首次匹配。 [0111] First match: From the list of available space, select the first available block to find the user, believed to be the first match. 首次匹配的目的是保证对用户请求的响应时间最短。 The purpose is to ensure that the first match the response time to user requests a minimum.

[0112] 系统节点选择的默认匹配方式是:首次匹配。 [0112] The default mode matching system node is selected: the first match.

[0113] 下面将详细描述本发明的服务节点的路由方法。 [0113] service node routing method of the present invention will be described in detail below.

[0114] 本发明的服务器层,在一个服务节点加入或离开网络时,都必须发送一个加入或离开的请求消息。 [0114] server layer of the present invention, when a join or leave the network service node, must send a request to join or leave a message. 对于每一个用户节点,在加入或离开网络时,必须发送一个加入或离开的请求消息到指定服务器。 For each user node, when joining or leaving the network must send a request to join or leave a message to the specified server. 对于非正常离开网络的情况,由错误处理模块自动完成网络状态的同步工作。 In the case of non-normal to leave the network, error handling module automatically by the network synchronization status.

[0115] 对于服务节点和用户点信息的存储,采用了“棋盘映射(CBMP) ”结构,以优化的负载均衡处理,减弱了网络”热点“效应,提高了服务节点的可靠性。 [0115] service node for storing user and point information, using a "checkerboard map (CBMP)" structure, to optimize the load balancing process, reduce network "hot spot" effect, improving the reliability of the service node.

[0116] 本发明的路由算法——CBMP算法的基本结构是:该算法给每个节点,指定一个m 位大小的标识符,这个标识符由基础的Hash算式给出,如SHA-I。 [0116] The basic structure of the routing algorithm --CBMP algorithm of the present invention is: The algorithm for each node, specify a size of m-bit identifier that is given by the equation Hash-based, such as SHA-I. 对于服务节点,标识符由其IP地址产生;对于用户节点,标识符在用户登陆时随机产生,在用户离开时销毁。 For service node identifier generated by its IP address; the user node identifier is randomly generated when the user logs destroyed when the user leaves. 标识符的长度,需要足够长度,以保证不会对不同节点产生相同的标识符。 Length of the identifier requires a sufficient length to ensure that the same will not produce a different node identifier. MEDDQSN系统中的用户标识符选为32位,服务节点标识符选为16位。 MEDDQSN system user identifier preferably 32, preferably 16 serving node identifier.

[0117] 如图7所示,本发明的棋盘映射表(CBMP)主要分为两个层的存储结构:服务节点存储表(SCB),用户节点存储表(CCB)。 [0117] As shown in FIG 7, the board mapping table (CBMP) of the present invention, the storage structure is divided into two layers: serving node storage table (SCB of), the user node storage table (CCB). 这两个存储表结构分别用于对服务节点(ServiceNode)和用户节点(ClientNode)的管理,每个服务节点记录所有在线的服务节点信息,并记录登陆到本节点的所有用户节点信息;每个用户节点只维护自己和与自己相关的节点的状态信息,对于其他节点信息,需向服务器发出请求后得到。 Respectively, these two memory table structure for managing service node (ServiceNode) and user nodes (ClientNode), each serving node records all online information service node, and the nodes record all users logged on to the local node; each users node only maintain state information relevant to themselves and their nodes to obtain information for the other nodes, the need to make a request to the server.

[0118] 本发明的路由算法可实现下述功能: [0118] routing algorithm according to the present invention can achieve the following functions:

[0119] 对于服务节点来说,可实现: [0119] For service node, it can be achieved:

[0120] (1)服务节点的加入(join):计算本节点标识符_>指定网络域和邻居节点_>上线通知_>同步SHT ; Online notification on a computing node identifier _> specify the network domain and neighbors _> _> synchronization SHT;: [0120] was added (1) a service node (join)

[0121] (2)服务节点的离开(leave):通知离线_>检查登陆用户_>通知用户疏散_>离线.一入, [0121] (2) away from the service node (leave): offline notification _> _ checking login> inform the user of the evacuation _> into a offline.

[0122] (3)服务节点的查找(find):定位服务节点_>发送查找通知; [0122] Find (3) service node (find): location services node _> Find transmitting notification;

[0123] (4)服务节点的监听(listen):接收其他服务节点的请求。 [0123] (4) monitoring services node (listen): receiving a service request from another node.

[0124] 对于用户节点来说,可实现: [0124] For the user node, it can be achieved:

[0125] (1)用户节点的加入(join):验证登陆_>通知上线_>标记状态为可用; [0125] (1) addition of user nodes (join): authentication log-_> _ line notification> labeled as available status;

[0126] (2)用户节点的离开(leave):通知离线_>中止共享进程_>标记状态为不可用; [0126] (2) leaves the user node (leave): offline notification _> _ sharing process abort> marked as unavailable state;

[0127] (3)用户节点的查找(find):登陆服务节点内查找_>如不满意,发出查找通知到其他服务节点_>更新可用状态;16[0128] (4)用户节点的监听(listen):接收服务节点其他用户节点的查找请求。 [0127] Finding (3) user node (find): Find a service node in the landing _> If not satisfied, issued a notice to find another service node _> update available; 16 [0128] (4) monitor user node ( listen): Find other user node node receives the service request.

[0129] 下面将结合图8〜图11详细说明本发明系统的数据文件的分布存储、传输方法。 [0129] below with reference to the distributed file system for storing data of the present invention, a transmission method described in detail in FIG. 11 8~ FIG.

[0130] 在本发明中,基于本发明系统的数据文件的分布存储方法包括: [0130] In the present invention, the distribution data file storage method of the present invention is based on the system comprising:

[0131] 用户节点利用服务节点选择将数据文件上传发布到网络中可用的用户节点上,形成分布节点; [0131] user node using the selected service node to upload the data file posted to the user nodes in the network is available to form a distribution node;

[0132] 在用户节点将数据上传后或在数据文件的传输过程中,服务节点自动调度整个骨干节点网络内的可用节点,分配承载任务,把用户节点上传分布到网络中的数据文件通过复制,分发存储到多个用户节点的共享空间上,形成多个共享节点,供用户节点下载或传输文件用,从而在网络上构建形成数据存储和传输的量子空间。 [0132] user node after upload data or during transmission of the data file, the service node automatically schedule available nodes throughout the backbone nodes of the network, allocating bearer task, the user node uploads distributed to the data file in the network by copying, distribution storage space shared to a plurality of user nodes, a plurality of shared node, a node for the user to download or transfer files to construct a quantum space formed of data storage and transmission over a network.

[0133] 并且,在分发存储的同时,保证一定量的数据冗余,以在一定范围的网络内,达到文件的均勻分布。 [0133] Further, simultaneous distribution of stored data to ensure a certain amount of redundancy to the network within a certain range, to achieve uniform distribution of the document. 并且,在完成上述分布存储过程后,用户节点还可利用包含有需要下载的数据文件的共享节点来完成所需数据文件的下载或传输。 And, after completion of the above-described procedure stored profile, the user node may also be required to complete the data file downloaded or transferred using the shared node with the data file to be downloaded.

[0134] 如图8所示,示出了用户节点向网络进行文件发布的工作流程,主要包括: [0134] As shown in FIG 8, shows the user node to the workflow file distribution network, including:

[0135] (1)、已注册并登陆用户节点1制作发布索引文件,并发送包含有该索引文件信息的上传请求到登陆服务节点1,其中,该索引文件至少包含有文件标识信息、文件分块信息, 该文件标识信息包括源用户节点ID信息、目的用户节点ID信息、文件物理信息和文件校验值。 [0135] (1), has been registered and logged in user node 1 production release of the index file, and sends contains the index file upload request information to the login service node 1, wherein the index file contains at least document identification information, file points block information, the file identification information of the user includes a source node ID, destination node ID, user information, file information, and the physical file checksum values.

[0136] (2)、服务节点1根据索引文件信息,查找可用节点状态;如果当前服务节点1的子网内节点不足以满足用户节点1的文件上传需求,则该服务节点1通知其他服务节点2需调用其子网内的用户节点2的共享空间,以得到足够的共享空间,提供给用户节点1使用; 如果当前服务节点1的子网内节点足以满足用户节点1的文件上传需求,则不需要进行服务节点1与服务节点2之间的查找;如果是文件传输,则需要定位目的节点的注册服务节点2,并通知该服务节点2; [0136] (2), the service node 1 according to the index file information, find an available node status; If the current node within the subnet is not sufficient to meet the service node 1 1 user node file upload needs, the service node 1 Notify other service node 2 to be called shared space user nodes within its subnetwork 2, in order to obtain enough shared space available to the user node 1 uses; subnets within the service node if the current node 1. node 1 is sufficient to meet the user's file upload needs, the You do not need to find between 1 and 2 service node serving node; if the file transfer, you need to locate a registered service node destination node 2, and notifies the service node 2;

[0137] (3)、服务节点1返回可用节点信息给用户节点1 ;接收到节点请求的其他服务节点2,也返回可用节点信息给用户节点1,并通告本边缘节点网内调用的用户节点2 ; [0137] (3), serving node 1 returns available node information to the user node 1; receives another service node requests 2, also returns available node information to the user node 1, and to inform the user node calls within this edge-node 2 ;

[0138] (4)、用户节点1根据返回的节点列表,发出连接请求; [0138] (4), user node list returned node 1, connection request is issued;

[0139] (5)、用户节点2的守护进程,接收到用户节点1的连接请求,返回当前状态及可用信息;如果共享空间已经不可用,则返回拒绝信息,并通知登陆服务节点2更改自己的状态; [0139] (5), the user node daemon 2, receives the user node a connection request 1 returns the current state and the available information; if the shared space is not available, it returns an information, and notifies the log in the service node 2 to change your status;

[0140] (6)用户节点1接收到节点的返回信息,确定是否开始传输;如果没有得到足够的节点响应,则重新向服务节点1发送新的请求,得到更多节点列表; [0140] (6) User node returns a node receiving the information, determines whether to start transmission; if not enough nodes in response, re-send a new request to a service node, a list of nodes get more;

[0141] (7)、用户节点间确定连接后,准备需要传输的文件块和相应的校验值,提交到传输管理模块开始传输文件块; After [0141] (7), the node determines the connection between users, ready to be transmitted and the file blocks corresponding check value, submitted to the transmission management module starts transmitting the file block;

[0142] (8)、文件块传输完成后,用户节点2注册文件状态给调用服务节点krverl。 [0142] (8), the file block transfer is complete, the user node status 2 register file to the calling serving node krverl.

[0143] 本实施例仅以用户节点1发布文件到用户节点2为例进行说明,但可以理解的是, 源用户节点在发布文件的时候,可以同时连接到网络中的多个用户节点上,将文件发布该多个用户节点上,这些用户节点即为分布节点。 [0143] The present embodiment only a user node publish files to the user node 2 as an example, it will be appreciated that the source user node when the release of the document, the user may be connected to a plurality of nodes in the network at the same time, the documents posted on the plurality of user nodes, the user node is the distribution node.

[0144] 如图9所示,结合本发明的分布式文件存储系统,以一个用户节点上传文件到网络并在网络节点中分布存储数据的过程为例,进一步详细说明本发明的存储系统的数据文件的上传发布、复制并分布存储的方法,其包括: [0144] As shown, the distributed file storage system in conjunction with the present invention, a user to upload files to the network node and distributed in a network node of an example process of storing data, the storage system of the present invention is described in further detail data 9 upload published files, copied and distributed storage method, comprising:

[0145] (1)用户节点4制作发布索引文件,并向服务节点D发出包含此索引文件的上传请求。 [0145] (1) user node 4 production release of the index file, and sends this index file upload request includes a service node D. 其中,该索引文件包含了文件的源用户节点ID、文件ID、文件大小、文件名、文件块大小等基本文件信息,以及其他一些关于文件分块信息、文件校验值等附件信息。 Wherein the index file includes file source node user ID, basic file ID, file size, file name, file block size and other information, and other information about the file attachment block partition information, file checksum values, etc. 通过该索引文件,即可快速查找、获得可用节点信息。 By the index file, you can quickly search, obtain information available nodes.

[0146] (2)服务节点D接到请求后,在服务节点D的文件存储表中记录此索引文件信息, 并从用户节点存储表中查找可用节点,如选择用户节点2、3为可用节点; [0146] (2) Service node D, upon request, the index file recorded in the information service node file storage table D, the user node and search the available node memory table, such as selecting an available node user nodes 2 ;

[0147] (3)服务节点D返回可用的节点信息给用户节点4 ;同时通知被选择的用户节点2 和用户节点3,通知征用的它们的共享空间,并使它们分别与用户节点4建立连接; [0147] (3) Available service node D returns the information to the user node node 4; while notifying the user selected node 2 and node 3 users, notification thereof requisition shared space, and which are connected with the user node establishing 4 ;

[0148] (4)用户节点4分别上传文件块1到用户节点3,上传文件块2到用户节点5 ; [0148] (4) 4 each user node to upload files to the user node 3 a block 1, block 2 upload files to the user node 5;

[0149] (5)文件上传完毕,用户节点3和用户节点5通知服务节点D和用户节点4完成状态(至此完成了文件的上传发布,其中用户节点3和用户节点5为上传发布过程的分布节点); [0149] (5) file is uploaded, the user node 3 and user node 5 Notification Service node D and user node 4 completion status (thus completing the upload published files, which the user node 3 and user node 5 is distributed to upload the publishing process node);

[0150] (6)服务节点D计算索引文件,得到文件分布级别(可根据索引文件中包含的文件标识符获取文件的优先级),并向其他服务节点A、B、C发出同步请求; [0150] (6) calculating an index file serving node D, to obtain the degree of distribution files (which can be an identifier acquisition priority file according to the file included in the index file), and issues a synchronization request other services nodes A, B, C;

[0151] (7)服务节点A、B、C接收到请求后,查找各自的网络域内的可用用户节点,如确定用户节点1、7、8、9可用,则通知用户节点1、7、8、9同步请求,空间征用大小;同时通知用户节点3、5需要向用户节点1、7、8、9复制文件块1、2,如将文件块1复制到用户节点7、8,将文件块2复制到用户节点1、9; After [0151] (7) serving Node A, B, C receiving the request, the respective network domain to find the available user node, such as node 1,7,8,9 determined that the user is available, the user is informed nodes 1,7,8 , 9 synchronization request, the size of the space requisition; while notifying the user node needs 3,5 1,2, as will be copied to the file block 1 1,7,8,9 user node to user node to copy the file blocks 7-8, the file block 2 1,9 copied to the user node;

[0152] (8)数据复制完毕后,用户节点1、7、8、9分别通知各自的边缘节点网内的服务节点A、B、C ;用户节点3、5通知服务节点D ; After replication [0152] (8) the data is completed, the user notifies the service node 1,7,8,9 node A in the respective edge-node, B, C; 3,5 notification user node serving node D;

[0153] (9)服务节点A、B、C通知服务节点D,文件分布存储完成,其中该用户节点1、7、8、 9为文件存储过程的共享节点,供其他用户节点文件下载/传输用。 [0153] (9) serving Node A, B, C notification service nodes D, distribution file storage is completed, wherein the user node, 7, 8, 9 for the shared node file stored procedures, file download by other user nodes / transfer use.

[0154] 较佳的,在本发明中,复制策略采用了基于位操作的数据统计分布算法。 [0154] Preferably, in the present invention, it uses a statistical distribution of the replication policy operations on data bits based on an algorithm. 首先通过统计数据,最大限度的获取可用节点的分布信息,包括时间分布域和地区分布域等信息,生成分布统计集中应用的信息库,提供给复制服务节点使用; First of all statistical data, maximize access to distribution information available nodes, including time distribution and geographical distribution of domain-domain information, etc., to generate statistical distributions repository for centralized applications, providing service to copy nodes;

[0155] 其次,数据复制服务节点,通过调用分布统计信息库,得到节点的当前和统计信息,基于信息的位操作,得到最优化的分布图; [0155] Next, the data replication service node, by calling the distribution of statistical information base to give the current node and statistics, based on the bit operation information to obtain a profile optimization;

[0156] 然后,复制服务节点提交复制策略,调度分布节点,完成数据的复制; [0156] Then, copy service node submit replication strategy, scheduling distribution node, complete copy of the data;

[0157] 最后,根据复制过程中反馈的数据信息,更新统计信息库相关项。 [0157] Finally, according to data replication feedback, update statistics database-related items.

[0158] 如图10所示,示出了本发明的用户节点查找、发现并把文件下载到本地磁盘的文件下载流程,其包括: [0158] As shown in FIG. 10, the present invention shows the user node search, find and download the file to the local disk file download process, which comprises:

[0159] (1)已注册并登陆用户节点1,发送下载请求到登陆服务节点1 ; [0159] (1) has been registered and logged in user node 1 sends a download request to the login service node 1;

[0160] (2)服务节点1根据请求文件信息,如请求下载文件的文件名、文件标识符(FCBID),查找网络中的共享节点;如果不能在本节点服务子网发现需要下载文件的全部信息,则需要发送查找请求到其他相关服务节点2,寻找可用的用户节点2 (该用户节点2即作为共享节点); [0160] (2) According to 1 request file serving node information, such as requests to download the file name of the file, a file identifier (FCBID), to find the shared node in the network; if not found in the present sub-node needs to download files and services all information search request needs to be sent to other service node 2, node 2 for an available user (i.e., user node as shared node 2);

[0161] (3)服务节点1和其他相关服务节点2返回本边缘节点网内和其他相关边缘节点网内有关于该需要下载文件的共享信息给该用户节点1 ;[0162] (4)用户节点1根据返回的共享节点列表,发出连接请求; [0161] (3) Service Node 1 and other related service node 2 returns within this edge-node and the other edge of the node network to share information about the need to download the file to the user node 1; [0162] (4) User node 1 node list sharing returned connection request is issued;

[0163] (5)用户节点2的守护进程,接收到用户节点1的连接请求,返回当前文件块状态; 如果文件块已经不可用,则返回拒绝信息,并通知登陆服务节点2更改自己状态; [0163] (5) User node daemon 2, receives the user node a connection request 1 returns the current file block state; if the file block is not already available, it returns an information, and notifies the log in the service node 2 changes its own state;

[0164] (6)用户节点1接收到节点的返回信息,确定是否开始下载;如果没有得到足够的节点响应,不能保证数据完整性,则重新向服务节点1发送新的请求,以得到更多的共享节点列表; [0164] (6) User node returns a node receiving the information, determines whether to start the download; if not enough nodes in response, data integrity can not be guaranteed, then re-send a new request to the service node 1, to obtain a more shared node list;

[0165] (7)用户节点间确定连接后,由共享节点准备需要下载的文件块和相应的校验值, 提交到传输管理模块开始传输文件块; Between [0165] (7) After determining the user nodes are connected, by a shared node ready to be downloaded file blocks and the corresponding check value, submitted to the transmission management module starts transmitting the file block;

[0166] (8)文件块传输完成后,用户节点2注册文件状态给调用服务节点1 ; [0166] (8) the file block transfer is complete, the user node registration file 2 serving node 1 to the calling state;

[0167] (9)下载完成后,用户节点1通知服务节点1。 After [0167] (9) the download is complete, the user notifies the service node Node 1 1.

[0168] 而对于文件传输,尤其是大文件的传输,其传输过程基本与上述过程类似,也要经过一个源文件的上传发布、复制分布存储的过程,将源文件发布存储到网络中的共享节点上,然后再通过目的节点请求文件下载到本地磁盘来实现大文件的传输。 [0168] For the file transfer, especially the transfer of large files, the transmission process is substantially similar to the process described above, but also through the release of a source file upload copy stored distribution process, the source file is stored in the network publish shared node, then the destination node by requesting the file downloaded to the local disk to realize the transmission of large files. 特别的是,在文件传输过程中,在上述文件上传过程的步骤2中,需要定位目的用户节点2的注册服务节点2, 并通知该服务节点2,其他步骤与上述文件上传发布、复制分布存储、下载的过程基本相同, 在此不再赘述。 In particular, the file transfer process, in step above file upload process 2, locate the destination user node registration service node 2 of 2, and notifies the service node 2, the other steps above file upload publish, reproduce distributed storage download process is basically the same, are not repeated here.

[0169] 另外,基于本发明的分布式文件存储系统,通过与无线、嵌入式设备的结合,可以供移动大容量存储等应用,进行大附件的邮件业务。 [0169] Further, distributed file storage system based on the present invention, by binding with a wireless, embedded devices, can be used for removable mass storage applications, e-mail attachments for large business.

[0170] 下面将结合图11A、11B、12分别说明本发明的分布式文件存储系统与无线终端、 邮件系统结合的应用。 [0170] below with reference to FIGS. 11A, 11B, 12, respectively, illustrate the application of distributed file storage system of the present invention in conjunction with a wireless terminal, the mail system.

[0171] 如图11A、1 IB所示,整个无线应用网络由分布式文件存储系统在internet网络上的服务节点和用户节点、移动运营商提供的无线网络和无线基站,用户的移动终端、第三方内容提供商等组成。 [0171] FIG. 11A, as shown by the entire wireless network application in a distributed file storage system internet service node on the network nodes and users, mobile operators provide wireless networks and radio base station, a mobile user terminal 1 IB, the first party content providers and so on. 其中,移动终端节点需要绑定到P2P网络存储系统的一个用户节点上, 该移动终端节点不提供共享空间,而是直接使用其绑定的用户节点的共享资源,其操作权限与被绑定的固定用户节点相同。 Wherein, the mobile terminal needs to bind to a node user nodes in the P2P network storage system, the mobile terminal node shared space is not available, but the use of shared resources directly bind the user node, the operation authority with the bound fixing the same user node. 并且,存储系统支持java环境和Symbian平台的设备, 以及提供第三方软件接口,可以做为移动传输的底层支持。 The storage system supports java environment devices and Symbian platforms, as well as third-party software interfaces, can be used as the underlying support mobile transmission.

[0172] 在本发明中,利用无线应用网络进行文件上传或下载等服务的过程包括如下步骤: [0172] In the present invention, the network using a wireless application services such as file upload or download process comprising the steps of:

[0173] (1)用户首先绑定移动终端到存储系统上的用户节点,然后可以使用该节点的ID 登陆,服务节点会标识该节点为移动状态,不作为分布存储节点使用。 [0173] (1) bind the mobile terminal user first user node to the storage system, the node may then use the login ID, the identifier of the serving node for the mobile node state, not as a distributed storage nodes. 该用户可以以普通方式上传文件或下载。 The user can upload or download files in the usual way.

[0174] (2)第三方软件可以调用存储系统的接口,提供其他增值服务。 [0174] (2) third-party software can store the system call interface, provide other value-added services.

[0175] 在无线应用过程中,对于移动终端的应用,需要运营商的支持。 [0175] In a wireless application process, the application for the mobile terminal, the operator needs to support. 在3G网络中,利用本发明的存储系统,可以对远程视频采集,IPTV提供优质的底层服务。 In the 3G network, using the storage system of the present invention, the remote video capture, IPTV provides high-quality underlying service.

[0176] 本发明的存储系统主要应用在互联网的文件分布存储、大文件传输、文件共享等方面。 [0176] storage system of the present invention is primarily used in the Internet distribution file storage, large file transfer, file sharing and so on. 下面以大附件邮件为例,进一步说明本发明的系统与第三方应用的结合应用。 Here a large e-mail attachments, for example, further described in conjunction with the application of the third application of the present invention.

[0177] 因为实际传输使用的是smtp/pop协议,普通邮箱提供的附件功能,在单个附件大小上有很大限制,一般邮箱为IOM左右,已经不能满足当前媒体应用等需求。 [0177] Because the actual transmission using smtp / pop protocol, accessory functions provided by ordinary mail, are very limited in the size of a single attachment, typically of approximately the IOM-mail, can not meet current needs media applications. 对于传送大文件的需求,普通方式只能用手动文件切割来处理,操作不方便,容易出错,发送速度低而且对于邮件系统有很大依赖。 Transmission requirements for large files, in a conventional manner can be handled manually paper cutting operation is inconvenient, error-prone, low transmission speed but also for a large mail system dependent.

[0178] 基于大文件传输的考虑,结合邮件系统与存储系统共同完成文件传输。 [0178] Based on consideration of large file transfers, e-mail systems and storage systems combine together to complete the file transfer. 可以解决单个附件大小的限制,提高了附件的传输效率。 You can solve a single attachment size limit and improve the transmission efficiency of attachments.

[0179] 作为存储系统对第三方软件扩展的支持,提供了一系列第三方组件,如对邮件系统的支持组件——“系统邮件代理模块”等,设置在存储系统用户节点和邮件服务器上,用于在邮件中有大附件时,调用存储系统中的共享空间来实现邮件中大附件的传输,如此即可解决了现有邮件系统中对于邮件附件大小的限制。 [0179] As the storage system extended support third-party software, a series of third-party components, such as support components mail system - "Mail System proxy module" and the like, provided on the storage system and a mail server user node, when there is a large attachment in the mail, call the shared space storage system to realize the transmission of large e-mail attachments, so you can solve the existing mail system attachment size limit for messages.

[0180] 例如,对于存储系统中的用户节点之间的大附件邮件传输,由于用户节点上安装有该邮件代理模块,可在进行大附件邮件传输时,直接调用存储系统中的共享空间来进行大附件的传输。 [0180] For example, for a large attachment between the user node storing the message in the transmission system, since the message proxy module mounted on the user node, when the message transfer large attachments may be performed, directly call the shared space to perform the storage system transmission of large attachments. 而对于不属于该存储系统中的用户来说,用户只需要在其邮件服务平台(如邮件服务器)上安装配置了相应的“系统邮件代理模块”作为邮件代理客户端组件,在进行大附件的邮件传输时,可与存储系统通信,调用存储系统中的共享空间,作为邮件中大附件的传输途径,这样就可以为用户提供高速的大附件传送功能。 For the storage system does not belong to the user, the user need only service platform (e.g., mail server) installation configuration corresponding "Agent System Messages module" Proxy Client as a mail message in its assembly, making large attachment when the message transfer, the storage system may be in communication with the storage space shared call system as mail transmission routes large attachments, so that the user can provide high-speed large attachments transfer function.

[0181] 如图12所示,示出了本发明的存储系统与邮件系统结合应用于大附件邮件的附件传输的结构。 [0181] As shown in FIG 12, it shows the structure of a storage system according to the present invention in conjunction with the email attachment system applied to the transmission of large e-mail attachments. 其中邮件系统内设置有一代理模块,与存储系统的服务器层网络连接,用于实现大附件邮件中附件的传输。 Wherein the messaging system is provided with a proxy module, connected to the server layer network storage system, for realizing the transmission of large email attachments attachments. 存储系统的服务器层的服务节点与邮件系统的针对不同环境下的邮件系统提供相应的代理程序,用于转发用户的附件传输请求。 Mail service node server system layer of the storage system for the agent to provide the corresponding mail system under different circumstances, for attachment to the user forwarded the transmission request. 在用户系统中,需要下载并安装存储系统的用户端代理,用于与存储系统网络连接。 In the user system, the need to download and install the client agent storage systems for connecting to the network storage system. 整个网络结构,可以利用现有的存储系统网络实现,可以保证用户的一致,并增加了系统的可用性(有足够多的节点)ο The whole network structure, the storage system can leverage existing network implementations, the user can ensure consistent, and increases the availability of the system (there are more than enough nodes) o

[0182] 如果没有大文件的传输,邮件系统单独工作,不涉及存储系统。 [0182] If there is no large file transfer, mail system working alone, do not involve the storage system. 如果用户发送者(Sender)发送一个大文件给用户接收者(Reciver),邮件系统会对附件的请求做一个判断,并通知“系统邮件代理”有用户请求,“系统邮件代理”通知联系邮件用户发送者(knder),启动文件上传流程;邮件正文由邮件系统发送给用户接收者(Reciver),其中包括附件的索引信息、用户的权限证明等。 If the user sender (Sender) to send a large file to the user receiver (Reciver), the system will request message with the attachment to make a judgment, and notice "system mail agent" requested by a user, "mail proxy system" to inform the user e-mail contact sender (knder), start the file upload process; the body of the message sent by the mail system to the user receiver (reciver), which includes indexing information about the attachment, the user's permissions proof. 用户接收者(Reciver)收到邮件后,如果点击下载附件,会直接连到P2P网络存储系统的服务节点,请求文件下载,服务节点验证用户的权限证明后,返回文件分布信息给用户接收者(Reciver),用户接收者(Reciver)开始连接其他用户节点,接收附件文件。 Users receiver (Reciver) after receiving the message, if you click to download attachments, directly connected to the service node P2P network storage systems, file download request, the service node to verify the user's permissions proof, return the file to the user distribution information recipients ( reciver), the recipient user (reciver) starts to connect to other user nodes, receiving a file attachment. 用户接收者(Reciver)接收完毕,通知服务节点“附件传输完毕”,服务节点通知文件分布节点删除该附件所占用空间,并通知邮件服务器接收附件完成,如果需要可以通知用户发送者(Sender)邮件成功。 Users receiver (Reciver) received, notification service node "Accessories transfer is completed", the service node notification document distribution node delete the attachment occupied space, and notifies the mail server receives the attachment is complete, if necessary, notify the user sender (Sender) Mail success.

[0183] 另外,跨越NAT的问题,对于国内互联网是一个基本的要点。 [0183] In addition, the problem across the NAT, the domestic Internet is a fundamental point. 本发明的MEDDQSN 除了支持基本的UDP跨越方式,如TURN、STUN、MIDCOM方式;同时,还提供基于TCP连接和TCP&ICMP方式,来保证数据传输过程中的流量控制、Qos等性能。 MEDDQSN the present invention, in addition to support substantially spanning the UDP mode, such as TURN, STUN, MIDCOM embodiment; while also providing TCP connections based on TCP & ICMP manner and to ensure that the flow control of data transmission, Qos properties.

[0184] 综上所述,本发明的存储系统作为一个基于internet的高效存储系统,既可以作为单独的应用提供给客户,也可以作为底层支持与大量第三方应用结合,提高整个网络传输的性能。 [0184] In summary, a storage system of the present invention as a highly efficient internet-based storage system, may be provided to a customer as separate applications, or as a large number of third-party applications and underlying support, and to improve the transmission performance of the entire network .

[0185] 当然,本发明还可有其他多种实施例,在不背离本发明精神及其实质的情况下,熟悉本领域的技术人员当可根据本发明作出各种相应的改变和变型,但这些相应的改变和变形都应属于本发明所附的权利要求的保护范围。 [0185] Of course, the present invention may have various other embodiments without departing from the spirit and the essence of the present invention, those skilled in the art can make various corresponding modifications and variations according to the present invention, but these corresponding changes and variations should fall within the scope of the appended claims.

Claims (26)

  1. 1. 一种在网络上构建数据量子空间的方法,其特征在于,该方法包括构建一分布式文件存储系统,该系统包括由若干个彼此网络连接的服务节点构成的骨干节点网,并以每个服务节点为中心构成一独立的边缘节点网,每一边缘节点网中包含有若干个用户节点,每个用户节点与其所属边缘节点网内的服务节点网络连接,且每个用户节点都具有一预定的共享空间;用户节点利用服务节点选择将数据文件的文件块上传发布到网络中可用的用户节点上,形成分布节点;在用户节点将数据文件的文件块上传后或在数据文件的文件块的传输过程中,服务节点自动调度整个骨干节点网络内的可用节点,分配承载任务,把用户请求上传或传输的数据文件的文件块通过复制,分发存储到多个用户节点的共享空间内,形成多个共享节点,供用户节点下载或传输文件用, 1. A method of constructing quantum space data on a network, characterized in that the method comprises constructing a distributed file storage system, the system comprising a network backbone nodes connected by a plurality of serving node network configuration to each other, and every a service node configured as the center of a separate network edge nodes, each edge node of the network contains a plurality of user nodes, the network service node within each user node is connected to its network edge node belongs, and every node has a user predetermined shared space; user node using a service node selects the file blocks of the data file uploaded published to the user nodes in the network is available to form a distribution node; the user node after the file block data file upload or file blocks in a data file during transmission, automatic scheduling service node available nodes throughout the network backbone nodes, load distribution task, the user request file block data file uploaded or transmitted by copying, distribution storage into the shared space a plurality of user nodes, forming a plurality of shared node, a node for the user to download or transfer files, 而在网络上构建形成数据存储和传输的量子空间。 Constructed subspace forming data stored and transmitted over the network.
  2. 2.根据权利要求1所述的方法,其特征在于,形成分布节点的过程包括用户节点向其所属的边缘节点网内的服务节点发出上传请求;该服务节点接到请求后,确定并返回可用的用户节点信息给该用户节点; 该用户节点根据返回的可用的用户节点信息将所需上传的数据文件的文件块发布到其中部分或全部可用节点上,形成分布节点。 2. The method according to claim 1, wherein the distribution node includes a process of forming a user node sends an upload request to its serving network node within the edge node belongs; the serving node after receiving the request, and return to the available the user node information to the user node; the user node based on the available information of the user node will return the required data uploaded file blocks to publish some or all available nodes form a distribution node.
  3. 3.根据权利要求2所述的方法,其特征在于,在发出上传请求前还包括用户节点制作发布索引文件,并在该上传请求中包含有索引文件信息。 3. The method according to claim 2, wherein, before sending the upload request further comprises a user node production publishing the index file, and includes information included in the index file upload request.
  4. 4.根据权利要求3所述的方法,其特征在于,该索引文件信息至少包含有文件标识信息、文件分块信息,该文件标识信息包括源用户节点ID信息、目的用户节点ID信息、文件物理信息和文件校验值。 4. The method according to claim 3, wherein the index file has the file identification information includes at least information file block information, the file identification information of the user includes a source node ID, destination node ID information of the user, the physical file information and file checksum value.
  5. 5.根据权利要求4所述的方法,其特征在于,该服务节点确定可用的用户节点信息的过程包括该用户节点所属的服务节点作为调用服务节点,在接到请求后,在其文件存储信息表中记录该索引文件信息,并根据索引文件信息,从用户节点信息表中查找可用节点;如果该用户节点所属的边缘节点网内的用户节点不足以满足该用户节点的文件上传需求,则该调用服务节点通知骨干节点网内的其他服务节点作为被调用服务节点,调用各自边缘节点网内的用户节点,以得到足够的共享空间,提供给该用户节点使用;该调用服务节点返回可用节点信息给该用户节点;接收到该用户节点请求的其他被调用服务节点,也返回可用节点信息给该用户节点,并通告各自维护的边缘节点网内被调用的用户节点;该用户节点根据返回的可用节点列表,选择其中部分用户节点 5. The method as claimed in claim 4, wherein, the user node information during the service node includes a service node determines an available node belongs as the user call the service node, upon request, information which is stored in the file the index file information table, and, from the user to find an available node in the node information table according to the index file information; user node in the network edge node if the node belongs to the user are insufficient to meet the needs of the user node uploads a file, the invoke services node notifies other serving nodes in the backbone nodes network as a service node is called, the calling user nodes within a respective edge-node to obtain enough shared space, providing to the user nodes; the call service node returns available node information to the user node; receiving another service node calls the user node request, returns an available node information to the user node, and notify the user node is called within a respective maintenance edge node of the network; the user node based on the return of the available node list, select the node in which some users 为分布节点,并向其发出连接请求;分布节点接收到该用户节点的连接请求,返回当前状态及可用信息;如果共享空间已经不可用,则返回拒绝信息,并通知其登陆的服务节点更改自己的状态;该用户节点接收到分布节点的返回信息,确定是否开始传输;如果该用户节点没有得到足够的可用节点响应,则重新向其所属的服务节点发送新的请求,以得到更多的可用节点列表。 The distribution node, and it sends a connection request; distribution node receives a connection request of the user node, and returns the current state of information available; if the shared space is not available, then refused to return information, and inform the service node to change their landing state; the distribution node to the user node receives the return information, determines whether to start transmission; node if the user does not get enough available nodes in response, re-send a new request to its serving node belongs, to obtain more available node list.
  6. 6.根据权利要求5所述的方法,其特征在于,在用户节点请求可用存储空间时,服务节点可通过以下方法之一选择可用节点:最佳匹配法,查找所有可用用户节点提供的可用空间块,和需要存储的文件块比较,相差最小的块,认为是最佳匹配结果,用以保证有足够大的空间块提供给其他用户使用;最差匹配法,查找所有可用用户节点提供的可用空间块,把最大的块提供给文件块使用,认为是最差匹配结果,用以保证剩余的空间块都足够大,不至于失去存储价值;首次匹配法,从可用的空间列表中,选取首先查找到的可用空间块给用户节点,认为是首次匹配,用以保证对用户请求的响应时间最短。 6. The method as claimed in claim 5, wherein, when the node requests the user memory space available, the service node may select an available one of the following nodes: the best matching method, to find all the available free space node provides users blocks, and the blocks need to store the file comparison, the smallest difference block, is considered the best match, there is enough space available to other blocks to ensure that users; worst matching, to find all the available user node provides the available space block, the biggest block to block file use, considered to be the worst match results to ensure that the remaining space blocks are large enough not to lose the stored value; the first match method, from the list of available space, select first found blocks to the available space user node, that is the first match, to ensure that the response time to user requests shortest.
  7. 7.根据权利要求6所述的方法,其特征在于,该用户节点是在可用节点列表中选择优先级较高的可用节点作为分布节点。 7. The method according to claim 6, characterized in that the node is a user selected higher priority available node as distribution node in the list of available nodes.
  8. 8.根据权利要求7所述的方法,其特征在于,该优先级由一与该骨干节点网的各个服务节点连接的汇集服务节点统计,计算每个用户节点的效率和可用度,排列用户节点在系统中的优先级。 8. The method according to claim 7, wherein the priority of a collection service by a node connected to each service node of the backbone network node statistics, computational efficiency and availability of each user node, the user node arrangement priority in the system.
  9. 9.根据权利要求5所述的方法,其特征在于,该用户节点上传数据文件的文件块的过程包含该用户节点与分布节点间确定连接后,准备需要传输的数据文件的文件块和相应的校验值,提交到其所属的服务节点中开始传输文件块;文件块传输完成后,分布节点注册文件状态给各自所属的服务节点和该用户节点所属的服务节点。 9. The method according to claim 5, characterized in that, during the user node uploading data files containing blocks is determined between the user node and the distribution node is connected to the data needed to prepare the file transfer and file blocks corresponding check value, which is submitted to the serving node belongs start the file transfer blocks; the file block transfer is complete, the state of distribution node registration file belongs to a respective service node and a service node to the user node belongs.
  10. 10.根据权利要求1所述的方法,其特征在于,该数据文件的文件块的复制分布存储过程包含在数据文件的文件块上传完毕后或在数据文件的文件块传输过程中,上传文件的用户节点所属的服务节点或文件传输的源用户节点所属的服务节点向其他服务节点发出同步请求;其他服务节点接收到请求后,查找各自边缘节点网内可用的用户节点,选择并调用其中部分用户节点作为共享节点,并向这些共享节点通知同步请求及共享空间征用大小;同时通知网络中存储有该数据文件的文件块的分布节点需要向该共享节点复制该数据文件的文件块;数据复制完毕后,各共享节点分别通知各自所属的边缘节点网的服务节点;各分布节点也分别通知各自所属的服务节点;各共享节点所属的服务节点通知各分布节点所属的服务节点,文件分布存储完成。 10. The method according to claim 1, wherein the distributed memory copy process of the data file of file blocks contained in the file block data uploaded file or file blocks in the data file transfer process, the file upload a source user node serving node or file transfer of the user node belongs serving node belongs issues a synchronization request to another serving node; other serving node after receiving the request, looks up the user node within the respective edge-node is available, select and invoke some users node as a shared node, and notifies the synchronization request and the size of the shared space requisition shared node; notify the network file blocks stored in the document distribution node data file blocks need to copy the data file to the shared node; data replication is completed after each shared node notifies the service node edge node network they belong to; each distribution node also notifies the service node they belong to; service nodes each shared node belongs to the notification service nodes of each distribution node belongs, document distribution storage completion.
  11. 11.根据权利要求10所述的方法,其特征在于,该服务节点是在可用的用户节点中选择优先级较高的用户节点作为共享节点。 11. The method according to claim 10, wherein the service node is to select the higher priority user node in a shared node available user nodes.
  12. 12.根据权利要求1所述的方法,其特征在于,还包括在用户节点进行数据文件的下载或传输的过程中,用户节点利用包含有所需下载或传输的数据文件的文件块的共享节点来完成所需数据文件的下载或传输。 12. The method according to claim 1, characterized in that the node further comprises a shared file block transfer or process data files to download at user node, the user node includes the use of a desired data file downloaded or transferred is required to complete the download or transfer data files.
  13. 13.根据权利要求12所述的方法,其特征在于,该数据文件下载的过程包含用户节点发送下载请求到其所属的服务节点;该服务节点查找共享节点;如果不能在该用户节点所属的边缘节点网内发现全部信息,则需要发送查找请求到其他相关服务节点;该服务节点和其他相关服务节点返回本边缘节点网内和其他相关边缘节点网内包含有关于该数据文件的共享信息的节点列表给该用户节点;该用户节点根据返回的节点列表,向共享节点发出连接请求; 共享节点接收到用户节点的连接请求,返回当前关于该数据文件的文件块状态;如果文件块已经不可用,则返回拒绝信息,并通知其所属的服务节点更改自己状态;用户节点接收到共享节点的返回信息,确定是否开始下载;如果没有得到足够的节点响应,不能保证数据完整性,则重新向其所属的服务节点发送 13. The method according to claim 12, wherein the data file comprises a process of downloading user node transmits a download request to its serving node belongs; service node searches the shared node; if not an edge belongs to the user node inner-node find all the information, it needs to send a search request to other service node; the service node and other related service node returns a node within this edge-node and the other edge of the node network with shared information about the data file list to the user node; the user node based on the list returned node, a connection request to the shared node; shared node receives the connection requesting user node, returns the current file block status on the data file; If a file block is not already available, rejection message is returned, and inform the serving node belongs to change their own state; the user node receives the shared node information returned, determining whether to start the download; if not enough nodes in response, can not guarantee data integrity, re belongs thereto the service node sends 的请求,以得到更多的节点列表;用户节点与共享节点之间确定连接后,准备需要下载的文件块和相应的校验值,提交到其所属的服务节点的传输管理模块开始传输文件块;文件块传输完成后,共享节点注册文件状态给各自所属的服务节点和调用文件的该用户节点所属的服务节点;下载完成后,该用户节点通知其所属的服务节点。 A request to obtain a list of additional nodes; determining the connections between the user node and the common node, to prepare the downloaded file blocks and corresponding check value, the transmission management module submitted to their respective serving node starts to transmit the file block ; block transfer is complete the file, the file status register shared node to the service serving node and each node calls the file belongs to the user node belongs; Once downloaded, the user node notifies its serving node belongs.
  14. 14.根据权利要求1所述的方法,其特征在于,上传之前还包含服务节点注册,服务节点启动后,需要配置自己的网络域和至少一个邻居节点,根据上述配置信息,服务节点发出上线通知,通告网络自己的状态和标识符,并同步得到自己网络域内的可用服务单元映射表,最后打开侦听进程,接受用户节点的连接请求;用户节点注册,用户节点启动后,如果是首次安装,需要连接到默认服务节点,注册自己的用户ID,服务节点根据用户节点提供的注册信息和源IP地址,生成唯一的用户节点标识符返回,并指定用户的首选服务节点;用户节点登陆,用户提供已经注册的ID和口令到首选服务节点,如果首选节点不可用,则计算二级服务节点,并尝试登陆;服务节点接收到登陆请求后,转发给协议模块处理, 生成用户本次会话的ID,向用户发出状态请求,要求用 14. The method according to claim 1, characterized in that, further comprising registering a service node before uploading the serving node starts to configure their network domains and at least one neighbor node, the online notification issued based on the configuration information, the service node advertise their status and network identifier, and synchronization is available map services unit within its own network, and finally open the listening process, accept connection requests user node; node registered user, the user restarted, if it is first installed, need to connect to the default service node, register their user ID, the service node based on the registration information provided by the user node and the source IP address, the user generates a unique node identifier return and specify the user's preferred service node; node user login, users registered ID and password to preferred service node, if the preferred node is not available, two service nodes are calculated, and attempts to log in; serving node after receiving the log-in request, forwarded to the protocol processing module generates a user ID of this session, issuing a status request to the user requirements for 户返回当前可用状态,并更新用户的在线状态。 Households return to the current availability status, and updates the user's online status.
  15. 15.根据权利要求14所述的方法,其特征在于,该服务节点标识符可为一 16位标识符, 表示该服务节点的信息;或者,该服务节点标识符可为一32位标识符,前16位表示该服务节点的信息,后16位保留,以提供服务节点新功能的扩充。 15. The method according to claim 14, wherein the serving node identifier may be a 16-bit identifier, information indicating the service node; or, the serving node identifier may be a 32-bit identifier, the first 16 bits represent the information service node, after 16 retained to provide expansion of the new features of the service node.
  16. 16.根据权利要求14所述的方法,其特征在于,该用户节点标符可为一 32位标识符, 前16位表示所属的边缘节点网,即该服务节点标识符中表示该服务节点的信息的前16位标识符;该用户节点标识符的后16位表示该用户节点自己的节点信息。 16. The method according to claim 14, wherein the user node identifier may be a standard 32-bit identifier, the front edge 16 represented ordinary network node, i.e. the serving node identifier indicates that the service node the first 16 bits of the identifier information; after the user node identifier 16 indicates that the user node's own node information.
  17. 17.根据权利要求14所述的方法,其特征在于,还包括对于服务节点的非正常失效,通过服务节点间心跳机制来保证失效节点的及早通告; 对于用户节点的非正常失效,包括可恢复的用户网络故障和不可恢复的用户节点信息故障两种情况,对于可恢复情况,标识用户状态为不可用;对于不可恢复情况,通知所有存储的信息源节点,作重新数据分布。 17. The method according to claim 14, characterized in that, further comprising a failure for non-normal serving node, serving node through the inter-heartbeat mechanism to ensure early failure nodes advertisement; abnormal failure for user node, including recoverable the user network fault information and fault both cases the user node unrecoverable, to restore the situation, the user identifies the status as unavailable; for non-recovery, all the notification information stored in the source node, data for re-distribution.
  18. 18.根据权利要求17所述的方法,其特征在于,服务节点只接受同一运营商用户节点的连接,以保证用户间数据传输的速度;只有当服务节点失效,没有其他服务节点可选的情况下,才允许用户节点跨运营商连接。 18. The method according to claim 17, wherein the service node only accepts user node connected to the same operator in order to ensure that the speed of data transfer between the user; only the case when the service node fails, there is no other alternative service node next, it allows the user node inter-operator connection.
  19. 19.根据权利要求14所述的方法,其特征在于,在用户节点注册、登陆之前,还包括以下步骤:由一服务节点定期采集域名分配组织公布的运营商数据,整理后生成一IP-运营商的对照关系表,并发布给其他服务节点;用户节点注册、登陆时,服务节点根据该对照关系表,决定允许用户节点登陆,或把用户节点的登陆请求转发到更适合匹配关系的服务节点。 19. The method according to claim 14, wherein the user node registration, before the landing, further comprising the step of: after the acquisition by a domain name service node tissue distribution data carriers released periodically, generating a finishing operation IP- business control table, and distributed to other service node; user node registration, landing, service node based on the comparison table, decided to allow the user node login, or the login node forwards the request to a more appropriate match relations service node .
  20. 20.一种分布式文件存储系统,其特征在于,该系统包括由若干个彼此网络连接的服务节点构成的骨干节点网,并以每个服务节点为中心构成一独立的边缘节点网,每一边缘节点网中包含有若干个用户节点,每个用户节点与其所属边缘节点网内的服务节点网络连接,且每个用户节点都具有一预定的共享空间;其中,该用户节点作为数据文件的分布载体,在网络上共享空间,并使用其它用户节点的共享空间,实现数据文件的文件块的上传、 分布存储、下载和传输;该服务节点用于支撑服务节点与用户节点之间的存储与查找,可控制完成数据文件的文件块的上传、分布存储、下载和传输。 20. A distributed file storage system, characterized in that the system includes a network backbone nodes connected by a plurality of serving node network configuration to each other, and each service node to the center edge constitute a separate network nodes, each network edge node comprises a plurality of user nodes, each service node within the network user node to which it belongs edge-node connection, the user node and each having a predetermined shared space; wherein the user node as a data file distribution carrier, block upload files shared on the network space, and the use of other shared space user node, data files, distributed memory, download and transfer; service node for storing the lookup service between the support node and the user node upload block, data files can control is completed, distributed memory, download and transfer.
  21. 21.根据权利要求20所述的系统,其特征在于,该系统还包括:一汇集服务节点,与骨干节点网网络连接,用于各服务节点与用户节点的数据统计、分析。 21. The system according to claim 20, characterized in that the system further comprises: a collection of service nodes, network nodes and a network backbone connection, each service node statistics for the user node, analysis.
  22. 22.根据权利要求21所述的系统,其特征在于,该服务节点包括:用户界面层模块,用于本地服务节点的配置与操作,并提供与汇集服务节点的通信功能;棋盘映射服务节点模块,其包含底层连接模块,用于提供网络连接接口给上层应用;监听模块,与该底层连接模块连接,用于监听节点的请求,并保持与其他节点的信息交换;用户节点存储模块,用于存储登陆到本服务节点的所有用户节点信息;服务节点存储模块,用于存储整个骨干节点网内所有服务节点信息;协议模块,分别与该用户节点存储模块、服务节点存储模块连接,其包含有质量控制模块,用于实现数据质量保证管理;节点管理模块,用于实现节点分布、查找;错误控制模块,用于实现节点失效的控制和管理;配置管理模块,用于配置系统参数;连接管理模块,用于本服务节点与其他节 22. The system according to claim 21, wherein the service node comprising: a user interface layer module, for the local serving node and its operation, and provides a communication function serving node pooled; board mapping module serving node which connection module comprising a bottom, for providing a network connection interface to an upper layer application; monitoring module, connected with the bottom of the connection module, listening for requesting node, and maintaining exchange information with other nodes; user node storing module, for login user node stores all information of the present serving node; serving node storage means for storing all service nodes within the entire backbone network node information; protocol module, a user node connected to the memory module, memory module serving node, which comprises quality control module for implementing data management of quality assurance; node management module, for implementing distributed nodes, find; error control module, and controls for implementing management node failure; configuration management module, configured to configure system parameters; connection Manager module, for the present node and other sections serving 的连接管理;系统心跳模块,用于保持连接。 Connection management; heartbeat module system for maintaining the connection.
  23. 23.根据权利要求21所述的系统,其特征在于,该一个或多个用户节点可为虚拟的超级节点,用于在一预定时间内保留特定用户数据的全部镜像信息,针对用户节点的不同级别和请求类型,选择性的使用这些虚拟的超级节点进行数据的传输。 23. The system according to claim 21, wherein the one or more nodes may be a virtual user super node for a specific user all image information retaining data within a predetermined time, for different user node request type and level, selective use of these virtual super nodes for data transmission.
  24. 24.根据权利要求21所述的系统,其特征在于,该用户节点包括: 系统支持层模块,用于为用户节点运行于不同的操作平台提供支持;底层连接模块,与该系统支持层模块连接,用于提供网络连接接口给上层应用; 监听模块,与该底层连接模块连接,用于监听服务节点或其他用户节点的连接请求; 文件发布模块,用于发布共享文件操作;本地文件管理模块,与该文件发布模块连接,用于数据文件的上传/下载,以及文件块的验证;共享空间管理模块,与该本地文件管理模块连接,用于统一管理用户节点共享给系统的磁盘空间,并动态分配共享空间的使用;传输模块,与该监听模块、共享空间管理模块连接,用于提供接口给上层结构,并通知数据文件的接收与发送。 24. The system according to claim 21, wherein the user node comprising: a system support layer module, for providing support for the user nodes running different operating platforms; underlying connection module connected to the system support layer module , for providing a network connection interface to an upper layer application; monitoring module, connected with the bottom connection module, listening for connection requests service node or other user nodes; document publishing means for publishing documents sharing operation; local file management module, publish module is connected with the file, the file data for upload / download, and verifying the file blocks; shared space management module, connected to the local file management module, for the unified management of the system to the user node shared disk space, and dynamic using the allocated shared space; transmission module, shared space management module connected with the monitoring module, for providing an interface to the upper structure, and notifies the receiving and sending data files.
  25. 25.根据权利要求21所述的系统,其特征在于,该系统还包括:移动终端,其通过移动运营商提供的无线网络和无线基站与系统中的服务节点连接, 并与该系统中的用户节点绑定,可直接使用共享资源进行文件的上传或下载。 25. The system according to claim 21, characterized in that the system further comprises: a mobile terminal, which is provided by the mobile radio network operator and a radio base station connected to the service node in the system, and the system user node binding, shared resources can be used directly upload or download files.
  26. 26.根据权利要求21所述的系统,其特征在于,该用户节点还包括:一邮件代理模块,用于在系统的用户节点之间实现大附件邮件中大附件的传输;或者用于与一对应设置有邮件代理模块的邮件服务平台配合,实现该系统的用户节点与邮件服务平台的用户之间的大附件邮件中附件的传输。 26. The system according to claim 21, wherein the user node further comprises: a mail agent module configured between a user node system that enables transmission of large e-mail attachments large attachments; or used with a provided corresponding to the mail agent module with the mail service platform, to achieve a large attachment mail transmission between the user node and the user of the mail service system platform attachment.
CN 200610114592 2006-11-16 2006-11-16 Method for constructing data quanta space in network and distributed file storage system CN101188569B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN 200610114592 CN101188569B (en) 2006-11-16 2006-11-16 Method for constructing data quanta space in network and distributed file storage system

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN 200610114592 CN101188569B (en) 2006-11-16 2006-11-16 Method for constructing data quanta space in network and distributed file storage system

Publications (2)

Publication Number Publication Date
CN101188569A true CN101188569A (en) 2008-05-28
CN101188569B true CN101188569B (en) 2011-05-04

Family

ID=39480764

Family Applications (1)

Application Number Title Priority Date Filing Date
CN 200610114592 CN101188569B (en) 2006-11-16 2006-11-16 Method for constructing data quanta space in network and distributed file storage system

Country Status (1)

Country Link
CN (1) CN101188569B (en)

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US8909738B2 (en) 2008-03-20 2014-12-09 Tajitshu Transfer Limited Liability Company Redundant data forwarding storage
US9203928B2 (en) 2008-03-20 2015-12-01 Callahan Cellular L.L.C. Data storage and retrieval

Families Citing this family (27)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20100250784A1 (en) * 2009-03-26 2010-09-30 Terascale Supercomputing Inc. Addressing Scheme and Message Routing for a Networked Device
CN101557421B (en) 2009-05-19 2012-02-01 北京北大方正电子有限公司 A network file distribution method and system
CN101562543B (en) 2009-05-25 2013-07-31 阿里巴巴集团控股有限公司 Cache data processing method and processing system and device thereof
CN101577626B (en) 2009-06-05 2011-04-13 西北工业大学 Method for monitoring initiative specific information dissemination based on eMule
CN102035866B (en) 2009-09-24 2014-06-04 腾讯科技(深圳)有限公司 Method and device for downloading data by P2P
CN101710901B (en) 2009-10-22 2012-12-05 乐视网信息技术(北京)股份有限公司 Distributed type storage system having p2p function and method thereof
CN102340497A (en) * 2010-07-23 2012-02-01 北京乐升科技有限公司 Online game system and online game resource processing method
CN102014150B (en) * 2010-09-29 2013-12-11 厦门市美亚柏科信息股份有限公司 Distributed small file storage system based on UDP (User Datagram Protocol) and data processing method thereof
US20130290529A1 (en) * 2011-01-10 2013-10-31 Storone Ltd. Large scale storage system
CN102209118A (en) * 2011-06-10 2011-10-05 成都勤智数码科技有限公司 Distributed mass data gathering method
CN103108005A (en) * 2011-11-11 2013-05-15 上海聚力传媒技术有限公司 Method, device and system for achieving data sharing in distributed storage system
CN102622187A (en) * 2011-11-28 2012-08-01 苏州奇可思信息科技有限公司 Network mapping based distributive file storage method
CN102624859A (en) * 2011-11-28 2012-08-01 苏州奇可思信息科技有限公司 Method for storing network files with multiple copies in distributed manner
CN102685240A (en) * 2012-04-19 2012-09-19 高剑青 Cloud storage based on E-mail
CN102722584B (en) * 2012-06-08 2015-07-15 北京天地云箱科技有限公司 Data storage system and method
CN102761607A (en) * 2012-06-21 2012-10-31 深圳市谷米科技有限公司 High-availability cache group implementation method applied to mobile Internet
US9612851B2 (en) 2013-03-21 2017-04-04 Storone Ltd. Deploying data-path-related plug-ins
CN103647797A (en) * 2013-11-15 2014-03-19 北京邮电大学 Distributed file system and data access method thereof
CN103970881A (en) * 2014-05-16 2014-08-06 浪潮(北京)电子信息产业有限公司 Method and system for achieving file uploading
CN103997540B (en) * 2014-06-10 2017-09-12 深圳市友华通信技术有限公司 Implementation of distributed storage network
CN104980417A (en) * 2014-08-06 2015-10-14 腾讯科技(深圳)有限公司 UGC file distribution method and equipment
CN104317716B (en) * 2014-10-30 2017-10-24 华为技术有限公司 The method of data transmission between the distributed nodes and the distributed node device
CN104932953B (en) * 2015-06-04 2017-11-21 华为技术有限公司 A data distribution method, a data storage method, apparatus and system related
CN104935668A (en) * 2015-06-26 2015-09-23 四川长虹电器股份有限公司 Distributed file system and data synchronization method therefor
CN105187551A (en) * 2015-09-29 2015-12-23 成都四象联创科技有限公司 Distributed computing method based on cloud platform
CN105227672B (en) * 2015-10-13 2018-04-17 国家电网公司 Method and system for data storage and access
CN105429942A (en) * 2015-10-28 2016-03-23 无锡澳优汇国际贸易有限公司 Enterprise internal transaction data rapid copy system

Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN1480859A (en) 2002-09-03 2004-03-10 鸿富锦精密工业(深圳)有限公司 Synchronous system in distributed files and method
US6711559B1 (en) 1999-12-27 2004-03-23 Fujitsu Limited Distributed processing system, apparatus for operating shared file system and computer readable medium
CN1567237A (en) 2003-06-09 2005-01-19 浪潮电子信息产业股份有限公司 Method for constructing high-available distributed memory system

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US6711559B1 (en) 1999-12-27 2004-03-23 Fujitsu Limited Distributed processing system, apparatus for operating shared file system and computer readable medium
CN1480859A (en) 2002-09-03 2004-03-10 鸿富锦精密工业(深圳)有限公司 Synchronous system in distributed files and method
CN1567237A (en) 2003-06-09 2005-01-19 浪潮电子信息产业股份有限公司 Method for constructing high-available distributed memory system

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US8909738B2 (en) 2008-03-20 2014-12-09 Tajitshu Transfer Limited Liability Company Redundant data forwarding storage
US9203928B2 (en) 2008-03-20 2015-12-01 Callahan Cellular L.L.C. Data storage and retrieval

Also Published As

Publication number Publication date Type
CN101188569A (en) 2008-05-28 application

Similar Documents

Publication Publication Date Title
Dannewitz et al. Network of information (netinf)–an information-centric networking architecture
US7257817B2 (en) Virtual network with adaptive dispatcher
US20030187974A1 (en) Broadcast tiers in decentralized networks
US5968121A (en) Method and apparatus for representing and applying network topological data
US20060168304A1 (en) Network traffic control in peer-to-peer environments
US8489701B2 (en) Private virtual LAN spanning a public network for connection of arbitrary hosts
US7177929B2 (en) Persisting node reputations in transient network communities
US20030217139A1 (en) Content tracking in transient communities
US7782866B1 (en) Virtual peer in a peer-to-peer network
US20030131061A1 (en) Transparent proxy server for instant messaging system and methods
Xylomenos et al. A survey of information-centric networking research.
US20080080392A1 (en) Virtual peer for a content sharing system
US20070064702A1 (en) Modifying operation of peer-to-peer networks based on integrating network routing information
US20030188019A1 (en) Providing management functions in decentralized networks
US7373394B1 (en) Method and apparatus for multicast cloud with integrated multicast and unicast channel routing in a content distribution network
US20130282860A1 (en) Name-Based Neighbor Discovery and Multi-Hop Service Discovery in Information-Centric Networks
US20120204224A1 (en) Method and Apparatus for a Control Plane to Manage Domain-Based Security and Mobility in an Information Centric Network
Raychaudhuri et al. Mobilityfirst: a robust and trustworthy mobility-centric architecture for the future internet
US20070237153A1 (en) Topology and routing model for a computer network
US20090222450A1 (en) System and a method for transferring email file attachments over a telecommunication network using a peer-to-peer connection
US20080071900A1 (en) Device and a method for communicating in a network
Seth et al. Low-cost communication for rural internet kiosks using mechanical backhaul
US20100094981A1 (en) Dynamically Deployable Self Configuring Distributed Network Management System
US20070094279A1 (en) Service provision in peer-to-peer networking environment
US20040107234A1 (en) Addressing method and system for using an anycast address

Legal Events

Date Code Title Description
C06 Publication
C10 Request of examination as to substance
C14 Granted
COR Bibliographic change or correction in the description

Free format text: CORRECT: ADDRESS; FROM: 100036 HAIDIAN, BEIJING TO: 100840 HAIDIAN, BEIJING

C41 Transfer of the right of patent application or the patent right
ASS Succession or assignment of patent right

Owner name: LI MING

Free format text: FORMER OWNER: RAO DAPING

Effective date: 20121025

ASS Succession or assignment of patent right

Owner name: BEIJING YUNHENG TECHNOLOGY RESEARCH INSTITUTE CO.,

Effective date: 20140929

C41 Transfer of the right of patent application or the patent right