CN1258921C - Distributive video order program system and its data recording and accessing method - Google Patents

Distributive video order program system and its data recording and accessing method Download PDF

Info

Publication number
CN1258921C
CN1258921C CN 02136350 CN02136350A CN1258921C CN 1258921 C CN1258921 C CN 1258921C CN 02136350 CN02136350 CN 02136350 CN 02136350 A CN02136350 A CN 02136350A CN 1258921 C CN1258921 C CN 1258921C
Authority
CN
China
Prior art keywords
data
server
data blocks
block
stored
Prior art date
Application number
CN 02136350
Other languages
Chinese (zh)
Other versions
CN1472963A (en
Inventor
夏建洲
刘湘宇
王�华
王怿忻
张建强
李加周
李喜欣
温央央
Original Assignee
中兴通讯股份有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 中兴通讯股份有限公司 filed Critical 中兴通讯股份有限公司
Priority to CN 02136350 priority Critical patent/CN1258921C/en
Publication of CN1472963A publication Critical patent/CN1472963A/en
Application granted granted Critical
Publication of CN1258921C publication Critical patent/CN1258921C/en

Links

Abstract

本发明公开了一种分布式视频点播系统及其实现数据存储和访问的方法,所述的存储方法包括以下步骤:数据存储的方法包括以下步骤:对数据文件按设定的方式进行数据分块;对设定数目的数据块计算形成校验数据块;将所述数据块和校验数据块按设定的方式轮流存储到设定的多个缓冲服务器中;在所述每一个缓冲服务器中生成相应的数据块存储的索引信息。 The present invention discloses a distributed video on demand system and its data storage and access method implementation, the storage method comprises the steps of: a data storage method comprising the steps of: the data file data block in the manner set ; the number of data blocks set computing parity data blocks is formed; the data blocks and parity data blocks is set to a plurality of turns stored in the cache server is set; in each of the buffer server generating index information stored in the corresponding data block. 采用本发明的技术方案,达到了提高存储效率的效果,节省了大量的存储空间;同时由于数据采用分布式存储,数据文件在多个缓冲服务器间分布,提高了数据的安全性;另一方面由于采用了优选的数据冗余校验的方法,提高数据分布的可靠性。 Aspect of the present invention, to achieve the effect of improving the storage efficiency and save a lot of storage space; the same time as distributed data storage, data files distributed across multiple servers buffer, improves the security of the data; on the other hand As a result of the preferred method for data redundancy checking, improve the reliability of data distribution.

Description

分布式视频点播系统及其实现数据存储和访问的方法 Distributed Video-on-demand system and its implementation of data storage and access methods

技术领域 FIELD

发明涉及分布式视频点播系统,尤其涉及数据的存储和访问。 VOD invention relates to distributed systems and more particularly relates to storing and accessing data.

背景技术 Background technique

数据存储是视频点播系统中涉及的一种关键技术。 Data storage is a key technology involved in video on demand system. 在分布式视频点播系统中,对数据进行存储有很特殊的要求。 In a distributed video on demand system, the data has a special storage requirements. 由于视频点播系统中对数据存储空间的需求非常大,因此,在分布式视频点播系统中,如何提高存储空间的利用率就成为一个很困难的问题。 Since the video-on-demand system, demand for data storage space is very large, therefore, in a distributed video-on-demand system, how to improve the utilization of storage space has become a very difficult problem.

在常规的分布式视频点播系统中,如采用美国专利56491%(Di由场时记。tora,mana沙ment sys扭.having a cache server ad me比记伪e挂fer)中描述的方法,所述分布式方法是基于一种地理区域上的划分,为了让所有的视频服务器共享某个或某些需要缓冲的数据流,就必须将数据复制到所有的缓冲服务器,这意味着数据在整个分布式视频点播系统中存在多个拷贝,因此数据存储空间的占用很大,整个分布式视频点播系统的数据存储空间利用率很低。 In conventional distributed video on demand system, as in U.S. Patent 56491% (Di by a field in mind when .tora, mana sand ment sys torsion .having a cache server ad me than hanging FER referred dummy e) of the method described, the said distributed approach is based on a division of the geographical area, in order to allow all of the servers share a video or some need to buffer the data stream, you must copy the data to all of the cache server, which means that data throughout the distribution VOD systems present multiple copies, and therefore occupies a large data storage space, data memory space of the whole distributed video on demand system utilization is low. 此外由于数据的完整拷贝在多个服务器上存在,大大地降低了数据的安全性,增加了数据被黑客等入侵者窃取的可能性。 Furthermore, since the presence of a complete copy of the data on multiple servers, greatly reduces the security of the data, increasing the possibility of data being stolen by hackers and other intruders.

发明内容 SUMMARY

本发明所要解决的技术问题是为了克服现有的分布式视频点播系统中数据流需要在分布的缓冲服务器多次拷贝而造成存储空间利用率低的缺点。 The present invention solves the technical problem to overcome existing distributed video on demand system requires multiple copies of the data stream in the buffer of the distribution server caused by the disadvantages of a low utilization rate of storage space. 实现本发明所要解决的技术问题而采取的技术方案概括如下:一方面,提出分布式视频点播系统中实现数据存储和访问的方法,其特征在于数据存储的方法包括以下步骤:A.对数据文件按设定的方式进行数据分块;B.对设定数目的数据块计算形成校验数据块; The present invention achieve technical problem to be solved by the technical solutions adopted are summarized as follows: In one aspect, a method of distributed video on demand system for data storage and access, characterized in that the data storage method comprising the steps of: A data file. the data block in the manner set;. B is set to the number of data blocks is calculated parity data blocks is formed;

C.将所述数据块和校验数据块按设定的方式轮流存储到设定的多个缓冲服务器中;D.在所述每一个缓冲服务器中生成相应的数据块存储的索引信息。 C. The data blocks and parity data blocks is set to a plurality of turns stored in the cache server is set;. D generates index information of the corresponding data block stored in each buffer of the server. 数据访问的方法包括以下步骤:a.客户端与服务器建立连接;b.客户端从索引服务器中读取要访问的数据文件对应的索引信息:c.按照索引信息在相应的多个缓冲服务器中读取数据块;d.对已读取达到组密度的数据块组进行去校验;e.客户端对读取的数据块进行重组,恢复数据文件。 Data access method comprising the steps of: a client establishes a connection with the server; B client reads index information corresponding to the data file to be accessed from the index server:... C according to the index information in the appropriate buffer plurality of servers read blocks;. D have to check on the read density data blocks group is reached groups;. E client of the read data blocks to restructure the data to restore the file.

另一方面,提出一种分布式视频点播系统,包括若干个视频客户端,若干个存贮数据文件的缓冲服务器和索引服务器,所述客户端和服务器通过通信网络进行连接和通讯,其特征在于:客户端向索引服务器发出视频文件请求,对从缓冲服务器读取的数据块重组;索引服务器,控制数据文件按设定方式被划分成数据块,对设定数目的数据块计算形成校验数据块,控制所述数据块和校验数据块按设定的方式分别轮流存储到多个数据块存储服务器;缓冲服务器,记录数据块存储的索引信息,并存储数据块。 On the other hand, provide a distributed video-on-demand system, comprising a plurality of video client, the cache server, and a plurality of index server storing data files, the client and server to connect and communicate through a communication network, wherein : the client sends a request to the index server the video file, the data block read from the buffer recombinant server; index server, the data file according to the set control mode is divided into data blocks, the number of data blocks set computing parity data is formed block, the control data block and the check data block is set by turns are stored in the plurality of data blocks stored in the server; caching server, recording the index information stored data block, and stores the data block.

采用本发明技术方案,与现有技术相比,取得了系统数据存储和访问方法上的进步,达到了提高存储效率的效果,节省了大量的存储空间。 With the technical scheme of the present invention, as compared with the prior art, progress has been made on the data storage system and access methods to achieve the effect of improving the storage efficiency and save a lot of storage space. 同时由于数据采用分布式存储,数据文件在多个缓冲服务器间分布,因此非法入侵者(如黑客),即使攻破了某个或某几个服务器,也无法获取完整的文件数据,因此,采取本发明的技术方案提高了数据的安全性;另一方面由于采用了优选的数据冗余校验的方法,即使某一个缓冲服务器出现异常,通过其他的缓冲服务器中存储的文件数据块也可以将受损的数据块恢复出来,从而实现对数据文件的重组和恢复,提高数据分布的可靠性。 At the same time due to the use of distributed data storage, data files across multiple servers distributed buffer, so intruders (such as hacking), even if one or a few break the server can not get the complete file data, therefore, take this aspect of the invention improves the security of data; the other hand, thanks to the preferred methods of data redundancy check, even if one server buffer abnormality occurs, the data block via another buffer may be stored in the server will be subject to loss recovered data blocks, enabling the recovery of recombinant and data files, to improve the reliability of data distribution.

附图说明 BRIEF DESCRIPTION

图1是本发明提供的分布式视频点播系统的一个实施例;图2是图1示出的系统的主索引信息表的数据结构:图3是图1示出的系统的二级索引信息的数据结构;图4是图1示出的系统的对数据文件的存储流程图;图5是图4中分块后校验数据的生成方法流程图;图6是图4中分布存储处理的流程图;图7是图1示出的系统中对数据进行访问的流程图。 FIG 1 is an embodiment of the distributed video on demand system according to the present invention provides; FIG. 2 is a data structure of the main index information table of the system shown in FIG. 1: FIG. 3 is a secondary index system shown in FIG. 1 information data structure; FIG. 4 is a flowchart of the data file stored in the system shown in FIG. 1; FIG. 5 is a block diagram illustrating the method of generating 4 equatorial check data flowchart; FIG. 6 is a flow in the distributed storage processing in FIG. 4 FIG.; FIG. 7 is a flowchart of a data access system shown in FIG. 1.

具体实施方式 Detailed ways

下面将结合附图,举例说明本发明的一个实施的方式. Following with reference, by way of illustration an embodiment of the present invention.

图1是本发明提供的分布式视频点播系统的一个实施例。 FIG 1 is a distributed video on demand system according to the present invention provides an embodiment. 在下面的描述中,假设视频点播系统客户端10通过通信网络20从索引服务器50发出数据请求,数据文件均匀存储在N(N为自然数)个二级服务器60上(本实施例基于两级结构,故将缓冲服务器称为二级服务器)。 In the following description, it is assumed that video on demand client 1020 requests the data from the index server 50 via a communication network, the data file stored in a uniform N (N is a natural number) on the secondary server 60 (in the present embodiment is based on two-stage structure , it will be referred to as secondary servers cache server). 用户的请求首先在索引服务器50处被处理,查找请求的数据在N个二级服务器60中的分布情况,然后由多个二级服务器60将请求的数据通过通信网络加发送到用户端10,用户端10完成数据的重新组合过程。 A user request is first processed at the index server 50 to find the distribution of the requested data in the secondary server 60 N, and a plurality of secondary servers 60 transmits the requested data to the client 10 was added via a communication network, client 10 to complete the re-assembly process data.

索引服务器50保存有多个目录表格。 Index server 50 stores multiple directories form. 对每个数据文件都生成一个记录。 Generates one record for each data file. 每当有数据文件被分布式存储一次,都生成一个对应的主索引信息30,主索引信息是一个由多行数据形成的表格,称为主索引数据表200,主索引数据表如图2所示。 Whenever distributed data files are stored once, it generates a corresponding primary index information 30, the main index table information is formed by a plurality of rows of data, called the primary index data table 200, the master index data in Table 2 in FIG. shows. 对于分布在各个服务器上的数据,在二级服务器上对应有二级索引数据表40。 For the distribution of data on each server, on the secondary server has a corresponding secondary index data table 40. 二级索引数据表如图3所示。 Two index data table as shown in FIG.

主索引数据表200包含的数据记录有:数据文件名,用来表示数据分布情况的分布度;分布存储的第一个二级服务器标识-初始服务器;用于标识一个块组中设定数据块数目的组密度;存储数据的二级服务器名序列。 Master data table 200 contains the index data is recorded: the data file name, used to represent the distribution of the distribution of the data; a first two distribution storage server identification - initial server; means for identifying a set of data blocks in the block group the number density of the group; two name server sequence storage of data. 如果客户端10需要请求某个数据文件,则通过连接索引服务器50获取主索引30,读取属于该请求文件的主索引数据表200,从中获得文件的分布信息、初始二级服务器、组密度和服务器名序列。 If the client needs to request a data file 10, 30 is acquired by connecting the main index index server 50, reads the index data of the main table 200 belonging to the requested document, derive distribution information file, the initial secondary servers, and group density the server name in the series.

获取了主索引信息后,客户端10将与在主索引数据表200中服务器名序列中的二级服务器一一建立连接,利用二级索引数据表210提供的检索信息请求数据,连接从第一个二级服务器一一初始服务器开始,直到最后一份数据块读完,整个过程结束。 After obtaining the primary indexing information, the client 10 establishes a connection with the secondary server eleven primary index table 200 in the data server name in the series, by using two index data retrieval table 210 provides information request data, from a first connection eleven initial secondary server starts the server, a data block until the last read, the process ends.

二级索引数据表210中包含第一个记录是数据文件名,数据文件名是和主索引数据表中的数据文件名字段相关联的,第二个记录是数据块的序号,用来标示数据块在源文件中的位置,第三个记录是前一个服务器,第四个记录是后一个服务器,这两个记录指示了相邻的数据块的存储位置,最后一个记录是数据块的存储地址,存储地址包含了数据块在外存储器70中的存储位置。 Two index data table 210 comprises a first data file name is recorded, the data file name is the main index and data file names in the data table associated with the segments, the second number is the recording data block, the data used to mark position of the block in the source file, and the third server is the previous record, the fourth record is a server, which indicates the storage location of recorded two adjacent data blocks, a record is stored in the last address of the data block memory address contains 70 storage locations in the data block in the external memory.

图4是图1示出的系统的对数据文件的存储流程图。 FIG 4 is a flowchart of the data file stored in the system shown in FIG. 在进行数据文件的存储时,首先执行由索引服务器读取文件进行分析的步骤300,然后执行根据文件分块的参数对文件进行分块的步骤310,在该步骤中,数据块的大小可以具体设定,各数据块之间可以是大小相等,也可以不等,在本实施例中,各数据块的大小是相等的,即采用均匀分块的方式;分块完成后将设定数量的数据块形成数据块组,执行对每个块组计算形成校验数据块的步骤320,然后将校验数据作为块组数据之一放入数据块组中,形成新的数据块组,最后执行将所有的数据块进行分布式轮流存储到指定的N个二级服务器的外存70中的步骤330。 When performing a data file is stored, the first step is performed by the index server 300 reads the file analysis, the file is then performed in block 310 according to the step profile of the block, in this step, the data block size can be specifically setting, among the data blocks may be of equal size, may be unequal, in the present embodiment, the size of each data block are equal, i.e. by way of a uniform block; block after completion of the set number of forming a data block group of data blocks formed in step 320 calculates the parity data blocks for each block group and the parity data block as one group of data into groups of data blocks, forming a new group of data blocks, and finally execute all data blocks stored in the step 330 turns distributed in the external memory 70 designated N secondary servers.

图5是图4中分块后校验数据的生成方法流程图,在已设定好数据块大小和块组中的块数目(即组密度)后,主要步骤如下: FIG 5 is a block diagram carve 4 check data generation method of the flowchart, the number of blocks in the data block size and the block group has been set up (i.e., set density), the main steps are as follows:

1、读取指定数据块数据的步骤400;2、判断是否达到了最后一个数据块的步骤410,如果达到最后一块转到步骤426处理,否则,执行步骤420,即判断数据块组指定的块数目(组密度)是否达到,如果没有达到,则继续读取下一个数据块,执行步骤400,否则表示已经读取了一个数据块组,执行步骤425;3、步骤425,对已读取得数据块组计算生成校验数据块,然后继续执行步骤400;4、步骤426,判断数据块组指定的块数目(组密度)是否达到,若达到,执行步骤440,否则执行步骤430;5、步骤430,对数量不足的数据块部分填充数据,以补充不足的数据块,补充数据块形成块组后,生成最后一个数据块组;6、步骤440,对最后一个数据块组计算生成校验数据块;7、步骤450,将所有的数据块连同校验数据块进行分布存储处理,具体见图6。 1, reads the specified data block in step 400; 2, it is determined whether the last data block in step 410, if it reaches a final process goes to step 426, otherwise, perform step 420, i.e., determines the specified block group of data blocks the number (density group) has reached, if not, the next block of data read continues, step 400, or that has been read a block of data sets, performing step 425; 3, step 425, to read the data acquired generating a checksum calculation block group of data blocks, and then proceed to step 400; 4, step 426, determines the number of blocks of data blocks designated group (group density) has reached, if reached, step 440 is executed, otherwise step 430; 5, step 430, on an insufficient number of data blocks partially filled data to supplement the data block, supplementary data blocks are formed after block group, the last group of data blocks generated; 6, step 440, the last set of computing a block of data to generate the parity data block; 7, step 450, all the data blocks together with the parity data blocks distributed storage processing, specifically shown in Figure 6.

在本实施例中,校验数据块的计算生成方法很多,这里举出几个具体的例子,假定组密度200d=M(M为自然数),相应地各个块组为D 1,Dz,...DM,待生成的校验数据为P,可以采取如下几种校验方法:(1)奇偶校验方法:又称为RAID方法,方法是,采用按位进行二进制加法的方法,按照这种方法,P=D1田DZ.....DM.或P=一(D10DZ,.....巩),其中“.”表示“异或”运算,“一”表示“非”运算。 In the present embodiment, the calculation method of generating parity data block number, a few specific examples are given here, assuming the density of group 200d = M (M is a natural number), each block corresponding group D 1, Dz, .. .dm, check data to be generated is P, can take the following several verification methods: (1) parity method: also known as RAID method, the method is a method for adding a binary bit, in this The method, P = Dl fields DZ ..... DM. or P = a (D10DZ, ..... Gong), where "." means "exclusive or" operation, "a" indicates "not" operation. 假设块组中的巩发生错误,那么相应地,对应该校验的数据恢复方法是:几=D1.几....环一P.Dk+1...田DM.或Dk二·(DI.DZ。...Dk-loP田Dk+1...。DM). Gongfa Sheng error assumed that the block groups, so accordingly, should verify data recovery method is: a few hundreds = D1 .... a ring P.Dk + 1 ... DM field or two-Dk ( DI.DZ .... Dk-loP field Dk + 1 .... DM).

(2)模运算方法:方法是,将块组看成二进制数,假设每个块的数二进制位数为K,将所有块组数相加,并对ZAK取模,即:P==(D1+DZ+...+nM)mod ZAK。 (2) molding operation method: method, the block group as binary numbers, assuming that the number of binary bits for each block K, adding the number of all the block groups, and modulo ZAK, namely: P == ( D1 + DZ + ... + nM) mod ZAK. 假设块组中的Dk发生错误,那么对应该校验的数据恢复方法是:Dk=P+2叹一(DI+DZ+...+Dk一1+D卜1...+DM)modZAK。 Suppose block group Dk error occurs, then the data recovery method to be verified is: Dk = P + 2 a sigh (DI + DZ + ... + Dk Bu a 1 + D 1 ... + DM) modZAK.

(3)镜像方法:该方法是对每个块生成一个相同的数据拷贝块,相当于在系统中生成两个相同的分布式拷贝。 (3) mirroring method: This method is to generate a copy of the same data block for each block, to generate the equivalent of two copies of the same distributed system.

方法还有很多,这里不再逐一举例,所有的方法都必须满足以下条件:一般地,校验数据P的生成方法是:对数据D1,DZ,...,DM·对应校验生成函数f(·),满足P二f(D,,DZ,...,DM),相应地,对于任意数据认,对应一个校验数据恢复函数g(·),满足Dk=g(D,,DZ,一Dk-1,P,D卜卜一DM),函数f(·)和g(·)可以相同,也可以不同。 There are many methods, there is no longer one by way of example, all methods must meet the following criteria: In general, the method of generating parity data P that is: the data D1, DZ, ..., DM · the corresponding parity generation function f (·), satisfies P = f (D ,, DZ, ..., DM), respectively, recognized for any data, check data corresponding to a recovery function G (·), satisfies Dk = g (D ,, DZ a Dk-1, P, D a pitapat the DM), the function f (·) and g (·) may be the same or different.

图6是图4中分布存储处理的流程图,包括以下步骤:1.步骤500,设定分布参数,即设定参与分布存储的服务器数目及所有的服务器名称,生成主索引数据表200;2.步骤510,设定起始服务器,即确定第一个开始存储的服务器;3.步骤520,读取数据块组;4.步骤530,分布式存储数据块组及生成二级索引,即从起始服务器开始,存储第一个数据块,在该服务器上生成该块的二级索引数据表信息210;然后顺次连接下一个服务器,存储第二个数据块,在该服务器上生成该块的二级索引数据表信息210,直到该块组的所有数据块分步存储到服务器上为止(包括对应的校验数据块);5.步骤540,判断该块组是否是最后一个块组,如果不是,则重新设定起始服务器,执行步骤510,如果是,那么表示所有的数据块组已经分布存储完毕,分布式存储结束;6、结束步骤550。 6 is a flowchart of distributed storage processing in FIG. 4, comprising the following steps: a step 500, the distribution parameter is set, i.e., a set number of servers and the server names all the participating distributed memory, the data table 200 to generate the main index; 2. in step 510, the server initial setting, i.e., determining a first start storing server;. 3 step 520, sets read data block;. 4, step 530, the distributed storage of data blocks and generating a set of secondary indexes, i.e. from origin server starts storing the first data block, the block that generates the secondary index information 210 in the data table on the server; then the next server, storing a second data blocks are sequentially connected, the block generated on the server the two data index table information 210 until all the data blocks of the block group stored until step (including the corresponding parity data blocks) on the server; 5. step 540 determines whether the block is the last block group set, If not, then the origin server is reset, step 510, and if so, all the data blocks showing distribution groups have been stored, the end of the distributed storage; 6, step 550 ends. 为了进一步提高数据的可靠性和灵活性,校验数据可以在块组中按照一定的规则指定位置,如在固定位置或动态位置,其中动态位置存储的可采用动态循环方法,如将校验数据在第一数据块组中放在第一块,第二数据块组中放在第二块,,......当校验数据块所在位置达到组密度时,又从第一块开始循环,可在实施过程中进行选择。 To further increase the reliability and flexibility, the check data can be specified location according to certain rules, such as in a fixed position or moving position, wherein the method of dynamic cycle can be stored in the dynamic position, as will be checking data in blocks in the first block in the first data block set, the second data block in the second block group ,, ...... when check data block reaches the location group density, and from a second start cycles, can be selected during implementation.

图7是图1示出的系统中对数据进行访问的流程图,主要包括如下步骤:1.步骤610,客户端10通过通信网络20与索引服务器50进行连接,请求某个数据文件;2.步骤620,连接建立后,索引服务器通过请求查找主索引信息,获取主索引30,获得标识该请求数据文件索引数据表200,从200中提取该文件的分布信息,包含文件名,分布度,初始服务器,组密度,服务器名序列等信息;3.步骤630,连接起始服务器;4.步骤640,调用读取二级索引信息过程,获取二级索引40,取得属于该请求文件的二级索引数据表210,从210中获得该数据块在服务器中的对应文件名、数据块号、上一个服务器、下一个服务器、存储位置等信息;5.步骤650,在外存70中读取数据块;6.步骤660,判断读取的数据块数是否达到为该文件指定组密度值:(1)如果达到组密度值,那么执行步骤680,即去校 FIG 7 is a flowchart of a data access system shown in FIG. 1, including the following steps: a step 610, the client 10 via the communication network 20 is connected with the index server 50, requests a data file; 2. step 620, after the connection is established, the index server 200, extracted by a request to find the primary index information acquiring primary index 30, obtain the identification of the requested data file index data in the table 200 distribution information of the file, including the file name, the distribution of initial the information server, the density of the group, the server name in the series and the like;. 3 step 630, the originating server is connected;. 4, step 640, the process calls to read two indexing information to acquire the secondary index 40, belonging to the request to obtain the secondary index file data table 210, 210 is obtained from the data block corresponding to the file server name, the data block number, a server, a server at the storage location; 5 step 650, the read data block in the external memory 70; 6. step 660, the number of data blocks is determined whether the read density value set for the file specified: (1) if the density reaches the value set, then step 680, i.e., go to the school 和数据块的重新组合过程,去校验重组过程将原来进行数据可靠性保证的冗余校验数据去掉。 Redundancy check data and re-assembly process the data block, the procedure to verify recombination original data to ensure the reliability removed. 继续执行步骤690;(2)如果没有达到组密度值,执行步骤670,即连接下一个服务器,然后继续执行步骤640;7.步骤690,判断数据块号,察看是否达到最后一个数据块:(1)若没有达到,执行步骤670,即连接下一个服务器,继续执行步骤640;(2)若已经达到最后一个数据块,那么整个数据检索访问过程结束。 Proceed to step 690; (2) If the density does not reach the set value, the step 670, i.e., a server connected to the next, and then proceed to step 640;. 7 step 690, judgment data block number, to see whether the last data block :( 1) If not reached, step 670, i.e. the connection to the next server, proceed to step 640; (2) if the last data block has been reached, then the entire data retrieval access procedure ends.

尽管参照实施例对所公开的涉及分布式视频点播系统及其实现数据存储和访问的方法进行了特别描述,本领域技术人员将能理解,在不偏离本发明的范围和精神的情况下,可以对它进行形式和细节的种种显而易见的修改,如可自选数据校验方法和数据分布密度,同时可以将实施例中的服务器分布的二级方式改为3级或多级方式,也可改为单级,索引信息表也可自行设计,增加新字段等。 While the method of Reference Example relates to distributed VOD system and implementing the data storage and access have been disclosed specifically described, those skilled in the art will appreciate, without departing from the scope and spirit of the present invention, can be form and details of its various obvious modifications, can be managed as data verification method and a data distribution density, while two embodiment examples of the embodiment may be distributed servers or multi-stage manner to level 3, can be changed to single stage, the index information table may also be designed to increase the new field and the like. 因此,以上描述的实施例是说明性的而不是限制性的,在不脱离本发明的精神和范围的情况下,所有的变化和修改都在本发明的范围之内。 Thus, embodiments described above are illustrative and not limiting, without departing from the spirit and scope of the present invention, all the variations and modifications are within the scope of the present invention.

Claims (10)

1.一种分布式视频点播系统中实现数据存储的方法,其特征在于数据存储的方法包括以下步骤:A.对数据文件按设定的方式进行数据分块;B.对设定数目的数据块计算形成校验数据块;C.将所述数据块和校验数据块按设定的方式轮流存储到设定的多个缓冲服务器中;D.在所述每一个缓冲服务器中生成相应的数据块存储的索引信息。 A method in a distributed video on demand system for data storage, wherein the method of storing data comprising the steps of: A data file for the data block in the manner set; B data set number. computing parity data blocks forming the block;. C to the data block and the check data block is set by a plurality of turns stored in the cache server settings;. D to form the corresponding buffer in each of said server index information stored data block.
2.一种分布式视频点播系统中实现数据访问的方法,其特征在于数据访问的方法包括以下步骤:a.客户端与服务器建立连接;b.客户端从索引服务器中读取要访问的数据文件对应的索引信息;c.按照索引信息在相应的多个缓冲服务器中读取数据块;d.对已读取达到组密度的数据块组进行去校验;e.客户端对读取的数据块进行重组,恢复数据文件。 2. A method in a distributed video on demand system for data access, wherein the data access method comprising the steps of: a client establishes a connection with the server; B client reads the index data to be accessed from the server. index information corresponding to the file;. c read in accordance with the index information corresponding to the plurality of data blocks in cache server; group D have been reached the read density data blocks group check to;.. e client to read restructuring the data block, the data files to restore.
3.根据权利要求1所述的存储方法,其特征在于对数据文件进行均匀的数据分块。 3. The storage method according to claim 1, wherein the data file uniform data block.
4.根据权利要求1所述的存储方法,其特征在于所述每个数据块组中,任一数据块可以通过校验数据块和其他数据块恢复。 The storage method according to claim 1, wherein each said group of data blocks, any of the data blocks may be recovered by the parity data blocks and other data blocks.
5.根据权利要求1所述的存储方法,其特征在于其中对设定数目的数据块计算形成校验数据块的步骤中,还包括以下步骤:B.1.判断包含文件最后一个数据块的数据组的数量,是否达到设定的数目,若达到,计算形成校验数据块;B.2.若没有达到,填充数量不足的数据块,然后计算形成校验数据块。 The storage method according to claim 1, characterized in that the number of data blocks set calculation step of forming parity data block, further comprising the step of:. B.1 comprises determining a last data block of the file number of data sets, whether the set number, if reached, calculates parity data blocks is formed;. B.2 if not reach, insufficient amount of padding data blocks, and calculates parity data blocks is formed.
6.根据权利要求1所述的存储方法,其特征在于其中将所述数据块按设定的方式轮流存储到多个服务器中步骤中,还包括以下步骤:C.1.设定分布存储缓冲服务器的数量和名称;C.2.确定初始存储缓冲服务器。 The storage method according to claim 1, characterized in that wherein said data blocks is set in turn to a plurality of servers storing step further comprises the step of:. C.1 set distribution storage buffer the number and name of the server;. C.2 determining an initial buffer storage server.
7.根据权利要求1所述的存储方法,其特征在于数据块组中的校验数据块存储位置是动态的。 The storage method according to claim 1, characterized in that the parity data blocks are stored in a data block group is dynamic.
8.根据权利要求1所述的存储方法,其特征在于其中在所述每一个缓冲服务器中生成相应的数据存储的索引信息的步骤中,索引信息包括以下内容:所存储的数据块对应的文件名;所存储的数据块序号;所存储数据块的相邻数据块存储的服务器名称;所存储的数据块在服务器中的存储地址。 The storage method according to claim 1, characterized in that wherein the step of generating the index information corresponding to data stored in each buffer of the server, the index information including the following: the data block stored in the corresponding name; product of the stored block number; the server name of the neighboring data blocks of the stored data block stored; data block stored in the server storage address.
9.根据权利要求2所述的数据访问方法,其特征在于其中按照索引信息在相应的多个服务器中读取数据块的步骤中还包括以下步骤:c.1.连接起始服务器;c.2.按索引信息读取文件对应的数据块;c.3.判断所读取的数据块数量是否等于组密度,若否,执行下一步,若是,执行客户端对读取的数据块进行重组,还原数据文件的步骤;c.4.连接下一个服务器,继续读取数据块,执行步骤c.2。 A data access method according to claim 2, characterized in that the step of indexing information in accordance with the read data block corresponding to the plurality of servers further comprises the steps of: c.1 connected to the originating server; c.. 2. press the index information file corresponding to the read data blocks;. c.3 number of data blocks is determined whether the read is equal to the density of the group, if not, the next step, if so, the client reads the data block to restructure step to restore the data file;. c.4 the connected server, continues to read the data block, step c.2.
10.分布式视频点播系统,包括若干个视频客户端,若干个存贮视频数据文件的缓冲服务器和索引服务器,所述客户端和服务器通过通信网络进行连接和通讯,其特征在于:客户端向索引服务器发出文件请求,对从缓冲服务器读取的数据块重组;索引服务器,控制数据文件按设定方式被划分成数据块,对设定数目的数据块计算形成校验数据块,控制所述数据块和校验数据块按设定的方式分别轮流存储到多个数据块存储服务器;缓冲服务器,记录数据块存储的索引信息,并存储数据块。 10. The distributed video-on-demand system, comprising a plurality of video client, a plurality of video data files stored in the cache server and the index server, the client and server to connect and communicate through a communication network, wherein: the client index file server sends a request to the data block read from the cache server recombinant; index server, the data file according to the set control mode is divided into data blocks, the number of data blocks set computing parity data blocks is formed, the control data blocks and parity data blocks, respectively, is set alternately to the plurality of data blocks stored in the storage server; caching server, recording the index information stored data block, and stores the data block.
CN 02136350 2002-07-30 2002-07-30 Distributive video order program system and its data recording and accessing method CN1258921C (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN 02136350 CN1258921C (en) 2002-07-30 2002-07-30 Distributive video order program system and its data recording and accessing method

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN 02136350 CN1258921C (en) 2002-07-30 2002-07-30 Distributive video order program system and its data recording and accessing method

Publications (2)

Publication Number Publication Date
CN1472963A CN1472963A (en) 2004-02-04
CN1258921C true CN1258921C (en) 2006-06-07

Family

ID=34146420

Family Applications (1)

Application Number Title Priority Date Filing Date
CN 02136350 CN1258921C (en) 2002-07-30 2002-07-30 Distributive video order program system and its data recording and accessing method

Country Status (1)

Country Link
CN (1) CN1258921C (en)

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2011131080A1 (en) * 2010-04-20 2011-10-27 杭州和利时自动化有限公司 Historical data processing method and device for distributed control system
CN104969192A (en) * 2013-02-27 2015-10-07 惠普发展公司,有限责任合伙企业 Selecting a backup type based on changed data

Families Citing this family (29)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2006126894A (en) 2004-10-26 2006-05-18 Sony Corp Content delivery method, program and information processor
CN100459701C (en) 2004-11-16 2009-02-04 南京大学 Self-adapting admission controlling method based on measurement for video frequency request system
CN1328879C (en) * 2005-01-26 2007-07-25 北京大学 Online backup method based on data block redundancy and virtualized storage
CN101000610B (en) 2006-01-11 2010-09-29 鸿富锦精密工业(深圳)有限公司;鸿海精密工业股份有限公司 Scatter storage system and method for file
CN100579016C (en) 2006-01-24 2010-01-06 华为技术有限公司 Distributing storage downloading system, device and method for network data
CN100477622C (en) 2006-06-28 2009-04-08 华中科技大学 Circular node organizing method in reciprocal network request broadcast system
CN100438625C (en) 2006-07-19 2008-11-26 杭州华三通信技术有限公司 Video monitoring data storage managing method and system
CN101155296B (en) 2006-09-29 2011-05-25 中国科学技术大学 Method for transmitting data
CN101193273B (en) 2006-11-20 2010-07-14 中兴通讯股份有限公司 A storage and playing method for real time multimedia image information
CN100556114C (en) 2006-11-23 2009-10-28 中兴通讯股份有限公司 Media slicing memory method
CN101146110B (en) 2007-09-25 2011-06-29 深圳市迅雷网络技术有限公司 A method for playing stream media
CN100553328C (en) 2007-10-10 2009-10-21 张福泉 Stream media VoD system for intelligent access data
CN101141627A (en) * 2007-10-23 2008-03-12 深圳市迅雷网络技术有限公司 Storage system and method of stream media file
CN101472166B (en) * 2007-12-26 2011-11-16 华为技术有限公司 Method for caching and enquiring content as well as point-to-point medium transmission system
CN102387179B (en) * 2010-09-02 2016-08-10 联想(北京)有限公司 Distributed file system and node, storage method and storage controlling method
CN102385606B (en) * 2010-09-03 2016-08-03 腾讯科技(深圳)有限公司 The access method of a kind of Distributed Data Warehouse and device
CN101997929A (en) * 2010-11-29 2011-03-30 北京卓微天成科技咨询有限公司 Data access method, device and system for cloud storage
CN102055797A (en) * 2010-11-29 2011-05-11 北京卓微天成科技咨询有限公司 Method, device and system for accessing cloud storage data
CN102595085B (en) * 2011-12-31 2015-05-20 杭州华三通信技术有限公司 Data storage management method and equipment thereof
CN103200204A (en) * 2012-01-04 2013-07-10 中兴通讯股份有限公司 Distributed transcoding system and method based on cloud storage
US8862847B2 (en) 2013-02-08 2014-10-14 Huawei Technologies Co., Ltd. Distributed storage method, apparatus, and system for reducing a data loss that may result from a single-point failure
CN103237063B (en) * 2013-04-11 2016-12-28 安科智慧城市技术(中国)有限公司 A kind of cloud storage method and cloud thereof control server, cloud storage server and system
CN103535014B (en) * 2013-04-23 2016-09-28 华为技术有限公司 A kind of network store system, data processing method and client
CN104216664B (en) * 2013-06-26 2017-06-09 华为技术有限公司 Network volume creation method, date storage method, storage device and storage system
CN103425785A (en) * 2013-08-22 2013-12-04 新浪网技术(中国)有限公司 Data storage system and user data storage and reading method thereof
US10241689B1 (en) 2015-06-23 2019-03-26 Amazon Technologies, Inc. Surface-based logical storage units in multi-platter disks
CN104980680B (en) * 2015-06-30 2017-11-03 银江股份有限公司 A kind of method of video data access
CN104954388B (en) * 2015-07-14 2018-03-20 电子科技大学 Towards the distributed more internal memory copy storage systems and method of video stream data
CN105760112B (en) * 2016-02-01 2017-03-15 惠州市蓝微新源技术有限公司 Extend date storage method, read method and the system of memory use time

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2011131080A1 (en) * 2010-04-20 2011-10-27 杭州和利时自动化有限公司 Historical data processing method and device for distributed control system
CN104969192A (en) * 2013-02-27 2015-10-07 惠普发展公司,有限责任合伙企业 Selecting a backup type based on changed data

Also Published As

Publication number Publication date
CN1472963A (en) 2004-02-04

Similar Documents

Publication Publication Date Title
US10127402B2 (en) Systematic erasure code encoding of data packages
US9940197B2 (en) Method and apparatus for slice partial rebuilding in a dispersed storage network
US8914667B2 (en) Identifying a slice error in a dispersed storage network
US9501358B2 (en) Adjusting a dispersal parameter of dispersedly stored data
US8856549B2 (en) Deleting encoded data slices in a dispersed storage network
Xiang et al. Optimal recovery of single disk failure in RDP code storage systems
US8918534B2 (en) Writing data slices to ready and non-ready distributed storage units in a distributed storage network
US9785503B2 (en) Method and apparatus for distributed storage integrity processing
US8732206B2 (en) Distributed storage timestamped revisions
US8725940B2 (en) Distributedly storing raid data in a raid memory and a dispersed storage network memory
US9927977B2 (en) Retrieving data segments from a dispersed storage network
US9195392B2 (en) Distributed storage method, apparatus, and system
US9304858B2 (en) Analyzing found data in a distributed storage and task network
JP5167243B2 (en) Storage Allocation and Erasure Coding Techniques for Scalable and Fault Tolerant Storage Systems
US10042703B2 (en) Encrypting data for storage in a dispersed storage network
US8954667B2 (en) Data migration in a dispersed storage network
US9658911B2 (en) Selecting a directory of a dispersed storage network
US8489915B2 (en) Method and apparatus for storage integrity processing based on error types in a dispersed storage network
US8352501B2 (en) Dispersed storage network utilizing revision snapshots
US9135115B2 (en) Storing data in multiple formats including a dispersed storage format
US8874990B2 (en) Pre-fetching data segments stored in a dispersed storage network
CN1201336C (en) Method of encoding data signals for storage
US20130198226A1 (en) Retrieving data utilizing a distributed index
US10445164B2 (en) Copying data in a dispersed storage network without replication
US20140304527A1 (en) Efficient memory utilization in a dispersed storage system

Legal Events

Date Code Title Description
C06 Publication
C10 Entry into substantive examination
C14 Grant of patent or utility model
EXPY Termination of patent right or utility model