CN100525435C - A data file storage method - Google Patents

Abstract

This invention discloses a data file storage method and a data request information process method based on the data/video service system. The storage method includes: said storage controller computes the storage position relating to each storage space based on the number of the output server and stored times of each data file, each storage position corresponds to one or more than one output servers. Said server stores the data files in the database in the related output servers by computation, said data request information method sends the data files required by the data required information to the data applier, which equalizes the storage on the output server.

Description

A kind of storage means of data file

Technical field

The present invention relates to a kind of storage means of data file, especially a kind of based on equalization data/video system, be used for the storage means of equally loaded data on flows file.

Background technology

Along with development of internet technology, the user increases day by day to the demand of video/data program request, addresses that need, and the data transmission scheme of point (video/data transmit leg) to multiple spot (a plurality of user) occurred.At present, great majority provide the system of video/data service, when handling the lot of data application, and general because hard disk bottleneck, bus bottleneck and network bottleneck cause the fan-out capability deficiency, can not support program request rate very big the time.Generally provide the system of video/data service to comprise storage control server and export server etc., described storage control server is responsible for distributing, the storage and the output of management service content, the message synchronization of related business process; Described export server is responsible for the balance of output queue, and stackable broadband exchange files interface is provided, and portion stores relevant business tine within it, and reliable data file management and protection are provided.

Generally, the request of data quantity of the system of video/data service may be bigger, since current mechanism be on the export server with multimachine between the storage identical file guarantee flow, the perhaps fixing memory location of program on export server, cause the load of balance export server well like this, cause part export server load capacity excessive, even operate at full capacity, and part export server resource is not fully utilized, even does not work.And,,, can't normally be transmitted away if the unusual cisco unity malfunction of certain station server wherein may cause asking the data file on this server to be lost because the file content in the current mechanism is to be stored in respectively on the different servers.

Summary of the invention

The objective of the invention is at the deficiencies in the prior art, a kind of storage means of the data file based on data/video service system is proposed, with being stored on the export server of data file equilibrium, be used for the flow of equalization data/Video service load by this method.

For achieving the above object, the invention provides a kind of storage means of the data file based on data/video service system, described system comprises storage control server and connected one or more export server and a material machine, described export server has unique sign, and this storage means may further comprise the steps:

Step 1, described storage control server are according to the number n of export server and the number of times k of each data file storage, and the memory location of calculating each memory space correspondence, described memory location are one or more export server;

Step 2, described storage control server store the data file in the database in the export server of the correspondence that calculates successively; And at stored file information generation memory location table.

Wherein, the memory location of each memory space correspondence described in the step 1 obtains by following algorithm: (wherein n is the CS0 number, and k is the storage number of times)

When memory space=1, to CS0 that should memory space be numbered 1,2...k;

When memory space=2, to CS0 that should memory space be numbered 2,3 ... k+1;

And the like,

When memory space=n-k+1, CS0 that should memory space is numbered (n-k+1), (n-k+2) ... n;

When memory space=n-k+2, CS0 that should memory space is numbered (n-k+2), (n-k+3) ... n, 1;

When memory space=n-k+3, the CS0 that correspondence should memory space is numbered (n-k+3), (n-k+4) ... n, 1,2;

Storage control server specifically may further comprise the steps the export server that the data file in the material machine stores the correspondence that calculates successively into described in the described step 2:

Step 2a, described storage control server are inquired about untapped memory space, obtain corresponding memory location, and obtain the filename of data file from database, and generate storing message, described storing message is sent to the export server of described memory location correspondence;

Step 2b, described export server are inquired about this data file and are stored this locality into according to the filename in the message from the material machine, return storing message to described storage control server;

Step 2c, described storage control server write down the user mode of this memory space and the data file information of having stored according to this storing message that returns, and described data file information of having stored comprises the Data Filename and the memory location thereof of having stored;

Whether all the storage of data file in step 2d, the described storage control server judgment data storehouse finishes, as does not have, and returns step 2a, finishes as whole storages, then finishes.

The present invention has set up in one sets of data/Video service the efficient response mechanism for big data on flows application.By above-mentioned storage means with the data file storage that interweaves, make relatively balance of the storage of data file on export server, guarantee smooth upgrade, and, because a plurality of copies of same data file are placed on many export servers, like this when the same data file of a plurality of user applies, can be from many machine outputs, data parallel output with user applies has improved the data fan-out capability, and, when the unusual cisco unity malfunction of a station server wherein, can not cause asking the data file on this export server to be lost,, still can normally be transmitted away because this document also has storage on export server.

The present invention is described in further detail below in conjunction with drawings and Examples.

Description of drawings

Fig. 1 is the structural representation of system of the present invention;

Fig. 2 is the memory allocation schematic diagram of data file;

Fig. 3 is the flow chart of the processing method of request message of the present invention;

Fig. 4 is the schematic diagram that concerns of total formation among the present invention and subqueue.

Embodiment

Referring to Fig. 1, structural representation for system of the present invention, a kind of system that is used for equalization data/Video service load flow comprises a storage control server (CSC) 1 and one or more export server (CS0) 21,22......2n, described CSC1 is connected with CS021, CS022......CS02n respectively, CSC1 sends to the CS0 that stores the corresponding data file with the data request information that receives, and CS0 exports corresponding data file after receiving this message.Described CSC1 is used for program request is integrated distribution; Described CS0 store with described storage allocation table in the identical data file in memory location, the response program request with the request data file send to data application side.

In order to realize the purpose of load an ancient unit of weight weighing apparatus, taked following technological means: at first, by storage means data file is stored in different CS0 and goes up (storage policy promptly interweaves), and generate the memory location table, by such data store organisation, make data file when storage, reach certain balance, then, when the request message of response data file, when output, can select the lighter CS0 output of corresponding load according to the loading condition of CS0.

So, the invention provides a kind of storage means of data file, at first, CSC is according to the number n of CS0 and the number of times k of needs storage, the memory location of calculating each memory space correspondence; And then the data file in the database stored in the memory location that calculates successively.

Concrete, be convenient management, CSC is at the number of times k that stores according to number n and the needs of CS0, when calculating the memory location of each memory space correspondence, generate a storage allocation table (SAT), the content of table comprises memory space numbering (StorageSpaceID), memory location (StoragePosition) and spatiality (Status), the corresponding StorageSpaceID of the data file of each storage, the StoragePosition that each StorageSpaceID is corresponding one, described memory location is each CS0 numbering of storing data files, the number of this CS0 numbering is the number of times of storage, and Status represents whether this locus is occupied.Specific embodiment is as follows: (wherein, the number n of CS0=4, the number of times k=2 of storage), this moment, the structure of described SAT table was as shown in table 1.

Table 1

StorageSpaceID	StoragePosition	Status
			1	CS01，CS02	Use
2	CS02，CS03	Use
			3	CS03，CS04	Use
4	CS04，CS01	Use
			5	CS01，CS02	Use
6	CS02，CS03	Do not use
			......	......	......

Wherein, the memory location of each memory space correspondence is that the CS0 numbering obtains by following algorithm: (wherein n is the CS0 number, and k is the storage number of times)

For (inti=0; I＜=allocation of space maximum; I++)

{ interim storage numbering=0

Initial storage numbering=i%n (symbol " % " be modulo operation)

for(intj＝1；j<k；j++)

{ current storage numbering=initial storage numbering+j;

If (current storage numbering〉n)

{ current storage numbering=current storage numbering-n}

Interim storage numbering=interim storage numbering+current storage numbering

}

Final storage numbering when the interim storage of record is numbered the space and is numbered i, i.e. export server numbering

}

According to above-mentioned algorithm, with n=4, k=2 is an example,

When StorageSpaceID=1, to CS0 that should memory space be numbered 1,2...k;

When StorageSpaceID=2, to CS0 that should memory space be numbered 2,3 ... k+1;

And the like,

When StorageSpaceID=n-k+1, CS0 that should memory space is numbered (n-k+1), (n-k+2) ... n;

When StorageSpaceID=n-k+2, CS0 that should memory space is numbered (n-k+2), (n-k+3) ... n, 1;

When StorageSpaceID=n-k+3, the CS0 that correspondence should memory space is numbered (n-k+3), (n-k+4) ... n, 1,2;

Shown in as above showing,

When as StorageSpaceID=1, CS0 that should memory space is numbered 1,2;

When StorageSpaceID=2, CS0 that should memory space is numbered 2,3;

When StorageSpaceID=3, CS0 that should memory space is numbered 3,4;

When StorageSpaceID=4, CS0 that should memory space is numbered 4,1;

And the like.

According to above-mentioned storage policy, characteristics have: it is unusual to guarantee that k-1 (k is the storage number of times) platform CS0 occurs, and in the time of can't normally moving, the operational process of whole system can not received influence; The load of farthest level and smooth CS0 avoid occurring separate unit CS0 overload operation, and other CS0 is no-load runs; Make things convenient for the dilatation of system, when promptly increasing the number of CS0 newly or changing the program storage number of times, do not need to revise original program, only need reconfigure system operational parameters, the initialization program gets final product.

When storage file promptly generates the SAT table, also write down the Data Filename and the memory location thereof of having stored, generate other memory location table (SPT) this moment, when carrying out storing data files according to above-mentioned storage policy, CS0 storage back is returned storage to CSC and is finished response message, the storage of CSC response CS0 is receiveed the response, the content of receiveing the response according to storage generates memory location table (SPT), come memory location (promptly storing the CS0 numbering of this data file) and other memory states of recording data files, the SPT table is interrelated by identical StorageSpaceID item with the SAT table.Corresponding with table 1, described SPT shows as shown in Figure 2, because the number of times of the storage that interweaves is 2, and each file storage 2 times, the CS0 of its storage is as shown in Figure 2.

After the weighing apparatus data/video service system according to above-mentioned storage means storing data files received one or more data request information, the method with the data file of this request message requests sends as shown in Figure 3, may further comprise the steps:

Step 1, described storage control server determine that according to memory location table content export server also is encapsulated into this export server information in the data request information according to the data request information that receives;

Data request information after step 2, described storage control server will encapsulate be added into the total formation of request of data and with the corresponding subqueue of described export server in;

Step 3, described storage control server send thread by request of data and extract data request information from described subqueues, and send to corresponding export server;

Step 4, described export server receive this data request information, and the data file of asking in this data request information is sent to data application side, and receive the response to the storage control server return data;

Step 5, described storage control server respond the data response message of described export server, revise total formation of request of data and corresponding subqueue.

Particularly, number with CS0 is 4, the storage number of times is that the system of 2 o'clock storage policy storing data files is an example, for responding each order request message, CSC can set up corresponding message queue, these message are managed, forwards is all finished by corresponding threading operation to CS0, only after CS0 returns response message, just formation is made amendment, promptly corresponding request message is deleted from formation, such operation can guarantee the reliability of message, avoids message to lose in the CSC system.

The quantity of possible unit interval request message is very many, pressure to CS is big, the method that CSC adopts classification to found the team and be listed as to such request message, i.e. a total formation and a plurality of subqueue, all request messages have been preserved in total formation, the CS0 of the corresponding response of each subqueue.The characteristics of graded queues are: data request information is assigned to corresponding subqueue according to the loading condition of each CS0, can guarantees the equilibrium of data distribution, avoid single CS0 load excessive; The parallel distribution of a plurality of formations can improve the efficient of message distribution; Subqueue can be according to the load of corresponding CS0, and control is distributed to the message number of this CS0, reduces the load pressure of CS0; Total formation has comprised the request message of all subqueues, even certain CS0 system can't operate as normal like this, can guarantee that also this CS0 correspondence subqueue message do not lose.Total formation among the present invention and subqueue concern schematic diagram as shown in Figure 4, in Fig. 4, receive 12 request messages with CSC, having 4 CS0 is example, wherein, and the corresponding CS021 of subqueue A1, the corresponding CS022 of subqueue A2, the corresponding CS023 of subqueue A3, the corresponding CS024 of subqueue A4.3 data request messages are arranged in each subqueue.

Receive the request message of data file as CSC1 after, determine to be specially the process of export server according to the data request information that receives:

At first, described CSC1 inquiry SPT table, the numbering that obtains the Data Filename corresponding and store all CS0 of this data file with the data request information that receives; For example, referring to Fig. 2, be file 1 as this data file, as can be known, store this data file all CS0 be numbered CS021, CS022,

Then, the number of the data file request message of described CSC1 inquiry and the corresponding subqueue 1,2 of more described export server CS021, CS022, if the present request message number of subqueue 1 is few, then CS021 is defined as exporting the export server of the data file of this data request information request.

After the export server numbering of the filename that obtains this data file 1 and this data file of output, CSC with these two information integrated be new request message, this request message is added in into total formation and the subqueue 1;

Described CSC1 sends thread by request of data and extract data request information from described subqueue 1, and sends to corresponding export server CS021;

Described CS021 receives this data request information, and the data file of asking in this data request information is sent to data application side, and receives the response to the CSC1 return data;

Described CSC1 responds the data response message of described CS021, revises total formation of request of data and corresponding subqueue 1, with this request message of handling deletion.

In addition, when to the request of a data file more for a long time, in order to satisfy the demand of data application side, can export by many export servers, at this moment, CSC is in the load number of CS0 relatively, when promptly it is to the queuing situation of just subqueue, selects the lighter one or more export server output of load.

When one of the output interface of CS0 is above, CS0 is in output data file, also need to select suitable output interface, for example, when CS021 has 3 output interfaces, after described CS021 receives this data request information, retrieve the data file number that the output interface of its storage inside need be exported, data file number according to this output interface needs output is determined its output pressure, and the data file number that needs are exported is the output interface that the light one or more output interfaces of output pressure are defined as exporting this data request information designated data files less.

It should be noted last that, above embodiment is only unrestricted in order to technical scheme of the present invention to be described, although the present invention is had been described in detail with reference to preferred embodiment, those of ordinary skill in the art is to be understood that, can make amendment or be equal to replacement technical scheme of the present invention, and not breaking away from the spirit and scope of technical solution of the present invention, it all should be encompassed in the middle of the claim scope of the present invention.

Claims (4)

1, a kind of storage means of the data file based on data/video service system, described system comprises storage control server, connected one or more export server, and material machine, described export server has unique sign, it is characterized in that this method may further comprise the steps:

Step 1, described storage control server are according to the number n of export server and the number of times k of each data file storage, calculate the memory location of each memory space correspondence, the described export server of described memory location correspondence and described storage number of times respective amount;

Step 2, described storage control server store the data file in the material machine in the export server of the correspondence that calculates successively; And at stored file information generation memory location table.

2, the storage means of the data file based on data/video service system according to claim 1 is characterized in that, the memory location of each memory space correspondence obtains by following algorithm in the described step 1:

For (int i=0; I＜=allocation of space maximum; I++)

{ interim storage numbering=0

Initial storage numbering=i%n (symbol " % " be modulo operation)

for(int?j＝1；j<k；j++)

{ current storage numbering=initial storage numbering+j;

If (current storage numbering〉n)

{ current storage numbering=current storage numbering-n}

Interim storage numbering=interim storage numbering+current storage numbering

}

Final storage numbering when the interim storage of record is numbered the space and is numbered i, i.e. output service

The device numbering

}。

3, the storage means of the data file based on data/video service system according to claim 1, it is characterized in that storage control server specifically may further comprise the steps the export server that the data file in the material machine stores the correspondence that calculates successively into described in the described step 2:

Step 2a, described storage control server are inquired about untapped memory space, obtain corresponding memory location, and obtain the filename of data file from database, and generate storing message, described storing message is sent to the export server of described memory location correspondence;

Step 2b, described export server are inquired about this data file and are stored this locality into according to the filename in the message from the material machine, return storing message to described storage control server;

Step 2c, described storage control server write down the user mode of this memory space and the data file information of having stored according to this storing message that returns;

Whether all the storage of data file in step 2d, the described storage control server judgment data storehouse finishes, as does not have, and returns step 2a, finishes as whole storages, then finishes.

4, the storage means of the data file based on data/video service system according to claim 1 is characterized in that the data file information of having stored in the described step 2 comprises the Data Filename and the memory location thereof of having stored.

Images