A kind of method and system of file real-time Transmission
Technical field
The present invention relates to the digital television receiving analysis technical field, relate in particular to a kind of method and system of file real-time Transmission.
Background technology
The digital television receiving analysis is a brand-new rating analysis field, the digital television receiving analytical system is applied to through the CHINA RFTCOM Co Ltd behind the digital improvement, be beneficial to set-top box real-time collecting user audience data, then from multi-level, multi-angle, piecewise analysis when carrying out, channel analysis, program analysis, advertisement analysis etc., thus the analysis indexes that obtains is used for explanation, analysis and prediction to spectators' viewing behavior.
Wherein user audience data is to use the user of set-top box to watch the recording of information of TV, comprises time, DVB channel, VOD program request film information etc., detail record user's viewing behavior information.
Data acquisition server can receive the user audience data of all set-top box of being managed, and press some cycles (as a hour), user audience data is saved as the file of certain format, filename is preserved according to the form of file type and date Hour Minute Second usually, be convenient to understand the type and the generation time of this document, regularly generate new data and be stored in the new file, and be stored in the disk array.Data acquisition server is distributed deployment, the set-top box in each data acquisition server management certain limit.
The user audience data file that data analysis center can be collected each data acquisition server is gathered together, and carries out analysis multi-level, multi-angle then.
Therefore, how with the user audience data file on each data acquisition server real-time, unduplicated, accurately, safety converge to the problem that data analysis center is a key.
Be exactly by the FTP transmission for the transmission of file method commonly used at present,, the respective file on the ftp server downloaded to this locality by the long-range ftp server that is connected to of ftp software.This technical scheme following shortcoming arranged:
Real-time is low, and can't judge end-of-file mark.Mode by FTP is difficult to determine whether file has stamped end mark, and then the file that will finish that can not be real-time is uploaded.
Automaticity is low.By ftp software, need the people to need downloaded files usually for going to check, automaticity is too low like this, and makes mistakes easily.
Very flexible.By ftp software or method, use complexity, and very flexible can not be obtained file server address and change situation automatically.
Summary of the invention
The objective of the invention is to propose a kind of method and system of file real-time Transmission, be applicable to the digital television receiving analytical system, can in real time the user audience data file of having finished be reported data analysis center.
For reaching this purpose, the present invention by the following technical solutions:
A kind of method of file real-time Transmission is applicable to the digital television receiving analytical system, may further comprise the steps:
A, the tabulation of File Agent maintenance documentation and file server corresponding file send tabulation;
B, file server sign in to described File Agent;
C, the regular inquiry file of described File Agent send tabulation, send tabulation according to described file server corresponding file, and the file that also is not transferred to described file server on the described listed files is sent to described file server.
Described file comprises user audience data, is regularly generated by data acquisition server.
Steps A further may further comprise the steps:
A1, described File Agent regularly read the filename of the file under the file storing directory in order;
A2, judge whether described file is present in the listed files in the internal memory, if, then return steps A 1, if not, then go to steps A 3;
A3, judge whether described file is scratch file, if, then return steps A 1, if not, then go to steps A 4;
A4, judge whether described file finishes, if not, then return steps A 1, if then go to steps A 5;
A5, described fileinfo is added in the described listed files.
Step B further may further comprise the steps:
B1, described file server read IP address, the port information of File Agent, the username and password of login from configuration file or database;
B2, described file server are set up socket according to IP address, the port information of described File Agent with described File Agent and are connected, and send log-on message;
B3, described File Agent carry out authentication to described file server.
Among the step C, described file sends the file send state information that described file server has been preserved in tabulation, comprises the transmit status of the overall budget number of the position of current transmission file listed files in internal memory, current transmission file, the bag sequence number that has sent, current bag, the transmit status of current file and the filename of current file.
Step C further may further comprise the steps:
C1, described the File Agent regularly described file server corresponding file of inquiry send tabulation, obtain described file server corresponding file send state information;
C2, described File Agent judge according to described file server corresponding file send state information whether current file is sent completely, if, then go to step C3, if not, then go to step C5;
C3, described File Agent judge whether to exist the file that does not send to described file server according to described listed files, if, then go to step C4, if not, then finish this transmission, return step C1 again;
C4, described File Agent send first bag of next file to described file server, and upgrade the file send state information of described file server, are sent completely up to All Files, go to step C1 again;
C5, described File Agent judge whether the bag of current file is sent completely, if, then go to step C6, if not, then go to step C7;
C6, send next bag, the transmit status of update package all is sent completely up to the bag of current file, and returns step C3;
C7, send the current bag of current file, all be sent completely up to the bag of current file, and return step C3.
Step C further comprises following steps:
Described file server is received the file bag of described File Agent transmission, and described file bag is saved in the local file, if preserve the failure of file bag, then sends the order that retransmits the current file bag to described File Agent.
A kind of system of file real-time Transmission, be applicable to the digital television receiving analytical system, comprise File Agent and file server, described File Agent is connected by network with described file server, described File Agent is used for maintenance documentation tabulation, regularly inquiry file sends tabulation and according to described file server corresponding file send state information, the file that also is not transferred to described file server on the listed files is sent to described file server, and described file server is used to receive the file that described File Agent sends.
Described File Agent is positioned on the data acquisition server, and described file server is positioned at data analysis center or file converges the center.
A File Agent correspondence is no less than one file server, and a file server correspondence is no less than one File Agent.
Adopted technical scheme of the present invention, by whether also having the new user audience data file of having finished not send to data analysis center on the regular judgment data acquisition server, guarantee in real time the user audience data file of having finished to be reported data analysis center.
Further, adopted technical scheme of the present invention, user audience data file upload operation, breakpoint transmission etc. all are to finish automatically, do not need artificial participation.
System's expansivity is strong, and the increase of data acquisition server only need be configured in the database, and data analysis center can be collected the file of the data acquisition server that increases newly automatically.
No matter be data acquisition server or the shutdown of data analysis center server and the instability of network, start or network still can continue to upload the file of not finishing after the UNICOM once more once more.
Description of drawings
Fig. 1 is the structural representation of file RTTS in the specific embodiment of the invention.
Fig. 2 is the flow chart of File Agent maintenance customer viewing behavior data file list in the specific embodiment of the invention.
Fig. 3 is the flow chart that File Agent and file server connect in the specific embodiment of the invention.
Fig. 4 is the flow chart that File Agent is given the real-time upload file of file server in the specific embodiment of the invention.
Embodiment
Further specify technical scheme of the present invention below in conjunction with accompanying drawing and by embodiment.
Fig. 1 is the structural representation of file RTTS in the specific embodiment of the invention.As shown in Figure 1, File Agent 104 is deployed on the data acquisition server 102, can finish the user audience data file that data acquisition server is produced and be uploaded to the file server 103 that is deployed in data analysis center 101.
File Agent can safeguard that the user audience data of regularly packing file is deleted expired user audience data file etc. to the user audience data file of this locality.
File Agent also has the file of the file send state information of file server to send tabulation by maintenance record, and file is sent tabulation be mapped on the hard disk by memory file, after file server and File Agent disconnect because of network or other reasons, after connecting once more, continue to upload the user audience data file of remainder or the function that packet reaches breakpoint transmission.
File server is deployed in data analysis center or alternative document converges the center, be used for receiving the user audience data file that each data acquisition server File Agent is uploaded, classification is kept on the disk then, and the data in the disk are put in order, packing, operations such as deletion.
File Agent is connected by network with file server, all supports one-to-many, can improve flexibility for system deployment like this.
Describe the flow process of file real-time Transmission below in detail.
It at first is the generation of user audience data file.The user audience data file is the object of system transmissions, it is the file that generates according to certain format, the name form of file name is generally the time that file generates, represent that as action_20090104_230054.dat this document is the file that generated 23: 0: 54 on the 4th January in 2009, this is for the ease of judge the rise time of this document from filename.
Generate a new file every some cycles (as a hour), the file of up-to-date generation is the file that is writing, and does not temporarily upload, and just uploads this document when waiting next new file to generate.
Next File Agent maintenance customer viewing behavior data file list.By reading the user audience data file, File Agent is safeguarded a listed files, and the file in this tabulation is the file that need be uploaded to file server.The periodic polling listed files adds new fileinfo in the listed files to, and expired fileinfo is deleted from listed files.
The user audience data file is divided into two classes: a class is the file of having write, and this document is the file before the one-period, and the sign of end is arranged; One class is to write the file of also not write, and does not finish sign.The operation of reading local user's viewing behavior data file is periodically to do, and in a very little time interval, will check the file under the file storing directory, checks new file.
Fig. 2 is the flow chart of File Agent maintenance customer viewing behavior data file list in the specific embodiment of the invention.As shown in Figure 2, this flow process may further comprise the steps:
Step 201, File Agent regularly read the filename of the file under the file storing directory in order one by one.
Step 202, the listed files in this document name and the internal memory is compared, judge whether this document is present in the listed files in the internal memory, if, then return step 201, if not, then go to step 203.
Step 203, judge whether this document is scratch file, if, then return step 201, if not, then go to step 204.
Step 204, judge whether this document finishes, if not, then return step 201, if then go to step 205.Wherein whether whether finish be to check to have and finish sign and judge by opening this document to this document, if the sign of end is arranged, then finishes; If do not finish sign, then do not finish.
Step 205, this document information is added in the listed files, returns step 201 again.
Be to connect between File Agent and the file server once more.Fig. 3 is the flow chart that File Agent and file server connect in the specific embodiment of the invention.As shown in Figure 3, this flow process may further comprise the steps:
Step 301, File Agent be the connection of monitoring file server always, and for File Agent, a file server is a client of File Agent, and File Agent then is a server.
Step 302, file server read IP address, the port information of File Agent, the username and password of login from configuration file or database.
Step 303, file server are set up socket according to IP address, port information and the File Agent of File Agent and are connected, and send log-on message.
Step 304, File Agent carry out authentication to file server, and both connect by the back.
It after File Agent and file server connect the process that File Agent is uploaded to the user audience data file file server.Fig. 4 is the flow chart that File Agent is given the real-time upload file of file server in the specific embodiment of the invention.As shown in Figure 4, this flow process may further comprise the steps:
Step 401, File Agent safeguard that the file of its corresponding file server sends tabulation, this document sends the file send state information that all respective file servers of this document agency have been preserved in tabulation, comprises the transmit status of the overall budget number of the position of current transmission file listed files in internal memory, current transmission file, the bag sequence number that has sent, current bag, the transmit status of current file and the filename of current file.
Step 402, File Agent are regularly inquired about this document and are sent tabulation, obtain the file send state information of each file server.
Step 403, according to the file send state information, judge whether the current file of this document server is sent completely, if, then go to step 404, if not, then go to step 406.
Step 404, the tabulation of File Agent inquiry file judge whether to exist the file that does not send to file server, if, then go to step 405, if not, then finish, return step 402 after a while.
Step 405, File Agent send first bag of next file to file server, and the file send state information of transaction file server, are sent completely up to All Files, return step 402 after a period of time.
Step 406, File Agent judge whether the bag of current file is sent completely, if, then go to step 407, if not, then go to step 408.
Step 407, send next bag, the transmit status of update package all is sent completely up to the bag of current file, and returns step 404.
Step 408, send the current bag of current file, all be sent completely up to the bag of current file, and return step 404.Since the listed files in the internal memory be with hard disk in file done File mapping, even file or bag have sent half like this, File Agent or file server are closed, and still can continue transmission after starting once more, but packets need are transmitted whole bag again.So-called here breakpoint transmission that Here it is.
File server is received the file bag of File Agent transmission, and the file bag is saved in the local file, if preserve the failure of file bag, then sends the order that retransmits the current file bag to File Agent.
After the file of a file server is sent completely, File Agent will be uploaded the file of next file server, and so circulation sends file in real time to all corresponding file servers.
Above flow process has been described the reciprocal process of File Agent and file server, whether wherein check has new file can guarantee the real-time Transmission of file in the listed files, because in case new file is arranged, this document information has just joined in the listed files, has just sent to file server as new file then.
The file of the file server that File Agent is safeguarded sends tabulation, write down the file send state information of each file server, owing to be to handle through the memory disk mapping techniques, what so just can guarantee that the information of the file that transmits between File Agent and the file server can safety is saved on the disk, no matter File Agent still is the file server process is died, can breakpoint transmission after restarting.
File for this locality, the file that surpasses a specified time is thought invalid data, also rejected the information of this document in the fileinfo tabulation of File Agent, can not upload, so for the processing of packing of this partial document, for example the file before three days compresses packing, delete original document then, to guarantee the reasonable utilization of disk space.
This embodiment has solved that user audience data file in the digital television system is efficient, the transmission of high real-time, can 24 hours uninterrupted automaticly finishes the reporting of data file; When distributed deployment, can flexible expansion, by increasing File Agent and, thereby reach the purpose of expansion management scope in information such as file server configuration this document agency's IP, port, user name, passwords.
The above; only for the preferable embodiment of the present invention, but protection scope of the present invention is not limited thereto, and anyly is familiar with the people of this technology in the disclosed technical scope of the present invention; the variation that can expect easily or replacement all should be encompassed within protection scope of the present invention.Therefore, protection scope of the present invention should be as the criterion with the protection range of claim.