CN110460876A

CN110460876A - Processing method, device and the electronic equipment of log is broadcast live

Info

Publication number: CN110460876A
Application number: CN201910756302.4A
Authority: CN
Inventors: 杜往泽; 黄洪波
Original assignee: Netease Hangzhou Network Co Ltd
Current assignee: Netease Hangzhou Network Co Ltd
Priority date: 2019-08-15
Filing date: 2019-08-15
Publication date: 2019-11-15

Abstract

The present invention provides a kind of processing method, device and electronic equipments that log is broadcast live, this method comprises: the log information that the client device for obtaining live streaming platform is sent；Log information is cleaned according to log cleaning rule；Statistics calculating is carried out to the log information after cleaning according to data statistics rule, obtains the corresponding live streaming statistic of attribute result of each storage dimension；According to each storage dimension in data storage rule, the corresponding live streaming statistic of attribute result of each storage dimension is stored.This mode that statistics calculating is carried out to log information, available reliable live streaming statistic of attribute result, and the live streaming statistic of attribute result is stored according to each storage dimension, it is easy to know the live streaming quality information in each dimension, the development that advanced optimizes for subsequent live streaming platform provides reliable data.

Description

Processing method, device and the electronic equipment of log is broadcast live

Technical field

The present invention relates to the technical field of information processing, more particularly, to a kind of processing method that log is broadcast live, device and Electronic equipment.

Background technique

In Internet era, how from acquisition data of interest always is the hot spot of research in mass data on line.Example Such as, for industry is broadcast live, as emerging mode of amusing and diverting oneself, it is young that more and more new era are attract.Live streaming Form it is varied, further include various joys in addition to the sports match of early stage, news hookup etc. are using other than direct-seeding The direct-seedings such as happy, teaching, live streaming are usually recipient of the manufacturer (lower referred to as main broadcaster) of live content with live content Information one-way transmission between (lower referred to as user).

With the extensive use of live streaming, it is also more and more that platform is broadcast live.It is all often that main broadcaster is logical on these live streaming platforms The information for crossing user feedback knows the quality condition of current live, such as true by the barrage of user feedback, picture, voice information Current live is determined with the presence or absence of Caton, delay, the nonsynchronous situation of sound picture.

The acquisition modes of above-mentioned live streaming quality are merely capable of that one people of main broadcaster is made to know the live streaming quality, and the live streaming quality It is relatively simple, it is difficult to which that the improvement that quality is broadcast live for live streaming platform provides reliable data.

Summary of the invention

The embodiment of the present invention be designed to provide it is a kind of be broadcast live log processing method, device and electronic equipment, with Quality is broadcast live in express statistic on various dimensions, improves live streaming quality for subsequent live streaming platform and provides reliable data.

It is provided in an embodiment of the present invention it is a kind of be broadcast live log processing method, applied to live streaming platform server device, Preset log processing rule is stored in the server device, the log processing rule includes log cleaning rule, number Rule and data storage rule according to statistics, which comprises

Obtain the log information that the client device of the live streaming platform is sent；The log information includes: quality index Storage dimension in the attribute information of information and the client device, the attribute information and the data storage rule is closed Connection；

The log information is cleaned according to the log cleaning rule；

Statistics calculating is carried out to the log information after cleaning according to the data statistics rule, obtains each described deposit Store up the corresponding live streaming statistic of attribute result of dimension；

According to each storage dimension in the data storage rule, it is corresponding straight to store each storage dimension Broadcast statistic of attribute result.

Further, the log processing rule includes multiple, and each log processing rule further includes having index word Section；Correspondingly, after the step of obtaining the log information of the client device transmission of the live streaming platform, the method is also wrapped Include: the log information matched with the index field of each log processing rule, according to matching result it is determining with it is described The matched log processing rule of log information is united using the log cleaning rule in the matched log processing rule, data Meter rule and data storage rule clean the log information, count calculating and storage processing.

Further, the step of obtaining the log information of the client device transmission of the live streaming platform includes: to receive institute State the log information that the client device of live streaming platform reports；The client device is during live streaming, according to the of setting Log information described in one periodic report；And/or the client device live streaming during, according to setting second round from Read log information in the designated storage area of the client device.

Further, the log cleaning rule includes: duplicate removal rule, mapping ruler and field supplement rule；Correspondingly, The step of cleaning according to the log cleaning rule to the log information includes: to believe according to the duplicate removal rule the log Breath carries out log duplicate removal processing；Field mapping processing is carried out to the log information after duplicate removal according to the mapping ruler；According to institute It states field supplement rule and field supplement process is carried out to field mapping treated log information, obtain complete clear of data field The log information after washing.

Further, the log cleaning rule further includes having splicing rule；Correspondingly, according to the duplicate removal rule to institute Stating the step of log information carries out log duplicate removal processing includes: the extraction log identification information in the log information；The day Will identification information includes: mark title and ident value；According to the splicing rule to the log identification information splicing；It will The keyword comparison stored in the keyword and keyword database that splicing obtains, determines the pass that splicing obtains Whether the keyword stored in keyword and the keyword database repeats；If repeating, the institute obtained with splicing is abandoned State log information corresponding to keyword；If not repeating, the keyword that the splicing obtains is saved to the key Word database, and set the effective time for the keyword that splicing obtains；The effective time is right for indicating it The keyword the answered effective time；Log information corresponding to the keyword in the obtained keyword database will be updated to make For the log information after duplicate removal.

Further, the step of carrying out field mapping processing to the log information after duplicate removal according to the mapping ruler is wrapped It includes: aiming field title corresponding with field name in the log information after duplicate removal is determined according to the mapping ruler； Field corresponding with the aiming field title in the log information after replacing duplicate removal using the aiming field title Title obtains field mapping treated the log information.

Further, field mapping treated log information is carried out at field supplement according to field supplement rule The step of reason includes: to believe according to field supplement rule the aiming field in field mapping treated the log information Breath carries out increasing changed handling, the log information after obtaining the complete cleaning of data field；The aiming field information is described Field information defined in field supplement rule.

Further, the data statistics rule is the data statistics rule for carrying time scale, storing dimension；Accordingly Ground carries out statistics calculating to the log information after cleaning according to the data statistics rule, obtains each storage dimension The step of spending corresponding live streaming statistic of attribute result includes: the aimed quality index letter in the log information after obtaining cleaning Breath；The aimed quality indication information is the quality index information to match with the time scale, the storage dimension；According to The data statistics rule carries out calculation processing to the aimed quality indication information, and it is right at its to obtain each storage dimension The live streaming statistic of attribute result for the time scale answered.

Further, the step of storing each storage dimension corresponding live streaming statistic of attribute result includes: according to institute It states storage dimension corresponding to live streaming statistic of attribute result and time scale determines target database table；The target database table With the database table to match for the storage dimension and the time scale；The live streaming statistic of attribute result is saved to institute It states in target database table.

Further, the method also includes: in response to the displaying operation for live streaming quality, the live streaming to storage Statistic of attribute result is shown.

Further, in response to being operated for the displaying of live streaming quality, to the live streaming statistic of attribute result of storage into The step of row is shown comprises determining that the data of the server device show demand；The data show that demand includes: to be presented Quality is broadcast live, shows that dimension, data show section and show time scale；According to the live streaming quality to be presented, the displaying Dimension and the displaying time scale determine display data library table；Display data library table is with described in database table wait open up Show live streaming quality, the database table for showing dimension and the displaying time scale and matching；From display data library table The middle data extracted in data displaying section, and the data obtained to extraction are shown.

Further, in response to being operated for the displaying of live streaming quality, to the live streaming statistic of attribute result of storage into The step of row is shown further include: obtain the data displaying request that the client of administrator is sent；The data show in request and take With live streaming quality to be presented, show that dimension, data show the parameter in section and displaying time scale；According to described to be presented straight It broadcasts quality, the displaying dimension and the displaying time scale and determines display data library table；Display data library table is data The database table to match in the table of library with live streaming quality, the displaying dimension and the displaying time scale to be presented；From The data in data displaying section are extracted in the table of the display data library, and are sent to the pipe for obtained data are extracted The client of reason person is shown.

Further, the method also includes: obtain log processing rule update request；It is carried in the update request Need the parameter and undated parameter of Update log processing rule；The undated parameter includes at least one of: at addition log The parameter of rule is managed, the parameter of log processing rule is modified and deletes the parameter of log processing rule；According to the day to be updated The parameter of will processing rule determines the log processing rule to be updated, and according to the undated parameter to the log to be updated Processing rule is updated.

The embodiment of the invention also provides a kind of processing unit that log is broadcast live, the server-side applied to live streaming platform is set It is standby, be stored with preset log processing rule in the server device, the log processing rule include log cleaning rule, Data statistics rule and data storage rule, described device include:

First acquisition unit, the log information that the client device for obtaining the live streaming platform is sent；The log Information includes: the attribute information of quality index information and the client device, and the attribute information and data storage are advised Storage dimension association in then；

Cleaning unit, for being cleaned according to the log cleaning rule to the log information；

Computing unit is counted, by carrying out based on statistics according to the data statistics rule to the log information after cleaning It calculates, obtains the corresponding live streaming statistic of attribute result of each storage dimension；

Storage unit, for according to each storage dimension in the data storage rule, storing each described deposit Store up the corresponding live streaming statistic of attribute result of dimension.

Further, the log processing rule includes multiple, and each log processing rule further includes having index word Section；Correspondingly, described device further include: matching unit, for by the log information and each log processing rule Index field matching, it is determining regular with the matched log processing of the log information according to matching result, using matched described Log cleaning rule, data statistics rule and data storage rule in log processing rule carry out the log information clear It washes, count calculating and storage processing.

Further, the first acquisition unit is also used to: receiving the day that the client device of the live streaming platform reports Will information；The client device reports the log information during live streaming, according to the period 1 of setting；And/or In During the client device live streaming, day is read from the designated storage area of the client device according to the second round of setting Will information.

Further, the log cleaning rule includes: duplicate removal rule, mapping ruler and field supplement rule；Correspondingly, The cleaning unit is also used to: carrying out log duplicate removal processing to the log information according to the duplicate removal rule；It is reflected according to described It penetrates rule and field mapping processing is carried out to the log information after duplicate removal；After handling according to field supplement rule field mapping Log information carry out field supplement process, the log information after obtaining the complete cleaning of data field.

Further, the log cleaning rule further includes having splicing rule；Correspondingly, the cleaning unit is also used to: Log identification information is extracted in the log information；The log identification information includes: mark title and ident value；According to institute Splicing rule is stated to the log identification information splicing；It will be deposited in keyword and keyword database that splicing obtains The keyword of storage compares, and determines that the keyword stored in the keyword and the keyword database that splicing obtains is No repetition；If repeating, log information corresponding to the keyword obtained with splicing is abandoned；It, will if not repeating The keyword that the splicing obtains is saved to the keyword database, and sets the keyword that splicing obtains Effective time；The effective time is used to indicate the keyword effective time corresponding to it；The pass that update is obtained Log information corresponding to keyword in keyword database is as the log information after duplicate removal.

Further, the cleaning unit is also used to: being believed according to the determining log with after duplicate removal of the mapping ruler The corresponding aiming field title of field name in breath；The log information after replacing duplicate removal using the aiming field title In field name corresponding with the aiming field title, obtain field mapping treated the log information.

Further, the cleaning unit is also used to: according to field supplement rule to field mapping treated institute The aiming field information stated in log information carries out increasing changed handling, the log letter after obtaining the complete cleaning of data field Breath；The aiming field information is field information defined in field supplement rule.

Further, the data statistics rule is the data statistics rule for carrying time scale, storing dimension；Accordingly Ground, the statistics computing unit are also used to: the aimed quality indication information in the log information after obtaining cleaning；The mesh Mark quality index information is the quality index information to match with the time scale, the storage dimension；According to the data Statistical rules carries out calculation processing to the aimed quality indication information, obtains each storage dimension in its corresponding time The live streaming statistic of attribute result of scale.

Further, the storage unit is also used to: according to storage dimension corresponding to the live streaming statistic of attribute result Target database table is determined with time scale；The target database table with for the storage dimension and the time scale phase The database table matched；The live streaming statistic of attribute result is saved into the target database table.

Further, described device further include: display unit, it is right for being operated in response to the displaying for being directed to live streaming quality The live streaming statistic of attribute result of storage is shown.

Further, the display unit is also used to: determining that the data of the server device show demand；The data Displaying demand includes: live streaming quality, displaying dimension, data displaying section and displaying time scale to be presented；According to described wait open up Show that live streaming quality, the displaying dimension and the displaying time scale determine display data library table；Display data library table is The database to match in database table with live streaming quality, the displaying dimension and the displaying time scale to be presented Table；From the data extracted in data displaying section in the table of the display data library, and the data obtained to extraction are opened up Show.

Further, the display unit is also used to: obtaining the data displaying request that the client of administrator is sent；It is described Data, which are shown, carries live streaming quality, displaying dimension, data displaying section and the parameter for showing time scale to be presented in request； Display data library table is determined according to live streaming quality, the displaying dimension and the displaying time scale to be presented；The exhibition Show database table be database table in live streaming quality, the displaying dimension and the displaying time scale phase to be presented The database table matched；From the data extracted in the table of the display data library in data displaying section, and extraction is obtained The client that data are sent to the administrator is shown.

Further, described device further include: second acquisition unit, the update for obtaining log processing rule are requested； It is described to update the parameter and undated parameter that log processing rule to be updated is carried in request；The undated parameter include with down toward One of few: the parameter of addition log processing rule modifies the parameter of log processing rule and deletes the parameter of log processing rule； Updating unit, for determining the log processing rule to be updated, and root according to the parameter of the log processing rule to be updated The log processing rule to be updated is updated according to the undated parameter.

The embodiment of the invention also provides a kind of electronic equipment, including processor and memory, the memory is stored with The machine-executable instruction that can be executed by the processor, the processor execute the machine-executable instruction to realize on The step of stating the processing method that log is broadcast live described in content.

The embodiment of the invention also provides a kind of computer readable storage medium, the computer-readable recording medium storage There is machine-executable instruction, the machine-executable instruction is when being called and being executed by processor, the machine-executable instruction The step of promoting the processor to realize the processing method of live streaming log described in above content.

In the processing method of above-mentioned live streaming log provided in an embodiment of the present invention, device and electronic equipment, server-side is set It is standby to be above stored with preset log processing rule, include in the log processing rule log cleaning rule, data statistics rule and Data storage rule first obtains the log information that the client device of live streaming platform is sent in the processing for being broadcast live log, The log information is cleaned according to log cleaning rule, the log information after cleaning is carried out according still further to the data statistics rule Statistics calculates, and according to each storage dimension in the data storage rule, stores the corresponding live streaming of each storage dimension Statistic of attribute result.It is this that live streaming log (log information that the client device of i.e. above-mentioned live streaming platform is sent) is counted The mode of calculating, available reliable live streaming statistic of attribute as a result, and the live streaming statistic of attribute result deposited according to each Storage dimension is stored, and is easy to know the live streaming quality information in each dimension, advanced optimizes hair for subsequent live streaming platform Exhibition provides reliable data, and alleviating the prior art cannot achieve live streaming quality the express statistic on various dimensions the technical issues of.

Detailed description of the invention

It, below will be to specific in order to illustrate more clearly of the specific embodiment of the invention or technical solution in the prior art Embodiment or attached drawing needed to be used in the description of the prior art be briefly described, it should be apparent that, it is described below Attached drawing is some embodiments of the present invention, for those of ordinary skill in the art, before not making the creative labor It puts, is also possible to obtain other drawings based on these drawings.

Fig. 1 is the schematic diagram of the application scenarios of live streaming log processing provided in an embodiment of the present invention；

Fig. 2 is the flow chart of the processing method of live streaming log provided in an embodiment of the present invention；

Fig. 3 is the method flow diagram provided in an embodiment of the present invention to log information cleaning；

Fig. 4 is the method flow diagram of log duplicate removal processing provided in an embodiment of the present invention；

Fig. 5 is the method flow diagram that the log information provided in an embodiment of the present invention to after cleaning carries out statistics calculating；

Fig. 6 is the displaying signal of the Caton quality for main broadcaster's dimension that time scale provided in an embodiment of the present invention is 1 minute Figure；

Fig. 7 is the flow chart of the update method of log processing rule provided in an embodiment of the present invention；

Fig. 8 is the schematic diagram of the processing unit of live streaming log provided in an embodiment of the present invention；

Fig. 9 is the schematic diagram of the processing unit of another live streaming log provided in an embodiment of the present invention；

Figure 10 is the schematic diagram of electronic equipment provided in an embodiment of the present invention.

Specific embodiment

Technical solution of the present invention is clearly and completely described below in conjunction with embodiment, it is clear that described reality Applying example is a part of the embodiment of the present invention, instead of all the embodiments.Based on the embodiments of the present invention, the common skill in this field Art personnel every other embodiment obtained without making creative work belongs to the model that the present invention protects It encloses.

For industry is broadcast live, related personnel supervises live streaming platform often through the information of live streaming quality, with The data that basis is provided for the optimized development of platform is broadcast live are supported.Wherein, live streaming quality can be set in the server-side of live streaming platform Standby upper statistics obtains.

The schematic diagram of the application scenarios of live streaming log processing shown in Figure 1, wherein illustrating the clothes of live streaming platform Be engaged in end equipment, and with the server device communicate to connect respectively live streaming platform client and administrator client.

Wherein, the client of platform is broadcast live during playing live streaming, can be calculated according to the Video stream information got The current log information about live streaming quality index is obtained, and then obtained log information is sent to the service of live streaming platform End equipment, then, the extraction that the server device that platform is broadcast live is broadcast live qualitative data to above-mentioned log information again calculate, from And the information of live streaming quality is obtained, and the information of obtained live streaming quality is sent to the client of administrator, for administrator Client the information of above-mentioned live streaming quality is shown and is analyzed.

But in view of the information of the obtained live streaming quality of server device of the live streaming platform of the prior art is more single (such as: only determine obtain live streaming platform Caton numerical value), be unable to get live streaming quality on different dimensions information (such as: Difference of the Caton under the dimensions such as different cities, different player versions, distinct device signal, difference CDN supplier), it is based on This, the embodiment of the present invention improves the treatment process of live streaming log, and improved processing method can obtain different dimensional Live streaming quality information on degree below describes to the process of the processing method of the live streaming log of inventive embodiments in detail.

Fig. 2 is a kind of flow chart of processing method that log is broadcast live according to an embodiment of the present invention, and this method can be applied In being stored with preset log processing rule in the server device of live streaming platform, the server device, above-mentioned log processing is advised It then include log cleaning rule, data statistics rule and data storage rule.With reference to Fig. 2, the processing method of above-mentioned live streaming log Include the following steps:

Step S202 obtains the log information that the client device of live streaming platform is sent.

Wherein, log information includes: the attribute information of quality index information and client device, and attribute information is deposited with data Storage dimension association in storage rule.

In embodiments of the present invention, the client device that platform is broadcast live (can be mobile terminal equipment, or the end PC is set It is standby, when for PC end equipment, it can be the player in PC end equipment, the webpage being also possible on the end PC, the embodiment of the present invention To above-mentioned client device without concrete restriction) when playing live streaming, client device can be according to the video flowing letter got Breath be calculated current quality index information (such as: Caton information, sound are drawn nonsynchronous information, frame per second information and network and are prolonged Slow information etc.), and above-mentioned quality index information is presented in log information.

It in addition to this, further include the attribute information for having client device in log information, above-mentioned attribute information may include: The type information (such as: IOS, Android, PC, web etc.) of client device, the version information of client device, CDN operation Quotient's information (such as: net place, Supreme Being's connection etc.), turn of tidal stream code information (such as: blue light, high definition, SD etc.), User IP information etc. and storage The associated information of dimension.

In addition, can be at the open source stream of Kafka in the log information that the client device for obtaining live streaming platform is sent Platform (its distributed post that can be realized high-throughput subscribes to message) first obtains above-mentioned log information, and then Kafka Log information is sent to server device again by open source stream process platform, so that server device obtains above-mentioned log information； It can be the log information that server device directly acquires client device transmission, the embodiment of the present invention is to its specific implementation It is not limited.

Step S204 cleans log information according to log cleaning rule.

Due to the version of client device, the otherness of implementation, the day that server device acquires is often resulted in The format of will information is inconsistent, it is therefore desirable to clean according to log cleaning rule to above-mentioned log information, obtains after cleaning Log information included in data be all valid data, field is regular and field is complete, be based on convenient for server device clear The processing that log information after washing is broadcast live log calculates, and then improves the efficiency that statistics calculates, meanwhile, it is finally obtained each The corresponding live streaming statistic of attribute result of a storage dimension also can be more accurate.

Step S206 carries out statistics calculating to the log information after cleaning according to data statistics rule, obtains each storage The corresponding live streaming statistic of attribute result of dimension.

When carrying out statistics calculating, the aimed quality indication information in the log information after can first obtaining cleaning is (specific Can be the quality index information to match with the SS in data statistics rule), and then according in data statistics rule Operation rule statistics calculating is carried out to above-mentioned aimed quality indication information, so that it is corresponding straight to obtain each storage dimension Broadcast statistic of attribute result.

The corresponding live streaming statistic of attribute result of above-mentioned each storage dimension can more comprehensively, complete reaction live streaming platform Live streaming quality, facilitate administrative staff be based on above-mentioned live streaming statistic of attribute result to live streaming platform comprehensively and effectively supervised, Also it is supported for the data that development provides the foundation that advanced optimize of subsequent live streaming platform.

Step S208 stores the corresponding live streaming of each storage dimension according to each storage dimension in data storage rule Statistic of attribute result.

The corresponding live streaming statistic of attribute of each storage dimension is stored as a result, it is possible to the management being more clear according to storage dimension The live streaming statistic of attribute of each storage dimension is as a result, be also convenient for subsequent searching live streaming quality to be presented in live streaming statistic of attribute result Statistical result improves the efficiency for searching live streaming statistic of attribute result to be presented.

In view of live streaming quality pack contain many various information (such as: Caton information, sound draw nonsynchronous information, frame Rate information and network delay information etc.), in order to comprehensively be counted to live streaming quality, the day stored in server device Will processing rule includes multiple, different log processing rules for counting to the live streaming quality of different aspect.

Since the quantity of log processing rule is multiple, so, log information is being carried out directly based on log processing rule Before broadcasting the processing of log, it is thus necessary to determine that it is regular with the matched log processing of log information, based on determining obtained log processing Rule carries out the statistics of quality index to log information.

And whether the index field in log processing rule is able to reflect out the log processing rule and matches with log information, In order to simplify realization process, the present embodiment is determining regular with the matched log processing of log information using index field as foundation, base In this, this gives the determining specific implementations with the matched log processing rule of log information:

Log information is matched with the index field of each log processing rule, according to matching result determination and log information Matched log processing rule, using log cleaning rule, data statistics rule and the data in matched log processing rule Storage rule cleans log information, counts calculating and storage processing.

Below with specific log information and log processing rule (here with the log for carrying out Caton statistic of attribute at Reason rule) for simply introduce the process of above-mentioned determination Yu the matched log processing rule of log information:

If log information are as follows:

Above-mentioned log information is the log information after formatted presentation, quick obtaining corresponding field when facilitating subsequent processing Information.

Log processing rule are as follows:

The index field of above-mentioned log processing rule are as follows: matched_rules field, when getting above-mentioned log information Afterwards, above-mentioned log information is matched with the index field (i.e. matched_rules field) of above-mentioned log processing rule, by With it is found that the information of the index field of above-mentioned log processing rule is log_type:PLAYER_INTERVAL, and above-mentioned day Also include in will information " log_type ": " PLAYER_INTERVAL " information, it is determined that above-mentioned log processing rule be and day The log processing rule of will information matches, it will be able to log cleaning rule, data statistics in the above-mentioned log processing rule of application Rule and data storage rule clean above-mentioned log information, count calculating and storage processing.

If getting a log information again, the information in " log_type " field in the log information is not PLAYER_INTERVAL, then illustrate that the index field of the log information and above-mentioned log processing rule mismatches, above-mentioned log Processing rule is not and the matched log processing of the log information is regular.

Server device can have different implementations when obtaining the log information that client device is sent.This reality It applies example and gives the realization side for the log information step that the client device for obtaining live streaming platform in above-mentioned steps S202 is sent Formula can specifically include the implementation of following (1) and/or (2):

(1) log information that the client device of live streaming platform reports is received；Client device during live streaming, according to The period 1 of setting reports log information.

(2) during client device is broadcast live, according to the second round of setting from the designated storage area of client device Read log information.

It should be noted that the above-mentioned period 1 can be identical as above-mentioned second round, the above-mentioned period 1 can also be with Above-mentioned second round is different.In addition, the above-mentioned period 1 can be for every the period of half a minute, the embodiment of the present invention is to above-mentioned the One period and above-mentioned second round are without concrete restriction.

In view of the format diversification for the log information that client device is sent, it is not easy to the subsequent statistics to log information Calculate, be based on this, present embodiments provide log cleaning rule, the log cleaning rule include: duplicate removal rule, mapping ruler and Field supplement rule.Log information is cleaned according to above-mentioned log cleaning rule, the log information obtained after cleaning is convenient for being used for Carry out the processing of subsequent live streaming log.With reference to the method flow diagram shown in Fig. 3 cleaned to log information, this method give upper The specific implementation according to log cleaning rule to log information cleaning step in step S204 is stated, this method includes as follows Step:

Step S301 carries out log duplicate removal processing to log information according to duplicate removal rule.

In view of client device is when being abnormal, duplicate log information may be sent to server device, and Duplicate log information will cause the repetition of subsequent live streaming statistic of attribute, and then influence final live streaming statistic of attribute result.Base In this, the embodiment of the present invention devises duplicate removal rule, and server device is made to carry out log to log information according to duplicate removal rule It handles again, so that the log information after duplicate removal is all valid data information, to improve final live streaming statistic of attribute result Accuracy.

With reference to the method flow diagram of log duplicate removal processing shown in Fig. 4, this method give in above-mentioned steps S301 according to Duplicate removal rule carries out the specific implementation of log duplicate removal processing step to log information, and this method comprises the following steps:

Step S401 extracts log identification information in log information.

Wherein, log identification information includes: mark title and ident value.

It include duplicate removal rule in log processing rule when realization, it is therein such as the log processing rule in the example above The information that duplicate removal rule includes by unique field.There it can be seen that field there are three including in above-mentioned unique field, Respectively keys field, fmt field and ttl field, wherein keys field indicates to obtain from log information for characterizing log The log identification information of information uniqueness；Fmt field indicates the splicing rule of log identification information；Ttl field is indicated to log The effective time of the keyword obtained after identification information splicing.

According to the log identification information extracting rule in above-mentioned duplicate removal rule (in the keys field i.e. in unique field Information) log identification information is extracted in log information.For example, for the log information in the example above, according to above-mentioned After log identification information extracting rule carries out the extraction of log identification information to above-mentioned log information, obtained log identification information Are as follows: PLAYER_INTERVAL, 1001,700700,30f8b5be8f6d8431,10028, ccandroid, 2019-07-12 09:41:11。

Step S402, according to splicing rule to log identification information splicing.

Continuation be illustrated with the citing in step S401, obtain above-mentioned log identification information (such as: PLAYER_ INTERVAL, 1001,700700,30f8b5be8f6d8431,10028, ccandroid, 2019-07-12 09:41:11) after, According to splicing regular (information i.e. in fmt field) to above-mentioned log identification information splicing, the key that splicing obtains Word are as follows: QUALITYCENTER_player_interval_buf_cnt_unique_PLAYER_INTE RVAL_ | _ 1001_ | _ 700700_|_30f8b5be8f6d8431_|_10028_|_ccandroid_|_2019-07-1209:41:11。

The keyword stored in keyword and keyword database that splicing obtains is compared, is determined by step S403 Whether the keyword stored in the keyword and keyword database that splicing obtains repeats；If repeating, then follow the steps S404；If not repeating, S405 and step S406 are thened follow the steps.

Step S404 abandons log information corresponding to the keyword obtained with splicing.

Step S405, the keyword that splicing is obtained are saved to keyword database, and are set splicing and obtained Keyword effective time.

When realization, according to the ttl field in duplicate removal rule the obtained keyword of information setting splicing it is effective when Between.Wherein, effective time is used to indicate the keyword effective time corresponding to it.

Step S406, after log information corresponding to the keyword in obtained keyword database will be updated as duplicate removal Log information.

In log information after carrying out the duplicate removal that above-mentioned log duplicate removal processing obtains do not include duplicate log information to get To log information be all effective data information, can be used in subsequent live streaming statistic of attribute.

After log information after obtaining duplicate removal, subsequent log cleaning process is continued to execute, comprising:

Step S302 carries out field mapping processing to the log information after duplicate removal according to mapping ruler.

In view of the log information that client device is sent is showed according to the format of client user's hobby, do not have Unified standard is not easy to the processing of subsequent live streaming log.Based on this, the embodiment of the present invention gives the specific of field mapping processing Implementation easily facilitates the processing of subsequent live streaming log so that treated that log information field is regular for field mapping.This Outside, mapping ruler used in field mapping treatment process, the quality index information also shown that it has, i.e. its expression user's sense The quality index information of interest, subsequent statistical only consider the interested quality index information of the user during calculating, ignore Other uninterested quality index information of user.

The implementation of field mapping processing may include the process of following (a) and (b):

(a) aiming field title corresponding with field name in the log information after duplicate removal is determined according to mapping ruler.

As the example above log processing rule in, letter that mapping ruler therein includes by key_mapping field Breath.There it can be seen that the mapping ruler gives the mapping ruler of cdn, server and video_ccid, that is, give respectively Aiming field title corresponding to cdn, server and video_ccid is gone out.The aiming field title be respectively as follows: cdn, Server and anchor.

(b) using field name corresponding with aiming field title in the log information after aiming field title replacement duplicate removal Claim, obtains field mapping treated log information.

Citing in such as above-mentioned (a), that is, using in the log information after aiming field title-anchor replacement duplicate removal Video_ccid, and then field mapping treated log information can be obtained.

In addition, by above-mentioned mapping ruler it is found that the information of cdn, server and anchor are useful quality index Information, the i.e. information of cdn, server and anchor are the interested quality index information of user, and other quality index letters Breath (such as: information, the information in live streaming place room of live state etc.) it is the uninterested quality index information of user, it is subsequent Statistics calculate during only consider cdn, server and anchor information (for example, the example above log processing rule in, After key_mapping field, and there is the information of cdn, server and anchor, expression used cdn, server and The information of anchor).

Treated that log information field is regular for the mapping of above-mentioned field, easily facilitates subsequent live streaming statistic of attribute.

After obtaining field mapping treated log information, subsequent log cleaning process is continued to execute, comprising:

Step S303 carries out field supplement process to field mapping treated log information according to field supplement rule, Log information after obtaining the complete cleaning of data field.

In view of the log information that client device is sent has that field missing, field are abnormal.The present invention is implemented Example gives the specific implementation of field supplement process, so that the data field of the log information after finally obtained cleaning is complete It is standby, to improve the validity of subsequent live streaming statistic of attribute.

The implementation of field supplement process can be with are as follows: according to field supplement rule, to field mapping, treated that log is believed Aiming field information in breath carries out increasing changed handling, the log information after obtaining the complete cleaning of data field.Wherein, target word Segment information is field information defined in field supplement rule.

As the example above log processing rule in, field therein supplement rule includes: that os_name field is included The information that the information and definition field that information that information, city field are included, isp field are included are included.

By the log information of the example above it is found that only containing user's ip information in the log information that client device is sent (the server field i.e. in log information) also needs to obtain urban information (the i.e. city word of user during actual count The information of section) and telecom operators' information (i.e. the information of isp field), for this purpose, self-defining ip2city method, ip2isp Method handles server field, and the information that server field is included is switched to the urban information and electricity of required user Letter operator's informaiton (such as: in field supplement rule ----city:{ key:server, method:$ ip2city } and isp: {key:server,method:$ip2isp})。

In addition, the os_name field of partial log information is " web " due to some historical reasons, also there is partial log letter The os_name field of breath is " web ' ", the two can be divided into two different classes when subsequent statistical calculates, and causes statistics wrong Accidentally.Therefore, above-mentioned os_name field is cleaned by the strip method of self-defining (such as: in field supplement rule ----os_name:{ key:os_name, method:$ strip }).

Further, it is also possible to supplement the definition in rule, such as above-mentioned log processing rule comprising other fields The information that field is included, indicate parsing clarity field supplement rule (such as: field supplement rule in ---- Definition:{ key:stream, method:$ stream2definition }), it can also include the field of numerical value shaping Supplement rule etc..

Above-mentioned realization process is illustrated by taking the implementation of python language as an example below, LogKeyMapping is Contain all processing methods (such as above-mentioned ip2city method, ip2isp method, strip method and Stream2definition object class), according to the subsequent parameter of method in log processing rule from LogKeyMapping In transfer corresponding method (getattr of specifically used python to transfer corresponding method), and then to the subsequent parameter of key Handled, thus obtain required information (such as: the information of above-mentioned urban information, telecom operators' information and clarity Deng).

When lacking some way in LogKeyMapping, the method that can will be lacked in advance using customized mode It is added in LogKeyMapping, therefrom to transfer use.

After the log information after being cleaned, it will be able to the processing of log is broadcast live to the log information after cleaning, To obtain the corresponding live streaming statistic of attribute result of each storage dimension.With reference to the log information shown in fig. 5 to after cleaning into Row statistics calculate method flow diagram, this method give in above-mentioned steps S206 according to data statistics rule to the day after cleaning Will information carries out the specific implementation that statistics calculates step, and above-mentioned data statistics rule is to carry time scale, storage dimension The data statistics rule of degree, this method comprises the following steps:

Step S501, the aimed quality indication information in log information after obtaining cleaning.

Wherein, aimed quality indication information is the quality index information to match with time scale, storage dimension.

Such as the log processing rule in the example above, operation field therein is meant that in data statistics rule Operation rule.Wherein, the operation rule of total field is { $ default:1 }, indicates often to receive the log after a cleaning The value of information, total all adds 1；The operation rule of buffered field is { $ and:{ bufcnt:{ $ gt:0 }, playtime:{ $ Gt:60 } } }, indicate that the value of bufcnt field is greater than 0, and the value of playtime field is greater than 60, then the value of buffered is 1, it is otherwise 0；The operation rule of percentage field is { $ div:[buffered, total] }, indicate buffered with The division arithmetic result of total.It should be noted that in above-mentioned operation rule: div indicates division, and and is indicated and logic, gt table Show and is greater than.

In addition, when the operation rule in above-mentioned data statistics rule is in practical application, it is also contemplated that the specific time Scale and storage dimension, when its be used to carry out according to 1 minute time scale and main broadcaster's dimension statistics calculate when, it is corresponding must Bufcnt value and playtime value (i.e. above-mentioned aimed quality index letter in by every 1 minute in the log information of each main broadcaster Breath).

Step S502 carries out calculation processing to aimed quality indication information according to data statistics rule, obtains each storage Live streaming statistic of attribute result of the dimension in its corresponding time scale.

Such as the citing in above-mentioned steps S501, obtain bufcnt value in every 1 minute in the log information of each main broadcaster and After playtime value, and then can be according to the operation rule (content i.e. in operation field) in data statistics rule To the bufcnt value and the progress calculation processing of playtime value in each log information, so that every 1 minute of each main broadcaster can be obtained Interior corresponding Caton number and Caton rate, it is similar to the above process for the calculating process of other storage dimensions, herein no longer into Row illustrates one by one.

The above process is illustrated with two examples again:

Example one: assuming that share 100 log informations about Caton in a certain minute, not according to any storage dimension into Row statistics calculates, and the value of total is 100, if to bufcnt value and the progress calculation processing of playtime value in each log information Afterwards, determining, which wherein has the bufcnt value of 10 log informations to be greater than 0 and playtime value, is greater than 60, then buffered value is 10, The percentage=buffered/total=10% being finally calculated according to operation rule, i.e., total Caton rate are 10%.

Example two: it assuming that sharing 100 log informations about Caton in a certain minute, is counted according to main broadcaster's dimension It calculates, wherein the Caton log information about A main broadcaster shares 50, i.e. total value is 50, if the Caton log to A main broadcaster is believed After bufcnt value and playtime value in breath carry out calculation processing, determine that the bufcnt value for wherein there are 2 log informations is greater than 0 And playtime value is greater than 60, then buffered value is 2, the percentage=being finally calculated according to operation rule Buffered/total=4%, i.e., when carrying out statistics calculating according to main broadcaster's dimension, the Caton of all clients user of A main broadcaster Rate is 4%.

Each storage dimension is being obtained after the live streaming statistic of attribute result of its corresponding time scale, can further deposited Store up the corresponding live streaming statistic of attribute result of each storage dimension.The present invention gives store to advise according to data in above-mentioned steps S208 Each storage dimension in then stores the specific implementation of the corresponding live streaming statistic of attribute result step of each storage dimension, Include the steps that it is following 1) and 2):

1) the storage dimension according to corresponding to live streaming statistic of attribute result and time scale determine target database table.

Wherein, target database table and the database table to match for storage dimension and time scale.

2) live streaming statistic of attribute result is saved into target database table.

When realization, such as the log processing rule in the example above, aggs field therein indicates data storage rule.Its In, value field indicates the data field for needing to store；Live streaming statistic of attribute result storage is defined in storage field Table structure, herein, table structure are column tableau formats, i.e., finally store live streaming statistic of attribute result to different database tables In.Every database table define storage dimension field (i.e. keys field) and database literary name section (i.e. mongo field, wherein Include database table table name and corresponding time scale).

In the log processing rule in the example above, first database table is not deposited according to any storage dimension Storage is player_buffer_cnt_sum_minute_%Y%m, player_ respectively wherein including three database tables Buffer_cnt_sum_hour_%Y%m and player_buffer_cnt_sum_day_%Y%m, %Y%m therein are to make With the timestamp combination of python, wherein %Y indicates the storage log corresponding time, and %m indicates that storage log is corresponding In month, such as the log information in the example above, log_time expression is currently in July, 2019, then corresponding database table are as follows: Player_buffer_cnt_sum_minute_201907, player_buffer_cnt_sum_hour_201907 with Player_buffer_cnt_sum_day_201907 realizes the strategy for dividing table automatically according to time scale, it is therefore prevented that single table Excessive problem.The corresponding time_delta in rear indicates the corresponding time scale of statistical data of table storage, such as table Player_buffer_cnt_sum_minute_201907 is to carry out statistics according to 1 minute for time scale to summarize, such as above-mentioned day Will information is the information for 2019-07-12 this minute of 09:41 summarized, and the bufcnt of the log information is 0, i.e. the log is believed (Numerical accuracy of bufcnt is by client relative program for the case where user of breath does not encounter Caton within that a bit of time Be responsible for, be not belonging to the contents of the present invention), only increase by 1 in total, by all clients equipment on line within this minute on The log information of report summarize union (example one in detailed process such as above-mentioned steps S502).Finally obtain 2019-07- In 12 this minutes of 09:41 the live streaming statistic of attribute of statistics calculating is not carried out as a result, in turn that this is straight according to any storage dimension Statistic of attribute result is broadcast to save into player_buffer_cnt_sum_minute_201907.

The corresponding live streaming quality of meaning and storage for other database tables of the log processing rule in the example above The process of statistical result is similar to the above process, and details are not described herein.

The management live streaming statistic of attribute that the mode of above-mentioned storage live streaming statistic of attribute result can be more clear is as a result, be convenient for Live streaming statistic of attribute result needed for subsequent more quick lookup.

After storing above-mentioned live streaming statistic of attribute result in server device, it will be able in response to the displaying for live streaming quality Operation, is shown the live streaming statistic of attribute result of storage.

This gives the modes that live streaming statistic of attribute result of the following two kinds to storage is shown, and are respectively as follows:

Mode one:

First determine that the data of server device show demand；Wherein, data show demand include: live streaming quality to be presented, Show that dimension, data show section and show time scale；According to live streaming quality to be presented, shows dimension and show time scale Determine display data library table；Display data library table be database table in live streaming quality to be presented, show dimension and displaying the time The database table that scale matches；From the data extracted in the table of display data library in data displaying section, and extraction is obtained Data are shown.

Mode two:

First obtain the data displaying request that the client of administrator is sent；Data, which are shown in request, carries live streaming to be presented Quality shows that dimension, data show section and show the parameter of time scale；According to live streaming quality to be presented, show dimension and Show that time scale determines display data library table；Display data library table is to tie up in database table with live streaming quality to be presented, displaying The database table that degree and displaying time scale match；From the data extracted in the table of display data library in data displaying section, and The client that the data that extraction obtains are sent to administrator is shown.

In above two exhibition method, the first exhibition method is the mode being shown in server device, second Kind mode is the mode being shown in the client of administrator, and the specific implementation process of two kinds of exhibition methods is similar, below Expansion explanation is only carried out by taking second of exhibition method as an example:

After the data that the client that server device gets administrator is sent show request, to the information wherein carried It is parsed, if parsing obtains the data and shows that the live streaming quality to be presented in request is Caton quality, displaying dimension is main broadcaster Dimension shows that time scale is 1 minute, then just first being determined for card in each log processing rule according to above- mentioned information The log processing rule of statistic of attribute, and then stored in the data of the log processing rule respectively for carrying out Caton statistic of attribute Main broadcaster's dimension is found in rule, as the example above log processing rule in, find in data storage rule in keys field Hold the field for being anchor, then, then the database table that determining time scale is 1 minute in keys field, as show number (in the log processing rule of such as the example above, meet the display data library table of above-mentioned requirements are as follows: player_buffer_ according to library table Cnt_anchor_%Y%m { time_delta:1M }), after obtaining display data library table, data are extracted from the table of display data library The number of (can be the section of certain time period, can also be Caton numerical digit in the section of main broadcaster etc. of preceding N) is shown in section According to finally the client for extracting obtained data and being sent to administrator is shown.It should be noted that in the visitor of administrator When being shown on the end of family, can the display form according to specified in the client of administrator be shown.As shown in fig. 6, it is given The displaying schematic diagram of the Caton quality for main broadcaster's dimension that time scale is 1 minute is gone out.In Fig. 6, most to preceding 10 Caton numbers The statistical result of big main broadcaster is shown.In practical operation, administrator need to only select to need the time zone of display data Between, and select to sort according to main broadcaster's dimension (i.e. main broadcaster CCID), search is clicked, figure shown in Fig. 6 can be obtained.It is bent in Fig. 6 Different horizontal lines, which are shown, above line chart represents different numbers, different main broadcaster's CCID (the i.e. log information of different digital representations In video_ccid, namely cleaning after anchor field in content).By this figure it can be found that the live streaming platform In, the main broadcaster ID of Caton the most is the user under 361433, and through counting, wherein average minute clock has 80 or so users that can meet The case where to Caton, continue the concrete condition for observing the main broadcaster therefore, it is necessary to administrative staff, investigation is confirmed whether to exist potential Problem is simultaneously handled in time.

Above-mentioned log processing rule can also update as needed, the update side of log processing rule shown in Figure 7 The flow chart of method, this method comprises the following steps:

Step S702 obtains the update request of log processing rule.

Wherein, the parameter and undated parameter that log processing rule to be updated is carried in request are updated；Undated parameter includes At least one of: the parameter of addition log processing rule modifies the parameter of log processing rule and deletes log processing rule Parameter.

In embodiments of the present invention, the update request of above-mentioned log processing rule can be the terminal device of project leader Be sent to server device update request (embodiment of the present invention, without concrete restriction, can also be administrator to which The terminal device of member is sent to update request of server device etc.), i.e., when administrative staff have new displaying demand, by its demand Inform project leader, and then project leader configures new log processing rule (i.e. further according to the displaying demand of administrative staff Above-mentioned update request in the application, certainly, updating request can also be other forms, such as: the update presented by code The form of request), then, the new log processing rule which is obtained is distributed to server device, to set to server-side The preset log processing rule of standby middle storage is updated.

Log processing rule as exemplified above is also easy to understand, so configuration is new it is found that its format and content are relatively fixed Log processing rule process be rapidly and efficiently, then, then the new log processing rule is distributed to server device Afterwards, server device be broadcast live the processing of log be also rapidly and efficiently, and can be convenient according to the demand of administrative staff To quality is broadcast live, the storage dimension of quality is broadcast live and the various rules that are directed to are added, delete and change, easily It is practical in operation.

Step S704 determines log processing rule to be updated according to the parameter of log processing rule to be updated, and according to more New parameter is treated Update log processing rule and is updated.

The above process can achieve the effect that with a specific example below and be illustrated:

In the log processing rule of the example above, rapidly and efficiently different storage dimension (i.e. aggs- can be configured Storage-keys configuration item), it is when if necessary to new storage dimension, then simple to increase a storage configuration, for it In the new storage dimension of keys field configuration, so that it is new to obtain this according to the processing method of above-mentioned live streaming log Storage dimension under live streaming quality.

Under practical application scene, is calculated if having carried out statistics by main broadcaster's dimension, it is most to illustrate Caton Several main broadcasters can be with to quickly have found wherein there is the main broadcaster of Caton situation, then when being analyzed for these main broadcasters Each main broadcaster is counted according to operator's dimension again and is calculated, confirms whether the Caton user of Caton main broadcaster both is from same Cdn operator (such as: the keys:[" anchor " in above-mentioned log processing rule, " cdn "] field), if so, then can be quick Cdn operator help processing follow-up is contacted, the processing speed of problem on line is greatly accelerated, reduces for user on line Influence.

Similarly, when simultaneously according to main broadcaster's dimension, city dimension, telecom operators' dimension (such as: above-mentioned log processing rule In keys:[" anchor ", " city ", " isp "]) carry out statistics calculate when, can quickly find under corresponding storage dimension With the presence or absence of abnormal conditions.And this programme provide method can be simple and quick configuration it is relevant storage dimension information, i.e., The live streaming quality under output associated storage dimension is generated, platform maintenance personnel is facilitated to configure according to oneself information of interest, And the live streaming quality under relevant dimension is obtained immediately, the operating condition for keeping maintenance personnel current for live streaming platform carries out comprehensive Solution, but also the verification analysis of platform live streaming problem is highly efficient.

Furthermore it is also possible to configure new live streaming quality, for example only carried out by taking Caton quality as an example in the embodiment of the present invention Illustrate, when network delay quality to be obtained, configuration rapidly and efficiently can also be carried out, and then the network that obtains rapidly and efficiently prolongs Information of the slow quality in each storage dimension.

The update method of above-mentioned log processing rule can carry out rapidly and efficiently more live streaming quality and storage dimension New configuration, and then combine the processing method of live streaming log to realize the live streaming quality from original log information to target and tieed up in each storage Rapidly and efficiently conversion on degree, to facilitate the live streaming quality condition that administrative staff learn full platform, and for therein thin Weak link is further strengthened and is promoted.

The embodiment of the invention also provides a kind of processing unit that log is broadcast live, the processing unit of the live streaming log is applied to The server device of platform is broadcast live, preset log processing rule is stored in server device, log processing rule includes day Will cleaning rule, data statistics rule and data storage rule, and the processing unit of the live streaming log is mainly used for executing this hair The processing method of live streaming log provided by bright embodiment above content, below to live streaming log provided in an embodiment of the present invention Processing unit does specific introduction.

Fig. 8 is a kind of schematic diagram of processing unit that log is broadcast live according to an embodiment of the present invention, as shown in figure 8, this is straight The processing unit for broadcasting log mainly includes first acquisition unit 10, cleaning unit 20, counts computing unit 30 and storage unit 40, Wherein:

First acquisition unit, the log information that the client device for obtaining live streaming platform is sent；Log information includes: The attribute information of quality index information and client device, attribute information are associated with the storage dimension in data storage rule；

Cleaning unit, for being cleaned according to log cleaning rule to log information；

Statistics computing unit is obtained for carrying out statistics calculating to the log information after cleaning according to data statistics rule The corresponding live streaming statistic of attribute result of each storage dimension；

Storage unit, for it is corresponding to store each storage dimension according to each storage dimension in data storage rule Statistic of attribute result is broadcast live.

In the processing unit of above-mentioned live streaming log provided in an embodiment of the present invention, it is stored in server device preset Log processing rule, the log processing rule in include log cleaning rule, data statistics rule and data storage rule, into When the processing of row live streaming log, the log information that the client device of live streaming platform is sent first is obtained, according to log cleaning rule The log information is cleaned, statistics calculating is carried out to the log information after cleaning according still further to the data statistics rule, and press According to each storage dimension in the data storage rule, the corresponding live streaming statistic of attribute result of each storage dimension is stored.This The mode of statistics calculating kind is carried out to live streaming log (log information that the client device of i.e. above-mentioned live streaming platform is sent), it can be with Obtain reliable live streaming statistic of attribute as a result, and the live streaming statistic of attribute result stored according to each storage dimension, It is easy to know the live streaming quality information in each dimension, the development that advanced optimizes for subsequent live streaming platform provides reliable number According to alleviating the prior art cannot achieve live streaming quality the express statistic on various dimensions the technical issues of.

Correspondingly, log processing rule includes multiple, and each log processing rule further includes having index field；Above-mentioned apparatus Further include: matching unit, it is true according to matching result for matching log information with the index field of each log processing rule The matched log processing rule of fixed and log information, unites using the log cleaning rule in matched log processing rule, data Meter rule and data storage rule clean log information, count calculating and storage processing.

Above-mentioned first acquisition unit is also used to: receiving the log information that the client device of live streaming platform reports；Client Equipment reports log information during live streaming, according to the period 1 of setting；And/or during client device live streaming, Log information is read from the designated storage area of client device according to the second round of setting.

Correspondingly, log cleaning rule includes: duplicate removal rule, mapping ruler and field supplement rule；Above-mentioned cleaning unit It is also used to: log duplicate removal processing being carried out to log information according to duplicate removal rule；According to mapping ruler to the log information after duplicate removal Carry out field mapping processing；Field supplement process is carried out to field mapping treated log information according to field supplement rule, Log information after obtaining the complete cleaning of data field.

Correspondingly, log cleaning rule further includes having splicing rule；Above-mentioned cleaning unit is also used to: being mentioned in log information Take log identification information；Log identification information includes: mark title and ident value；Log identification information is spelled according to splicing rule Connect processing；The keyword comparison that will be stored in keyword and keyword database that splicing obtains, determines that splicing obtains To keyword and keyword database in the keyword that stores whether repeat；If repeating, what discarding was obtained with splicing Log information corresponding to keyword；If not repeating, the keyword that splicing is obtained is saved to keyword database, and The effective time for the keyword that setting splicing obtains；When effective time is used to indicate that the keyword corresponding to it to be effective Between；Log information corresponding to the keyword in obtained keyword database will be updated as the log information after duplicate removal.

Above-mentioned cleaning unit is also used to: corresponding with field name in the log information after duplicate removal according to mapping ruler determination Aiming field title；Word corresponding with aiming field title in log information after replacing duplicate removal using aiming field title Name section obtains field mapping treated log information.

Above-mentioned cleaning unit is also used to: according to field supplement rule to the target in field mapping treated log information Field information carries out increasing changed handling, the log information after obtaining the complete cleaning of data field；Aiming field information is field benefit Fill field information defined in rule.

Correspondingly, data statistics rule is the data statistics rule for carrying time scale, storing dimension；Above-mentioned statistics meter It calculates unit to be also used to: the aimed quality indication information in log information after obtaining cleaning；Aimed quality indication information be and when Between scale, the quality index information that matches of storage dimension；Aimed quality indication information is counted according to data statistics rule Calculation processing obtains each storage dimension in the live streaming statistic of attribute result of its corresponding time scale.

Said memory cells are also used to: being determined according to storage dimension corresponding to live streaming statistic of attribute result and time scale Target database table；Target database table and the database table to match for storage dimension and time scale；It will live streaming quality system Meter result is saved into target database table.

Above-mentioned apparatus further include: display unit, for the live streaming in response to the displaying operation for live streaming quality, to storage Statistic of attribute result is shown.

Above-mentioned display unit is also used to: determining that the data of server device show demand；Data show that demand includes: wait open up Show live streaming quality, show that dimension, data show section and show time scale；According to live streaming quality to be presented, show dimension and Show that time scale determines display data library table；Display data library table is to tie up in database table with live streaming quality to be presented, displaying The database table that degree and displaying time scale match；From the data extracted in the table of display data library in data displaying section, and The data obtained to extraction are shown.

Above-mentioned display unit is also used to: obtaining the data displaying request that the client of administrator is sent；Data show request In carry it is to be presented live streaming quality, show dimension, data show section and show time scale parameter；According to be presented straight Quality is broadcast, show dimension and shows that time scale determines display data library table；Display data library table is in database table and wait open up Show live streaming quality, show dimension and shows the database table that time scale matches；Data exhibition is extracted from the table of display data library Show the data in section, and the client that the data that extraction obtains are sent to administrator is shown.

The technical effect of the processing unit of live streaming log provided by the embodiment of the present invention, realization principle and generation is with before The embodiment of the method stated in the processing method of live streaming log is identical, and to briefly describe, Installation practice part does not refer to place, can With reference to corresponding contents in preceding method embodiment.

Fig. 9 is a kind of schematic diagram of processing unit that log is broadcast live according to an embodiment of the present invention, as shown in figure 9, this is straight It further include second acquisition unit 50 and updating unit 60 on the basis of broadcasting the processing unit device shown in Fig. 8 of log, in which:

Second acquisition unit, the update for obtaining log processing rule are requested；It updates in request and carries day to be updated The parameter and undated parameter of will processing rule；Undated parameter includes at least one of: the parameter of addition log processing rule is repaired Change the parameter of log processing rule and deletes the parameter of log processing rule；

Updating unit, for determining log processing rule to be updated, and root according to the parameter of log processing rule to be updated Update log processing rule is treated according to undated parameter to be updated.

The technical effect and preceding method embodiment phase of device provided by the embodiment of the present invention, realization principle and generation Together, to briefly describe, Installation practice part does not refer to place, can refer to corresponding contents in preceding method embodiment.

The embodiment of the invention also provides a kind of electronic equipment, for running the processing method of above-mentioned live streaming log；Referring to Shown in Figure 10, which includes memory 101 and processor 102, wherein memory 101 is for storing one or more Computer instruction, one or more computer instruction are executed by processor 102, to realize the processing method of above-mentioned live streaming log.

Further, electronic equipment shown in Fig. 10 further includes bus 103 and communication interface 104, processor 102, communication Interface 104 and memory 101 are connected by bus 103.

Wherein, memory 101 may include high-speed random access memory (RAM, Random Access Memory), It may further include non-labile memory (non-volatile memory), for example, at least a magnetic disk storage.By extremely A few communication interface 104 (can be wired or wireless) is realized logical between the system network element and at least one other network element Letter connection, can be used internet, wide area network, local network, Metropolitan Area Network (MAN) etc..Bus 103 can be isa bus, pci bus or Eisa bus etc..Bus can be divided into address bus, data/address bus, control bus etc..For convenient for indicating, only with one in Figure 10 Four-headed arrow indicates, it is not intended that an only bus or a type of bus.

Processor 102 may be a kind of IC chip, the processing capacity with signal.It is above-mentioned during realization Each step of method can be completed by the integrated logic circuit of the hardware in processor 102 or the instruction of software form.On The processor 102 stated can be general processor, including central processing unit (Central Processing Unit, abbreviation CPU), network processing unit (Network Processor, abbreviation NP) etc.；It can also be digital signal processor (Digital Signal Processor, abbreviation DSP), specific integrated circuit (Application Specific Integrated Circuit, abbreviation ASIC), field programmable gate array (Field-Programmable Gate Array, abbreviation FPGA) or Person other programmable logic device, discrete gate or transistor logic, discrete hardware components.It may be implemented or execute sheet Disclosed each method, step and logic diagram in inventive embodiments.General processor can be microprocessor or the processing Device is also possible to any conventional processor etc..The step of method in conjunction with disclosed in the embodiment of the present invention, can be embodied directly in Hardware decoding processor executes completion, or in decoding processor hardware and software module combination execute completion.Software mould Block can be located at random access memory, flash memory, read-only memory, programmable read only memory or electrically erasable programmable storage In the storage medium of this fields such as device, register maturation.The storage medium is located at memory 101, and processor 102 reads memory Information in 101, in conjunction with its hardware complete previous embodiment method the step of.

The embodiment of the invention also provides a kind of computer readable storage medium, which has Machine-executable instruction, when being called and being executed by processor, machine-executable instruction promotes to handle the machine-executable instruction Device realizes the processing method of above-mentioned live streaming log, and specific implementation can be found in embodiment of the method, and details are not described herein.

The computer program of the processing method of live streaming log, device and electronic equipment provided by the embodiment of the present invention produces Product, the computer readable storage medium including storing program code, before the instruction that said program code includes can be used for execution Method described in the embodiment of the method for face, specific implementation can be found in embodiment of the method, and details are not described herein.

It is apparent to those skilled in the art that for convenience and simplicity of description, the device of foregoing description And/or the specific work process of electronic equipment, it can refer to corresponding processes in the foregoing method embodiment, details are not described herein.

Finally, it should be noted that embodiment described above, only a specific embodiment of the invention, to illustrate the present invention Technical solution, rather than its limitations, scope of protection of the present invention is not limited thereto, although with reference to the foregoing embodiments to this hair It is bright to be described in detail, those skilled in the art should understand that: anyone skilled in the art In the technical scope disclosed by the present invention, it can still modify to technical solution documented by previous embodiment or can be light It is readily conceivable that variation or equivalent replacement of some of the technical features；And these modifications, variation or replacement, do not make The essence of corresponding technical solution is detached from the spirit and scope of technical solution of the embodiment of the present invention, should all cover in protection of the invention Within the scope of.Therefore, the protection scope of the present invention shall be subject to the protection scope of the claims.

Claims

1. a kind of processing method that log is broadcast live, which is characterized in that applied to the server device of live streaming platform, the server-side Preset log processing rule is stored in equipment, the log processing rule includes log cleaning rule, data statistics rule With data storage rule, which comprises

Obtain the log information that the client device of the live streaming platform is sent；The log information includes: quality index information With the attribute information of the client device, the attribute information is associated with the storage dimension in the data storage rule；

The log information is cleaned according to the log cleaning rule；

Statistics calculating is carried out to the log information after cleaning according to the data statistics rule, obtains each storage dimension Spend corresponding live streaming statistic of attribute result；

According to each storage dimension in the data storage rule, the corresponding live streaming matter of each storage dimension is stored Measure statistical result.

2. the method according to claim 1, wherein the log processing rule includes multiple, each day It further includes having index field that will, which handles rule,；

After the step of obtaining the log information of the client device transmission of the live streaming platform, the method also includes:

The log information is matched with the index field of each log processing rule, according to matching result it is determining with it is described The matched log processing rule of log information is united using the log cleaning rule in the matched log processing rule, data Meter rule and data storage rule clean the log information, count calculating and storage processing.

3. the method according to claim 1, wherein the day that the client device for obtaining the live streaming platform is sent The step of will information includes:

Receive the log information that the client device of the live streaming platform reports；The client device is pressed during live streaming The log information is reported according to the period 1 of setting；

And/or

During client device live streaming, according to the second round of setting from the designated storage area of the client device Read log information.

4. the method according to claim 1, wherein the log cleaning rule includes: duplicate removal rule, mapping rule Then rule is supplemented with field；

The step of cleaning according to the log cleaning rule to the log information include:

Log duplicate removal processing is carried out to the log information according to the duplicate removal rule；

Field mapping processing is carried out to the log information after duplicate removal according to the mapping ruler；

Field supplement process is carried out to field mapping treated log information according to field supplement rule, obtains data word The log information after the complete cleaning of section.

5. according to the method described in claim 4, it is characterized in that, the log cleaning rule further includes having splicing rule；

Include: to the step of log information progress log duplicate removal processing according to the duplicate removal rule

Log identification information is extracted in the log information；The log identification information includes: mark title and ident value；

According to the splicing rule to the log identification information splicing；

The keyword comparison that will be stored in keyword and keyword database that splicing obtains, determines what splicing obtained Whether the keyword stored in the keyword and the keyword database repeats；

If repeating, log information corresponding to the keyword obtained with splicing is abandoned；

If not repeating, the keyword that the splicing obtains is saved to the keyword database, and set stitching portion Manage the effective time of the obtained keyword；The effective time is used to indicate the keyword effective time corresponding to it；

Log information corresponding to the keyword in the obtained keyword database will be updated as the day after duplicate removal Will information.

6. according to the method described in claim 4, it is characterized in that, according to the mapping ruler to the log information after duplicate removal into Row field mapping processing the step of include:

Aiming field title corresponding with field name in the log information after duplicate removal is determined according to the mapping ruler；

It is corresponding with the aiming field title in the log information after replacing duplicate removal using the aiming field title Field name obtains field mapping treated the log information.

7. according to the method described in claim 4, it is characterized in that, after handling according to field supplement rule field mapping Log information carry out field supplement process the step of include:

The aiming field information in field mapping treated the log information is increased according to field supplement rule Changed handling, the log information after obtaining the complete cleaning of data field；The aiming field information is field supplement Field information defined in rule.

8. the method according to claim 1, wherein the data statistics rule is to carry time scale, deposit Store up the data statistics rule of dimension；

Statistics calculating is carried out to the log information after cleaning according to the data statistics rule, obtains each storage dimension The step of spending corresponding live streaming statistic of attribute result include:

The aimed quality indication information in the log information after obtaining cleaning；The aimed quality indication information be with it is described The quality index information that time scale, the storage dimension match；

Calculation processing is carried out to the aimed quality indication information according to the data statistics rule, obtains each storage dimension Spend the live streaming statistic of attribute result in its corresponding time scale.

9. according to the method described in claim 8, it is characterized in that, storing the corresponding live streaming quality system of each storage dimension Count result the step of include:

Target database table is determined according to storage dimension corresponding to the live streaming statistic of attribute result and time scale；The mesh Mark database table and the database table to match for the storage dimension and the time scale；

The live streaming statistic of attribute result is saved into the target database table.

10. the method according to claim 1, wherein the method also includes:

In response to the displaying operation for live streaming quality, the live streaming statistic of attribute result of storage is shown.

11. according to the method described in claim 10, it is characterized in that, in response to being operated for the displaying of live streaming quality, to depositing Live streaming statistic of attribute result the step of being shown of storage includes:

Determine that the data of the server device show demand；The data show that demand includes: live streaming quality to be presented, displaying Dimension, data show section and show time scale；

Display data library table is determined according to live streaming quality, the displaying dimension and the displaying time scale to be presented；Institute State display data library table be database table in live streaming quality, the displaying dimension and the displaying time scale to be presented The database table to match；

From the data extracted in data displaying section in the table of the display data library, and the data obtained to extraction are opened up Show.

12. according to the method described in claim 10, it is characterized in that, in response to being operated for the displaying of live streaming quality, to depositing The step of live streaming statistic of attribute result of storage is shown further include:

Obtain the data displaying request that the client of administrator is sent；The data, which are shown in request, carries live streaming matter to be presented Amount shows that dimension, data show section and show the parameter of time scale；

From the data extracted in the table of the display data library in data displaying section, and it is sent to obtained data are extracted The client of the administrator is shown.

13. the method according to claim 1, wherein the method also includes:

Obtain the update request of log processing rule；It is described update carried in request the parameter of log processing rule to be updated with Undated parameter；The undated parameter includes at least one of: parameter, the modification log processing rule of addition log processing rule Parameter and delete log processing rule parameter；

The log processing rule to be updated is determined according to the parameter of the log processing rule to be updated, and according to the update Parameter is updated the log processing rule to be updated.

14. a kind of processing unit that log is broadcast live, which is characterized in that applied to the server device of live streaming platform, the service Preset log processing rule is stored in end equipment, the log processing rule includes log cleaning rule, data statistics rule Then with data storage rule, described device includes:

First acquisition unit, the log information that the client device for obtaining the live streaming platform is sent；The log information It include: the attribute information of quality index information and the client device, in the attribute information and the data storage rule Storage dimension association；

Computing unit is counted, for carrying out statistics calculating to the log information after cleaning according to the data statistics rule, Obtain the corresponding live streaming statistic of attribute result of each storage dimension；

Storage unit, for storing each storage dimension according to each storage dimension in the data storage rule Spend corresponding live streaming statistic of attribute result.

15. device according to claim 14, which is characterized in that described device further include:

Second acquisition unit, the update for obtaining log processing rule are requested；Day to be updated is carried in the update request The parameter and undated parameter of will processing rule；The undated parameter includes at least one of: the ginseng of addition log processing rule The parameter of number, the parameter of modification log processing rule and deletion log processing rule；

Updating unit, for determining the log processing rule to be updated according to the parameter of the log processing rule to be updated, And the log processing rule to be updated is updated according to the undated parameter.

16. a kind of electronic equipment, which is characterized in that including processor and memory, the memory is stored with can be described The machine-executable instruction that processor executes, the processor execute the machine-executable instruction with realize claim 1 to The step of processing method of 13 described in any item live streaming logs.

17. a kind of computer readable storage medium, which is characterized in that the computer-readable recording medium storage has machine that can hold Row instruction, for the machine-executable instruction when being called and being executed by processor, the machine-executable instruction promotes the place Manage the step of device realizes the processing method of the described in any item live streaming logs of claim 1 to 13.