CN107124320A - Traffic data monitoring method and device and server - Google Patents
Traffic data monitoring method and device and server Download PDFInfo
- Publication number
- CN107124320A CN107124320A CN201710524384.0A CN201710524384A CN107124320A CN 107124320 A CN107124320 A CN 107124320A CN 201710524384 A CN201710524384 A CN 201710524384A CN 107124320 A CN107124320 A CN 107124320A
- Authority
- CN
- China
- Prior art keywords
- data
- flows
- predicted
- current log
- log data
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L41/00—Arrangements for maintenance, administration or management of data switching networks, e.g. of packet switching networks
- H04L41/14—Network analysis or design
- H04L41/147—Network analysis or design for predicting network behaviour
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L41/00—Arrangements for maintenance, administration or management of data switching networks, e.g. of packet switching networks
- H04L41/14—Network analysis or design
- H04L41/145—Network analysis or design involving simulating, designing, planning or modelling of a network
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L43/00—Arrangements for monitoring or testing data switching networks
- H04L43/06—Generation of reports
- H04L43/067—Generation of reports using time frame reporting
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/50—Network services
- H04L67/535—Tracking the activity of the user
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L41/00—Arrangements for maintenance, administration or management of data switching networks, e.g. of packet switching networks
- H04L41/06—Management of faults, events, alarms or notifications
- H04L41/0681—Configuration of triggering conditions
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L41/00—Arrangements for maintenance, administration or management of data switching networks, e.g. of packet switching networks
- H04L41/06—Management of faults, events, alarms or notifications
- H04L41/069—Management of faults, events, alarms or notifications using logs of notifications; Post-processing of notifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L43/00—Arrangements for monitoring or testing data switching networks
- H04L43/02—Capturing of monitoring data
- H04L43/028—Capturing of monitoring data by filtering
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L43/00—Arrangements for monitoring or testing data switching networks
- H04L43/16—Threshold monitoring
Landscapes
- Engineering & Computer Science (AREA)
- Computer Networks & Wireless Communication (AREA)
- Signal Processing (AREA)
- Computer Hardware Design (AREA)
- General Engineering & Computer Science (AREA)
- Management, Administration, Business Operations System, And Electronic Commerce (AREA)
Abstract
The invention discloses a method, a device and a server for monitoring flow data, wherein the method for monitoring the flow data comprises the following steps: acquiring current log data; preprocessing current log data to generate flow data to be predicted; inputting flow data to be predicted into a pre-trained deep learning model; outputting a prediction result corresponding to flow data to be predicted through a pre-trained deep learning model; and monitoring the flow data according to the prediction result. According to the traffic data monitoring method, the traffic data monitoring device and the traffic data monitoring server, the deep learning model is adopted to monitor the traffic data, so that the accuracy of the traffic data is improved, the complexity of traffic data monitoring is reduced, the operation cost is reduced, and the traffic data monitoring method, the traffic data monitoring device and the traffic data monitoring server are more intelligent.
Description
Technical field
The present invention relates to Internet technical field, more particularly to a kind of monitoring method of data on flows, device and server.
Background technology
The monitoring and control of internet traffic, have highly important effect for moving advertising system.It can help matchmaker
Body is controlled and monitoring data on flows, and is fed back in time.At present, it is main that method is monitored using threshold value, for different media, from
Different dimensions, using different threshold values, monitors axle, to be monitored to data on flows using the time as advancing.When certain is one-dimensional
When the numerical value of degree exceeds its corresponding threshold value, mail alarm can be carried out or directly closed down.Media are graded, can be high-quality
Advertisement distribute high-quality media, so as to reach benefit.But, above-mentioned monitoring method is, it is necessary to which business personnel is subjective
Judge, high is required to specialty, and be adjusted correspondingly difficulty, not enough intelligently.
The content of the invention
It is contemplated that at least solving one of technical problem in correlation technique to a certain extent.Therefore, the present invention
One purpose is to propose a kind of monitoring method of data on flows, data on flows is supervised by using deep learning model
Control, improves the degree of accuracy of data on flows, reduces the complexity of data on flows monitoring, cuts operating costs, more intelligent.
Second object of the present invention is to propose a kind of supervising device of data traffic.
Third object of the present invention is to propose a kind of server.
Fourth object of the present invention is to propose a kind of storage medium.
The 5th purpose of the present invention is to propose a kind of application program.
To achieve these goals, first aspect present invention embodiment proposes a kind of monitoring method of data traffic, bag
Include：Obtain current log data；
The current log data are pre-processed, data on flows to be predicted is generated；
The data on flows to be predicted is inputted to the deep learning model of training in advance；
Predicted the outcome by the way that the deep learning model output data on flows to be predicted of the training in advance is corresponding；
Data on flows is monitored according to described predict the outcome.
Optionally, the current log data are pre-processed, generates data on flows to be predicted, including：
The current log data are classified；
Corresponding current log data are cleaned and extracted according to classification；
Regularization processing is carried out to the current log data, and generates data on flows to be predicted.
Optionally, corresponding current log data are cleaned and extracted according to classification, including：
The noise data in the current log data is filtered according to default filtering rule.
Optionally, the deep learning model of the training in advance includes range model and depth model, the range model
Connected with the depth model by logic loss function.
Optionally, data on flows is monitored according to described predict the outcome, including：
When described predict the outcome beyond predetermined threshold value, warning information is generated；
Feedback information of the business personnel to the warning information is received, and the warning information and the feedback information are entered
Row statistics；
The data on flows is graded according to statistical information.
Optionally, the monitoring method of data traffic also includes：Train the deep learning model.
Optionally, the deep learning model is trained, including：
Obtain history log data；
The history log data is pre-processed, historical traffic data is generated；
The deep learning model is trained according to the historical traffic data.
The monitoring method of the data traffic of the embodiment of the present invention, is supervised by using deep learning model to data on flows
Control, improves the degree of accuracy of data on flows, reduces the complexity of data on flows monitoring, cuts operating costs, more intelligent.
Second aspect of the present invention embodiment proposes a kind of supervising device of data traffic, including：
Acquisition module, for obtaining current log data；Pretreatment module, it is pre- for being carried out to the current log data
Processing, generates data on flows to be predicted；
Input module, for the data on flows to be predicted to be inputted to the deep learning model of training in advance；
Output module, the data on flows correspondence to be predicted is exported for the deep learning model by the training in advance
Predict the outcome；
Monitoring module, is monitored for being predicted the outcome according to data on flows.
Optionally, the pretreatment module, is used for：The current log data are classified；According to classification to corresponding
Current log data cleaned and extracted；
Regularization processing is carried out to the current log data, and generates data on flows to be predicted.
Optionally, the pretreatment module, is used for：Making an uproar in the current log data is filtered according to default filtering rule
Sound data.
Optionally, the deep learning model of the training in advance includes range model and depth model, the range model
Connected with the depth model by logic loss function.
Optionally, the monitoring module, is used for：When described predict the outcome beyond predetermined threshold value, warning information is generated；
Feedback information of the business personnel to the warning information is received, and the warning information and the feedback information are entered
Row statistics；The data on flows is graded according to statistical information.
Optionally, described device also includes：Training module, for training the deep learning model.
Optionally, the training module, is used for：Obtain history log data；The history log data is located in advance
Reason, generates historical traffic data；The deep learning model is trained according to the historical traffic data.
The supervising device of the data traffic of the embodiment of the present invention, is supervised by using deep learning model to data on flows
Control, improves the degree of accuracy of data on flows, reduces the complexity of data on flows monitoring, cuts operating costs, more intelligent.
Third aspect present invention embodiment proposes a kind of server, including：Processor, memory, communication interface and total
Line；The processor, the memory and the communication interface are connected by the bus and complete mutual communication；It is described
Memory storage executable program code；The processor by read the executable program code stored in the memory come
Operation program corresponding with the executable program code, for performing the prison of the data on flows described in first aspect embodiment
Prosecutor method.
Fourth aspect present invention embodiment proposes a kind of storage medium, wherein, the storage medium, which is used to store, to be applied
Program, the application program is used for the monitoring method for operationally performing the data on flows described in first aspect embodiment.
Fifth aspect present invention embodiment proposes a kind of application program, wherein, the application program is used for operationally
Perform the monitoring method of the data on flows described in first aspect embodiment.
Brief description of the drawings
Fig. 1 is the flow chart of the monitoring method of data on flows according to an embodiment of the invention；
Fig. 2 is the flow chart of the monitoring method of data on flows in accordance with another embodiment of the present invention；
Fig. 3 is the structural representation of the supervising device of data on flows according to an embodiment of the invention；
Fig. 4 is the structural representation of the supervising device of data on flows in accordance with another embodiment of the present invention；
Fig. 5 is the structural representation of server according to an embodiment of the invention.
Embodiment
Embodiments of the invention are described below in detail, the example of the embodiment is shown in the drawings, wherein from beginning to end
Same or similar label represents same or similar element or the element with same or like function.Below with reference to attached
The embodiment of figure description is exemplary, it is intended to for explaining the present invention, and be not considered as limiting the invention.
Below with reference to the accompanying drawings monitoring method, device and the server of the data on flows of the embodiment of the present invention described.
Fig. 1 is the flow chart of the monitoring method of data on flows according to an embodiment of the invention.
As shown in figure 1, the monitoring method of data on flows may include：
S101, obtains current log data.
In one embodiment of the invention, when being monitored to data on flows, current log data can be obtained.Currently
Daily record data may include showing advertisement amount, ad click amount, ad click rate, ad conversion rates and advertising income.Current log
Minute, hour, day, moon etc. can be used as magnitude unit by data, for example, the daily record of nearest 10 minutes can be taken, as current
Daily record data.
Current log data are pre-processed by S102, generate data on flows to be predicted.
After current log data are obtained, current log data can be pre-processed, generate data on flows to be predicted.
In one embodiment of the invention, current log data can be classified first, then according to classification to phase
The current log data answered are cleaned and extracted, then carry out regularization processing to current log data, and generate stream to be predicted
Measure data.For example：Current log data, which can be divided into, shows click data, installation data, metering data.Wherein, hits are showed
According to can be showing advertisement amount, ad click amount；Installation data can be ad conversion rates；Metering data can be that advertisement is received
Enter.Different classification is directed to respectively, and above-mentioned data are cleaned and extracted, so as to generate data on flows to be predicted.Wherein, may be used
To filter the noise data in current log data according to default filtering rule.Certainly, it may also include it in current log data
His type, is such as used for the type for describing to show advertisement terminal, is mobile phone tablet personal computer or computer, such data are to reality
The traffic monitoring on border does not produce influence, therefore can be cleaned.Advised in addition, different filterings can be set according to different demands
Then, for example：Some demands only focus on the high data of ad click amount, then may filter that ad click amount is less than the data of 100 times；
Some demand concern ad conversion rates and advertising income, then may filter that the data such as showing advertisement amount, ad click amount.
S103, data on flows to be predicted is inputted to the deep learning model of training in advance.
Wherein, the deep learning model of training in advance may include range model and depth model, range model and depth mould
Type is connected by a logic loss function.
Range model is a general linear model：Y=Wtx+b.Wherein, y is predicted value, x=x1, x2 ...,
Xn } it is the vector that n feature is constituted, w={ w1, w2 ..., wn } is model parameter, and b is deviation.
Depth model is a feed forward neural networking, embedded vectorial random initializtion.Used in low latitude insertion vector
Hiding layer, there is following calculating function in each hidden layer：a(l+1)=f (w(l)a(l)+b(l)), wherein l is the number of plies, and f is to excite
Function, w(l)It is l layers of skew and excitation values of Model Weight.
Range model and depth model are connected by a logic loss function.FTRL (Follow can be used
TheRegularized Leader) algorithm, as the optimization of range model, the excellent of depth model is used as by the use of AdaGrad algorithms
Change.Finally, generation deep learning model P (Y=1 | X)=σ (WT wide[X, φ (X)]+WT deep a(lf)+ b), wherein, Y is two-value
Tag along sort, σ is S function, and φ is the middle conversion of original feature, and b is deviant, WT wideRange vector model weight to
Amount, WT deepIt is depth model weight vectors.
In one embodiment of the invention, data on flows to be predicted can be inputted to the deep learning mould of training in advance
Type.
S104, predicts the outcome by the way that the deep learning model output data on flows to be predicted of training in advance is corresponding.
Input to deep learning model, carried out using the deep learning model pre- by data on flows to be predicted
Survey, so as to export, data on flows to be predicted is corresponding to predict the outcome.For example：By the ad click amount of nearest 1 hour, predict
Subsequent time period such as the ad click amount in 1 hour future.Again for example：The advertising income of nearest 10 minutes is 100,000 yuan, then may be used
Predict advertising income of following half an hour etc..
S105, is monitored according to predicting the outcome to data on flows.
After output predicts the outcome, data on flows can be monitored according to predicting the outcome.
In one embodiment of the invention, when predicting the outcome beyond predetermined threshold value, warning information can be generated, to carry
Business personnel's data on flows of waking up occurs abnormal.Business personnel can be directed to abnormal information, the feedback operation such as be adjusted correspondingly.
After business personnel is received to the feedback information of warning information, warning information and feedback information can be counted, then basis
Statistical information is graded to data on flows.As an example it is assumed that following one day showing advertisement amount of prediction, if predicted the outcome
Compared to the previous day data on flows incrementss beyond 20%, then it is believed that the result is abnormal, early warning letter can be generated
Breath, reminding business personnel are handled in time.The amount of showing of the advertisement is for example reduced, it is tended to be normal.Then, can be according to life
Into the number of times of warning information etc., to evaluate data on flows, then matching for high-quality advertisement and high-quality flow, reality realized based on grading
Existing precise control of flew, and benefit.
The monitoring method of the data on flows of the embodiment of the present invention, is supervised by using deep learning model to data on flows
Control, improves the degree of accuracy of data on flows, reduces the complexity of data on flows monitoring, cuts operating costs, more intelligent.
Fig. 2 is the flow chart of the monitoring method of data on flows in accordance with another embodiment of the present invention.
As shown in Fig. 2 the monitoring method of data on flows may include：
S201, trains deep learning model.
In one embodiment of the invention, history log data can be obtained, history log data is pre-processed, it is raw
Into historical traffic data, then deep learning model can be trained according to historical traffic data.For example：It can be wrapped in history log data
Include showing advertisement amount, the ad click amount of history, the ad click rate of history, the ad conversion rates of history and the history of history
The daily historical data of advertising income, such as the whole of last year.According to above-mentioned historical data, it the pre- place such as can be cleaned, be extracted to it
Reason, generates historical traffic data.Using the feature of historical traffic data, training vector is converted into, and then train deep learning mould
Type.
S202, obtains current log data.
In one embodiment of the invention, when being monitored to data on flows, current log data can be obtained.Currently
Daily record data may include showing advertisement amount, ad click amount, ad click rate, ad conversion rates and advertising income.Current log
Minute, hour, day, moon etc. can be used as magnitude unit by data, for example, the daily record of nearest 10 minutes can be taken, as current
Daily record data.
Current log data are pre-processed by S203, generate data on flows to be predicted.
After current log data are obtained, current log data can be pre-processed, generate data on flows to be predicted.
In one embodiment of the invention, current log data can be classified first, then according to classification to phase
The current log data answered are cleaned and extracted, then carry out regularization processing to current log data, and generate stream to be predicted
Measure data.For example：Current log data, which can be divided into, shows click data, installation data, metering data.Wherein, hits are showed
According to can be showing advertisement amount, ad click amount；Installation data can be ad conversion rates；Metering data can be that advertisement is received
Enter.Different classification is directed to respectively, and above-mentioned data are cleaned and extracted, so as to generate data on flows to be predicted.Wherein, may be used
To filter the noise data in current log data according to default filtering rule.Certainly, it may also include it in current log data
His type, is such as used for the type for describing to show advertisement terminal, is mobile phone tablet personal computer or computer, such data are to reality
The traffic monitoring on border does not produce influence, therefore can be cleaned.Advised in addition, different filterings can be set according to different demands
Then, for example：Some demands only focus on the high data of ad click amount, then may filter that ad click amount is less than the data of 100 times；
Some demand concern ad conversion rates and advertising income, then may filter that the data such as showing advertisement amount, ad click amount.
S204, data on flows to be predicted is inputted to the deep learning model of training in advance.
Wherein, the deep learning model of training in advance may include range model and depth model, range model and depth mould
Type is connected by a logic loss function.
Range model is a general linear model：Y=Wtx+b.Wherein, y is predicted value, x=x1, x2 ...,
Xn } it is the vector that n feature is constituted, w={ w1, w2 ..., wn } is model parameter, and b is deviation.
Depth model is a feed forward neural networking, embedded vectorial random initializtion.Used in low latitude insertion vector
Hiding layer, there is following calculating function in each hidden layer：a(l+1)=f (w(l)a(l)+b(l)), wherein l is the number of plies, and f is to excite
Function, w(l)It is l layers of skew and excitation values of Model Weight.
Range model and depth model are connected by a logic loss function.FTRL (Follow can be used
TheRegularized Leader) algorithm, as the optimization of range model, the excellent of depth model is used as by the use of AdaGrad algorithms
Change.Finally, generation deep learning model P (Y=1 | X)=σ (WT wide[X, φ (X)]+WT deep a(lf)+ b), wherein, Y is two-value
Tag along sort, σ is S function, and φ is the middle conversion of original feature, and b is deviant, WT wideRange vector model weight to
Amount, WT deepIt is depth model weight vectors.
In one embodiment of the invention, data on flows to be predicted can be inputted to the deep learning mould of training in advance
Type.
S205, predicts the outcome by the way that the deep learning model output data on flows to be predicted of training in advance is corresponding.
Input to deep learning model, carried out using the deep learning model pre- by data on flows to be predicted
Survey, so as to export, data on flows to be predicted is corresponding to predict the outcome.For example：By the ad click amount of nearest 1 hour, predict
Subsequent time period such as the ad click amount in 1 hour future.Again for example：The advertising income of nearest 10 minutes is 100,000 yuan, then may be used
Predict advertising income of following half an hour etc..
S206, is monitored according to predicting the outcome to data on flows.
After output predicts the outcome, data on flows can be monitored according to predicting the outcome.
In one embodiment of the invention, when predicting the outcome beyond predetermined threshold value, warning information can be generated, to carry
Business personnel's data on flows of waking up occurs abnormal.Business personnel can be directed to abnormal information, the feedback operation such as be adjusted correspondingly.
After business personnel is received to the feedback information of warning information, warning information and feedback information can be counted, then basis
Statistical information is graded to data on flows.As an example it is assumed that following one day showing advertisement amount of prediction, if predicted the outcome
Compared to the previous day data on flows incrementss beyond 20%, then it is believed that the result is abnormal, early warning letter can be generated
Breath, reminding business personnel are handled in time.The amount of showing of the advertisement is for example reduced, it is tended to be normal.Then, can be according to life
Into the number of times of warning information etc., to evaluate data on flows, then matching for high-quality advertisement and high-quality flow, reality realized based on grading
Existing precise control of flew, and benefit.
The monitoring method of the data on flows of the embodiment of the present invention, deep learning mould is trained by substantial amounts of history log data
Type, then data on flows is monitored using the deep learning model trained, the degree of accuracy of data on flows is further increased,
The complexity of data on flows monitoring is reduced, is cut operating costs, it is more intelligent.
To achieve the above object, the present invention also proposes a kind of supervising device of data on flows.
Fig. 3 is the structural representation of the supervising device of data on flows according to an embodiment of the invention.
As shown in figure 3, the supervising device of the data on flows may include：Acquisition module 110, pretreatment module 120, input mould
Block 130, output module 140 and monitoring module 150.
Acquisition module 110 is used to obtain current log data.
In one embodiment of the invention, when being monitored to data on flows, acquisition module 110, which can be obtained, works as the day before yesterday
Will data.Current log data may include that showing advertisement amount, ad click amount, ad click rate, ad conversion rates and advertisement are received
Enter.Minute, hour, day, moon etc. can be used as magnitude unit by current log data, for example, can take the day of nearest 10 minutes
Will, is used as current log data.
Pretreatment module 120 is used to pre-process current log data, generates data on flows to be predicted.
Optionally, pretreatment module 120 is used for：Current log data are classified；According to classification to corresponding current
Daily record data is cleaned and extracted；Regularization processing is carried out to current log data, and generates data on flows to be predicted.
Optionally, pretreatment module 120 is additionally operable to：The noise number in current log data is filtered according to default filtering rule
According to.
In one embodiment of the invention, current log data can be classified first, then according to classification to phase
The current log data answered are cleaned and extracted, then carry out regularization processing to current log data, and generate stream to be predicted
Measure data.For example：Current log data, which can be divided into, shows click data, installation data, metering data.Wherein, hits are showed
According to can be showing advertisement amount, ad click amount；Installation data can be ad conversion rates；Metering data can be that advertisement is received
Enter.Different classification is directed to respectively, and above-mentioned data are cleaned and extracted, so as to generate data on flows to be predicted.Wherein, may be used
To filter the noise data in current log data according to default filtering rule.Certainly, it may also include it in current log data
His type, is such as used for the type for describing to show advertisement terminal, is mobile phone tablet personal computer or computer, such data are to reality
The traffic monitoring on border does not produce influence, therefore can be cleaned.Advised in addition, different filterings can be set according to different demands
Then, for example：Some demands only focus on the high data of ad click amount, then may filter that ad click amount is less than the data of 100 times；
Some demand concern ad conversion rates and advertising income, then may filter that the data such as showing advertisement amount, ad click amount.
Input module 130 is used to input data on flows to be predicted to the deep learning model of training in advance.
Wherein, the deep learning model of training in advance may include range model and depth model, range model and depth mould
Type is connected by a logic loss function.
Range model is a general linear model：Y=Wtx+b.Wherein, y is predicted value, x=x1, x2 ...,
Xn } it is the vector that n feature is constituted, w={ w1, w2 ..., wn } is model parameter, and b is deviation.
Depth model is a feed forward neural networking, embedded vectorial random initializtion.Used in low latitude insertion vector
Hiding layer, there is following calculating function in each hidden layer：a(l+1)=f (w(l)a(l)+b(l)), wherein l is the number of plies, and f is to excite
Function, w(l)It is l layers of skew and excitation values of Model Weight.
Range model and depth model are connected by a logic loss function.FTRL (Follow can be used
TheRegularized Leader) algorithm, as the optimization of range model, the excellent of depth model is used as by the use of AdaGrad algorithms
Change.Finally, generation deep learning model P (Y=1 | X)=σ (WT wide[X, φ (X)]+WT deep a(lf)+ b), wherein, Y is two-value
Tag along sort, σ is S function, and φ is the middle conversion of original feature, and b is deviant, WT wideRange vector model weight to
Amount, WT deepIt is depth model weight vectors.
Output module 140 is used to export the corresponding prediction of data on flows to be predicted by the deep learning model of training in advance
As a result.
Input to deep learning model, carried out using the deep learning model pre- by data on flows to be predicted
Survey, so as to export, data on flows to be predicted is corresponding to predict the outcome.For example：By the ad click amount of nearest 1 hour, predict
Subsequent time period such as the ad click amount in 1 hour future.Again for example：The advertising income of nearest 10 minutes is 100,000 yuan, then may be used
Predict advertising income of following half an hour etc..
Monitoring module 150 is used to be monitored data on flows according to predicting the outcome.
Optionally, monitoring module 150 is used for：When predicting the outcome beyond predetermined threshold value, warning information is generated；Reception business
Personnel count to the feedback information of warning information to warning information and feedback information；According to statistical information to flow number
According to being graded.As an example it is assumed that following one day showing advertisement amount of prediction, if predicted the outcome compared to the stream of the previous day
The incrementss of data are measured beyond 20%, then it is believed that the result is abnormal, warning information can be generated, reminding business personnel and
Shi Jinhang processing.The amount of showing of the advertisement is for example reduced, it is tended to be normal.Then, can be according to the secondary of the warning information of generation
Number etc., to evaluate data on flows, then based on grading realizes matching for high-quality advertisement and high-quality flow, realizes precise control of flew,
And benefit.
It should be appreciated that the supervising device on the data on flows in above-described embodiment, wherein modules perform behaviour
The concrete mode of work is described in detail in the embodiment about the monitoring method of data on flows, will not do herein in detail
It is thin to illustrate explanation.
The supervising device of the data on flows of the embodiment of the present invention, is supervised by using deep learning model to data on flows
Control, improves the degree of accuracy of data on flows, reduces the complexity of data on flows monitoring, cuts operating costs, more intelligent.
In addition, as shown in figure 4, the supervising device of the data on flows of the embodiment of the present invention may also include training module 160.
Training module 160 is used to train deep learning model.
Optionally, training module 160 is used to obtain history log data；History log data is pre-processed, generated
Historical traffic data；Deep learning model is trained according to historical traffic data.
In one embodiment of the invention, history log data can be obtained, history log data is pre-processed, it is raw
Into historical traffic data, then deep learning model can be trained according to historical traffic data.For example：It can be wrapped in history log data
Include showing advertisement amount, the ad click amount of history, the ad click rate of history, the ad conversion rates of history and the history of history
The daily historical data of advertising income, such as the whole of last year.According to above-mentioned historical data, it the pre- place such as can be cleaned, be extracted to it
Reason, generates historical traffic data.Using the feature of historical traffic data, training vector is converted into, and then train deep learning mould
Type.
The supervising device of the data on flows of the embodiment of the present invention, deep learning mould is trained by substantial amounts of history log data
Type, then data on flows is monitored using the deep learning model trained, the degree of accuracy of data on flows is further increased,
The complexity of data on flows monitoring is reduced, is cut operating costs, it is more intelligent.
In order to realize above-described embodiment, the present invention also proposes a kind of server.
Fig. 5 is the structural representation of server according to an embodiment of the invention.
As shown in figure 5, the server includes processor 51, memory 52, communication interface 53 and bus 54, wherein：Processing
Device 51, memory 52 and communication interface 53 are connected by bus 54 and complete mutual communication；The storage of memory 52 is executable
Program code；Processor 51 is run and executable program code by reading the executable program code stored in memory 52
Corresponding program, for performing following steps：
S101 ', obtains current log data.
In one embodiment of the invention, when being monitored to data on flows, current log data can be obtained.Currently
Daily record data may include showing advertisement amount, ad click amount, ad click rate, ad conversion rates and advertising income.Current log
Minute, hour, day, moon etc. can be used as magnitude unit by data, for example, the daily record of nearest 10 minutes can be taken, as current
Daily record data.
Current log data are pre-processed by S102 ', generate data on flows to be predicted.
After current log data are obtained, current log data can be pre-processed, generate data on flows to be predicted.
In one embodiment of the invention, current log data can be classified first, then according to classification to phase
The current log data answered are cleaned and extracted, then carry out regularization processing to current log data, and generate stream to be predicted
Measure data.For example：Current log data, which can be divided into, shows click data, installation data, metering data.Wherein, hits are showed
According to can be showing advertisement amount, ad click amount；Installation data can be ad conversion rates；Metering data can be that advertisement is received
Enter.Different classification is directed to respectively, and above-mentioned data are cleaned and extracted, so as to generate data on flows to be predicted.Wherein, may be used
To filter the noise data in current log data according to default filtering rule.Certainly, it may also include it in current log data
His type, is such as used for the type for describing to show advertisement terminal, is mobile phone tablet personal computer or computer, such data are to reality
The traffic monitoring on border does not produce influence, therefore can be cleaned.Advised in addition, different filterings can be set according to different demands
Then, for example：Some demands only focus on the high data of ad click amount, then may filter that ad click amount is less than the data of 100 times；
Some demand concern ad conversion rates and advertising income, then may filter that the data such as showing advertisement amount, ad click amount.
S103 ', data on flows to be predicted is inputted to the deep learning model of training in advance.
Wherein, the deep learning model of training in advance may include range model and depth model, range model and depth mould
Type is connected by a logic loss function.
Range model is a general linear model：Y=Wtx+b.Wherein, y is predicted value, x=x1, x2 ...,
Xn } it is the vector that n feature is constituted, w={ w1, w2 ..., wn } is model parameter, and b is deviation.
Depth model is a feed forward neural networking, embedded vectorial random initializtion.Used in low latitude insertion vector
Hiding layer, there is following calculating function in each hidden layer：a(l+1)=f (w(l)a(l)+b(l)), wherein l is the number of plies, and f is to excite
Function, w(l)It is l layers of skew and excitation values of Model Weight.
Range model and depth model are connected by a logic loss function.FTRL (Follow can be used
TheRegularized Leader) algorithm, as the optimization of range model, the excellent of depth model is used as by the use of AdaGrad algorithms
Change.Finally, generation deep learning model P (Y=1 | X)=σ (WT wide[X, φ (X)]+WT deep a(lf)+ b), wherein, Y is two-value
Tag along sort, σ is S function, and φ is the middle conversion of original feature, and b is deviant, WT wideRange vector model weight to
Amount, WT deepIt is depth model weight vectors.
In one embodiment of the invention, data on flows to be predicted can be inputted to the deep learning mould of training in advance
Type.
S104 ', predicts the outcome by the way that the deep learning model output data on flows to be predicted of training in advance is corresponding.
Input to deep learning model, carried out using the deep learning model pre- by data on flows to be predicted
Survey, so as to export, data on flows to be predicted is corresponding to predict the outcome.For example：By the ad click amount of nearest 1 hour, predict
Subsequent time period such as the ad click amount in 1 hour future.Again for example：The advertising income of nearest 10 minutes is 100,000 yuan, then may be used
Predict advertising income of following half an hour etc..
S105 ', is monitored according to predicting the outcome to data on flows.
After output predicts the outcome, data on flows can be monitored according to predicting the outcome.
In one embodiment of the invention, when predicting the outcome beyond predetermined threshold value, warning information can be generated, to carry
Business personnel's data on flows of waking up occurs abnormal.Business personnel can be directed to abnormal information, the feedback operation such as be adjusted correspondingly.
After business personnel is received to the feedback information of warning information, warning information and feedback information can be counted, then basis
Statistical information is graded to data on flows.As an example it is assumed that following one day showing advertisement amount of prediction, if predicted the outcome
Compared to the previous day data on flows incrementss beyond 20%, then it is believed that the result is abnormal, early warning letter can be generated
Breath, reminding business personnel are handled in time.The amount of showing of the advertisement is for example reduced, it is tended to be normal.Then, can be according to life
Into the number of times of warning information etc., to evaluate data on flows, then matching for high-quality advertisement and high-quality flow, reality realized based on grading
Existing precise control of flew, and benefit.
The server of the embodiment of the present invention, is monitored by using deep learning model to data on flows, improves stream
The degree of accuracy of data is measured, the complexity of data on flows monitoring is reduced, cuts operating costs, it is more intelligent.
To achieve the above object, the present invention also proposes a kind of storage medium, wherein, the storage medium, which is used to store, to be applied
Program, the application program is used for the monitoring method for operationally performing the data on flows described in first aspect embodiment.
To achieve the above object, the present invention also proposes a kind of application program, wherein, the application program is used for operationally
Perform the monitoring method of the data on flows described in first aspect embodiment.
In the description of the invention, it is to be understood that term " " center ", " longitudinal direction ", " transverse direction ", " length ", " width ",
" thickness ", " on ", " under ", "front", "rear", "left", "right", " vertical ", " level ", " top ", " bottom " " interior ", " outer ", " up time
The orientation or position relationship of the instruction such as pin ", " counterclockwise ", " axial direction ", " radial direction ", " circumference " be based on orientation shown in the drawings or
Position relationship, is for only for ease of the description present invention and simplifies description, rather than indicate or imply that the device or element of meaning must
There must be specific orientation, with specific azimuth configuration and operation, therefore be not considered as limiting the invention.
In addition, term " first ", " second " are only used for describing purpose, and it is not intended that indicating or implying relative importance
Or the implicit quantity for indicating indicated technical characteristic.Thus, define " first ", the feature of " second " can express or
Implicitly include at least one this feature.In the description of the invention, " multiple " are meant that at least two, such as two, three
It is individual etc., unless otherwise specifically defined.
In the present invention, unless otherwise clearly defined and limited, term " installation ", " connected ", " connection ", " fixation " etc.
Term should be interpreted broadly, for example, it may be fixedly connected or be detachably connected, or integrally；Can be that machinery connects
Connect or electrically connect；Can be joined directly together, can also be indirectly connected to by intermediary, can be in two elements
The connection in portion or the interaction relationship of two elements, unless otherwise clear and definite restriction.For one of ordinary skill in the art
For, the concrete meaning of above-mentioned term in the present invention can be understood as the case may be.
In the present invention, unless otherwise clearly defined and limited, fisrt feature can be with "above" or "below" second feature
It is that the first and second features are directly contacted, or the first and second features pass through intermediary mediate contact.Moreover, fisrt feature exists
Second feature " on ", " top " and " above " but fisrt feature are directly over second feature or oblique upper, or be merely representative of
Fisrt feature level height is higher than second feature.Fisrt feature second feature " under ", " lower section " and " below " can be
One feature is immediately below second feature or obliquely downward, or is merely representative of fisrt feature level height less than second feature.
In the description of this specification, reference term " one embodiment ", " some embodiments ", " example ", " specifically show
The description of example " or " some examples " etc. means to combine specific features, structure, material or the spy that the embodiment or example are described
Point is contained at least one embodiment of the present invention or example.In this manual, to the schematic representation of above-mentioned term not
Identical embodiment or example must be directed to.Moreover, specific features, structure, material or the feature of description can be with office
Combined in an appropriate manner in one or more embodiments or example.In addition, in the case of not conflicting, the skill of this area
Art personnel can be tied the not be the same as Example or the feature of example and non-be the same as Example or example described in this specification
Close and combine.
Although embodiments of the invention have been shown and described above, it is to be understood that above-described embodiment is example
Property, it is impossible to limitation of the present invention is interpreted as, one of ordinary skill in the art within the scope of the invention can be to above-mentioned
Embodiment is changed, changed, replacing and modification.
Claims (10)
1. a kind of monitoring method of data on flows, it is characterised in that including：
Obtain current log data；
The current log data are pre-processed, data on flows to be predicted is generated；
The data on flows to be predicted is inputted to the deep learning model of training in advance；
Predicted the outcome by the way that the deep learning model output data on flows to be predicted of the training in advance is corresponding；
Data on flows is monitored according to described predict the outcome.
2. the method as described in claim 1, it is characterised in that pre-processed to the current log data, generation treats pre-
Measurement of discharge data, including：
The current log data are classified；
Corresponding current log data are cleaned and extracted according to classification；
Regularization processing is carried out to the current log data, and generates data on flows to be predicted.
3. method as claimed in claim 2, it is characterised in that according to classification corresponding current log data are carried out cleaning with
Extract, including：
The noise data in the current log data is filtered according to default filtering rule.
4. the method as described in claim 1, it is characterised in that the deep learning model of the training in advance includes：
Range model and depth model, the range model and the depth model are connected by logic loss function.
5. the method as described in claim 1, it is characterised in that be monitored according to described predict the outcome to data on flows, bag
Include：
When described predict the outcome beyond predetermined threshold value, warning information is generated；
Feedback information of the business personnel to the warning information is received, and the warning information and the feedback information are united
Meter；
The data on flows is graded according to statistical information.
6. the method as described in claim 1, it is characterised in that also include：
Train the deep learning model.
7. method as claimed in claim 6, it is characterised in that the training deep learning model, including：
Obtain history log data；
The history log data is pre-processed, historical traffic data is generated；
The deep learning model is trained according to the historical traffic data.
8. a kind of supervising device of data on flows, it is characterised in that including：
Acquisition module, for obtaining current log data；
Pretreatment module, for being pre-processed to the current log data, generates data on flows to be predicted；
Input module, for the data on flows to be predicted to be inputted to the deep learning model of training in advance；
Output module, exports the data on flows to be predicted corresponding pre- for the deep learning model by the training in advance
Survey result；
Monitoring module, is monitored for being predicted the outcome according to data on flows.
9. device as claimed in claim 8, it is characterised in that the pretreatment module, is used for：
The current log data are classified；
Corresponding current log data are cleaned and extracted according to classification；
Regularization processing is carried out to the current log data, and generates data on flows to be predicted.
10. device as claimed in claim 9, it is characterised in that the pretreatment module, is used for：
The noise data in the current log data is filtered according to default filtering rule.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710524384.0A CN107124320A (en) | 2017-06-30 | 2017-06-30 | Traffic data monitoring method and device and server |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710524384.0A CN107124320A (en) | 2017-06-30 | 2017-06-30 | Traffic data monitoring method and device and server |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN107124320A true CN107124320A (en) | 2017-09-01 |
Family
ID=59731175
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201710524384.0A Pending CN107124320A (en) | 2017-06-30 | 2017-06-30 | Traffic data monitoring method and device and server |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN107124320A (en) |
Cited By (15)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107809331A (en) * | 2017-10-25 | 2018-03-16 | 北京京东尚科信息技术有限公司 | The method and apparatus for identifying abnormal flow |
| CN107948166A (en) * | 2017-11-29 | 2018-04-20 | 广东亿迅科技有限公司 | Traffic anomaly detection method and device based on deep learning |
| CN108712292A (en) * | 2018-05-29 | 2018-10-26 | 广州大学 | A kind of network flow type prediction method based on deep learning |
| CN108880888A (en) * | 2018-06-20 | 2018-11-23 | 浙江工商大学 | A kind of SDN network method for predicting based on deep learning |
| CN109120463A (en) * | 2018-10-15 | 2019-01-01 | 新华三大数据技术有限公司 | Method for predicting and device |
| CN109787958A (en) * | 2018-12-15 | 2019-05-21 | 深圳先进技术研究院 | Network flow real-time detection method and detection terminal, computer readable storage medium |
| CN109903142A (en) * | 2019-03-21 | 2019-06-18 | 中国联合网络通信集团有限公司 | A kind of flow packet matching process and device |
| CN109995592A (en) * | 2019-04-09 | 2019-07-09 | 中国联合网络通信集团有限公司 | Quality of service monitoring method and equipment |
| CN110009429A (en) * | 2019-04-10 | 2019-07-12 | 金瓜子科技发展（北京）有限公司 | A kind of method, apparatus and computer equipment of predicted flow rate data |
| CN110445629A (en) * | 2018-05-03 | 2019-11-12 | 佛山市顺德区美的电热电器制造有限公司 | A kind of server concurrency prediction technique and device |
| WO2020082853A1 (en) * | 2018-10-24 | 2020-04-30 | 珠海格力电器股份有限公司 | Method and apparatus for monitoring network security, air conditioner and household appliance |
| CN112101400A (en) * | 2019-12-19 | 2020-12-18 | 国网江西省电力有限公司电力科学研究院 | Industrial control system abnormality detection method, equipment, server and storage medium |
| CN113052383A (en) * | 2021-03-29 | 2021-06-29 | 上海酷量信息技术有限公司 | Income prediction method and device based on machine learning |
| CN113872873A (en) * | 2021-09-29 | 2021-12-31 | 西安交通大学 | Multi-scene cross-layer congestion control method suitable for 5G new application |
| US11556953B2 (en) | 2020-03-10 | 2023-01-17 | Google Llc | Methods, systems, and media for timing presentation of content |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102663617A (en) * | 2012-03-20 | 2012-09-12 | 亿赞普(北京)科技有限公司 | Method and system for prediction of advertisement clicking rate |
| US20150127595A1 (en) * | 2013-11-01 | 2015-05-07 | Numenta, Inc. | Modeling and detection of anomaly based on prediction |
| CN105654200A (en) * | 2015-12-30 | 2016-06-08 | 上海珍岛信息技术有限公司 | Deep learning-based advertisement click-through rate prediction method and device |
| CN106713169A (en) * | 2016-11-25 | 2017-05-24 | 东软集团股份有限公司 | Method of controlling flow bandwidth and apparatus thereof |
-
2017
- 2017-06-30 CN CN201710524384.0A patent/CN107124320A/en active Pending
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102663617A (en) * | 2012-03-20 | 2012-09-12 | 亿赞普(北京)科技有限公司 | Method and system for prediction of advertisement clicking rate |
| US20150127595A1 (en) * | 2013-11-01 | 2015-05-07 | Numenta, Inc. | Modeling and detection of anomaly based on prediction |
| CN105654200A (en) * | 2015-12-30 | 2016-06-08 | 上海珍岛信息技术有限公司 | Deep learning-based advertisement click-through rate prediction method and device |
| CN106713169A (en) * | 2016-11-25 | 2017-05-24 | 东软集团股份有限公司 | Method of controlling flow bandwidth and apparatus thereof |
Cited By (22)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107809331A (en) * | 2017-10-25 | 2018-03-16 | 北京京东尚科信息技术有限公司 | The method and apparatus for identifying abnormal flow |
| CN107948166B (en) * | 2017-11-29 | 2020-09-25 | 广东亿迅科技有限公司 | Deep learning-based traffic anomaly detection method and device |
| CN107948166A (en) * | 2017-11-29 | 2018-04-20 | 广东亿迅科技有限公司 | Traffic anomaly detection method and device based on deep learning |
| CN110445629A (en) * | 2018-05-03 | 2019-11-12 | 佛山市顺德区美的电热电器制造有限公司 | A kind of server concurrency prediction technique and device |
| CN108712292A (en) * | 2018-05-29 | 2018-10-26 | 广州大学 | A kind of network flow type prediction method based on deep learning |
| CN108712292B (en) * | 2018-05-29 | 2021-04-02 | 广州大学 | Network flow type prediction method based on deep learning |
| CN108880888A (en) * | 2018-06-20 | 2018-11-23 | 浙江工商大学 | A kind of SDN network method for predicting based on deep learning |
| CN109120463A (en) * | 2018-10-15 | 2019-01-01 | 新华三大数据技术有限公司 | Method for predicting and device |
| CN109120463B (en) * | 2018-10-15 | 2022-01-07 | 新华三大数据技术有限公司 | Flow prediction method and device |
| CN111092850B (en) * | 2018-10-24 | 2021-06-04 | 珠海格力电器股份有限公司 | Method and device for monitoring network security, air conditioner and household appliance |
| WO2020082853A1 (en) * | 2018-10-24 | 2020-04-30 | 珠海格力电器股份有限公司 | Method and apparatus for monitoring network security, air conditioner and household appliance |
| CN111092850A (en) * | 2018-10-24 | 2020-05-01 | 珠海格力电器股份有限公司 | Method and device for monitoring network security, air conditioner and household appliance |
| CN109787958A (en) * | 2018-12-15 | 2019-05-21 | 深圳先进技术研究院 | Network flow real-time detection method and detection terminal, computer readable storage medium |
| CN109787958B (en) * | 2018-12-15 | 2021-05-25 | 深圳先进技术研究院 | Network flow real-time detection method, detection terminal and computer readable storage medium |
| CN109903142A (en) * | 2019-03-21 | 2019-06-18 | 中国联合网络通信集团有限公司 | A kind of flow packet matching process and device |
| CN109995592A (en) * | 2019-04-09 | 2019-07-09 | 中国联合网络通信集团有限公司 | Quality of service monitoring method and equipment |
| CN110009429A (en) * | 2019-04-10 | 2019-07-12 | 金瓜子科技发展（北京）有限公司 | A kind of method, apparatus and computer equipment of predicted flow rate data |
| CN110009429B (en) * | 2019-04-10 | 2021-04-16 | 金瓜子科技发展（北京）有限公司 | Method and device for predicting flow data and computer equipment |
| CN112101400A (en) * | 2019-12-19 | 2020-12-18 | 国网江西省电力有限公司电力科学研究院 | Industrial control system abnormality detection method, equipment, server and storage medium |
| US11556953B2 (en) | 2020-03-10 | 2023-01-17 | Google Llc | Methods, systems, and media for timing presentation of content |
| CN113052383A (en) * | 2021-03-29 | 2021-06-29 | 上海酷量信息技术有限公司 | Income prediction method and device based on machine learning |
| CN113872873A (en) * | 2021-09-29 | 2021-12-31 | 西安交通大学 | Multi-scene cross-layer congestion control method suitable for 5G new application |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN107124320A (en) | Traffic data monitoring method and device and server | |
| Kim | Toward a successful CRM: variable selection, sampling, and ensemble | |
| US8849732B2 (en) | Adaptive remote maintenance of rolling stocks | |
| CN111242171B (en) | Model training and diagnosis prediction method and device for network faults and electronic equipment | |
| CN109300039A (en) | The method and system of intellectual product recommendation are carried out based on artificial intelligence and big data | |
| WO2000007113A1 (en) | Automatic freeway incident detection system using artificial neural networks and genetic algorithms | |
| Huang et al. | Using artificial neural networks to establish a customer-cancellation prediction model | |
| US20240085870A1 (en) | Predictive modeling and control for water resource infrastructure | |
| CN108091132A (en) | A kind of traffic flow forecasting method and device | |
| CN106934412A (en) | A kind of user behavior sorting technique and system | |
| Wang et al. | Traffic flow forecasting based on a hybrid model | |
| Iliopoulou et al. | Identifying spatio-temporal patterns of bus bunching in urban networks | |
| CN110458325A (en) | A kind of traffic zone passenger flow forecasting and device in short-term | |
| CN113114529A (en) | KPI (Key Performance indicator) abnormity detection method and device based on condition variation automatic encoder | |
| Frizelle et al. | The measurement of complexity in production and other commercial systems | |
| Shoorkand et al. | A hybrid CNN-LSTM model for joint optimization of production and imperfect predictive maintenance planning | |
| Chuwang et al. | Short-term urban rail transit passenger flow forecasting based on fusion model methods using univariate time series | |
| CN117610734A (en) | Deep learning-based user behavior prediction method, system and electronic equipment | |
| CN112686448B (en) | Loss early warning method and system based on attribute data | |
| CN115375474A (en) | Information prompting method and device and electronic equipment | |
| Li et al. | A study on customer churn of commercial banks based on learning from label proportions | |
| CN112232945B (en) | Method and device for determining personal client credit | |
| CN111581508B (en) | Service monitoring method, device, equipment and storage medium | |
| JP7458183B2 (en) | Energy demand fluctuation pattern classification method and system | |
| CN114358559A (en) | Electric power fee-hastening strategy design method, system and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20170901 |
|
| RJ01 | Rejection of invention patent application after publication |