CN103873499B - Data capture method and device - Google Patents
Data capture method and device Download PDFInfo
- Publication number
- CN103873499B CN103873499B CN201210533354.3A CN201210533354A CN103873499B CN 103873499 B CN103873499 B CN 103873499B CN 201210533354 A CN201210533354 A CN 201210533354A CN 103873499 B CN103873499 B CN 103873499B
- Authority
- CN
- China
- Prior art keywords
- response time
- task
- cgi
- estimated
- current
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Landscapes
- Information Transfer Between Computers (AREA)
- Computer And Data Communications (AREA)
Abstract
This application provides data capture method and device.Wherein, this method includes：Receive the request of data from client；It is determined that the CGI(Common gateway interface) CGI for handling the request of data, handled according to priority below high to low order tasks carrying successively to the CGI：The task current response time is estimated, detects whether the response time estimated meets and imposes a condition, if it is, performing the task by the CGI, if not, updating the response time estimated, determines whether to perform the task by the CGI according to the response time after updating.Using the present invention, can realize that CGI obtains data according to network condition, it is to avoid due to CGI it is unavailable caused by page response speed is slow and CGI500 mistakes.
Description
Technical field
The application is related to Internet technology, more particularly to data capture method and device.
Background technology
In the Internet, applications, when the page needs to obtain data, client can be initiated to ask to Web server.Web takes
CGI (the CGI of business device：Common Gateway Interface) understand according to the corresponding data of the acquisition request, and
When the data of acquisition are returned into client, backstage is asked to handle the data in a serial fashion, the processing of return afterwards
Data afterwards are to client.Here, so-called CGI, it is physically, substantially one section of application program, operates in Web server
On, there is provided the interface that same client is communicated for a kind of instrument communicated for Web server with client.
In actual applications, CGI obtain data can be influenceed by network condition, when due to network condition cause CGI without
When method obtains corresponding data or causes the CGI still not obtain corresponding data more than setting time, Web server can be terminated
CGI continues to obtain data, and this can delay page response speed, or even common CGI500 mistakes occur.
The content of the invention
This application provides data capture method and device, to realize that CGI obtains data according to network condition, it is to avoid due to
Page response speed is slow caused by CGI is unavailable and CGI500 mistakes.
The technical scheme that the application is provided includes：
A kind of data capture method, this method includes：
A kind of data capture method, it is characterised in that this method includes：
Receive the request of data from client；
It is determined that the CGI(Common gateway interface) CGI for handling the request of data, wherein, the CGI has what is be pre-configured with
At least one task, each task of the CGI, with corresponding data type, for obtaining its corresponding data type
Data, also, priority is assigned in each task of the CGI；
Handled according to priority below high to low order tasks carrying successively to the CGI：
The task current response time is estimated, detects whether the response time estimated meets and imposes a condition, if it is, by institute
State CGI and perform the task, if not, updating the response time estimated, determine whether according to the response time after updating by described
CGI performs the task；
The response time that the renewal is estimated includes：
If the priority value of task is less than or equal to the priority threshold value of setting, it is setting to update the response time estimated
Value；
The response time according to after updating determines whether that performing the task by the CGI includes：
If the response time value after updating is setting value, control CGI abandons performing the task.
A kind of data acquisition facility applied on Web server, the device includes：
Receiving unit, for receiving the request of data from client；
Determining unit, for the CGI(Common gateway interface) CGI for determining to be used to handle the request of data, wherein, the CGI tools
There is at least one task being pre-configured with, each task of the CGI is right for obtaining its with corresponding data type
The data for the data type answered, also, priority is assigned in each task of the CGI；
Processing unit, for being handled according to priority below high to low order tasks carrying successively to the CGI：
The task current response time is estimated, detects whether the response time estimated meets and imposes a condition, if it is, being held by the CGI
The row task, if not, updating the response time estimated, determines whether to be performed by the CGI according to the response time after updating
The task；
The processing unit more new task current response time includes：
If the priority value of task is less than or equal to the priority threshold value of setting, it is setting to update the response time estimated
Value；
The processing unit determines whether that performing task by CGI includes according to the response time after updating：
If the response time value after updating is setting value, control CGI abandons performing the task.
As can be seen from the above technical solutions, in the present invention, whether CGI performs task and depends on what is estimated for the task
Response time, and the response time that this is estimated reflects the current network condition of task, this realizes CGI according to Task Network shape
Condition performs task and obtains data, it is to avoid cause the problem of CGI is unavailable to be brought such as page response speed due to network condition difference
Spend slow and CGI500 mistakes etc.；
Further, the present invention by for CGI task distribute priority, and according to priority from high to low order according to
Secondary each tasks carrying to the CGI is handled, and this ensures that the high task of priority is performed as much as possible, it is ensured that the net of user
Network is accessed.
Brief description of the drawings
Fig. 1 is method flow diagram provided in an embodiment of the present invention；
Fig. 2 is step 104 implementation process figure provided in an embodiment of the present invention；
Fig. 3 is step 105 implementation process figure provided in an embodiment of the present invention；
Fig. 4 a are application example schematic diagram provided in an embodiment of the present invention；
Fig. 4 b are the application example schematic diagram that existing CGI performs task；
Fig. 5 is structure drawing of device provided in an embodiment of the present invention.
Embodiment
In order that the object, technical solutions and advantages of the present invention are clearer, below in conjunction with the accompanying drawings with specific embodiment pair
The present invention is described in detail.
Referring to Fig. 1, Fig. 1 is method flow diagram provided in an embodiment of the present invention.Wherein, the present invention is applied to Web server,
As shown in figure 1, the method that the present invention is provided may include following steps：
Step 101, the request of data from client is received.
Generally, when webpage needs data to update, it can trigger client and send request of data to Web server.And due to
The present invention is applied to Web server, and therefore, it can receive the request of data from client.
Step 102, it is determined that CGI for handling the request of data.
On Web server, it has been compiled multiple CGI in advance, wherein, different CGI has different functions, because
This, in this step 102, after the request of data from client is received, the CGI that can be compiled according to Web server has
Function determine be currently which CGI is the request of data is handled by.
In the present invention, CGI is preconfigured at least one task, wherein, can be according to following original when for CGI configuration tasks
Then perform：An a kind of task of data of data type as CGI will be obtained, or, it is two or more associated by obtaining
The data of data type as CGI a task.Such as, CGI can obtain the data of following four data type：Class1, class
Type 2, type 3, type 4, based on will obtain a kind of data of data type as the description of a CGI task, then match somebody with somebody for CGI
Put following 4 tasks：Task 1, task 2, task 3, task 4, wherein, task 1 is used to obtain the corresponding data of Class1, task 2
For obtaining the corresponding data of type 2, task 3 is used to obtain the corresponding data of type 3, and task 4 is corresponding for obtaining type 4
Data.And be based on obtaining the data of two or more associated data types as the description of a CGI task, then
Then, if Class1 is unrelated to type 4 with type 2, type 3 is unrelated with Class1, type 2, type 4, and type 2 and type 4 are associated,
Such as, type 2 is user's identity type, and type 4 is user basic information type, and both of which is identified by user to be associated, then
For following 3 tasks of CGI configurations：Task 1, task 2, task 3, wherein, task 1 is used for the data for obtaining Class1, and task 2 is used
In obtaining type 2, the data of type 4, task 3 is used for the data for obtaining type 3.
It in summary it can be seen, in the present invention, each task that CGI is configured, with corresponding data type, for obtaining
Take the data of its corresponding data type.
In addition, in the present invention, being also assigned with priority for CGI each task.Wherein, each task for being CGI is distributed
Priority can be realized by following steps：The significance level of the data obtained according to task is that each task distributes priority, its
In, the data significance level that task is obtained is higher, then the priority of the task is higher, and the data significance level that task is obtained is got over
Low, then the priority of the task is lower.Such as, CGI is configured with following 3 tasks：Task 1, task 2, task 3, wherein, according to
Product demand finds the data significance level highest that task 1 is obtained, and the data significance level that task 2 is obtained is taken second place, and task 3 is obtained
Data significance level it is minimum, based on this, just distribute highest priority for task 1, such as value is 4, be that task 2 is distributed time
High priority, such as value are 3, are that task 3 distributes minimum priority, such as value is 2.
Step 103, according to priority from high to low order successively to each task execution step 104 and step 105.
Step 104, the task current response time is estimated.Step 105 is performed afterwards.
As one embodiment of the present of invention, in this step 104, the response time for estimating task can be by flowing shown in Fig. 2
Cheng Shixian, wouldn't be repeated here.
Step 105, whether the response time that detection is estimated, which meets, imposes a condition, if it is, performing this by the CGI
Business, if not, updating the response time estimated, determines whether to perform the task by the CGI according to the response time after updating.
As one embodiment of the present of invention, this step 105 can be realized by flow shown in Fig. 3, wouldn't be repeated here.
So far, flow shown in Fig. 1 is completed.
From flow can be seen that the present invention shown in Fig. 1, CGI whether perform task place one's entire reliance upon it is pre- for the task
The response time estimated, the response time estimated reflects the current network condition of task, that is to say, that the present invention is according to task
Current network conditions determine CGI whether perform task, this obviously can avoid due to network condition difference and cause CGI unavailable
The problem of bringing such as page response speed is slow and CGI500 mistakes etc..
Flow shown in Fig. 2 is described below：
Referring to Fig. 2, Fig. 2 is step 104 implementation process figure provided in an embodiment of the present invention.As shown in Fig. 2 the flow can be wrapped
Include following steps：
Step 201, detect the CGI by the end of currently whether the excessively described task of executed, if it is, perform step 202,
If not, performing step 205.
Step 202, the task is calculated when the last time having been performed the response of the actual generation of the rear task by CGI
Between.
After CGI has performed a subtask, the time for starting the time performed according to the task and completing to perform is easy to
Determine the actual response time of the task.
Step 203, referred to using the response time and the last response time fluctuation determined for the task of calculating
Number determines the current response time index of oscillation of the task.
In the present invention, the current response time index of oscillation reflects the fluctuation of the task current network conditions.
As one embodiment of the present of invention, step 203 can use index moving average (EMA：Exponential
Moving Average) the current response time index of oscillation of algorithm calculating task.EMA algorithms are introduced in stock application
, and applied to the present invention, it is the network condition variation tendency that the task is monitored by using the EMA algorithms, with after an action of the bowels
The continuous strategy that task is performed according to the timely decision-making CGI of monitored results.
Wherein, step 203 use the current response time index of oscillation of EMA algorithm calculating tasks for：
Yn+1=[2*Xn+1+(N-1)*Yn]/(N+1)；(formula 1)
Wherein, Yn+1Represent the current response time index of oscillation, Xn+1Represent that the task is performed in the last time by CGI
The response time of the complete rear actual generation of task, YnRepresent that the last response time fluctuation determined for the task refers to
Number, N is the decimation factor of setting, can be set according to actual conditions, here without limiting.
It should be noted that the fluctuation of current response time that this step 203 is the calculating task by taking EMA algorithms as an example refers to
Number, can also use the current response time index of oscillation of other modes calculating task, and the present invention is not specific to be limited.
Step 204, the task current response time is estimated using the current response time index of oscillation.
As one embodiment of the present of invention, this step 204 can estimate the response of the task currently using below equation
Time Tdo：
Tdo=Thwm+p* (Tmax-Thwm) (formula 2)
Wherein, Tdo represents the response time estimated, Thwm be for task maximum response time set in advance,
Tmax is CGI maximum response time set in advance, and it can be relevant with Thwm, such as Q times for Thwm, wherein, Q values are
Any value in 1.3 to 2, as one embodiment of the present of invention, Tmax can value be 5；P is represented by equation 3 below：
P=(Thwm-Yn+1)/(Thwm-Y0) (formula 3).
It should be noted that when this step 204 simply estimates the response of task currently by taking above-mentioned formula 2 and formula 3 as an example
Between, the task current response time can also be estimated using other modes, the present invention is not specific to be limited.
Step 205, by Y set in advance0It is defined as the current response time index of oscillation of the task, is rung using current
The index of oscillation estimates the task current response time between seasonable.
In this step 205, the task current response time and step are estimated using the current response time index of oscillation
204 is similar, repeats no more here.
So far, flow shown in Fig. 2 is completed.The task current response time can be estimated by flow shown in Fig. 2, with after an action of the bowels
Whether the continuous response time estimated according to this performs the task come decision-making by CGI, specifically flow as shown in Figure 3.
Flow shown in Fig. 3 is described below：
Referring to Fig. 3, Fig. 3 is step 105 implementation process figure provided in an embodiment of the present invention.Preferably, it is somebody's turn to do for ease of understanding
Flow, is only designated as task 1 by the task that this flow is related to below and is described, as shown in figure 3, the flow may include：
Step 301, calculating task 1 is currently estimated response time and CGI short time consumption (Tcgi) sum.
In this step 301, the Tcgi is：Since CGI receiving the request of data to the current consumed time.
Step 302, whether the result of calculation of judgment step 301 is less than or equal to CGI maximum response time set in advance
(Tmax), imposed a condition if it is, determining that the response time estimated meets, step 303 is performed, otherwise, it determines the response estimated
Time is unsatisfactory for imposing a condition, and performs step 304.
Step 303, the task 1 is performed by CGI.Terminate current process.
From step 303 as can be seen that when the result of calculation of step 301 is less than or equal to Tmax, it is meant that current network
It is in good condition, the task can be continued executing with by CGI.Now, be not in because network condition influences and causes CGI can not
With such as CGI500 mistakes brought, renewal of the page is slow the problems such as.
Step 304, judge whether the priority of the task 1 is less than or equal to the priority threshold value of setting, if it is, updating pre-
The response time estimated is setting value, if not, updating the difference that the response time estimated is Tmax and Tcgi.Step is performed afterwards
305。
When the response time estimated is setting value, represent that the current network condition of task 1 is poor, to avoid CGI from holding
By network condition influences and causes unavailable the brought such as CGI500 mistakes of CGI, renewal of the page slow during the row task 1
The problems such as, it is necessary to CGI abandon perform the task 1, be specifically shown in step 305.
As one embodiment of the present of invention, the setting value in step 304 can be 0.
Step 305, judge the response time after updating, whether value is setting value, if it is, control CGI abandons performing
The task 1, otherwise, the task 1 is performed by CGI.
So far, flow shown in Fig. 3 is terminated.
Citing description is carried out to the method that the present invention is provided below by a specific embodiment：
If receiving after the request of data from client, determine that CGI1 is used to perform the data according to the request of data
Request, wherein, the task that CGI1 has is followed successively by A, B, C, D, the priority values of four tasks is arranged in order as 1,2,3,
4, wherein, task D priority-level highest then, first estimates the task D current response times for task D and (is designated as EMA
(D, N)), and perform following steps：
Step a, detects whether response time EMA (D, N) and CGI Tcgi sums are less than or equal to Tmax, if it is, holding
Whether row task D, otherwise, Detection task D priority value are less than or equal to the priority threshold value of setting, if it is, will be directed to
The response time that task D is estimated is set to setting value such as 0, if not, will be set to for the task D response times estimated
Tmax and Tcgi difference.
Wherein, steps described above a can be described by following procedure language：
If (Tcgi+EMA (D, N))<=Tmax)
Do D,
Else
If (D priority<=given threshold)
Stop D, END EMA (D, N)=setting value
Else
EMA (D, N)=Tmax-Tcgi；
Similarly, similar above-mentioned step is also performed for task A, B, C, that is, realized according to network condition decision task
Strategy, specifically as shown in fig. 4 a, this performs the mode of task compared to the existing CGI shown in Fig. 4 b, realizes CGI according to network
Situation obtains data, it is to avoid cause the problem of CGI is unavailable to be brought such as page response speed slow due to network condition difference and
CGI500 mistakes etc..
The method that the present invention is provided is described above, the device that the present invention is provided is described below：
Referring to Fig. 5, Fig. 5 is structure drawing of device provided in an embodiment of the present invention.Wherein, the device is applied to Web service
Data acquisition facility on device, as shown in figure 5, the device includes：
Receiving unit, for receiving the request of data from client；
Determining unit, for the CGI(Common gateway interface) CGI for determining to be used to handle the request of data, wherein, the CGI tools
There is at least one task being pre-configured with, each task of the CGI is right for obtaining its with corresponding data type
The data for the data type answered, also, priority is assigned in each task of the CGI；
Processing unit, for being handled according to priority below high to low order tasks carrying successively to the CGI：
The task current response time is estimated, detects whether the response time estimated meets and imposes a condition, if it is, being held by the CGI
The row task, if not, updating the response time estimated, determines whether to be performed by the CGI according to the response time after updating
The task.
In the present invention, as shown in figure 5, the processing unit estimates the task current response time by following subelement：
Detection sub-unit, for detect the CGI by the end of currently whether the excessively described task of executed,
Determination subelement, for when the testing result of the detection sub-unit is no, by Y set in advance0It is defined as
The current response time index of oscillation of the task；
Computation subunit, for the testing result in the detection sub-unit for when being, the calculating task is nearest one
The secondary response time that the actual generation of the rear task has been performed by CGI, institute is directed to using the response time and the last time of calculating
The response time index of oscillation for stating task determination determines the current response time index of oscillation of the task；
Subelement is estimated, for estimating the task current response time using the current response time index of oscillation, its
In, the current response time index of oscillation reflects the fluctuation of the task current network conditions.
Preferably, in the present invention, the computation subunit determines the current response time index of oscillation by below equation：
Yn+1=[2*Xn+1+(N-1)*Yn]/(N+1)；
Wherein, Yn+1Represent the current response time index of oscillation, Xn+1Represent that the task is performed in the last time by CGI
The response time of the complete rear actual generation of task, YnRepresent that the last response time fluctuation determined for the task refers to
Number, N is the decimation factor of setting.
Preferably, in the present invention, the subelement of estimating estimates the task current response time by below equation：
Tdo=Thwm+p* (Tmax-Thwm),
Wherein, Tdo represents the response time estimated, Thwm be for task maximum response time set in advance,
Tmax is CGI maximum response time set in advance, and P is represented by below equation：
P=(Thwm-Yn+1)/(Thwm-Y0), Yn+1Represent the current response time index of oscillation.
In the present invention, the processing unit Detection task current response time whether meet impose a condition including：
Detect whether the response time estimated is less than or equal to CGI set in advance with CGI short time consumption Tcgi sums maximum
Response time Tmax, imposes a condition if it is, determining that the response time estimated meets, otherwise, it determines the response time estimated is not
Satisfaction imposes a condition；
The Tcgi is：Since CGI receiving the request of data to the current consumed time
In the present invention, the processing unit more new task current response time includes：
Judge whether the priority value of task is less than or equal to the priority threshold value of setting,
If it is, it is setting value to update the response time estimated；
If not, updating the difference that the response time estimated is Tmax and Tcgi, the Tmax is CGI set in advance maximum
Response time.
In the present invention, the processing unit determines whether that performing task by CGI includes according to the response time after updating：
Judge the response time after updating, whether value is setting value, if it is, control CGI abandons performing the task,
Otherwise, the task is performed by CGI.
So far, the apparatus structure description shown in Fig. 5 is completed.
As can be seen from the above technical solutions, in the present invention, whether CGI performs task and depends on what is estimated for the task
Response time, and the response time that this is estimated reflects the current network condition of task, this realizes networks of the CGI according to task
Situation performs task and obtains data, it is to avoid cause the problem of CGI is unavailable to be brought such as page response due to network condition difference
Speed is slow and CGI500 mistakes etc.；
Further, the present invention by for CGI task distribute priority, and according to priority from high to low order according to
Secondary each tasks carrying to the CGI is handled, and this ensures that the high task of priority is performed as much as possible, it is ensured that the net of user
Network is accessed.
Further, the present invention ensure that the stabilization of task according to the strategy of network condition decision task, and accordingly
The response speed of customer terminal webpage is improved, the robustness of task in the case of user's unaware, can be improved.
The foregoing is merely illustrative of the preferred embodiments of the present invention, is not intended to limit the invention, all essences in the present invention
God is with principle, and any modification, equivalent substitution and improvements done etc. should be included within the scope of protection of the invention.
Claims (13)
1. a kind of data capture method, it is characterised in that this method includes：
Receive the request of data from client；
It is determined that the CGI(Common gateway interface) CGI for handling the request of data, wherein, the CGI, which has, to be pre-configured with least
One task, each task of the CGI, with corresponding data type, the number for obtaining its corresponding data type
According to, also, priority is assigned in each task of the CGI；
Handled according to priority below high to low order tasks carrying successively to the CGI：
The task current response time is estimated, detects whether the response time estimated meets and imposes a condition, if it is, by described
CGI performs the task, if not, updating the response time estimated, determines whether according to the response time after updating by the CGI
Perform the task；
The response time that the renewal is estimated includes：
If the priority value of task is less than or equal to the priority threshold value of setting, it is setting value to update the response time estimated；
The response time according to after updating determines whether that performing the task by the CGI includes：
If the response time value after updating is setting value, control CGI abandons performing the task.
2. according to the method described in claim 1, it is characterised in that each task of the CGI is allocated by following steps
Priority：
The significance level of the data obtained according to task is that each task distributes priority, wherein, the data that task is obtained are important
Degree is higher, then the priority of the task is higher, and the data significance level that task is obtained is lower, then the priority of the task is got over
It is low.
3. according to the method described in claim 1, it is characterised in that described to estimate the task current response time and include：
Detect the CGI by the end of currently whether the excessively described task of executed,
If not, by Y set in advance0It is defined as the current response time index of oscillation of the task, utilizes the current response time
The index of oscillation estimates the task current response time；
If it is, calculating the response time that the task has been performed the actual generation of the rear task in the last time by CGI, utilize
The response time of calculating and the last response time index of oscillation determined for the task determine the current of the task
The response time index of oscillation, the task current response time is estimated using the current response time index of oscillation；
Wherein, the current response time index of oscillation reflects the fluctuation of the task current network conditions.
4. method according to claim 3, it is characterised in that described to utilize the response time calculated and the last determination
The response time index of oscillation determine that the current response time index of oscillation of task includes：
The current response time index of oscillation is determined using below equation：
Yn+1=[2*Xn+1+(N-1)*Yn]/(N+1)；
Wherein, Yn+1Represent the current response time index of oscillation, Xn+1Represent the task after the last time has been performed by CGI
The response time of the actual generation of task, YnRepresent the last response time index of oscillation determined for the task, N
For the decimation factor of setting.
5. method according to claim 3, it is characterised in that the current response time index of oscillation of the utilization is estimated described
The task current response time includes：
The task current response time is estimated using below equation：
Tdo=Thwm+p* (Tmax-Thwm),
Wherein, Tdo represents the response time estimated, and Thwm is for task maximum response time set in advance, Tmax
For CGI maximum response time set in advance, P is represented by below equation：
P=(Thwm-Yn+1)/(Thwm-Y0), Yn+1Represent the current response time index of oscillation.
6. according to the method described in claim 1, it is characterised in that described to detect whether the response time estimated meets setting bar
Part includes：
Detect whether the response time estimated is less than or equal to CGI peak responses set in advance with CGI short time consumption Tcgi sums
Time Tmax, imposes a condition if it is, determining that the response time estimated meets, otherwise, it determines the response time estimated is unsatisfactory for
Impose a condition；
The Tcgi is：Since CGI receiving the request of data to the current consumed time.
7. according to the method described in claim 1, it is characterised in that the response time that the renewal is estimated further comprises：
If the priority value of task is more than the priority threshold value of setting, update response time for estimating for Tmax and Tcgi it
Difference, the Tmax is CGI maximum response time set in advance, and the Tcgi is：Since CGI receiving the request of data
To the current consumed time.
8. a kind of data acquisition facility applied on Web server, it is characterised in that the device includes：
Receiving unit, for receiving the request of data from client；
Determining unit, for the CGI(Common gateway interface) CGI for determining to be used to handle the request of data, wherein, the CGI has pre-
At least one task first configured, each task of the CGI is corresponding for obtaining its with corresponding data type
The data of data type, also, priority is assigned in each task of the CGI；
Processing unit, for being handled according to priority below high to low order tasks carrying successively to the CGI：Estimate
The task current response time, detect whether the response time estimated meets and impose a condition, should if it is, being performed by the CGI
Task, if not, updating the response time estimated, determines whether to perform this by the CGI according to the response time after updating
Business；
The processing unit more new task current response time includes：
If the priority value of task is less than or equal to the priority threshold value of setting, it is setting value to update the response time estimated；
The processing unit determines whether that performing task by CGI includes according to the response time after updating：
If the response time value after updating is setting value, control CGI abandons performing the task.
9. device according to claim 8, it is characterised in that the processing unit is estimated task by following subelement and worked as
The preceding response time：
Detection sub-unit, for detect the CGI by the end of currently whether the excessively described task of executed,
Determination subelement, for when the testing result of the detection sub-unit is no, by Y set in advance0It is defined as described appoint
The current response time index of oscillation of business；
Computation subunit, for the testing result in the detection sub-unit for when being, the calculating task is in the last quilt
CGI has performed the response time of the actual generation of the rear task, is appointed using the response time and the last time of calculating for described
The response time index of oscillation that business is determined determines the current response time index of oscillation of the task；
Subelement is estimated, for estimating the task current response time using the current response time index of oscillation, wherein, when
The preceding response time index of oscillation reflects the current network condition of the task.
10. device according to claim 9, it is characterised in that the computation subunit is determined current by below equation
The response time index of oscillation：
Yn+1=[2*Xn+1+(N-1)*Yn]/(N+1)；
Wherein, Yn+1Represent the current response time index of oscillation, Xn+1Represent the task after the last time has been performed by CGI
The response time of the actual generation of task, YnRepresent the last response time index of oscillation determined for the task, N
For the decimation factor of setting.
11. device according to claim 9, it is characterised in that the subelement of estimating estimates task by below equation
The current response time：
Tdo=Thwm+p* (Tmax-Thwm),
Wherein, Tdo represents the response time estimated, and Thwm is for task maximum response time set in advance, Tmax
For CGI maximum response time set in advance, P is represented by below equation：
P=(Thwm-Yn+1)/(Thwm-Y0), Yn+1Represent the current response time index of oscillation.
12. device according to claim 8, it is characterised in that the processing unit Detection task current response time
Whether satisfaction impose a condition including：
Detect whether the response time estimated is less than or equal to CGI peak responses set in advance with CGI short time consumption Tcgi sums
Time Tmax, imposes a condition if it is, determining that the response time estimated meets, otherwise, it determines the response time estimated is unsatisfactory for
Impose a condition；
The Tcgi is：Since CGI receiving the request of data to the current consumed time.
13. device according to claim 8, it is characterised in that the processing unit more new task current response time
Further comprise：
If priority value is more than the priority threshold value of setting, the difference that the response time estimated is Tmax and Tcgi, institute are updated
Tmax is stated for CGI maximum response time set in advance, the Tcgi is：CGI since being received the request of data to work as
The preceding consumed time.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201210533354.3A CN103873499B (en) | 2012-12-12 | 2012-12-12 | Data capture method and device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201210533354.3A CN103873499B (en) | 2012-12-12 | 2012-12-12 | Data capture method and device |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN103873499A CN103873499A (en) | 2014-06-18 |
| CN103873499B true CN103873499B (en) | 2017-11-07 |
Family
ID=50911624
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201210533354.3A Active CN103873499B (en) | 2012-12-12 | 2012-12-12 | Data capture method and device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN103873499B (en) |
Families Citing this family (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106097088A (en) * | 2016-06-07 | 2016-11-09 | 中国建设银行股份有限公司 | The processing method of accounting data and system |
| CN106534281B (en) * | 2016-10-25 | 2019-09-24 | Oppo广东移动通信有限公司 | A kind of response method of request of data, apparatus and system |
| CN109857479A (en) * | 2018-12-14 | 2019-06-07 | 平安科技（深圳）有限公司 | Interface data processing method, device, computer equipment and storage medium |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| KR20030028273A (en) * | 2001-09-28 | 2003-04-08 | 주식회사 케이티 | Method for administering data in web environment |
| CN1595931A (en) * | 2004-06-30 | 2005-03-16 | 港湾网络有限公司 | Route processing method for modules of border gateway protocol (BGP) |
| CN101043529A (en) * | 2006-03-21 | 2007-09-26 | 腾讯科技（深圳）有限公司 | Universal gateway interface and method for obtaining data using the same |
| CN101631066A (en) * | 2008-07-14 | 2010-01-20 | 中兴通讯股份有限公司 | Method and system for realizing terminal quality of service of internet voice transmission system |
| CN101895544A (en) * | 2010-07-15 | 2010-11-24 | 深圳市普联技术有限公司 | Multi-user control method and multi-user control system for communication equipment and server |
| CN102238204A (en) * | 2010-04-23 | 2011-11-09 | 腾讯科技（深圳）有限公司 | Network data acquisition method and system |
-
2012
- 2012-12-12 CN CN201210533354.3A patent/CN103873499B/en active Active
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| KR20030028273A (en) * | 2001-09-28 | 2003-04-08 | 주식회사 케이티 | Method for administering data in web environment |
| CN1595931A (en) * | 2004-06-30 | 2005-03-16 | 港湾网络有限公司 | Route processing method for modules of border gateway protocol (BGP) |
| CN101043529A (en) * | 2006-03-21 | 2007-09-26 | 腾讯科技（深圳）有限公司 | Universal gateway interface and method for obtaining data using the same |
| CN101631066A (en) * | 2008-07-14 | 2010-01-20 | 中兴通讯股份有限公司 | Method and system for realizing terminal quality of service of internet voice transmission system |
| CN102238204A (en) * | 2010-04-23 | 2011-11-09 | 腾讯科技（深圳）有限公司 | Network data acquisition method and system |
| CN101895544A (en) * | 2010-07-15 | 2010-11-24 | 深圳市普联技术有限公司 | Multi-user control method and multi-user control system for communication equipment and server |
Also Published As
| Publication number | Publication date |
|---|---|
| CN103873499A (en) | 2014-06-18 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Fan et al. | Self-triggered consensus for multi-agent systems with zeno-free triggers | |
| US10140142B2 (en) | Grouping and placement of virtual machines based on similarity and correlation of functional relations | |
| CN109936517B (en) | Adaptive dynamic flow distribution method in mimicry defense | |
| US20150074198A1 (en) | Social network grouping method and system, and computer storage medium | |
| JP2008027442A (en) | Sub-task processor distribution scheduling | |
| CN103873499B (en) | Data capture method and device | |
| US20010016876A1 (en) | Method and apparatus to provide high precision packet traversal time statistics in a heterogeneous network | |
| CN109491788A (en) | A kind of virtual platform implementation of load balancing and device | |
| CN104683457A (en) | Concurrency control method and device | |
| CN104424240B (en) | Multilist correlating method, main service node, calculate node and system | |
| CN103942197A (en) | Data monitoring processing method and device | |
| CN101331715A (en) | Node discovery involving multiple node enumerators | |
| KR20190131174A (en) | Apparatus and method of providing a cloud-based batch service | |
| CN104301241B (en) | A kind of SOA dynamic load distributing methods and system | |
| CN101331714A (en) | Node discovery involving stealth node enumerators | |
| CN112465615A (en) | Bill data processing method, device and system | |
| EP2212786B1 (en) | Method and device for agile computing | |
| Subramanian et al. | Online incentive mechanism design for smartphone crowd-sourcing | |
| GB2507816A (en) | Calculating timeout for remote task execution from network delays and processing duration on local application/hardware replica | |
| Motahari-Nezhad et al. | Protocol discovery from imperfect service interaction logs | |
| CN105227384A (en) | Method of commerce between online transaction processing system and transaction system | |
| CN101981908B (en) | Monitoring system | |
| Marendić et al. | An investigation into the performance of reduction algorithms under load imbalance | |
| CN102868594B (en) | Method and device for message processing | |
| CN104113590A (en) | Copy selection method based on copy response time prediction |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |