WO2011009345A1

WO2011009345A1 - Method and device for smoothing data processing

Info

Publication number: WO2011009345A1
Application number: PCT/CN2010/073664
Authority: WO
Inventors: 龚力; 李蕊; 杨建军
Original assignee: 中兴通讯股份有限公司
Priority date: 2009-07-23
Filing date: 2010-06-08
Publication date: 2011-01-27
Also published as: CN101615135B; CN101615135A

Abstract

A method for smoothing data processing is provided. The method includes: determining data processing quantity of a task with a fitting function based on an expected execution time of the task; and controlling the execution of the task according to the determined data processing quantity. A device for smoothing data processing is provided as well. This solution predicts the data processing quantity of the task to enable the processing time of the task to be controlled and reduces the passivity of the task executing, thus facilitates improving the stability and predictability of the data processing in the high capacity system.

Description

Method and device for smoothing data processing

The present invention relates to data processing techniques in the field of communications, and more particularly to a method and apparatus for smoothing data processing. Background technique

In an intelligent network system, it is often necessary to configure tasks in the background of the system to process user data, such as deletion of user information. In a large-capacity system, because the total amount of user data is very large, these tasks often take a lot of time and the system load is very large. In the commercial intelligent network system, due to the user's usage habits, a large amount of one-time data is often generated at special days such as the end of the month or the holiday, which causes the data volume of the system to be processed in a single day to fluctuate greatly, and the processing time fluctuates drastically. The system stability and predictability required by commercial systems are inconsistent and steps need to be taken to smooth out such fluctuations. Summary of the invention

In view of this, it is a primary object of the present invention to provide a method and apparatus for smoothing data processing to smooth fluctuations in data throughput and processing time in an intelligent network system.

In order to achieve the above object, the technical solution of the present invention is achieved as follows:

The invention discloses a method for smoothing data processing, the method comprising:

The data processing amount of the task is determined by the fitting function according to the expected execution time of the task; the execution of the task is controlled according to the determined data processing amount.

Further, the fitting function is used to describe the amount of data processing corresponding to the expected execution time of the data processing amount and the data processing time. The method further includes: presetting a forced end time of the task, determining whether the task execution time exceeds the forced end time during the task execution; forcibly ending the task when the determination is exceeded; and continuing execution when the determination is not exceeded The task until the end of the task.

The method further includes: at the end of the task, recording the actual end time of the task, and correcting the fit function based on the actual end time of the record.

Further, the expected execution time of the task is determined according to the execution time of the task of the history, or is determined by the difference between the expected end time of the set task and the current time.

The present invention also provides an apparatus for smoothing data processing, the apparatus comprising:

a data processing quantity determining module, configured to determine a data processing amount of the task by using a fitting function according to an expected execution time of the task;

The task execution control module is configured to control execution of the task according to the determined data processing amount.

The function is used to describe the data processing amount and the data processing time. Correspondingly, the data processing amount determining module is further configured to substitute the expected execution time of the task into the fitting function. A data processing amount corresponding to the expected execution time is obtained.

The device further includes: a determining module, configured to determine whether the task execution time exceeds a predetermined forced end time during the execution of the task, and notify the task execution control module to forcibly end the task when the determination is exceeded; At the time, the task execution control module is notified to continue executing the task until the task execution ends.

The apparatus further includes: a function correction module, configured to: at the end of the execution of the task, record an actual end time of the task, correct the fitting function according to the actual end time of the record, and provide the modified fitting function A module for data throughput determination.

The expected execution time of the task is determined by the execution time of the history, or by the difference between the expected end time of the set task and the current time.

A method and apparatus for smoothing data processing provided by the present invention, using time versus data The metric is predicted, and the appropriate data processing amount is calculated according to the expected processing time, which can smooth the processing of the data, and realize the control of the task processing time, so that the actual processing time of the data is close to the expected processing time; And online correction according to the actual processing time, realizes the system's self-adaptation, reduces the volatility of task execution, and is beneficial to improve the stability and predictability of data processing in large-capacity systems. DRAWINGS

1 is a flow chart of a method for smoothing data processing according to the present invention;

FIG. 1 is a schematic structural diagram of a device for smooth data processing according to the present invention. detailed description

The technical solutions of the present invention are further elaborated below in conjunction with the accompanying drawings and specific embodiments. In order to achieve smooth data processing, the core idea of the present invention is: according to the expected execution time of the task, and determining the data processing amount of the task through the fitting function, and then controlling the execution of the task according to the determined data processing amount, thereby realizing the task Control of processing time. The fitting function is used to describe the relationship between the data processing amount and the data processing time. Correspondingly, the expected execution time of the task is substituted into the fitting function, that is, the data processing amount corresponding to the expected execution time can be obtained. Preferably, at the end of the task execution, the actual end time of the task may be recorded, and the fitting function is corrected according to the actual end time of the record, and the modified fitting function is applied to subsequent task processing; , can achieve dynamic adaptive adjustment of the fitting function to achieve the purpose of reducing the volatility of task execution.

In an actual application, the expected execution time of the task may be an absolute time set by the user, that is, the expected execution time of the task is determined by the difference between the expected end time of the task and the current time; of course, according to a specific algorithm. Obtaining the expected execution time of the task, for example: According to the above idea, the present invention provides a method for smoothing data processing, as shown in FIG. 1 As shown, it mainly includes the following steps:

Step 101: Set an expected end time of the task.

The user sets the corresponding expected end time according to the task to be performed.

Step 102: Obtain a difference between an expected end time of the task and the current time, and record it as At. The set expected end time is subtracted from the current time to get At, At is the expected execution time of the task, which is used to indicate the expected time span of the task execution.

Step 103: Substituting the obtained expected execution time At into the fitting function to obtain a data processing amount. The fitting function is denoted as F ( ), F ( ) is used to describe the relationship between the data processing amount and the data processing time. Substituting At into F ( ) can obtain the data processing amount corresponding to At. Among them, F ( ) can give an empirical value in the process of implementation, and correct it in the subsequent operation; and F ( ) is not limited to the function, but can also be realized by the calculation processing method such as process.

The invention uses the data processing amount calculated by the fitting function F ( ) as the actual data processing amount of the task, and thereby controls the actual execution time of the task, thereby improving the stability and predictability of the data processing.

Step 104: Control the execution of the task according to the obtained data processing amount, and record the task execution time.

The task is executed according to the obtained data processing amount, and when the actual data processing amount of the task reaches the above data processing amount, the operation of the control task execution ends, and the task execution time is recorded.

Step 105: Compare the recorded task execution time with the preset forced end time, and determine whether the task execution time exceeds the forced end time. If yes, execute step 106; otherwise, perform step 107.

Step 106: When it is determined that the task execution time exceeds the forced end time, the task is forcibly ended, and the process proceeds to step 107.

Step 107: At the end of the task, record the actual end time of the task, and correct the fitting function according to the actual end time of the task. ₇ | Step 106, the actual end time of the task is the forced end time; proceed to step 105, if the task execution time does not exceed the forced end time, the task continues to execute, if the task continues during the execution of the task If the execution time reaches the forced end time, the task is forcibly terminated. The forced end time is the actual end time of the task. Alternatively, the actual end time of the task is recorded until the forced end time has not been reached until the task is completed.

By setting the forced end time, the present invention forcibly ends the task when the task execution time exceeds the set forced end time, thereby achieving absolute controllability of the task execution time.

The actual end time of the task is subtracted from the time when the task starts executing, and the actual execution time of the task is obtained, and the fitting function is corrected according to the function relationship between the actual execution time of the task and the data processing amount. By using the actual execution time to dynamically correct the fit function, the fit function can track the actual situation in time, thus more effectively fitting the functional relationship between data processing time and data throughput. The dynamic adaptive correction of the fit function can be used, but not limited to, a linear fit.

During the subsequent task execution, the above steps 101~107 are repeated, except that the modified fitting function will be used. It can be seen that through such a cyclic operation, dynamic adaptive adjustment of the fitting function can be realized, thereby obtaining a fitting function more in line with the actual situation, so as to better achieve the purpose of reducing the volatility of task execution.

In order to implement the above method for smoothing data processing, the present invention further provides an apparatus for smoothing data processing. As shown in FIG. 2, the apparatus includes: a data processing amount determining module 10 and a task execution control module 20. The data processing amount determining module 10 is configured to determine, according to an expected execution time of the task, a data processing amount of the task by using a fitting function; the fitting function is used to describe a function relationship between the data processing amount and the data processing time, and corresponding The data processing amount determining module 10 substitutes the expected execution time of the task into the fitting function, that is, obtains the data processing amount corresponding to the expected execution time. The task execution control module 20 is connected to the data processing amount determining module 10 for using the data processing amount. The execution of the data throughput control task determined by the module 10 is determined.

Preferably, the apparatus further includes a determination module 30 and a function correction module 40 on the basis of the above modules. The determining module 30 is configured to connect the task execution control module 20, configured to determine whether the task execution time exceeds a predetermined forced end time during the execution of the task, and notify the task execution control module 20 to end the task when the determination is exceeded, to implement the task. The execution time is absolutely controllable. The function correction module 40, the connection data processing amount determining module 10 and the task execution control module 20, for recording the actual end time of the task at the end of the task execution, correcting the fitting function according to the actual end time of the record, and correcting The resulting fit function is provided to the data throughput determination module 10 to effect dynamic adaptive adjustment of the fit function. The dynamic adaptive correction of the fit function can be used, but not limited to, a linear fit.

In summary, the method and apparatus of the present invention use time to predict the amount of data processing, and calculate an appropriate amount of data processing according to the expected processing time, which can smooth the processing of data and achieve control of task processing time. The actual processing time of the data is close to the expected processing time; the fitting function is proposed, and the online correction is performed according to the actual processing time, the system is adaptive, the volatility of the task execution is reduced, and the data processing of the large-capacity system is improved. Stability and predictability.

The above is only the preferred embodiment of the present invention and is not intended to limit the scope of the present invention.

Claims

Claim

A method for smoothing data processing, the method comprising:

2. The method of smoothing data processing according to claim 1, wherein the fitting function is used to describe a relationship between a data processing amount and a data processing time; and correspondingly, the expected execution time of the task is substituted into the The fitting function is obtained to obtain a data processing amount corresponding to the expected execution time.

The method of smoothing data processing according to claim 1, wherein the method further comprises: presetting a forced end time of the task, and determining whether the task execution time exceeds the forced end during the task execution process Time; when the judgment is exceeded, the task is forcibly terminated; when the judgment is not exceeded, the task is continued until the task ends.

4. The method of smoothing data processing according to claim 3, wherein the method further comprises: at the end of the task, recording an actual end time of the task, and comparing the fitting function according to the actual end time of the record. Make corrections.

The method for smoothing data processing according to any one of claims 1 to 4, wherein the expected execution time of the task is determined according to an execution time of the history recorded task, or by a set task The difference between the expected end time and the current time is determined.

6. A device for smoothing data processing, the device comprising:

7. The apparatus for smoothing data processing according to claim 6, wherein the fitting function is used to describe a function relationship between a data processing amount and a data processing time; Correspondingly, the data processing amount determining module is further configured to substitute the expected execution time of the task into the fitting function to obtain a data processing amount corresponding to the expected execution time.

8. The apparatus for smoothing data processing according to claim 6, wherein the device enters a predetermined forced end time, and when the determination is exceeded, notifying the task execution control module to forcibly end the task; when the determination is not exceeded Notifying the task execution control module to continue executing the task until the task execution ends.

9. The apparatus for smoothing data processing according to claim 8, wherein the apparatus further comprises: a function correction module, configured to record an actual end time of the task at the end of execution of the task, according to an actual end time of the record The fitting function is corrected, and the modified fitting function is provided to the data processing amount determining module.

10. The apparatus for smoothing data processing according to any one of claims 6 to 9, characterized in that the expected execution time of the task is from the execution time of the history, or the expected end time and current time of the set task. The difference is determined.