CN111353629A

CN111353629A - Frequency updating method and device

Info

Publication number: CN111353629A
Application number: CN201811582332.XA
Authority: CN
Inventors: 张琳; 王本玉; 马昭; 吴敏礽; 金晶
Original assignee: SF Technology Co Ltd
Current assignee: SF Technology Co Ltd
Priority date: 2018-12-24
Filing date: 2018-12-24
Publication date: 2020-06-30
Anticipated expiration: 2038-12-24
Also published as: CN111353629B

Abstract

The application discloses a frequency updating method and a device, training set data on a time axis are obtained, the time interval of the training set is [1, 2, …, k + i-1], the total amount of a sample set on the time axis is T, k is a time base point of the training set, i takes a value (1, 2,. once, T-k-h +1), and h is a period number needing extrapolation; inputting training set data into a training set fitting model; outputting data of a time point k + h + i-1 through a training set fitting model; calculating the error of the time point k + h + i-1; repeating steps S1-S3; calculating the average error of the verification set data, wherein the verification set data are the data of all time points k + h + i-1; the update frequency is determined. According to the technical scheme provided by the embodiment of the application, the relation between the updating frequency and the error is determined through a cross validation method of time series prediction, the more reasonable updating frequency is further provided, and the improvement of the prediction effect and the optimization of frequency updating are realized.

Description

Frequency updating method and device

Technical Field

The present invention relates to the field of logistics, and in particular, to a method and apparatus for frequency updating.

Background

The error (mean absolute percentage error) is an important index for measuring the time sequence prediction effect. The long-term time sequence prediction has heteroscedasticity and hysteresis effect, and the prediction result is not stable on a time axis. From the time span of prediction, long-term prediction (extrapolation period exceeds 30 days) has the characteristics of low precision and large trend variation. The long-term prediction error of the daily data has strong correlation with the updating frequency, and generally, the more frequent the prediction updating is, the higher the accuracy is, i.e., the smaller the error is. However, considering the limited computing resources and the deviation of daily logistics components, a trade-off needs to be made between the update frequency and the prediction error, so as to achieve a better prediction accuracy with a smaller update frequency (i.e. update period) under the guarantee of the computing resources.

How to obtain the relationship between the update frequency and the prediction error; the cross validation method is commonly used for evaluating the prediction performance of the model, and the basic idea is to group the original data, randomly extract one part as a training set to train the model, and randomly extract the other part as a test set to evaluate the model, however, the conventional cross validation method randomly divided into several parts is not suitable for the timing problem because of the following reasons:

1. irreversibility of timing. The basic idea of timing prediction is to infer the future by learning the history, requiring that the test set must be later in time than the training set;

2. the test set is arbitrarily selected. The test set of the traditional k-fold cross validation method is selected quite randomly, namely, the test set is randomly divided into k parts, and the sample size of each training set is random. However, the time series prediction depends on the quality of the historical data, the more historical inputs can provide better model fitting, and therefore the time series data cross validation must retain data for a certain time as a training set.

Disclosure of Invention

In view of the above-mentioned drawbacks and deficiencies of the prior art, it is desirable to provide a frequency updating method and apparatus.

In a first aspect, a method for frequency update is provided, which includes the steps of:

s1: acquiring training set data on a time axis, wherein the time interval of the training set is [1, 2, …, k + i-1], the total amount of the sample set on the time axis is T, k is a time base point of the training set, i takes a value (1, 2, 1, T-k-h +1), and h is the period number needing extrapolation;

s2: inputting the training set data into a training set fitting model;

s3: outputting data of a time point k + h + i-1 through the training set fitting model;

s4: calculating the error of the time point k + h + i-1;

s5: repeating steps S1-S4;

s6: calculating the average error of the verification set data, wherein the verification set data are the data of all time points k + h + i-1;

s7: and determining the updating frequency according to the relation graph of the average error and the updating frequency.

In a second aspect, a frequency updating apparatus is provided, which includes:

the acquisition module is used for acquiring training set data on a time axis, the time interval of the training set is [1, 2, …, k + i-1], the total amount of the sample set on the time axis is T, k is a time base point of the training set, and i takes a value (1, 2., T-k-h + 1);

the input module is used for inputting the training set data into a training set fitting model;

the output module is used for outputting data of a time point k + h + i-1 through the training set fitting model;

the point error calculation module is used for calculating the error of the time point k + h + i-1;

the average error calculation module is used for calculating the average error of verification set data, wherein the verification set data are data of all time points k + h + i-1;

and the updating frequency determining module is used for determining the updating frequency according to the relation graph of the average error and the updating frequency.

According to the technical scheme provided by the embodiment of the application, the relation between the updating frequency and the error is determined through a cross validation method of time series prediction, the more reasonable updating frequency is further provided, and the improvement of the prediction effect and the optimization of frequency updating are realized.

Drawings

Other features, objects and advantages of the present application will become more apparent upon reading of the following detailed description of non-limiting embodiments thereof, made with reference to the accompanying drawings in which:

FIG. 1 is a flowchart illustrating a frequency updating method according to the present embodiment;

FIG. 2 is a schematic structural diagram of a frequency updating apparatus in this embodiment;

FIGS. 3-5 are schematic diagrams of the prediction variation in this embodiment;

FIGS. 6-7 are graphs showing the relationship between the update frequency and the average error in this embodiment;

FIG. 8 is a graph illustrating the relationship between the update frequency and the calculation cost in the present embodiment;

fig. 9 is a schematic structural diagram of an apparatus provided in this embodiment.

Detailed Description

The present application will be described in further detail with reference to the following drawings and examples. It is to be understood that the specific embodiments described herein are merely illustrative of the relevant invention and not restrictive of the invention. It should be noted that, for convenience of description, only the portions related to the present invention are shown in the drawings.

It should be noted that the embodiments and features of the embodiments in the present application may be combined with each other without conflict. The present application will be described in detail below with reference to the embodiments with reference to the attached drawings.

Referring to fig. 1, the present embodiment provides a frequency updating method, including:

s2: inputting the training set data into a training set fitting model;

s4: calculating the error of the time point k + h + i-1;

s5: repeating steps S1-S4;

s7: and determining an updating frequency range according to the average error and the updating frequency relation graph.

The frequency updating method provided in the embodiment is based on daily prediction, the updating frequency is determined through the method, the frequency is further optimized, and by taking the daily prediction updating frequency of the logistics quantity as an example, the past data cannot be predicted by adopting future data, namely, the irreversibility of the data on time cannot be changed; the long-term trend, multi-frequency and isochronous components are extracted according to the necessary historical information to ensure that the training set number is sufficient. Assuming that the total observed value of the time sequence sample set is T, fixing a time window k as a base point of a training set to ensure that credible prediction can be obtained; according to the above method, the following two embodiments are provided,

firstly, when h is 1, testing data after a training set, selecting a point k + i on a time axis as a verification set according to the steps, inputting the data of the training set into a training set fitting model to obtain data of the point k + i, calculating an error of the point k +1, repeating the steps, predicting forward in each rolling period, and continuously rolling along with the prediction, wherein the point with a hatched line in the middle is the verification set, and the solid points in the front are the training set;

secondly, when h is not 1, supposing that extrapolation is needed for multiple periods, selecting a point k + h + i-1 on a time axis as a test set, wherein the steps are the same as the steps described above, as shown in fig. 4 and 5, the h period is predicted forward with continuous rolling prediction, fig. 4 is a predicted graph of two rolling periods, fig. 5 is a rolling graph of three prediction periods, and the longer the time needed to predict is, the larger the h value is.

In this embodiment, the relationship between the update frequency and the error is determined by a cross validation method of time series prediction, and a more reasonable update frequency is further proposed, so that the prediction effect is improved.

Further, k is a natural number not less than 0.5T.

The frequency update in this embodiment is realized by time series prediction, where a sufficient training set needs to be set to facilitate the accuracy of the verification set calculated according to the fitting model, and therefore, data of at least half of the total amount of the sample set is selected as the training set.

Further, the error of the time point k + h + i-1 is | (X-Y)/X |, where X is an actual measurement value and Y is a training simulation value, and the average error is (∑ ((X-Y)/X) × 100%)/N, where N is ∑ (k + h + i-1).

In this embodiment, the calculation formulas of the verification set time point error and the average error are given.

Further, if the average error is not greater than a first threshold, determining an update frequency according to the average error, and if the average error exceeds the first threshold, not updating the frequency.

Further, the method also comprises the step of drawing a relation graph of the average error and the updating frequency;

the method specifically comprises the following steps: selecting the mesh points with the average error smaller than a second threshold value and the mesh points with the average error between the second threshold value and the first threshold value, wherein the time length T1 is a training set, the set positions are predicted forward along the time axis, the prediction times are the updating frequency,

inputting a training set with the time length of T1 into a training set fitting model, obtaining an average error under each updating frequency, and respectively determining a relation graph of the average error of the mesh points with the errors below a second threshold and the mesh points with the errors between the second threshold and the first threshold and the updating frequency.

In this embodiment, a relationship between an average error and an update frequency is determined according to a certain error, where mesh points with an error below a first threshold are determined to be optimized by the update frequency, mesh points with an error exceeding the first threshold are not optimized by the method, a lowest threshold for optimization can be determined by the above method, and after the lowest threshold exceeds the first threshold, update frequency optimization needs to be performed by other methods, so as to ensure an optimization effect, where the first threshold is set to be 35% to 45%, and 40% is preferably used as the first threshold in this embodiment; further, the error is divided into two parts, that is, the error is below a second threshold and between the second threshold and the first threshold, where the second threshold is set to be smaller than the first threshold, the range within the first threshold is subdivided to a certain extent, and the update frequency settings under different thresholds are different, so that the optimization scheme in this embodiment is more detailed, where the second threshold is set to be between 15% and 25%, in this embodiment, 20% is preferably used as the second threshold, in this embodiment, a certain time length is set as the training set, and the set times are all predicted forward, in this embodiment, 30 times are preferably used, and the average error of each time is obtained according to the steps S1-S6, so that a relation graph of the average error and the update frequency is further drawn.

Then, the update frequency is determined according to the average error, different errors of the optimization method in this embodiment correspond to different update frequencies, when the average error is less than 20%, the relationship between the average error and the update frequency is as shown in fig. 6, when frequency update is performed, theoretically, the smaller the average error is, the better the effect is, for example, a mesh point with an error within 20%, the prediction frequency is shortened from once update every 31 days to once update every 7 days, the total average error is reduced from about 14.5% to about 10%, and the average accuracy is increased by 4.5%; when the average error is between 20% and 40%, the relationship between the average error and the update frequency is shown in fig. 7, for example, for a mesh point with an error within 20-40%, the prediction frequency is shortened from update every 31 days to update every 7 days, the overall average error is reduced from about 29% to about 19%, and the average accuracy is increased by 10%.

Further, if the total amount of sample sets T includes a special time point, the selection of T needs to meet the requirement that T > -k +2n, where n is the number of verification set data.

In the process of carrying out prediction optimization according to the day, the deviation of some special time points, such as holidays or weekends, to the whole prediction needs to be considered, therefore, on a day time line meeting the special time points, certain adjustment needs to be carried out, firstly, the total amount of a sample set is expanded, the total amount of the sample set at least comprises a training set and twice the length of a verification set, and enough verification sets are ensured to be available in the calculation of average errors so as to remove the influence of the special time points; and further, the intersection of the verification sets is obtained in the calculation of the average error, and the number of the intersection is n.

Further, the method also comprises the following steps: and determining the updating frequency according to the updating frequency and the calculation cost relation graph. In this embodiment, in addition to the relationship between the update frequency and the average error, a relationship between the update frequency and the calculation cost needs to be considered, and a relationship diagram is shown in fig. 8, theoretically, in order to save resources and cost, the update frequency needs to be determined with a lower calculation cost, so that an update frequency range below a certain average error is determined according to the relationship diagram between the average error and the update frequency, and then the update frequency with the lowest calculation cost in the update frequency range is selected as the optimized update frequency according to the relationship diagram between the update frequency and the calculation cost shown in fig. 8; the method can provide a reasonable updating frequency, balance average errors and calculation cost, and improve the long-term prediction effect.

As shown in fig. 2, the present embodiment further provides a frequency updating apparatus, including:

and the updating frequency determining module is used for determining an updating frequency range according to the average error and the updating frequency relation graph.

The working principle of the device shown in fig. 2 refers to the method shown in fig. 1, and is not described in detail here.

Further, the system further comprises a second update frequency determination module, configured to determine the update frequency according to the update frequency and the calculation cost relationship diagram.

In this embodiment, in addition to the relationship between the update frequency and the average error, a relationship between the update frequency and the calculation cost needs to be considered, and a relationship diagram is shown in fig. 8, theoretically, in order to save resources and cost, the update frequency needs to be determined with a lower calculation cost, so that an update frequency range below a certain average error is determined according to the relationship diagram between the average error and the update frequency, and then the update frequency with the lowest calculation cost in the update frequency range is selected as the optimized update frequency according to the relationship diagram between the update frequency and the calculation cost shown in fig. 8; the method can provide a reasonable updating frequency, balance average errors and calculation cost, and improve the long-term prediction effect.

Further, selecting the mesh points with the average error smaller than a second threshold value and the mesh points with the average error between the second threshold value and the first threshold value, using the time length T1 as a training set, predicting the set times forward along the time axis, wherein the predicted times are the updating frequencies, inputting the training set with the time length T1 into a training set fitting model to obtain the average error under each updating frequency,

and the relational graph drawing module is used for determining a relational graph of the average error of the mesh points with the errors below a second threshold value and the updating frequency and a relational graph of the average error of the mesh points with the errors between the second threshold value and the first threshold value and the updating frequency according to the average error and the updating frequency of each updating frequency.

In this embodiment, the device is further used to determine a relationship graph between the average error and the update frequency, and for a specific working principle, reference is made to the relationship graph drawing method shown in the above step, which is not described herein again.

As shown in fig. 9, as another aspect, the present application also provides an apparatus 300 including one or more Central Processing Units (CPUs) 301 that can perform various appropriate actions and processes according to a program stored in a Read Only Memory (ROM)302 or a program loaded from a storage section 308 into a Random Access Memory (RAM) 303. In the RAM 303, various programs and data necessary for the operation of the system 300 are also stored. The CPU 301, ROM 302, and RAM 303 are connected to each other via a bus 304. An input/output (I/O) interface 305 is also connected to bus 304.

The following components are connected to the I/O interface 305: an input portion 306 including a keyboard, a mouse, and the like; an output section 307 including a display such as a Cathode Ray Tube (CRT), a Liquid Crystal Display (LCD), and the like, and a speaker; a storage section 308 including a hard disk and the like; and a communication section 309 including a network interface card such as a LAN card, a modem, or the like. The communication section 309 performs communication processing via a network such as the internet. A drive 310 is also connected to the I/O interface 305 as needed. A removable medium 311 such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, or the like is mounted on the drive 310 as necessary, so that a computer program read out therefrom is mounted into the storage section 308 as necessary.

In particular, the process described above with reference to fig. 1 may be implemented as a computer software program, according to an embodiment of the present disclosure. For example, embodiments of the present disclosure include a computer program product comprising a computer program tangibly embodied on a machine-readable medium, the computer program comprising program code for performing a sorting configuration method. In such an embodiment, the computer program may be downloaded and installed from a network through the communication section 309, and/or installed from the removable medium 311.

The flowchart and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments of the present invention. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that, in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams and/or flowchart illustration, and combinations of blocks in the block diagrams and/or flowchart illustration, can be implemented by special purpose hardware-based systems which perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.

As yet another aspect, the present application also provides a computer-readable storage medium, which may be the computer-readable storage medium included in the apparatus in the above embodiments; or it may be a separate computer readable storage medium not incorporated into the device. The computer readable storage medium stores one or more programs for use by one or more processors in performing the frequency update methods described herein.

The units or modules described in the embodiments of the present application may be implemented by software or hardware. The described units or modules may also be provided in a processor, for example, each of the described units may be a software program provided in a computer or a mobile intelligent device, or may be a separately configured hardware device. Wherein the designation of a unit or module does not in some way constitute a limitation of the unit or module itself.

The above description is only a preferred embodiment of the application and is illustrative of the principles of the technology employed. It will be appreciated by a person skilled in the art that the scope of the invention as referred to in the present application is not limited to the embodiments with a specific combination of the above-mentioned features, but also covers other embodiments with any combination of the above-mentioned features or their equivalents without departing from the inventive concept. For example, the above features may be replaced with (but not limited to) features having similar functions disclosed in the present application.

Claims

1. A method for frequency updating, comprising the steps of:

s2: inputting the training set data into a training set fitting model;

s4: calculating the error of the time point k + h + i-1;

s5: repeating steps S1-S4;

2. The frequency updating method according to claim 1, further comprising the steps of: and determining the updating frequency according to the updating frequency and the calculation cost relation graph.

3. The frequency updating method according to claim 1 or 2, wherein k is a natural number not less than 0.5T.

4. The method of claim 1, wherein the time k + h + i-1 has an error | (X-Y)/X |, where X is an actual measurement value and Y is a training simulation value, and wherein the average error is (∑ ((X-Y)/X) × 100%)/N, where N is ∑ (k + h + i-1).

5. The method according to claim 4, wherein if the average error is not greater than a first threshold, determining an update frequency according to the average error, and if the average error exceeds the first threshold, not updating the frequency.

6. The frequency updating method of claim 5, further comprising the steps of plotting the average error versus the update frequency;

the method specifically comprises the following steps: selecting the mesh points with the average error smaller than a second threshold value and the mesh points with the average error between the second threshold value and the first threshold value, using the time length T1 as a training set, predicting the set times forward along the time axis, wherein the predicted times are the updating frequency,

7. The method according to claim 1 or 2, wherein if the total amount of sample sets T includes a specific time point, T > k +2n is selected, and n is the number of verification set data.

8. A frequency updating apparatus, comprising:

and the first updating frequency determining module is used for determining an updating frequency range according to the average error and the updating frequency relation graph.

9. The frequency updating apparatus of claim 7, further comprising a second updating frequency determining module for determining the updating frequency according to the relationship between the updating frequency and the computation cost.

10. The frequency updating apparatus according to claim 7, wherein a grid point having an average error smaller than a second threshold and a grid point having an average error between the second threshold and a first threshold are selected, the time length T1 is a training set, a set number of times is predicted forward along the time axis, the predicted number is the updating frequency, the training set having the time length T1 is input into the training set fitting model to obtain the average error at each updating frequency,