CN108038085A - The data calibration method and device of real-time task - Google Patents
The data calibration method and device of real-time task Download PDFInfo
- Publication number
- CN108038085A CN108038085A CN201711447741.4A CN201711447741A CN108038085A CN 108038085 A CN108038085 A CN 108038085A CN 201711447741 A CN201711447741 A CN 201711447741A CN 108038085 A CN108038085 A CN 108038085A
- Authority
- CN
- China
- Prior art keywords
- real
- time
- result
- data
- time task
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F17/00—Digital computing or data processing equipment or methods, specially adapted for specific functions
- G06F17/10—Complex mathematical operations
- G06F17/18—Complex mathematical operations for evaluating statistical data, e.g. average values, frequency distributions, probability functions, regression analysis
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q30/00—Commerce
- G06Q30/02—Marketing; Price estimation or determination; Fundraising
- G06Q30/0201—Market modelling; Market analysis; Collecting market data
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q30/00—Commerce
- G06Q30/02—Marketing; Price estimation or determination; Fundraising
- G06Q30/0201—Market modelling; Market analysis; Collecting market data
- G06Q30/0202—Market predictions or forecasting for commercial activities
Landscapes
- Engineering & Computer Science (AREA)
- Business, Economics & Management (AREA)
- Physics & Mathematics (AREA)
- Accounting & Taxation (AREA)
- Development Economics (AREA)
- Finance (AREA)
- Strategic Management (AREA)
- Data Mining & Analysis (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Entrepreneurship & Innovation (AREA)
- Mathematical Analysis (AREA)
- Economics (AREA)
- General Business, Economics & Management (AREA)
- Game Theory and Decision Science (AREA)
- Marketing (AREA)
- Mathematical Physics (AREA)
- Pure & Applied Mathematics (AREA)
- Computational Mathematics (AREA)
- Mathematical Optimization (AREA)
- Life Sciences & Earth Sciences (AREA)
- Evolutionary Biology (AREA)
- Operations Research (AREA)
- Probability & Statistics with Applications (AREA)
- Bioinformatics & Computational Biology (AREA)
- Algebra (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Databases & Information Systems (AREA)
- Software Systems (AREA)
- General Engineering & Computer Science (AREA)
- Management, Administration, Business Operations System, And Electronic Commerce (AREA)
Abstract
The invention discloses a kind of data calibration method of real-time task, including step：The off-line calculation of time segment is carried out to multiple history real time datas of real-time task, obtains off-line calculation result corresponding with each period；According to off-line calculation as a result, carrying out calibration process to real time data result of the real-time task in current slot, display data result of the real-time task in current slot is obtained.The embodiment of the invention also discloses a kind of data calibration device of real-time task.A technical solution in above-mentioned technical proposal is handled by carrying out the off-line calculation of time segment to history real time data of the real-time task before current slot, obtains corresponding to the off-line calculation result of each period；Off-line calculation result is called to carry out the display data result that calibration process obtains real-time task and is used for current slot to the real time data result in current slot.So as to fulfill the data result of real-time exhibition, more accurately purpose, user experience are more preferable.
Description
Technical field
The present invention relates to real time data computing technique field, a kind of data calibration method more particularly to real-time task and
Device.
Background technology
With modern commerce and information-based development, the calculating processing requirement for data, such as data processing speed, place
Reason scale and the precision of processing etc., are all stepping up.For the occasion of internet+business, such as market or net
Stand, it will usually need to grasp the critical data situations such as user volume, trading volume, preference data or click volume in time, so as to and
Shi Jinhang decision-makings adjust.In order to obtain timely critical data feedback, the technology of mainstream be by Real-Time Task System, such as
Storm (distributed real-time platform) stream calculation frame is calculated in real time, is fed back with obtaining required critical data.However, pass
Data in the Real-Time Task System of system to be likely to occur the different of task collapse since there are of short duration time delay, and Computational frame
Often, so that, obtaining results of calculation, there are deviation.
The content of the invention
Based on above-mentioned analysis, it is necessary to for results of calculation asking there are deviation obtained by traditional Real-Time Task System
A kind of topic, there is provided the data calibration device of a kind of data calibration method of real-time task and real-time task.
The embodiment of the present invention provides a kind of data calibration method of real-time task, including step：
The off-line calculation of time segment is carried out to multiple history real time datas of real-time task, is obtained corresponding with each period
Off-line calculation result；
According to the off-line calculation as a result, carrying out school to real time data result of the real-time task in current slot
Quasi- processing, obtains display data result of the real-time task in current slot.
In one of the embodiments, according to the off-line calculation as a result, to the real-time task in current slot
Real time data result carry out calibration process, the step of obtaining display data result of the real-time task in current slot
Before, further include step：
According to the real time data in the current slot as a result, determining the execution state of the real-time task；
The predetermined scheme of the calibration process is determined according to the execution state；Wherein, the predetermined scheme includes combining
Processing or data prediction simulation process.
In one of the embodiments, the display data result includes calibration data result；
According to the off-line calculation as a result, carrying out school to real time data result of the real-time task in current slot
Quasi- processing, in the step of obtaining display data result of the real-time task in current slot, including：
When the execution state of the definite real-time task is normal condition, to the off-line calculation knot of a upper period
Fruit is combined processing with the real time data result in the current slot, obtains the calibration data result.
In one of the embodiments, the combination processing is included at the direct adduction processing of index or index net value adduction
Reason.
In one of the embodiments, the display data result includes analogue data result；
According to the off-line calculation as a result, carrying out school to real time data result of the real-time task in current slot
Quasi- processing, in the step of obtaining display data result of the real-time task in current slot, further includes：
Obtain the real time data result after the calibration of each period before the current slot；Wherein, after calibration
The real time data result is as a result, in corresponding each period according to each period corresponding off-line calculation
The real time data result is calibrated, the real time data result after being calibrated；
When the execution state of the definite real-time task is abnormality, according to the real time data knot after each calibration
Fruit carries out data prediction simulation process, obtains the analogue data result.
In one of the embodiments, the duration of the period is greater than or equal to 15 minutes.
The embodiment of the present invention also provides a kind of data calibration device of real-time task, including：
Off-line calculation module, for multiple history real time datas of real-time task to be carried out with the off-line calculation of time segment,
Obtain off-line calculation result corresponding with each period；
Calibration process module, for according to the off-line calculation as a result, to the real-time task in current slot
Real time data result carries out calibration process, obtains display data result of the real-time task in current slot.
In one of the embodiments, the data calibration device of the real-time task further includes task status and determines mould
Block, for according to the real time data in the current slot as a result, determine the real-time task execution state；
Processing scheme determining module, for determining the predetermined scheme of the calibration process according to the execution state；Wherein,
The predetermined scheme includes combining processing or data prediction simulation process.
The embodiment of the present invention also provides a kind of computer equipment, including memory and processor, is stored on the memory
There is the computer program that can be run on the processor, the processor realizes the reality when performing the computer program
When task data calibration method the step of.
The embodiment of the present invention also provides a kind of computer-readable recording medium, is stored thereon with computer program, the meter
The step of data calibration method of the real-time task is realized when calculation machine program is executed by processor.
A technical solution in above-mentioned technical proposal has the following advantages that and beneficial effect：
By to real-time task before current slot history real time data carry out time segment off-line calculation at
Reason, obtains corresponding to the off-line calculation result of each period；Off-line calculation result is called to the real time data knot in current slot
Fruit carries out calibration process and obtains the display data result that real-time task is used for current slot.The embodiment of the present invention solves tradition
Real-Time Task System obtained by results of calculation there are the problem of deviation, reaching to make real-time exhibition result more accurately effect,
So as to preferably improve user experience.
Brief description of the drawings
Fig. 1 is the data calibration method flow diagram of the real-time task of one embodiment of the invention；
Fig. 2 is the data calibration method flow diagram of the real-time task of another embodiment of the present invention；
Fig. 3 is the data calibration method idiographic flow schematic diagram of the real-time task of another embodiment of the present invention；
Fig. 4 is the modular structure schematic diagram of the data calibration device of the real-time task of one embodiment of the invention.
Embodiment
The data calibration method of real-time task to the present invention and the embodiment of device are made detailed below in conjunction with the accompanying drawings
Thin description.The preferred embodiment of the present invention is given in attached drawing.But the present invention can come in fact in many different forms
It is existing, however it is not limited to embodiment described herein.On the contrary, the purpose for providing these embodiments is to make the disclosure
More thorough and comprehensive.
Calculate in real time, be to want to obtain the key message of real-time for user, such as the current flow of the people in market, website are worked as
The information such as preceding pageview, the current click volume of shopping website part commodity, pointedly to formulate the development of the demand of marketing strategy
And a kind of data come calculate processing mode in real time.The basic demand calculated in real time is its real-time.In today of mass data,
The challenge of bigger is faced with to the demand that real time data is shown, conventional unit real time data computing technique is powerless should
It is right.With the fast development of distributed computing technology, distributed real-time data computing technique is increasingly taken seriously.It is corresponding to occur
A variety of real-time distributed Computational frames.Wherein representative is to develop more early and widely used Storm stream calculation frames
Frame.
Storm stream calculations frame transmits real data flow in itself, using backup and confirms that character (ACK) replys confirmation machine
System, ensures that the data that each flows into can be processed to.During realizing the present invention, inventor has found, based on Storm
In the Real-Time Task System that stream calculation frame is established, on the one hand, data are subscribed to message system (Kafka) from distributed post and flowed into
There is very short delay into Storm stream calculation frames, when judging in program process time field, such as accurately
Judge to 1 time field when small, have partial data due to postponing misjudged filter out；Over time, thus make
Into calculating deviation can be exaggerated.On the other hand calculating task it is possible that abnormal collapse, flows into real-time after exception in real time
Data can not all be calculated, and cause the wireless normal presentation of real time data under abnormal time.Therefore the real-time number of actual count
According to and actual value can be there are deviation, deviation will influence whether user experience with the continuous increase of time.
Referring to Fig. 1, to solve results of calculation obtained by above-mentioned traditional Real-Time Task System, there are the technology of deviation
Problem, the embodiment of the present invention provide a kind of data calibration method of real-time task, comprise the following steps：
Multiple history real time datas of real-time task are carried out the off-line calculation of time segment, obtained and each period by S12
Corresponding off-line calculation result.
Wherein, history real time data can be before real-time task goes to current slot, all have been subjected to real-time meter
The real time data of calculation process, be it is opposite with for current slot.The duration of period can according to for obtain a time
Section off-line calculation result and carry out the required duration of off-line calculation to set.For example, the period to a scheduled duration
Interior history real time data carries out off-line calculation, and requires the precision of the off-line calculation result of gained to reach certain rank, then
Foregoing off-line calculation process, which proceeds to, just obtains required off-line calculation result i.e. only.In this way, can the off-line calculation process
All required duration is set as the duration of above-mentioned period.When the duration of period for example can be 1 small.It is appreciated that
In same Real-Time Task System, the duration of each period can be identical.
Optionally, the mode of time segment may, for example, be：In a manner of time shaft, by 1 it is small when in units of, will by work as
1 before the preceding moment it is small when duration be set to current slot t0, then it is current between section t0A upper period be t-1, upper one
Time period t-1A period upper again be then t-2, and so on.In this way, Real-Time Task System can be in t-1The knot of period
At the beam moment, start to t-1All history real time datas in period are settled accounts offline；To current slot t0At the end of
Carve, for t-1The off-line calculation of period terminates and obtains t-1The offline checkout result of period.In this way, for every other
Period can be made identical similarly to understand.
Specifically, real-time task is in the process of implementation, Real-Time Task System can be to continuing by Storm stream calculation frames
The real time data of output divides on a time period, and at the end of any one period, i.e., to all real-time in the period
Data carry out off-line calculation, obtain an off-line calculation result for corresponding to the period.It is appreciated that by Storm flowmeters
The real time data in the period that frame persistently exports is calculated, may correspond to the result of calculation of multiple real time datas, namely
Real time data result.In this way, after whole real-time task starts, the real time data of each period is carried out corresponding
Off-line calculation, obtains corresponding off-line calculation result of each period.
In wherein an optional embodiment, the duration of period may be greater than or equal to 15 minutes, less than 15 points
Clock will be too short due to duration less than 10 minutes, cause off-line calculation not have time enough to complete and substantially reduce calculating essence
Degree, is unfavorable for follow-up data calibration.But calculating task is relatively simple, data volume is less or computational accuracy requires relatively low feelings
Condition, can be set to the duration less than 15 minutes, and in other words, the present embodiment is only to provide optional embodiment, and not exclusive
Setting means；Therefore, the duration of period can according to for obtain a period reasonable off-line calculation result carry out from
The required duration of line computation, carrys out the specific duration for being set in more than 15 minutes.The duration of period be preferably 1 it is small when, such as
This, the smaller real-time task real-time calculation error accumulative with the time and can ensure the off-line calculation process of each period
There can be enough time completion.
S14, according to off-line calculation as a result, being carried out to real time data result of the real-time task in current slot at calibration
Reason, obtains display data result of the real-time task in current slot.
Wherein, calibration process can be：To the corresponding off-line calculation result of the upper period and current slot
Interior each real time data result carries out data adduction, obtains the calibration process of calibration data result.Calibration process can also be：Root
According to before current slot, two or more continuous no data exception history periods it is corresponding, two or two
More than off-line calculation as a result, carry out data prediction and simulation, obtain one closest to should respectively have real-time number in current slot
According to the analogue data result of result.Display data result can be current slot show user real time data as a result,
It can include foregoing calibration data result and analogue data result.
Specifically, Real-Time Task System can be according to the off-line calculation result of a upper period to current slot
Interior real time data result carries out calibration process, obtains display data result of the real-time task in current slot.Appoint in real time
Business system can also carry out data prediction according to two or more the off-line calculation result before current slot
And simulation, obtain real-time task accurate display data result in current slot.
In this way, Real-Time Task System carries out time segment by the history real time data exported to Storm stream calculations frame
Off-line calculation processing, obtain correspond to each period off-line calculation result；Off-line calculation result is called in current slot
Real time data result carry out calibration process obtain real-time task be used for current slot display data as a result, solving tradition
Real-Time Task System obtained by results of calculation there are the problem of deviation, having reached makes real-time exhibition result more accurately imitate
Fruit, is substantially improved the fault freedom of real-time task, preferably improves user experience.
Referring to Fig. 2, in one embodiment, before step S14, step can also be included：
S13, according to the real time data in current slot as a result, determining the execution state of real-time task.
Wherein, the execution state of real-time task can be real time data result there are deviation but still can have real time data knot
Fruit shows the displaying normal condition of user or abnormal collapse occurs in the real-time calculating task of current slot, causes
No real time data shows the displaying abnormality of user in current slot.
It is appreciated that Real-Time Task System can be in current slot real time data result carry out calibration process it
Before, the execution state of real-time task is determined according to the real time data result in current slot.For example, in current slot,
When Storm stream calculation frames have the real time data result in output current slot, the execution state for determining real-time task is
Normal condition.In current slot, the real time data result in current slot is not exported when Storm stream calculations frame
When, the execution state for determining real-time task is abnormality.In this way, Real-Time Task System is by prejudging the execution shape of real-time task
State, the real time data result that can be effectively directed in the current slot under different conditions carry out calibration process, save system
Spending and raising calibration efficiency.
S135, the predetermined scheme of calibration process is determined according to execution state.
Wherein, predetermined scheme can be the combination processing in above-described embodiment or data prediction and simulation process.
Specifically, real-time task can be called and be pre-stored in after the execution state of real-time task is determined according to execution state
Predetermined scheme in system carries out required calibration process, in this way, called for different execution states at suitable calibration
The specific predetermined scheme of reason, can save overhead and improve data calibration efficiency.
Referring to Fig. 3, in one of the embodiments, specifically, for step S14, step can be included：
S142, when the execution state of definite real-time task is normal condition, to the off-line calculation result of a upper period
Processing is combined with the real time data result in current slot, obtains calibration data result.
Wherein, can be the direct adduction processing of index or the processing of index net value adduction with reference to processing.For example, for website
Number of starts index, can be handled the direct adduction of number of starts index in surrounding time section, obtained result is calibration
Number of starts data afterwards.In another example for any active ues quantitative index of website, typically directly adduction can produce repetition meter
Calculate, therefore BloomFilter (binary vector data structure) algorithm can be utilized, judge any active ues in data structure service
Whether any active ues in device (Redis) were integrated into the same day has existed.On the day of being calculated and being appended to there is no if
In any active ues set, net value adduction processing is carried out so as to fulfill to any active ues quantitative index.
Specifically, Real-Time Task System called the upper time when the execution state of definite real-time task is normal condition
The off-line calculation of section is as a result, and carried out the real time data result in the off-line calculation result of a upper period and current slot
With reference to processing, calibration data result is obtained.In this way, being handled by combining, the data result for showing user can be greatly improved
Accuracy.
In another embodiment, specifically, for step S14, step can also be included：
S144, obtains the real time data result after the calibration of each period before current slot.
Wherein, the real time data result after calibration can be as a result, to correspondence according to corresponding off-line calculation of each period
Each period in real time data result calibrated, the real time data result after the calibration of obtained each period.
It is appreciated that Real-Time Task System is after the off-line calculation result of a period is obtained, when can also be according to this
Between section off-line calculation result in the period real time data result carry out calibration process, to calibrate the institute in the period
There is real time data as a result, so as to obtain the real time data result after the calibration in the period.Real-Time Task System is appointed in real time
In the execution time of business, above-mentioned calibration process process can be carried out for the real time data result in each period.
So as to which over time, Real-Time Task System can obtain each period corresponding off-line calculation result and every
The real time data result after calibration in one period.
S146, when the execution state of definite real-time task is abnormality, according to the real time data result after each calibration
Data prediction simulation process is carried out, obtains analogue data result.
Wherein, the real time data result after each calibration can be within the same day, the execution state of real-time task is abnormal shape
Real time data result after calibrated before the period of state, in all periods.Real time data result after each calibration
Quantity can be two or more quantity.
It is appreciated that Real-Time Task System is obtained when the execution state of definite real-time task is abnormality according to foregoing
Real time data after each calibration taken is as a result, progress data prediction and simulation process, obtain one and exist closest to current slot
The analogue data result of due real-time exhibition data result during normal condition.In this way, in the feelings of real-time task short time exception
Under condition, remain able to obtain relatively accurate real-time exhibition data result, greatly improve the fault-tolerance of real-time calculating task, carry
User experience is risen.
Optionally, by taking any active ues of application as an example, Real-Time Task System reads the current of real-time task in interface routine
When time segment data is invalid (null), namely during no data, it is abnormality to determine real-time task.Real-Time Task System is read
Down to date it is all carried out calibration periods in the number of starts, Add User number and active users, determine weighting
The average number of starts afterwards and the sum of the number that Adds User, with the ratio of the averagely number of starts；If the ratio is more than setting value, such as
Setting value is 1, then judges that the random difference that the number of starts reduces corresponding proportion is simulation active users.Simulate active users
One kind in i.e. foregoing analogue data result.
In one embodiment, the real time data result in the various embodiments described above and off-line calculation result can be stored in point
In cloth memory object caching system (Memcached).In this way, it can be obtained more using the data-query interfaces of Memcached
Fast data query reading speed.
Referring to Fig. 4, the embodiment of the present invention provides a kind of data calibration device 100 of real-time task, including off-line calculation
Module 22 and calibration process module 24.Off-line calculation module 22 is used to carry out timesharing to multiple history real time datas of real-time task
Between section off-line calculation, obtain off-line calculation result corresponding with each period.Calibration process module 24 is by according to based on offline
The real time data result progress calibration process as a result, to real-time task in current slot is calculated, obtains real-time task current
Display data result in period.
In this way, the data calibration device 100 of the real-time task of the embodiment of the present invention, by off-line calculation module 22 to real-time
History real time data of the task before current slot carries out the off-line calculation processing of time segment, obtains corresponding to each period
Off-line calculation result.Calibration process module 24 calls off-line calculation result to carry out the real time data result in current slot
Calibration process obtains the display data result that real-time task is used for current slot.Solve obtained by traditional Real-Time Task System
Results of calculation preferably improves user's body there are the problem of deviation, reaching to make real-time exhibition result more accurately effect
Test.
In another embodiment, optionally, it is true can also to include task status for the data calibration device 100 of real-time task
Cover half block 23 and processing scheme determining module 231.Task status determining module 23 is used for according to the real-time number in current slot
According to as a result, determining the execution state of real-time task.Processing scheme determining module 231 is used to determine calibration process according to execution state
Predetermined scheme.In this way, by prejudging the execution state of real-time task and determining predetermined scheme come into the hand-manipulating of needle according to execution state
To the calibration process of property, the real time data result in current slot that can be effectively directed under different conditions is carried out at calibration
Reason, can save the overhead of the data calibration device 100 of real-time task and improve calibration efficiency.
In one embodiment, there is provided a kind of computer equipment, the computer equipment can be common computer or Special electric
Brain.The computer equipment includes memory and processor.The computer program that can be run on a processor is stored with memory.
When processor performs the computer program on memory, following steps can be performed：Multiple history of real-time task are counted in real time
According to the off-line calculation for carrying out time segment, off-line calculation result corresponding with each period is obtained；It is according to off-line calculation as a result, right
Real time data result of the real-time task in current slot carries out calibration process, obtains real-time task in current slot
Display data result.
In an optional embodiment, foregoing computer equipment, the computer journey on memory is performed in processor
During sequence, the step of can also carry out the data calibration method of real-time task in the various embodiments described above.
In one embodiment, a kind of computer-readable recording medium is also provided, is deposited on the computer-readable recording medium
Contain computer program.When the computer program is executed by processor, it may be such that processor performs following steps：To real-time task
Multiple history real time datas carry out time segment off-line calculation, obtain off-line calculation result corresponding with each period；Root
According to off-line calculation as a result, carrying out calibration process to real time data result of the real-time task in current slot, appointed in real time
Display data result of the business in current slot.
In an optional embodiment, the computer program on foregoing computer-readable recording medium is held by processor
During row, the step of can also carry out the data calibration method of real-time task in the various embodiments described above.
It will appreciated by the skilled person that realizing all or part of flow in above-described embodiment method, being can
To be completed by the relevant hardware of computer program instructions, the computer program can be stored in computer-readable storage
In medium, as independent production marketing or use.When the computer program is executed by processor, it may be such that processor is held
The all or part of step of the row such as embodiment of above-mentioned each method.Wherein, the storage medium can be magnetic disc, it is CD, read-only
Storage memory (Read-Only Memory, ROM) or random access memory (Random Access Memory, RAM) etc..
Embodiment described above only expresses the several embodiments of the present invention, its description is more specific and detailed, but simultaneously
Therefore the limitation to the scope of the claims of the present invention cannot be interpreted as.It should be pointed out that for those of ordinary skill in the art
For, without departing from the inventive concept of the premise, various modifications and improvements can be made, these belong to the guarantor of the present invention
Protect scope.Therefore, the protection domain of patent of the present invention should be determined by the appended claims.
Claims (10)
1. a kind of data calibration method of real-time task, it is characterised in that including step：
Multiple history real time datas of real-time task are carried out with the off-line calculation of time segment, obtain it is corresponding with each period from
Line computation result；
According to the off-line calculation as a result, being carried out to real time data result of the real-time task in current slot at calibration
Reason, obtains display data result of the real-time task in current slot.
2. the data calibration method of real-time task according to claim 1, it is characterised in that according to the off-line calculation knot
Fruit, carries out calibration process to real time data result of the real-time task in current slot, obtains the real-time task and exist
Before the step of display data result in current slot, step is further included：
According to the real time data in the current slot as a result, determining the execution state of the real-time task；
The predetermined scheme of the calibration process is determined according to the execution state；Wherein, the predetermined scheme includes combining and handles
Or data prediction simulation process.
3. the data calibration method of real-time task according to claim 2, it is characterised in that the display data result bag
Include calibration data result；
According to the off-line calculation as a result, being carried out to real time data result of the real-time task in current slot at calibration
In the step of managing, obtaining display data result of the real-time task in current slot, including：
When the execution state of the definite real-time task is normal condition, the off-line calculation knot corresponding to a upper period
Fruit is combined processing with the real time data result in the current slot, obtains the calibration data result.
4. the data calibration method of real-time task according to claim 3, it is characterised in that the combination processing includes referring to
Mark direct adduction processing or the processing of index net value adduction.
5. the data calibration method of real-time task according to claim 2, it is characterised in that the display data result bag
Include analogue data result；
According to the off-line calculation as a result, being carried out to real time data result of the real-time task in current slot at calibration
In the step of managing, obtaining display data result of the real-time task in current slot, further include：
Obtain the real time data result after the calibration of each period before the current slot；Wherein, it is described after calibration
Real time data result is as a result, to described in corresponding each period according to each period corresponding off-line calculation
Real time data result is calibrated, the real time data result after being calibrated；
Determine the real-time task execution state be abnormality when, according to the real time data result after each calibration into
Line number obtains the analogue data result it is predicted that simulation process.
6. the data calibration method of real-time task according to any one of claims 1 to 5, it is characterised in that the time
The duration of section is greater than or equal to 15 minutes.
A kind of 7. data calibration device of real-time task, it is characterised in that including：
Off-line calculation module, for multiple history real time datas of real-time task to be carried out with the off-line calculation of time segment, obtains
Off-line calculation result corresponding with each period；
Calibration process module, for according to the off-line calculation as a result, real-time in current slot to the real-time task
Data result carries out calibration process, obtains display data result of the real-time task in current slot.
8. the data calibration device of real-time task according to claim 7, it is characterised in that further include：
Task status determining module, for according to the real time data in the current slot as a result, determine the real-time task
Execution state；
Processing scheme determining module, for determining the predetermined scheme of the calibration process according to the execution state；Wherein, it is described
Predetermined scheme includes combining processing or prognosis modelling processing.
9. a kind of computer equipment, it is characterised in that including memory and processor, being stored with the memory can be described
The computer program run on processor, the processor are realized when performing the computer program as claim 1-6 is any
The step of data calibration method of real-time task described in.
10. a kind of computer-readable recording medium, is stored thereon with computer program, it is characterised in that the computer program
The step of data calibration method such as claim 1-6 any one of them real-time tasks is realized when being executed by processor.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201711447741.4A CN108038085A (en) | 2017-12-27 | 2017-12-27 | The data calibration method and device of real-time task |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201711447741.4A CN108038085A (en) | 2017-12-27 | 2017-12-27 | The data calibration method and device of real-time task |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN108038085A true CN108038085A (en) | 2018-05-15 |
Family
ID=62097555
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201711447741.4A Pending CN108038085A (en) | 2017-12-27 | 2017-12-27 | The data calibration method and device of real-time task |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN108038085A (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109597842A (en) * | 2018-12-14 | 2019-04-09 | 深圳前海微众银行股份有限公司 | Data real-time computing technique, device, equipment and computer readable storage medium |
| CN112445583A (en) * | 2019-08-28 | 2021-03-05 | 北京沃东天骏信息技术有限公司 | Task management method, task management system, electronic device, and storage medium |
Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20120239956A1 (en) * | 2011-03-17 | 2012-09-20 | Keun-Joo Kwon | Method of reducing power consumption of system software using query scheduling of application and apparatus for reducing power consumption using the method |
| CN104424294A (en) * | 2013-09-02 | 2015-03-18 | 阿里巴巴集团控股有限公司 | Information processing method and information processing device |
| CN104484673A (en) * | 2014-12-05 | 2015-04-01 | 南京大学 | Data complementation method for pattern recognition application of real-time data flow |
| CN106528847A (en) * | 2016-11-24 | 2017-03-22 | 北京集奥聚合科技有限公司 | Multi-dimensional processing method and system for massive data |
| CN106844083A (en) * | 2017-02-20 | 2017-06-13 | 重庆邮电大学 | A kind of fault-tolerance approach and system perceived towards stream calculation system exception |
| CN106909598A (en) * | 2016-07-01 | 2017-06-30 | 阿里巴巴集团控股有限公司 | It is a kind of to ensure processing method, the apparatus and system for calculating data consistency |
| CN107122431A (en) * | 2017-04-14 | 2017-09-01 | 浙江数链科技有限公司 | A kind of real-time calculating platform and the method for computing data based on real-time calculating platform |
| CN107305501A (en) * | 2016-04-25 | 2017-10-31 | 百度在线网络技术（北京）有限公司 | A kind of processing method and system of multithread stream data |
-
2017
- 2017-12-27 CN CN201711447741.4A patent/CN108038085A/en active Pending
Patent Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20120239956A1 (en) * | 2011-03-17 | 2012-09-20 | Keun-Joo Kwon | Method of reducing power consumption of system software using query scheduling of application and apparatus for reducing power consumption using the method |
| CN104424294A (en) * | 2013-09-02 | 2015-03-18 | 阿里巴巴集团控股有限公司 | Information processing method and information processing device |
| CN104484673A (en) * | 2014-12-05 | 2015-04-01 | 南京大学 | Data complementation method for pattern recognition application of real-time data flow |
| CN107305501A (en) * | 2016-04-25 | 2017-10-31 | 百度在线网络技术（北京）有限公司 | A kind of processing method and system of multithread stream data |
| CN106909598A (en) * | 2016-07-01 | 2017-06-30 | 阿里巴巴集团控股有限公司 | It is a kind of to ensure processing method, the apparatus and system for calculating data consistency |
| CN106528847A (en) * | 2016-11-24 | 2017-03-22 | 北京集奥聚合科技有限公司 | Multi-dimensional processing method and system for massive data |
| CN106844083A (en) * | 2017-02-20 | 2017-06-13 | 重庆邮电大学 | A kind of fault-tolerance approach and system perceived towards stream calculation system exception |
| CN107122431A (en) * | 2017-04-14 | 2017-09-01 | 浙江数链科技有限公司 | A kind of real-time calculating platform and the method for computing data based on real-time calculating platform |
Non-Patent Citations (1)
| Title |
|---|
| HUSTFXJ: "Windowing in Apache Storm/Jstorm", 《HTTP://CHINAUNIX.NET/UID-29038263-ID-5765585.HTML》 * |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109597842A (en) * | 2018-12-14 | 2019-04-09 | 深圳前海微众银行股份有限公司 | Data real-time computing technique, device, equipment and computer readable storage medium |
| CN112445583A (en) * | 2019-08-28 | 2021-03-05 | 北京沃东天骏信息技术有限公司 | Task management method, task management system, electronic device, and storage medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Bormetti et al. | Modelling systemic price cojumps with Hawkes factor models | |
| CN107220845A (en) | User purchases probabilistic forecasting/user quality and determines method, device and electronic equipment again | |
| CN108052979A (en) | The method, apparatus and equipment merged to model predication value | |
| CA2364920A1 (en) | Method and device for evaluation of financial derivatives using sparse grids | |
| KR20200003109A (en) | Method and apparatus for setting sample weight, electronic device | |
| CN107992978A (en) | It is a kind of to net the method for prewarning risk and relevant apparatus for borrowing platform | |
| CN109583729B (en) | Data processing method and device for platform online model | |
| CN108038085A (en) | The data calibration method and device of real-time task | |
| CN110517142B (en) | Method and device for outputting policy evaluation information | |
| Fox et al. | Nonlinear regression, nonlinear least squares, and nonlinear mixed models in R | |
| CN110363661A (en) | Bank liquidity prediction technique and device | |
| CN110232582A (en) | A kind of pricing method of used car, apparatus and system | |
| CN107909465A (en) | A kind of method of calibration and device of cash in banks account interest | |
| Ambler et al. | Endogenous business cycle propagation and the persistence problem: The role of labor-market frictions | |
| US11281557B2 (en) | Estimating treatment effect of user interface changes using a state-space model | |
| JP2002109208A (en) | Credit risk managing method, analysis model deciding method, analyzing server and analysis model deciding device | |
| CN106919706A (en) | The method and device that data update | |
| WO2019221844A1 (en) | Item-specific value optimization tool | |
| CN106776757A (en) | User completes the indicating means and device of Net silver operation | |
| CN113902260A (en) | Information prediction method, information prediction device, electronic equipment and medium | |
| Bushe et al. | Prices, activity, and machinery exports: An analysis based on new price data | |
| CN110135972A (en) | A kind of method, apparatus, system and recording medium for improving user and moving branch rate | |
| CN112001774A (en) | Neural network-based tobacco input quantity research and judgment method and system | |
| CN116630051A (en) | Real-time calculation system, method and equipment for yield curve and bond estimation | |
| CN111222993A (en) | Fund recommendation method and device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| TA01 | Transfer of patent application right |
Effective date of registration: 20220119 Address after: Room 1423, No. 1256 and 1258, Wanrong Road, Jing'an District, Shanghai 200040 Applicant after: Tianyi Digital Life Technology Co.,Ltd. Address before: 1 / F and 2 / F, East Garden, Huatian International Plaza, 211 Longkou Middle Road, Tianhe District, Guangzhou, Guangdong 510630 Applicant before: Century Dragon Information Network Co.,Ltd. |
|
| TA01 | Transfer of patent application right | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20180515 |
|
| RJ01 | Rejection of invention patent application after publication |