CN109697247A - A kind of detection method and device of data accuracy - Google Patents

A kind of detection method and device of data accuracy Download PDF

Info

Publication number
CN109697247A
CN109697247A CN201811648569.3A CN201811648569A CN109697247A CN 109697247 A CN109697247 A CN 109697247A CN 201811648569 A CN201811648569 A CN 201811648569A CN 109697247 A CN109697247 A CN 109697247A
Authority
CN
China
Prior art keywords
time
difference value
result data
data
detection
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN201811648569.3A
Other languages
Chinese (zh)
Other versions
CN109697247B (en
Inventor
韩红根
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Beijing QIYI Century Science and Technology Co Ltd
Original Assignee
Beijing QIYI Century Science and Technology Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Beijing QIYI Century Science and Technology Co Ltd filed Critical Beijing QIYI Century Science and Technology Co Ltd
Priority to CN201811648569.3A priority Critical patent/CN109697247B/en
Publication of CN109697247A publication Critical patent/CN109697247A/en
Application granted granted Critical
Publication of CN109697247B publication Critical patent/CN109697247B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06QINFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
    • G06Q30/00Commerce
    • G06Q30/02Marketing; Price estimation or determination; Fundraising
    • G06Q30/0241Advertisements

Abstract

This application discloses a kind of detection method and device of data accuracy, wherein method includes: to calculate the first difference value, and the first difference value is the difference value in result data sequence accessed by current sensing time between result data and previous result data;Result data sequence is arranged to obtain by result data according to sequence of the acquired time after arriving first;Difference value is the parameter value of the variation degree between the reflection result data;The difference value calculated separately based on detection time sequence, acquisition in the preceding preset quantity detection time of current sensing time;Detection time sequence is arranged to obtain by detection time according to the sequence after arriving first;In the case where meeting preset condition, determine that result data acquired in current sensing time is accurate.By the application, can determine whether result data acquired in current sensing time is accurate.

Description

A kind of detection method and device of data accuracy
Technical field
This application involves data processing field more particularly to a kind of detection method and device of data accuracy.
Background technique
Currently, there is high requirement to the accuracy of result data in many application scenarios.
For example, needing to obtain the data generated based on certain timestamps in video ads scene and the time being calculated Stab the value of corresponding pre-set level.For example, it is desired to obtain 14: 20 that the mass data of 14 points of generations in 20 minutes is calculated Divide corresponding exposure rate value, the reference how many to account deposit amount as user's decision.
Due to the acquired accuracy for sometime stabbing corresponding pre-set level value, there is great shadow to the decision of user It rings, therefore, it is necessary to guarantee the acquired accuracy for sometime stabbing corresponding pre-set level value.
Summary of the invention
This application provides a kind of detection method and device of data accuracy, it is therefore intended that solves detection generation time stamp Whether the value of corresponding pre-set level is accurate.
To achieve the goals above, this application provides following technical schemes:
This application discloses a kind of detection methods of data accuracy, comprising:
The first difference value is calculated, first difference value is knot accessed by current sensing time in result data sequence Difference value between fruit data and previous result data;The result data sequence by result data according to the acquired time from Sequence after arriving first arranges to obtain;The difference value is the parameter value of the variation degree between the reflection result data;
Based on detection time sequence, the preceding preset quantity detection time obtained in the current sensing time is calculated separately Obtained difference value;The detection time sequence is arranged to obtain by detection time according to the sequence after arriving first;
In the case where meeting preset condition, determine that result data acquired in the current sensing time is accurate.
Wherein, the preset condition includes: that acquired preset quantity difference value and first difference value are small In preset threshold.
Wherein, the preset condition includes: that acquired preset quantity difference value and first difference value are small There is no the calculating task being not carried out in the preset threshold and generated calculating task, the calculating task is used for At least one calculated result is calculated in the data generated to default equipment at least once;The current sensing time is obtained The result data taken is the calculated data of last time at least one described calculated result
Wherein, further includes:
There is no the calculating task that is not carried out in generated calculating task, and preset quantity difference value and described In the case that first difference value is not all less than the preset threshold, result data acquired in the current sensing time is determined Inaccuracy;
When the time reaching first object timestamp, last time is obtained from least one described calculated result and is calculated Data be result data, and the step of executing the first difference value of the calculating;The first object timestamp is preset more It is stabbed in a timestamp greater than the minimum time of current sensing time;In preset multiple timestamps between two neighboring timestamp When a length of preset duration.
Wherein, further includes:
In the case where there is the calculating task being not carried out in generated calculating task, determine that current sensing time is obtained The result data inaccuracy taken;
Preset duration needed for one calculating task of quantity and execution for the calculating task being not carried out according to described in, determines Total duration needed for the calculating task being not carried out described in completion;
It determines and postpones the obtained timestamp of total duration in current sensing time as reference time stamp;
The minimum time stamp of reference time stamp will be greater than in preset multiple timestamps, is determined as the second target Timestamp;
When the time reaching second object time stamp, cut-off described the is obtained from least one described calculated result The two object times calculated data of stamp last time are result data, and the step of executing the first difference value of the calculating.
Wherein, the difference value between the current sensing time obtains result data and previous result data, by with Under type is calculated:
Calculate the difference between the result data and previous result data that the current sensing time obtains;
Calculating the ratio between the difference and target duration is the difference value;A length of acquisition is described previous when the target Corresponding duration when the corresponding detection time of a result data and current detection.
Present invention also provides a kind of detection devices of data accuracy, comprising:
Computing unit, for calculating the first difference value, when first difference value is current detection in result data sequence Between difference value between accessed result data and previous result data;The result data sequence by result data according to Sequence of the acquired time after arriving first arranges to obtain;The difference value is the variation degree between the reflection result data Parameter value;
First acquisition unit obtains the preceding preset quantity in the current sensing time for being based on detection time sequence The difference value that a detection time calculates separately;The detection time sequence is arranged by detection time according to the sequence after arriving first Column obtain;
First determination unit, in the case where meeting preset condition, determining acquired in the current sensing time Result data is accurate.
Wherein, the preset condition in first determination unit, comprising: acquired preset quantity difference value and institute It states the first difference value and is respectively less than preset threshold.
Wherein, the preset condition in first determination unit, comprising: acquired preset quantity difference value and institute The first difference value is stated to be respectively less than in the preset threshold and generated calculating task there is no the calculating task being not carried out, At least one calculated result is calculated in the data that the calculating task is used to generate default equipment at least once;It is described Result data acquired in current sensing time is the calculated data of last time at least one described calculated result.
Wherein, further includes:
Second determination unit, for the calculating task being not carried out, and present count to be not present in generated calculating task In the case where a difference value and first difference value are measured not all less than the preset threshold, when determining the current detection Between acquired result data inaccuracy;
Second acquisition unit, for when the time reaching first object timestamp, from least one described calculated result Obtaining the calculated data of last time is result data, and the step of executing the first difference value of the calculating;First mesh Marking timestamp is the minimum time stamp for being greater than current sensing time in preset multiple timestamps;Preset multiple timestamps In when a length of preset duration between two neighboring timestamp.
Wherein, further includes:
Third determination unit, in the case where for there is the calculating task being not carried out in generated calculating task, really Determine the inaccuracy of result data acquired in current sensing time;
4th determination unit, one calculating task institute of quantity and execution of the calculating task for being not carried out according to described in The preset duration needed, total duration needed for determining the calculating task being not carried out described in completing;
5th determination unit, for determining that postponing the obtained timestamp of total duration in current sensing time is reference Timestamp;
6th determination unit, for the minimum time of reference time stamp will to be greater than in preset multiple timestamps Stamp is determined as the second object time stamp;
Third acquiring unit, for being tied from least one described calculating when the time reaching second object time stamp It obtains that end second object time calculated data of stamp last time be result data in fruit, and executes and described calculate the The step of one difference value.
Wherein, the computing unit, is specifically used for:
Calculate the difference between the result data and previous result data that the current sensing time obtains;
Calculating the ratio between the difference and target duration is the difference value;A length of acquisition is described previous when the target Corresponding duration when the corresponding detection time of a result data and current detection.
In the detection method and device of data accuracy described herein, the first difference value is calculated, wherein the first difference Value is the difference value in result data sequence accessed by current sensing time between result data and previous result data;Its In, result data sequence is to arrange to obtain according to sequence of the acquired time after arriving first by result data;When based on detection Between sequence, the difference value that calculates separately of preceding preset quantity detection time in current sensing time is obtained, at this point, obtaining Using current sensing time as starting point, the difference that is obtained respectively according to multiple detection times of the sequence of detection time from back to front Be worth to get to including the difference value that is obtained comprising current sensing time with continuous multiple difference values, due to difference value reflection Be variation degree between result data parameter, therefore, those skilled in the art can based on acquired multiple difference values, Determine whether result data acquired in current sensing time is accurate.
Detailed description of the invention
In order to illustrate the technical solutions in the embodiments of the present application or in the prior art more clearly, to embodiment or will show below There is attached drawing needed in technical description to be briefly described, it should be apparent that, the accompanying drawings in the following description is only this Some embodiments of application for those of ordinary skill in the art without creative efforts, can be with It obtains other drawings based on these drawings.
Fig. 1 is a kind of Application Scenarios-Example figure of the detection device of data accuracy provided by the present application;
Fig. 2 is the calculation method embodiment for the value that a kind of generation time provided by the present application stabs corresponding pre-set level Flow chart;
Fig. 3 is a kind of flow chart of the detection calculating task provided by the present application with the presence or absence of the embodiment of the method for accumulation;
Fig. 4 is whether the newest value that a kind of detection target generation time provided by the present application stabs corresponding pre-set level is quasi- The flow chart of true embodiment of the method;
Fig. 5 be it is provided by the present application another detection target generation time stab corresponding pre-set level newest value whether The flow chart of accurate embodiment of the method;
Fig. 6 is a kind of structural schematic diagram of the detection device embodiment of data accuracy provided by the embodiments of the present application.
Specific embodiment
Inventor has found under study for action, for calculating the data sometime stabbing the value of corresponding pre-set level and being based on Quantity be very big, so need to carry out based on these data repeatedly calculate can just obtain the pre-set level finally take Value.But during obtaining the final value of pre-set level, the corresponding some medians of pre-set level are also obtained, such as The corresponding pre-set level value of the timestamp that fruit obtains is a median of the corresponding pre-set level of the timestamp, so that being based on The median of acquired pre-set level leads to the decision of mistake.
Fig. 1 is the Application Scenarios-Example figure of the detection device of the data accuracy of the application, includes that ad log takes in Fig. 1 The detection device of business device and data accuracy.Wherein, ad log server generates advertisement log data stream;Data accuracy inspection Survey device, for detect any generation time in advertisement log data stream stab corresponding pre-set level newest value it is whether quasi- Really.
Data accuracy detection device in Fig. 1 can integrate in ad log server, can also be independently arranged.
Below in conjunction with the attached drawing in the embodiment of the present application, technical solutions in the embodiments of the present application carries out clear, quasi- It really describes, it is clear that described embodiments are only a part of embodiments of the present application, instead of all the embodiments.It is based on Embodiment in the application, it is obtained by those of ordinary skill in the art without making creative efforts every other Embodiment shall fall in the protection scope of this application.
Fig. 2 is the calculation method for the value that a kind of generation time provided by the present application stabs corresponding pre-set level, including with Lower step:
S201, the data in message queue are parsed, a plurality of data after being parsed.
In the present embodiment, data producer is by generated real-time data transmission to message queue, so that follow-up data Processing unit is extracted data from message queue and is handled extracted data.Wherein, data producer is to generate number According to equipment, for example, log server.
By taking log server generates ad log as an example, generated ad log is exported to message queue, subsequent number Ad log is extracted from message queue according to processing unit and the processing such as analytical Calculation is carried out to extracted ad log.
In the present embodiment, using for message queue from generation data to the process of processing data to be in real time, i.e., Generated data can be transmitted to message queue after generating data by data producer, then, as long as depositing in message queue The data in message queue can be handled in data processing device, all generate data without data producer After the completion, data processing equipment could carry out subsequent processings to generated all data, so ensure that data from generate to The real-time of the process of processing.
In this step, the data in message queue are parsed, a plurality of data after being parsed, wherein parsing Every data afterwards is all corresponding with the generation time stamp of the data.Wherein, the generation time stamp of any one data refers to this At the time of data generate;For example, 10,000 datas that ad log server was generated at 14: 23 30:, at this point, this 10,000 Data generation time stamp be exactly 14 points 23 seconds 30 minutes.
In the present embodiment, the process data in message queue parsed, can using distributed mode into Row parsing, for example, can be parsed simultaneously to the data in message queue using multiple processors.
S202, according to generation time stab sequencing, by the preset quantity data in a plurality of data parsed into Row saves.
The generation time stamp of the value accuracy of subsequent determination pre-set level to be detected for convenience, in the present embodiment, According to the sequencing that generation time is stabbed, the data of preset quantity item are saved in a plurality of data analytically gone out.Specifically , log delay table can be stored in.
In the present embodiment, due to process that the data in message queue are parsed can using distributed mode into Row processing.For example, requiring the movement for executing this step after parsing data for any one processor.
It should be noted that a plurality of parsing data are obtained after S201 parses the data in message queue, for Data after the parsing that S201 is obtained, S202 and the two steps of S203 can execute parallel.
S203, the value that pre-set level is calculated according to the data after parsing, and save and calculate according to the sequencing of calculating The value of obtained pre-set level.
In this step, pre-set level can be determined according to the actual situation by technical staff, and the present embodiment is not to pre- If index limits.The value of pre-set level is calculated to which content in the data after parsing, and how to calculate default finger The process of target value is the prior art, and which is not described herein again.
In the present embodiment, since a generation time stamp corresponds to many datas, to a generation time stamp pair The data answered calculate the process of the value of pre-set level, it may be necessary to which the multiple calculating of progress can just obtain generation time stamp and correspond to Pre-set level final value.In this step, it according to the sequencing of calculating, saves generation time and stabs corresponding default finger It is marked on each calculated result, i.e., from the angle of time, the number of the calculated result saved is successively increased.Wherein, often Secondary calculated result includes: the generation time stamp of data, this pre-set level (for example, exposure rate) calculated, this calculates Value (for example, exposure rate value) of the pre-set level arrived etc..
In the present embodiment, the process that corresponding data calculate the value of pre-set level is stabbed for any one generation time In, each parameter corresponds to a set of logic of propositions for being calculated data.It wherein, will be to any one generation time The process that corresponding data calculate the value of pre-set level is stabbed, multiple calculating tasks are generated, in the process for executing calculating task It is possible that failure, so that generated calculating task is actually but not carried out, for convenience, the present embodiment will have been generated But the calculating task being not carried out is known as the calculating task accumulated.
It should be noted that above-mentioned S201~S203 is the process that the data in message queue are carried out with analytical Calculation, under S301~the S303 in face is the detection process to calculating task state, and in the present embodiment, the process of analytical Calculation and calculating are appointed The process of the detection of business state is to execute parallel.
Specifically, Fig. 3 is a kind of method of the detection calculating task disclosed in the present application with the presence or absence of accumulation, including following step It is rapid:
S301, every preset duration, detect whether in the presence of accumulation calculating task, if it is present execute S302, it is no Then, S301 is executed.
In the present embodiment, it is carried out in calculating process to the data after parsing, can have an interface, if there is heap Long-pending calculating task can show the quantity for the calculating task currently accumulated in the interface.For the heap being shown in interface Long-pending calculating task is that how to determine is the prior art, and which is not described herein again.
In this step, it detects whether the calculating task in the presence of accumulation, can be accumulated by crawler technology from display is current Calculating task interface in crawl the calculating task currently accumulated, if crawling the quantity for the calculating task currently accumulated It is not zero, then it represents that there is currently the calculating tasks of accumulation;Otherwise, if the quantity for crawling the calculating task currently accumulated is Zero, then it represents that there is currently no the calculating tasks of accumulation.
A length of delay duration when needed for S302, the determining calculating task for completing accumulation.
In this step, a default calculating duration is corresponding with for each calculating task, therefore, by the calculating of accumulation The total quantity of task and the default product for calculating duration, duration needed for exactly completing the calculating task of accumulation, for the side of description Just, it is known as the calculated duration of institute to postpone duration.
S303, calculating task state and data delay are saved, and returns and executes S301.
In the present embodiment, calculating task state includes stacking states and normal condition, and the calculating if there is accumulation is appointed Business, then calculating task state is stacking states, and if there is no the calculating task of accumulation, then calculating task state is normal shape State.A calculating task state is detected every preset duration, in this step, when the calculating task state that will test is with delay Length is saved.
Above-mentioned S201~S203 and S301~S303 is to carry out generation time in the application and stabbing corresponding default finger Whether target value accurately does basis, i.e., whether the value that any one generation time stabs corresponding pre-set level is accurately examined It surveys, needs to be detected based on the obtained result of S201~S203 and S301~S303.
It should be noted that above-mentioned S201~S203 calculate generation time stab corresponding pre-set level value process, Whether the value that above-mentioned S301~S303 stabs corresponding pre-set level to the detection process and generation time of calculating task is accurate Detection process these three processes execute parallel.
Stabbing the value of corresponding pre-set level for generation time, whether accurate detection process is introduced following.
Due to stabbing the process that corresponding data are parsed and calculated to each generation time that data producer generates, be by It is successively carried out according to the sequencing of generation time stamp, therefore, the newest value that each generation time stabs corresponding pre-set level is quasi- The true time is also to have sequencing, i.e., the newest value that forward generation time stabs corresponding pre-set level is also first to reach To accurate.
For example, data producer generate data generation time stamp be followed successively by 14 points 30 seconds 23 minutes, 14 points 31 seconds 23 minutes With 14 points 32 seconds 23 minutes, it is also 14 points of 23 minutes 30 seconds, 14 that the newest value of corresponding pre-set level, which reaches accurate sequencing, Point 31 seconds 23 minutes and 14 points 32 seconds 23 minutes.
I.e. when the newest value of 14 points of 30 seconds 23 minutes corresponding pre-set levels does not reach accurate, 14 points 23 minutes 31 The newest value of second and 14 points of 32 seconds 23 minutes corresponding pre-set levels will not reach accurately, therefore, if 14 points of detection While whether the newest value of 23 points of 30 seconds corresponding pre-set levels accurate, detection 14 points 31 seconds 23 minutes and 14 points 32 seconds 23 minutes Whether the newest value of corresponding pre-set level is accurate, reduces detection efficiency.
Therefore, in the present embodiment, in order to improve detection efficiency, pass through each generation time in detection sliding time window The whether accurate mode of value of corresponding pre-set level is stabbed, each generation time stamp that successively detection data producer is sequentially generated Whether the newest value of corresponding pre-set level is accurate.
Wherein, sliding time window refers to the time range that point is constituted from start time point to the end time.Wherein, at this In this implementation, start time point is the minimum generation time in the not accurate generation time stamp of value of corresponding pre-set level Stamp, end time point are that log obtained above postpones any one generation time stamp for being more than or equal to start time point in table.
For example, minimum time stamp in current not accurate generation time stamp for 15 points 25 seconds 20 minutes, log postpones in table Generation time stamp be respectively 15 points 26 seconds 20 minutes, 15 points 27 seconds 20 minutes, 15 points 28 seconds 20 minutes and 15 points 29 seconds 20 minutes, then terminate Time point can postpone any one generation time stamp in this four generation time stamps in table for log.
In this step, each generation time for being included to detection sliding time window is needed to stab corresponding pre-set level Whether value is accurate.For example, including 3 generation time stamps in sliding time window, in this step, need to detect this 3 productions Whether the value of the corresponding pre-set level of life timestamp is accurate.
In the present embodiment, any one generation time in sliding time window is stabbed, detects generation time stamp The whether accurate process of the value of corresponding pre-set level be it is identical, for convenience introduce, the present embodiment is with time slip-window For any one generation time stamp (target generation time stamp) in mouthful, introduces and detect the corresponding default finger of generation time stamp The whether accurate process of target value.
Specifically, Fig. 4 is that the embodiment of the present application discloses a kind of detection target generation time and stabs corresponding pre-set level The whether accurate method of newest value.
The detection process is the process that a circulation executes, when reaching with default detection when multiple detections at a length of interval Between in a detection time when, triggering execute one-time detection process, until certain execute detection process in judge to meet When preset condition, then it represents that the value that the target generation time stabs corresponding pre-set level is accurate.
Wherein, each detection process all corresponds to a detection time, requires to obtain pre-set level in each detection process Value (calculated result) in the calculated pre-set level of last time value, that is, obtain pre-set level newest value, For convenience, calculated result acquired in each detection process is known as result data.
Since the present embodiment is the process that a circulation executes, for convenience, when by according to detection after arriving first Between the sequence that arranges be known as detection time sequence, will be arranged according to detection time from rear acquired result data is arrived first To sequence be known as result data sequence.
Specifically, may comprise steps of:
S401, when reaching the target detection time, it is corresponding default that target generation time stamp is obtained from result data table The newest value of index.
In the present embodiment, for convenience, any one generation time stamp in sliding time window is known as mesh Mark generation time stamp.In this step, multiple detections at a length of interval when the initial value of target detection time is with default detection First detection time in time, first detection time can be set by the user.
In the present embodiment, it is being counted every time due to having recorded pre-set level according to the sequencing of calculating in result data table It is corresponding default to obtain target generation time stamp in this step from result data table for obtained value (calculated result) The newest value of index.
Since the calculating process for the value for calculating pre-set level includes calculating at least once, a calculating is calculated every time As a result, also, the detection process of the data accuracy of the calculating process and the present embodiment of the value of pre-set level is mutually independent Process.Therefore, in this step, the newest value of acquired pre-set level is off current sensing time last time and is counted The value of calculating, for convenience, when the newest value of acquired pre-set level is known as result data, i.e. current detection Between acquired result data.
S402, the newest value for calculating pre-set level acquired in the newest value last time for the pre-set level that this is obtained Between difference value, obtain the first difference value.
In the present embodiment, this refers to current sensing time, the last time refer in history detection time with current detection Temporally adjacent detection time.It is there is no the last time, i.e., pre- acquired in last time if the present embodiment is to execute for the first time If the newest value of index is sky.
In this step, difference reflection be the adjacent pre-set level obtained twice newest value between variation degree. Specifically, difference value can be difference or change rate, wherein difference be this obtain pre-set level newest value with it is upper Difference between the newest value of the pre-set level once obtained;Change rate is difference and this is obtained and the last time obtained The ratio at interval.Certainly, in practical applications, difference can also be other content, and the present embodiment is not to the particular content of difference It limits, as long as difference can reflect between the newest value that the adjacent target generation time obtained twice stabs corresponding pre-set level Variation degree.
For convenience, the pre-set level newest value and the last time of this pre-set level obtained obtained it is newest Difference value between value is known as the first difference value.
S403, it is based on detection time sequence, the preceding preset quantity detection time obtained in current sensing time is counted respectively Obtained difference value.
When this step is to execute for the first time, calculated separately in the preceding preset quantity detection time of current sensing time The difference value arrived is sky.
S404, judge whether to meet preset condition, if it is satisfied, then S405 is executed, if conditions are not met, then executing S406.
In this step, preset condition includes: that the first difference value and acquired preset quantity difference value are both less than pre- If threshold value, wherein preset threshold is the same numerical value.
If this step is to execute for the first time, a difference is only existed, which is exactly the first difference value, acquired Difference value is sky.Acquired difference value can be regarded as infinity, at this point, when executing this step for the first time, judging result one It is set to and is unsatisfactory for preset condition.
It should be noted that the preset condition in this step is a kind of implementation, in practice, those skilled in the art Member can preset condition determines according to actual conditions particular content.
S405, determine target generation time stab corresponding pre-set level newest value it is accurate.
In this step, determine target generation time stab corresponding pre-set level newest value it is accurate.
S406, determine that target generation time stabs the newest value inaccuracy of corresponding pre-set level.
In this step, it is inaccurate for determining that target generation time stabs the corresponding newest value of pre-set level.
S407, the target detection time is updated.
In the present embodiment, whether the newest value that detection target generation time stabs corresponding pre-set level is accurately circulation It carries out, intercycle is default detection duration, i.e. detection time is distributed according to default detection duration.For example, default detection Shi Changwei mono- minute, detection time be followed successively by 14 points 30 minutes, 14 points 31 minutes, 14 points 32 minutes ....
In this step, the target detection time is the minimum time for being greater than current sensing time in preset multiple timestamps Stamp, wherein current sensing time is the target detection time reached in S401 in this implementation procedure.For example, this is executed In the process the target detection time in S401 be 14 points 30 minutes, then in this step, the target detection time be 14 points 31 minutes so that Reach 14: 31 timesharing in the time, continues to execute S401.
Sampling process due to calculating pre-set level in practice is likely to occur failure, and calculating task is caused to generate accumulation, When calculating task generates accumulation, acquired preset quantity difference value and first difference value are judged in S404 Respectively less than preset threshold still stabs the process that corresponding data calculate the value of pre-set level for the target generation time, should There is also uncalculated data in the corresponding data of target generation time stamp, i.e., the newest value of current obtained pre-set level It is not the final calculation result that the generation time stabs corresponding all data;But judge to meet preset condition according to S404, I.e. the target generation time stab corresponding pre-set level newest value it is accurate;Therefore, to target generation time stamp pair Accurately the accuracy of this testing result is lower for the newest value for the pre-set level answered.
The newest value of corresponding pre-set level accurate detection result is stabbed for the target generation time in order to improve Accuracy, detection target generation time stabs the whether accurate process of newest value of corresponding pre-set level, as shown in Figure 5.
In the detection process, in each implementation procedure, in addition to including acquired preset quantity difference in preset condition Different value and first difference value are respectively less than except preset threshold, further include that there is no be not carried out in generated calculating task Calculating task.It is respectively less than pre- in acquired preset quantity difference value and the first difference value i.e. in some detection time If threshold value, and there is no the calculating tasks being not carried out in generated calculating task, then and target generation time stamp is corresponding pre- If the newest value of index is accurate in the detection time, otherwise, it determines the target generation time stabs corresponding pre-set level most New value is in detection time inaccuracy, when reaching the target detection time, continues to execute according to above-mentioned thinking, should until meeting When preset condition, it is determined that the newest value that the target generation time stabs corresponding pre-set level is accurate.
Specifically, the process may comprise steps of:
S501, when reaching the target detection time, it is corresponding default that target generation time stamp is obtained from result data table The newest value of index.
S502, the newest value for calculating the pre-set level that this is obtained and the newest of pre-set level acquired in the last time take Difference value between value obtains the first difference value.
S503, it is based on detection time sequence, the preceding preset quantity detection time obtained in current sensing time is counted respectively Obtained difference value.
The implementation detail of S501~S503 S401~S403 corresponding with Fig. 4 is identical, and which is not described herein again.
S504, judge whether to meet preset condition, if it is satisfied, then S505 is executed, if conditions are not met, then executing S506.
In this step, preset condition includes: that acquired preset quantity difference value and the first difference value are respectively less than Preset threshold, and there is no the calculating tasks being not carried out in generated calculating task.
If the case where being unsatisfactory for preset condition includes:
The first situation: there is the calculating task being not carried out in generated calculating task;
Second situation: the calculating task being not carried out, and acquired present count are not present in generated calculating task A difference value and the first difference value are measured not all less than preset threshold.
S505, determine target generation time stab corresponding pre-set level newest value it is accurate.
S506, determine that target generation time stabs the newest value inaccuracy of corresponding pre-set level.
S506, the target detection time is updated.
In this step, if the calculating task being not carried out is not present in generated calculating task, but it is acquired Preset quantity difference value and the first difference value be not all less than preset threshold, then when the detection of detection trigger process next time Between for according to it is default detection duration distribution multiple detection times in, greater than current sensing time minimum time stab.For example, working as Preceding detection time be 14 points 30 minutes, preset detection when it is 1 minute a length of, then the target detection time be 14 points 31 minutes.For the side of description Just, will in generated calculating task there is no the calculating task that is not carried out, and acquired preset quantity difference value and First difference value not all less than preset threshold in this case, the detection time determined be known as first object detection when Between, using the first object detection time as the target detection time.
If there is the calculating task being not carried out in generated calculating task, i.e., what is obtained in this implementation procedure is default The newest value of index may be the newest value of the pre-set level obtained in last time implementation procedure, therefore, even if acquired Preset quantity difference value and the first difference value are respectively less than preset threshold, can not ensure that target generation time stamp is corresponding pre- If the newest value of index is accurate.
Due to that can determine that delay duration, i.e., the calculating task accumulated after the delay duration could calculate completion, because This, determines that the detection time that detects next time of triggering is in preset multiple detection times stamps, from current sensing time to pusher Minimum time stamp after the slow delay duration, for convenience, is known as the second target detection for the detection time determined Time continues to execute detection when reaching the target detection time using the second target detection time as the target detection time Process.
Specifically, determining that the method for determination for triggering the detection time of detection process next time includes:
A1, to calculate current sensing time to postpone the time point after the delay duration be to stab the reference time.
A2, it is detected in multiple detection times that duration is distributed from default, determines the minimum time for being greater than reference time stamp Stamp was the second target detection time.
For example, current sensing time be 14 points 30 minutes, when detection, is 2 minutes a length of, and when delay is 3 minutes a length of, then the second mesh Mark detection time be 14 points 34 minutes.
The utility model has the advantages that in the present embodiment, being parsed to the data that data producer in message queue generates and right The data that any generation time stabs after corresponding parsing are successively calculated, and are obtained the generation time and are stabbed corresponding pre-set level Multiple values;The value that corresponding pre-set level is stabbed based on the generation time is detected the generation time and stabs corresponding pre-set level Newest value it is whether accurate.Specifically, in each detection process, obtaining generation time stamp by the detection process of circulation The newest value of corresponding pre-set level determines the first difference value based on acquired newest value, judges that the generation time is stabbed Whether the newest value of corresponding pre-set level is accurate.
Wherein, judge whether accurately a kind of mode may include: acquired preset quantity to the newest value of pre-set level A difference value and the first difference value are respectively less than preset threshold.
In practice, in the program failure that the process of the value of corresponding pre-set level is stabbed in calculating generation time, So that there is the calculating task being not carried out in generated calculating task, so that the calculating task of accumulation is produced, at this point, due to Program mal causes the newest value of the pre-set level repeatedly obtained during repeated detection to be the same median, this When, judge that acquired preset quantity difference value and the first difference value are respectively less than preset threshold, but pre-set level Newest value is inaccurate.In order to improve pre-set level newest value accurate detection result accuracy, the present embodiment removes The acquired preset quantity difference value of judgement and the first difference value are respectively less than outside preset threshold, while also judging to have generated Calculating task in the presence or absence of the calculating task that is not carried out, only in acquired preset quantity difference value and described the One difference value is respectively less than preset threshold, also, there is no when the calculating task being not carried out in generated calculating task, determining should The newest value that generation time stabs corresponding pre-set level is accurate, at this point, testing result accuracy with higher.
In order to improve detection efficiency, when there is the calculating task being not carried out in generated calculating task, this is calculated not Total duration needed for the calculating task of execution, and the second target detection time after current sensing time postpones the total duration into Capable detection process next time.
Fig. 6 is a kind of detection device of data accuracy disclosed in the present application, comprising:
Computing unit 601, for calculating the first difference value, first difference value is current detection in result data sequence Difference value between result data accessed by time and previous result data;The result data sequence is pressed by result data It arranges to obtain according to sequence of the acquired time after arriving first;The difference value is the variation degree between the reflection result data Parameter value;
First acquisition unit 602 obtains the preceding present count in the current sensing time for being based on detection time sequence Measure the difference value that a detection time calculates separately;The detection time sequence is by detection time according to the sequence after arriving first Arrangement obtains;
First determination unit 603, in the case where meeting preset condition, determining acquired in the current sensing time Result data it is accurate.
Wherein, the preset condition in first determination unit 603, comprising: acquired preset quantity difference value with And first difference value is respectively less than preset threshold.
Wherein, the preset condition in first determination unit 603, comprising: acquired preset quantity difference value with And first difference value is respectively less than the calculating for being not present and being not carried out in the preset threshold and generated calculating task and appoints At least one calculated result is calculated in business, the data that the calculating task is used to generate default equipment at least once; Result data acquired in the current sensing time is the calculated data of last time at least one described calculated result.
Wherein, further includes:
Second determination unit, for the calculating task being not carried out, and present count to be not present in generated calculating task In the case where a difference value and first difference value are measured not all less than the preset threshold, when determining the current detection Between acquired result data inaccuracy;
Second acquisition unit, for when the time reaching first object timestamp, from least one described calculated result Obtaining the calculated data of last time is result data, and the step of executing the first difference value of the calculating;First mesh Marking timestamp is the minimum time stamp for being greater than current sensing time in preset multiple timestamps;Preset multiple timestamps In when a length of preset duration between two neighboring timestamp.
Wherein, further includes:
Third determination unit, in the case where for there is the calculating task being not carried out in generated calculating task, really Determine the inaccuracy of result data acquired in current sensing time;
4th determination unit, one calculating task institute of quantity and execution of the calculating task for being not carried out according to described in The preset duration needed, total duration needed for determining the calculating task being not carried out described in completing;
5th determination unit, for determining that postponing the obtained timestamp of total duration in current sensing time is reference Timestamp;
6th determination unit, for the minimum time of reference time stamp will to be greater than in preset multiple timestamps Stamp is determined as the second object time stamp;
Third acquiring unit, for being tied from least one described calculating when the time reaching second object time stamp It obtains that end second object time calculated data of stamp last time be result data in fruit, and executes and described calculate the The step of one difference value.
Wherein, the computing unit 601, is specifically used for:
Calculate the difference between the result data and previous result data that the current sensing time obtains;
Calculating the ratio between the difference and target duration is the difference value;A length of acquisition is described previous when the target Corresponding duration when the corresponding detection time of a result data and current detection.
If function described in the embodiment of the present application method is realized in the form of SFU software functional unit and as independent production Product when selling or using, can store in a storage medium readable by a compute device.Based on this understanding, the application is real The part for applying a part that contributes to existing technology or the technical solution can be embodied in the form of software products, The software product is stored in a storage medium, including some instructions are used so that a calculating equipment (can be personal meter Calculation machine, server, mobile computing device or network equipment etc.) execute each embodiment the method for the application whole or portion Step by step.And storage medium above-mentioned include: USB flash disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), with Machine accesses various Jie that can store program code such as memory (RAM, Random Access Memory), magnetic or disk Matter.
Each embodiment in this specification is described in a progressive manner, the highlights of each of the examples are with it is other The difference of embodiment, same or similar part may refer to each other between each embodiment.
The foregoing description of the disclosed embodiments makes professional and technical personnel in the field can be realized or use the application. Various modifications to these embodiments will be readily apparent to those skilled in the art, as defined herein General Principle can be realized in other embodiments without departing from the spirit or scope of the application.Therefore, the application It is not intended to be limited to the embodiments shown herein, and is to fit to and the principles and novel features disclosed herein phase one The widest scope of cause.

Claims (12)

1. a kind of detection method of data accuracy characterized by comprising
The first difference value is calculated, first difference value is number of results accessed by current sensing time in result data sequence According to the difference value between previous result data;The result data sequence is by result data according to the acquired time from arriving first Sequence afterwards arranges to obtain;The difference value is the parameter value of the variation degree between the reflection result data;
Based on detection time sequence, the preceding preset quantity detection time obtained in the current sensing time calculates separately to obtain Difference value;The detection time sequence is arranged to obtain by detection time according to the sequence after arriving first;
In the case where meeting preset condition, determine that result data acquired in the current sensing time is accurate.
2. the method according to claim 1, wherein the preset condition includes: acquired preset quantity Difference value and first difference value are respectively less than preset threshold.
3. the method according to claim 1, wherein the preset condition includes: acquired preset quantity Difference value and first difference value are respectively less than to be not present in the preset threshold and generated calculating task and be not carried out Calculating task, data of the calculating task by generating to default equipment are calculated based at least one at least once Calculate result;Result data acquired in the current sensing time is to calculate for the last time at least one described calculated result Data.
4. according to the method described in claim 3, it is characterized by further comprising:
The calculating task being not carried out, and preset quantity difference value and described first are not present in generated calculating task In the case that difference value is not all less than the preset threshold, determine that result data acquired in the current sensing time is inaccurate Really;
When the time reaching first object timestamp, the calculated number of last time is obtained from least one described calculated result According to for result data, and the step of executing the first difference value of the calculating;When the first object timestamp is preset multiple Between stamp in be greater than current sensing time minimum time stab;In preset multiple timestamps between two neighboring timestamp when A length of preset duration.
5. according to the method described in claim 3, it is characterized by further comprising:
In the case where there is the calculating task being not carried out in generated calculating task, determine acquired in current sensing time Result data inaccuracy;
Preset duration needed for one calculating task of quantity and execution for the calculating task being not carried out according to described in, determines and completes Total duration needed for the calculating task being not carried out;
It determines and postpones the obtained timestamp of total duration in current sensing time as reference time stamp;
The minimum time stamp of reference time stamp will be greater than in preset multiple timestamps, was determined as the second object time Stamp;
When the time reaching second object time stamp, is obtained from least one described calculated result and end second mesh Marking the calculated data of timestamp last time is result data, and the step of executing the first difference value of the calculating.
6. the method according to claim 1, wherein the current sensing time obtain result data with it is previous Difference value between a result data, is calculated in the following manner:
Calculate the difference between the result data and previous result data that the current sensing time obtains;
Calculating the ratio between the difference and target duration is the difference value;It is a length of when the target to obtain the previous knot Corresponding duration when the corresponding detection time of fruit data and current detection.
7. a kind of detection device of data accuracy characterized by comprising
Computing unit, for calculating the first difference value, first difference value is current sensing time institute in result data sequence Difference value between the result data and previous result data that get;The result data sequence is by result data according to being obtained Sequence of the time taken after arriving first arranges to obtain;The difference value is the parameter of the variation degree between the reflection result data Value;
First acquisition unit, for being based on detection time sequence, the preceding preset quantity obtained in the current sensing time is examined The difference value that the survey time calculates separately;The detection time sequence is arranged by detection time according to the sequence after arriving first It arrives;
First determination unit, in the case where meeting preset condition, determining result acquired in the current sensing time Data are accurate.
8. device according to claim 7, which is characterized in that the preset condition in first determination unit, comprising: institute The preset quantity difference value of acquisition and first difference value are respectively less than preset threshold.
9. device according to claim 7, which is characterized in that the preset condition in first determination unit, comprising: institute The preset quantity difference value of acquisition and first difference value are respectively less than the preset threshold and generated calculate is appointed There is no the calculating task being not carried out in business, data of the calculating task by generating to default equipment are carried out based at least once Calculation obtains at least one calculated result;Result data acquired in the current sensing time is at least one described calculated result The middle calculated data of last time.
10. device according to claim 9, which is characterized in that further include:
Second determination unit, for the calculating task being not carried out, and preset quantity to be not present in generated calculating task In the case that difference value and first difference value be not all less than the preset threshold, the current sensing time institute is determined The result data inaccuracy of acquisition;
Second acquisition unit, for being obtained from least one described calculated result when the time reaching first object timestamp Calculated data are result data, and the step of executing the first difference value of the calculating for the last time;When the first object Between stamp be preset multiple timestamps in be greater than current sensing time minimum time stab;Phase in preset multiple timestamps When a length of preset duration between adjacent two timestamps.
11. device according to claim 9, which is characterized in that further include:
Third determination unit, in the case where for there is the calculating task being not carried out in generated calculating task, determination is worked as The inaccuracy of result data acquired in preceding detection time;
4th determination unit, the quantity of calculating task for being not carried out according to described in and executes needed for a calculating task Preset duration, total duration needed for determining the calculating task being not carried out described in completing;
5th determination unit, for determining that postponing the obtained timestamp of total duration in current sensing time is the reference time Stamp;
6th determination unit, for the minimum time for being greater than reference time stamp in preset multiple timestamps to be stabbed, It is determined as the second object time stamp;
Third acquiring unit, for the time reach second object time stamp when, from least one described calculated result It is result data that acquisition, which ends the calculated data of the second object time stamp last time, and it is poor to execute the calculating first The step of different value.
12. device according to claim 7, which is characterized in that the computing unit is specifically used for:
Calculate the difference between the result data and previous result data that the current sensing time obtains;
Calculating the ratio between the difference and target duration is the difference value;It is a length of when the target to obtain the previous knot Corresponding duration when the corresponding detection time of fruit data and current detection.
CN201811648569.3A 2018-12-30 2018-12-30 Method and device for detecting data accuracy Active CN109697247B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201811648569.3A CN109697247B (en) 2018-12-30 2018-12-30 Method and device for detecting data accuracy

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201811648569.3A CN109697247B (en) 2018-12-30 2018-12-30 Method and device for detecting data accuracy

Publications (2)

Publication Number Publication Date
CN109697247A true CN109697247A (en) 2019-04-30
CN109697247B CN109697247B (en) 2021-05-18

Family

ID=66233122

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201811648569.3A Active CN109697247B (en) 2018-12-30 2018-12-30 Method and device for detecting data accuracy

Country Status (1)

Country Link
CN (1) CN109697247B (en)

Cited By (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111563078A (en) * 2020-07-15 2020-08-21 浙江大华技术股份有限公司 Data quality detection method and device based on time sequence data and storage device
WO2021098021A1 (en) * 2019-11-20 2021-05-27 珠海格力电器股份有限公司 Data anomaly statistical alarm method and device, and electronic equipment
CN113189664A (en) * 2021-04-26 2021-07-30 拉扎斯网络科技(上海)有限公司 Object placing state detection method and device
CN115017099A (en) * 2022-08-08 2022-09-06 深圳市华曦达科技股份有限公司 Distributed network task cooperation method and system

Citations (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN104965773A (en) * 2015-07-09 2015-10-07 网易(杭州)网络有限公司 Terminal, jamming detection method, device as well as game jamming detection method and device
KR20150120924A (en) * 2015-10-15 2015-10-28 (주) 솔텍시스템 Apparatus, method and computer readable recording medium for compressing time series processing data
CN105187863A (en) * 2015-07-31 2015-12-23 小米科技有限责任公司 Advertisement playing method and device
CN105653407A (en) * 2015-12-08 2016-06-08 网易(杭州)网络有限公司 Terminal, jam measuring method, device, game jam measuring method and apparatus
CN106095787A (en) * 2016-05-30 2016-11-09 重庆大学 A kind of Symbolic Representation method of time series data
CN107423435A (en) * 2017-08-04 2017-12-01 电子科技大学 The multi-level method for detecting abnormality of multidimensional space-time data
CN107871190A (en) * 2016-09-23 2018-04-03 阿里巴巴集团控股有限公司 A kind of operational indicator monitoring method and device
CN107968731A (en) * 2016-10-20 2018-04-27 腾讯科技(深圳)有限公司 The aobvious number method for detecting abnormality of one kind and server
CN108959174A (en) * 2018-07-27 2018-12-07 中国大唐集团新能源科学技术研究院有限公司 A kind of calculation method of wind power system generated energy

Patent Citations (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN104965773A (en) * 2015-07-09 2015-10-07 网易(杭州)网络有限公司 Terminal, jamming detection method, device as well as game jamming detection method and device
CN105187863A (en) * 2015-07-31 2015-12-23 小米科技有限责任公司 Advertisement playing method and device
KR20150120924A (en) * 2015-10-15 2015-10-28 (주) 솔텍시스템 Apparatus, method and computer readable recording medium for compressing time series processing data
CN105653407A (en) * 2015-12-08 2016-06-08 网易(杭州)网络有限公司 Terminal, jam measuring method, device, game jam measuring method and apparatus
CN106095787A (en) * 2016-05-30 2016-11-09 重庆大学 A kind of Symbolic Representation method of time series data
CN107871190A (en) * 2016-09-23 2018-04-03 阿里巴巴集团控股有限公司 A kind of operational indicator monitoring method and device
CN107968731A (en) * 2016-10-20 2018-04-27 腾讯科技(深圳)有限公司 The aobvious number method for detecting abnormality of one kind and server
CN107423435A (en) * 2017-08-04 2017-12-01 电子科技大学 The multi-level method for detecting abnormality of multidimensional space-time data
CN108959174A (en) * 2018-07-27 2018-12-07 中国大唐集团新能源科学技术研究院有限公司 A kind of calculation method of wind power system generated energy

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
李琦: "基于多源数据的交通状态监测与预测方法研究", 《中国博士学位论文全文数据库信息科技辑》 *

Cited By (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2021098021A1 (en) * 2019-11-20 2021-05-27 珠海格力电器股份有限公司 Data anomaly statistical alarm method and device, and electronic equipment
CN111563078A (en) * 2020-07-15 2020-08-21 浙江大华技术股份有限公司 Data quality detection method and device based on time sequence data and storage device
CN111563078B (en) * 2020-07-15 2020-11-10 浙江大华技术股份有限公司 Data quality detection method and device based on time sequence data and storage device
CN113189664A (en) * 2021-04-26 2021-07-30 拉扎斯网络科技(上海)有限公司 Object placing state detection method and device
CN113189664B (en) * 2021-04-26 2022-04-22 拉扎斯网络科技(上海)有限公司 Object placing state detection method and device
CN115017099A (en) * 2022-08-08 2022-09-06 深圳市华曦达科技股份有限公司 Distributed network task cooperation method and system

Also Published As

Publication number Publication date
CN109697247B (en) 2021-05-18

Similar Documents

Publication Publication Date Title
CN109697247A (en) A kind of detection method and device of data accuracy
WO2021098021A1 (en) Data anomaly statistical alarm method and device, and electronic equipment
US8170894B2 (en) Method of identifying innovations possessing business disrupting properties
US9645909B2 (en) Operation management apparatus and operation management method
CN110601900A (en) Network fault early warning method and device
CN106598822B (en) A kind of abnormal deviation data examination method and device for Capacity Assessment
US20190028556A1 (en) System and method for measuring user engagement
CN104618949B (en) A kind of complaint prediction technique and device based on arma modeling
US20190324794A1 (en) Real-Time Data Processing Method and Apparatus
Lee et al. Smart phone power model generation using use pattern analysis
CN114623939A (en) Method, device, equipment and medium for determining pulse frequency
CN105242873B (en) The acquisition of the performance data of cloud computing system and storage method and device
CN103428733B (en) A kind of Forecasting Methodology and device
JP6995146B2 (en) Performance analysis of adaptive applications
CN111611521B (en) Flow cheating monitoring method and device, electronic equipment and storage medium
CN104462116B (en) Data selection method and device
JP6192432B2 (en) Risk weighing system
Akca et al. Run-time measurement of cosmic functional size for java business applications: Initial results
Jain et al. Software reliability growth model (SRGM) with imperfect debugging, fault reduction factor and multiple change-point
CN113343458B (en) Engine sensor selection method and device, electronic equipment and storage medium
CN109614570A (en) Predict the method and device of section water quality parameter data
CN103593426B (en) A kind of commercial articles searching and offer method and device
CN103885716B (en) Touch-screen setting-out method of testing and setting-out velocity measuring device
CN104657614A (en) Product site failure rate calculation method
Inoue et al. On estimation of number of detectable software faults under budget constraint

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant