CN110399366A - Data filtering method, device, server and computer readable storage medium - Google Patents
Data filtering method, device, server and computer readable storage medium Download PDFInfo
- Publication number
- CN110399366A CN110399366A CN201910694094.XA CN201910694094A CN110399366A CN 110399366 A CN110399366 A CN 110399366A CN 201910694094 A CN201910694094 A CN 201910694094A CN 110399366 A CN110399366 A CN 110399366A
- Authority
- CN
- China
- Prior art keywords
- data
- threshold value
- frequency threshold
- abnormal
- target
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/21—Design, administration or maintenance of databases
- G06F16/215—Improving data quality; Data cleansing, e.g. de-duplication, removing invalid entries or correcting typographical errors
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/23—Updating
- G06F16/2358—Change logging, detection, and notification
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/90—Details of database functions independent of the retrieved data types
- G06F16/95—Retrieval from the web
- G06F16/953—Querying, e.g. by the use of web search engines
- G06F16/9535—Search customisation based on user profiles and personalisation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q30/00—Commerce
- G06Q30/02—Marketing; Price estimation or determination; Fundraising
- G06Q30/0241—Advertisements
- G06Q30/0277—Online advertisement
Abstract
The present invention provides a kind of data filtering method, device, server and computer readable storage medium, is related to Internet technical field.This method comprises: determining second frequency threshold value according to preset first frequency threshold value, the ratio of the data volume of target abnormal data and initial abnormal data is preset ratio threshold value；From first frequency threshold value and second frequency threshold value, target frequency threshold value is determined；According to target frequency threshold value, ad data to be monitored is filtered.By choosing a frequency threshold from first frequency threshold value and second frequency threshold value as target frequency threshold value, the abnormal data in ad data is filtered out by target frequency threshold value again, different frequency thresholds can be chosen in varied situations as target frequency threshold value, so as to avoid being filtered normal data, abnormal data can also accurately be filtered, the accuracy for improving abnormal data in filtering advertising data improves the accuracy of monitoring of the advertisement effect.
Description
Technical field
The present invention relates to Internet technical field, in particular to a kind of data filtering method, device, server and
Computer readable storage medium.
Background technique
It, can be by internet to user's advertisement with the continuous development of Internet technology.Accurately to monitor advertisement
Effect is launched, ad data can be filtered, to be analyzed and processed according to the ad data after filtering.
In the related technology, ad data can be filtered according to predeterminated frequency threshold value, to determine in the ad data
Abnormal data, and the abnormal data is filtered out.The abnormal data can be motion frequency such as advertisement point in ad data
The amount of hitting is greater than or equal to the data of the frequency threshold.
But predeterminated frequency threshold value is easy to be cracked by reverse-engineering, it, can be very light once frequency threshold is cracked
Easy carry out flow cheating, so that the abnormal data in ad data is difficult to be detected, so that monitoring of the advertisement is imitated
Fruit is not accurate enough.
Summary of the invention
It is an object of the present invention in view of the deficiency of the prior art, provide a kind of data filtering method, device,
Server and computer readable storage medium are difficult to be detected with the abnormal data solved in ad data, so that extensively
Accuse the not accurate enough problem of monitoring effect.
To achieve the above object, technical solution used in the embodiment of the present invention is as follows:
In a first aspect, the embodiment of the invention provides a kind of data filtering methods, which comprises
Second frequency threshold value is determined according to preset first frequency threshold value, and the corresponding target of the second frequency threshold value is abnormal
The data volume of data is more than the data volume of the corresponding initial abnormal data of the first frequency threshold value, and, the target exception number
Ratio according to the data volume with the initial abnormal data is preset ratio threshold value；
From the first frequency threshold value and the second frequency threshold value, target frequency threshold value is determined；
According to the target frequency threshold value, ad data to be monitored is filtered, to filter out in the ad data
Abnormal data.
Optionally, described from the first frequency threshold value and the second frequency threshold value, it determines target frequency threshold value, wraps
It includes:
The ad data is filtered by the first frequency threshold value and the second frequency threshold value respectively, is obtained
First abnormal ad data and the second abnormal ad data；
If the ratio of the data volume between the described second abnormal ad data and the first abnormal ad data is greater than institute
Preset ratio threshold value is stated, then the second frequency threshold value is determined as the target frequency threshold value；
If the ratio of the data volume between the second abnormal ad data and the first abnormal ad data be less than or
Equal to the preset ratio threshold value, then the first frequency threshold value is determined as the target frequency threshold value.
It is optionally, described that second frequency threshold value is determined according to preset first frequency threshold value, comprising:
Sample ad data is filtered according to the first frequency threshold value, determines the institute in the sample ad data
State initial abnormal data；
The first frequency threshold value is handled, the frequency threshold that obtains that treated；
Treated that frequency threshold is filtered the sample ad data according to described, determines the sample advertisement number
The target abnormal data in；
If the data volume of the target abnormal data is more than the data volume of the initial abnormal data, and, the target is different
The ratio of the data volume of regular data and the initial abnormal data be the preset ratio threshold value, it is determined that it is described treated frequency
Rate threshold value is the second frequency threshold value.
Optionally, described that the first frequency threshold value is handled, the frequency threshold that obtains that treated, comprising:
The first frequency threshold value is carried out it is double or halve processing, obtain described in treated frequency threshold.
Optionally, the method also includes:
If the data volume of the target abnormal data is less than the data volume of the initial abnormal data, alternatively, the target
The ratio of the data volume of abnormal data and the initial abnormal data be not equal to the preset ratio threshold value, then to the processing after
Frequency threshold be updated processing, updated frequency threshold is obtained, until the updated frequency threshold is obtained
The ratio of the data volume of abnormal data and the data volume of the initial abnormal data is equal to the preset ratio threshold value.
Second aspect, the embodiment of the invention also provides a kind of data filtering device, described device includes:
Determining module, for determining second frequency threshold value, the second frequency threshold value according to preset first frequency threshold value
The data volume of corresponding target abnormal data is more than the data volume of the corresponding initial abnormal data of the first frequency threshold value, and,
The ratio of the data volume of the target abnormal data and the initial abnormal data is preset ratio threshold value；
Module is chosen, for determining target frequency threshold value from the first frequency threshold value and the second frequency threshold value；
Filtering module, for being filtered to ad data to be monitored, to filter out according to the target frequency threshold value
State the abnormal data in ad data.
Optionally, the selection module is also used to respectively through the first frequency threshold value and the second frequency threshold value
The ad data is filtered, the first abnormal ad data and the second abnormal ad data are obtained；If described second is abnormal
The ratio of data volume between ad data and the first abnormal ad data is greater than the preset ratio threshold value, then will be described
Second frequency threshold value is determined as the target frequency threshold value；If the described second abnormal ad data and the first abnormal advertisement number
The ratio of data volume between is less than or equal to the preset ratio threshold value, then is determined as the first frequency threshold value described
Target frequency threshold value.
Optionally, the determining module is also used to be filtered sample ad data according to the first frequency threshold value,
Determine the initial abnormal data in the sample ad data；The first frequency threshold value is handled, is handled
Frequency threshold afterwards；Treated that frequency threshold is filtered the sample ad data according to described, determines the sample
The target abnormal data in ad data；If the data volume of the target abnormal data is more than the initial abnormal data
Data volume, and, the ratio of the data volume of the target abnormal data and the initial abnormal data is the preset ratio threshold value,
Treated described in then determining, and frequency threshold is the second frequency threshold value.
Optionally, the determining module is also used to carry out double to the first frequency threshold value or halves processing, obtains institute
Frequency threshold of stating that treated.
Optionally, described device further include:
Update module, if being less than the data volume of the initial abnormal data for the data volume of the target abnormal data,
Alternatively, the ratio of the data volume of the target abnormal data and the initial abnormal data is not equal to the preset ratio threshold value,
Then treated that frequency threshold is updated processing to described, updated frequency threshold is obtained, until the updated frequency
The ratio of the data volume of the obtained abnormal data of rate threshold value and the data volume of the initial abnormal data is equal to the default ratio
Example threshold value.
The third aspect, the embodiment of the invention also provides a kind of servers, comprising: processor, storage medium and bus, institute
It states storage medium and is stored with the executable machine readable instructions of the processor, when server operation, the processor
By the bus communication between the storage medium, the processor executes the machine readable instructions, holds when executing
The step of capable data filtering method as described in first aspect is any.
Fourth aspect, it is described computer-readable to deposit the embodiment of the invention also provides a kind of computer readable storage medium
It is stored with computer program on storage media, is executed as described in first aspect is any when the computer program is run by processor
The step of data filtering method.
The beneficial effects of the present invention are:
The embodiment of the present invention determines second frequency threshold value according to preset first frequency threshold value, and from first frequency threshold value and
In second frequency threshold value, target frequency threshold value is determined, ad data to be monitored is filtered further according to target frequency threshold value,
To filter out the abnormal data in ad data.By choosing a frequency threshold from first frequency threshold value and second frequency threshold value
The abnormal data in ad data is filtered out as target frequency threshold value, then by target frequency threshold value, it can be in varied situations
Different frequency thresholds is chosen as target frequency threshold value, it, can also be to different so as to avoid being filtered normal data
Regular data is accurately filtered, and the accuracy of abnormal data in filtering advertising data is improved, and improves monitoring of the advertisement effect
Accuracy.
Detailed description of the invention
In order to illustrate the technical solution of the embodiments of the present invention more clearly, below will be to needed in the embodiment attached
Figure is briefly described, it should be understood that the following drawings illustrates only certain embodiments of the present invention, therefore is not construed as pair
The restriction of range for those of ordinary skill in the art without creative efforts, can also be according to this
A little attached drawings obtain other relevant attached drawings.
Fig. 1 is the flow diagram for the data filtering method that one embodiment of the invention provides；
Fig. 2 be another embodiment of the present invention provides data filtering method flow diagram；
Fig. 3 is the schematic diagram for the data filtering device that one embodiment of the invention provides；
Fig. 4 be another embodiment of the present invention provides data filtering device schematic diagram；
Fig. 5 is the structural schematic diagram for the server that one embodiment of the invention provides.
Specific embodiment
In order to make the object, technical scheme and advantages of the embodiment of the invention clearer, below in conjunction with the embodiment of the present invention
In attached drawing, technical scheme in the embodiment of the invention is clearly and completely described, it is clear that described embodiment is
A part of the embodiment of the present invention, instead of all the embodiments.
Before the application files an application, related art scheme are as follows: server can be according to predeterminated frequency threshold value, to advertisement
Data are filtered, and to determine the abnormal data in the ad data, and are filtered out to the abnormal data.
Technical problem caused by it are as follows: predeterminated frequency threshold value can be cracked by reverse-engineering, be preset
The corresponding parameter value of frequency threshold, to cause to be difficult in the filtering for no longer triggering abnormal data according to the predeterminated frequency threshold value
Abnormal data is detected, so that monitoring of the advertisement result inaccuracy.
In order to solve the above-mentioned technical problem, the embodiment of the present invention provides a data filtering method.Its core improvement is:
Second frequency threshold value is determined according to preset first frequency threshold value, and chooses mesh from first frequency threshold value and second frequency threshold value
Frequency threshold is marked, ad data is filtered further according to target frequency threshold value, the accuracy of Exception Filter data is improved, mentions
The high accuracy of monitoring of the advertisement.
Technical solution of the present invention is illustrated below by possible implementation.
Fig. 1 is the flow diagram for the data filtering method that one embodiment of the invention provides, as shown in Figure 1, this method can
It is executed by server, which can be the server of data filtering, alternatively, monitoring server etc., this method can include:
Step 101 determines second frequency threshold value according to preset first frequency threshold value.
Wherein, the data volume of the corresponding target abnormal data of second frequency threshold value is corresponding more than first frequency threshold value initial
The data volume of abnormal data, and, the ratio of the data volume of target abnormal data and initial abnormal data is preset ratio threshold value.
In order to accurately be filtered to the abnormal data in ad data, avoiding can not be by first frequency threshold value to exception
Data are filtered, and can be adjusted to preset first frequency threshold value, to obtain that more abnormal datas can be filtered
Second frequency threshold value.
Wherein, which can analyze to obtain according to Historical Monitoring data, the Historical Monitoring data
The record that as ad data is monitored.
In a kind of optional embodiment, the available first frequency threshold value of server, and according to first frequency threshold value
Parameter value, is increased or is reduced to the parameter value, the first frequency threshold value after being adjusted, so that by adjusting after
The abnormal data that one frequency threshold is filtered, more than the initial abnormal number being filtered by first frequency threshold value
According to.
Further, in order to avoid being filtered to normal data, during being determined to second frequency threshold value,
The data volume for the abnormal data that can be filtered according to the first frequency threshold value by adjusting after determines adjusted
Whether one frequency threshold meets preset condition, to avoid being filtered normal data, to improve the standard of abnormal data filtering
True property.
Step 102, from first frequency threshold value and second frequency threshold value, determine target frequency threshold value.
In practical applications, it may include motion frequency in ad data close to first frequency threshold value but not triggered
The abnormal flow of filter, it is also possible to not include above-mentioned abnormal flow, then can be determined for different situations and use first frequency threshold
Value or second frequency threshold value are filtered ad data.
It is corresponding, then a frequency threshold can be chosen from first frequency threshold value and second frequency threshold value as target
Frequency threshold, to filter out the abnormal data in ad data by target frequency threshold value in the next steps.
In a kind of optional embodiment, whether it includes activity frequency that server can be determined first in ad data to be monitored
Rate is close to first frequency threshold value but not the abnormal data of triggering filtering, can be with if in ad data including the abnormal data
Using second frequency threshold value as target frequency threshold value.But, however, it is determined that do not include the abnormal data in ad data, then can incite somebody to action
First frequency threshold value is as target frequency threshold value.
Step 103, according to target frequency threshold value, ad data to be monitored is filtered, to filter out in ad data
Abnormal data.
It, then can be according to the corresponding parameter value of target frequency threshold value, to ad data after determining target frequency threshold value
In the motion frequencies of each data be compared, thus the abnormal data in filtering advertising data.
In a kind of optional embodiment, the corresponding motion frequency of each data in the available ad data of server
Parameter value, and the parameter value of the corresponding motion frequency of each data parameter value corresponding with target frequency threshold value is compared.
If the parameter value of the corresponding motion frequency of some data is greater than or equal to the corresponding parameter value of target frequency threshold value, can determine
The data are abnormal data.
It is corresponding, after each data are compared with target frequency threshold value in ad data, it can determine advertisement
Abnormal data in data realizes the filtering to abnormal data.
In conclusion data filtering method provided in an embodiment of the present invention, determines according to preset first frequency threshold value
Two frequency thresholds, and from first frequency threshold value and second frequency threshold value, target frequency threshold value is determined, further according to target frequency threshold
Value is filtered ad data to be monitored, to filter out the abnormal data in ad data.By from first frequency threshold value and
A frequency threshold is chosen in second frequency threshold value filters out ad data as target frequency threshold value, then by target frequency threshold value
In abnormal data, different frequency thresholds can be chosen in varied situations as target frequency threshold value, so as to avoid
Normal data is filtered, abnormal data can also accurately be filtered, improve abnormal data in filtering advertising data
Accuracy, improve the accuracy of monitoring of the advertisement effect.
Fig. 2 be another embodiment of the present invention provides data filtering method flow diagram, as shown in Fig. 2, this method
Include:
Step 201 is filtered sample ad data according to first frequency threshold value, determines first in sample ad data
Beginning abnormal data.
In order to avoid there is abnormal data in ad data to be monitored, ad data can be carried out by frequency threshold
Filtering, filters out the abnormal data in ad data.And it may include motion frequency in ad data close to first frequency
Threshold value but not the abnormal data of triggering filtering, then need to be adjusted first frequency threshold value, to obtain filter condition more
Add stringent second frequency threshold value.
And during determining second frequency threshold value, sample ad data can be carried out by first frequency threshold value
Filter, obtains initial abnormal data, can be according to the initial abnormal data to the ginseng of second frequency threshold value so as in the next steps
Numerical value is adjusted.
Step 202 handles first frequency threshold value, the frequency threshold that obtains that treated.
In order to obtain the stringenter second frequency threshold value of misgivings condition, can be to first frequency threshold value at
Reason, the frequency threshold that obtains that treated, so that abnormal data that frequency threshold is filtered is more by the way that treated.
In a kind of optional embodiment, the parameter value of first frequency threshold value can be adjusted, such as to the parameter
Value is increasedd or decreased, then can be using the first frequency threshold value after adjusting parameter value as treated frequency threshold.
For example, double or halve processing, the frequency threshold that obtains that treated can be carried out to first frequency threshold value.Namely
It is that, if the corresponding parameter value of first frequency threshold value is 100, can be adjusted to the parameter value 100, obtaining parameter value is 50
Or 200 treated frequency threshold.It is, of course, also possible to be carried out otherwise to the corresponding parameter value of first frequency threshold value
Adjustment, such as the parameter value can be adjusted to 150, the embodiment of the present application does not limit this.
Step 203, according to treated, frequency threshold is filtered sample ad data, determines in sample ad data
Target abnormal data.
In order to determine treated, and whether frequency threshold has tightened up filter condition compared to first frequency threshold value,
Can by the way that treated, frequency threshold is filtered sample ad data, so that target abnormal data is obtained, so as to rear
, can be by the initial abnormal data obtained in target abnormal data and step 201 in continuous step, the frequency threshold that determines that treated
Whether value has tightened up filter condition.
Since this step 203 is by treated process that frequency threshold is filtered sample ad data, with step
201 processes that are filtered to sample ad data by first frequency threshold value are similar, and details are not described herein.
If step 204, the data volume of target abnormal data are more than the data volume of initial abnormal data, and, target exception number
Ratio according to the data volume with initial abnormal data is preset ratio threshold value, it is determined that treated, and frequency threshold is second frequency
Threshold value.
It, can be according to target abnormal data and initial abnormal data after obtaining target abnormal data and initial abnormal data
Corresponding data volume, treated whether frequency threshold has been obtained by filtration more abnormal datas for judgement.If target is abnormal
The corresponding data volume of data is greater than the corresponding data volume of initial abnormal data, then explanation treated frequency threshold has tightened up
Filter condition.
Further, in order to avoid treated, frequency threshold is filtered the normal data in ad data, improves
The accuracy of Exception Filter data can be defined the ratio of the data volume of target abnormal data and initial abnormal data,
When the ratio is equal with preset ratio threshold value, can determining that treated, frequency threshold will not be filtered normal data.
It is corresponding, the corresponding data volume of available target abnormal data data volume corresponding with initial abnormal data it
Between ratio, if the ratio is equal with preset ratio threshold value, can will treated frequency threshold as second frequency threshold value.
But if the data volume of target abnormal data is less than the data volume of initial abnormal data, alternatively, target abnormal data
Be not equal to preset ratio threshold value with the ratio of the data volume of initial abnormal data, then it can frequency threshold carries out more to treated
New processing, obtains updated frequency threshold, until the data volume of the obtained abnormal data of updated frequency threshold and just
The ratio of the data volume of beginning abnormal data is equal to preset ratio threshold value.
In a kind of optional embodiment, if the data volume of target abnormal data is less than the data volume of initial abnormal data,
First frequency threshold value can then be handled according to the reverse mode that first frequency threshold value in step 202 is handled, be obtained
Updated frequency threshold.
For example, being operated if carrying out increased adjustment to the parameter value of first frequency threshold value in step 202, in this step
Parameter value can be operated in 204 using reduced adjustment, obtain updated frequency threshold.
In another optional embodiment, if the ratio of the data volume of target abnormal data and initial abnormal data differs
In preset ratio threshold value, then it can be greater than preset ratio threshold value according to the ratio or the ratio is less than preset ratio threshold value, use
To treated, frequency threshold is adjusted different modes.
If the ratio is greater than preset ratio threshold value, reverse adjustment can be carried out to treated frequency threshold, obtained more
Frequency threshold after new that is to say the opposite adjustment mode pair for using and being handled in step 202 first frequency threshold value
Treated, and frequency threshold is adjusted, so that the Stringency of the filter condition of updated frequency threshold declines.
If the ratio is less than preset ratio threshold value, can continue to carry out first frequency threshold value using with step 202
To treated, frequency threshold is adjusted the consistent mode of processing, obtains updated frequency threshold, so that after updating
Frequency threshold filter condition it is stringenter.
After obtaining updated frequency threshold, step 203 can be repeated to step 204, until target exception number
According to data volume be more than initial abnormal data data volume, and, the ratio of the data volume of target abnormal data and initial abnormal data
Value is preset ratio threshold value, then can be using updated frequency threshold as second frequency threshold value.
It should be noted that the embodiment of the present application be only be illustrated by taking the sequence of step 201 to step 204 as an example, but
Being may include other sequentially to determine that second frequency threshold value, the embodiment of the present application do not limit this in practical applications.Example
Such as, step 202 can be first carried out, then is performed simultaneously step 201 and step 203, finally executes step 204.
Step 205, from first frequency threshold value and second frequency threshold value, determine target frequency threshold value.
It can be by according to the actual conditions of ad data to be detected, determination after determining second frequency threshold value
Two frequency thresholds are filtered, or are filtered by first frequency threshold value, that is to say, by first frequency threshold value or the second frequency
Rate threshold value is chosen for target frequency threshold value, real to different in ad data by target frequency threshold value so as in the next steps
Regular data is filtered.
Optionally, first ad data can be filtered by first frequency threshold value and second frequency threshold value respectively, is obtained
To the first abnormal ad data and the second abnormal ad data, further according to the second abnormal ad data and the first abnormal ad data
Between data volume ratio, determine using which frequency threshold as target frequency threshold value.
If the ratio of the data volume between the second abnormal ad data and the first abnormal ad data is greater than preset ratio threshold
Value, illustrates in ad data that there are motion frequencies close to first frequency threshold value but not the abnormal data of triggering filtering, then can be with
Second frequency threshold value is determined as target frequency threshold value.
But if the ratio of the data volume between the second abnormal ad data and the first abnormal ad data is less than or equal to
Preset ratio threshold value illustrates that motion frequency is not present in ad data close to first frequency threshold value but not triggers the different of filtering
First frequency threshold value can be then determined as target frequency threshold value, avoided in the next steps in ad data by regular data
Normal data is filtered.
Step 206, according to target frequency threshold value, ad data to be monitored is filtered, to filter out in ad data
Abnormal data.
The process of this step 206 and the process of step 103 are similar, and details are not described herein.
In conclusion data filtering method provided in an embodiment of the present invention, determines according to preset first frequency threshold value
Two frequency thresholds, and from first frequency threshold value and second frequency threshold value, target frequency threshold value is determined, further according to target frequency threshold
Value is filtered ad data to be monitored, to filter out the abnormal data in ad data.By from first frequency threshold value and
A frequency threshold is chosen in second frequency threshold value filters out ad data as target frequency threshold value, then by target frequency threshold value
In abnormal data, different frequency thresholds can be chosen in varied situations as target frequency threshold value, so as to avoid
Normal data is filtered, abnormal data can also accurately be filtered, improve abnormal data in filtering advertising data
Accuracy, improve the accuracy of monitoring of the advertisement effect.
Further, by using different frequency thresholds as target frequency threshold value in varied situations, and pass through mesh
Mark frequency threshold ad data is filtered, avoid ad data there is no motion frequency close to first frequency threshold value,
But not the abnormal data for triggering filtering, is filtered the problem of causing normal data to be filtered yet by second frequency threshold value,
Improve the accuracy of ad data filtering.
Further, by continuous iteration adjustment treated frequency threshold, second frequency threshold value is obtained, so that the second frequency
The problem of rate threshold filtering obtains the data volume of abnormal data by considered critical, avoids to normal data filtering, also avoids
Incomplete problem is filtered to abnormal data, improves the accuracy of ad data filtering.
Fig. 3 is the schematic diagram for the data filtering device that one embodiment of the invention provides, as shown in figure 3, the device specifically wraps
It includes:
Determining module 301, for determining second frequency threshold value according to preset first frequency threshold value, the second frequency threshold value
The data volume of corresponding target abnormal data is more than the data volume of the corresponding initial abnormal data of the first frequency threshold value, and, it should
The ratio of the data volume of target abnormal data and the initial abnormal data is preset ratio threshold value；
Module 302 is chosen, for determining target frequency threshold value from the first frequency threshold value and the second frequency threshold value；
Filtering module 303, for being filtered to ad data to be monitored, to filter out according to the target frequency threshold value
Abnormal data in the ad data.
Optionally, the selection module 302 is also used to respectively by the first frequency threshold value and the second frequency threshold value to this
Ad data is filtered, and obtains the first abnormal ad data and the second abnormal ad data；If the second abnormal ad data
The ratio of data volume between the first abnormal ad data is greater than the preset ratio threshold value, then the second frequency threshold value is true
It is set to the target frequency threshold value；If the ratio of the data volume between the second abnormal ad data and the first abnormal ad data
Less than or equal to the preset ratio threshold value, then the first frequency threshold value is determined as the target frequency threshold value.
Optionally, the determining module 301 is also used to be filtered sample ad data according to the first frequency threshold value,
Determine the initial abnormal data in the sample ad data；The first frequency threshold value is handled, the frequency that obtains that treated
Rate threshold value；Frequency threshold after managing according to this is filtered the sample ad data, determines in the sample ad data
The target abnormal data；If the data volume of the target abnormal data is more than the data volume of the initial abnormal data, and, the target is different
The ratio of the data volume of regular data and the initial abnormal data is the preset ratio threshold value, it is determined that should treated frequency threshold
For the second frequency threshold value.
Optionally, the determining module 301 is also used to carry out double to the first frequency threshold value or halves processing, is somebody's turn to do
Treated frequency threshold.
Optionally, referring to fig. 4, the device further include:
Update module 304, if it is less than the data volume of the initial abnormal data for the data volume of the target abnormal data, or
The ratio of the data volume of person, the target abnormal data and the initial abnormal data is not equal to the preset ratio threshold value, then at this
Frequency threshold after reason is updated processing, obtains updated frequency threshold, until obtained by the updated frequency threshold
Abnormal data data volume and the initial abnormal data data volume ratio be equal to the preset ratio threshold value.
In conclusion data filtering device provided in an embodiment of the present invention, determines according to preset first frequency threshold value
Two frequency thresholds, and from first frequency threshold value and second frequency threshold value, target frequency threshold value is determined, further according to target frequency threshold
Value is filtered ad data to be monitored, to filter out the abnormal data in ad data.By from first frequency threshold value and
A frequency threshold is chosen in second frequency threshold value filters out ad data as target frequency threshold value, then by target frequency threshold value
In abnormal data, different frequency thresholds can be chosen in varied situations as target frequency threshold value, so as to avoid
Normal data is filtered, abnormal data can also accurately be filtered, improve abnormal data in filtering advertising data
Accuracy, improve the accuracy of monitoring of the advertisement effect.
The method that above-mentioned apparatus is used to execute previous embodiment offer, it is similar that the realization principle and technical effect are similar, herein not
It repeats again.
The above module can be arranged to implement one or more integrated circuits of above method, such as: one
Or multiple specific integrated circuits (Application Specific Integrated Circuit, abbreviation ASIC), or, one
Or multi-microprocessor (digital singnal processor, abbreviation DSP), or, one or more field programmable gate
Array (Field Programmable Gate Array, abbreviation FPGA) etc..For another example, when some above module passes through processing elements
When the form of part scheduler program code is realized, which can be general processor, such as central processing unit (Central
Processing Unit, abbreviation CPU) or it is other can be with the processor of caller code.For another example, these modules can integrate
Together, it is realized in the form of system on chip (system-on-a-chip, abbreviation SOC).
Fig. 5 is the structural schematic diagram for the server that one embodiment of the invention provides, which, which can be, has data mistake
Filter the calculating equipment of function.
The server includes: processor 501, storage medium 502 and bus 503.
The storage medium 502 is stored with the executable machine readable instructions of the processor 501, when the server is transported
It when row, is communicated between the processor 501 and the storage medium 502 by the bus 503, the processor 501 executes
The machine readable instructions, execute above method embodiment when executing.Specific implementation is similar with technical effect, here not
It repeats again.
Optionally, it the present invention also provides a kind of computer readable storage medium, is deposited on the computer readable storage medium
Computer program is contained, the computer program executes above method embodiment when being run by processor.
In several embodiments provided by the present invention, it should be understood that disclosed device and method can pass through it
Its mode is realized.For example, the apparatus embodiments described above are merely exemplary, for example, the division of the unit, only
Only a kind of logical function partition, there may be another division manner in actual implementation, such as multiple units or components can be tied
Another system is closed or is desirably integrated into, or some features can be ignored or not executed.Another point, it is shown or discussed
Mutual coupling, direct-coupling or communication connection can be through some interfaces, the INDIRECT COUPLING or logical of device or unit
Letter connection can be electrical property, mechanical or other forms.
The unit as illustrated by the separation member may or may not be physically separated, aobvious as unit
The component shown may or may not be physical unit, it can and it is in one place, or may be distributed over multiple
In network unit.It can select some or all of unit therein according to the actual needs to realize the mesh of this embodiment scheme
's.
It, can also be in addition, the functional units in various embodiments of the present invention may be integrated into one processing unit
It is that each unit physically exists alone, can also be integrated in one unit with two or more units.Above-mentioned integrated list
Member both can take the form of hardware realization, can also realize in the form of hardware adds SFU software functional unit.
The above-mentioned integrated unit being realized in the form of SFU software functional unit can store and computer-readable deposit at one
In storage media.Above-mentioned SFU software functional unit is stored in a storage medium, including some instructions are used so that a computer
Equipment (can be personal computer, server or the network equipment etc.) or processor (English: processor) execute this hair
The part steps of bright each embodiment the method.And storage medium above-mentioned includes: USB flash disk, mobile hard disk, read-only memory
(English: Read-Only Memory, abbreviation: ROM), random access memory (English: Random Access Memory, letter
Claim: RAM), the various media that can store program code such as magnetic or disk.
Upper is only the specific embodiment of the application, but the protection scope of the application is not limited thereto, any to be familiar with sheet
Those skilled in the art within the technical scope of the present application, can easily think of the change or the replacement, and should all cover at this
Within the protection scope of application.Therefore, the protection scope of the application should be subject to the protection scope in claims.
Claims (12)
1. a kind of data filtering method, which is characterized in that the described method includes:
Second frequency threshold value is determined according to preset first frequency threshold value, the corresponding target abnormal data of the second frequency threshold value
Data volume be more than the corresponding initial abnormal data of the first frequency threshold value data volume, and, the target abnormal data and
The ratio of the data volume of the initial abnormal data is preset ratio threshold value；
From the first frequency threshold value and the second frequency threshold value, target frequency threshold value is determined；
According to the target frequency threshold value, ad data to be monitored is filtered, it is different in the ad data to filter out
Regular data.
2. the method as described in claim 1, which is characterized in that described from the first frequency threshold value and the second frequency threshold
In value, target frequency threshold value is determined, comprising:
The ad data is filtered by the first frequency threshold value and the second frequency threshold value respectively, obtains first
Abnormal ad data and the second abnormal ad data；
If the ratio of the data volume between the described second abnormal ad data and the first abnormal ad data is greater than described pre-
If proportion threshold value, then the second frequency threshold value is determined as the target frequency threshold value；
If the ratio of the data volume between the described second abnormal ad data and the first abnormal ad data is less than or equal to
The first frequency threshold value is then determined as the target frequency threshold value by the preset ratio threshold value.
3. the method as described in claim 1, which is characterized in that described to determine second frequency according to preset first frequency threshold value
Threshold value, comprising:
Sample ad data is filtered according to the first frequency threshold value, is determined described first in the sample ad data
Beginning abnormal data；
The first frequency threshold value is handled, the frequency threshold that obtains that treated；
Treated that frequency threshold is filtered the sample ad data according to described, determines in the sample ad data
The target abnormal data；
If the data volume of the target abnormal data is more than the data volume of the initial abnormal data, and, the target exception number
Ratio according to the data volume with the initial abnormal data is the preset ratio threshold value, it is determined that treated the frequency threshold
Value is the second frequency threshold value.
4. method as claimed in claim 3, which is characterized in that it is described that the first frequency threshold value is handled, it obtains everywhere
Frequency threshold after reason, comprising:
The first frequency threshold value is carried out it is double or halve processing, obtain described in treated frequency threshold.
5. method as claimed in claim 3, which is characterized in that the method also includes:
If the data volume of the target abnormal data is less than the data volume of the initial abnormal data, alternatively, the target is abnormal
The ratio of data and the data volume of the initial abnormal data be not equal to the preset ratio threshold value, then to it is described treated frequency
Rate threshold value is updated processing, obtains updated frequency threshold, until the updated obtained exception of frequency threshold
The ratio of the data volume of data and the data volume of the initial abnormal data is equal to the preset ratio threshold value.
6. a kind of data filtering device, which is characterized in that described device includes:
Determining module, for determining that second frequency threshold value, the second frequency threshold value are corresponding according to preset first frequency threshold value
Target abnormal data data volume be more than the corresponding initial abnormal data of the first frequency threshold value data volume, and, it is described
The ratio of the data volume of target abnormal data and the initial abnormal data is preset ratio threshold value；
Module is chosen, for determining target frequency threshold value from the first frequency threshold value and the second frequency threshold value；
Filtering module, it is described wide to filter out for being filtered to ad data to be monitored according to the target frequency threshold value
Accuse the abnormal data in data.
7. device as claimed in claim 6, which is characterized in that the selection module is also used to respectively through first frequency
Rate threshold value and the second frequency threshold value are filtered the ad data, obtain the first abnormal ad data and the second exception
Ad data；If the ratio of the data volume between the described second abnormal ad data and the first abnormal ad data is greater than institute
Preset ratio threshold value is stated, then the second frequency threshold value is determined as the target frequency threshold value；If the described second abnormal advertisement
The ratio of data volume between data and the first abnormal ad data is less than or equal to the preset ratio threshold value, then by institute
It states first frequency threshold value and is determined as the target frequency threshold value.
8. device as claimed in claim 6, which is characterized in that the determining module is also used to according to the first frequency threshold
Value is filtered sample ad data, determines the initial abnormal data in the sample ad data；To described first
Frequency threshold is handled, the frequency threshold that obtains that treated；According to treated the frequency threshold to the sample advertisement
Data are filtered, and determine the target abnormal data in the sample ad data；If the number of the target abnormal data
It is more than the data volume of the initial abnormal data according to amount, and, the data of the target abnormal data and the initial abnormal data
The ratio of amount is the preset ratio threshold value, it is determined that treated the frequency threshold is the second frequency threshold value.
9. device as claimed in claim 8, which is characterized in that the determining module is also used to the first frequency threshold value
It carries out double or halves processing, obtain treated the frequency threshold.
10. device as claimed in claim 8, which is characterized in that described device further include:
Update module, if being less than the data volume of the initial abnormal data for the data volume of the target abnormal data, alternatively,
The ratio of the data volume of the target abnormal data and the initial abnormal data is not equal to the preset ratio threshold value, then to institute
Stating treated, frequency threshold is updated processing, obtains updated frequency threshold, until the updated frequency threshold
The ratio of the data volume of obtained abnormal data and the data volume of the initial abnormal data is equal to the preset ratio threshold value.
11. a kind of server characterized by comprising processor, storage medium and bus, the storage medium storage is
The executable machine readable instructions of processor are stated, when server operation, between the processor and the storage medium
By the bus communication, the processor executes the machine readable instructions, and such as claim 1 to 5 times is executed when executing
The step of data filtering method described in one.
12. a kind of computer readable storage medium, which is characterized in that be stored with computer on the computer readable storage medium
Program, the computer program execute the step of data filtering method as claimed in claim 1 to 5 when being run by processor
Suddenly.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910694094.XA CN110399366A (en) | 2019-07-29 | 2019-07-29 | Data filtering method, device, server and computer readable storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910694094.XA CN110399366A (en) | 2019-07-29 | 2019-07-29 | Data filtering method, device, server and computer readable storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN110399366A true CN110399366A (en) | 2019-11-01 |
Family
ID=68326712
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910694094.XA Pending CN110399366A (en) | 2019-07-29 | 2019-07-29 | Data filtering method, device, server and computer readable storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110399366A (en) |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP2008311719A (en) * | 2007-06-12 | 2008-12-25 | Nippon Telegr & Teleph Corp <Ntt> | Threshold setting method, system, and program |
| CN107168854A (en) * | 2017-06-01 | 2017-09-15 | 北京京东尚科信息技术有限公司 | Detection method, device, equipment and readable storage medium storing program for executing are clicked in Internet advertising extremely |
| CN109639633A (en) * | 2018-11-02 | 2019-04-16 | 平安科技（深圳）有限公司 | Abnormal flow data identification method, device, medium and electronic equipment |
| CN109802973A (en) * | 2019-03-15 | 2019-05-24 | 北京百度网讯科技有限公司 | Method and apparatus for detection flows |
| CN110008096A (en) * | 2018-11-29 | 2019-07-12 | 阿里巴巴集团控股有限公司 | Data monitoring method, device, electronic equipment and computer readable storage medium |
-
2019
- 2019-07-29 CN CN201910694094.XA patent/CN110399366A/en active Pending
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP2008311719A (en) * | 2007-06-12 | 2008-12-25 | Nippon Telegr & Teleph Corp <Ntt> | Threshold setting method, system, and program |
| CN107168854A (en) * | 2017-06-01 | 2017-09-15 | 北京京东尚科信息技术有限公司 | Detection method, device, equipment and readable storage medium storing program for executing are clicked in Internet advertising extremely |
| CN109639633A (en) * | 2018-11-02 | 2019-04-16 | 平安科技（深圳）有限公司 | Abnormal flow data identification method, device, medium and electronic equipment |
| CN110008096A (en) * | 2018-11-29 | 2019-07-12 | 阿里巴巴集团控股有限公司 | Data monitoring method, device, electronic equipment and computer readable storage medium |
| CN109802973A (en) * | 2019-03-15 | 2019-05-24 | 北京百度网讯科技有限公司 | Method and apparatus for detection flows |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109858461B (en) | Method, device, equipment and storage medium for counting dense population | |
| CN108038546B (en) | Method and apparatus for compressing neural networks | |
| CN107491252B (en) | Read the statistical method and electronic equipment of duration | |
| CN110210006A (en) | A kind of data screening method and data screening device | |
| CN109460432A (en) | A kind of data processing method and system | |
| CN110347724A (en) | Abnormal behaviour recognition methods, device, electronic equipment and medium | |
| CN108365989A (en) | Event-handling method and device | |
| CN106685749B (en) | The method of inspection and device of network flow | |
| CN109951804A (en) | A kind of stream of people's amount estimation method and device | |
| CN110210558A (en) | Assess the method and device of neural network performance | |
| CN108011615A (en) | A kind of method and apparatus of signal processing | |
| CN110008049A (en) | A kind of data correcting method based on internet platform, terminal device and medium | |
| CN103810241B (en) | Filter method and device that a kind of low frequency is clicked on | |
| CN110399366A (en) | Data filtering method, device, server and computer readable storage medium | |
| CN109559172A (en) | Data processing method, device, electronic equipment and computer readable storage medium | |
| CN110826583A (en) | Fault determination method and device, storage medium and electronic device | |
| CN115952398B (en) | Traditional calculation method, system and storage medium based on data of Internet of things | |
| CN109643307B (en) | Stream processing system and method | |
| CN109951609A (en) | A kind of malicious call number processing method and device | |
| CN113449008B (en) | Modeling method and device | |
| CN112242959B (en) | Micro-service current-limiting control method, device, equipment and computer storage medium | |
| CN114139705A (en) | Structured pruning method based on image frequency response | |
| CN110275496A (en) | A kind of method and apparatus of more time series timestamp alignment | |
| CN111309706A (en) | Model training method and device, readable storage medium and electronic equipment | |
| CN110298690A (en) | Object class purpose period judgment method, device, server and readable storage medium storing program for executing |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20191101 |