CN108491875A - A kind of data exception detection method, device, equipment and medium - Google Patents
A kind of data exception detection method, device, equipment and medium Download PDFInfo
- Publication number
- CN108491875A CN108491875A CN201810225915.0A CN201810225915A CN108491875A CN 108491875 A CN108491875 A CN 108491875A CN 201810225915 A CN201810225915 A CN 201810225915A CN 108491875 A CN108491875 A CN 108491875A
- Authority
- CN
- China
- Prior art keywords
- data
- daily paper
- detected
- training
- tested
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
- G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
- G06F18/211—Selection of the most significant subset of features
Abstract
The embodiment of the invention discloses a kind of data exception detection method, device, equipment and media.The method includes：The training data corresponding to daily paper data to be detected and the daily paper data to be detected is obtained, wherein the training data was determined according to generated times of the daily paper data to be detected；Normal data section is generated according to the training data and preset distributed model；The daily paper data to be detected are matched with the normal data section of generation, determine whether the daily paper data to be detected are abnormal data according to matching result.The present invention proposes a kind of report data method for detecting abnormality with adaptivity so that the detection of daily paper data is quicker, can automatically find data exception in time, and improve abnormal data traces efficiency, reduces the coverage of abnormal data.
Description
Technical field
The present embodiments relate to information technology field more particularly to a kind of data exception detection method, device, equipment and
Medium.
Background technology
With the high speed development of Internet technology and e-commerce, online transaction is growing day by day.Currently, producing for convenience
Product, operation and related management personnel are for statistical analysis to daily transaction data, use the mode of timed task to transaction
And other information datas carry out automatically generating for the T+1 of daily paper data.Wherein, " T " refers to the day of trade, and " T+1 " refers to time of the day of trade
Day.Automatically generating for T+1 refers to automatically generating the daily paper data of the day of trade in the next day of the day of trade.
For aforesaid way, whether related personnel could have found daily paper data after daily paper data are compared, analyzed
In the presence of abnormal (sharp increase or rapid drawdown of such as trading volume), related technical personnel is then notified to trace abnormal.In current day
It during count off is according to anomaly, automatically generates to staff and notes abnormalities from daily paper, when generally undergoing longer one section
Between (short then half a day, long then a couple of days).It can be seen that the detection time of abnormal transaction data is long, need to check after detecting and
Positioning relevant daily record data becomes more difficult, to reduce abnormal efficiency of tracing, and may expand abnormal influence model
It encloses.
Invention content
In view of the above-mentioned problems, an embodiment of the present invention provides a kind of data exception detection method, device, equipment and medium,
To realize quickly detection abnormal data, that improves abnormal data traces efficiency.
In a first aspect, an embodiment of the present invention provides a kind of data exception detection methods, including：
The training data corresponding to daily paper data to be detected and the daily paper data to be detected is obtained, wherein the training
Data were determined according to generated times of the daily paper data to be detected；
Normal data section is generated according to the training data and preset distributed model；
The daily paper data to be detected are matched with the normal data section of generation, described in matching result determination
Whether daily paper data to be detected are abnormal data.
Second aspect, the embodiment of the present invention additionally provide a kind of data exception detection device, including：
Data acquisition module, for obtaining the training corresponding to daily paper data to be detected and the daily paper data to be detected
Data, wherein the training data was determined according to generated times of the daily paper data to be detected；
Section generation module, for generating normal data section according to the training data and preset distributed model；
Data detection module, for the daily paper data to be detected to be matched with the normal data section of generation, according to
Determine whether the daily paper data to be detected are abnormal data according to matching result.
The third aspect, the embodiment of the present invention additionally provide a kind of data exception detection device, and the equipment includes：
One or more processors；
Storage device, for storing one or more programs；
When one or more of programs are executed by one or more of processors so that one or more of processing
Device realizes the data exception detection method provided such as any embodiment of the present invention.
Fourth aspect, the embodiment of the present invention additionally provide a kind of computer readable storage medium, are stored thereon with computer
Program realizes the data exception detection method provided such as any embodiment of the present invention when the program is executed by processor.
The embodiment of the present invention by obtaining the training data corresponding to daily paper data to be detected and daily paper data to be detected,
Wherein training data was determined according to generated times of daily paper data to be detected so that training data can with data to be tested
Adaptive matching；Normal data section is generated according to training data and preset distributed model；By daily paper data to be detected and life
At normal data section matched, determine whether daily paper data to be detected are abnormal data according to matching result, it is proposed that
A kind of report data method for detecting abnormality with adaptivity so that the detection of daily paper data is quicker, can be automatically
Data exception is found in time, and improve abnormal data traces efficiency, reduces the coverage of abnormal data.
Description of the drawings
Fig. 1 is the flow chart of data exception detection method in the embodiment of the present invention one；
Fig. 2 a are the flow charts of data exception detection method in the embodiment of the present invention two；
Fig. 2 b are the schematic diagrames in the corresponding normal data section of Gaussian distribution model in the embodiment of the present invention two；
Fig. 3 is the flow chart of data exception detection method in the embodiment of the present invention three；
Fig. 4 is the structural schematic diagram of data exception detection device in the embodiment of the present invention four；
Fig. 5 is the structural schematic diagram of data exception detection device in the embodiment of the present invention five.
Specific implementation mode
The present invention is described in further detail with reference to the accompanying drawings and examples.It is understood that this place is retouched
The specific embodiment stated is used only for explaining the present invention rather than limitation of the invention.It also should be noted that in order to just
Only the parts related to the present invention are shown in description, attached drawing rather than entire infrastructure.
Embodiment one
Fig. 1 is the flow chart of data exception detection method in the embodiment of the present invention one, and the present embodiment is applicable to detection day
Normal transaction data is with the presence or absence of abnormal situation.This method can be executed by data exception detection device, which may be used
The mode of software and/or hardware is realized, for example, the device is configured in data exception detection device.As shown in Figure 1, the party
Method specifically includes：
Training data corresponding to S110, acquisition daily paper data to be detected and the daily paper data to be detected.
In general, daily paper data have usually been carried out T+1 by the statistics to current transaction related data and management for convenience
Automatically generate, i.e., the next day of the day of trade generate the day of trade on the day of daily paper data.In the present embodiment, by having been formed
Daily paper data extraction, obtain the training data corresponding to daily paper data to be detected and daily paper data to be detected.
It should be noted that with the development of transaction business, daily paper data (such as turnover) can be dynamic by certain trend
State changes, thus recent data could preferably reflect whether newest data are normal.Therefore, using the close of data to be tested
Issue is according to as the training data corresponding to data to be tested so that the detection of data has certain adaptivity.Namely
It says, the generated time of daily paper data generated time and data to be tested corresponding to training data have correspondence.Specifically,
Training data can be the daily paper data in data to be tested preset time period.It is true according to the generated time of data to be tested
Determining training data can be in data to be tested difference, and the modification training data of adaptability makes the detection of data to be tested have
Certain adaptivity.
It in the present embodiment, can the transaction to the day of trade after the next day of the day of trade and the daily paper data of the day of trade generate
Data are detected, to judge the transaction data of the day of trade with the presence or absence of abnormal.
For example, after the next day of the day of trade generates the daily paper data of the day of trade, the daily paper data of the day of trade of generation are made
For data to be tested, and according to the date of formation of day of trade daily paper data and preset period obtain daily paper data to be detected and
Its corresponding training data.Wherein preset time period can be the period corresponding to training data, as training data can be
The daily paper data in a period before the date of formation of daily paper data corresponding to data to be tested, such as to be detected
Daily paper data before the date of formation of daily paper data corresponding to data in 20 consecutive days.Preset time period may be to remove
Except date corresponding to training data, also includes the period on the date corresponding to data to be tested, such as directly acquire and waiting for
Daily paper data before the detection data date of formation in 21 consecutive days obtain data to be tested and training data simultaneously.Again
Time corresponding to data to be tested from data to be tested are extracted, using remaining data as training data.
For example, after the daily paper data that on March 12nd, 2018 generates on March 11st, 2018, according to daily paper data to be detected
March 12 2018 generated time obtains daily paper data to be detected, i.e. daily paper data corresponding to 11 days March in 2018；According to
March 12 2018 daily paper data generated time and 20 consecutive days of preset time period to be detected obtain corresponding to data to be tested
Training data, that is, obtain 2 month -2018 years on the 20th March 11 in 2018 corresponding to daily paper data as training data.
S120, normal data section is generated according to the training data and preset distributed model.
In general, current transaction data are as trade date is by certain trend dynamic change, meet certain distribution
Rule.Therefore distributed model can be pre-set, determines the specific distribution that training data is met, built and marked according to distributed model
Quasi- data interval, is detected data to be tested.
In the present embodiment, the training data corresponding to foundation data to be tested and the generation of preset distributed model are used for
Detect the normal data section of data to be tested.Optionally, the tool that current transaction data are met is calculated by training data
Body distributed model calculates normal data section according to distributed model and preset interval computation mode.Wherein, distributed model
Can be Gauss model, card side's model, t distributed models, bi-distribution model etc..
S130, the daily paper data to be detected are matched with the normal data section of generation, it is true according to matching result
Whether the fixed daily paper data to be detected are abnormal data.
In the present embodiment, daily paper data to be detected are matched with the normal data section of generation, is judged to be detected
Whether data are in normal data section.
Optionally, described to determine whether data to be tested are abnormal data according to matching result, including：
When the data to be tested are in the data interval, judge the data to be tested for normal data；Alternatively,
When the data to be tested are outside the data interval, judge the data to be tested for abnormal data.
The embodiment of the present invention by obtaining the training data corresponding to daily paper data to be detected and daily paper data to be detected,
Wherein training data was determined according to generated times of daily paper data to be detected so that training data can with data to be tested
Adaptive matching；Normal data section is generated according to training data and preset distributed model；By daily paper data to be detected and life
At normal data section matched, determine whether daily paper data to be detected are abnormal data according to matching result, it is proposed that
A kind of report data method for detecting abnormality with adaptivity so that the detection of daily paper data is quicker, can be automatically
Data exception is found in time, and improve abnormal data traces efficiency, reduces the coverage of abnormal data.
Embodiment two
Fig. 2 a are the flow charts of data exception detection method in the embodiment of the present invention two, and the present embodiment is in above-described embodiment
On the basis of further optimized.As shown in Figure 2 a, the method includes：
Training data corresponding to S210, acquisition daily paper data to be detected and the daily paper data to be detected.
S220, go out the trained number from existing daily paper extracting data according to generated times of the daily paper data to be detected
According to.
Optionally, after extracting the daily paper data needed for Data Detection, when according to the generation of daily paper data to be detected
Between determine periods of daily paper data corresponding to training data, according to the period determined from existing daily paper extracting data
Go out training data.
For example, if the period of the daily paper data corresponding to setting training data was 20 consecutive days, obtain existing
The daily paper data of preceding 20 consecutive days are as training data in daily paper data.Alternatively, when according to the generation of daily paper data to be detected
Between, determine the date corresponding to training data, the date corresponding to training data goes out from existing daily paper extracting data
Training data.
S230, the characteristic value that the distributed model is determined according to the training data.
In the present embodiment, the characteristic value for the distributed model that historical trading data is met is calculated according to training data.
Wherein, distributed model can be Gaussian Profile, chi square distribution etc..
Optionally, distributed model is Gaussian distribution model.Gaussian Profile also known as normal distribution are one in mathematics, physics
And all very important probability distribution in the fields such as engineering.There are two parameters for Gaussian Profile tool：The random variable of continuous type of μ and σ
Distribution, wherein parameter μ is the mean value for the stochastic variable for deferring to Gaussian Profile, parameter σ2It is this variance of a random variable, parameter σ is
The standard deviation of this stochastic variable, in general, Gaussian Profile is denoted as N (μ, σ2).Using Gaussian Profile as distributed model so that
The structure of model is simple, calculates simply, and the performance loss of computer is low.
Optionally, the characteristic value that distributed model can be calculated according to the characteristic value calculation formula of distributed model, with determination
The specific distribution that training data is met.
Optionally, described to determine the distributed mode according to the training data when distributed model is Gaussian distribution model
The characteristic value of type, including：
Determine the mean value and standard deviation of the Gaussian Profile that the training data is met.
Optionally, the calculation formula that training data is substituted into Gaussian Profile mean value and standard deviation respectively calculates trained number
According to the mean value and standard deviation of the Gaussian Profile met.
Wherein, the calculation formula of the mean value of Gaussian Profile can be：
The calculation formula of the standard deviation of Gaussian Profile can be：
Wherein n is the number of training data, XiIndicate that the numerical value of training data, μ indicate the mean value of Gaussian Profile, parameter σ
Indicate the standard deviation of Gaussian Profile.
S240, the normal data area is generated according to the characteristic value and preset configuration parameter using the distributed model
Between.
In the present embodiment, criterion numeral is generated according to the characteristic value of distributed model and the calculation formula in normal data section
According to section.
Optionally, when distributed model is Gauss model, according to the mean value and standard deviation of Gauss model difference and can match
It sets parameter and calculates normal data section：(μ-t*σ,μ+t*σ).Wherein, μ is the mean value of Gaussian Profile, and parameter σ is Gaussian Profile
Standard deviation, t are the configuration parameter in normal data section.Wherein, t is positive number.In the present embodiment, parameter can be configured by modification
Value adjustment normal data section range, and then the stringency of anomaly data detection is adjusted.
Fig. 2 b are the schematic diagrames in the corresponding normal data section of Gaussian distribution model in the embodiment of the present invention two, there is shown with
Normal data section corresponding when taking different numerical value t.It can be seen that t values are bigger by Fig. 2 b, normal data section
Range is bigger, and the stringency of anomaly data detection is lower；T values are smaller, and the range in normal data section is with regard to smaller, abnormal number
It is higher according to the stringency of detection.The value of parameter t is configured by modification, the range in adjustment normal data section makes abnormal data
The stringency of detection can be configured with detection demand.
S250, the daily paper data to be detected are matched with the normal data section of generation, it is true according to matching result
Whether the fixed daily paper data to be detected are abnormal data.
The technical solution of the embodiment of the present invention is embodied on the basis of said program according to training data and preset
Distributed model generates the process in normal data section, and the characteristic value of the distributed model of its satisfaction, root are calculated by training data
Normal data section is generated according to the characteristic value of distributed model and preset configuration parameter.Using this method, can be matched according to modification
The range in the value adjustment normal data section of parameter is set, and then adjusts the stringency of data exception detection.
Embodiment three
Fig. 3 is the flow chart of data exception detection method in the embodiment of the present invention three.The present embodiment is carried based on above-mentioned
A kind of preferred embodiment is supplied.As shown in figure 3, this method includes：
S310, daily paper data are generated.
In the present embodiment, T+1 daily paper is automatically generated according to the timed task of setting, i.e., generates and hands in the next day of the day of trade
Daily paper data on the day of Yi, and using the daily paper data being newly generated as data to be tested.
S320, training data is pulled.
Optionally, pulled from database recent daily paper data (it is 20 days such as nearly, but do not include newest number to be detected
According to) it is used as training data.
S330, the mean value and standard deviation for calculating the Gaussian Profile that training data is met.
The equal of the Gaussian Profile that training data is met is calculated by the calculation formula of Gaussian Profile mean value and standard deviation
Value and standard deviation.The calculation formula of wherein mean value and standard deviation can be found in above-described embodiment.
S340, normal data section is calculated.
Calculated mean value, standard deviation and preset configuration parameter are substituted into normal data interval computation formula, calculated
Go out standard data interval.
S350, judge latest data whether in normal data interval range.
Optionally, using the daily paper data being newly generated as data to be tested, by data to be tested and normal data section
Matched, judge data to be tested whether in standard interval range if so, execute S360；If it is not, executing S370.
S360, judgement data are normal data.
S370, judgement data are abnormal data.
Example IV
Fig. 4 is the structural schematic diagram of data exception detection device in the embodiment of the present invention four.Software may be used in the device
And/or the mode of hardware is realized, such as device can be configured in data exception detection device, as shown in figure 4, described device packet
It includes：
Data acquisition module 410, for obtaining corresponding to daily paper data to be detected and the daily paper data to be detected
Training data, wherein the training data was determined according to generated times of the daily paper data to be detected；
Section generation module 420, for generating normal data section according to the training data and preset distributed model；
Data detection module 430, for the daily paper data to be detected to be matched with the normal data section of generation,
Determine whether the daily paper data to be detected are abnormal data according to matching result.
On the basis of said program, the section generation module 420 includes：
Training data extraction unit, for the generated time according to the daily paper data to be detected from existing daily paper data
Extract the training data；
Characteristic value determination unit, the characteristic value for determining the distributed model according to the training data；
Section generation unit, for generating institute according to the characteristic value and preset configuration parameter using the distributed model
State normal data section.
On the basis of said program, the distributed model is Gaussian Profile.
On the basis of said program, described device further includes：
Data to be tested extraction module, for being carried out by the normal data section of the daily paper data to be detected and generation
Before matching, according to the period to be detected, go out data to be tested from existing daily paper extracting data.
On the basis of said program, the data detection module 430 is specifically used for：
When the data to be tested are in the data interval, judge the data to be tested for normal data；
Alternatively, when the data to be tested are outside the data interval, judge the data to be tested for abnormal data.
The data exception detection device that the embodiment of the present invention is provided can perform the number that any embodiment of the present invention is provided
According to method for detecting abnormality, have the corresponding function module of execution method and advantageous effect.
Embodiment five
Fig. 5 is the structural schematic diagram of data exception detection device in the embodiment of the present invention five.Fig. 5 shows real suitable for being used for
The block diagram of the example data abnormality detecting apparatus 512 of existing embodiment of the present invention.The data exception detection device that Fig. 5 is shown
512 be only an example, should not bring any restrictions to the function and use scope of the embodiment of the present invention.
As shown in figure 5, data exception detection device 512 is showed in the form of universal computing device.Data exception detection is set
Standby 512 component can include but is not limited to：One or more processing unit 516, system storage 528 connect not homology
The bus 518 of system component (including system storage 528 and processing unit 516).
Bus 518 indicates one or more in a few class bus structures, including memory bus or Memory Controller,
Peripheral bus, graphics acceleration port, processing unit 516 or total using the local of the arbitrary bus structures in a variety of bus structures
Line.For example, these architectures include but not limited to industry standard architecture (ISA) bus, microchannel architecture
(MAC) bus, enhanced isa bus, Video Electronics Standards Association (VESA) local bus and peripheral component interconnection (PCI) are total
Line.
Data exception detection device 512 typically comprises a variety of computer system readable media.These media can be appointed
What usable medium that can be accessed by data exception detection device 512, including volatile and non-volatile media, it is moveable and
Immovable medium.
System storage 528 may include the computer system readable media of form of volatile memory, such as deposit at random
Access to memory (RAM) 530 and/or cache memory 532.Data exception detection device 512 may further include other
Removable/nonremovable, volatile/non-volatile computer system storage medium.Only as an example, storage device 534 can
For reading and writing immovable, non-volatile magnetic media (Fig. 5 do not show, commonly referred to as " hard disk drive ").Although in Fig. 5
It is not shown, can provide for the disc driver to moving non-volatile magnetic disk (such as " floppy disk ") read-write, and pair can
The CD drive that mobile anonvolatile optical disk (such as CD-ROM, DVD-ROM or other optical mediums) is read and write.In these situations
Under, each driver can be connected by one or more data media interfaces with bus 518.Memory 528 may include
There is one group of (for example, at least one) program module, these program modules to be configured at least one program product, the program product
To execute the function of various embodiments of the present invention.
Program/utility 540 with one group of (at least one) program module 542, can be stored in such as memory
In 528, such program module 542 includes but not limited to operating system, one or more application program, other program modules
And program data, the realization of network environment may be included in each or certain combination in these examples.Program module 542
Usually execute the function and/or method in embodiment described in the invention.
Data exception detection device 512 can also be (such as keyboard, sensing equipment, aobvious with one or more external equipments 514
Show device 524 etc.) communication, the equipment interacted with the data exception detection device 512 can be also enabled a user to one or more
Communication, and/or it is any with enabling the data exception detection device 512 to be communicated with one or more of the other computing device
Equipment (such as network interface card, modem etc.) communicates.This communication can be carried out by input/output (I/O) interface 522.
Also, data exception detection device 512 can also pass through network adapter 520 and one or more network (such as LAN
(LAN), wide area network (WAN) and/or public network, such as internet) communication.As shown, network adapter 520 passes through bus
518 communicate with other modules of data exception detection device 512.It should be understood that although not shown in the drawings, can be different in conjunction with data
Normal detection device 512 uses other hardware and/or software module, including but not limited to：Microcode, device driver, at redundancy
Manage unit, external disk drive array, RAID system, tape drive and data backup storage system etc..
Processing unit 516 is stored in program in system storage 528 by operation, to perform various functions using with
And data processing, such as realize the data exception detection method that the embodiment of the present invention is provided, this method includes：
The training data corresponding to daily paper data to be detected and the daily paper data to be detected is obtained, wherein the training
Data were determined according to generated times of the daily paper data to be detected；
Normal data section is generated according to the training data and preset distributed model；
The daily paper data to be detected are matched with the normal data section of generation, described in matching result determination
Whether daily paper data to be detected are abnormal data.
Certainly, it will be understood by those skilled in the art that processing unit can also realize that any embodiment of the present invention is provided
Data exception detection method technical solution.
Embodiment six
The embodiment of the present invention six additionally provides a kind of computer readable storage medium, is stored thereon with computer program, should
Realize that the data exception detection method provided such as the embodiment of the present invention, this method include when program is executed by processor：
The training data corresponding to daily paper data to be detected and the daily paper data to be detected is obtained, wherein the training
Data were determined according to generated times of the daily paper data to be detected；
Normal data section is generated according to the training data and preset distributed model；
The daily paper data to be detected are matched with the normal data section of generation, described in matching result determination
Whether daily paper data to be detected are abnormal data.
Certainly, a kind of computer readable storage medium that the embodiment of the present invention is provided, the computer program stored thereon
The method operation being not limited to the described above, can also be performed in the data exception detection method that any embodiment of the present invention is provided
Relevant operation.
The arbitrary of one or more computer-readable media may be used in the computer storage media of the embodiment of the present invention
Combination.Computer-readable medium can be computer-readable signal media or computer readable storage medium.It is computer-readable
Storage medium for example may be-but not limited to-the system of electricity, magnetic, optical, electromagnetic, infrared ray or semiconductor, device or
Device, or the arbitrary above combination.The more specific example (non exhaustive list) of computer readable storage medium includes：Tool
There are one or the electrical connection of multiple conducting wires, portable computer diskette, hard disk, random access memory (RAM), read-only memory
(ROM), erasable programmable read only memory (EPROM or flash memory), optical fiber, portable compact disc read-only memory (CD-
ROM), light storage device, magnetic memory device or above-mentioned any appropriate combination.In this document, computer-readable storage
Medium, which can be any, includes or the tangible medium of storage program, which can be commanded execution system, device or device
Using or it is in connection.
Computer-readable signal media may include in a base band or as the data-signal that a carrier wave part is propagated,
Wherein carry computer-readable program code.Diversified forms may be used in the data-signal of this propagation, including but unlimited
In electromagnetic signal, optical signal or above-mentioned any appropriate combination.Computer-readable signal media can also be that computer can
Any computer-readable medium other than storage medium is read, which can send, propagates or transmit and be used for
By instruction execution system, device either device use or program in connection.
The program code for including on computer-readable medium can transmit with any suitable medium, including --- but it is unlimited
In wireless, electric wire, optical cable, RF etc. or above-mentioned any appropriate combination.
It can be write with one or more programming languages or combinations thereof for executing the computer that operates of the present invention
Program code, described program design language include object oriented program language-such as Java, Smalltalk, C++,
Further include conventional procedural programming language-such as " C " language or similar programming language.Program code can be with
It fully executes, partly execute on the user computer on the user computer, being executed as an independent software package, portion
Divide and partly executes or executed on a remote computer or server completely on the remote computer on the user computer.
Be related in the situation of remote computer, remote computer can pass through the network of any kind --- including LAN (LAN) or
Wide area network (WAN)-be connected to subscriber computer, or, it may be connected to outer computer (such as carried using Internet service
It is connected by internet for quotient).
Note that above are only presently preferred embodiments of the present invention and institute's application technology principle.It will be appreciated by those skilled in the art that
The present invention is not limited to specific embodiments described here, can carry out for a person skilled in the art it is various it is apparent variation,
It readjusts and substitutes without departing from protection scope of the present invention.Therefore, although being carried out to the present invention by above example
It is described in further detail, but the present invention is not limited only to above example, without departing from the inventive concept, also
May include other more equivalent embodiments, and the scope of the present invention is determined by scope of the appended claims.
Claims (10)
1. a kind of data exception detection method, which is characterized in that including：
The training data corresponding to daily paper data to be detected and the daily paper data to be detected is obtained, wherein the training data
It is to be determined according to generated times of the daily paper data to be detected；
Normal data section is generated according to the training data and preset distributed model；
The daily paper data to be detected are matched with the normal data section of generation, are determined according to matching result described to be checked
Survey whether daily paper data are abnormal data.
2. according to the method described in claim 1, it is characterized in that, described according to the training data and preset distributed model
Normal data section is generated, including：
Generated time according to the daily paper data to be detected goes out the training data from existing daily paper extracting data；
The characteristic value of the distributed model is determined according to the training data；
The normal data section is generated according to the characteristic value and preset configuration parameter using the distributed model.
3. according to the method described in claim 2, it is characterized in that, the distributed model is Gaussian Profile.
4. according to the method described in claim 3, it is characterized in that, described determine the distributed model according to the training data
Characteristic value, including：
Determine the mean value and standard deviation of the Gaussian Profile that the training data is met.
5. according to the method described in claim 1, it is characterized in that, it is described according to matching result determine data to be tested whether be
Abnormal data, including：
When the data to be tested are in the data interval, judge the data to be tested for normal data；
Alternatively,
When the data to be tested are outside the data interval, judge the data to be tested for abnormal data.
6. a kind of data exception detection device, which is characterized in that including：
Data acquisition module, for obtaining the training number corresponding to daily paper data to be detected and the daily paper data to be detected
According to wherein the training data was determined according to generated times of the daily paper data to be detected；
Section generation module, for generating normal data section according to the training data and preset distributed model；
Data detection module, for the daily paper data to be detected to be matched with the normal data section of generation, foundation
Determine whether the daily paper data to be detected are abnormal data with result.
7. device according to claim 6, which is characterized in that the section generation module includes：
Training data extraction unit, for the generated time according to the daily paper data to be detected from existing daily paper extracting data
Go out the training data；
Characteristic value determination unit, the characteristic value for determining the distributed model according to the training data；
Section generation unit, for generating the mark according to the characteristic value and preset configuration parameter using the distributed model
Quasi- data interval.
8. device according to claim 6, which is characterized in that the data detection module is specifically used for：
When the data to be tested are in the data interval, judge the data to be tested for normal data；
Alternatively,
When the data to be tested are outside the data interval, judge the data to be tested for abnormal data.
9. a kind of data exception detection device, which is characterized in that the equipment includes：
One or more processors；
Storage device, for storing one or more programs；
When one or more of programs are executed by one or more of processors so that one or more of processors are real
The now data exception detection method as described in any in claim 1-5.
10. a kind of computer readable storage medium, is stored thereon with computer program, which is characterized in that the program is by processor
The data exception detection method as described in any in claim 1-5 is realized when execution.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810225915.0A CN108491875A (en) | 2018-03-19 | 2018-03-19 | A kind of data exception detection method, device, equipment and medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810225915.0A CN108491875A (en) | 2018-03-19 | 2018-03-19 | A kind of data exception detection method, device, equipment and medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN108491875A true CN108491875A (en) | 2018-09-04 |
Family
ID=63318393
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201810225915.0A Pending CN108491875A (en) | 2018-03-19 | 2018-03-19 | A kind of data exception detection method, device, equipment and medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN108491875A (en) |
Cited By (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109739838A (en) * | 2018-12-06 | 2019-05-10 | 中科恒运股份有限公司 | Method for processing abnormal data and device |
| CN110299208A (en) * | 2019-05-22 | 2019-10-01 | 平安科技（深圳）有限公司 | Disease surveillance data exception detection method, system, equipment and storage medium |
| CN110399409A (en) * | 2019-07-29 | 2019-11-01 | 中国工商银行股份有限公司 | Transaction method for monitoring abnormality and device |
| CN110399268A (en) * | 2019-07-26 | 2019-11-01 | 阿里巴巴集团控股有限公司 | A kind of method, device and equipment of anomaly data detection |
| CN110930913A (en) * | 2019-12-10 | 2020-03-27 | 京东方科技集团股份有限公司 | Display compensation data, data detection method and device and display panel |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106548035A (en) * | 2016-11-24 | 2017-03-29 | 腾讯科技（深圳）有限公司 | A kind of diagnostic method and device of data exception |
| US20180053109A1 (en) * | 2016-08-18 | 2018-02-22 | International Business Machines Corporation | Confidence intervals for anomalies in computer log data |
| CN107766299A (en) * | 2017-10-24 | 2018-03-06 | 携程旅游信息技术（上海）有限公司 | The abnormal monitoring method of data target and its system, storage medium, electronic equipment |
-
2018
- 2018-03-19 CN CN201810225915.0A patent/CN108491875A/en active Pending
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20180053109A1 (en) * | 2016-08-18 | 2018-02-22 | International Business Machines Corporation | Confidence intervals for anomalies in computer log data |
| CN106548035A (en) * | 2016-11-24 | 2017-03-29 | 腾讯科技（深圳）有限公司 | A kind of diagnostic method and device of data exception |
| CN107766299A (en) * | 2017-10-24 | 2018-03-06 | 携程旅游信息技术（上海）有限公司 | The abnormal monitoring method of data target and its system, storage medium, electronic equipment |
Cited By (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109739838A (en) * | 2018-12-06 | 2019-05-10 | 中科恒运股份有限公司 | Method for processing abnormal data and device |
| CN110299208A (en) * | 2019-05-22 | 2019-10-01 | 平安科技（深圳）有限公司 | Disease surveillance data exception detection method, system, equipment and storage medium |
| CN110399268A (en) * | 2019-07-26 | 2019-11-01 | 阿里巴巴集团控股有限公司 | A kind of method, device and equipment of anomaly data detection |
| CN110399268B (en) * | 2019-07-26 | 2023-09-26 | 创新先进技术有限公司 | Abnormal data detection method, device and equipment |
| CN110399409A (en) * | 2019-07-29 | 2019-11-01 | 中国工商银行股份有限公司 | Transaction method for monitoring abnormality and device |
| CN110399409B (en) * | 2019-07-29 | 2022-02-08 | 中国工商银行股份有限公司 | Transaction abnormity monitoring method and device |
| CN110930913A (en) * | 2019-12-10 | 2020-03-27 | 京东方科技集团股份有限公司 | Display compensation data, data detection method and device and display panel |
| WO2021115307A1 (en) * | 2019-12-10 | 2021-06-17 | 京东方科技集团股份有限公司 | Method and apparatus for processing data, and display panel |
| CN110930913B (en) * | 2019-12-10 | 2021-10-22 | 京东方科技集团股份有限公司 | Display compensation data, data detection method and device and display panel |
| US11645978B2 (en) | 2019-12-10 | 2023-05-09 | Hefei Xinsheng Optoelectronics Technology Co., Ltd. | Data processing method and device, and display panel |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN108491875A (en) | A kind of data exception detection method, device, equipment and medium | |
| US20210035126A1 (en) | Data processing method, system and computer device based on electronic payment behaviors | |
| CN113360580B (en) | Abnormal event detection method, device, equipment and medium based on knowledge graph | |
| US20210117978A1 (en) | Graph decomposition for fraudulent transaction analysis | |
| CN112131322B (en) | Time sequence classification method and device | |
| CN108564944B (en) | Intelligent control method, system, equipment and storage medium | |
| CN109951449A (en) | A kind of abnormal login detecting method, device, electronic equipment and storage medium | |
| CN112613584A (en) | Fault diagnosis method, device, equipment and storage medium | |
| US20230066703A1 (en) | Method for estimating structural vibration in real time | |
| CN112990035A (en) | Text recognition method, device, equipment and storage medium | |
| US20220229904A1 (en) | Privacy-protection-based data processing model acquisition method and apparatus, terminal device and storage medium | |
| CN108538290A (en) | A kind of intelligent home furnishing control method based on audio signal detection | |
| CN112598784B (en) | Method and device for adjusting contour scanning result, electronic equipment and storage medium | |
| Barlowe et al. | Multivariate visual explanation for high dimensional datasets | |
| CN114490990B (en) | Method, device, equipment and storage medium for determining text to be annotated | |
| CN112541557B (en) | Training method and device for generating countermeasure network and electronic equipment | |
| CN111859985B (en) | AI customer service model test method and device, electronic equipment and storage medium | |
| CN113379469A (en) | Abnormal flow detection method, device, equipment and storage medium | |
| CN104333479B (en) | Server power supply annexation detection method and detecting system | |
| CN114141236A (en) | Language model updating method and device, electronic equipment and storage medium | |
| CN113657516A (en) | Method and device for processing medical transaction data, electronic equipment and storage medium | |
| CN114782668A (en) | Model aggregation method, device and system and electronic equipment | |
| CN110796561A (en) | Influence maximization method and device based on three-hop velocity attenuation propagation model | |
| CN113791391B (en) | Interference characteristic parameter identification method, device, storage and equipment | |
| CN114722061B (en) | Data processing method and device, equipment and computer readable storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20180904 |
|
| RJ01 | Rejection of invention patent application after publication |