CN105930328A - Analytical method and system for abnormal data - Google Patents
Analytical method and system for abnormal data Download PDFInfo
- Publication number
- CN105930328A CN105930328A CN201510896335.0A CN201510896335A CN105930328A CN 105930328 A CN105930328 A CN 105930328A CN 201510896335 A CN201510896335 A CN 201510896335A CN 105930328 A CN105930328 A CN 105930328A
- Authority
- CN
- China
- Prior art keywords
- data
- preset kind
- resolved
- dirty
- normal
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 230000002159 abnormal effect Effects 0.000 title claims abstract description 72
- 238000004458 analytical method Methods 0.000 title claims abstract description 48
- 238000000034 method Methods 0.000 claims abstract description 32
- 230000009286 beneficial effect Effects 0.000 abstract description 9
- 238000012217 deletion Methods 0.000 description 6
- 230000037430 deletion Effects 0.000 description 6
- 238000010276 construction Methods 0.000 description 4
- 238000010586 diagram Methods 0.000 description 2
- 238000005516 engineering process Methods 0.000 description 2
- 241001269238 Data Species 0.000 description 1
- 241000406668 Loxodonta cyclotis Species 0.000 description 1
- 238000006243 chemical reaction Methods 0.000 description 1
- 238000013524 data verification Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 238000004880 explosion Methods 0.000 description 1
- 230000002349 favourable effect Effects 0.000 description 1
- 238000005457 optimization Methods 0.000 description 1
- 238000005096 rolling process Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/23—Updating
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Data Mining & Analysis (AREA)
- Databases & Information Systems (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Selective Calling Equipment (AREA)
Abstract
The invention discloses an analytical method and system for abnormal data. The method includes the steps of receiving data to be analyzed from a data server; analyzing the received data to be analyzed, and determining whether dirty data of a preset type are present in the data to be analyzed; if so, determining an operating type corresponding to the present dirty data of the preset type; and finding out the determined operating type from operating types of null data of a preset type, finding out a function corresponding to the operating type that is found out of the empty data of the preset type, and calling out the function, which is found out, for data processing of the dirty data of the preset type. The invention has the beneficial effects of easily finding out the dirty data from the data to be analyzed and timely processing the dirty data that is found out, reduces the analytical burden of the system and the volume of a system analyzer, and improves the data processing efficiency.
Description
Technical field
The present invention relates to technical field of data processing, particularly relate to the analytic method of a kind of abnormal data
And system.
Background technology
Along with making constant progress of data processing technique, in the epoch of current explosion type information, greatly
Data are also required to new tupe just can be had more powerful decision edge, see clearly discovery power and flow process
The magnanimity of optimization ability, high growth rate and diversified feature, so can become favourable letter
Breath assets, for people to use.
At present, process the mathematical logic returned about data for data server in the industry, for
Improve the probability not having dirty data in the data returned, then need to add in resolver a large amount of
Check code carry out dirty data verification；This processing mode wastes man-hour, and increases Solutions of Systems
The scale of construction of parser, adds the parsing burden of Solutions of Systems parser.
Summary of the invention
In view of the foregoing, it is necessary to analytic method and the system of a kind of abnormal data are provided, in order to:
From data to be resolved, find out dirty data easily and the dirty data found out is processed in time, subtracting
The parsing burden of light system.
The invention discloses the analytic method of a kind of abnormal data, comprise the following steps:
Data to be resolved are received from data server；
The data to be resolved received are resolved, and judges whether described data to be resolved exist
Preset kind dirty data；
If judging described data to be resolved exist preset kind dirty data, it is determined that go out existence
Action type corresponding to described preset kind dirty data；
From the action type of preset kind sky data, find the action type determined, find out and look into
The function that the action type of the preset kind sky data found is corresponding, calls the function found out to described
Preset kind dirty data carries out data process.
Preferably, the described data to be resolved to receiving resolve, and judge described number to be resolved
Whether there is preset kind dirty data according to, further comprise the steps of: afterwards
According to normal resolution logic, the normal data of non-preset kind dirty data is resolved, raw
Become corresponding analysis result.
Preferably, described according to normal resolution logic, the normal data to non-preset kind dirty data
Resolve, generate corresponding analysis result, including:
According to normal resolution logic, resolve in the normal data of non-preset kind dirty data and whether exist
Preset kind abnormal conditions；
If described normal data exists preset kind abnormal conditions, then for there is described default class
The normal data of type abnormal conditions does not carry out any operation, generates and resolves daily record and record the different of correspondence
Reason condition.
Preferably, described preset kind abnormal conditions include:
Array Bound, character string position cross the border, empty numerical value and/or null key.
Preferably, described preset kind sky data include:
Null character string, empty dictionary, empty array, null set and/or empty numerical value.
Corresponding to the analytic method of a kind of abnormal data disclosed above, the invention also discloses one
Plant the resolution system of abnormal data, including:
Data reception module, for receiving data to be resolved from data server；
Data resolution module, for resolving the data to be resolved received, and treats described in judgement
Resolve in data and whether there is preset kind dirty data；
Data processing module, is used for:
If described data resolution module judges to exist in described data to be resolved the dirty number of preset kind
According to, it is determined that go out the action type corresponding to described preset kind dirty data existed；
From the action type of preset kind sky data, find the action type determined, find out and look into
The function that the action type of the preset kind sky data found is corresponding, calls the function found out to described
Preset kind dirty data carries out data process.
Preferably, described data resolution module is additionally operable to:
According to normal resolution logic, the normal data of non-preset kind dirty data is resolved, raw
Become corresponding analysis result.
Preferably, described data resolution module includes:
Data parsing unit, for according to normal resolution logic, resolves non-preset kind dirty data
Whether normal data exists preset kind abnormal conditions；
, if there are preset kind abnormal conditions, then in described normal data in exception processing unit
Do not carry out any operation for the normal data that there are described preset kind abnormal conditions, generate and resolve
Daily record also records corresponding abnormal conditions.
Preferably, described preset kind abnormal conditions include:
Array Bound, character string position cross the border, empty numerical value and/or null key.
Preferably, described preset kind sky data include:
Null character string, empty dictionary, empty array, null set and/or empty numerical value.
The analytic method of a kind of abnormal data of the present invention and system can reach following beneficial effect:
By receiving data to be resolved from data server；The data to be resolved received are carried out
Resolve, and judge whether described data to be resolved exist preset kind dirty data；If judging institute
State and data to be resolved exist preset kind dirty data, it is determined that go out the described preset kind existed dirty
Action type corresponding to data；Find from the action type of preset kind sky data and determine
Action type, find out the function that the action type of the preset kind sky data found is corresponding, adjust
With the function found out, described preset kind dirty data is carried out data process；Have and solve from waiting easily
Analysis data find out dirty data the beneficial effect processed the dirty data found out in time, alleviates
The scale of construction resolving burden and Solutions of Systems parser of system, improves data-handling efficiency.
Accompanying drawing explanation
Fig. 1 is the schematic flow sheet of a kind of embodiment of the analytic method of abnormal data of the present invention；
Fig. 2 is the schematic flow sheet of the another embodiment of the analytic method of abnormal data of the present invention；
Fig. 3 is in the analytic method of abnormal data of the present invention, in embodiment described in Fig. 2 the one of step S50
Plant the schematic flow sheet of embodiment；
Fig. 4 is the block diagram of a kind of embodiment of the resolution system of abnormal data of the present invention；
Fig. 5 is in the resolution system of abnormal data of the present invention, parsing module 70 in embodiment described in Fig. 4
The block diagram of a kind of embodiment.
The realization of embodiment of the present invention purpose, functional characteristics and advantage will in conjunction with the embodiments, with reference to attached
Figure is described further.
Detailed description of the invention
Technical scheme is further illustrated below in conjunction with Figure of description and specific embodiment.
Should be appreciated that specific embodiment described herein, only in order to explain the present invention, is not used to limit
Determine the present invention.
Analytic method and the system of abnormal data of the present invention may operate in client, it is also possible to run
At server end, the data such as returned data server in the data parser of client are entered
Row resolves, thus determines whether dirty data；One data parser can also be set at server end,
The data directly returned data server resolve, thus can also judge whether dirty number
According to.
It addition, in the analytic method of abnormal data of the present invention and system, described dirty data is permissible
Be interpreted as: the data in origin system not in given scope or meaningless for practical business,
Or data form is illegal, and in origin system, there is nonstandard coding and ambiguous business is patrolled
Volume.
In database technology, dirty data produces interim renewal in (dirty reading).Affairs A update
Certain data item X, but for a certain reason, affairs A there is a problem, A is returned in being intended to
Rolling.But before rollback, another affairs B have read the value (after A updates) of data item X, A
Rollback affairs, data item recovered initial value.Being exactly one and " face of data item X that affairs B read
Time " value, i.e. dirty data.Popular says, when affairs are accessing data, and logarithm
According to being modified, and this amendment is not the most submitted in data base；At this moment, another one thing
Business also accesses these data, then employs these data.Because these data are the most not submit to
Data, then these data that another one affairs are read are dirty datas, are done according to dirty data
Operation be probably incorrect.
The analytic method of abnormal data of the present invention and system is follow-up no longer will go to live in the household of one's in-laws on getting married foregoing
State.
The invention provides the analytic method of a kind of abnormal data, in order to: easily from number to be resolved
Find out dirty data according to and the dirty data found out is processed in time, alleviating the parsing burden of system.
As it is shown in figure 1, the analytic method of embodiment of the present invention abnormal data may be implemented as the step described
Rapid S10-S40:
Step S10, from data server, receive data to be resolved；
Step S20, the data to be resolved received are resolved, and judge in described data to be resolved
Whether there is preset kind dirty data；
In the embodiment of the present invention, connect from data server for carrying out the resolution system of data parsing
After receiving data to be resolved, the data to be resolved received are resolved, it is judged that this is to be resolved
Whether data exist preset kind dirty data.
In a preferred embodiment, described preset kind dirty data is empty object.Resolving
Time, each data to be resolved received by resolution system identification, it is judged that above-mentioned data to be resolved
In whether be sky object.
In the analytic method of embodiment of the present invention abnormal data, described " object " is appreciated that
For: people to carry out anything studied；Such as, from simplest integer to complicated aircraft
Can regard object as Deng all, it can not only represent concrete things, moreover it is possible to represents abstract rule, meter
Draw or event etc..Such as, object can be data, array, specific people etc..
If step S30 is judged to there is preset kind dirty data in described data to be resolved, it is determined that
Go out the action type corresponding to described preset kind dirty data existed；
If judging to there is preset kind dirty data in described data to be resolved, then resolution system determines
Go out the action type corresponding to preset kind dirty data existed.Such as, for above-mentioned preset kind
Dirty data, resolution system determines with the action type corresponding to above-mentioned preset kind dirty data and is: wound
Build, call, update, read and/or the action type such as deletion.
Step S40, from the action type of preset kind sky data, find the operation class determined
Type, find out the function that the action type of the preset kind sky data found is corresponding, calls and finds out
Function carries out data process to described preset kind dirty data.
After determining the action type that above-mentioned preset kind dirty data is corresponding, resolution system first from
The action type of preset kind sky data finds the action type determined, then finds out lookup
The function that the action type of the preset kind sky data arrived is corresponding, finally directly invokes the function found out
Preset kind dirty data is carried out data process.
In a preferred embodiment, described preset kind sky data include but not limited to:
Null character string, empty dictionary, empty array, null set and/or empty numerical value.
Such as, in a concrete application scenarios, when above-mentioned preset kind dirty data is empty object,
If it is determined that empty action type corresponding to object is deletion action, it is determined that go out preset kind sky number
According to delete behaviour process function Y corresponding to operation, and call the process function Y determined, to empty right
The data of elephant carry out deletion action.
The analytic method of embodiment of the present invention abnormal data can reach following beneficial effect:
By receiving data to be resolved from data server；The data to be resolved received are carried out
Resolve, and judge whether described data to be resolved exist preset kind dirty data；If judging institute
State and data to be resolved exist preset kind dirty data, it is determined that go out the described preset kind existed dirty
Action type corresponding to data；Find from the action type of preset kind sky data and determine
Action type, find out the function that the action type of the preset kind sky data found is corresponding, adjust
With the function found out, described preset kind dirty data is carried out data process；Have and solve from waiting easily
Analysis data find out dirty data the beneficial effect processed the dirty data found out in time, alleviates
The scale of construction resolving burden and Solutions of Systems parser of system, improves data-handling efficiency.
Description based on embodiment described in Fig. 1, in the analytic method of abnormal data of the present invention, resolves system
System, for the normal data of non-preset kind dirty data, still resolves according to normal resolution logic,
And generate the analysis result of correspondence.
As in figure 2 it is shown, in the analytic method of embodiment of the present invention abnormal data, implementing described in Fig. 1
" step S20, the data to be resolved received being resolved, and judge described data to be resolved of example
In whether there is preset kind dirty data " after, it is also possible to be embodied as steps as described below S50；
Step S50, according to normal resolution logic, the normal data of non-preset kind dirty data is carried out
Resolve, generate corresponding analysis result.
In the embodiment of the present invention, in order to ensure that resolution system can to non-preset kind dirty data just
Regular data normally resolves, and resolution system, for above-mentioned normal data, is patrolled according to normal parsing
Collect and resolve, and generate the analysis result of correspondence；Such as, generate corresponding parsing daily record, and
The concrete parsing situation corresponding to whole resolving is recorded in resolving daily record.
In a preferred embodiment, as it is shown on figure 3, in the analytic method of abnormal data of the present invention,
In embodiment described by Fig. 2, " step S50, according to normal resolution logic, dirty to non-preset kind
The normal data of data resolves, and generates corresponding analysis result ", may be implemented as described below
Step S510-S520:
Step S510, according to normal resolution logic, resolve the normal data of non-preset kind dirty data
In whether there are preset kind abnormal conditions；
If there are preset kind abnormal conditions, then for existence in the described normal data of step S520
The normal data of described preset kind abnormal conditions does not carry out any operation, generates and resolves daily record and remember
The abnormal conditions that record is corresponding.
In the embodiment of the present invention, for the normal data of non-preset kind dirty data, resolution system exists
During parsing, according to normal resolution logic, first determine whether the normal data of this non-preset kind dirty data
In whether there are preset kind abnormal conditions.
In the embodiment of the present invention, described preset kind abnormal conditions include but not limited to:
Array Bound, character string position cross the border, empty numerical value (empty value) and/or null key (empty
key)；Wherein, described " Array Bound " is it is to be understood that there are 10 units in an array
Element, crosses the border when we access the element beyond these 10 elements when exactly；Such as, the is accessed
11 elements." crossing the border in character string position " it is to be understood that there are 10 characters in an array, when
We cross the border the when of accessing the element beyond these 10 characters exactly；Such as: access the 12nd word
Symbol.
If resolution system parses there is preset kind abnormal conditions, then pin in above-mentioned normal data
The normal data that there are described preset kind abnormal conditions is not carried out any operation, only generates parsing
Daily record also records corresponding abnormal conditions.If resolution system parses in above-mentioned normal data and does not deposits
In preset kind abnormal conditions, then always according to normal resolution logic, above-mentioned normal data is entered
Row resolves, and generates corresponding analysis result；Such as, generate corresponding parsing daily record, and resolving
Daily record records the concrete parsing situation corresponding to whole resolving.
In the analytic method of embodiment of the present invention abnormal data, for non-preset kind dirty data just
Regular data, resolves according to normal resolution logic, generates corresponding analysis result；Have into one
Step improves the beneficial effect of data parsing efficiency；It is simultaneous for the analysis result generated, it is simple to follow-up
Carry out inquiring about and reference for parsing situation, improve the convenience of data parsing, also in certain journey
Consumer's Experience is improve on degree.
Parsing side corresponding to a kind of abnormal data described by figure 1 above, Fig. 2 and Fig. 3 embodiment
Method, present invention also offers the resolution system of a kind of abnormal data；As shown in Figure 4, the present invention implements
The resolution system of example abnormal data includes: data reception module 60, data resolution module 70 and data
Processing module 80；Wherein:
Data reception module 60, for receiving data to be resolved from data server；
Data resolution module 70, for resolving the data to be resolved received, and judges described
Whether data to be resolved exist preset kind dirty data；
In the embodiment of the present invention, data reception module 60 receives to be resolved from data server
After data, the data to be resolved that data reception module 60 is received by data resolution module 70 solve
Analysis, it is judged that whether there is preset kind dirty data in these data to be resolved.
In a preferred embodiment, described preset kind dirty data is empty object.Data solution
Analysis module 70 resolve time, each data to be resolved received by resolution system identification, it is judged that
Whether above-mentioned data to be resolved are sky object.
In the resolution system of embodiment of the present invention abnormal data, described " object " is appreciated that
For: people to carry out anything studied；Such as, from simplest integer to complicated aircraft
Can regard object as Deng all, it can not only represent concrete things, moreover it is possible to represents abstract rule, meter
Draw or event etc..Such as, object can be data, array, specific people etc..
Data processing module 80, is used for:
If described data resolution module 70 judges to exist in described data to be resolved the dirty number of preset kind
According to, it is determined that go out the action type corresponding to described preset kind dirty data existed；From default class
The action type of type sky data finds the action type determined, finds out the default class found
The function that the action type of type sky data is corresponding, calls the function found out number dirty to described preset kind
According to carrying out data process.
Such as, for above-mentioned preset kind dirty data, data processing module 80 determines to be preset with above-mentioned
Action type corresponding to type dirty data is: creates, call, update, read and/or deletion etc.
Action type.
After determining the action type that above-mentioned preset kind dirty data is corresponding, data processing module 80
From the action type of preset kind sky data, find the action type determined, find out and find
Function corresponding to the action type of preset kind sky data, call the function found out and preset described
Type dirty data carries out data process.
In a preferred embodiment, described preset kind sky data include but not limited to:
Null character string, empty dictionary, empty array, null set and/or empty numerical value.
Such as, in a concrete application scenarios, when above-mentioned preset kind dirty data is empty object,
Data processing module 80 is if it is determined that empty action type corresponding to object is deletion action, it is determined that
Go out the process function Y corresponding with deleting behaviour's operation of preset kind sky data, and call the place determined
The data of empty object are carried out deletion action by reason function Y.
The resolution system of embodiment of the present invention abnormal data can reach following beneficial effect:
By receiving data to be resolved from data server；The data to be resolved received are carried out
Resolve, and judge whether described data to be resolved exist preset kind dirty data；If judging institute
State and data to be resolved exist preset kind dirty data, it is determined that go out the described preset kind existed dirty
Action type corresponding to data；Find from the action type of preset kind sky data and determine
Action type, find out the function that the action type of the preset kind sky data found is corresponding, adjust
With the function found out, described preset kind dirty data is carried out data process；Have and solve from waiting easily
Analysis data find out dirty data the beneficial effect processed the dirty data found out in time, alleviates
The scale of construction resolving burden and Solutions of Systems parser of system, improves data-handling efficiency.
Description based on embodiment described in Fig. 4, in the resolution system of abnormal data of the present invention, described number
According to parsing module 70 for the normal data of non-preset kind dirty data, still patrol according to normal parsing
Collect and resolve, and generate the analysis result of correspondence.
Referring once again to Fig. 4, as shown in Figure 4, described data resolution module 70 is additionally operable to:
According to normal resolution logic, the normal data of non-preset kind dirty data is resolved, raw
Become corresponding analysis result.
In the embodiment of the present invention, in order to ensure that data resolution module 70 can number dirty to non-preset kind
According to normal data normally resolve, data resolution module 70 for above-mentioned normal data, according to
Normal resolution logic resolves, and generates the analysis result of correspondence；Such as, data parsing mould
Block 70 generates the parsing daily record of correspondence, and records corresponding to whole resolving in resolving daily record
Specifically resolve situation.
In a preferred embodiment, as it is shown in figure 5, in the resolution system of abnormal data of the present invention,
Data resolution module 70 described by Fig. 4 embodiment includes:
Data parsing unit 710, for according to normal resolution logic, resolves non-preset kind dirty data
Normal data in whether there are preset kind abnormal conditions；
Exception processing unit 720, if there are preset kind abnormal conditions in described normal data,
Then do not carry out any operation for the normal data that there are described preset kind abnormal conditions, generate and solve
Analysis daily record also records corresponding abnormal conditions.
In the embodiment of the present invention, for the normal data of non-preset kind dirty data, data parsing mould
Block 70, when resolving, according to normal resolution logic, is first judged this non-pre-by data parsing unit 710
If whether the normal data of type dirty data exists preset kind abnormal conditions.
In the embodiment of the present invention, described preset kind abnormal conditions include but not limited to:
Array Bound, character string position cross the border, empty object, empty numerical value (empty value) and/or empty close
Key word (empty key)；Wherein, described " Array Bound " is it is to be understood that in an array
There are 10 elements, cross the border exactly when we access the element beyond these 10 elements when；Such as,
Access the 11st element." crossing the border in character string position " is it is to be understood that there are 10 words in an array
Symbol, crosses the border when we access the element beyond these 10 characters when exactly；Such as: access the
12 characters.
If data parsing unit 710 parses there are preset kind exception feelings in above-mentioned normal data
Condition, then exception processing unit 720 does not enters for the normal data that there are described preset kind abnormal conditions
Any operation of row, only generates and resolves daily record and record the abnormal conditions of correspondence.If data parsing list
Unit 710 parses and there are not preset kind abnormal conditions in above-mentioned normal data, then always according to normally
Resolution logic, above-mentioned normal data is resolved, generates corresponding analysis result；Such as,
Generate corresponding parsing daily record, and record corresponding to whole resolving in resolving daily record concrete
Parsing situation.
In the resolution system of embodiment of the present invention abnormal data, for non-preset kind dirty data
Normal data, resolves according to normal resolution logic, generates corresponding analysis result；Have into
One step improves the beneficial effect of data parsing efficiency；It is simultaneous for the analysis result generated, it is simple to after
Continue and carry out inquiring about and reference for parsing situation, improve the convenience of data parsing, also necessarily
Consumer's Experience is improve in degree.
It should be noted that in this article, term " include ", " comprising " or any other band
Having nonexcludability to contain the word of meaning, its effect is to show to include the process of a series of key element, side
Method, article or system not only include those key elements, but also other including being not expressly set out
Key element, or also include the key element intrinsic for this process, method, article or system.
In the case of there is no more restriction, statement " including ... " key element limited, not
Get rid of in including the process of this key element, method, article or system, there is also other identical want
Element.
The invention described above embodiment sequence number, just to describing, does not represent the quality of embodiment.
Through the above description of the embodiments, those skilled in the art is it can be understood that arrive
Above-described embodiment method can add the mode of required general hardware platform by software and realize, certainly
Can also pass through hardware, but a lot of in the case of the former is more preferably embodiment.Based on such reason
Solving, the part that prior art is contributed by technical scheme the most in other words can be with
The form of software product embodies, this computer software product be stored in a storage medium (as
ROM/RAM, magnetic disc, CD) in, including some instructions with so that a station terminal equipment (can
To be mobile phone, computer, server, or the network equipment etc.) perform each embodiment of the present invention
Described method.
The foregoing is only the preferred embodiments of the present invention, not thereby limit its scope of the claims, all
It is the equivalent structure utilizing description of the invention and accompanying drawing content to be made or equivalence flow process conversion, directly
Or indirectly it is used in other relevant technical fields, the most in like manner it is included in the patent protection model of the present invention
In enclosing.
Claims (10)
1. the analytic method of an abnormal data, it is characterised in that comprise the following steps:
Data to be resolved are received from data server；
The data to be resolved received are resolved, and judges whether described data to be resolved exist
Preset kind dirty data；
If judging described data to be resolved exist preset kind dirty data, it is determined that go out existence
Action type corresponding to described preset kind dirty data；
From the action type of preset kind sky data, find the action type determined, find out and look into
The function that the action type of the preset kind sky data found is corresponding, calls the function found out to described
Preset kind dirty data carries out data process.
2. the method for claim 1, it is characterised in that the described number to be resolved to receiving
According to resolving, and judge whether described data to be resolved exist preset kind dirty data, afterwards
Further comprise the steps of:
According to normal resolution logic, the normal data of non-preset kind dirty data is resolved, raw
Become corresponding analysis result.
3. method as claimed in claim 2, it is characterised in that described according to normal resolution logic,
The normal data of non-preset kind dirty data is resolved, generates corresponding analysis result, including:
According to normal resolution logic, resolve in the normal data of non-preset kind dirty data and whether exist
Preset kind abnormal conditions；
If described normal data exists preset kind abnormal conditions, then for there is described default class
The normal data of type abnormal conditions does not carry out any operation, generates and resolves daily record and record the different of correspondence
Reason condition.
4. method as claimed in claim 3, it is characterised in that described preset kind abnormal conditions
Including:
Array Bound, character string position cross the border, empty numerical value and/or null key.
5. the method as described in any one of Claims 1-4, it is characterised in that described preset kind
Empty data include:
Null character string, empty dictionary, empty array, null set and/or empty numerical value.
6. the resolution system of an abnormal data, it is characterised in that including:
Data reception module, for receiving data to be resolved from data server；
Data resolution module, for resolving the data to be resolved received, and treats described in judgement
Resolve in data and whether there is preset kind dirty data；
Data processing module, is used for:
If described data resolution module judges to exist in described data to be resolved the dirty number of preset kind
According to, it is determined that go out the action type corresponding to described preset kind dirty data existed；
From the action type of preset kind sky data, find the action type determined, find out and look into
The function that the action type of the preset kind sky data found is corresponding, calls the function found out to described
Preset kind dirty data carries out data process.
7. system as claimed in claim 6, it is characterised in that described data resolution module is also used
In:
According to normal resolution logic, the normal data of non-preset kind dirty data is resolved, raw
Become corresponding analysis result.
8. system as claimed in claim 7, it is characterised in that described data resolution module includes:
Data parsing unit, for according to normal resolution logic, resolves non-preset kind dirty data
Whether normal data exists preset kind abnormal conditions；
, if there are preset kind abnormal conditions, then in described normal data in exception processing unit
Do not carry out any operation for the normal data that there are described preset kind abnormal conditions, generate and resolve
Daily record also records corresponding abnormal conditions.
9. system as claimed in claim 8, it is characterised in that described preset kind abnormal conditions
Including:
Array Bound, character string position cross the border, empty numerical value and/or null key.
10. the system as described in any one of claim 6 to 9, it is characterised in that described default class
Type sky data include:
Null character string, empty dictionary, empty array, null set and/or empty numerical value.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201510896335.0A CN105930328A (en) | 2015-12-08 | 2015-12-08 | Analytical method and system for abnormal data |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201510896335.0A CN105930328A (en) | 2015-12-08 | 2015-12-08 | Analytical method and system for abnormal data |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN105930328A true CN105930328A (en) | 2016-09-07 |
Family
ID=56840291
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201510896335.0A Pending CN105930328A (en) | 2015-12-08 | 2015-12-08 | Analytical method and system for abnormal data |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN105930328A (en) |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108170652A (en) * | 2017-12-29 | 2018-06-15 | 北京酷我科技有限公司 | A kind of method of iOS dictionaries security solution |
| CN111158960A (en) * | 2019-12-31 | 2020-05-15 | 北京讯鸟软件有限公司 | Distributed abnormal data processing method and device based on memory |
| CN114840599A (en) * | 2022-07-05 | 2022-08-02 | 杭州广立微电子股份有限公司 | Semiconductor source data parsing method, ETL system, computer device and product |
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103593352A (en) * | 2012-08-15 | 2014-02-19 | 阿里巴巴集团控股有限公司 | Method and device for cleaning mass data |
| CN104268216A (en) * | 2014-09-24 | 2015-01-07 | 江苏名通信息科技有限公司 | Data cleaning system based on internet information |
-
2015
- 2015-12-08 CN CN201510896335.0A patent/CN105930328A/en active Pending
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103593352A (en) * | 2012-08-15 | 2014-02-19 | 阿里巴巴集团控股有限公司 | Method and device for cleaning mass data |
| CN104268216A (en) * | 2014-09-24 | 2015-01-07 | 江苏名通信息科技有限公司 | Data cleaning system based on internet information |
Non-Patent Citations (2)
| Title |
|---|
| POWER0405HF: "数据清洗", 《HTTPS://BLOG.CSDN.NET/POWER0405HF/ARTICLE/DETAILS/49644041》 * |
| 王武: "数据清洗方法研究及工具设计", 《中国优秀硕士学位论文全文数据库信息科技辑》 * |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108170652A (en) * | 2017-12-29 | 2018-06-15 | 北京酷我科技有限公司 | A kind of method of iOS dictionaries security solution |
| CN111158960A (en) * | 2019-12-31 | 2020-05-15 | 北京讯鸟软件有限公司 | Distributed abnormal data processing method and device based on memory |
| CN114840599A (en) * | 2022-07-05 | 2022-08-02 | 杭州广立微电子股份有限公司 | Semiconductor source data parsing method, ETL system, computer device and product |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN103297435B (en) | A kind of abnormal access behavioral value method and system based on WEB daily record | |
| CN113011889B (en) | Account anomaly identification method, system, device, equipment and medium | |
| CN108959244A (en) | The method and apparatus of address participle | |
| US11568344B2 (en) | Systems and methods for automated pattern detection in service tickets | |
| CN113328994A (en) | Malicious domain name processing method, device, equipment and machine readable storage medium | |
| CN105930328A (en) | Analytical method and system for abnormal data | |
| CN112258254A (en) | Internet advertisement risk monitoring method and system based on big data architecture | |
| CN116980162A (en) | Cloud audit data detection method, device, equipment, medium and program product | |
| CN111586695A (en) | Short message identification method and related equipment | |
| CN115204733A (en) | Data auditing method and device, electronic equipment and storage medium | |
| CN116862243B (en) | Enterprise risk analysis prediction method, system and medium based on neural network | |
| CN106484913A (en) | Method and server that a kind of Target Photo determines | |
| CN111988327B (en) | Threat behavior detection and model establishment method and device, electronic equipment and storage medium | |
| CN116739408A (en) | Power grid dispatching safety monitoring method and system based on data tag and electronic equipment | |
| CN111083705A (en) | Group-sending fraud short message detection method, device, server and storage medium | |
| CN116159310A (en) | Data processing method, device, electronic equipment and storage medium | |
| CN112651026B (en) | Application version mining method and device with service safety problem | |
| CN111581533B (en) | Method and device for identifying state of target object, electronic equipment and storage medium | |
| CN114491528A (en) | Malicious software detection method, device and equipment | |
| CN113326688A (en) | Ideological and political theory word duplication checking processing method and device | |
| CN113688346A (en) | Illegal website identification method, device, equipment and storage medium | |
| CN113220947A (en) | Method and device for encoding event characteristics | |
| CN112333708A (en) | Telecommunication fraud detection method and system based on bidirectional gating circulation unit | |
| CN111523034A (en) | Application processing method, device, equipment and medium | |
| CN113065348B (en) | Internet negative information monitoring method based on Bert model |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20160907 |
|
| RJ01 | Rejection of invention patent application after publication |