CN106156886A - A kind of method and system based on business system Supplementing Data rule application flow - Google Patents
A kind of method and system based on business system Supplementing Data rule application flow Download PDFInfo
- Publication number
- CN106156886A CN106156886A CN201610509098.2A CN201610509098A CN106156886A CN 106156886 A CN106156886 A CN 106156886A CN 201610509098 A CN201610509098 A CN 201610509098A CN 106156886 A CN106156886 A CN 106156886A
- Authority
- CN
- China
- Prior art keywords
- data
- carrier data
- unit
- carrier
- completion
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q10/00—Administration; Management
- G06Q10/04—Forecasting or optimisation specially adapted for administrative or management purposes, e.g. linear programming or "cutting stock problem"
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/25—Integrating or interfacing systems involving database management systems
- G06F16/254—Extract, transform and load [ETL] procedures, e.g. ETL data flows in data warehouses
Landscapes
- Engineering & Computer Science (AREA)
- Business, Economics & Management (AREA)
- Theoretical Computer Science (AREA)
- Human Resources & Organizations (AREA)
- Databases & Information Systems (AREA)
- Economics (AREA)
- Strategic Management (AREA)
- General Physics & Mathematics (AREA)
- Physics & Mathematics (AREA)
- Operations Research (AREA)
- Quality & Reliability (AREA)
- Tourism & Hospitality (AREA)
- Marketing (AREA)
- General Business, Economics & Management (AREA)
- Entrepreneurship & Innovation (AREA)
- Development Economics (AREA)
- Game Theory and Decision Science (AREA)
- Data Mining & Analysis (AREA)
- General Engineering & Computer Science (AREA)
- Management, Administration, Business Operations System, And Electronic Commerce (AREA)
Abstract
The method and system based on business system Supplementing Data rule application flow of the present invention, complexity and particularity for business system data, the process application flow of whole carrier data is improved targetedly and optimized, have employed the strategy of preposition calculating, have employed confidence level calculating simultaneously and path analysis technology embeds whole flow process, single table storage+distributed storage+locally stored storage scheme and the data analysis based on single table storage, not only increase the calculated performance of whole flow process, save substantial amounts of time cost, the condition decreasing policy development limits；Also substantially increase the safety and reliability of carrier data simultaneously.
Description
Technical field
The present invention relates to big technical field of data processing, particularly relating to one should based on business system Supplementing Data rule
By the method and system of flow process.
Background technology
People mention more and more for the concept of big data in recent years, big data (big data), or title flood tide data,
Magnanimity data, refers to carry out, with conventional software instrument, the data that catch, manage and process in the time range that can bear
Set.Big data processing technique, covers data analysis and excavates, and visualization presents and applies, analyses and prediction of data etc..
And carrier data process, not only there is the feature of mass data and simultaneously but also there is oneself data structure and data are special
Point.Carrier data, refers to the visit between the information of relevant equipment itself that Provider Equipment produces and equipment and equipment
Ask the data of information.The data form of operator's initial acquisition and information format all have the special nature of oneself.At data acquisition
Collection and data protocol reduction phase, business system data use syslog and snmp protocol to be acquired and data protocol mostly
Reduction, but, gather and protocol assembly to carrier data and traditional data make a marked difference part.Such as, pass
The syslog data of system include two territories of TIMESTAMP and HOSTNAME at title division, and can only comprise in HOSTNAME territory
One host address or an IP, carrier data not only contains host address, also can comprise multiple IP address simultaneously,
Carrier data is mainly reflected in the information of the aspects such as equipment, assets, authority, account number.
Existing technical scheme, after data acquisition and protocol assembly (TCP or UDP), data-storing
Module carries out a point table association storage to the data collecting, or carries out data directory storage.After storage finishes, data model
Processing module is associated on corresponding data model according to different list structures and table and processes accordingly.In mass data
Under background, the drawback that data are operated after finishing by storage again is very big, time-consumingly extremely grows extremely inefficient.
Secondly, in memory phase, prior art, use multilist storage, for the storage scheme of operator's data,
Generally refer to various information in carrier data to be stored separately, such as the assets information of carrier data, principal and subordinate's account, user
The unpack such as information, security incident is stored in order to reduce the memory space taking as far as possible, preferably realizes data pair
The service logic answered.But, the Main Function of carrier data is to be analyzed locating to data according to the strategy formulated and rule
Reason the result finally drawing needs, the time that the resolving of low volume data needs is negligible, but operation quotient
According to an important feature be exactly that data volume is huge, multilist storage needs the time consuming flood tide to go to be associated inquiry, configuration
The also existence because of multilist of the rule resolving and strategy makes its difficulty configuring and complex operation degree improve several order of magnitude, its
The drawback of efficiency and the too fat to move parsing can't bear will be more serious slowly.Formulate need according to the data processing model that point table is carried out
The data relationship of multilist to be considered and the association of data source, restrictive condition becomes with associating the quantity locking the rule making formulation
Huge, the content of rule also can become tediously long, and efficiency and the flexibility of whole flow chart of data processing are greatly diminished.
In addition, the epoch becoming more and more important in this information security, carrier data is to the requirement of secure context also increasingly
Height, the safety and reliability of data has to obtain the guarantee of necessity, and existing technology is in carrier data handling process
In necessary guarantee is lacked to the safety and reliability of data.
The purpose of the present invention is exactly that the flow process processing above-mentioned carrier data is optimized, to improve at carrier data
The flexibility of reason, freedom, safety and reliability.
Content of the invention
It is an object of the invention to be achieved through the following technical solutions.
According to the embodiment of the present invention, a kind of side based on business system Supplementing Data rule application flow is proposed
Method, described method comprises the steps:
S1, the formulation carrying out carrier data parsing strategy；
S2, obtain carrier data in real time by data source port；
S3, real-time preposition calculating is carried out to the described carrier data being in during reception；
S4, the carrier data performing real-time preposition calculating is carried out multivariant mixing storage；
S5, the formulation being stored in row Stakeout & Homicide Preservation Strategy based on multivariant mixing；
S6, the carrier data to storage carry out statistical analysis, generate forecast model and visualization result, and return to fortune
The commercial family of battalion uses.
According to the preferred embodiment of the present invention, described step S3 the carrier data being in during reception is carried out
Real-time preposition calculating includes:
S3-1, described acquisition carrier data access when start normalized process；
S3-2, to after standardization carrier data perform completion process；
S3-3, the carrier data execution merger to after completion process, filtration, Screening Treatment.
According to the embodiment of the present invention, after data carry out the completion process described in step S3-2, data are performed:
S3a, confidence level matching primitives, comprising:
Sensitive data information in S3a-1, extracted data；
S3a-2, simultaneously extract comprise in preset Provider Equipment object repository with described sensitive data letter
Cease the standard information of corresponding data；
S3a-3, these key data information by extraction in data are entered one by one with the described corresponding informance in knowledge base
Row comparison.
According to the embodiment of the present invention, when the preposition calculating of step S3, perform: S3b, path mark process.
According to the embodiment of the present invention, step S4 the carrier data performing real-time preposition calculating is carried out in many ways
The mixing storage of case specifically includes:
S4-1, Standardization Strategies, completion strategy and the merger screening and filtering formulated in the Data Analysis policy development stage
Strategy carries out single table storage of relevant database；
S4-2, the actual data after preposition calculating is processed carry out distributed storage and locally stored, follow-up statistics
During analysis, the principle of data acquisition is nearby principle and local first principle.
According to the embodiment of the present invention, the carrier data to storage of described step S6 carries out statistical analysis, generates
Forecast model and visualization result, and return to provider customer use specifically include: the analysis of the confidence level of receiving step S3a
Result and the extraction operation carrying out path mark according to step S3b, carry out path analysis according to path mark, enter to user
Row is shown.
According to the embodiment of the present invention, it is also proposed that perform above-mentioned based on business system Supplementing Data rule application flow
The system of method, described system includes: resolve policy development unit, data access unit, preposition computing unit, mixing storage
Unit, analysis strategy formulation unit and statistical analysis unit, wherein,
Described parsing policy development unit, formulates the strategy of carrier data standardization, completion and merger filtering screening；
Described data access unit, obtains carrier data in real time by data source port；
Described preposition computing unit carries out real-time preposition calculating to the carrier data being in during reception；
Described hybrid memory cell carries out multivariant mixing storage to the carrier data performing real-time preposition calculating；
Described analysis strategy formulates the formulation that unit is stored in row Stakeout & Homicide Preservation Strategy based on multivariant mixing；
Described statistical analysis unit carries out statistical analysis to carrier data, generates forecast model and visualization result, and
Return to provider customer to use.
According to the embodiment of the present invention, described preposition computing unit includes:
Standardization unit, starts normalized process when carrier data accesses；
Completion processing unit, performs completion process to the carrier data after standardization；And
Merger screening unit, the carrier data after processing completion performs merger, filtration, Screening Treatment.
According to the embodiment of the present invention, described system also includes confidence level matching primitives unit, after processing completion
Data perform confidence level matching primitives.
According to the embodiment of the present invention, described system also includes path mark processing unit, respectively to standardization,
Data execution path mark process after completion process and merger Screening Treatment.
It is an advantage of the current invention that: for complexity and the particularity of business system data, to whole carrier data
Process application flow improve targetedly and optimize, have employed the strategy of preposition calculating, simultaneously Promethean have employed
Confidence level calculates and path analysis technology embeds whole flow process, the storage scheme of single table storage+distributed storage+locally stored with
And the data analysis based on single table storage, not only increase the calculated performance of whole flow process, save substantial amounts of time cost, subtract
The condition having lacked policy development limits；Also substantially increase the safety and reliability of carrier data simultaneously.
Brief description
By reading the detailed description of hereafter preferred embodiment, various other advantage and benefit common for this area
Technical staff will be clear from understanding.Accompanying drawing is only used for illustrating the purpose of preferred embodiment, and is not considered as to the present invention
Restriction.And it in whole accompanying drawing, is denoted by the same reference numerals identical parts.In the accompanying drawings:
Fig. 1 shows the method based on business system Supplementing Data rule application flow according to embodiment of the present invention
Flow chart；
Fig. 2 shows the system based on business system Supplementing Data rule application flow according to embodiment of the present invention
Structured flowchart.
Detailed description of the invention
It is more fully described the illustrative embodiments of the disclosure below with reference to accompanying drawings.Although accompanying drawing shows these public affairs
The illustrative embodiments opened, it being understood, however, that may be realized in various forms the disclosure and the reality that should not illustrated here
The mode of executing is limited.On the contrary, these embodiments are provided to be able to be best understood from the disclosure, and can be by these public affairs
What the scope opened was complete conveys to those skilled in the art.
According to the embodiment of the present invention, a kind of side based on business system Supplementing Data rule application flow is proposed
Method, as it is shown in figure 1, described method comprises the steps:
S1, the formulation carrying out carrier data parsing strategy；Specifically include:
S1-1, the Standardization Strategies formulating data, such as a certain field of data starts to terminate this section to a certain field
Content be source IP, be initially the description of contents of event from certain one piece of data, define the process etc. of each field；
S1-2, formulate completion strategy, for example, judge that whether the content of a certain field in data is sky, if if sky then
Demand according to carrier data and knowledge base carry out the completion of field contents.Except the completion to some necessary informations, it is possible to
Completion demand with self-defining some personalizations of increase；
S1-3, formulation merger filtering screening strategy, by arranging various judgment rule and policy condition, in data stream
The data meeting resolve, and carry out to data filtering in accordance with regular combined screening.
According to this resolve strategy can high-freedom degree, self-defining daily record is standardized, completion, merger, mistake
Filter operation.
S2, obtain carrier data in real time by data source port.
S3, real-time preposition calculating is carried out to the carrier data being in during reception.Including:
S3-1, start normalized process when carrier data accesses, specifically include: according to the standardization arranging
The data of input are resolved by strategy, insignificant data are split, mark into univocal field, such as parses
The source IP of data, Target IP, forwarding IP, the time of event generation, event content etc..
S3-2, the completion process to carrier data for the execution；After being standardized processing, according to the benefit having made
Full strategy continues executing with and carries out completion operation to data, and whether the field that contrast has parsed lacks the field of some necessity
And carry out the completion of these fields according to the existing knowledge base of operator, including assets, operation system, resource group, security domain, factory
Business, account number, authority, mandate etc. operator must field.
S3-3, perform to the merger of carrier data, filtration, Screening Treatment；While completion, according to existing filtration
The further process to data of merger screening strategy, according to the policy condition setting, within the time window determining, to
Field through resolving simultaneously completion merges, filters, screens.Through the step for after, carrier data i.e. formally becomes
Type.
The processing procedure of step S3 is all to operate during data access, is i.e. before storing just
Completing the process to carrier data, using this mode of operation, write-once database avoids the need for follow-up write behaviour
Make, read operation is only had to data later, greatly reduce the number of operations to database, improve the efficiency processing data.
According to the embodiment of the present invention, in order to improve the safety and stability of data, when data carry out step S3-2
After described completion is processed, identifiability has reached levels of the standard substantially, performs data:
S3a, confidence level matching primitives；Concrete implementation method includes:
Sensitive prime data message in S3a-1, extracted data, the title of the device object of such as equipment, equipment self
IP, the IP of affiliated security domain, affiliated resource group IP；
S3a-2, simultaneously extract comprise in preset Provider Equipment object repository with described sensitive data letter
Cease the standard information of corresponding data；
S3a-3, these key data information by extraction in data are entered one by one with the described corresponding informance in knowledge base
Row comparison, overall matching information is α, and the match is successful, and bar number is β, then confidence levels γ is:
It when γ value is more than 7, is high confidence level, is typically credible between 4 to 6, be then insincere below 4, build
View is abandoned.The analysis result of confidence level will carry out the analysis to user and represents in step s 6.
And, according to the embodiment of the present invention, when the preposition calculating of step S3, perform:
S3b, path mark process；The whole process processing on stream due to data, be for a user one black
Case pattern, according to the path analysis of this programme, by final data content exposure, user can observe that this data are passed through
Which processes step, makes whole data flow transparence, allow user data are had one cognitive very intuitively, be greatly improved
The reliability of data and the transparency, also further add the security of data, and concrete implementation method is as follows:
S3b-1, when carrying out the data initial normalisation of step S3-1, carry out path mark for the first time to data, newly-built one
The individual field being specifically used to as ID of trace route path, fills into identified value；
S3b-2, during the completion of step S3-2, the field of completion is carried out second time mark；
S3b-3, the filtration of step S3-3 and merging phase, if a certain data has carried out the operation of filtration and merger,
Carry out third time path mark to this data.
Final in step S6, carry out the extraction operation of path mark, carry out path analysis according to path mark, enter to user
Row is shown.
Owing to adding confidence level matching primitives and path mark process in whole flow process, to each carrier data
Carry out track of the whole process mark and carry out contrasting safely coupling during analyzing, through calculating confidence level, passing through road
Footpath mark obtains the reduction to data path, selects data for user, greatly improve data security and
Stability.
S4, the carrier data performing real-time preposition calculating is carried out multivariant mixing storage.Specifically include:
S4-1, the Standardization Strategies formulated in the Data Analysis policy development stage, completion strategy, merger screening and filtering plan
Slightly these data carry out single table storage of relevant database.Although the strategy type for the formulation of carrier data is different,
But tactful corresponding field similarity is very high, single table storage can meet required various strategies completely.
S4-2, the actual data after preposition calculating is processed carry out distributed storage+locally stored, and data are once located
Reason finishes, and will be replicated and be distributed among different storage points, and the rule of storage is according to date, data type, source IP, mesh
The classification of IP etc. concrete similar index data are distributed in multiple stage disk.Number during follow-up statistical analysis
It is nearby principle and local first principle according to the principle using, thus while distributed storage, reached locally stored mesh
's.
From the aspect of memory space takies, the shared space of single table storage is that the space taking than multilist storage is wanted
Greatly, prior art is mainly stored by structured database, and data source is different, so being processed by single, and can be with logarithm
According to storing, and the cost of storage can be reduced by association, but under the background of operator's big data quantity, according to original storage
Mode, then, when calculating complicated rule and policy model, the consumption of the calculating resource of multilist association can be arrived greatly and cannot be undertaken calculating
Cost.Use single table storage, the time cost of saving and the carrying cost calculating resources costs increase to be far longer than, so
It is the cost being exchanged for substantial amounts of calculating by sacrificing a small amount of carrying cost in this way, calculating resource is saved
Low.
Meanwhile, using distributed memory and locally stored, distributed storage can improve the dilatation ability of storage greatly,
The ability of storage management data can reach TB level even PB level, the similar index of the area principle using during distributed storage, with
When the characteristic of data distribution and the ability of parallel data processing make the efficiency of process data be further enhanced, when an equipment
Do not interfere with being normally carried out of whole flow chart of data processing in the case of damaging or being in paralysis, also improve the safety of data yet
Property.
S5, the formulation being stored in row Stakeout & Homicide Preservation Strategy based on multivariant mixing.
According to Stakeout & Homicide Preservation Strategy, can to the stored carrier data that finishes of pre-treatment carry out statistical
Analysis, generates forecast model and visualization result.
In preposition calculation stages, have been carried out resolving strategy, i.e. the execution of filtering screening merger strategy, this step only needs
Perform query analysis class strategy；Carry out multivariant mixing memory phase, index principle and nearby according to distributed storage class
Principle and local first principle, the speed of data query is exceedingly fast, and the rule under simultaneously single table schema and strategy are without the concern for many
Table relation and the association of data source, a rule only needs a data base querying to realize, the Policy model training time shortens
(training method can use in prior art ripe learning training algorithm), wall scroll rule and strategy can corresponding several even
Hundreds of data originating tasks of dozens of, adaptability is extremely strong.Simultaneously locally stored can realize looking into of participle and not participle flexibly
Inquiry, can reach to obtain result in the cutting inquiry of any dimension, can quickly lay down a regulation very flexibly and strategy with this
Model, makes calculated performance be greatly improved.And can more also avoid in the formulation of rule and strategy
Loaded down with trivial details configuration, thus it is high to eliminate prior art time cost, the very poor shortcoming of flexibility；By the regular mould optimizing
Type and Policy model, the efficiency carrying out statistical analysis also can be improved accordingly.
S6, the carrier data to storage carry out statistical analysis, generate forecast model and visualization result, and return to fortune
The commercial family of battalion uses.
Described statistical analysis includes the analysis result of the confidence level of receiving step S3a and enters walking along the street according to step S3b
The extraction operation of footpath mark, carries out path analysis according to path mark, is shown to user.
According to another embodiment of the present invention, it is also proposed that perform above-mentioned based on the rule application of business system Supplementing Data
The system of the method for flow process, as in figure 2 it is shown, described system includes: resolve policy development unit, data access unit, preposition meter
Calculation unit, hybrid memory cell, analysis strategy formulation unit and statistical analysis unit, wherein,
Described parsing policy development unit, formulates the strategy of carrier data standardization, completion and merger filtering screening；
Described data access unit, obtains carrier data in real time by data source port；
Described preposition computing unit carries out real-time preposition calculating to the carrier data being in during reception；Before described
Put computing unit to include:
Standardization unit, starts normalized process, specifically includes when carrier data accesses: according to setting
The data of input are resolved by the Standardization Strategies put, and insignificant data are split, mark into univocal field,
Such as parse the source IP of data, time that Target IP, forwarding IP, event occur, event content etc..
Completion processing unit, performs the completion process to carrier data；After being standardized processing, according to having made
The completion strategy reserved continues executing with and carries out completion operation to data, and whether the field that contrast has parsed lacks some must
The field wanted and the completion carrying out these fields according to the existing knowledge base of operator, including assets, operation system, resource group,
Security domain, manufacturer, account number, authority, mandate etc. operator must field.
Merger screening unit, performs the merger to carrier data, filtration, Screening Treatment；While completion, according to
Some filtration merger screening strategy further process to data, according to the policy condition setting, at the time window determining
Within, to having resolved and the field of completion merges, filters, screens.
Described hybrid memory cell carries out multivariant mixing storage to the carrier data performing real-time preposition calculating,
Specifically include:
The Standardization Strategies formulated in the Data Analysis policy development stage, completion strategy, merger screening and filtering strategy this
A little data carry out single table storage of relevant database.
Actual data after preposition calculating is processed carries out distributed storage+locally stored.
Described analysis strategy formulates the formulation that unit is stored in row Stakeout & Homicide Preservation Strategy based on multivariant mixing.
Described statistical analysis unit carries out statistical analysis to carrier data, generates forecast model and visualization result, and
Return to provider customer to use.
According to the embodiment of the present invention, in order to improve the safety and stability of data, described system also includes credible
Degree matching primitives unit, described confidence level matching primitives unit is for after data carry out completion process, and identifiability is substantially
Through reaching levels of the standard, confidence level matching primitives is performed to data；Described confidence level matching primitives unit specifically includes:
Sensitive data extracting unit, the sensitive prime data message in extracted data, the name of the device object of such as equipment
Title, the IP of the IP of equipment self, affiliated security domain, affiliated resource group IP；
Corresponding standard information extraction unit, extract comprise in preset Provider Equipment object repository with described
The standard information of the corresponding data of sensitive data information；
Comparing unit, by these key data information of extraction in data with the described corresponding informance in knowledge base one by one
Comparing, overall matching information is α, and the match is successful, and bar number is β, then confidence levels γ is:
It when γ value is more than 7, is high confidence level, is typically credible between 4 to 6, be then insincere below 4, build
View is abandoned.
According to the embodiment of the present invention, described system also includes path mark processing unit, is used for execution route mark
Process；Described path mark processing unit specifically includes:
Data, when carrying out data initial normalisation, are carried out for the first time path mark by the first mark unit, newly-built one specially
Door is used as the field of ID of trace route path, fills into identified value；
Second mark unit, carries out second time mark during completion is processed to the field of completion；
3rd mark unit, filter and merging phase, if a certain data has carried out the operation of filtration and merger, right
This data carries out third time path mark.
Described statistical analysis unit receives the analysis result of the confidence level of confidence level matching primitives unit and beats from path
Mark processing unit carries out the extraction operation of path mark, carries out path analysis according to path mark, is shown to user.
The above, the only present invention preferably detailed description of the invention, but protection scope of the present invention is not limited thereto,
Any those familiar with the art in the technical scope that the invention discloses, the change that can readily occur in or replacement,
All should cover within protection scope of the present invention.Therefore, protection scope of the present invention answers the described protection model with claim
Enclose and be as the criterion.
Claims (10)
1. the method based on business system Supplementing Data rule application flow, described method comprises the steps:
S1, the formulation carrying out carrier data parsing strategy；
S2, obtain carrier data in real time by data source port；
S3, real-time preposition calculating is carried out to the described carrier data being in during reception；
S4, the carrier data performing real-time preposition calculating is carried out multivariant mixing storage；
S5, the formulation being stored in row Stakeout & Homicide Preservation Strategy based on multivariant mixing；
S6, the carrier data to storage carry out statistical analysis, generate forecast model and visualization result, and return to operator
User uses.
2. a method of claim 1, described step S3 the carrier data being in during reception is carried out
Real-time preposition calculating includes:
S3-1, described acquisition carrier data access when start normalized process；
S3-2, to after standardization carrier data perform completion process；
S3-3, the carrier data execution merger to after completion process, filtration, Screening Treatment.
3. data, after data carry out the completion process described in step S3-2, are held by a method as claimed in claim 2
OK: S3a, confidence level matching primitives, comprising:
Sensitive data information in S3a-1, extracted data；
S3a-2, simultaneously extract that comprise in preset Provider Equipment object repository with described sensitive data information pair
The standard information of the data answered；
S3a-3, by these key data information and the described corresponding informance in knowledge base of extraction in data one by one than
Right.
4. a method as claimed in claim 3, when the preposition calculating of step S3, performs: S3b, at the mark of path
Reason.
5. a method as claimed in claim 4, step S4 the carrier data performing real-time preposition calculating is carried out
Multivariant mixing storage specifically includes:
S4-1, Standardization Strategies, completion strategy and the merger screening and filtering strategy formulated in the Data Analysis policy development stage
Carry out single table storage of relevant database；
S4-2, the actual data after preposition calculating is processed carry out distributed storage and locally stored, follow-up statistical analysis
During the principle of data acquisition be nearby principle and local first principle.
6. a method as claimed in claim 5, described step S6 carry out statistical analysis to carrier data, generate prediction
Model and visualization result, and return to provider customer use specifically include: the analysis result of the confidence level of receiving step S3a
And the extraction operation carrying out path mark according to step S3b, carry out path analysis according to path mark, open up to user
Show.
7. one kind perform the method based on the regular application flow of business system Supplementing Data such as one of claim 1-6 be
System, described system includes: resolve policy development unit, data access unit, preposition computing unit, hybrid memory cell, analysis
Policy development unit and statistical analysis unit, wherein,
Described parsing policy development unit, formulates the strategy of carrier data standardization, completion and merger filtering screening；
Described data access unit, obtains carrier data in real time by data source port；
Described preposition computing unit carries out real-time preposition calculating to the carrier data being in during reception；
Described hybrid memory cell carries out multivariant mixing storage to the carrier data performing real-time preposition calculating；
Described analysis strategy formulates the formulation that unit is stored in row Stakeout & Homicide Preservation Strategy based on multivariant mixing；And
Described statistical analysis unit carries out statistical analysis to carrier data, generates forecast model and visualization result, and returns
Use to provider customer.
8. a system as claimed in claim 7, described preposition computing unit includes:
Standardization unit, starts normalized process when carrier data accesses；
Completion processing unit, performs completion process to the carrier data after standardization；And
Merger screening unit, the carrier data after processing completion performs merger, filtration, Screening Treatment.
9. a system as claimed in claim 8, described system also includes confidence level matching primitives unit, after processing completion
Data perform confidence level matching primitives.
10. a system as claimed in claim 9, described system also includes path mark processing unit, respectively to standardization
Data execution path mark process after process, completion process and merger Screening Treatment.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201610509098.2A CN106156886A (en) | 2016-06-30 | 2016-06-30 | A kind of method and system based on business system Supplementing Data rule application flow |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201610509098.2A CN106156886A (en) | 2016-06-30 | 2016-06-30 | A kind of method and system based on business system Supplementing Data rule application flow |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN106156886A true CN106156886A (en) | 2016-11-23 |
Family
ID=57350713
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201610509098.2A Pending CN106156886A (en) | 2016-06-30 | 2016-06-30 | A kind of method and system based on business system Supplementing Data rule application flow |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN106156886A (en) |
Cited By (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108664497A (en) * | 2017-03-30 | 2018-10-16 | 大有秦鼎（北京）科技有限公司 | The method and apparatus of Data Matching |
| CN110928484A (en) * | 2018-09-19 | 2020-03-27 | 上海仪电（集团）有限公司中央研究院 | Hybrid cloud storage method based on software defined storage |
| CN111078679A (en) * | 2019-12-23 | 2020-04-28 | 用友网络科技股份有限公司 | Data report generation method and device and computer readable storage medium |
| CN113052205A (en) * | 2021-03-05 | 2021-06-29 | 广州默尼互联网信息有限公司 | Lying-in woman data classification method, device, equipment and medium based on machine learning |
| CN113313362A (en) * | 2021-05-12 | 2021-08-27 | 平安科技（深圳）有限公司 | Enterprise ESG index determination method based on data completion and related products |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20130191328A1 (en) * | 2012-01-25 | 2013-07-25 | Sap Ag | Standardized framework for reporting archived legacy system data |
| CN105512201A (en) * | 2015-11-26 | 2016-04-20 | 晶赞广告（上海）有限公司 | Data collection and processing method and device |
| CN105631027A (en) * | 2015-12-30 | 2016-06-01 | 中国农业大学 | Data visualization analysis method and system for enterprise business intelligence |
-
2016
- 2016-06-30 CN CN201610509098.2A patent/CN106156886A/en active Pending
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20130191328A1 (en) * | 2012-01-25 | 2013-07-25 | Sap Ag | Standardized framework for reporting archived legacy system data |
| CN105512201A (en) * | 2015-11-26 | 2016-04-20 | 晶赞广告（上海）有限公司 | Data collection and processing method and device |
| CN105631027A (en) * | 2015-12-30 | 2016-06-01 | 中国农业大学 | Data visualization analysis method and system for enterprise business intelligence |
Cited By (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108664497A (en) * | 2017-03-30 | 2018-10-16 | 大有秦鼎（北京）科技有限公司 | The method and apparatus of Data Matching |
| CN108664497B (en) * | 2017-03-30 | 2020-11-03 | 大有秦鼎（北京）科技有限公司 | Data matching method and device |
| CN110928484A (en) * | 2018-09-19 | 2020-03-27 | 上海仪电（集团）有限公司中央研究院 | Hybrid cloud storage method based on software defined storage |
| CN110928484B (en) * | 2018-09-19 | 2023-12-22 | 上海仪电（集团）有限公司中央研究院 | Hybrid cloud storage method based on software defined storage |
| CN111078679A (en) * | 2019-12-23 | 2020-04-28 | 用友网络科技股份有限公司 | Data report generation method and device and computer readable storage medium |
| CN111078679B (en) * | 2019-12-23 | 2023-06-16 | 用友网络科技股份有限公司 | Method and device for generating data report and computer readable storage medium |
| CN113052205A (en) * | 2021-03-05 | 2021-06-29 | 广州默尼互联网信息有限公司 | Lying-in woman data classification method, device, equipment and medium based on machine learning |
| CN113313362A (en) * | 2021-05-12 | 2021-08-27 | 平安科技（深圳）有限公司 | Enterprise ESG index determination method based on data completion and related products |
| CN113313362B (en) * | 2021-05-12 | 2024-01-16 | 平安科技（深圳）有限公司 | Enterprise ESG index determining method based on data completion and related products |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN106156886A (en) | A kind of method and system based on business system Supplementing Data rule application flow | |
| Holm et al. | Automatic data collection for enterprise architecture models | |
| CN100463461C (en) | Active network safety loophole detector | |
| CN101950271B (en) | Modeling technology-based software security test method | |
| CN102196478B (en) | Diagnosis method and system for failure of network management system | |
| CN106790001A (en) | Multisystem role-security management method and system based on unified interface | |
| CN102932195A (en) | Networking protocol analysis-based business analysis monitoring method and system | |
| CN112328585A (en) | Data processing method and device | |
| CN108429747A (en) | A kind of extensive Web server information collecting method | |
| US9372664B2 (en) | Comparing event data sets | |
| CN111046000A (en) | Government data exchange sharing oriented security supervision metadata organization method | |
| CN113542074A (en) | Method and system for visually managing east-west network traffic of kubernets cluster | |
| CN114547077A (en) | Intelligent processing system and method for basic government affair form data | |
| Välja et al. | Automated architecture modeling for enterprise technology manageme using principles from data fusion: A security analysis case | |
| CN116032581A (en) | Network equipment security management method and electronic equipment | |
| CN108109071A (en) | The monitoring method and electronic equipment dynamically associated based on personnel's social relationships | |
| WO2022031184A1 (en) | System for intelligent risk and vulnerability management for infrastructure elements | |
| CN112363996A (en) | Method, system, and medium for building a physical model of a power grid knowledge graph | |
| CN115296892B (en) | Data information service system | |
| CN109947403B (en) | Decomposition and modeling method of safety target and related equipment | |
| CN116257404A (en) | Log analysis method and computing device | |
| Korzeniowski et al. | Discovering interactions between applications with log analysis | |
| CN107888561A (en) | Towards the security service combined system of civil aircraft | |
| CN113076308B (en) | Space-time big data service system | |
| CN110209885A (en) | A kind of figure querying method and system |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20161123 |
|
| RJ01 | Rejection of invention patent application after publication |