Invention content
The invention is intended to provide a kind of enterprises system data exchange monitoring method, can make enterprise to operation system it
Between the data transmitted supervised and controlled.
In order to solve the above-mentioned technical problem, this patent provides following basic technology scheme:
Enterprises system data exchange monitoring method, including the following contents:
Data acquisition step, operation system ask summed data by data interface module to service module transmission;
Service processing step, service module receive business and ask summed data, according to the request of operation system by data into
Row handles and is sent to corresponding operation system;
Data monitoring step, the data monitoring step be used for data acquisition step and service processing step data into
Row monitoring and record, data monitoring step specifically include the following contents:
Error detection step, error detection submodule detection data obtaining step, service processing step and data monitoring
The mistake occurred in step;
Data statistics step, data statistics submodule count information resources in the data transmission procedure of each operation system
Service condition;
Logging step, log recording submodule is according to data acquisition step, service processing step, error detection step
Daily record data is generated with the data in data statistics step.
In technical solution of the present invention, by data interface module by enterprises system data exchange monitoring method with it is each
Operation system establishes data connection, and each operation system can send request or number by data interface module to service module
According to by data interface module to each operation system transmission data, error detection submodule can detect service module in real time
Mistake in platform operational process, such as data formatting error, operation system mistake, network connection error, data transmission fault,
The daily record data that log recording submodule can will generate during service module processing transmission data, such as transmission time pass
Defeated content, data exchange amount, error message etc., are recorded, and administrative staff is facilitated to be checked in the later stage;Data statistics submodule
The service condition of each information resources in the data exchange process between each operation system and platform is counted, such as a data
Visit capacity, access frequency, the data exchange amount of operation system, the request amount of an operation system etc., facilitate administrator's root
Each operation system is allocated according to the service condition of information resources, such as improves the load energy of the big operation system of visit capacity
Power controls data exchange amount of operation system etc..
Further, alarm step is further included, the alarm step specifically includes:
Step 1:Alarm submodule obtains errorlevel and quantity;
Step 2:Alarm submodule goes out alert level according to errorlevel and quantity according to preset weight calculation;
Step 3:Alarm submodule sends alarm according to alert level to administrative staff corresponding with alert level.
Alarm submodule can send alarm when detecting mistake to administrative staff, can remind administrator at the first time
Member is handled, and is sent alarm to corresponding administrative staff according to the number of the rank of mistake, can more accurately be notified each
The corresponding administrative staff of a errorlevel specify the Limitation on Liability of administrative staff.
Further, data are further included and keep in step, data temporary storage module is detecting that service module sends data to industry
When mistake occurring in systematic procedure of being engaged in, data to be sent are stored.
Data to be sent can be stored by data temporary storage module, with latency services system it is normal after again into
Row is sent.Avoid loss of data.
Further, the daily record data is including in requested service system, target service system, time, request content, data
Appearance and error logging.
According to requested service system, target service system, time, the flow direction of data is conveniently traced, according to data content, is asked
It seeks content and error logging can facilitate mistake is investigated and repaired.
Further, the service processing step includes:
Whether just data format verification step, data format verification submodule judge the data format of operation system transmission
Really, it is continued with if correct, stop the processing to the data if incorrect and generates format error signal.
It whether correct detects operation system transmission data form, filters out wrong data, wrong data is avoided to be sent to it
His operation system.
Further, the data format verification step specifically includes:
Step 1:Data format verification submodule obtains the data type and data name of data to be tested;
Step 2:Data format verification submodule searches the normal data lattice of the data according to data type and data name
Formula terminates data format verification step if not finding, and is performed if the data format for having found standard next
Step;
Step 3:The number of data is compared data format verification submodule with the data format of standard, and judgement is
No identical, if so, judgement form is correct, verification passes through, if it is not, then judging format error, verification does not pass through, data format school
Test the service processing step of the submodule block end data.
Specific embodiment
Below by specific embodiment, the present invention is described in further detail:
As shown in Figure 1, the present embodiment enterprises system data exchange monitoring method has used following data monitoring system
System, the data monitoring system include data interface module, service module, data monitoring module and data temporary storage module, wherein:
Data interface module asks summed data for each operation system to service module transmission, and data interface module is also used
In service module to each operation system transmission data;
Service module is used to handle data according to the request of operation system and be sent to corresponding operation system;Clothes
Module of being engaged in includes data format and verifies submodule, and format check submodule is used to detect and verify the data transmitted by operation system
Whether form is correct, and data format verification submodule includes format match unit and format check unit, and format match unit is used
In the data format according to data type and data the name acquiring data standard, format check unit is used for according to the data mark
Accurate data format verifies whether the form of the data is correct.
Format match unit is according to the form of data type and data name acquiring data standard and then can be to data
Form is verified.
Data monitoring module, data monitoring module include error detection submodule, log recording submodule, alarm submodule
With data statistics submodule, error detection submodule is used for the mistake during detection service module process data.
Log recording submodule is used to record the daily record data of the generation in service module processing data procedures, daily record data
Including requested service system, target service system, time, request content, data content and error logging.According to requested service
System, target service system, time conveniently trace the flow direction of data, can according to data content, request content and error logging
Mistake is investigated and is repaired to facilitate.
Data statistics submodule is used for the service condition of information resources in the data transmission procedure for counting each operation system,
Data statistics submodule includes handling duration statistic unit, data volume statistic unit, access frequency statistic unit, visit capacity statistics
Unit, handling duration statistic unit are same as the total duration that statistical fractals resume module each operation system is spent, data volume
Statistic unit for counting the total amount of data that each operation system is handled and transmitted by service module, unite by access frequency
Meter unit is used to count the access frequency of each operation system and data, and visit capacity statistic unit is used to count each business
The visit capacity of system and data.
Spend total duration, access frequency, visit capacity can by the total amount of data of statistical service system and data, data processing
It is administrative staff in the adjustment for carrying out operation system and data to judge the liveness of each operation system and significance level
It provides data to support, in order to which the resource of business to business be reasonably configured and manage.
The wrong grade of alarm submodule memory storage and corresponding administrative staff, alarm submodule are used in error detection
Alarm is sent to corresponding administrative staff according to errorlevel and number when module detects mistake.Pipe can be reminded at the first time
Reason personnel are handled, and are sent alarm to corresponding administrative staff according to the number of the rank of mistake, can more accurately be led to
Know the corresponding administrative staff of each errorlevel, specify the Limitation on Liability of administrative staff.
The platform further includes data cleansing module, and data cleansing module is used for the data content sent to each operation system
It is detected, matches and cleans, data cleansing module includes:
Master data management module examines submodule including primary data store submodule and master data creates submodule, main number
Master data list and data cleaning rule list are stored with according to sub-module stored, master data and cleaning rule in master data list
Cleaning rule in list corresponds, and master data creates submodule for creating master data and data corresponding with master data
Cleaning rule is simultaneously respectively stored into master data list and cleaning rule list;
Data operation modules, data operation modules are used to send out each operation system according to the data cleansing rule of master data
The data sent are matched and carry out cleaning replacement to matched data.
Many times, same data can have a different form of presentation, between branch company particularly from different places,
Due to the difference of region, different calls is might have for same content, even same company, in different development
Period may also can change to the cognition of same content, often will appear number in each operation system of a company
According to skimble-scamble problem, for example, for the name of a client company, exist referred to as in operation system and there are full name;
It for the title of a product, with the expansion of product function, needs to be modified, causes product presence in operation system more
A title etc.;
It is stored by master data management module and manages master data list, each master data includes in master data list
Corresponding data cleansing rule, the number that master data of the data operation modules in master data list sends each operation system
According to being cleaned, each data for meeting data cleansing rule is replaced, thus service module is finally returned to each operation system
The critical data returned all is the master data of standard, and so as to solve, the existence form of same data in enterprise operation system is inconsistent to be made
The problem of into data corruption.
Master data creates submodule and includes master data input unit, master data searching unit, Keywords matching unit sum number
According to cleaning rule generation unit.
Master data input unit is used to input the master data to be established for user, and master data searching unit is used to search and test
It demonstrate,proves whether master data input by user has existed in master data list, avoids repeating to establish master data.
Keywords matching unit can obtain the data content transmitted between each operation system from daily record data automatically
The data similar to master data input by user are matched in record to select for user as cleaning keyword, and user is made to recognize respectively
The present existing data similar to master data in the data of a operation system transmission;
Data cleansing rule generating unit can generate number according to master data input by user and the cleaning keyword of selection
According to cleaning rule.Data cleansing rule includes cleaning keyword and keyword is associated with matching, and matching association keyword is used in number
Data are matched together with cleaning keyword according to during cleaning, what data cleansing rule generating unit can be selected according to user
It cleans keyword extraction matching association keyword to select for user, such as Business Name, user has selected to contain company name
After the cleaning keyword of title, other the shared associations for the keyword that data cleansing rule generating unit is selected automatically according to user
Associated data is grouped by data, such as CompanyAddress according to similarity, is selected for user, and user is selected as matching
It is associated with keyword.
Data operation modules include data and replace submodule and backup and reduction submodule, and data replace submodule and are used for basis
Data in the data that the data cleansing rule of master data transmits each operation system by data read-write module match,
Specifically, it selects all matched data, then again with matching association key first to clean keyword as matching keyword
Word screens all matched data, the data for being associated with keyword with matching and matching is selected, then according to data cleansing
Rule carries out cleaning replacement to matched data.It is carried out by the data that data operation modules transmit each operation system clear
It washes, replaces each data for meeting data cleansing rule, so as to solve same data in the data of enterprise operation system transmission
Existence form inconsistent the problem of causing data corruption.
Matching association keyword can to avoid same content data by mistake replacement, for example, operation system transmission number
According to it is middle there are two data be " X companies ", but what it referred to is different companies, has different addresses, management functions, and
One entitled " Chongqing X science and technology limited Companies " is established by this system to clean the data of operation system, manage afterwards
" X companies " as cleaning keyword, is just needed administrative staff's selection and confirms that matching is associated with key accordingly by reason personnel at this time
Word is such as associated with keyword using CompanyAddress as matching, so as to enable a system to correctly distinguish two " X companies ", avoids accidentally grasping
Make.
Daily record data further includes master data cleaning record, which further includes data cleansing reminding module, and data cleansing carries
Awake module can be obtained according to data content of the cleaning record to transmit in daily record data corresponding to cleaned data
Classification, and send prompting message, such as data content is related to contract to the responsible person of corresponding classification, then notify Finance Department
And business department, data are related to product, then notify production division and sales department, so as to make master data is quickly expanded should
With being also convenient for after master data accidentally changed, related personnel is traced.
This method includes the following contents:
Data acquisition step, operation system ask summed data by data interface module to service module transmission;
Service processing step, service module receive business and ask summed data, according to the request of operation system by data into
Row handles and is sent to corresponding operation system;
Data monitoring step, data monitoring step are used to supervise the data of data acquisition step and service processing step
Control and record, data monitoring step specifically include the following contents:
Error detection step, error detection submodule detection data obtaining step, service processing step and data monitoring
The mistake occurred in step;
Data statistics step, data statistics submodule count information resources in the data transmission procedure of each operation system
Service condition;
Logging step, log recording submodule is according to data acquisition step, service processing step, error detection step
Daily record data is generated with the data in data statistics step.
Alarm step is further included, alarm step specifically includes:
Step 1:Alarm submodule obtains errorlevel and quantity;
Step 2:Alarm submodule goes out alert level according to errorlevel and quantity according to preset weight calculation;
Step 3:Alarm submodule sends alarm according to alert level to administrative staff corresponding with alert level.
It further includes data and keeps in step, data temporary storage module is detecting that service module sends data to operation system mistake
When mistake occurring in journey, data to be sent are stored.
Service processing step includes:
Whether just data format verification step, data format verification submodule judge the data format of operation system transmission
Really, it is continued with if correct, stop the processing to the data if incorrect and generates format error signal.
Data format verification step specifically includes:
Step 1:Data format verification submodule obtains the data type and data name of data to be tested;
Step 2:Data format verification submodule searches the normal data lattice of the data according to data type and data name
Formula terminates data format verification step if not finding, and is performed if the data format for having found standard next
Step;
Step 3:The number of data is compared data format verification submodule with the data format of standard, and judgement is
No identical, if so, judgement form is correct, verification passes through, if it is not, then judging format error, verification does not pass through, data format school
Test the service processing step of the submodule block end data.
Data cleansing step is further included, the data cleansing step is used for that the data that operation system inputs cleaned
Filter, data cleansing step include:
Master data creates step, and user creates submodule by master data and establishes new master data, and by primary data store
Into the master data list of primary data store submodule, master data includes data cleansing rule;
Data cleansing step, data operation modules read the data that each operation system is sent, and the number that will be read
It is matched according to the master data in master data list, the data that will match to are regular according to the data cleansing of corresponding master data
It is replaced.
Master data creates step and includes master data checking procedure and master data generation step, and master data checking procedure is specifically wrapped
It includes:
Step 1:Master data creates submodule and obtains master data keyword input by user;
Step 2:Master data creates submodule and reads the master data list in the block of primary data store submodule, and user is inputted
Master data keyword matched with the master data in master data list;
Step 3:Master data create submodule judge whether it is identical in master data keyword input by user
Master data, if it is, terminating the foundation of this master data, if it is not, then performing next step;
Step 4:It is more than preset value that master data, which creates submodule and filters out with master data keyword similarity input by user,
Master data;
Step 5:Master data creates submodule and performs step 6 to step 8 to the master data that each is screened:
Step 6:Master data create submodule obtain master data that master data keyword input by user transmits with screening it
Between difference;
Step 7:Master data creates submodule and obtains the corresponding score value of each difference, calculates the total score of difference;
Step 8:Master data creates submodule and judges the total score of difference whether in preset disparity range, if
It is, then by the master data labeled as suspicion data, if it is not, then skipping the master data;
Step 9:Master data creates submodule and filters out all labels checks really for the master data of data for user
Recognize.
Master data generation step specifically includes:
Step 1:Master data creates submodule and is obtained from daily record data in the data transmitted between each operation system
Hold, master data keyword input by user is matched with the data of each operation system;
Step 2:It is more than preset value that master data, which creates submodule and filters out with master data keyword similarity input by user,
Data as cleaning keyword for user selection;
Step 3:Master data creates cleaning keyword that submodule is selected according to user from the content that operation system is transmitted
The data larger with cleaning keyword relevance are filtered out, and data are sorted out according to similarity, generation association matching is closed
Keyword is selected for user;
Step 4:Master data create submodule according to user select according to master data input by user and selection cleaning
Keyword is associated with keyword generation data cleansing rule with matching.
Above is only the embodiment of the present invention, and the common sense such as well known concrete structure and characteristic are not made excessively herein in scheme
Description, all common of technical field that the present invention belongs to before one skilled in the art know the applying date or priority date
Technological know-how can know the prior art all in the field, and with using routine experiment means before the date
Ability, one skilled in the art with reference to self-ability can improve under the enlightenment that the application provides and implement we
Case, some typical known features or known method should not become the barrier that one skilled in the art implement the application
Hinder.It should be pointed out that for those skilled in the art, without departing from the structure of the invention, if can also make
Dry modification and improvement, these should also be considered as protection scope of the present invention, these all do not interfere with the effect that the present invention implemented and
Practical applicability.The scope of protection required by this application should be based on the content of the claims, the specific reality in specification
Apply the content that the records such as mode can be used for explaining claim.