CN104486337A - Data validity verification method and device - Google Patents
Data validity verification method and device Download PDFInfo
- Publication number
- CN104486337A CN104486337A CN201410773933.4A CN201410773933A CN104486337A CN 104486337 A CN104486337 A CN 104486337A CN 201410773933 A CN201410773933 A CN 201410773933A CN 104486337 A CN104486337 A CN 104486337A
- Authority
- CN
- China
- Prior art keywords
- data
- verification
- application scenarios
- project
- verified
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L63/00—Network architectures or network communication protocols for network security
- H04L63/12—Applying verification of the received information
Landscapes
- Engineering & Computer Science (AREA)
- Computer Security & Cryptography (AREA)
- Computer Hardware Design (AREA)
- Computing Systems (AREA)
- General Engineering & Computer Science (AREA)
- Computer Networks & Wireless Communication (AREA)
- Signal Processing (AREA)
- Management, Administration, Business Operations System, And Electronic Commerce (AREA)
Abstract
The invention discloses a data validity verification method and a data validity verification device. The data validity verification method comprises the following steps: acquiring metadata; confirming at least one application scenery of the metadata; confirming configuration parameters corresponding to the application scenery according to the application scenery, wherein the configuration parameters are used for representing testing environment data of the application scenery, and the testing environment data at least comprise an account login parameter of a present application scenery; respectively loading the configuration parameters of any one application scenery to the metadata, and generating at least one data to be verified; acquiring verification items corresponding to any one application scenery; verifying the data to be verified according to the verification items, thereby obtaining a verification result. By the adoption of the data validity verification method and the data validity verification device, the problems that in the prior art, a data validity checking device is poor in stability and high in maintenance cost and risk as a checking method used in the data validity checking device is high in fragmentization, poor in reusability, poor in expandability, high in coupling property and poor in evolution property are solved.
Description
Technical field
The present invention relates to field of information processing, in particular to a kind of Validation of Data method and device.
Background technology
Along with the development of the Internet, each large on-line search service provider attracts third party to carry out application innovation and integration by open platform, such as: an apt word added to clinch the point for Baidu's phoenix nest, alliance of Baidu.com and Qihoo (360 search).They also provide a whole set of online management platform separately simultaneously.If need the client of millions keyword and multi-platform input concerning one, management and optimize their bad dream beyond doubt.Therefore third party's outline management system, integrates main flow on-line search service, for user provides one-stop full platform offline Advertising Management System.Offline advertisement management system is except cannot meeting real-time data analysis and checking demand, and it is all real-time that other additions and deletions change.In order to ensure this off-line system and each platform on-line system data consistency, off-line system inside needs periodically to carry out one-way data synchronous (on-line system data syn-chronization is to off-line system) with each large platform.
In data synchronization process, two key elements are had to need special concern:
First: the validity of data;
Second: the consistency of data;
For above 2 points, obviously can see that data validity is the prerequisite of data consistency, if effectively dirty data cannot be shielded, just cannot ensure data robustness, let alone consistency, all can produce follow-up data analysis and excavation and hinder and impact.Meanwhile, the conditions and environment producing dirty data may have similitude because of the difference of platform, also variant.
Prior art major part is for embody rule scene, and be distributed in business procession by inspection logic, fragmentation is very serious.And, be difficult to like this make inspection logic have durability.System is in evolutionary process, and data validity inspection rectification work is carried out in the mode of patch often on the original basis.While running counter to the open design principle closed, the stability of system is impacted.When system is to a certain extent huge, once find that multi-platform appearance checks leak, then need to go deep into each business processing logic inside and revise, maintenance cost and risk increase severely.
Above scheme has following defect:
1, fragmentation.Maintenance cost is high, influential system stability, and along with System Development, risk is high.
2, poor expandability.Run counter to the open design principle closed.
3, coupling is high.As a service logic part, certain concrete application scenarios of alignment processing.
4, durability is poor.Tackle general inspection logic, durability is poor, is at most the multiplexing of logic.
5, developmental capacity is low.Lack developmental capacity flexibly, lead one and hole whole body, evolution cost is high.
The inspection method fragmentation adopted for data validity testing fixture in prior art is high, durability is poor, poor expandability, coupling are high and developmental capacity is low, the problem that the poor stability of the data validity testing fixture caused, maintenance cost and risk are high, not yet proposes effective solution at present.
Summary of the invention
Main purpose of the present invention is to provide a kind of Validation of Data method and device, to solve in prior art.The inspection method fragmentation that data validity testing fixture adopts is high, durability is poor, poor expandability, coupling are high and developmental capacity is low, the problem that the poor stability of the data validity testing fixture caused, maintenance cost and risk are high.
To achieve these goals, according to an aspect of the embodiment of the present invention, a kind of Validation of Data method is provided.The method comprises: obtain metadata; Determine at least one application scenarios verifying metadata; According to application scenarios, determine the configuration parameter corresponding with application scenarios, wherein, configuration parameter is for characterizing the test environment data of application scenarios, and test environment data at least comprise: the Account Logon parameter of current application scene; Configuration parameter corresponding for any one application scenarios is respectively loaded on metadata, generates at least one data to be verified, wherein, the metadata configuration parameter corresponding with any one application scenarios has incidence relation; Obtain the Verification Project that any one application scenarios is corresponding, wherein, the verification step that Verification Project carries out at application scenarios testing for characterizing data to be verified; Verify data to be verified based on Verification Project, be verified result.
To achieve these goals, according to the another aspect of the embodiment of the present invention, provide a kind of Validation of Data device, this device comprises the first acquisition module, for obtaining metadata; First determination module, for determining at least one application scenarios verifying metadata; Second determination module, for according to application scenarios, determines the configuration parameter corresponding with application scenarios, and wherein, configuration parameter is for characterizing the test environment data of application scenarios, and test environment data at least comprise: the Account Logon parameter of current application scene; First generation module, for configuration parameter corresponding for any one application scenarios is respectively loaded on metadata, generates at least one data to be verified, and wherein, the metadata configuration parameter corresponding with any one application scenarios has incidence relation; Second acquisition module, for obtaining Verification Project corresponding to any one application scenarios, wherein, the verification step that Verification Project carries out at application scenarios testing for characterizing data to be verified; Authentication module, for verifying data to be verified based on Verification Project, is verified result.
According to inventive embodiments, by obtaining metadata; Determine at least one application scenarios verifying metadata; According to application scenarios, determine the configuration parameter corresponding with application scenarios, wherein, configuration parameter is for characterizing the test environment data of application scenarios, and test environment data at least comprise: the Account Logon parameter of current application scene; Configuration parameter corresponding for any one application scenarios is respectively loaded on metadata, generates at least one data to be verified, wherein, the metadata configuration parameter corresponding with any one application scenarios has incidence relation; Obtain the Verification Project that any one application scenarios is corresponding, wherein, the verification step that Verification Project carries out at application scenarios testing for characterizing data to be verified; Data to be verified are verified based on Verification Project, be verified result, solve that the inspection method fragmentation that data validity testing fixture in prior art adopts is high, durability is poor, poor expandability, coupling are high and developmental capacity is low, the problem that the poor stability of the data validity testing fixture caused, maintenance cost and risk are high.Achieve the modularization and durability that improve verification msg validity method, reduce the effect of maintenance cost and evolution cost.
Accompanying drawing explanation
The accompanying drawing forming a application's part is used to provide a further understanding of the present invention, and schematic description and description of the present invention, for explaining the present invention, does not form inappropriate limitation of the present invention.In the accompanying drawings:
Fig. 1 is the flow chart of the Validation of Data method according to the embodiment of the present invention one;
Fig. 2 is the flow chart according to the preferred Validation of Data method of the embodiment of the present invention one;
Fig. 3 is the Validation of Data system block diagram according to the embodiment of the present invention;
Fig. 4 is the structural representation of the Validation of Data device according to the embodiment of the present invention two;
Fig. 5 is the structural representation according to the preferred Validation of Data device of the embodiment of the present invention two; And
Fig. 6 is the structural representation according to the preferred Validation of Data device of the embodiment of the present invention two.
Embodiment
It should be noted that, when not conflicting, the embodiment in the application and the feature in embodiment can combine mutually.Below with reference to the accompanying drawings and describe the present invention in detail in conjunction with the embodiments.
The present invention program is understood better in order to make those skilled in the art person, below in conjunction with the accompanying drawing in the embodiment of the present invention, technical scheme in the embodiment of the present invention is clearly and completely described, obviously, described embodiment is only the embodiment of a part of the present invention, instead of whole embodiments.Based on the embodiment in the present invention, those of ordinary skill in the art, not making the every other embodiment obtained under creative work prerequisite, should belong to the scope of protection of the invention.
It should be noted that, term " first ", " second " etc. in specification of the present invention and claims and above-mentioned accompanying drawing are for distinguishing similar object, and need not be used for describing specific order or precedence.Should be appreciated that the data used like this can be exchanged, in the appropriate case so that embodiments of the invention described herein.In addition, term " comprises " and " having " and their any distortion, intention is to cover not exclusive comprising, such as, contain those steps or unit that the process of series of steps or unit, method, system, product or equipment is not necessarily limited to clearly list, but can comprise clearly do not list or for intrinsic other step of these processes, method, product or equipment or unit.
Embodiment 1
Embodiments provide a kind of Validation of Data method.
Fig. 1 is the flow chart of the Validation of Data method according to the embodiment of the present invention.As shown in Figure 1, it is as follows that the method comprising the steps of:
Step S21, obtains metadata.
Concrete, the metadata in above-mentioned steps S21 is be invested in the initial data in each application scenarios, and, above-mentioned meta-data pack contains the information for self-described, wherein, and metadata, be a kind of data of self descriptiveness, it needs to be predefined in metadatabase, and change is smaller.Such as: a parameter, it can tell user, and whether what Class Type is this parameter be, parameter name, can empty and parameter value be how many, and observability, and user can carry out the maintenance of metadata by inking device.
Step S23, determines at least one application scenarios verifying metadata.
Concrete, by above-mentioned steps S23, determine the application scenarios that above-mentioned metadata is thrown in, these application scenarioss can be one, also can be multiple.
Step S25, according to application scenarios, determines the configuration parameter corresponding with application scenarios, and wherein, configuration parameter is for characterizing the test environment data of application scenarios, and test environment data at least comprise: the Account Logon parameter of current application scene.
Concrete, above-mentioned steps S25, according to one or more application scenarios, determines the configuration parameter corresponding with different application scene.These configuration parameters can comprise the Account Logon parameter of application scenarios.
In the middle of practical application, application scenarios is for Baidu's phoenix nest, and in synchronous Baidu phoenix nest synchrodata process, parameter needs the data file list of Profile, account ID and the current account related to belonging to account.
Step S27, is respectively loaded on metadata by configuration parameter corresponding for any one application scenarios, generates at least one data to be verified, and wherein, the metadata configuration parameter corresponding with any one application scenarios has incidence relation.
Concrete, each configuration parameter corresponding with application scenarios is bound with metadata by above-mentioned steps S27 respectively, generates the to be verified data for validation verification corresponding with each scene respectively.
In the middle of practical application, by carrying out man-to-man mapping to metadata and configuration parameter, thus effectively can complete data the School Affairs of concrete metadata is bound.
Step S29, obtains the Verification Project that any one application scenarios is corresponding, wherein, and the verification step that Verification Project carries out at application scenarios testing for characterizing data to be verified.
Concrete, by above-mentioned steps S29, obtain the Verification Project corresponding with any one application scenarios.The data verification that Verification Project comprises may relate to many-sided checking, and also can carry out according to a fixed sequence between various checking, and these checking items are directed to a certain application scenarios provides data verification service, is together to form the entirety of a Verification Project.
These Verification Projects can be stored in the public Verification Project in the middle of publicly-owned storehouse, also can be the privately owned Verification Project that creates voluntarily of demand that user is directed to oneself and be stored in the middle of private database.
In the middle of practical application, application scenarios is for Baidu's phoenix nest, the full account of Baidu's phoenix nest is comprised as whether download file validity, the superior and the subordinate's solid data relation lose and the early warning etc. of empty data by the data verification of process, and this several checking forms a Verification Project jointly.
Step S31, verifies data to be verified based on Verification Project, is verified result.
Concrete, above-mentioned steps S31, according to each verification step in Verification Project, treats verification msg and verifies.
By above-mentioned steps S21 to step S31, configuration parameter corresponding with application scenarios for metadata is mated, generate the data to be verified that can use for different application scene, and specify the Verification Project corresponding with application scenarios according to different application scenarioss.Perform each verification step in Verification Project successively.Obtain the result of metadata in application scenarios.
In summary, the invention solves that the inspection method fragmentation that data validity testing fixture in prior art adopts is high, durability is poor, poor expandability, coupling are high and developmental capacity is low, the problem that the poor stability of the data validity testing fixture caused, maintenance cost and risk are high, achieve the modularization and durability that improve verification msg validity method, reduce the effect of maintenance cost and evolution cost.
Preferably, in the above embodiments of the present application, verify data to be verified in step S31 based on Verification Project, be verified in result, step comprises:
Step S311, create proof rule according to Verification Project, wherein, proof rule is for characterizing the restriction relation between each Verification Project.
Step S313, according to proof rule, treats verification msg and verifies, wherein, treats the method that verification msg carries out verifying and at least comprises: metadata must defeated item checking, the superior and the subordinate's entity relationship checking and empty data verification.
Concrete, by step S311 and step S313, for each Verification Project adds restriction relation, the verification step in Verification Project is retrained.The Verification Project not conforming to specification to some by restriction relation identifies, processes.Wherein, checking item verifies certain aspect of the data of scene.Authentication unit is then formed by a series of checking item dynamic combined.Thus the checking of some row can be completed.Restriction relation at least can comprise: step constraint, enforceability constraint, enable constraint etc.Such as: enable constraint, can control section or whole Verification Projects can not perform.
Wherein, metadata must be verified as by defeated item: current scene data verification must to specify premised on a several metadata, and metadata must the feature of defeated item be to require that this metadata must be in binding state, and the data of binding can not be sky.
The method of the superior and the subordinate's entity relationship checking is: comprise subordinate to higher level, subordinate is as the checking of this relation of a higher level's part.Such as, trigger the destination address comprised in metadata, access corresponding subpage frame according to destination address, judge the validity of corresponding subpage frame.
The verification method of empty data verification is: verify the data content of metadata in application scenarios, judges whether metadata appears as empty situation in application scenarios.Such as: verify that whether the data obtained from on-line system are empty, or whether verification msg is the valid format type of specifying.
Certainly can also add the method for other checking for validation verification, not enumerate herein.
Preferably, as shown in Figure 2, in the above embodiments of the present application, verify data to be verified in step S31 based on Verification Project, after being verified result, method also comprises:
Step S33, according to application scenarios, configuration parameter, Verification Project, proof rule and the result, generates checking report.
Concrete, by step S33, respectively record is carried out to the content such as checking scene, configuration parameter, Verification Project, the result of metadata, generate and be directed to the various certificate parameters of a certain group metadata at each application scenarios.The data verification of application scenarios, need concrete contextual data and concrete metadata to be bound by adapter, that is metadata is the carrier of True Data, participates in proof procedure.
Preferably, in the above embodiments of the present application, in step S33 according to application scenarios, configuration parameter, Verification Project, proof rule and the result, after generating checking report, method also comprises:
Step S341, identifies the error message in checking report, determines type of error.
Step S343, according to type of error, divides error level.
Concrete, by step S341 and step S343, according to the checking report generated, error message in checking report is identified, determine the type of error occurred in proof procedure, and change a point error level according to type of error, and point out accordingly, process.
In the middle of practical application, in the process of service metadata validation verification, define different warning levels and corresponding diagnostic message according to the failure state of metadata, system can generate diagnosis report and validation verification result automatically.Audit report is for following the trail of the effect of proof procedure and early warning, so that the action of responding property process in time.The result of validation verification, be then used for the condition of next process of data syn-chronization, determines whether allow this synchrodata to enter business processing flow by synchronization program.
Such as: find Business Entity the superior and the subordinate lost contact, once there is the situation of above-mentioned lost contact, system can determine wrong error level according to generating the result, and system, according to error level, judges whether to allow to carry out follow-up synchronized process.
Preferably, in the above embodiments of the present application, in step S31 based on the described data to be verified of described Verification Project checking, after being verified result, method also comprises:
Step S35, judges the versatility of Verification Project, and divide the project generic grade of Verification Project, wherein, project generic grade at least comprises: total level, privately owned level.
Step S36, according to project generic grade, is stored in Verification Project storehouse, and wherein, Verification Project storehouse at least comprises: Verification Project private database and Verification Project have storehouse, for storing Verification Project.
Concrete, by step S35 and step S36, the Verification Project of establishment can be preserved, according to the general degree of the Verification Project preserved at application scenarios, judge the project generic grade of Verification Project.According to project generic grade, Verification Project can be divided into: publicly-owned level and privately owned level.The Verification Project of total level is stored in Verification Project and has storehouse, the Verification Project of privately owned level is stored in Verification Project private database.The publicly-owned storehouse of Verification Project is public available as seen; And Verification Project private database is that certain business use scenes is specifically used, other scene cannot use.So just effectively distinguish except individual example and public situation.Along with development and the analysis of business, we can progressively by certain, the abstract socialization of some check item (namely being transferred to the process in publicly-owned storehouse by private database).
In the middle of practical application, early stage, we docked Baidu's phoenix nest system, the early warning of the validity check of data checks include file, the superior and the subordinate's entity relationship lost contact and empty data.Later stage docking 360 search (an apt word added to clinch the point in Qihoo) system, file is invalid and the problem of the superior and the subordinate's entity lost contact to find existence too.We only need these two check item communizations (by abstract means, going business), thus completion system develop and design multiplexing.
Preferably, in the above embodiments of the present application, in step S313 according to described proof rule, after verifying described data to be verified, method also comprises:
Step S37, judges the versatility of proof rule, and divide the regular all-purpose grade of proof rule, wherein, regular all-purpose grade at least comprises: total level, privately owned level.
Step S38, according to regular all-purpose grade, is stored in proof rule storehouse, and wherein, proof rule storehouse at least comprises: proof rule private database and proof rule have storehouse, for storing proof rule.
Concrete, by step S37 and step S38, the proof rule of establishment can be preserved, according to the general degree of the proof rule preserved at application scenarios, judge the regular all-purpose grade of proof rule.According to regular all-purpose grade, proof rule can be divided into: publicly-owned level and privately owned level.The proof rule of total level is stored in proof rule and has storehouse, the proof rule of privately owned level is stored in proof rule private database.The publicly-owned storehouse of proof rule is public available as seen; And proof rule private database is that certain business use scenes is specifically used, other scene cannot use.So just effectively distinguish except individual example and public situation.Along with development and the analysis of business, we can progressively by certain, the abstract socialization of some check item (namely being transferred to the process in publicly-owned storehouse by private database).
By the above embodiments of the present application, can solve and check logic fragmentation.Realize low coupling, high cohesion, strengthen the durability of Verification Project.The autgmentability of verification method is improved greatly, is greatly reduced follow-up maintenance cost.And by visual configuration tool, the management of support system and evolution effectively.
Further, the function that said method can be realized, builds Validation of Data system.Be respectively Validation of Data Operation system setting difference in functionality module.As shown in Figure 3, system, with being divided into 7 modules, is respectively: metadatabase device, adaptive device, demo plant, Verification Project storehouse, constraint device, diagnosis report device and inking device.
Metadatabase, for storing the metadata needing checking.
Adaptive device, for according to application scenarios, determines the configuration parameter corresponding with application scenarios.
Constraint fills, for creating proof rule according to Verification Project.And according to proof rule, treat verification msg and verify.
Diagnosis report device, according to application scenarios, configuration parameter, Verification Project, proof rule and the result, generates checking report.Error message in checking report is identified, determines type of error.And according to type of error, divide error level.
Inking device, for obtaining Verification Project corresponding to any one application scenarios.
Demo plant, for verifying data to be verified based on Verification Project, is verified result.
Verification Project storehouse, for judging the versatility of Verification Project, divides the project generic grade of Verification Project.And according to project generic grade, be stored in Verification Project storehouse.
Embodiment 2
The embodiment of the present invention additionally provides a kind of Validation of Data device, as shown in Figure 4, this device can comprise: the first acquisition module 40, first determination module 42, second determination module 44, first generation module 46, second acquisition module 48 and authentication module 50.
Wherein, the first acquisition module 40, for obtaining metadata.
Concrete, above-mentioned first acquisition module 40, metadata for obtaining is be invested in the initial data in each application scenarios, further, above-mentioned meta-data pack contains the information for self-described, wherein, metadata, be a kind of data of self descriptiveness, it needs to be predefined in metadatabase, and change is smaller.Such as: a parameter, it can tell user, and whether what Class Type is this parameter be, parameter name, can empty and parameter value be how many, and observability, and user can carry out the maintenance of metadata by inking device.
First determination module 42, for determining at least one application scenarios verifying metadata.
Concrete, by the first determination module 42, determine the application scenarios that above-mentioned metadata is thrown in, these application scenarioss can be one, also can be multiple.
Second determination module 44, for according to application scenarios, determines the configuration parameter corresponding with application scenarios, and wherein, configuration parameter is for characterizing the test environment data of application scenarios, and test environment data at least comprise: the Account Logon parameter of current application scene.
Concrete, above-mentioned second determination module 44, according to one or more application scenarios, determines the configuration parameter corresponding with different application scene.These configuration parameters can comprise the Account Logon parameter of application scenarios.
In the middle of practical application, application scenarios is for Baidu's phoenix nest, and in synchronous Baidu phoenix nest synchrodata process, parameter needs the data file list of Profile, account ID and the current account related to belonging to account.
First generation module 46, for configuration parameter corresponding for any one application scenarios is respectively loaded on metadata, generates at least one data to be verified, and wherein, the metadata configuration parameter corresponding with any one application scenarios has incidence relation.
Concrete, each configuration parameter corresponding with application scenarios is bound with metadata by above-mentioned first generation module 46 respectively, generates the to be verified data for validation verification corresponding with each scene respectively.
In the middle of practical application, by carrying out man-to-man mapping to metadata and configuration parameter, thus effectively can complete data the School Affairs of concrete metadata is bound.
Second acquisition module 48, for obtaining Verification Project corresponding to any one application scenarios, wherein, the verification step that Verification Project carries out at application scenarios testing for characterizing data to be verified.
Concrete, by above-mentioned second acquisition module 48, obtain the Verification Project corresponding with any one application scenarios.The data verification that Verification Project comprises may relate to many-sided checking, and also can carry out according to a fixed sequence between various checking, and these checking items are directed to a certain application scenarios provides data verification service, is together to form the entirety of a Verification Project.
These Verification Projects can be stored in the public Verification Project in the middle of publicly-owned storehouse, also can be the privately owned Verification Project that creates voluntarily of demand that user is directed to oneself and be stored in the middle of private database.
In the middle of practical application, application scenarios is for Baidu's phoenix nest, the full account of Baidu's phoenix nest is comprised as whether download file validity, the superior and the subordinate's solid data relation lose and the early warning etc. of empty data by the data verification of process, and this several checking forms a Verification Project jointly.
Authentication module 50, for verifying data to be verified based on Verification Project, is verified result.
Concrete, above-mentioned authentication module 50, according to each verification step in Verification Project, treats verification msg and verifies.
By above-mentioned first acquisition module 40, first determination module 42, second determination module 44, first generation module 46, second acquisition module 48 and authentication module 50, configuration parameter corresponding with application scenarios for metadata is mated, the data to be verified that generation can use for different application scene, and specify the Verification Project corresponding with application scenarios according to different application scenarioss.Perform each verification step in Verification Project successively.Obtain the result of metadata in application scenarios.
In summary, the invention solves that the inspection method fragmentation that data validity testing fixture in prior art adopts is high, durability is poor, poor expandability, coupling are high and developmental capacity is low, the problem that the poor stability of the data validity testing fixture caused, maintenance cost and risk are high, achieve the modularization and durability that improve verification msg validity method, reduce the effect of maintenance cost and evolution cost.
Further, the step that can be realized by authentication module 50 is comprised:
Create proof rule according to Verification Project, wherein, proof rule is for characterizing the restriction relation between each Verification Project.
According to proof rule, treat verification msg and verify, wherein, treat the method that verification msg carries out verifying and at least comprise: metadata must defeated item checking, the superior and the subordinate's entity relationship checking and empty data verification.
Concrete, above-mentioned steps adds restriction relation for each Verification Project, retrains the verification step in Verification Project.The Verification Project not conforming to specification to some by restriction relation identifies, processes.Wherein, checking item verifies certain aspect of the data of scene.Authentication unit is then formed by a series of checking item dynamic combined.Thus the checking of some row can be completed.Restriction relation at least can comprise: step constraint, enforceability constraint, enable constraint etc.Such as: enable constraint, can control section or whole Verification Projects can not perform.
Wherein, metadata must be verified as by defeated item: current scene data verification must to specify premised on a several metadata, and metadata must the feature of defeated item be to require that this metadata must be in binding state, and the data of binding can not be sky.
The method of the superior and the subordinate's entity relationship checking is: comprise subordinate to higher level, subordinate is as the checking of this relation of a higher level's part.Such as, trigger the destination address comprised in metadata, access corresponding subpage frame according to destination address, judge the validity of corresponding subpage frame.
The verification method of empty data verification is: verify the data content of metadata in application scenarios, judges whether metadata appears as empty situation in application scenarios.Such as: verify that whether the data obtained from on-line system are empty, or whether verification msg is the valid format type of specifying.
Certainly can also add the method for other checking for validation verification, not enumerate herein.
Preferably, as shown in Figure 5, in the above embodiments of the present application, device also comprises: the second generation module 52.
Wherein, the second generation module 52, for according to application scenarios, configuration parameter, Verification Project, proof rule and the result, generates checking report.
Concrete, by the second generation module 52, respectively record is carried out to the content such as checking scene, configuration parameter, Verification Project, the result of metadata, generate and be directed to the various certificate parameters of a certain group metadata at each application scenarios.The data verification of application scenarios, need concrete contextual data and concrete metadata to be bound by adapter, that is metadata is the carrier of True Data, participates in proof procedure.
Preferably, as shown in Figure 6, in the above embodiments of the present application, device also comprises: the 3rd determination module 531 and the first judge module 533.
Wherein, the 3rd determination module 531, for identifying the error message in checking report, determines type of error.
First judge module 533, for according to type of error, divides error level.
Concrete, by above-mentioned 3rd determination module 531 and the first judge module 533.According to the checking report generated, the error message in checking report is identified, determines the type of error occurred in proof procedure, and change a point error level according to type of error, and point out accordingly, process.
In the middle of practical application, in the process of service metadata validation verification, define different warning levels and corresponding diagnostic message according to the failure state of metadata, system can generate diagnosis report and validation verification result automatically.Audit report is for following the trail of the effect of proof procedure and early warning, so that the action of responding property process in time.The result of validation verification, be then used for the condition of next process of data syn-chronization, determines whether allow this synchrodata to enter business processing flow by synchronization program.
Such as: find Business Entity the superior and the subordinate lost contact, once there is the situation of above-mentioned lost contact, system can determine wrong error level according to generating the result, and system, according to error level, judges whether to allow to carry out follow-up synchronized process.
Further, in the above embodiments of the present application, said apparatus can also comprise: the second judge module 54 and the first memory module 55.
Wherein, the second judge module 54, for judging the versatility of described Verification Project, divides the project generic grade of described Verification Project, and wherein, described project generic grade at least comprises: total level, privately owned level;
First memory module 55, for according to described project generic grade, is stored in Verification Project storehouse, and wherein, described Verification Project storehouse at least comprises: Verification Project private database and Verification Project have storehouse, for storing described Verification Project.
Concrete, by the second judge module 54 and the first memory module 55, the Verification Project of establishment can be preserved, according to the general degree of the Verification Project preserved at application scenarios, judge the project generic grade of Verification Project.According to project generic grade, Verification Project can be divided into: publicly-owned level and privately owned level.The Verification Project of total level is stored in Verification Project and has storehouse, the Verification Project of privately owned level is stored in Verification Project private database.The publicly-owned storehouse of Verification Project is public available as seen; And Verification Project private database is that certain business use scenes is specifically used, other scene cannot use.So just effectively distinguish except individual example and public situation.Along with development and the analysis of business, we can progressively by certain, the abstract socialization of some check item (namely being transferred to the process in publicly-owned storehouse by private database).
In the middle of practical application, early stage, we docked Baidu's phoenix nest system, the early warning of the validity check of data checks include file, the superior and the subordinate's entity relationship lost contact and empty data.Later stage docking 360 search (an apt word added to clinch the point in Qihoo) system, file is invalid and the problem of the superior and the subordinate's entity lost contact to find existence too.We only need these two check item communizations (by abstract means, going business), thus completion system develop and design multiplexing.
Preferably, in the above embodiments of the present application, said apparatus also comprises: the 3rd judge module 56 and the second memory module 57.
Wherein, the 3rd judge module 56, for judging the versatility of described proof rule, divides the regular all-purpose grade of described proof rule, and wherein, described regular all-purpose grade at least comprises: total level, privately owned level;
Second memory module 57, for according to described regular all-purpose grade, is stored in proof rule storehouse, and wherein, described proof rule storehouse at least comprises: proof rule private database and proof rule have storehouse, for storing described proof rule.
Concrete, by the 3rd judge module 56 and the second memory module 57, the proof rule of establishment can be preserved, according to the general degree of the proof rule preserved at application scenarios, judge the regular all-purpose grade of proof rule.According to regular all-purpose grade, proof rule can be divided into: publicly-owned level and privately owned level.The proof rule of total level is stored in proof rule and has storehouse, the proof rule of privately owned level is stored in proof rule private database.The publicly-owned storehouse of proof rule is public available as seen; And proof rule private database is that certain business use scenes is specifically used, other scene cannot use.So just effectively distinguish except individual example and public situation.Along with development and the analysis of business, we can progressively by certain, the abstract socialization of some check item (namely being transferred to the process in publicly-owned storehouse by private database).
By the above embodiments of the present application, can solve and check logic fragmentation.Realize low coupling, high cohesion, strengthen the durability of Verification Project.The autgmentability of verification method is improved greatly, is greatly reduced follow-up maintenance cost.And by visual configuration tool, the management of support system and evolution effectively.
It should be noted that, for aforesaid each embodiment of the method, in order to simple description, therefore it is all expressed as a series of combination of actions, but those skilled in the art should know, the present invention is not by the restriction of described sequence of movement, because according to the present invention, some step can adopt other orders or carry out simultaneously.Secondly, those skilled in the art also should know, the embodiment described in specification all belongs to preferred embodiment, and involved action and module might not be that the present invention is necessary.
In the above-described embodiments, the description of each embodiment is all emphasized particularly on different fields, in certain embodiment, there is no the part described in detail, can see the associated description of other embodiments.
In several embodiments that the application provides, should be understood that, disclosed device, the mode by other realizes.Such as, device embodiment described above is only schematic, the such as division of described unit, be only a kind of logic function to divide, actual can have other dividing mode when realizing, such as multiple unit or assembly can in conjunction with or another system can be integrated into, or some features can be ignored, or do not perform.Another point, shown or discussed coupling each other or direct-coupling or communication connection can be by some interfaces, and the indirect coupling of device or unit or communication connection can be electrical or other form.
The described unit illustrated as separating component or can may not be and physically separates, and the parts as unit display can be or may not be physical location, namely can be positioned at a place, or also can be distributed in multiple network element.Some or all of unit wherein can be selected according to the actual needs to realize the object of the present embodiment scheme.
In addition, each functional unit in each embodiment of the present invention can be integrated in a processing unit, also can be that the independent physics of unit exists, also can two or more unit in a unit integrated.Above-mentioned integrated unit both can adopt the form of hardware to realize, and the form of SFU software functional unit also can be adopted to realize.
If described integrated unit using the form of SFU software functional unit realize and as independently production marketing or use time, can be stored in a computer read/write memory medium.Based on such understanding, the part that technical scheme of the present invention contributes to prior art in essence in other words or all or part of of this technical scheme can embody with the form of software product, this computer software product is stored in a storage medium, comprises all or part of step of some instructions in order to make a computer equipment (can be personal computer, mobile terminal, server or the network equipment etc.) perform method described in each embodiment of the present invention.And aforesaid storage medium comprises: USB flash disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), portable hard drive, magnetic disc or CD etc. various can be program code stored medium.
The foregoing is only the preferred embodiments of the present invention, be not limited to the present invention, for a person skilled in the art, the present invention can have various modifications and variations.Within the spirit and principles in the present invention all, any amendment done, equivalent replacement, improvement etc., all should be included within protection scope of the present invention.
Claims (11)
1. a Validation of Data method, is characterized in that, comprising:
Obtain metadata;
Determine at least one application scenarios verifying described metadata;
According to described application scenarios, determine the configuration parameter corresponding with described application scenarios, wherein, described configuration parameter is for characterizing the test environment data of described application scenarios, and described test environment data at least comprise: the Account Logon parameter of current application scene;
Configuration parameter corresponding for application scenarios described in any one is respectively loaded on described metadata, generates at least one data to be verified, wherein, the described metadata configuration parameter corresponding with application scenarios described in any one has incidence relation;
Obtain the Verification Project that application scenarios described in any one is corresponding, wherein, the verification step that described Verification Project carries out at described application scenarios testing for characterizing described data to be verified;
Based on the described data to be verified of described Verification Project checking, be verified result.
2. method according to claim 1, is characterized in that, described based on the described data to be verified of described Verification Project checking, the step being verified result comprises:
Create proof rule according to described Verification Project, wherein, described proof rule is for characterizing the restriction relation described in each between Verification Project;
According to described proof rule, described data to be verified are verified, wherein, the method that described data to be verified are verified at least are comprised: metadata must the checking of defeated item, the superior and the subordinate's entity relationship checking and empty data verification.
3. method according to claim 2, is characterized in that, described based on the described data to be verified of described Verification Project checking, after being verified result, described method also comprises:
According to described application scenarios, described configuration parameter, described Verification Project, described proof rule and described the result, generate checking report.
4. method according to claim 3, is characterized in that, described according to described application scenarios, described configuration parameter, described Verification Project, described proof rule and described the result, after generating checking report, method also comprises:
Error message in described checking report is identified, determines type of error;
According to described type of error, divide error level.
5. method according to claim 1, is characterized in that, described based on the described data to be verified of described Verification Project checking, after being verified result, described method also comprises:
Judge the versatility of described Verification Project, divide the project generic grade of described Verification Project, wherein, described project generic grade at least comprises: total level, privately owned level;
According to described project generic grade, be stored in Verification Project storehouse, wherein, described Verification Project storehouse at least comprises: Verification Project private database and Verification Project have storehouse, for storing described Verification Project.
6. method according to claim 2, is characterized in that, described according to described proof rule, after verifying described data to be verified, described method also comprises:
Judge the versatility of described proof rule, divide the regular all-purpose grade of described proof rule, wherein, described regular all-purpose grade at least comprises: total level, privately owned level;
According to described regular all-purpose grade, be stored in proof rule storehouse, wherein, described proof rule storehouse at least comprises: proof rule private database and proof rule have storehouse, for storing described proof rule.
7. a Validation of Data device, is characterized in that, comprising:
First acquisition module, for obtaining metadata;
First determination module, for determining at least one application scenarios verifying described metadata;
Second determination module, for according to described application scenarios, determine the configuration parameter corresponding with described application scenarios, wherein, described configuration parameter is for characterizing the test environment data of described application scenarios, and described test environment data at least comprise: the Account Logon parameter of current application scene;
First generation module, for configuration parameter corresponding for application scenarios described in any one is respectively loaded on described metadata, generate at least one data to be verified, wherein, the described metadata configuration parameter corresponding with application scenarios described in any one has incidence relation;
Second acquisition module, for obtaining Verification Project corresponding to application scenarios described in any one, wherein, the verification step that described Verification Project carries out at described application scenarios testing for characterizing described data to be verified;
Authentication module, for based on the described data to be verified of described Verification Project checking, is verified result.
8. device according to claim 7, is characterized in that, described device also comprises:
Second generation module, for according to described application scenarios, described configuration parameter, described Verification Project, proof rule and described the result, generates checking report.
9. device according to claim 8, is characterized in that, described device also comprises:
3rd determination module, for identifying the error message in described checking report, determines type of error;
First judge module, for according to described type of error, divides error level.
10. device according to claim 7, it is characterized in that, described device also comprises:
Second judge module, for judging the versatility of described Verification Project, divides the project generic grade of described Verification Project, and wherein, described project generic grade at least comprises: total level, privately owned level;
First memory module, for according to described project generic grade, is stored in Verification Project storehouse, and wherein, described Verification Project storehouse at least comprises: Verification Project private database and Verification Project have storehouse, for storing described Verification Project.
11. devices according to claim 8, is characterized in that, described device also comprises:
3rd judge module, for judging the versatility of described proof rule, divides the regular all-purpose grade of described proof rule, and wherein, described regular all-purpose grade at least comprises: total level, privately owned level;
Second memory module, for according to described regular all-purpose grade, is stored in proof rule storehouse, and wherein, described proof rule storehouse at least comprises: proof rule private database and proof rule have storehouse, for storing described proof rule.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201410773933.4A CN104486337B (en) | 2014-12-12 | 2014-12-12 | Validation of Data method and device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201410773933.4A CN104486337B (en) | 2014-12-12 | 2014-12-12 | Validation of Data method and device |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN104486337A true CN104486337A (en) | 2015-04-01 |
| CN104486337B CN104486337B (en) | 2018-05-18 |
Family
ID=52760841
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201410773933.4A Active CN104486337B (en) | 2014-12-12 | 2014-12-12 | Validation of Data method and device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN104486337B (en) |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104865346A (en) * | 2015-04-22 | 2015-08-26 | 北京交通大学 | Method and apparatus for verification of data acquired by vehicle tail gas detection system |
| CN110602056A (en) * | 2019-08-21 | 2019-12-20 | 厦门网宿有限公司 | Service parameter transmission method and device |
| CN110781637A (en) * | 2019-10-14 | 2020-02-11 | 珠海泰芯半导体有限公司 | Chip verification auxiliary environment and chip verification system |
| CN113986943A (en) * | 2021-12-28 | 2022-01-28 | 畅捷通信息技术股份有限公司 | Data consistency verification method, device and medium based on scene data stamp |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101291188A (en) * | 2008-06-17 | 2008-10-22 | 中兴通讯股份有限公司 | Data synchronizing system and method for television service network with mobile terminals |
| CN101996371A (en) * | 2009-08-17 | 2011-03-30 | 华为技术有限公司 | Advertisement putting method, device and system |
| CN102457555A (en) * | 2010-10-28 | 2012-05-16 | 中兴通讯股份有限公司 | Security system and method for distributed storage |
| CN103699693A (en) * | 2014-01-10 | 2014-04-02 | 中国南方电网有限责任公司 | Metadata-based data quality management method and system |
-
2014
- 2014-12-12 CN CN201410773933.4A patent/CN104486337B/en active Active
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101291188A (en) * | 2008-06-17 | 2008-10-22 | 中兴通讯股份有限公司 | Data synchronizing system and method for television service network with mobile terminals |
| CN101996371A (en) * | 2009-08-17 | 2011-03-30 | 华为技术有限公司 | Advertisement putting method, device and system |
| CN102457555A (en) * | 2010-10-28 | 2012-05-16 | 中兴通讯股份有限公司 | Security system and method for distributed storage |
| CN103699693A (en) * | 2014-01-10 | 2014-04-02 | 中国南方电网有限责任公司 | Metadata-based data quality management method and system |
Cited By (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104865346A (en) * | 2015-04-22 | 2015-08-26 | 北京交通大学 | Method and apparatus for verification of data acquired by vehicle tail gas detection system |
| CN104865346B (en) * | 2015-04-22 | 2017-02-01 | 北京交通大学 | Method and apparatus for verification of data acquired by vehicle tail gas detection system |
| CN110602056A (en) * | 2019-08-21 | 2019-12-20 | 厦门网宿有限公司 | Service parameter transmission method and device |
| CN110781637A (en) * | 2019-10-14 | 2020-02-11 | 珠海泰芯半导体有限公司 | Chip verification auxiliary environment and chip verification system |
| CN110781637B (en) * | 2019-10-14 | 2023-05-02 | 珠海泰芯半导体有限公司 | Chip verification auxiliary environment and chip verification system |
| CN113986943A (en) * | 2021-12-28 | 2022-01-28 | 畅捷通信息技术股份有限公司 | Data consistency verification method, device and medium based on scene data stamp |
| CN113986943B (en) * | 2021-12-28 | 2022-04-22 | 畅捷通信息技术股份有限公司 | Data consistency verification method, device and medium based on scene data stamp |
Also Published As
| Publication number | Publication date |
|---|---|
| CN104486337B (en) | 2018-05-18 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Klein et al. | A systematic review of system-of-systems architecture research | |
| CN105389256A (en) | Unit testing method and system | |
| CN109522228B (en) | Interface automation test data construction method, device, platform and storage medium | |
| CN104133765B (en) | The test case sending method of network activity and test case server | |
| Washizaki et al. | Software-engineering design patterns for machine learning applications | |
| CN104486337A (en) | Data validity verification method and device | |
| CN109740129B (en) | Report generation method, device and equipment based on blockchain and readable storage medium | |
| CN101819551B (en) | Method and system for recording service interaction data of enterprise message system | |
| CN112631846A (en) | Fault drilling method and device, computer equipment and storage medium | |
| CN102567172A (en) | Concurrent workload simulation for application performance testing | |
| CN107632827A (en) | The generation method and device of the installation kit of application | |
| CN108460068A (en) | Method, apparatus, storage medium and the terminal that report imports and exports | |
| CN105607997A (en) | Background service test method, apparatus and system for software product | |
| CN107168844B (en) | Performance monitoring method and device | |
| Safonova | On one approach to modeling enterprise-wide function-oriented computer networks | |
| CN104471530A (en) | Executable software specification generation | |
| CN106055326A (en) | Program processing method and dispatch server | |
| Luo et al. | Clustering and tailoring user session data for testing web applications | |
| CN107203471B (en) | Joint debugging method, service platform and computer storage medium | |
| CN113449444B (en) | Simulation method, computer device and storage medium for domain engineering | |
| CN112035366B (en) | Test case generation method, device and equipment | |
| CN116521509A (en) | Intelligent contract testing method, device, equipment, storage medium and product | |
| CN114650211A (en) | Fault repairing method, device, electronic equipment and computer readable storage medium | |
| Togay et al. | Rule based axiomatic design theory guidance for software development | |
| Zhang et al. | Integration design and model transformation for cyber physical systems |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant | ||
| PE01 | Entry into force of the registration of the contract for pledge of patent right |
Denomination of invention: Data validity verification method and device Effective date of registration: 20190531 Granted publication date: 20180518 Pledgee: Shenzhen Black Horse World Investment Consulting Co., Ltd. Pledgor: Beijing Guoshuang Technology Co.,Ltd. Registration number: 2019990000503 |
|
| PE01 | Entry into force of the registration of the contract for pledge of patent right | ||
| CP02 | Change in the address of a patent holder |
Address after: 100083 No. 401, 4th Floor, Haitai Building, 229 North Fourth Ring Road, Haidian District, Beijing Patentee after: BEIJING GRIDSUM TECHNOLOGY Co.,Ltd. Address before: 100086 Beijing city Haidian District Shuangyushu Area No. 76 Zhichun Road cuigongfandian 8 layer A Patentee before: BEIJING GRIDSUM TECHNOLOGY Co.,Ltd. |
|
| CP02 | Change in the address of a patent holder |