CN113569047A - Intersystem data verification method, device, equipment and readable storage medium - Google Patents
Intersystem data verification method, device, equipment and readable storage medium Download PDFInfo
- Publication number
- CN113569047A CN113569047A CN202110833541.2A CN202110833541A CN113569047A CN 113569047 A CN113569047 A CN 113569047A CN 202110833541 A CN202110833541 A CN 202110833541A CN 113569047 A CN113569047 A CN 113569047A
- Authority
- CN
- China
- Prior art keywords
- data
- fields
- model
- field
- downstream
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000000034 method Methods 0.000 title claims abstract description 40
- 238000013524 data verification Methods 0.000 title claims description 35
- 238000011144 upstream manufacturing Methods 0.000 claims abstract description 149
- 238000013507 mapping Methods 0.000 claims abstract description 136
- XLYOFNOQVPJJNP-UHFFFAOYSA-N water Substances O XLYOFNOQVPJJNP-UHFFFAOYSA-N 0.000 claims abstract description 66
- 238000012549 training Methods 0.000 claims description 102
- 238000004364 calculation method Methods 0.000 claims description 21
- 238000004422 calculation algorithm Methods 0.000 claims description 17
- 238000013145 classification model Methods 0.000 claims description 17
- 238000004590 computer program Methods 0.000 claims description 9
- 230000008859 change Effects 0.000 claims description 5
- 238000012545 processing Methods 0.000 claims description 3
- 238000001514 detection method Methods 0.000 abstract description 9
- 238000000275 quality assurance Methods 0.000 abstract description 2
- 238000004891 communication Methods 0.000 description 7
- 230000006870 function Effects 0.000 description 4
- 230000003993 interaction Effects 0.000 description 4
- 230000002452 interceptive effect Effects 0.000 description 4
- 230000005236 sound signal Effects 0.000 description 4
- 238000010586 diagram Methods 0.000 description 3
- 230000005540 biological transmission Effects 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000003287 optical effect Effects 0.000 description 2
- 238000003491 array Methods 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 238000007689 inspection Methods 0.000 description 1
- 210000001503 joint Anatomy 0.000 description 1
- 230000008569 process Effects 0.000 description 1
- 230000003068 static effect Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/30—Information retrieval; Database structures therefor; File system structures therefor of unstructured textual data
- G06F16/35—Clustering; Classification
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/30—Information retrieval; Database structures therefor; File system structures therefor of unstructured textual data
- G06F16/38—Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually
- G06F16/383—Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually using metadata automatically derived from the content
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/24—Classification techniques
- G06F18/241—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
- G06F18/2413—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches based on distances to training or reference patterns
- G06F18/24147—Distances to closest patterns, e.g. nearest neighbour classification
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Data Mining & Analysis (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- Library & Information Science (AREA)
- Databases & Information Systems (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Evolutionary Computation (AREA)
- Evolutionary Biology (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Bioinformatics & Computational Biology (AREA)
- Artificial Intelligence (AREA)
- Life Sciences & Earth Sciences (AREA)
- Management, Administration, Business Operations System, And Electronic Commerce (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The invention relates to the technical field of quality assurance, in particular to a method, a device, equipment and a readable storage medium for verifying intersystem data, which establish a first model; acquiring running water data; bringing a plurality of upstream fields in the pipeline data into the first model to obtain a plurality of second fields; the method comprises the steps of establishing a mapping relation between the upstream field and the downstream field in the first model, bringing the upstream field into the first model to obtain the second field for predicting the downstream field, and comparing the downstream field with the upstream field to detect whether the upstream field is consistent with the downstream field, so that the problem of missed detection caused by the fact that the consistency of the upstream field and the downstream field is detected manually in the prior art is solved.
Description
Technical Field
The invention relates to the technical field of quality assurance, in particular to a method, a device and equipment for verifying intersystem data and a readable storage medium.
Background
The method is characterized in that a long service link exists in a bank, complex service scenes of interactive serial execution between systems according to a dependency relationship need to be guaranteed, correct service execution can be guaranteed only by guaranteeing correct data of upstream and downstream systems, a large number of long transaction chains in the bank span interactive scenes among a plurality of systems, and the data consistency of the upstream and downstream systems is generally verified by relying on manual operation to have the risk of artificial omission.
Disclosure of Invention
The present invention is directed to a method, an apparatus, a device and a readable storage medium for data verification between systems, so as to solve the above problems.
In order to achieve the above object, the embodiments of the present application provide the following technical solutions:
in one aspect, an embodiment of the present application provides a method for data verification between systems, where the method includes: establishing a first model, wherein the first model comprises a plurality of first submodels, and the first submodel comprises a mapping relation between an upstream field and a plurality of corresponding downstream fields; acquiring the flow data, wherein the flow data comprises a plurality of upstream fields and a plurality of downstream fields, and the number of the upstream fields in one flow data is the same as that of the downstream fields;
bringing the upstream fields in the running water data into the first model to obtain second fields, wherein the second fields are downstream fields which are derived by the upstream fields in the running water data through the mapping relation in the first model and correspond to the upstream fields in the running water data; comparing whether the plurality of second fields are the same as the downstream fields in the plurality of corresponding pipeline data, and if each second field is the same as the downstream fields in the plurality of pipeline data, determining that the upstream fields in the plurality of pipeline data are consistent with the downstream fields in the plurality of pipeline data.
Optionally, after the establishing the first model, the method further includes:
acquiring training data, wherein the training data comprises a plurality of the running data;
calling the flow data in one training data;
bringing the running water data in the training data into the first model, changing the mapping relation in the first model through a learning classification model K nearest neighbor algorithm, and outputting the changed first model;
calling the running water data in another unused training data, bringing the running water data into the current first model, changing the mapping relation in the first model through a learning classification model K nearest neighbor algorithm, and outputting the changed first model until the running water data in the training data is used;
and outputting the trained first model.
Optionally, after all the running water data in the training data is used, the method further includes:
sequentially calling the flow data in the training data, and respectively bringing the flow data in the training data into the current first model to obtain a plurality of second fields corresponding to the flow data in the training data one by one;
respectively detecting whether the second field corresponding to the running water data in the training data is the same as the downstream field corresponding to the running water data in the training data, if the second fields in the training data are the same as the downstream fields corresponding to the training data, outputting the trained first model, if the second fields in any one of the training data are different from the downstream fields corresponding to the training data, returning to bring the running water data in the training data into the first model, changing the mapping relation in the first model by learning a classification model K nearest neighbor algorithm, and outputting the changed first model.
Optionally, the establishing a first model includes:
obtaining a mapping data, said mapping data comprising one said upstream field and a plurality of corresponding said downstream fields;
establishing a mapping relation between one upstream field in the mapping data and one downstream field in the mapping data one by one until the upstream field in the mapping data and the downstream fields in the plurality of mapping data have a mapping relation, and obtaining a first sub-model corresponding to the upstream field in the mapping data, wherein the first sub-model comprises the mapping relation between the upstream field and the plurality of downstream fields;
acquiring another upstream field which is not established with a mapping relation, executing the one-by-one establishment of the mapping relation between one upstream field in the mapping data and one downstream field in the mapping data until the upstream field in the mapping data and the downstream fields in the mapping data have the mapping relation, and acquiring a first submodel until each upstream field corresponds to one first submodel;
outputting a plurality of the first submodels, wherein the first model is composed of a plurality of the first submodels.
Optionally, the bringing the plurality of upstream fields into the first model to obtain a plurality of second fields includes:
respectively bringing the upstream field in each pipeline data into the first model, finding one first sub-model corresponding to the upstream field in the pipeline data, and assigning the downstream field in the corresponding first sub-model to the second field;
outputting a plurality of the second fields.
Optionally, the comparing whether the plurality of second fields are the same as the downstream fields in the plurality of corresponding pipeline data includes:
if any one second field is different from the corresponding downstream fields in the plurality of the running water data, sending an alarm instruction, wherein the alarm instruction is an alarm prompt for manually verifying the running water data;
backing up the pipelined data that is manually verified and the upstream field is consistent with the downstream field.
In a second aspect, an embodiment of the present application provides an intersystem data verification system, where the system includes a first computing module, a first data obtaining module, a second computing module, and a third computing unit; the first calculation module is used for establishing a first model, the first model comprises a plurality of first submodels, and the first submodel comprises a mapping relation between an upstream field and a plurality of corresponding downstream fields; the device comprises a first data acquisition module, a second data acquisition module and a data processing module, wherein the first data acquisition module is used for acquiring the pipeline data, the pipeline data comprises a plurality of upstream fields and a plurality of downstream fields, and the number of the upstream fields in one pipeline data is the same as that of the downstream fields; a second calculation module, configured to bring the upstream fields in the running water data into the first model to obtain second fields, where the second fields are downstream fields corresponding to the upstream fields in the running water data, and the downstream fields are derived by the upstream fields in the running water data through the mapping relationship in the first model; a third calculating module, configured to compare whether the plurality of second fields are the same as the downstream fields in the plurality of corresponding pipelined data, and if each of the second fields is the same as the downstream fields in the plurality of pipelined data, determine that the upstream fields in the plurality of pipelined data are consistent with the downstream fields in the plurality of pipelined data.
Optionally, the first computing module comprises:
a first data acquisition unit, configured to acquire training data, where the training data includes a plurality of the running water data;
the second data acquisition unit is used for calling the running water data in the training data;
the first calculation unit is used for substituting the running water data in the training data into the first model, changing the mapping relation in the first model through a learning classification model K nearest neighbor algorithm, and outputting the changed first model;
the second calculation unit is used for calling the running water data in another unused training data, bringing the running water data into the current first model, changing the mapping relation in the first model through a learning classification model K neighbor algorithm, and outputting the changed first model until the running water data in the training data is used;
and the third calculation unit is used for outputting the trained first model.
Optionally, the second computing unit includes:
the first calculation subunit is configured to sequentially retrieve the flow data in the training data, and bring the flow data in the training data into the current first model respectively to obtain a plurality of second fields corresponding to the plurality of flow data in the training data one to one;
a second calculating subunit, configured to detect whether the second field corresponding to the pipeline data in the training data is the same as the downstream field corresponding to the pipeline data in the training data, respectively, output the trained first model if the plurality of second fields in the training data are the same as the downstream fields corresponding to the training data, return to bring the pipeline data in the training data into the first model if any of the second fields in the training data is not the same as the downstream fields corresponding to the training data, change the mapping relationship in the first model by using a learning classification model K nearest neighbor algorithm, and output the changed first model.
Optionally, the first computing module comprises:
a third data obtaining unit, configured to obtain mapping data, where the mapping data includes one upstream field and a plurality of corresponding downstream fields;
a fourth calculating unit, configured to establish a mapping relationship between one upstream field in the mapping data and one downstream field in the mapping data one by one until the upstream field in the mapping data and the downstream fields in the multiple mapping data all have a mapping relationship, and obtain a first sub-model corresponding to the upstream field in the mapping data, where the first sub-model includes a mapping relationship between the upstream field and the multiple downstream fields;
a fifth calculating unit, configured to obtain another upstream field that is not mapped, and execute the one-by-one mapping between one upstream field in the mapping data and one downstream field in the mapping data until the upstream field in the mapping data and the downstream fields in the multiple mapping data have a mapping, so as to obtain a first submodel until each upstream field corresponds to one first submodel;
and the sixth calculating unit is used for outputting a plurality of first submodels, and the first model is composed of a plurality of first submodels.
Optionally, the second computing module comprises:
a seventh calculating unit, configured to bring the upstream field in each of the pipeline data into the first model, find one of the first sub-models corresponding to the upstream field in the pipeline data, and assign the downstream field in the corresponding first sub-model to the second field;
an eighth calculation unit configured to output a plurality of the second fields.
Optionally, the third computing module comprises:
a ninth calculating unit, configured to send an alert instruction if any one of the second fields is different from the corresponding downstream fields in the plurality of running water data, where the alert instruction is an alert prompt that the running water data needs to be manually verified;
a tenth computing unit for backing up the pipelined data that is manually verified and the upstream field is consistent with the downstream field.
In a third aspect, an embodiment of the present application provides an inter-system data verification apparatus, which includes a memory and a processor. The memory is used for storing a computer program; the processor is used for realizing the steps of the intersystem data verification method when executing the computer program.
In a fourth aspect, an embodiment of the present application provides a readable storage medium, where a computer program is stored on the readable storage medium, and when the computer program is executed by a processor, the computer program implements the steps of the inter-system data verification method described above.
The invention has the beneficial effects that:
according to the invention, the mapping relation of the upstream field and the downstream field is established in the first model, the upstream field is brought into the first model to obtain the second field for predicting the downstream field, and the downstream field and the second field are compared to realize the function of detecting whether the upstream field and the downstream field are consistent, namely, the function of automatically detecting whether the interaction information between different systems is consistent is realized, so that the condition that the finally output data is wrong due to inconsistency of the interaction information between the systems is effectively prevented, and the problem of missed detection or false detection possibly caused by the fact that the upstream field and the downstream field are consistent through manual detection in the traditional banking industry is solved.
Additional features and advantages of the invention will be set forth in the description which follows, and in part will be obvious from the description, or may be learned by the practice of the embodiments of the invention. The objectives and other advantages of the invention will be realized and attained by the structure particularly pointed out in the written description and claims hereof as well as the appended drawings.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings needed to be used in the embodiments will be briefly described below, it should be understood that the following drawings only illustrate some embodiments of the present invention and therefore should not be considered as limiting the scope, and for those skilled in the art, other related drawings can be obtained according to the drawings without inventive efforts.
FIG. 1 is a flow chart of a method for verifying data between systems according to an embodiment of the present invention;
FIG. 2 is a schematic structural diagram of an inter-system data verification system according to an embodiment of the present invention;
FIG. 3 is a schematic structural diagram of an inter-system data verification apparatus according to an embodiment of the present invention;
Detailed Description
In order to make the objects, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are some, but not all, embodiments of the present invention. The components of embodiments of the present invention generally described and illustrated in the figures herein may be arranged and designed in a wide variety of different configurations. Thus, the following detailed description of the embodiments of the present invention, presented in the figures, is not intended to limit the scope of the invention, as claimed, but is merely representative of selected embodiments of the invention. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
It should be noted that: like reference numbers or letters refer to like items in the following figures, and thus, once an item is defined in one figure, it need not be further defined or explained in subsequent figures. Meanwhile, in the description of the present invention, the terms "first", "second", and the like are used only for distinguishing the description, and are not to be construed as indicating or implying relative importance.
Example 1
As shown in fig. 1, the present embodiment provides an intersystem data verification method including step S1, step S2, step S3, and step S4.
S1, establishing a first model, wherein the first model comprises a plurality of first submodels, and the first submodels comprise a mapping relation between an upstream field and a plurality of corresponding downstream fields; the first model comprises the mapping relation between each upstream field and a plurality of corresponding downstream fields;
s2, acquiring running water data, wherein the running water data comprises a plurality of upstream fields and a plurality of downstream fields, and the number of the upstream fields in one running water data is the same as that of the downstream fields; the stream data can be account data corresponding to a serial number, namely account information of a customer, because one business of a bank needs to relate to a plurality of different systems in the bank, the account data can be transmitted among the different systems when a business worker operates, and data when two adjacent different systems are in butt joint are the upstream data and the downstream data.
S3, bringing the upstream fields in the flow data into the first model to obtain second fields, wherein the second fields are downstream fields which are derived by the upstream fields in the flow data through the mapping relation in the first model and correspond to the upstream fields in the flow data;
the system automatically brings the upstream data in the running data into the first model to obtain a second field, and judges whether the transmission of the account data corresponding to the running number between two adjacent systems has errors or not by comparing the second field with the corresponding downstream field in the running data, namely whether the upstream data is consistent with the downstream data or not.
And S4, comparing whether the second fields are the same as the downstream fields in the corresponding pipeline data or not, and if each second field is the same as the downstream fields in the pipeline data, judging that the upstream fields in the pipeline data are consistent with the downstream fields in the pipeline data.
If the detection result is that the upstream field and the downstream field are consistent, the system determines that there is no problem in data transmission of the account data between different systems, and usually the step is completed by manual checking.
According to the invention, the mapping relation of the upstream field and the downstream field is established in the first model, the upstream field is brought into the first model to obtain the second field for predicting the downstream field, and the downstream field and the second field are compared to realize the function of detecting whether the upstream field and the downstream field are consistent, namely, the function of automatically detecting whether the interaction information between different systems is consistent is realized, so that the condition that the finally output data is wrong due to inconsistency of the interaction information between the systems is effectively prevented, and the problem of missed detection or false detection possibly caused by the fact that the upstream field and the downstream field are consistent through manual detection in the traditional banking industry is solved.
In a specific embodiment of the present disclosure, after the step S1, the method may further include a step S11, a step S12, a step S13, a step S14, and a step S15.
S11, acquiring training data, wherein the training data comprises a plurality of running data;
s12, calling the flow data in one training data;
s13, bringing the flow data in the training data into the first model, changing the mapping relation in the first model through a learning classification model K nearest neighbor algorithm, and outputting the changed first model;
s14, calling the running water data in another unused training data, bringing the running water data into the current first model, changing the mapping relation in the first model through a learning classification model K neighbor algorithm, and outputting the changed first model until the running water data in the training data is used;
and S15, outputting the trained first model.
The method of this embodiment is used to optimize the first model, and the training data including a plurality of pipeline data continuously optimizes and trains the first model, so that an upstream field in each pipeline data in the training data can output a second field identical to a plurality of corresponding downstream fields in the training data through the trained first model.
In a specific embodiment of the present disclosure, the step S14 may further include a step S141 and a step S142.
S141, sequentially calling the flow data in the training data, and respectively bringing the flow data in the training data into the current first model to obtain a plurality of second fields corresponding to the flow data in the training data one by one;
step S142, respectively detecting whether the second field corresponding to the pipeline data in the training data is the same as the downstream field corresponding to the pipeline data in the training data, if the plurality of second fields in the training data are the same as the downstream fields corresponding to the training data, outputting the trained first model, and if the second field in any one of the training data is not the same as the downstream field corresponding to the training data, returning to step S13.
The embodiment discloses a process of how the first model verifies whether the upstream field and the downstream field are consistent, and also discloses a judgment mode for adjusting the first model.
In a specific embodiment of the present disclosure, the step S1 may further include a step S16, a step S17, a step S18, and a step S19.
S16, obtaining mapping data, wherein the mapping data comprises one upstream field and a plurality of corresponding downstream fields;
s17, establishing a mapping relation between one upstream field in the mapping data and one downstream field in the mapping data one by one until the upstream field in the mapping data and the downstream fields in the plurality of mapping data have a mapping relation, and obtaining a first sub-model corresponding to the upstream field in the mapping data, wherein the first sub-model comprises the mapping relation between the upstream field and the downstream fields;
s18, obtaining another upstream field which is not subjected to mapping relationship establishment, executing the mapping relationship between one upstream field in the mapping data and one downstream field in the mapping data established one by one until the upstream field in the mapping data and the downstream fields in the mapping data have the mapping relationship, and obtaining a first submodel until each upstream field corresponds to one first submodel;
and S19, outputting a plurality of first submodels, wherein the first submodels are composed of a plurality of first submodels.
The embodiment discloses a method for establishing the first model, which includes acquiring all the upstream data and the downstream data, and sequentially establishing a mapping relationship between each upstream field and a plurality of corresponding upstream fields, wherein the plurality of mapping relationships form the first model.
In a specific embodiment of the present disclosure, the step S3 may further include a step S31 and a step S32.
S31, respectively bringing the upstream field in each pipeline data into the first model, finding one first sub-model corresponding to the upstream field in the pipeline data, and assigning the downstream field in the corresponding first sub-model to the second field;
and S32, outputting a plurality of second fields.
In a specific embodiment of the present disclosure, the step S4 may further include a step S41 and a step S42.
S41, if any one second field is different from the corresponding downstream fields in the plurality of running water data, sending an alarm instruction, wherein the alarm instruction is an alarm prompt for manually verifying the running water data;
step S42, backing up the pipelining data which is manually verified and the upstream field is consistent with the downstream field.
And recording the mapping relation between the upstream field and the downstream field which are not covered by the training data, and continuously further optimizing the first model by collecting the backup running water data and introducing the backup running water data into the training data.
The invention has a long service link in the bank, a complex service scene of interactive series execution between systems according to the dependency relationship needs to ensure the data correctness of upstream and downstream systems, the service execution can be ensured to be correct, a large number of long transaction chains in the bank span the interactive scene among a plurality of systems, the data consistency of the upstream and downstream systems is generally verified by depending on manual operation to have the risk of artificial omission of inspection, the invention establishes the mapping relationship between the upstream field and the downstream field in the first model, a second field for predicting a downstream field is obtained by bringing the upstream field into a first model, by comparing the downstream field with the second field, whether the upstream field and the downstream field are consistent or not is detected, and the problem of missed detection caused by the fact that the consistency of the upstream field and the downstream field is detected manually in the traditional method is solved.
Example 2
As shown in fig. 2, the present embodiment provides an intersystem data verification system, which includes a first calculation module 71, a first data acquisition module 72, a second calculation module 73, and a third calculation unit 74;
a first calculating module 71, configured to establish a first model, where the first model includes a plurality of first submodels, and the first submodel includes a mapping relationship between an upstream field and a plurality of corresponding downstream fields;
a first data obtaining module 72, configured to obtain pipelined data, where the pipelined data includes multiple upstream fields and multiple downstream fields, and the number of the upstream fields in one pipelined data is the same as the number of the downstream fields;
a second calculating module 73, configured to bring the upstream fields in the running water data into the first model to obtain second fields, where the second fields are downstream fields corresponding to the upstream fields in the running water data, derived by the upstream fields in the running water data through the mapping relationship in the first model;
a third calculating module 74, configured to compare whether the plurality of second fields are the same as the downstream fields in the plurality of corresponding pipelined data, and if each of the plurality of second fields is the same as the downstream field in the plurality of pipelined data, determine that the upstream field in the plurality of pipelined data is consistent with the downstream field in the plurality of pipelined data.
In a specific embodiment of the present disclosure, the first calculation module 71 includes:
a first data obtaining unit 711, configured to obtain training data, where the training data includes a plurality of the pipeline data;
a second data obtaining unit 712, configured to retrieve the flow data in one of the training data;
a first calculating unit 713, configured to bring the flow data in the training data into the first model, change the mapping relationship in the first model through a learning classification model K-nearest neighbor algorithm, and output the changed first model;
a second calculating unit 714, configured to retrieve the running water data in another unused training data, bring the running water data into the current first model, change the mapping relationship in the first model through a learning classification model K-nearest neighbor algorithm, and output the changed first model until all the running water data in the training data is used;
a third calculating unit 715, configured to output the trained first model.
In a specific embodiment of the present disclosure, the second calculating unit 714 includes:
a first calculating subunit 7141, configured to sequentially retrieve the flow data in the training data, and bring the flow data in the training data into the current first model, respectively, to obtain a plurality of second fields corresponding to the plurality of flow data in the training data one to one;
a second calculating subunit 7142, configured to detect whether the second field corresponding to the pipeline data in the training data is the same as the downstream field corresponding to the pipeline data in the training data, respectively, output the trained first model if the plurality of second fields in the training data are the same as the downstream field corresponding to the training data, return the pipeline data in the training data to the first model if any of the second fields in the training data is not the same as the downstream field corresponding to the training data, change the mapping relationship in the first model by using a learning classification model K nearest neighbor algorithm, and output the changed first model.
In a specific embodiment of the present disclosure, the first calculation module 71 includes:
a third data obtaining unit 716, configured to obtain mapping data, where the mapping data includes one upstream field and a plurality of corresponding downstream fields;
a fourth calculating unit 717, configured to establish a mapping relationship between one of the upstream fields in the mapping data and one of the downstream fields in the mapping data one by one until the upstream field in the mapping data and the downstream fields in the plurality of mapping data all have a mapping relationship, and obtain a first sub-model corresponding to the upstream field in the mapping data, where the first sub-model includes a mapping relationship between the upstream field and the plurality of downstream fields;
a fifth calculating unit 718, configured to obtain another upstream field that is not mapped, and execute the one-by-one mapping between one upstream field in the mapping data and one downstream field in the mapping data until the upstream field in the mapping data and the downstream fields in the multiple mapping data have a mapping, so as to obtain a first sub-model until each upstream field corresponds to one first sub-model;
a sixth calculating unit 719, configured to output a plurality of the first sub-models, where the first model is composed of a plurality of the first sub-models.
In a specific embodiment of the present disclosure, the second calculating module 73 includes:
a seventh calculating unit 731, configured to bring the upstream field in each of the pipeline data into the first model, find one of the first sub-models corresponding to the upstream field in the pipeline data, and assign the downstream field in the corresponding first sub-model to the second field;
an eighth calculating unit 732, configured to output a plurality of the second fields.
In one embodiment of the present disclosure, the third calculation module 74 includes:
a ninth calculating unit 741, configured to send an alert instruction if any one of the second fields is different from the corresponding downstream fields in the plurality of running water data, where the alert instruction is an alert prompt that the running water data needs to be manually verified;
a tenth calculation unit 742 for backing up the pipelined data that is manually verified and the upstream field is consistent with the downstream field.
It should be noted that, regarding the apparatus in the above embodiment, the specific manner in which each module performs the operation has been described in detail in the embodiment related to the method, and will not be elaborated herein.
Example 3
Corresponding to the above method embodiment, the embodiment of the present disclosure further provides an inter-system data verification device, and a reference may be made to the inter-system data verification device and the inter-system data verification method described above in a corresponding manner.
Fig. 3 is a block diagram illustrating an inter-system data verification apparatus 800, according to an example embodiment. As shown in fig. 3, the inter-system data verification apparatus 800 may include: a processor 801, a memory 802. The inter-system data verification device 800 may also include one or more of a multimedia component 803, an input/output (I/O) interface 804, and a communication component 805.
The processor 801 is configured to control the overall operation of the inter-system data verification apparatus 800, so as to complete all or part of the steps in the inter-system data verification method. The memory 402 is used to store various types of data to support the operation of the inter-system data authentication device 800, which may include, for example, instructions for any application or method operating on the inter-system data authentication device 800, as well as application-related data, such as contact data, transceived messages, pictures, audio, video, and so forth. The Memory 802 may be implemented by any type of volatile or non-volatile Memory device or combination thereof, such as Static Random Access Memory (SRAM), Electrically Erasable Programmable Read-Only Memory (EEPROM), Erasable Programmable Read-Only Memory (EPROM), Programmable Read-Only Memory (PROM), Read-Only Memory (ROM), magnetic Memory, flash Memory, magnetic disk or optical disk. The multimedia components 803 may include screen and audio components. Wherein the screen may be, for example, a touch screen and the audio component is used for outputting and/or inputting audio signals. For example, the audio component may include a microphone for receiving external audio signals. The received audio signal may further be stored in the memory 802 or transmitted through the communication component 805. The audio assembly also includes at least one speaker for outputting audio signals. The I/O interface 804 provides an interface between the processor 801 and other interface modules, such as a keyboard, mouse, buttons, etc. These buttons may be virtual buttons or physical buttons. The communication component 805 is used for wired or wireless communication between the intersystem data authentication device 800 and other devices. Wireless communication, such as Wi-Fi, bluetooth, Near Field Communication (NFC), 2G, 3G, or 4G, or a combination of one or more of them, so that the corresponding communication component 805 may include: Wi-Fi module, bluetooth module, NFC module.
In an exemplary embodiment, the inter-system data verification apparatus 800 may be implemented by one or more Application Specific Integrated Circuits (ASICs), Digital Signal Processors (DSPs), Digital Signal Processing Devices (DSPDs), Programmable Logic Devices (PLDs), Field Programmable Gate Arrays (FPGAs), controllers, microcontrollers, microprocessors or other electronic components for performing the above-described inter-system data verification method.
In another exemplary embodiment, a computer readable storage medium comprising program instructions which, when executed by a processor, implement the steps of the above-described inter-system data verification method is also provided. For example, the computer readable storage medium may be the above-mentioned memory 802 comprising program instructions that are executable by the processor 801 of the inter-system data authentication apparatus 800 to perform the above-mentioned inter-system data authentication method.
Example 4
Corresponding to the above method embodiment, the embodiment of the present disclosure further provides a readable storage medium, and a readable storage medium described below and an inter-system data verification method described above may be referred to correspondingly.
A readable storage medium, on which a computer program is stored, which, when being executed by a processor, implements the steps of the inter-system data verification method of the above-mentioned method embodiments.
The readable storage medium may be a usb disk, a removable hard disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a magnetic disk or an optical disk, and various other readable storage media capable of storing program codes.
The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention, and various modifications and changes may be made by those skilled in the art. Any modification, equivalent replacement, or improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention.
Claims (14)
1. An intersystem data verification method, comprising:
establishing a first model, wherein the first model comprises a plurality of first submodels, and the first submodel comprises a mapping relation between an upstream field and a plurality of corresponding downstream fields;
acquiring the flow data, wherein the flow data comprises a plurality of upstream fields and a plurality of downstream fields, and the number of the upstream fields in one flow data is the same as that of the downstream fields;
bringing the upstream fields in the running water data into the first model to obtain second fields, wherein the second fields are downstream fields which are derived by the upstream fields in the running water data through the mapping relation in the first model and correspond to the upstream fields in the running water data;
comparing whether the plurality of second fields are the same as the downstream fields in the plurality of corresponding pipeline data, and if each second field is the same as the downstream fields in the plurality of pipeline data, determining that the upstream fields in the plurality of pipeline data are consistent with the downstream fields in the plurality of pipeline data.
2. The method for data verification between systems according to claim 1, wherein after the establishing the first model, further comprising:
acquiring training data, wherein the training data comprises a plurality of the running data;
calling the flow data in one training data;
bringing the running water data in the training data into the first model, changing the mapping relation in the first model through a learning classification model K nearest neighbor algorithm, and outputting the changed first model;
calling the running water data in another unused training data, bringing the running water data into the current first model, changing the mapping relation in the first model through a learning classification model K nearest neighbor algorithm, and outputting the changed first model until the running water data in the training data is used;
and outputting the trained first model.
3. The method of claim 2, wherein the step of verifying the running data until all of the running data in the training data is used further comprises:
sequentially calling the flow data in the training data, and respectively bringing the flow data in the training data into the current first model to obtain a plurality of second fields corresponding to the flow data in the training data one by one;
respectively detecting whether the second field corresponding to the running water data in the training data is the same as the downstream field corresponding to the running water data in the training data, if the second fields in the training data are the same as the downstream fields corresponding to the training data, outputting the trained first model, if the second fields in any one of the training data are different from the downstream fields corresponding to the training data, returning to bring the running water data in the training data into the first model, changing the mapping relation in the first model by learning a classification model K nearest neighbor algorithm, and outputting the changed first model.
4. The method of claim 1, wherein the establishing a first model comprises:
obtaining a mapping data, said mapping data comprising one said upstream field and a plurality of corresponding said downstream fields;
establishing a mapping relation between one upstream field in the mapping data and one downstream field in the mapping data one by one until the upstream field in the mapping data and the downstream fields in the plurality of mapping data have a mapping relation, and obtaining a first sub-model corresponding to the upstream field in the mapping data, wherein the first sub-model comprises the mapping relation between the upstream field and the plurality of downstream fields;
acquiring another upstream field which is not established with a mapping relation, executing the one-by-one establishment of the mapping relation between one upstream field in the mapping data and one downstream field in the mapping data until the upstream field in the mapping data and the downstream fields in the mapping data have the mapping relation, and acquiring a first submodel until each upstream field corresponds to one first submodel;
outputting a plurality of the first submodels, wherein the first model is composed of a plurality of the first submodels.
5. The method of claim 1, wherein substituting the plurality of upstream fields into the first model to obtain a plurality of second fields comprises:
respectively bringing the upstream field in each pipeline data into the first model, finding one first sub-model corresponding to the upstream field in the pipeline data, and assigning the downstream field in the corresponding first sub-model to the second field;
outputting a plurality of the second fields.
6. The method of claim 1, wherein comparing whether the plurality of second fields are the same as the plurality of downstream fields in the pipeline data comprises:
if any one second field is different from the corresponding downstream fields in the plurality of the running water data, sending an alarm instruction, wherein the alarm instruction is an alarm prompt for manually verifying the running water data;
backing up the pipelined data that is manually verified and the upstream field is consistent with the downstream field.
7. An intersystem data verification system, comprising:
the first calculation module is used for establishing a first model, the first model comprises a plurality of first submodels, and the first submodel comprises a mapping relation between an upstream field and a plurality of corresponding downstream fields;
the device comprises a first data acquisition module, a second data acquisition module and a data processing module, wherein the first data acquisition module is used for acquiring the pipeline data, the pipeline data comprises a plurality of upstream fields and a plurality of downstream fields, and the number of the upstream fields in one pipeline data is the same as that of the downstream fields;
a second calculation module, configured to bring the upstream fields in the running water data into the first model to obtain second fields, where the second fields are downstream fields corresponding to the upstream fields in the running water data, and the downstream fields are derived by the upstream fields in the running water data through the mapping relationship in the first model;
a third calculating module, configured to compare whether the plurality of second fields are the same as the downstream fields in the plurality of corresponding pipelined data, and if each of the second fields is the same as the downstream fields in the plurality of pipelined data, determine that the upstream fields in the plurality of pipelined data are consistent with the downstream fields in the plurality of pipelined data.
8. The system for data verification between systems according to claim 7, wherein said first computing module comprises:
a first data acquisition unit, configured to acquire training data, where the training data includes a plurality of the running water data;
the second data acquisition unit is used for calling the running water data in the training data;
the first calculation unit is used for substituting the running water data in the training data into the first model, changing the mapping relation in the first model through a learning classification model K nearest neighbor algorithm, and outputting the changed first model;
the second calculation unit is used for calling the running water data in another unused training data, bringing the running water data into the current first model, changing the mapping relation in the first model through a learning classification model K neighbor algorithm, and outputting the changed first model until the running water data in the training data is used;
and the third calculation unit is used for outputting the trained first model.
9. The inter-system data verification system of claim 8, wherein the second block of computing units comprises:
the first calculation subunit is configured to sequentially retrieve the flow data in the training data, and bring the flow data in the training data into the current first model respectively to obtain a plurality of second fields corresponding to the plurality of flow data in the training data one to one;
a second calculating subunit, configured to detect whether the second field corresponding to the pipeline data in the training data is the same as the downstream field corresponding to the pipeline data in the training data, respectively, output the trained first model if the plurality of second fields in the training data are the same as the downstream fields corresponding to the training data, return to bring the pipeline data in the training data into the first model if any of the second fields in the training data is not the same as the downstream fields corresponding to the training data, change the mapping relationship in the first model by using a learning classification model K nearest neighbor algorithm, and output the changed first model.
10. The system for data verification between systems according to claim 7, wherein said first computing module comprises:
a third data obtaining unit, configured to obtain mapping data, where the mapping data includes one upstream field and a plurality of corresponding downstream fields;
a fourth calculating unit, configured to establish a mapping relationship between one upstream field in the mapping data and one downstream field in the mapping data one by one until the upstream field in the mapping data and the downstream fields in the multiple mapping data all have a mapping relationship, and obtain a first sub-model corresponding to the upstream field in the mapping data, where the first sub-model includes a mapping relationship between the upstream field and the multiple downstream fields;
a fifth calculating unit, configured to obtain another upstream field that is not mapped, and execute the one-by-one mapping between one upstream field in the mapping data and one downstream field in the mapping data until the upstream field in the mapping data and the downstream fields in the multiple mapping data have a mapping, so as to obtain a first submodel until each upstream field corresponds to one first submodel;
and the sixth calculating unit is used for outputting a plurality of first submodels, and the first model is composed of a plurality of first submodels.
11. The system for data verification between systems according to claim 7, wherein said second computing module comprises:
a seventh calculating unit, configured to bring the upstream field in each of the pipeline data into the first model, find one of the first sub-models corresponding to the upstream field in the pipeline data, and assign the downstream field in the corresponding first sub-model to the second field;
an eighth calculation unit configured to output a plurality of the second fields.
12. The system for data verification between systems according to claim 7, wherein said third computing module comprises:
a ninth calculating unit, configured to send an alert instruction if any one of the second fields is different from the corresponding downstream fields in the plurality of running water data, where the alert instruction is an alert prompt that the running water data needs to be manually verified;
a tenth computing unit for backing up the pipelined data that is manually verified and the upstream field is consistent with the downstream field.
13. An intersystem data verification apparatus, comprising:
a memory for storing a computer program;
a processor for implementing the steps of the inter-system data verification method as claimed in any one of claims 1 to 3 when executing said computer program.
14. A readable storage medium, characterized by: the readable storage medium has stored thereon a computer program which, when being executed by a processor, carries out the steps of the inter-system data authentication method according to any one of claims 1 to 3.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110833541.2A CN113569047B (en) | 2021-07-23 | 2021-07-23 | Intersystem data verification method, device, equipment and readable storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110833541.2A CN113569047B (en) | 2021-07-23 | 2021-07-23 | Intersystem data verification method, device, equipment and readable storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN113569047A true CN113569047A (en) | 2021-10-29 |
| CN113569047B CN113569047B (en) | 2024-07-12 |
Family
ID=78166500
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110833541.2A Active CN113569047B (en) | 2021-07-23 | 2021-07-23 | Intersystem data verification method, device, equipment and readable storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113569047B (en) |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20160150012A1 (en) * | 2014-11-25 | 2016-05-26 | Nimble Storage, Inc. | Content-based replication of data between storage units |
| CN109241068A (en) * | 2018-08-22 | 2019-01-18 | 中国平安人寿保险股份有限公司 | The method, apparatus and terminal device that foreground and background data compares |
| CN111061833A (en) * | 2019-12-10 | 2020-04-24 | 北京明略软件系统有限公司 | Data processing method and device, electronic equipment and computer readable storage medium |
| CN112463780A (en) * | 2020-12-02 | 2021-03-09 | 中国工商银行股份有限公司 | Data quality inspection method and device |
-
2021
- 2021-07-23 CN CN202110833541.2A patent/CN113569047B/en active Active
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20160150012A1 (en) * | 2014-11-25 | 2016-05-26 | Nimble Storage, Inc. | Content-based replication of data between storage units |
| CN109241068A (en) * | 2018-08-22 | 2019-01-18 | 中国平安人寿保险股份有限公司 | The method, apparatus and terminal device that foreground and background data compares |
| CN111061833A (en) * | 2019-12-10 | 2020-04-24 | 北京明略软件系统有限公司 | Data processing method and device, electronic equipment and computer readable storage medium |
| CN112463780A (en) * | 2020-12-02 | 2021-03-09 | 中国工商银行股份有限公司 | Data quality inspection method and device |
Also Published As
| Publication number | Publication date |
|---|---|
| CN113569047B (en) | 2024-07-12 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN108427632B (en) | Automatic test method and device | |
| CN112650685B (en) | Automatic test method, device, electronic equipment and computer storage medium | |
| CN112256593B (en) | Program processing method and device, computer equipment and readable storage medium | |
| US11743133B2 (en) | Automatic anomaly detection | |
| US20220270355A1 (en) | Distributed learning method, server and application using identification card recognition model, and identification card recognition method using the same | |
| CN112988444B (en) | Processing method, processing device and processing equipment for server cluster fault diagnosis, method for server fault diagnosis and computer-readable storage medium | |
| CN114417995A (en) | Pipe network risk early warning method and device, computer equipment and storage medium | |
| CN110865806B (en) | Code processing method, device, server and storage medium | |
| CN111241873A (en) | Image reproduction detection method, training method of model thereof, payment method and payment device | |
| CN106709335B (en) | Vulnerability detection method and device | |
| CN111309584A (en) | Data processing method and device, electronic equipment and storage medium | |
| CN112882957A (en) | Test task validity checking method and device | |
| CN113569047B (en) | Intersystem data verification method, device, equipment and readable storage medium | |
| CN112598027B (en) | Abnormality identification method and device for equipment, terminal equipment and storage medium | |
| CN115205619A (en) | Training method, detection method, device and storage medium for detection model | |
| CN113986618A (en) | Cluster brain split automatic repairing method, system, device and storage medium | |
| US10901726B2 (en) | Intelligent visual regression system | |
| CN113240088A (en) | Training method of text intention recognition model | |
| CN110765006A (en) | Flow testing method and device, computer readable storage medium and electronic device | |
| CN118012674B (en) | Back fault recovery method, cloud operating system and intelligent computing platform | |
| CN113570368B (en) | Block chain mixed signature verification method and system | |
| CN116702073B (en) | Fault event prompting method and device, electronic equipment and storage medium | |
| CN117640357A (en) | Processing method and device for network element data acquisition abnormality and related equipment | |
| CN116545632A (en) | Joint debugging tool, joint debugging method, control device and readable storage medium | |
| CN115118959A (en) | Video docking test method and system |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |