CN115357625A - Structured data comparison method and device, electronic equipment and storage medium - Google Patents
Structured data comparison method and device, electronic equipment and storage medium Download PDFInfo
- Publication number
- CN115357625A CN115357625A CN202211085688.9A CN202211085688A CN115357625A CN 115357625 A CN115357625 A CN 115357625A CN 202211085688 A CN202211085688 A CN 202211085688A CN 115357625 A CN115357625 A CN 115357625A
- Authority
- CN
- China
- Prior art keywords
- comparison
- data
- structured data
- field
- structured
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 58
- 238000012545 processing Methods 0.000 claims abstract description 34
- 238000012163 sequencing technique Methods 0.000 claims abstract description 32
- 230000006870 function Effects 0.000 claims description 33
- 238000004590 computer program Methods 0.000 claims description 10
- 238000007781 pre-processing Methods 0.000 claims description 10
- 238000006243 chemical reaction Methods 0.000 claims description 4
- 238000010586 diagram Methods 0.000 description 8
- 238000013508 migration Methods 0.000 description 8
- 230000005012 migration Effects 0.000 description 8
- 230000008859 change Effects 0.000 description 7
- 238000005516 engineering process Methods 0.000 description 4
- 238000012423 maintenance Methods 0.000 description 4
- 230000005540 biological transmission Effects 0.000 description 3
- 238000004891 communication Methods 0.000 description 3
- 230000008878 coupling Effects 0.000 description 3
- 238000010168 coupling process Methods 0.000 description 3
- 238000005859 coupling reaction Methods 0.000 description 3
- 238000012217 deletion Methods 0.000 description 3
- 230000037430 deletion Effects 0.000 description 3
- 238000013461 design Methods 0.000 description 3
- 238000011161 development Methods 0.000 description 3
- 230000018109 developmental process Effects 0.000 description 3
- 230000008569 process Effects 0.000 description 3
- 230000000694 effects Effects 0.000 description 2
- 230000014509 gene expression Effects 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 230000003287 optical effect Effects 0.000 description 2
- 238000012795 verification Methods 0.000 description 2
- 230000002159 abnormal effect Effects 0.000 description 1
- 230000009471 action Effects 0.000 description 1
- 230000032683 aging Effects 0.000 description 1
- 230000001413 cellular effect Effects 0.000 description 1
- 230000009977 dual effect Effects 0.000 description 1
- 230000000977 initiatory effect Effects 0.000 description 1
- 230000007246 mechanism Effects 0.000 description 1
- 230000002093 peripheral effect Effects 0.000 description 1
- 230000003252 repetitive effect Effects 0.000 description 1
- 230000003068 static effect Effects 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/24—Querying
- G06F16/245—Query processing
- G06F16/2455—Query execution
- G06F16/24564—Applying rules; Deductive queries
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/23—Updating
- G06F16/2365—Ensuring data consistency and integrity
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/24—Querying
- G06F16/245—Query processing
- G06F16/2453—Query optimisation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/25—Integrating or interfacing systems involving database management systems
- G06F16/258—Data format conversion from or to a database
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F40/00—Handling natural language data
- G06F40/20—Natural language analysis
- G06F40/205—Parsing
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Databases & Information Systems (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- Computational Linguistics (AREA)
- General Health & Medical Sciences (AREA)
- Health & Medical Sciences (AREA)
- Audiology, Speech & Language Pathology (AREA)
- Artificial Intelligence (AREA)
- Computer Security & Cryptography (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The application relates to the technical field of data processing, and provides a structured data comparison method, a device, an electronic device and a storage medium, wherein the method comprises the following steps: acquiring a structured data set and application scene requirements corresponding to the structured data set, wherein the structured data set comprises first structured data and second structured data, the first structured data is reference source data, and the second structured data is changed data; acquiring a field comparison rule based on application scene requirements, and analyzing the structured data set based on the field comparison rule to obtain comparison item numbers and sequencing fields; grouping the structured data sets based on the comparison item numbers to obtain at least one group of comparison data; and aiming at each group of comparison data, sequencing the comparison data based on the sequencing field, and sequentially comparing to obtain a comparison result. Therefore, the field comparison rules meeting the requirements of various application scenes can be supported, the comparison of batch data is supported, and the comparison efficiency is improved.
Description
Technical Field
The present application relates to the field of data processing technologies, and in particular, to a method and an apparatus for comparing structured data, an electronic device, and a storage medium.
Background
The structured data is also called row data, has specific fields, can be represented and stored by using a relational database, can logically express the implemented data by using a two-dimensional table structure, and needs to compare the data before and after migration when the structured data is migrated so as to ensure the correctness of the data migration work.
In the prior art, when structured data is compared, a structured data file may be loaded into a relational database such as Oracle/Mysql, and the two tables may be associated with each other in a Full Join (SQL Full Join) manner, so as to compare character strings in the two tables.
However, the above data comparison method depends on the performance of the stand-alone database, and when there is batch data for comparison, it takes a lot of time, and the comparison efficiency is poor.
Disclosure of Invention
The application provides a structured data comparison method, a structured data comparison device, an electronic device and a storage medium, which can support field comparison rules meeting requirements of various application scenes, and simultaneously support comparison of batch data, and improve comparison efficiency.
In a first aspect, the present application provides a method for structured data alignment, the method comprising:
acquiring a structured data set and application scene requirements corresponding to the structured data set, wherein the structured data set comprises first structured data and second structured data, the first structured data is reference source data, and the second structured data is changed data;
acquiring a field comparison rule based on the application scene requirement, and analyzing the structured data set based on the field comparison rule to obtain a comparison item number and a sequencing field;
grouping the structured data set based on the comparison item numbers to obtain at least one group of comparison data;
and aiming at each group of comparison data, sequencing the comparison data based on the sequencing field, and sequentially comparing to obtain a comparison result.
Optionally, analyzing the structured data set based on the field comparison rule to obtain a comparison item number and a sorting field, including:
acquiring a corresponding comparison field in the field comparison rule, wherein the comparison field comprises a type field, a function field and a number field;
and acquiring corresponding data in the structured data set based on the comparison field, and preprocessing the data based on the field comparison rule to obtain a comparison item number and a sequencing field corresponding to the data.
Optionally, preprocessing the data based on the field comparison rule to obtain a comparison item number and a sorting field corresponding to the data, including:
identifying character strings to be compared in the data based on the type fields, and splicing the character strings to be compared; the character strings to be compared comprise a main key character string and a service character string;
processing the spliced character strings to be compared based on the function fields to obtain comparison character strings;
and assembling the comparison character strings by using the number fields, and identifying comparison item numbers and sequencing fields corresponding to the assembled comparison character strings.
Optionally, grouping the structured data set based on the comparison item numbers includes:
and acquiring a primary key character string corresponding to each data in the structured data set, and grouping the structured data set based on the primary key character string and the comparison item number.
Optionally, the comparing data are sorted based on the sorting field and sequentially compared to obtain a comparison result, including:
acquiring the number of data pieces in each group of comparison data, and judging whether the number of the data pieces is more than 1;
if so, sorting the comparison data based on the sorting field, and sequentially comparing to obtain a comparison result;
if not, obtaining a comparison result based on the source of the comparison data.
Optionally, obtaining an alignment result based on the source of the alignment data includes:
if the comparison data is from first structured data, determining that the comparison data is deleted data;
and if the comparison data is from the second structured data, determining the comparison data as the newly added data.
Optionally, obtaining a structured data set and an application scenario requirement corresponding to the structured data set includes:
acquiring real-time data in different databases by using a standardized interface, and performing format conversion on the real-time data to obtain a structured data set;
and aiming at the structured data sets corresponding to different databases, acquiring the application scene requirements corresponding to each structured data set.
Optionally, the method further includes:
after the comparison result is obtained, judging whether the comparison result is consistent;
if so, comparing the structured data set again at preset time intervals, and covering the last comparison result after obtaining the comparison result so as to verify the accuracy of comparison;
and if not, generating an alarm prompt to remind the user to check the comparison result.
In a second aspect, the present application provides a structured data alignment apparatus, including:
the system comprises an acquisition module, a processing module and a processing module, wherein the acquisition module is used for acquiring a structured data set and application scene requirements corresponding to the structured data set, the structured data set comprises first structured data and second structured data, the first structured data is reference source data, and the second structured data is changed data;
the analysis module is used for acquiring a field comparison rule based on the application scene requirement, and analyzing the structured data set based on the field comparison rule to obtain a comparison item number and a sequencing field;
the grouping module is used for grouping the structured data set based on the comparison item numbers to obtain at least one group of comparison data;
and the comparison module is used for sequencing the comparison data based on the sequencing fields aiming at each group of comparison data and sequentially comparing the sequencing data to obtain a comparison result.
In a third aspect, the present application provides an electronic device, comprising: a processor, and a memory communicatively coupled to the processor;
the memory stores computer-executable instructions;
the processor executes computer-executable instructions stored by the memory to implement the method of any of the first aspects.
In a fourth aspect, the present application provides a computer-readable storage medium having stored thereon computer-executable instructions for implementing the method according to any one of the first aspect when executed by a processor.
In a fifth aspect, the present application provides a computer program product comprising program code for performing the method according to any of the first aspect when the computer program is run by a computer.
In summary, the present application provides a method, an apparatus, an electronic device, and a storage medium for comparing structured data, which may obtain a field comparison rule based on an application scene requirement by obtaining a structured data set and an application scene requirement corresponding to the structured data set, and further analyze the structured data set based on the field comparison rule to obtain a comparison item number and a sorting field; grouping the structured data sets based on the comparison item numbers to obtain at least one group of comparison data; further, for each group of comparison data, sequencing the comparison data based on a sequencing field, and sequentially comparing to obtain a comparison result, wherein the structured data set comprises first structured data and second structured data, the first structured data is reference source data, and the second structured data is changed data; therefore, the field comparison rules meeting the requirements of various application scenes can be supported, the comparison of batch data is supported, and the comparison efficiency is improved while the correctness of data comparison is ensured.
Drawings
The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the application and, together with the description, serve to explain the principles of the application.
Fig. 1 is a schematic view of an application scenario provided in an embodiment of the present application;
fig. 2 is a schematic flowchart of a method for comparing structured data according to an embodiment of the present disclosure;
FIG. 3 is a block diagram of a standardized interface provided in an embodiment of the present application;
fig. 4 is a schematic flowchart of a batch data comparison method according to an embodiment of the present application;
fig. 5 is a schematic flowchart of a real-time data comparison method according to an embodiment of the present disclosure;
fig. 6 is a schematic structural diagram of a structured data comparing apparatus according to an embodiment of the present application;
fig. 7 is a schematic structural diagram of an electronic device according to an embodiment of the present application.
With the above figures, there are shown specific embodiments of the present application, which will be described in more detail below. These drawings and written description are not intended to limit the scope of the inventive concepts in any manner, but rather to illustrate the inventive concepts to those skilled in the art by reference to specific embodiments.
Detailed Description
Reference will now be made in detail to the exemplary embodiments, examples of which are illustrated in the accompanying drawings. The following description refers to the accompanying drawings in which the same numbers in different drawings represent the same or similar elements unless otherwise indicated. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present application. Rather, they are merely examples of apparatus and methods consistent with certain aspects of the present application, as detailed in the appended claims.
In the embodiments of the present application, terms such as "first" and "second" are used to distinguish the same or similar items having substantially the same function and action. For example, the first device and the second device are only used for distinguishing different devices, and the sequence order thereof is not limited. Those skilled in the art will appreciate that the terms "first," "second," etc. do not denote any order or quantity, nor do the terms "first," "second," etc. denote any order or importance.
It is noted that, in the present application, words such as "exemplary" or "for example" are used to mean exemplary, illustrative, or descriptive. Any embodiment or design described herein as "exemplary" or "e.g.," is not necessarily to be construed as preferred or advantageous over other embodiments or designs. Rather, use of the word "exemplary" or "such as" is intended to present concepts related in a concrete fashion.
In the present application, "at least one" means one or more, "a plurality" means two or more. "and/or" describes the association relationship of the associated objects, meaning that there may be three relationships, e.g., a and/or B, which may mean: a exists alone, A and B exist simultaneously, and B exists alone, wherein A and B can be singular or plural. The character "/" generally indicates that the former and latter associated objects are in an "or" relationship. "at least one of the following" or similar expressions refer to any combination of these items, including any combination of the singular or plural items. For example, at least one (one) of a, b, or c, may represent: a, b, c, a-b, a-c, b-c, or a-b-c, wherein a, b, c may be single or multiple.
The application is described below with reference to the accompanying drawings, and fig. 1 is a schematic view of an application scenario provided by an embodiment of the application, and a structured data comparison method provided by the application may be applied to the application scenario shown in fig. 1. The application scenario includes: a first database 1011, a second database 1012, a data processing platform 102 and a display terminal 103; when data in the first database 1011 is migrated to the second database 1012, the data before and after the migration needs to be compared.
Specifically, the data processing platform 102 may obtain the reference source data from the first database 1011, obtain the migrated data from the second database 1012, and correspondingly obtain a configuration file corresponding to the application scenario, where the configuration file is a manually pre-configured field comparison rule applicable to the application scenario, and the configuration file includes a comparison field and a comparison rule that need to be performed for data comparison, and further, the data processing platform 102 parses the configuration file to obtain the field comparison rule, and compares the reference source data obtained from the first database 1011 with the migrated data obtained from the second database 1012 by using the field comparison rule to obtain a comparison result, records the data that are not compared, generates an alarm prompt, and sends the alarm prompt to the display terminal 103, so that the user operation display terminal 103 searches for the data that are not compared to perform verification.
It should be noted that the display terminal 103 may be a display device corresponding to the data processing platform 102, or may also be a terminal device of a user, which is not specifically limited in this embodiment, in this application, the data processing platform 102 may compare data from multiple databases, or compare changed data in one database, and is not limited to the above two databases for data comparison in a data migration scenario, and the number of databases and an application scenario are not specifically limited in this embodiment.
The terminal device may be a wireless terminal or a wired terminal. A wireless terminal may refer to a device that provides voice and/or other traffic data connectivity to a user, a handheld device having wireless connection capability, or other processing device connected to a wireless modem. A wireless terminal, which may be a mobile terminal such as a mobile phone (or called a "cellular" phone) and a computer having a mobile terminal, for example, a portable, pocket, hand-held, computer-included or vehicle-mounted mobile device, may communicate with one or more core Network devices via a Radio Access Network (RAN), and exchange languages and/or data with the RAN. For another example, the Wireless terminal may be a Personal Communication Service (PCS) phone, a cordless phone, a Session Initiation Protocol (SIP) phone, a Wireless Local Loop (WLL) station, a Personal Digital Assistant (PDA), and the like. A wireless Terminal may also be referred to as a system, a Subscriber Unit (Subscriber Unit), a Subscriber Station (Subscriber Station), a Mobile Station (Mobile), a Remote Station (Remote Station), a Remote Terminal (Remote Terminal), an Access Terminal (Access Terminal), a User Terminal (User Terminal), a User Agent (User Agent), and a User Device or User Equipment (User Equipment), which are not limited herein. Optionally, the terminal device may also be a smart phone, a tablet computer, or the like.
Therefore, the comparison of the structured data can be applied to scenes such as data migration or online maintenance transaction, and particularly, when the structured data is migrated, the data before and after migration needs to be compared to ensure the correctness of data migration work; when the transaction is maintained online, the difference before and after the data modification needs to be seen at the front end in real time, that is, the difference before and after the data modification needs to be recorded when the transaction is maintained online.
In a possible implementation manner, when structured data is compared, structured data files can be loaded into a relational database such as Oracle/Mysql and the like, two tables are associated in a full-connection manner, and then character strings in the two tables are compared.
However, the above data comparison method needs to load the file into a library, and depends on the performance of a stand-alone database, and when there is batch data for comparison, it takes a lot of time, and the comparison efficiency is poor.
It should be noted that when there is batch data to perform comparison, multiple comparison rules are also needed to meet different comparison requirements, but the implementation of complex comparison rules has a certain threshold for developers, and for the requirement of real-time comparison, if the comparison is implemented based on online transaction, it needs to be bound with transaction service logic, which makes online transaction very complex.
In order to solve the above problems, the present application provides a structured data comparison method, which uses a Spark program for development, that is, a single-machine database is replaced by a distributed technology, field comparison rules under different application scenarios are loaded and analyzed through the Spark program, and then a structured data set is analyzed based on the field comparison rules to obtain comparison item numbers and sorting fields, the structured data set includes reference source data and changed data, further, the structured data set is grouped by using the comparison item numbers to obtain at least one set of comparison data, further, the comparison data in the same group is sorted based on the sorting fields to perform data comparison, that is, the comparison data in the same record and the same comparison item number are put together and sorted for comparison, so that field comparison rules meeting requirements of various application scenarios can be supported, thereby supporting flexible comparison of batch data and improving comparison efficiency.
The technical solution of the present application will be described in detail below with specific examples. These several specific embodiments may be combined with each other below, and details of the same or similar concepts or processes may not be repeated in some embodiments. Embodiments of the present application will be described below with reference to the accompanying drawings.
Fig. 2 is a schematic flowchart of a method for comparing structured data according to an embodiment of the present application, where as shown in fig. 2, the method for comparing structured data includes the following steps:
s201, acquiring a structured data set and application scene requirements corresponding to the structured data set, wherein the structured data set comprises first structured data and second structured data, the first structured data is reference source data, and the second structured data is changed data.
In this embodiment of the present application, the first structured data and the second structured data may be from the same database or different databases, the structured data may be a database table or a data file, the first structured data is reference source data, that is, original data stored in the database, and the second structured data is changed data, such as data after addition, deletion, update, and migration, which is not specifically limited in this embodiment of the present application.
Optionally, the second structured data may be change data captured in real time from a redolog log by a real-time data change acquisition component, the change data includes a full field recorded after addition, deletion and update, and then the first structured data includes a full field recorded before addition, deletion and update, so that the application can acquire the real-time data and compare the real-time data.
S202, field comparison rules are obtained based on the application scene requirements, and the structured data set is analyzed based on the field comparison rules to obtain comparison item numbers and sequencing fields.
In the embodiment of the application, the field comparison rule is a configuration file defined in advance, different application scenes correspond to different configuration files, the configuration file comprises a comparison field and a comparison rule, the comparison field is a field type corresponding to data in a structured data set, such as a field name, a type field, a function field, a number field and the like, the comparison rule can be that the data are processed by using the obtained function field and then are sequentially compared, and/or the data are combined by using the number field and then are sequentially compared, and/or the data corresponding to a certain type field are sequentially compared and equally.
In this step, the structured data set is analyzed based on the field comparison rule, that is, a corresponding comparison field in the field comparison rule is analyzed, and a comparison item number and a sequencing field corresponding to the structured data set are obtained by using the comparison field, where the comparison item number is used to mark a field to be compared with comparison data, and if 200010 indicates that 9 th to 10 th fields in a piece of data need to be compared; the sorting field may refer to a field that may be used for sorting in the data, such as an operator number, a mechanism number, an update timestamp, and the like, and may be designed by user, which is not specifically limited in this embodiment of the present application.
S203, grouping the structured data sets based on the comparison item numbers to obtain at least one group of comparison data.
In this step, the structured data sets are grouped based on the comparison item numbers, that is, the character strings to be compared are assembled based on the comparison item numbers, and further, the comparison data of the same record and the same comparison item number are grouped into one group to obtain at least one group of comparison data.
S204, aiming at each group of comparison data, sorting the comparison data based on the sorting fields, and sequentially comparing to obtain comparison results.
In this step, the comparison data is sorted based on the sorting field, for example, the data update time stamp corresponding to each group of structured data sets sorts the data in each group according to the sequence based on the data update time stamp, and further, the sorted data is sequentially compared to obtain a comparison result; the comparison result comprises a result with consistent data and a result with inconsistent data, and the inconsistent data can be stored aiming at the result with inconsistent data so as to facilitate subsequent query.
Therefore, the application provides a structured data comparison method, which can obtain a structured data set and an application scene requirement corresponding to the structured data set, and obtain a field comparison rule based on the application scene requirement, and further analyze the structured data set based on the field comparison rule to obtain a comparison item number and a sequencing field; grouping the structured data sets based on the comparison item numbers to obtain at least one group of comparison data; further, for each group of comparison data, sorting the comparison data based on the sorting field, and sequentially comparing to obtain a comparison result, wherein the structured data set comprises reference source data and changed data; therefore, the structured data comparison method can support the field comparison rules meeting the requirements of various application scenes to carry out data comparison, and simultaneously supports the comparison of batch data, thereby ensuring the correctness of data comparison and improving the comparison efficiency.
Optionally, analyzing the structured data set based on the field comparison rule to obtain a comparison item number and a sorting field, including:
acquiring a corresponding comparison field in the field comparison rule, wherein the comparison field comprises a type field, a function field and a number field;
and acquiring corresponding data in the structured data set based on the comparison field, and preprocessing the data based on the field comparison rule to obtain a comparison item number and a sequencing field corresponding to the data.
In this embodiment of the present application, the comparison field may include a type field, a function field, and a number field, where the type field is used to mark the type of field comparison, such as Y, N, K, S, C, HB, and D, where Y indicates that the field is used for comparison, N indicates that the field is not used for comparison, K indicates that the field is used as a primary key, C indicates that the field is a fixed field and can be designed by human-defined definition, S indicates that the field corresponds to a comparison item number, HB indicates that the field participates in comparison as part of a joint field, and D indicates that the field is a sorting field; the function field may refer to a function expression for processing a field, such as trim, to _ char, lower, and the like, where the to _ char indicates that the data type of the field is converted into a character type, the trim indicates that spaces in the data of the field are deleted, the trim indicates that the data of the field is converted into a lower-case character string, and if there is no function field, the trim indicates that the field is not required to be processed; the number field is used for identifying the comparison result of each service field, and the comparison result can be screened according to the number, for example, the number of the comparison item of the 9 th to 10 th bytes of a certain data is 200010.
For example, comparison fields corresponding to field comparison rules in the embodiment of the present application may be as shown in table 1, and based on the comparison fields, field comparison rules meeting requirements of different application scenarios may be formed:
TABLE 1
The first column represents english names of fields corresponding to different fields in the data, which is only described above by way of example, and the english names of fields corresponding to fields in the data may also be represented in other forms.
It can be understood that, if data in the structured data set corresponds to a function field, a function corresponding to the function field may be used to perform function processing on the field, and further, the data after function processing is used for data comparison, optionally, a common string processing function is preset in the field comparison rule, and meanwhile, the field comparison rule may also support a user to define a function to extend a function type corresponding to the function field, which is not specifically limited in the embodiment of the present application.
Therefore, the embodiment of the application supports complex and various field comparison rules, and can preprocess data by acquiring the comparison fields in the field comparison rules, thereby improving the processing flexibility.
Optionally, preprocessing the data based on the field comparison rule to obtain a comparison item number and a sorting field corresponding to the data, including:
identifying character strings to be compared in the data based on the type fields, and splicing the character strings to be compared; the character strings to be compared comprise a main key character string and a service character string;
processing the spliced character strings to be compared based on the function fields to obtain comparison character strings;
and assembling the comparison character strings by using the number fields, and identifying the number and the sequencing fields of the comparison items corresponding to the assembled comparison character strings.
In this embodiment, the primary key character string may refer to character string data corresponding to the primary key, such as cst _ id, and the service character string refers to character string data corresponding to a plurality of service fields constituting a certain piece of data, such as a plurality of service fields whose contact addresses include country, province, city, county, detailed address, etc., that is, country a, country B, city C, city D, county E.
Optionally, if the data is composed of a plurality of service fields, each field is separated by | @ | such as data corresponding to a contact address: therefore, the present application supports merging of a plurality of business fields as one field for comparison, but if any one of the merged fields changes, the merged field as a whole needs to be used as an output result for data comparison.
It can be understood that a piece of data can be divided into a plurality of service fields according to bytes for comparison, so that a certain service field in a piece of data can represent one service meaning, and other adjacent service fields can represent other service meanings.
In this step, the character strings to be compared in the data are identified based on the type fields, and unnecessary comparison information can be removed, for example, the main key fields in the data are identified and the main key character strings are spliced based on the identified main key fields; further, the number field is utilized to assemble the comparison character strings, if each comparison item is numbered, the character strings to be compared are assembled, and then the comparison item number and the sequencing field corresponding to the assembled comparison character strings are identified; and fixed output fields corresponding to the comparison character strings can be identified for data comparison, so that the comparison accuracy is improved.
Therefore, the embodiment of the application can support complex comparison logic, for example, data can be divided into a plurality of service fields for comparison, a support function processes the data, unnecessary comparison information can be removed, and the application scene is wide.
Optionally, grouping the structured data set based on the comparison item numbers includes:
and acquiring a primary key character string corresponding to each data in the structured data set, and grouping the structured data set based on the primary key character string and the comparison item number.
It is to be appreciated that the present application can group the structured data set based on the primary key string.
Therefore, the structured data set can be grouped in a mode of combining the primary key character string and the comparison item number, and the structured data set can also be grouped only on the basis of the primary key character string or the comparison item number, so that the grouping accuracy is improved.
Optionally, the sorting the comparison data based on the sorting field, and sequentially comparing to obtain a comparison result, including:
acquiring the number of data in each group of comparison data, and judging whether the number of the data is more than 1;
if yes, sorting the comparison data based on the sorting field, and sequentially comparing to obtain a comparison result;
if not, obtaining a comparison result based on the source of the comparison data.
For example, in the application scenario of fig. 1, after the data processing platform 102 groups the reference source data acquired from the first database 1011 and the migrated data acquired from the second database 1012 based on the comparison item numbers to obtain at least one set of comparison data, the data processing platform 102 may acquire the number N of data pieces in each set of comparison data, and determine whether the number N of data pieces is greater than 1; if so, sorting the comparison data based on the sorting field, and sequentially comparing to obtain a comparison result; if not, a comparison result is obtained from the first database 1011 or the second database 1012 based on the comparison data, where N is a positive integer greater than or equal to 1.
It can be understood that preset requirements can also be set in the embodiment of the present application, and whether the preset requirements are met is determined based on the number of data pieces, so as to determine whether the comparison data needs to be sorted and then compared, so as to obtain a comparison result.
Therefore, the embodiment of the application can be used for sorting and comparing the data by using the sorting field based on whether the number of the data pieces in each group of comparison data meets the preset requirement, so as to obtain the comparison result, and when the number of the data pieces does not meet the preset requirement, the comparison result can be directly determined according to the source of the comparison data, so that the comparison efficiency is improved.
Optionally, obtaining an alignment result based on the source of the alignment data includes:
if the comparison data is from the first structured data, determining the comparison data as deleted data;
and if the comparison data is from the second structured data, determining the comparison data as the newly added data.
In this step, if it is determined that the comparison data is from the first structured data but not present in the second structured data, it is determined that the comparison data is deleted data; and if the comparison data is determined to be from the second structured data but not from the first structured data, the comparison data is the newly added data.
Therefore, the comparison result can be directly obtained based on the source of the comparison data, and the timeliness of the comparison is improved.
Optionally, obtaining a structured data set and an application scenario requirement corresponding to the structured data set includes:
acquiring real-time data in different databases by using a standardized interface, and performing format conversion on the real-time data to obtain a structured data set;
and aiming at the structured data sets corresponding to different databases, acquiring the application scene requirements corresponding to each structured data set.
In the embodiment of the application, the standardized interface may refer to a designed unified interface for calling data in different databases, and the standardized interface is a code frame defined in advance, and when the data in different databases are called, only corresponding parameter values in the code frame need to be modified.
In this step, the Kafka message middleware can be used to read the real-time changed data in the database, but the results of analyzing the changed data from different database products are very different, so that the changed data needs to be standardized, that is, the format of the acquired real-time data needs to be converted to obtain a structured data set for data comparison.
The data in batch data comparison is in a text format, the standardized output in the real-time data comparison process is also in the text format, and the text file formats of the data and the text file formats are consistent.
For example, fig. 3 is a schematic diagram of a framework of a standardized interface provided in an embodiment of the present application; as shown in fig. 3, the embodiment of the present application provides a unified standardized interface, which is used for calling change data of each database, where each change data corresponds to a corresponding implementation class, and the calling of each implementation class needs to be implemented based on the standardized interface; the implementation classes corresponding to the change data of the databases such as Mysql, oracle, DB2, oceanBase, etc. include: the data type, processinsertRec, processdeleterrec, processUpdateNotKeyRec (updating non-primary key common field), and processUpdateKeyRec (updating primary key field) are realized based on a standardized interface, and therefore, for various types of databases, a user only needs to realize the standardized interface.
It should be noted that, different implementation classes in the database are called to correspond to different comparison results, taking an Oracle database as an example, the comparison result of each dml operation includes a transaction start mark and a transaction end mark, and only one result statement is added/deleted; updating the comparison result corresponding to the non-primary key common field to be the previous record or the next record; the comparison results corresponding to the update primary key field are V, D, I three statements, while for the update primary key field of the OceanBase database, the corresponding comparison results are a delete operation, an insert operation, and other dml operations, the corresponding comparison result is a record containing information before and after change, and different implementation classes in different databases correspond to different comparison results, which is not described in detail herein.
It can be understood that the real-time comparison usually requires on-line maintenance transaction or a special comparison program at the back end to perform customized comparison on data changed each time, and the development and maintenance cost is very high, and the repetitive workload is large.
Optionally, the method further includes:
after the comparison result is obtained, judging whether the comparison result is consistent;
if so, comparing the structured data set again at preset time intervals, and covering the last comparison result after obtaining the comparison result so as to verify the accuracy of comparison;
if not, generating an alarm prompt to remind the user to check the comparison result.
In this application embodiment, the preset time may refer to a set time for performing the comparison again, a specific numerical value corresponding to the preset time is not limited in this application embodiment, and may be one day, and the content and the transmission form of the alarm prompt are not specifically limited in this application embodiment, and the alarm prompt may be generated based on inconsistent data, for example, the alarm prompt is "data 1 in table a is inconsistent with data 2 in table B", and the transmission form may be a short message, a display frame, or the like.
It should be noted that although the real-time data comparison makes up for the disadvantage of poor timeliness of batch processing, there is also a case that the real-time data may be lost, so that, in the present application, after the interval is preset, the data comparison is executed again, and the comparison result obtained covers the last comparison result, that is, the data reading application is realized through the database idempotent method to have no sense of repeated comparison, thereby ensuring the timeliness of the data and ensuring the integrity of the data.
Therefore, the embodiment of the application reduces the situation that the real-time data is possibly lost through a re-execution mode, improves the data integrity while improving the data timeliness, and can also generate an alarm prompt to prompt a user of the abnormal situation of the comparison result so as to facilitate the follow-up data calling for verification and improve the convenience.
With reference to the foregoing embodiment, the structured data comparison method provided by the present application supports two data comparison manners, namely real-time comparison implemented based on a flink datastream technology and batch comparison implemented based on a flink dataset technology, where real-time comparison logic and batch comparison logic are consistent, comparison calibers are consistent, and both of the real-time comparison logic and the batch comparison logic share a core code, and specifically, fig. 4 is a schematic flow diagram of the batch data comparison method provided by the embodiment of the present application; as shown in fig. 4, the batch data comparison method includes the following steps:
step 1: starting a Spark program, loading and analyzing a configuration file (application scene requirement), reading data in the file A and the file B from the data file, and/or reading data in the table a and the table B from the database table, analyzing the data according to the configuration file, and executing the step 2.
It should be noted that, the data in the file a and the table a are reference source data, and the data in the file B and the table B are changed data, in this embodiment of the present application, the data in the file a and the file B may be subjected to comparison processing alone, the data in the table a and the table B may be subjected to comparison processing alone, the data in the file a and the file B may be subjected to comparison processing separately, and the data in the table a and the table B may be subjected to comparison processing separately, which is not limited in this embodiment of the present application.
Step 2: merging two parts of data in the file A and the file B, and/or merging two parts of data in the table a and the table B, grouping input data (structured data sets) according to a main key and a comparison item number to obtain at least one group of comparison data, and sequencing the comparison data in the same group according to a data update timestamp, wherein the number of data pieces in the same group is marked as N, further determining a comparison result according to the number of the data pieces N, wherein N is a positive integer greater than or equal to 1, and further executing the step 3.
And 3, step 3: if the number of the data is 1, and the data is determined to be from the file A and/or the table a, and the file B and/or the table B does not have the data, it is indicated that the record corresponding to the data is deleted, if the data is determined to be from the file B and/or the table B, and the file A and/or the table a does not have the data, it is indicated that the record corresponding to the data is newly added, and if the number of the data is greater than 1, the sorted results can be sequentially compared to obtain a comparison result, wherein the data corresponding to the compared inconsistent items can be recorded in the comparison result for the user to check.
Fig. 5 is a schematic flowchart of a real-time data comparison method according to an embodiment of the present disclosure; as shown in fig. 5, the real-time data comparison method includes the following steps:
the method comprises the following steps: the method comprises the steps of starting a Spark Streaming program, loading and analyzing a configuration file (application scene requirement), calling a unified standardized interface by utilizing Kafka message middleware to collect changed data and original data in an Oracle database in real time, standardizing changed records into json character strings in a unified format in order to adapt to various database products, correspondingly, further unifying values of data updating types, and adapting records for updating a main key, namely converting the changed data and the original data into the unified format, wherein the data in the Oracle database is modified and then called changed data, and further subsequent steps are executed.
It should be noted that the subsequent steps are similar to step 2 and step 3 in the embodiment shown in fig. 4, and details can be referred to the description of the embodiment shown in fig. 4, and are not described here.
This application supports real-time and batch dual mode to the comparison of structured data, all supports complicated diversified field to compare the rule with the form of configuration file, and study and use threshold are lower for the user need not repeated development, and this application still presets commonly used string processing function in addition, supports self-defined function with spread function processing function, and flows to carry out the data comparison in the mode of combining in batches, when reducing later stage operation and maintenance cost, promotes the ageing.
In the foregoing embodiment, the method for comparing structured data provided in the embodiment of the present application is described, and in order to implement each function in the method provided in the embodiment of the present application, the electronic device serving as an execution subject may include a hardware structure and/or a software module, and implement each function in the form of a hardware structure, a software module, or a hardware structure plus a software module. Whether any of the above-described functions is implemented as a hardware structure, a software module, or a hardware structure plus a software module depends upon the particular application and design constraints imposed on the technical solution.
For example, fig. 6 is a schematic structural diagram of a structured data comparing apparatus according to an embodiment of the present application, and as shown in fig. 6, the apparatus includes: an acquisition module 610, an analysis module 620, a grouping module 630 and a comparison module 640; the obtaining module 610 is configured to obtain a structured data set and an application scenario requirement corresponding to the structured data set, where the structured data set includes first structured data and second structured data, the first structured data is reference source data, and the second structured data is changed data;
the parsing module 620 is configured to obtain a field comparison rule based on the application scenario requirement, and parse the structured data set based on the field comparison rule to obtain a comparison item number and a sorting field;
the grouping module 630 is configured to group the structured data set based on the comparison item numbers to obtain at least one group of comparison data;
the comparison module 640 is configured to sort, for each group of comparison data, the comparison data based on the sorting field, and compare the comparison data in sequence to obtain a comparison result.
Optionally, the parsing module 620 includes an obtaining unit and a preprocessing unit;
specifically, the obtaining unit is configured to obtain a corresponding comparison field in the field comparison rule, where the comparison field includes a type field, a function field, and a number field;
the preprocessing unit is used for acquiring corresponding data in the structured data set based on the comparison fields, and preprocessing the data based on the field comparison rule to obtain comparison item numbers and sequencing fields corresponding to the data.
Optionally, the preprocessing unit is specifically configured to:
identifying character strings to be compared in the data based on the type fields, and splicing the character strings to be compared; the character strings to be compared comprise a main key character string and a service character string;
processing the spliced character strings to be compared based on the function fields to obtain comparison character strings;
and assembling the comparison character strings by using the number fields, and identifying comparison item numbers and sequencing fields corresponding to the assembled comparison character strings.
Optionally, the grouping module 630 is specifically configured to:
and acquiring a primary key character string corresponding to each data in the structured data set, and grouping the structured data set based on the primary key character string and the comparison item number.
Optionally, the comparing module 640 includes a determining unit, a comparing unit and a determining unit;
specifically, the judging unit is configured to obtain the number of data pieces in each group of comparison data, and judge whether the number of data pieces is greater than 1;
the comparison unit is used for sorting the comparison data based on the sorting field when the number of the data pieces is more than 1, and sequentially comparing the data pieces to obtain a comparison result;
the determining unit is configured to obtain an alignment result based on a source of the alignment data when the number of the data pieces is equal to 1.
Optionally, the determining unit is specifically configured to:
if the comparison data is from the first structured data, determining the comparison data as deleted data;
and if the comparison data come from the second structured data, determining that the comparison data are newly added data.
Optionally, the obtaining module 610 is specifically configured to:
acquiring real-time data in different databases by using a standardized interface, and performing format conversion on the real-time data to obtain a structured data set;
and aiming at the structured data sets corresponding to different databases, acquiring the application scene requirements corresponding to each structured data set.
Optionally, the apparatus further includes a determining module, where the determining module is configured to:
after the comparison result is obtained, judging whether the comparison result is consistent;
if so, comparing the structured data set again at preset time intervals, and covering the last comparison result after obtaining the comparison result so as to verify the accuracy of comparison;
if not, generating an alarm prompt to remind the user to check the comparison result.
For a specific implementation principle and effect of the structured data comparison device provided in the embodiment of the present application, reference may be made to the relevant description and effect corresponding to the above embodiment, which is not described herein in any greater detail.
An embodiment of the present application further provides a schematic structural diagram of an electronic device, and fig. 7 is a schematic structural diagram of an electronic device provided in an embodiment of the present application, and as shown in fig. 7, the electronic device may include: a processor 701 and a memory 702 communicatively coupled to the processor; the memory 702 stores computer programs; the processor 701 executes the computer program stored in the memory 702, so that the processor 701 executes the method according to any of the embodiments.
The memory 702 and the processor 701 may be connected by a bus 703.
Embodiments of the present application further provide a computer-readable storage medium, in which computer program execution instructions are stored, and the computer program execution instructions, when executed by a processor, are used to implement the method according to any of the foregoing embodiments of the present application.
The embodiment of the present application further provides a chip for executing the instruction, where the chip is used to execute the method in any of the foregoing embodiments executed by the electronic device in any of the foregoing embodiments of the present application.
Embodiments of the present application further provide a computer program product, which includes program code for performing the method described in any of the foregoing embodiments performed by the electronic device in any of the foregoing embodiments of the present application when the computer program is run by a computer.
In the technical scheme of the application, the collection, storage, use, processing, transmission, provision, disclosure and other processing of the related information such as financial data or user data and the like all accord with the regulations of related laws and regulations and do not violate the good custom of the public order.
In the several embodiments provided in the present application, it should be understood that the disclosed apparatus and method may be implemented in other manners. For example, the above-described apparatus embodiments are merely illustrative, and for example, a division of modules is merely a division of logical functions, and an actual implementation may have another division, for example, a plurality of modules or components may be combined or integrated into another system, or some features may be omitted, or not executed. In addition, the shown or discussed coupling or direct coupling or communication connection between each other may be through some interfaces, indirect coupling or communication connection between devices or modules, and may be in an electrical, mechanical or other form.
Modules described as separate parts may or may not be physically separate, and parts displayed as modules may or may not be physical units, may be located in one position, or may be distributed on a plurality of network units. Some or all of the modules may be selected according to actual needs to implement the solution of the present embodiment.
In addition, functional modules in the embodiments of the present application may be integrated into one processing unit, or each module may exist alone physically, or two or more modules are integrated into one unit. The unit formed by the modules can be realized in a hardware form, and can also be realized in a form of hardware and a software functional unit.
The integrated module implemented in the form of a software functional module may be stored in a computer-readable storage medium. The software functional module is stored in a storage medium and includes several instructions for causing a computer device (which may be a personal computer, a server, or a network device) or a processor to execute some steps of the methods described in the embodiments of the present application.
It should be understood that the Processor may be a Central Processing Unit (CPU), other general purpose Processor, a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), etc. A general purpose processor may be a microprocessor or the processor may be any conventional processor or the like. The steps of the methods disclosed in the incorporated application may be directly implemented by a hardware processor, or may be implemented by a combination of hardware and software modules in the processor.
The Memory may include a Random Access Memory (RAM), and may further include a Non-volatile Memory (NVM), for example, at least one magnetic disk Memory, and may also be a usb disk, a removable hard disk, a read-only Memory, a magnetic disk or an optical disk.
The bus may be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, an Extended ISA (EISA) bus, or the like. The bus may be divided into an address bus, a data bus, a control bus, etc. For ease of illustration, the buses in the figures of the present application are not limited to only one bus or one type of bus.
The storage medium may be implemented by any type of volatile or non-volatile Memory device or combination thereof, such as Static Random Access Memory (SRAM), electrically Erasable Programmable Read Only Memory (EEPROM), erasable Programmable Read-Only Memory (EPROM), programmable Read-Only Memory (PROM), read-Only Memory (ROM), magnetic Memory, flash Memory, magnetic disk or optical disk. A storage media may be any available media that can be accessed by a general purpose or special purpose computer.
An exemplary storage medium is coupled to the processor such the processor can read information from, and write information to, the storage medium. Of course, the storage medium may also be integral to the processor. The processor and the storage medium may reside in an Application Specific Integrated Circuits (ASIC). Of course, the processor and the storage medium may reside as discrete components in an electronic device or host device.
The above description is only a specific implementation of the embodiments of the present application, but the scope of the embodiments of the present application is not limited thereto, and any changes or substitutions within the technical scope disclosed in the embodiments of the present application should be covered by the scope of the embodiments of the present application. Therefore, the protection scope of the embodiments of the present application shall be subject to the protection scope of the claims.
Claims (12)
1. A method for comparing structured data, the method comprising:
acquiring a structured data set and application scene requirements corresponding to the structured data set, wherein the structured data set comprises first structured data and second structured data, the first structured data is reference source data, and the second structured data is changed data;
acquiring a field comparison rule based on the application scene requirement, and analyzing the structured data set based on the field comparison rule to obtain a comparison item number and a sequencing field;
grouping the structured data set based on the comparison item numbers to obtain at least one group of comparison data;
and aiming at each group of comparison data, sequencing the comparison data based on the sequencing field, and sequentially comparing to obtain a comparison result.
2. The method of claim 1, wherein parsing the structured data set based on the field alignment rule to obtain an alignment number and a ranking field comprises:
acquiring a corresponding comparison field in the field comparison rule, wherein the comparison field comprises a type field, a function field and a number field;
and acquiring corresponding data in the structured data set based on the comparison field, and preprocessing the data based on the field comparison rule to obtain a comparison item number and a sequencing field corresponding to the data.
3. The method of claim 2, wherein preprocessing the data based on the field alignment rule to obtain an alignment number and a sorting field corresponding to the data comprises:
identifying character strings to be compared in the data based on the type fields, and splicing the character strings to be compared; the character strings to be compared comprise a main key character string and a service character string;
processing the spliced character strings to be compared based on the function fields to obtain comparison character strings;
and assembling the comparison character strings by using the number fields, and identifying comparison item numbers and sequencing fields corresponding to the assembled comparison character strings.
4. The method of claim 1, wherein grouping the structured data set based on the alignment numbers comprises:
and acquiring a primary key character string corresponding to each data in the structured data set, and grouping the structured data set based on the primary key character string and the comparison item number.
5. The method of claim 1, wherein sorting the comparison data based on the sorting field and sequentially comparing the sorted comparison data to obtain a comparison result comprises:
acquiring the number of data in each group of comparison data, and judging whether the number of the data is more than 1;
if yes, sorting the comparison data based on the sorting field, and sequentially comparing to obtain a comparison result;
if not, obtaining a comparison result based on the source of the comparison data.
6. The method of claim 5, wherein obtaining alignment results based on the source of the alignment data comprises:
if the comparison data is from the first structured data, determining the comparison data as deleted data;
and if the comparison data is from the second structured data, determining the comparison data as the newly added data.
7. The method of claim 1, wherein obtaining a structured data set and application scenario requirements corresponding to the structured data set comprises:
acquiring real-time data in different databases by using a standardized interface, and performing format conversion on the real-time data to obtain a structured data set;
and aiming at the structured data sets corresponding to different databases, acquiring the application scene requirements corresponding to each structured data set.
8. The method according to any one of claims 1-7, further comprising:
after the comparison result is obtained, judging whether the comparison result is consistent;
if yes, comparing the structured data set again at preset time intervals, and covering the last comparison result after obtaining the comparison result so as to verify the accuracy of the comparison;
and if not, generating an alarm prompt to remind the user to check the comparison result.
9. A structured data alignment apparatus, comprising:
the system comprises an acquisition module, a processing module and a processing module, wherein the acquisition module is used for acquiring a structured data set and application scene requirements corresponding to the structured data set, the structured data set comprises first structured data and second structured data, the first structured data is reference source data, and the second structured data is changed data;
the analysis module is used for acquiring a field comparison rule based on the application scene requirement, and analyzing the structured data set based on the field comparison rule to obtain a comparison item number and a sequencing field;
the grouping module is used for grouping the structured data set based on the comparison item numbers to obtain at least one group of comparison data;
and the comparison module is used for sequencing the comparison data based on the sequencing fields aiming at each group of comparison data and sequentially comparing the sequencing data to obtain a comparison result.
10. An electronic device, comprising: a processor, and a memory communicatively coupled to the processor;
the memory stores computer-executable instructions;
the processor executes computer-executable instructions stored by the memory to implement the method of any of claims 1-8.
11. A computer-readable storage medium having computer-executable instructions stored thereon, which when executed by a processor, perform the method of any one of claims 1-8.
12. A computer program product comprising a program code for performing the method of any one of claims 1-8 when the computer program is run by a computer.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211085688.9A CN115357625A (en) | 2022-09-06 | 2022-09-06 | Structured data comparison method and device, electronic equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211085688.9A CN115357625A (en) | 2022-09-06 | 2022-09-06 | Structured data comparison method and device, electronic equipment and storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN115357625A true CN115357625A (en) | 2022-11-18 |
Family
ID=84005920
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202211085688.9A Pending CN115357625A (en) | 2022-09-06 | 2022-09-06 | Structured data comparison method and device, electronic equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN115357625A (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116628107A (en) * | 2023-05-30 | 2023-08-22 | 曙光云计算集团有限公司 | Data comparison method, device, equipment and medium |
| CN118689912A (en) * | 2024-08-27 | 2024-09-24 | 杭州乒乓智能技术有限公司 | Data comparison method and device, business data comparison system and electronic device |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111782657A (en) * | 2020-07-08 | 2020-10-16 | 上海乾臻信息科技有限公司 | Data processing method and device |
| CN112199935A (en) * | 2020-09-24 | 2021-01-08 | 建信金融科技有限责任公司 | Data comparison method and device, electronic equipment and computer readable storage medium |
| CN113642311A (en) * | 2021-08-12 | 2021-11-12 | 北京奇艺世纪科技有限公司 | Data comparison method and device, electronic equipment and storage medium |
| CN114461611A (en) * | 2022-01-26 | 2022-05-10 | 连连（杭州）信息技术有限公司 | Data comparison method and device, electronic equipment and storage medium |
-
2022
- 2022-09-06 CN CN202211085688.9A patent/CN115357625A/en active Pending
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111782657A (en) * | 2020-07-08 | 2020-10-16 | 上海乾臻信息科技有限公司 | Data processing method and device |
| CN112199935A (en) * | 2020-09-24 | 2021-01-08 | 建信金融科技有限责任公司 | Data comparison method and device, electronic equipment and computer readable storage medium |
| CN113642311A (en) * | 2021-08-12 | 2021-11-12 | 北京奇艺世纪科技有限公司 | Data comparison method and device, electronic equipment and storage medium |
| CN114461611A (en) * | 2022-01-26 | 2022-05-10 | 连连（杭州）信息技术有限公司 | Data comparison method and device, electronic equipment and storage medium |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116628107A (en) * | 2023-05-30 | 2023-08-22 | 曙光云计算集团有限公司 | Data comparison method, device, equipment and medium |
| CN118689912A (en) * | 2024-08-27 | 2024-09-24 | 杭州乒乓智能技术有限公司 | Data comparison method and device, business data comparison system and electronic device |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111177176B (en) | Data detection method, device and storage medium | |
| CN115357625A (en) | Structured data comparison method and device, electronic equipment and storage medium | |
| CN115061721A (en) | Report generation method and device, computer equipment and storage medium | |
| CN111400392B (en) | Multi-source heterogeneous data processing method and device | |
| CN110795455A (en) | Dependency relationship analysis method, electronic device, computer device and readable storage medium | |
| US20120143893A1 (en) | Pattern Matching Framework for Log Analysis | |
| CN108334609B (en) | Method, device, equipment and storage medium for realizing JSON format data access in Oracle | |
| CN110427188B (en) | Configuration method, device, equipment and storage medium of single-test assertion program | |
| CN111522817A (en) | Table content mapping system, method and non-transitory storage medium | |
| CN113268500B (en) | Service processing method and device and electronic equipment | |
| CN113297251A (en) | Multi-source data retrieval method, device, equipment and storage medium | |
| CN114741392A (en) | Data query method and device, electronic equipment and storage medium | |
| CN116795845A (en) | Data display method, device, terminal equipment and readable storage medium | |
| CN112506931A (en) | Data query method and device, electronic equipment and storage medium | |
| CN116775599A (en) | Data migration method, device, electronic equipment and storage medium | |
| CN107341105A (en) | Information processing method, terminal and server | |
| CN115757174A (en) | Database difference detection method and device | |
| CN115757481A (en) | Data migration method, device, equipment and storage medium | |
| CN114281845A (en) | Index generation method and device, electronic equipment and readable storage medium | |
| CN114238326A (en) | Data batch comparison method, device, equipment and storage medium | |
| CN109785099B (en) | Method and system for automatically processing service data information | |
| CN110781182B (en) | Automatic encoding method and device for check logic and computer equipment | |
| US9251253B2 (en) | Expeditious citation indexing | |
| CN113934430A (en) | Data retrieval analysis method and device, electronic equipment and storage medium | |
| CN109840213B (en) | Test data creating method, device, terminal and storage medium for GUI test |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |