CN116719854A - Data comparison method, device, computer equipment and storage medium - Google Patents

Data comparison method, device, computer equipment and storage medium Download PDF

Info

Publication number
CN116719854A
CN116719854A CN202310796853.XA CN202310796853A CN116719854A CN 116719854 A CN116719854 A CN 116719854A CN 202310796853 A CN202310796853 A CN 202310796853A CN 116719854 A CN116719854 A CN 116719854A
Authority
CN
China
Prior art keywords
comparison
configuration information
data
scanning
storage
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202310796853.XA
Other languages
Chinese (zh)
Inventor
麦明瑶
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Ping An Property and Casualty Insurance Company of China Ltd
Original Assignee
Ping An Property and Casualty Insurance Company of China Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Ping An Property and Casualty Insurance Company of China Ltd filed Critical Ping An Property and Casualty Insurance Company of China Ltd
Priority to CN202310796853.XA priority Critical patent/CN116719854A/en
Publication of CN116719854A publication Critical patent/CN116719854A/en
Pending legal-status Critical Current

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/24—Querying
    • G06F16/245—Query processing
    • G06F16/2455—Query execution
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F11/00—Error detection; Error correction; Monitoring
    • G06F11/07—Responding to the occurrence of a fault, e.g. fault tolerance
    • G06F11/14—Error detection or correction of the data by redundancy in operation
    • G06F11/1402—Saving, restoring, recovering or retrying
    • G06F11/1446—Point-in-time backing up or restoration of persistent data
    • G06F11/1448—Management of the data involved in backup or backup restore
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
    • G06Q40/00—Finance; Insurance; Tax strategies; Processing of corporate or income taxes
    • Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
    • Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
    • Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Physics & Mathematics (AREA)
  • Business, Economics & Management (AREA)
  • General Engineering & Computer Science (AREA)
  • Quality & Reliability (AREA)
  • Databases & Information Systems (AREA)
  • Data Mining & Analysis (AREA)
  • Computational Linguistics (AREA)
  • Accounting & Taxation (AREA)
  • Development Economics (AREA)
  • Economics (AREA)
  • Finance (AREA)
  • Marketing (AREA)
  • Strategic Management (AREA)
  • Technology Law (AREA)
  • General Business, Economics & Management (AREA)
  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)

Abstract

The embodiment of the application belongs to the fields of big data and financial science and technology, and relates to a data comparison method, a device, computer equipment and a storage medium, wherein the method comprises the following steps: obtaining a plurality of storage object scanning results, wherein the plurality of storage object scanning results are obtained according to the scanning of a plurality of storage objects, and the plurality of storage objects comprise a database and a historical backup thereof, or a data table and a historical backup thereof; obtaining comparison configuration information aiming at a plurality of storage object scanning results, wherein the comparison configuration information comprises comparison configuration items and rule items; performing data comparison on the plurality of storage object scanning results according to the comparison configuration items in the comparison configuration information to obtain a primary comparison result; and generating a target comparison result according to the rule items in the comparison configuration information and the primary comparison result. In addition, the application also relates to a blockchain technology, and a target comparison junctionFruits may be stored in the blockchain 。 The application improves the flexibility of the data comparison tool for data comparison.

Description

Data comparison method, device, computer equipment and storage medium
Technical Field
The present application relates to the field of big data and financial science and technology, and in particular, to a data comparison method, apparatus, computer device and storage medium.
Background
Data comparison is a common data processing method, and the existing data comparison is usually implemented through a database data comparison tool, which can perform comparison between libraries. For example, in the field of financial science and technology, a company may have two databases for storing data under two types of services, some of the customer data in the databases may be from the same customer, and the databases of the two services may be compared by a database data comparison tool to detect a data difference according to the two databases.
The existing data comparison tool is relatively rigid in function, comparison objects can only realize comparison among different databases, full-quantity comparison can only be performed, and the configuration function is single. However, in practice, the comparison needs are various, so that the existing data comparison tool is lack of flexibility.
Disclosure of Invention
The embodiment of the application aims to provide a data comparison method, a data comparison device, computer equipment and a storage medium, so as to solve the problem of low data comparison flexibility.
In order to solve the above technical problems, the embodiment of the present application provides a data comparison method, which adopts the following technical scheme:
Obtaining a plurality of storage object scanning results, wherein the storage object scanning results are obtained according to scanning of a plurality of storage objects, and the storage objects comprise a database and a historical backup thereof, or a data table and a historical backup thereof;
obtaining comparison configuration information aiming at the scanning results of the plurality of storage objects, wherein the comparison configuration information comprises comparison configuration items and rule items;
performing data comparison on the plurality of storage object scanning results according to the comparison configuration items in the comparison configuration information to obtain a primary comparison result;
and generating a target comparison result according to the rule items in the comparison configuration information and the primary comparison result.
In order to solve the above technical problems, the embodiment of the present application further provides a data comparison device, which adopts the following technical scheme:
the system comprises a scanning acquisition module, a storage object scanning module and a storage object scanning module, wherein the scanning acquisition module is used for acquiring a plurality of storage object scanning results, the storage object scanning results are obtained according to scanning of a plurality of storage objects, and the storage objects comprise a database and a historical backup thereof or a data table and a historical backup thereof;
the configuration acquisition module is used for acquiring comparison configuration information aiming at the plurality of storage object scanning results, wherein the comparison configuration information comprises comparison configuration items and rule items;
The data comparison module is used for comparing the data of the plurality of storage object scanning results according to the comparison configuration items in the comparison configuration information to obtain a primary comparison result;
and the result generation module is used for generating a target comparison result according to the rule items in the comparison configuration information and the primary comparison result.
In order to solve the above technical problems, the embodiment of the present application further provides a computer device, which adopts the following technical schemes:
obtaining a plurality of storage object scanning results, wherein the storage object scanning results are obtained according to scanning of a plurality of storage objects, and the storage objects comprise a database and a historical backup thereof, or a data table and a historical backup thereof;
obtaining comparison configuration information aiming at the scanning results of the plurality of storage objects, wherein the comparison configuration information comprises comparison configuration items and rule items;
performing data comparison on the plurality of storage object scanning results according to the comparison configuration items in the comparison configuration information to obtain a primary comparison result;
and generating a target comparison result according to the rule items in the comparison configuration information and the primary comparison result.
In order to solve the above technical problems, an embodiment of the present application further provides a computer readable storage medium, which adopts the following technical schemes:
Obtaining a plurality of storage object scanning results, wherein the storage object scanning results are obtained according to scanning of a plurality of storage objects, and the storage objects comprise a database and a historical backup thereof, or a data table and a historical backup thereof;
obtaining comparison configuration information aiming at the scanning results of the plurality of storage objects, wherein the comparison configuration information comprises comparison configuration items and rule items;
performing data comparison on the plurality of storage object scanning results according to the comparison configuration items in the comparison configuration information to obtain a primary comparison result;
and generating a target comparison result according to the rule items in the comparison configuration information and the primary comparison result.
Compared with the prior art, the embodiment of the application has the following main beneficial effects: the method comprises the steps of obtaining a plurality of storage object scanning results, wherein the storage object scanning results are obtained by scanning storage objects, and the storage objects can be databases and historical backups thereof or data tables and historical backups thereof, so that the flexibility of the objects for data comparison is ensured; the method comprises the steps of obtaining comparison configuration information aiming at a plurality of storage object scanning results, wherein the comparison configuration information comprises comparison configuration items and rule items, and personalized configuration of data comparison can be realized; performing data comparison on the multiple storage object scanning results according to the comparison configuration items to obtain primary comparison results; the primary comparison result reflects the difference between the scanning results of the storage objects, and the rule items are used for generating conclusive information according to the primary comparison result so as to generate a final target comparison result; according to the application, the setting can be carried out from the object of data comparison to the comparison process, so that the flexibility of the data comparison tool in data comparison is improved.
Drawings
In order to more clearly illustrate the solution of the present application, a brief description will be given below of the drawings required for the description of the embodiments of the present application, it being apparent that the drawings in the following description are some embodiments of the present application, and that other drawings may be obtained from these drawings without the exercise of inventive effort for a person of ordinary skill in the art.
FIG. 1 is an exemplary system architecture diagram in which the present application may be applied;
FIG. 2 is a flow chart of one embodiment of a data comparison method according to the present application;
FIG. 3 is a schematic diagram of an embodiment of a data contrast device according to the present application;
FIG. 4 is a schematic structural diagram of one embodiment of a computer device in accordance with the present application.
Detailed Description
Unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs; the terminology used in the description of the applications herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the application; the terms "comprising" and "having" and any variations thereof in the description of the application and the claims and the description of the drawings above are intended to cover a non-exclusive inclusion. The terms first, second and the like in the description and in the claims or in the above-described figures, are used for distinguishing between different objects and not necessarily for describing a sequential or chronological order.
Reference herein to "an embodiment" means that a particular feature, structure, or characteristic described in connection with the embodiment may be included in at least one embodiment of the application. The appearances of such phrases in various places in the specification are not necessarily all referring to the same embodiment, nor are separate or alternative embodiments mutually exclusive of other embodiments. Those of skill in the art will explicitly and implicitly appreciate that the embodiments described herein may be combined with other embodiments.
In order to make the person skilled in the art better understand the solution of the present application, the technical solution of the embodiment of the present application will be clearly and completely described below with reference to the accompanying drawings.
As shown in fig. 1, a system architecture 100 may include terminal devices 101, 102, 103, a network 104, and a server 105. The network 104 is used as a medium to provide communication links between the terminal devices 101, 102, 103 and the server 105. The network 104 may include various connection types, such as wired, wireless communication links, or fiber optic cables, among others.
The user may interact with the server 105 via the network 104 using the terminal devices 101, 102, 103 to receive or send messages or the like. Various communication client applications, such as a web browser application, a shopping class application, a search class application, an instant messaging tool, a mailbox client, social platform software, etc., may be installed on the terminal devices 101, 102, 103.
The terminal devices 101, 102, 103 may be various electronic devices having a display screen and supporting web browsing, including but not limited to smartphones, tablet computers, electronic book readers, MP3 players (Moving Picture Experts Group Audio Layer III, dynamic video expert compression standard audio plane 3), MP4 (Moving Picture Experts Group Audio Layer IV, dynamic video expert compression standard audio plane 4) players, laptop and desktop computers, and the like.
The server 105 may be a server providing various services, such as a background server providing support for pages displayed on the terminal devices 101, 102, 103.
It should be noted that, the data comparison method provided in the embodiment of the present application is generally executed by a server, and accordingly, the data comparison device is generally disposed in the server.
It should be understood that the number of terminal devices, networks and servers in fig. 1 is merely illustrative. There may be any number of terminal devices, networks, and servers, as desired for implementation.
With continued reference to FIG. 2, a flow chart of one embodiment of a data comparison method in accordance with the present application is shown. The data comparison method comprises the following steps:
Step S201, a plurality of storage object scan results are obtained, wherein the plurality of storage object scan results are obtained according to the scan of the plurality of storage objects, and the plurality of storage objects include a database and a history backup thereof, or a data table and a history backup thereof.
In this embodiment, the electronic device (e.g., the server shown in fig. 1) on which the data comparison method operates may communicate with the terminal device through a wired connection or a wireless connection. It should be noted that the wireless connection may include, but is not limited to, 3G/4G/5G connection, wiFi connection, bluetooth connection, wiMAX connection, zigbee connection, UWB (ultra wideband) connection, and other now known or later developed wireless connection.
Wherein the storage object is an object storing data, for example, a database may be used as the storage object; the data table can also be used as a storage object; even if the data table is a table in the database, the data table may be a storage object alone, but the level thereof is different from the database. The application can backup the database/data table to obtain the historical backup of the database/data table, and the historical backup of the database/data table can also be used as a storage object. When scanning or data comparison is performed, the historical backup of the database and the database are regarded as the same-level objects, and the historical backup of the data table and the data table are regarded as the same-level objects.
Specifically, a plurality of storage objects may be scanned in advance, so as to obtain a storage object scanning result corresponding to each storage object.
Step S202, obtaining comparison configuration information of a plurality of storage object scanning results, wherein the comparison configuration information comprises comparison configuration items and rule items.
Specifically, comparison configuration information aiming at a plurality of storage object scanning results is obtained, the comparison configuration information comprises comparison configuration items and rule items, and personalized configuration can be carried out on comparison processing by setting the comparison configuration information so as to meet various comparison requirements; wherein, the comparison configuration item can be used for comparing which parts (such as a data table in a database or fields in the data table) in the scanning result of the storage object; the rule item is then used for the generation of the comparison result.
Step S203, data comparison is carried out on the plurality of storage object scanning results according to the comparison configuration items in the comparison configuration information, and a primary comparison result is obtained.
Specifically, the application adopts independently written data comparison tools, supports the scanning of storage objects, and supports the personalized configuration of the data comparison process. And the data comparison tool performs data comparison on the plurality of storage object scanning results according to the comparison configuration items in the comparison configuration information to obtain a primary comparison result.
Step S204, generating a target comparison result according to the rule items in the comparison configuration information and the primary comparison result.
Specifically, the data comparison tool needs to generate a target comparison result according to a rule item, wherein the rule item is used for indicating the data comparison tool to generate conclusive information. For example, according to the difference threshold value in the rule item, judging the difference degree of the scanning results of a plurality of storage objects; and generating presentation information in a specific format according to the rule items, and generating a final target comparison result together with the primary comparison result.
In this embodiment, a plurality of storage object scanning results are obtained, where the storage object scanning results are obtained by scanning a storage object, and the storage object may be a database and its historical backup, or a data table and its historical backup, so as to ensure flexibility of the object for data comparison; the method comprises the steps of obtaining comparison configuration information aiming at a plurality of storage object scanning results, wherein the comparison configuration information comprises comparison configuration items and rule items, and personalized configuration of data comparison can be realized; performing data comparison on the multiple storage object scanning results according to the comparison configuration items to obtain primary comparison results; the primary comparison result reflects the difference between the scanning results of the storage objects, and the rule items are used for generating conclusive information according to the primary comparison result so as to generate a final target comparison result; according to the application, the setting can be carried out from the object of data comparison to the comparison process, so that the flexibility of the data comparison tool in data comparison is improved.
Further, the step of obtaining the scan results of the plurality of storage objects may include: responding to the received scanning selection instruction, and selecting a plurality of storage objects; receiving scanning configuration information aiming at a plurality of storage objects, wherein the scanning configuration information is used for setting a part to be scanned, and the part to be scanned is from the plurality of storage objects; checking the scanning configuration information according to a preset scanning checking rule; and when the scanning configuration information passes the verification, scanning the plurality of storage objects according to the scanning configuration information to obtain storage object scanning results corresponding to the plurality of storage objects respectively.
Specifically, the user may operate the terminal and trigger a scan selection instruction, and the data comparison tool selects a plurality of storage objects according to the received scan selection instruction.
The data comparison tool receives the scan configuration information for a plurality of storage objects, and the scan configuration information can be configured through a visual configuration page, or the historical scan configuration information is used as the current scan configuration information (for example, the scan is set as a timing task, the historical scan configuration information is obtained when the timing task is triggered, or the user selects the historical scan configuration information, edits and adjusts the historical scan configuration information to obtain the current scan configuration information). The scan configuration information is used to set a portion to be scanned, where the portion to be scanned is from a storage object, and the portion to be scanned refers to which portion of data in the storage object is scanned (for example, which data table in the database is scanned, or which fields in the data table are scanned), and it is understood that the portion to be scanned may be set as the whole database, or the whole data table.
The data comparison tool is preset with a scanning verification rule, and before scanning, the scanning configuration information needs to be verified according to the scanning verification rule to verify whether the scanning configuration information is legal or not. In the verification, one of the more important verification dimensions is to verify whether the storage objects are objects of the same level, for example, whether the storage objects are identical to a database or an identical data table, because the scanning is usually performed before and after the data comparison, and the level of the scanning results of a plurality of storage objects needs to be ensured to be the same during the data comparison.
When the scanning configuration information passes the verification, the data comparison tool scans a plurality of storage objects according to the scanning configuration information to obtain storage object scanning results corresponding to the storage objects respectively.
In this embodiment, a plurality of storage objects are selected, and scan configuration information is received, where the scan configuration information is used to set scan content, so as to implement personalized scan; and checking whether the scanning configuration information is legal or not according to the scanning checking rule, and scanning a plurality of storage objects according to the scanning configuration information to obtain storage object scanning results respectively corresponding to the plurality of storage objects when the scanning configuration information passes the checking, so that the accuracy of the storage object scanning results is ensured.
Further, before the step of selecting the plurality of storage objects in response to the received scan selection instruction, the method may further include: determining a data table which can be backed up, wherein the data table which can be backed up is determined according to a received table setting instruction or a storage value evaluation value of the data table; when a triggered backup instruction is received, carrying out backup processing on the backup data table to obtain historical backup of the backup data table; a historical backup of the backupable data table is set as a storage object.
Specifically, although both the database and the data table can be backed up, the data amount of the database is generally large and the storage content is large, so that the data table is generally selected to be backed up, and the backupable data table needs to be predetermined.
The data table capable of being backed up refers to a data table needing to be backed up, and can receive a table setting instruction sent by a user, and the data table is set as the data table capable of being backed up according to the table setting instruction; or, according to a preset algorithm, calculating a storage value evaluation value of the data table, wherein the storage value evaluation value represents the storage value of the data in the data table, and according to the storage value evaluation value, the backup data table can be automatically selected. In one embodiment, a storage evaluation index of each data table is obtained, the storage evaluation index includes information such as the access times of each data table, the level of a task associated with the data table, the level of a user accessing the data table, and the like in a preset time, and the storage evaluation index is subjected to weighted calculation to obtain a storage value evaluation value of each data table.
When a triggered backup instruction is received, backup processing is performed on the backup data table, so that the historical backup of the backup data table is obtained, and the historical backup of the backup data table is also set as a storage object. The historical backup of the data table is also set as a storage object, so that the front-back comparison of the same storage object at different times can be realized, which is not possessed by the existing data comparison tool.
In this embodiment, according to a table setting instruction or a storage value evaluation value of a data table, a data table that can be backed up is determined, where the data table that can be backed up is a data table that needs to be backed up; when a backup instruction is received, carrying out backup processing on the backup data table to obtain the historical backup of the backup data table, setting the historical backup of the backup data table as a storage object, and ensuring that the data comparison of the same object at different time can be realized later.
Further, the step of obtaining the comparison configuration information of the scan results for the plurality of storage objects may include: acquiring historical comparison configuration information associated with a plurality of storage object scanning results from a configuration information base, and generating current comparison configuration information based on the historical comparison configuration information; or displaying the visual configuration page through the terminal, and generating comparison configuration information according to configuration operation triggered in the visual configuration page.
Specifically, a configuration information base is set in the data comparison tool, the configuration information base stores historical comparison configuration information, the historical comparison configuration information is comparison configuration information used in the past, and the data comparison tool can persistence the used comparison configuration information.
And acquiring history comparison configuration information associated with a plurality of storage object scanning results from a configuration information base, wherein the association refers to that the history comparison configuration information contains the storage object corresponding to the current storage object scanning result. The historical comparison configuration information can be directly used as current comparison configuration information (for example, the comparison configuration information does not need to be configured independently each time in a timing task scene), or the user adjusts the historical comparison configuration information to obtain the current applicable comparison configuration information. The used historical configuration information is subjected to persistence and re-utilization, so that the generation speed of the comparison configuration information can be improved.
The data comparison tool can also display a visual configuration page through the terminal, and a user can perform configuration operation obtained by looking up in the visual configuration page in modes of dragging, choosing, inputting text and the like, so as to generate comparison configuration information.
In this embodiment, the comparison configuration information may be generated according to historical comparison configuration information in the configuration information base, or generated according to configuration operation in the visual configuration page, so as to enrich the acquisition mode of the comparison configuration information.
Further, the step S203 may include: respectively acquiring parts to be compared in a plurality of storage object scanning results according to the comparison configuration information; respectively calculating hash values of the parts to be compared; when the obtained hash values are inconsistent, comparing the parts to be compared through a preset multidimensional comparison algorithm to obtain a primary comparison result.
Specifically, the to-be-compared parts in the multiple storage object scanning results are respectively obtained according to the comparison configuration information, for example, a scanning table in the database scanning results is used as the to-be-compared part, or scanning contents corresponding to certain fields in the database scanning results are used as the to-be-compared part; it will be appreciated that the entire database scan or data table scan may also be used as the portion to be compared.
And respectively calculating hash values of the parts to be compared, wherein the hash values are unique data fingerprints. By comparing the hash values, whether the scanning results of all the storage objects are consistent or not can be rapidly judged, so that the calculation cost caused by the comparison of the data piece by piece is avoided.
When the obtained hash values are inconsistent, comparing the parts to be compared through a preset multidimensional comparison algorithm to obtain a primary comparison result. The multi-dimensional comparison algorithm includes a comparison algorithm of a plurality of dimensions, for example, a difference is determined by a row level difference algorithm by comparing each row of data in the scan results of the respective storage objects; similar data is matched by a Longest Common Subsequence (LCS) algorithm, a fuzzy matching algorithm, etc., and the correlation between them is determined.
In the embodiment, firstly, the hash value of each part to be compared is calculated, the whole comparison is carried out according to the hash value, and when the hash values are consistent, the comparison can be ended, so that the data processing amount is reduced; when the obtained hash values are inconsistent, comparing the parts to be compared through a preset multidimensional comparison algorithm to obtain a primary comparison result, and ensuring the accuracy of the primary comparison result.
Further, comparing the rule items in the configuration information to include the pre-warning rule item and the format rule item, the step S204 may include: determining the difference degree between the scanning results of the plurality of storage objects according to the early warning rule items and the primary comparison result, and generating early warning information according to the difference degree; performing format conversion treatment on the primary comparison result according to the format rule item to obtain a secondary comparison result; and generating a target comparison result according to the primary comparison result, the early warning information and the secondary comparison result.
Specifically, the rule items include an early warning rule item and a format rule item. The primary comparison result can record and store differences between object scanning results, such as the number of difference records (how many differences exist), difference fields (which fields have differences), and difference data details (specific difference conditions); the early warning rule item records an early warning rule, the difference degree among a plurality of storage object scanning results can be determined according to the early warning rule item and the primary comparison result, and early warning information is generated according to the difference degree. For example, when the number of difference records between the two stored object scanning results is greater than a preset threshold, the difference degree is indicated to be greater, and corresponding early warning information is generated.
Performing format conversion treatment on the primary comparison result according to the format rule item to obtain a secondary comparison result; the format conversion process may be processing the difference item or generating a secondary comparison result in a particular format (e.g., chart, etc.). And generating a final target comparison result according to the primary comparison result, the early warning information and the secondary comparison result.
In this embodiment, the difference between the scanning results of the plurality of storage objects is determined according to the early warning rule item and the primary comparison result, so as to generate early warning information; and carrying out format conversion treatment on the primary comparison result according to the format rule item to obtain a secondary comparison result, realizing personalized processing treatment on the primary comparison result, and generating a final target comparison result.
It should be emphasized that, to further ensure the privacy and security of the target comparison result, the target comparison result may also be stored in a node of a blockchain.
The blockchain is a novel application mode of computer technologies such as distributed data storage, point-to-point transmission, consensus mechanism, encryption algorithm and the like. The Blockchain (Blockchain), which is essentially a decentralised database, is a string of data blocks that are generated by cryptographic means in association, each data block containing a batch of information of network transactions for verifying the validity of the information (anti-counterfeiting) and generating the next block. The blockchain may include a blockchain underlying platform, a platform product services layer, an application services layer, and the like.
Those skilled in the art will appreciate that implementing all or part of the above described methods may be accomplished by computer readable instructions stored in a computer readable storage medium that, when executed, may comprise the steps of the embodiments of the methods described above. The storage medium may be a nonvolatile storage medium such as a magnetic disk, an optical disk, a Read-Only Memory (ROM), or a random access Memory (Random Access Memory, RAM).
It should be understood that, although the steps in the flowcharts of the figures are shown in order as indicated by the arrows, these steps are not necessarily performed in order as indicated by the arrows. The steps are not strictly limited in order and may be performed in other orders, unless explicitly stated herein. Moreover, at least some of the steps in the flowcharts of the figures may include a plurality of sub-steps or stages that are not necessarily performed at the same time, but may be performed at different times, the order of their execution not necessarily being sequential, but may be performed in turn or alternately with other steps or at least a portion of the other steps or stages.
With further reference to fig. 3, as an implementation of the method shown in fig. 2, the present application provides an embodiment of a data comparison device, where the embodiment of the device corresponds to the embodiment of the method shown in fig. 2, and the device is particularly applicable to various electronic devices.
As shown in fig. 3, the data comparing apparatus 300 according to the present embodiment includes: a scan acquisition module 301, a configuration acquisition module 302, a data comparison module 303, and a result generation module 304, wherein:
The scan obtaining module 301 is configured to obtain a plurality of storage object scan results, where the plurality of storage object scan results are obtained according to scanning a plurality of storage objects, and the plurality of storage objects include a database and a history backup thereof, or a data table and a history backup thereof.
The configuration obtaining module 302 is configured to obtain comparison configuration information for the scan results of the plurality of storage objects, where the comparison configuration information includes comparison configuration items and rule items.
The data comparison module 303 is configured to compare data of the plurality of stored object scan results according to the comparison configuration item in the comparison configuration information, so as to obtain a primary comparison result.
The result generating module 304 is configured to generate a target comparison result according to the rule item and the primary comparison result in the comparison configuration information.
In this embodiment, a plurality of storage object scanning results are obtained, where the storage object scanning results are obtained by scanning a storage object, and the storage object may be a database and its historical backup, or a data table and its historical backup, so as to ensure flexibility of the object for data comparison; the method comprises the steps of obtaining comparison configuration information aiming at a plurality of storage object scanning results, wherein the comparison configuration information comprises comparison configuration items and rule items, and personalized configuration of data comparison can be realized; performing data comparison on the multiple storage object scanning results according to the comparison configuration items to obtain primary comparison results; the primary comparison result reflects the difference between the scanning results of the storage objects, and the rule items are used for generating conclusive information according to the primary comparison result so as to generate a final target comparison result; according to the application, the setting can be carried out from the object of data comparison to the comparison process, so that the flexibility of the data comparison tool in data comparison is improved.
In some alternative implementations of the present embodiment, the scan acquisition module 301 may include: the device comprises an object selecting sub-module, a scanning receiving sub-module, an information checking sub-module and an object scanning sub-module, wherein:
and the object selection sub-module is used for responding to the received scanning selection instruction and selecting a plurality of storage objects.
And the scanning receiving sub-module is used for receiving scanning configuration information aiming at a plurality of storage objects, wherein the scanning configuration information is used for setting a part to be scanned, and the part to be scanned is from the plurality of storage objects.
And the information verification sub-module is used for verifying the scanning configuration information according to a preset scanning verification rule.
And the object scanning sub-module is used for scanning the plurality of storage objects according to the scanning configuration information when the scanning configuration information passes the verification, so as to obtain storage object scanning results respectively corresponding to the plurality of storage objects.
In this embodiment, a plurality of storage objects are selected, and scan configuration information is received, where the scan configuration information is used to set scan content, so as to implement personalized scan; and checking whether the scanning configuration information is legal or not according to the scanning checking rule, and scanning a plurality of storage objects according to the scanning configuration information to obtain storage object scanning results respectively corresponding to the plurality of storage objects when the scanning configuration information passes the checking, so that the accuracy of the storage object scanning results is ensured.
In some optional implementations of the present embodiment, the scan acquisition module 301 may further include: the data table determining sub-module, the backup processing sub-module and the backup setting sub-module, wherein:
and the data table determining sub-module is used for determining a data table which can be backed up, and the data table which can be backed up is determined according to the received table setting instruction or the storage value evaluation value of the data table.
And the backup processing sub-module is used for carrying out backup processing on the backup data table when receiving the triggered backup instruction to obtain the historical backup of the backup data table.
And the backup setting sub-module is used for setting the historical backup of the backup data table as a storage object.
In this embodiment, according to a table setting instruction or a storage value evaluation value of a data table, a data table that can be backed up is determined, where the data table that can be backed up is a data table that needs to be backed up; when a backup instruction is received, carrying out backup processing on the backup data table to obtain the historical backup of the backup data table, setting the historical backup of the backup data table as a storage object, and ensuring that the data comparison of the same object at different time can be realized later.
In some alternative implementations of the present embodiment, the configuration acquisition module 302 may include: history acquisition sub-module and page acquisition sub-module, wherein:
The history acquisition sub-module is used for acquiring history comparison configuration information related to a plurality of storage object scanning results from the configuration information base and generating current comparison configuration information based on the history comparison configuration information.
The page acquisition sub-module is used for displaying the visual configuration page through the terminal and generating comparison configuration information according to configuration operation triggered in the visual configuration page.
In this embodiment, the comparison configuration information may be generated according to historical comparison configuration information in the configuration information base, or generated according to configuration operation in the visual configuration page, so as to enrich the acquisition mode of the comparison configuration information.
In some alternative implementations of the present embodiment, the data comparison module 303 may include: comparing and obtaining a sub-module, a Ha Xiji operator module and a comparison sub-module, wherein:
and the comparison acquisition sub-module is used for respectively acquiring the parts to be compared in the plurality of storage object scanning results according to the comparison configuration information.
And the hash calculation submodule is used for calculating hash values of the parts to be compared respectively.
And the comparison sub-module is used for comparing the parts to be compared through a preset multidimensional comparison algorithm when the obtained hash values are inconsistent, so as to obtain a primary comparison result.
In the embodiment, firstly, the hash value of each part to be compared is calculated, the whole comparison is carried out according to the hash value, and when the hash values are consistent, the comparison can be ended, so that the data processing amount is reduced; when the obtained hash values are inconsistent, comparing the parts to be compared through a preset multidimensional comparison algorithm to obtain a primary comparison result, and ensuring the accuracy of the primary comparison result.
In some optional implementations of this embodiment, the rule items in the comparison configuration information include early warning rule items and format rule items, and the result generating module 304 may include: the device comprises an early warning generation sub-module, a conversion processing sub-module and a result generation sub-module, wherein:
and the early warning generation sub-module is used for determining the difference degree between the scanning results of the plurality of storage objects according to the early warning rule items and the primary comparison result and generating early warning information according to the difference degree.
And the conversion processing sub-module is used for carrying out format conversion processing on the primary comparison result according to the format rule item to obtain a secondary comparison result.
And the result generation sub-module is used for generating a target comparison result according to the primary comparison result, the early warning information and the secondary comparison result.
In this embodiment, the difference between the scanning results of the plurality of storage objects is determined according to the early warning rule item and the primary comparison result, so as to generate early warning information; and carrying out format conversion treatment on the primary comparison result according to the format rule item to obtain a secondary comparison result, realizing personalized processing treatment on the primary comparison result, and generating a final target comparison result.
In order to solve the technical problems, the embodiment of the application also provides computer equipment. Referring specifically to fig. 4, fig. 4 is a basic structural block diagram of a computer device according to the present embodiment.
The computer device 4 comprises a memory 41, a processor 42, a network interface 43 communicatively connected to each other via a system bus. It should be noted that only computer device 4 having components 41-43 is shown in the figures, but it should be understood that not all of the illustrated components are required to be implemented and that more or fewer components may be implemented instead. It will be appreciated by those skilled in the art that the computer device herein is a device capable of automatically performing numerical calculations and/or information processing in accordance with predetermined or stored instructions, the hardware of which includes, but is not limited to, microprocessors, application specific integrated circuits (Application Specific Integrated Circuit, ASICs), programmable gate arrays (fields-Programmable Gate Array, FPGAs), digital processors (Digital Signal Processor, DSPs), embedded devices, etc.
The computer equipment can be a desktop computer, a notebook computer, a palm computer, a cloud server and other computing equipment. The computer equipment can perform man-machine interaction with a user through a keyboard, a mouse, a remote controller, a touch pad or voice control equipment and the like.
The memory 41 includes at least one type of readable storage medium including flash memory, hard disk, multimedia card, card memory (e.g., SD or DX memory, etc.), random Access Memory (RAM), static Random Access Memory (SRAM), read Only Memory (ROM), electrically Erasable Programmable Read Only Memory (EEPROM), programmable Read Only Memory (PROM), magnetic memory, magnetic disk, optical disk, etc. In some embodiments, the storage 41 may be an internal storage unit of the computer device 4, such as a hard disk or a memory of the computer device 4. In other embodiments, the memory 41 may also be an external storage device of the computer device 4, such as a plug-in hard disk, a Smart Media Card (SMC), a Secure Digital (SD) Card, a Flash Card (Flash Card) or the like, which are provided on the computer device 4. Of course, the memory 41 may also comprise both an internal memory unit of the computer device 4 and an external memory device. In this embodiment, the memory 41 is typically used to store an operating system and various application software installed on the computer device 4, such as computer readable instructions of a data comparison method. Further, the memory 41 may be used to temporarily store various types of data that have been output or are to be output.
The processor 42 may be a central processing unit (Central Processing Unit, CPU), controller, microcontroller, microprocessor, or other data processing chip in some embodiments. The processor 42 is typically used to control the overall operation of the computer device 4. In this embodiment, the processor 42 is configured to execute computer readable instructions stored in the memory 41 or process data, such as computer readable instructions for executing the data comparison method.
The network interface 43 may comprise a wireless network interface or a wired network interface, which network interface 43 is typically used for establishing a communication connection between the computer device 4 and other electronic devices.
The computer device provided in this embodiment may perform the above-described data comparison method. The data comparison method may be the data comparison method of each of the above embodiments.
In this embodiment, a plurality of storage object scanning results are obtained, where the storage object scanning results are obtained by scanning a storage object, and the storage object may be a database and its historical backup, or a data table and its historical backup, so as to ensure flexibility of the object for data comparison; the method comprises the steps of obtaining comparison configuration information aiming at a plurality of storage object scanning results, wherein the comparison configuration information comprises comparison configuration items and rule items, and personalized configuration of data comparison can be realized; performing data comparison on the multiple storage object scanning results according to the comparison configuration items to obtain primary comparison results; the primary comparison result reflects the difference between the scanning results of the storage objects, and the rule items are used for generating conclusive information according to the primary comparison result so as to generate a final target comparison result; according to the application, the setting can be carried out from the object of data comparison to the comparison process, so that the flexibility of the data comparison tool in data comparison is improved.
The present application also provides another embodiment, namely, a computer-readable storage medium storing computer-readable instructions executable by at least one processor to cause the at least one processor to perform the steps of the data comparison method as described above.
In this embodiment, a plurality of storage object scanning results are obtained, where the storage object scanning results are obtained by scanning a storage object, and the storage object may be a database and its historical backup, or a data table and its historical backup, so as to ensure flexibility of the object for data comparison; the method comprises the steps of obtaining comparison configuration information aiming at a plurality of storage object scanning results, wherein the comparison configuration information comprises comparison configuration items and rule items, and personalized configuration of data comparison can be realized; performing data comparison on the multiple storage object scanning results according to the comparison configuration items to obtain primary comparison results; the primary comparison result reflects the difference between the scanning results of the storage objects, and the rule items are used for generating conclusive information according to the primary comparison result so as to generate a final target comparison result; according to the application, the setting can be carried out from the object of data comparison to the comparison process, so that the flexibility of the data comparison tool in data comparison is improved.
From the above description of the embodiments, it will be clear to those skilled in the art that the above-described embodiment method may be implemented by means of software plus a necessary general hardware platform, but of course may also be implemented by means of hardware, but in many cases the former is a preferred embodiment. Based on such understanding, the technical solution of the present application may be embodied essentially or in a part contributing to the prior art in the form of a software product stored in a storage medium (e.g. ROM/RAM, magnetic disk, optical disk) comprising instructions for causing a terminal device (which may be a mobile phone, a computer, a server, an air conditioner, or a network device, etc.) to perform the method according to the embodiments of the present application.
It is apparent that the above-described embodiments are only some embodiments of the present application, but not all embodiments, and the preferred embodiments of the present application are shown in the drawings, which do not limit the scope of the patent claims. This application may be embodied in many different forms, but rather, embodiments are provided in order to provide a thorough and complete understanding of the present disclosure. Although the application has been described in detail with reference to the foregoing embodiments, it will be apparent to those skilled in the art that modifications may be made to the embodiments described in the foregoing description, or equivalents may be substituted for elements thereof. All equivalent structures made by the content of the specification and the drawings of the application are directly or indirectly applied to other related technical fields, and are also within the scope of the application.

Claims (10)

1. A method of data comparison comprising the steps of:
obtaining a plurality of storage object scanning results, wherein the storage object scanning results are obtained according to scanning of a plurality of storage objects, and the storage objects comprise a database and a historical backup thereof, or a data table and a historical backup thereof;
obtaining comparison configuration information aiming at the scanning results of the plurality of storage objects, wherein the comparison configuration information comprises comparison configuration items and rule items;
performing data comparison on the plurality of storage object scanning results according to the comparison configuration items in the comparison configuration information to obtain a primary comparison result;
and generating a target comparison result according to the rule items in the comparison configuration information and the primary comparison result.
2. The data comparison method of claim 1, wherein the step of obtaining a plurality of stored object scan results comprises:
responding to the received scanning selection instruction, and selecting a plurality of storage objects;
receiving scanning configuration information for the plurality of storage objects, wherein the scanning configuration information is used for setting a part to be scanned, and the part to be scanned is from the plurality of storage objects;
Checking the scanning configuration information according to a preset scanning checking rule;
and when the scanning configuration information passes the verification, scanning the plurality of storage objects according to the scanning configuration information to obtain storage object scanning results respectively corresponding to the plurality of storage objects.
3. The data comparison method according to claim 2, further comprising, prior to the step of selecting a plurality of memory objects in response to the received scan selection instruction:
determining a data table capable of being backed up, wherein the data table capable of being backed up is determined according to a received table setting instruction or a storage value evaluation value of the data table;
when a triggered backup instruction is received, carrying out backup processing on the backup data table to obtain historical backup of the backup data table;
and setting the historical backup of the backup data table as a storage object.
4. The data comparison method according to claim 1, wherein the step of acquiring comparison configuration information for the plurality of stored object scan results comprises:
acquiring historical comparison configuration information associated with the plurality of storage object scanning results from a configuration information base, and generating current comparison configuration information based on the historical comparison configuration information; or,
And displaying the visual configuration page through the terminal, and generating comparison configuration information according to configuration operation triggered in the visual configuration page.
5. The data comparison method according to claim 1, wherein the step of comparing the plurality of stored object scan results according to the comparison configuration item in the comparison configuration information to obtain a primary comparison result comprises:
respectively acquiring parts to be compared in the scanning results of the plurality of storage objects according to the comparison configuration information;
respectively calculating hash values of the parts to be compared;
when the obtained hash values are inconsistent, comparing the parts to be compared through a preset multidimensional comparison algorithm to obtain a primary comparison result.
6. The data comparison method according to claim 1, wherein the rule items in the comparison configuration information include early warning rule items and format rule items, and the step of generating a target comparison result according to the rule items in the comparison configuration information and the primary comparison result includes:
determining the difference degree between the plurality of storage object scanning results according to the early warning rule items and the primary comparison result, and generating early warning information according to the difference degree;
Performing format conversion processing on the primary comparison result according to the format rule item to obtain a secondary comparison result;
and generating a target comparison result according to the primary comparison result, the early warning information and the secondary comparison result.
7. A data contrast device, comprising:
the system comprises a scanning acquisition module, a storage object scanning module and a storage object scanning module, wherein the scanning acquisition module is used for acquiring a plurality of storage object scanning results, the storage object scanning results are obtained according to scanning of a plurality of storage objects, and the storage objects comprise a database and a historical backup thereof or a data table and a historical backup thereof;
the configuration acquisition module is used for acquiring comparison configuration information aiming at the plurality of storage object scanning results, wherein the comparison configuration information comprises comparison configuration items and rule items;
the data comparison module is used for comparing the data of the plurality of storage object scanning results according to the comparison configuration items in the comparison configuration information to obtain a primary comparison result;
and the result generation module is used for generating a target comparison result according to the rule items in the comparison configuration information and the primary comparison result.
8. The data comparison device of claim 7, wherein the data comparison module comprises:
The comparison acquisition sub-module is used for respectively acquiring the parts to be compared in the plurality of storage object scanning results according to the comparison configuration information;
the hash calculation submodule is used for calculating hash values of the parts to be compared respectively;
and the comparison sub-module is used for comparing the parts to be compared through a preset multidimensional comparison algorithm when the obtained hash values are inconsistent, so as to obtain a primary comparison result.
9. A computer device comprising a memory having stored therein computer readable instructions which when executed by a processor implement the steps of the data comparison method of any of claims 1 to 6.
10. A computer readable storage medium having stored thereon computer readable instructions which when executed by a processor implement the steps of the data comparison method according to any of claims 1 to 6.
CN202310796853.XA 2023-06-30 2023-06-30 Data comparison method, device, computer equipment and storage medium Pending CN116719854A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202310796853.XA CN116719854A (en) 2023-06-30 2023-06-30 Data comparison method, device, computer equipment and storage medium

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202310796853.XA CN116719854A (en) 2023-06-30 2023-06-30 Data comparison method, device, computer equipment and storage medium

Publications (1)

Publication Number Publication Date
CN116719854A true CN116719854A (en) 2023-09-08

Family

ID=87869678

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202310796853.XA Pending CN116719854A (en) 2023-06-30 2023-06-30 Data comparison method, device, computer equipment and storage medium

Country Status (1)

Country Link
CN (1) CN116719854A (en)

Similar Documents

Publication Publication Date Title
CN110334144B (en) Message display method, device, computer equipment and storage medium
CN112181835B (en) Automatic test method, device, computer equipment and storage medium
CN113220734A (en) Course recommendation method and device, computer equipment and storage medium
CN112363814A (en) Task scheduling method and device, computer equipment and storage medium
CN113496017B (en) Verification method, device, equipment and storage medium
CN116956326A (en) Authority data processing method and device, computer equipment and storage medium
CN116383787A (en) Page creation method, page creation device, computer equipment and storage medium
CN113283759B (en) Account risk portrait updating method, device, equipment and storage medium
CN111221817B (en) Service information data storage method, device, computer equipment and storage medium
CN112182107B (en) List data acquisition method, device, computer equipment and storage medium
CN116719854A (en) Data comparison method, device, computer equipment and storage medium
CN113434069A (en) Menu configuration method, device, equipment and storage medium
CN113806372B (en) New data information construction method, device, computer equipment and storage medium
CN109947701B (en) File import method, device, equipment and storage medium based on NSQ message queue
CN117827814A (en) Data verification method, device, computer equipment and storage medium
CN116701488A (en) Data processing method, device, computer equipment and storage medium
CN116206032A (en) Task verification method, device, computer equipment and medium thereof
CN116737437A (en) Data analysis method, device, computer equipment and storage medium
CN116775186A (en) Page data processing method and device, computer equipment and storage medium
CN116932486A (en) File generation method, device, computer equipment and storage medium
CN116842541A (en) Data encryption and decryption processing method and device, computer equipment and storage medium
CN113553519A (en) Interface address display method and device, computer equipment and storage medium
CN113504957A (en) Table data processing method and device, computer equipment and storage medium
CN116643884A (en) Data computing method, device, equipment and storage medium based on rule engine
CN116738948A (en) Data processing method, device, computer equipment and storage medium

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination