CN111506578B

CN111506578B - Service data verification method, device, equipment and storage medium

Info

Publication number: CN111506578B
Application number: CN202010309495.1A
Authority: CN
Inventors: 黄蔚; 胡盼盼; 卢道和
Original assignee: WeBank Co Ltd
Current assignee: WeBank Co Ltd
Filing date: 2020-04-17
Publication date: 2024-07-09
Anticipated expiration: 2040-04-17

Abstract

The invention discloses a method, a device, equipment and a storage medium for verifying service data, wherein the method comprises the steps of obtaining standard table data in a first database, slicing the standard table data according to preset data quantity, and generating each sliced table data corresponding to the standard table data; acquiring each data identifier in the fragment table data, and calculating a fragment data check value corresponding to the fragment table data; and according to each data identifier in the fragment table data and the fragment data verification value corresponding to the fragment table data, performing data fragmentation on the to-be-verified table data in the second database to be verified, and performing data verification on the fragmented to-be-verified table data. According to the invention, the table data is subjected to the fragment verification, so that the calculation time of the data verification value is shortened, and the whole table data is prevented from influencing the external service of the database by carrying out fragment comparison verification on the table data, so that the data verification efficiency is improved.

Description

Service data verification method, device, equipment and storage medium

Technical Field

The present invention relates to the technical field of financial science and technology (Fintech), and in particular, to a method, an apparatus, a device, and a computer readable storage medium for verifying service data.

Background

With the development of computer technology, more and more technologies are applied in the financial field, and the traditional financial industry is gradually changed to the financial technology (Finteh), so that the database synchronization technology is not exceptional, but the database synchronization technology is also required to be higher due to the requirements of safety and real-time performance of the financial industry. For banking systems, the service data to be stored is increased, but the MySQL database is a stand-alone database, and only one physical server can store the data, and the storage capacity is limited, so that massive data of the banking system cannot be stored. Because of the horizontally unlimited expansion of the capacity of the distributed database, data in MySQL databases is typically migrated to the distributed database. Before switching the connection database of the banking system from the MySQL database to the distributed database, it is necessary to confirm whether the data of the MySQL database and the distributed database are consistent. At present, the data checking method for the MySQL database and the distributed database is as follows: and calculating check codes for the full-table data of the MySQL database and the distributed database respectively, and carrying out data check through the two full-table data check codes. However, the method of generating the check code for data check on the whole table of the database has the problems of long time consumption and low check efficiency.

Disclosure of Invention

The invention mainly aims to provide a method, a device, equipment and a computer readable storage medium for verifying service data, and aims to solve the technical problems of long time consumption and low verification efficiency in the existing service data verification mode.

In order to achieve the above object, the present invention provides a method for verifying service data, the method for verifying service data comprising the steps of:

Obtaining standard table data in a first database, and slicing the standard table data according to a preset data amount to generate each piece of slicing table data corresponding to the standard table data;

Acquiring each data identifier in the fragment table data, and calculating a fragment data check value corresponding to the fragment table data;

And according to each data identifier in the fragment table data and the fragment data verification value corresponding to the fragment table data, performing data fragmentation on the to-be-verified table data in the second database to be verified, and performing data verification on the fragmented to-be-verified table data.

Optionally, the step of performing data slicing on the table data to be checked in the second database to be checked according to each data identifier in the sliced table data and the sliced data check value corresponding to the sliced table data, and performing data check on the sliced table data to be checked specifically includes:

Obtaining the fragment table data in the first database as current fragment data, and adding pessimistic locks to the current fragment data in the first database according to each data identifier in the current fragment data;

judging whether related fragment data corresponding to the current fragment data exists in the to-be-checked table data or not;

If the related fragment data exists in the to-be-checked table data, calculating a related data check value corresponding to the related fragment data;

And carrying out data verification on the related sliced data according to the related data verification value and the sliced data verification value corresponding to the current sliced data so as to finish the data verification of the table data to be verified after slicing.

Optionally, after the step of performing data verification on the relevant sliced data according to the relevant data verification value and the sliced data verification value corresponding to the current sliced data to complete the data verification of the table data to be verified after slicing, the method further includes:

and when detecting a data verification completion instruction of the related sliced data, releasing pessimistic lock of the current sliced data in the first database.

Optionally, the method for verifying service data further includes:

And when the locking time interval of the pessimistic lock exceeds the preset time, releasing the pessimistic lock of the current sliced data, and stopping data verification of the related sliced data.

Optionally, the step of obtaining each data identifier in the fragment table data and calculating the fragment data check value corresponding to the fragment table data specifically includes:

Acquiring each data identifier in the fragment table data and acquiring all field values in the fragment table data;

and calculating the fragment data check value corresponding to the fragment table data according to a preset check value calculation function and all field values in the fragment table data.

Optionally, the step of calculating the fragment data check value corresponding to the fragment table data according to a preset check value calculation function and all field values in the fragment table data specifically includes:

Dividing all the field values into field values of each row according to row identifications, and adding separators between every two field values of each row to combine the field values of each row to generate corresponding character strings;

And converting the self character strings corresponding to the field values of each row into unique check codes through a bitwise exclusive OR function, and taking the unique check codes as the check values of the fragment data corresponding to the fragment table data.

Optionally, the step of converting the self-string corresponding to the field value of each row into a unique check code through a bitwise exclusive or function, as a piece data check value corresponding to the piece table data specifically includes:

Calculating the row check codes of the character strings corresponding to the field values of each row through the check value calculation function, and converting each row check code into each row check code of an unsigned integer through the type conversion function;

And converting the row check codes of the unsigned integers corresponding to the field values of each row into unique check codes through a bitwise exclusive OR function, and taking the unique check codes as the check values of the sliced data corresponding to the sliced table data.

Performing data slicing on the table data to be checked in the second database to be checked according to each data identifier in the table data and the slicing data check value corresponding to the table data;

And acquiring a currently idle check thread, and carrying out concurrent data check on the fragmented data of the table to be checked through a plurality of currently idle check threads.

In addition, in order to achieve the above object, the present invention further provides a service data verification device, where the service data verification device includes:

the table data slicing module is used for acquiring standard table data in the first database, slicing the standard table data according to a preset data amount and generating slicing table data corresponding to the standard table data;

the check value calculation module is used for acquiring each data identifier in the fragment table data and calculating a fragment data check value corresponding to the fragment table data;

And the table data verification module is used for carrying out data slicing on the table data to be verified in the second database to be verified according to each data identifier in the sliced table data and the sliced data verification value corresponding to the sliced table data, and carrying out data verification on the sliced table data to be verified.

Optionally, the table data verification module specifically includes:

The system comprises a sliced data locking unit, a pessimistic lock adding unit and a data processing unit, wherein the sliced data locking unit is used for acquiring sliced table data in the first database as current sliced data, and adding pessimistic locks to the current sliced data in the first database according to each data identifier in the current sliced data;

the fragment data judging unit is used for judging whether related fragment data corresponding to the current fragment data exists in the to-be-checked table data;

The first check value calculation unit is used for calculating a relevant data check value corresponding to the relevant fragment data if the relevant fragment data exists in the to-be-checked table data;

And the slicing data verification unit is used for carrying out data verification on the relevant slicing data according to the relevant data verification value and the slicing data verification value corresponding to the current slicing data so as to finish the data verification of the table data to be verified after slicing.

Optionally, the check value calculation module specifically includes:

A data field value obtaining unit, configured to obtain each data identifier in the fragment table data, and obtain all field values in the fragment table data;

And the second check value calculation unit is used for calculating the fragment data check value corresponding to the fragment table data according to a preset check value calculation function and all field values in the fragment table data.

Optionally, the second check value calculation unit is further configured to:

In addition, to achieve the above object, the present invention further provides a service data verification device, where the service data verification device includes: the system comprises a memory, a processor and a service data verification program stored in the memory and capable of running on the processor, wherein the service data verification program realizes the steps of the service data verification method when being executed by the processor.

In addition, in order to achieve the above object, the present invention also provides a computer-readable storage medium having stored thereon a verification program of service data, which when executed by a processor, implements the steps of the service data verification method as described above.

The invention provides a checking method of service data, which comprises the steps of obtaining standard table data in a first database, and slicing the standard table data according to preset data quantity to generate each sliced table data corresponding to the standard table data; acquiring each data identifier in the fragment table data, and calculating a fragment data check value corresponding to the fragment table data; and according to each data identifier in the fragment table data and the fragment data verification value corresponding to the fragment table data, performing data fragmentation on the to-be-verified table data in the second database to be verified, and performing data verification on the fragmented to-be-verified table data. By the method, the table data are subjected to the fragment verification, so that the calculation time of the data verification value is shortened, the table data in the second database are subjected to fragment comparison verification through the fragment data verification value of each fragment table data in the first database, the influence of the whole table data verification on the external service of the database is avoided, the data verification efficiency is improved, and the technical problems of long time consumption and low verification efficiency existing in the conventional business data verification method are solved. In addition, through the scheme, the database is subjected to the fragment verification, and when certain fragment table data is verified, other fragment table data can continue to provide services, so that the data verification efficiency is improved, the condition that the whole database is subjected to data verification and the writing operation of an application program to the database is stopped is avoided, and the online data verification without shutdown is realized.

Drawings

FIG. 1 is a schematic diagram of a device architecture of a hardware operating environment according to an embodiment of the present invention;

FIG. 2 is a flow chart of a first embodiment of a method for verifying service data according to the present invention;

Fig. 3 is a schematic diagram of the data synchronization process of the present invention.

The achievement of the objects, functional features and advantages of the present invention will be further described with reference to the accompanying drawings, in conjunction with the embodiments.

Detailed Description

It should be understood that the specific embodiments described herein are for purposes of illustration only and are not intended to limit the scope of the invention.

Referring to fig. 1, fig. 1 is a schematic device structure of a hardware running environment according to an embodiment of the present invention.

The verification device of the service data in the embodiment of the invention can be a PC or a server device, and a Java virtual machine is operated on the verification device.

As shown in fig. 1, the verification device for service data may include: a processor 1001, such as a CPU, a network interface 1004, a user interface 1003, a memory 1005, a communication bus 1002. Wherein the communication bus 1002 is used to enable connected communication between these components. The user interface 1003 may include a Display, an input unit such as a Keyboard (Keyboard), and the optional user interface 1003 may further include a standard wired interface, a wireless interface. The network interface 1004 may optionally include a standard wired interface, a wireless interface (e.g., WI-FI interface). The memory 1005 may be a high-speed RAM memory or a stable memory (non-volatile memory), such as a disk memory. The memory 1005 may also optionally be a storage device separate from the processor 1001 described above.

It will be appreciated by those skilled in the art that the device structure shown in fig. 1 is not limiting of the device and may include more or fewer components than shown, or may combine certain components, or a different arrangement of components.

As shown in fig. 1, an operating system, a network communication module, a user interface module, and a verification program of service data may be included in a memory 1005 as one type of computer storage medium.

In the device shown in fig. 1, the network interface 1004 is mainly used for connecting to a background server, and performing data communication with the background server; the user interface 1003 is mainly used for connecting a client (user side) and performing data communication with the client; and the processor 1001 may be used to call a verification program of service data stored in the memory 1005 and perform operations in a verification method of service data described below.

Based on the hardware structure, the embodiment of the business data verification method is provided.

Referring to fig. 2, fig. 2 is a flow chart of a first embodiment of a method for verifying service data according to the present invention, where the method for verifying service data includes:

Step S10, standard table data in a first database are obtained, and the standard table data are segmented according to preset data quantity, so that each piece of segmented table data corresponding to the standard table data are generated;

Because MySQL database is a stand-alone database, i.e. only one physical server can store data, and the hard disk capacity of one physical server is limited, i.e. the server typically has a capacity of 3T. In the financial industry, with the increasing number of customers, a stand-alone database cannot store business data of a lower financial system. Because the capacity of the distributed database has the characteristic of horizontal infinite expansion, the data of the MySQL database is generally migrated to the distributed database so as to solve the capacity problem of the database. The specific data synchronization process is as shown in fig. 3, firstly stopping the application program from writing data into the MySQL database, then using a third party component to analyze the binary file binlog file of the MySQL database, then recording the SQL sentence operation on the MySQL database, and then performing the same data processing operation on the distributed database in real time synchronization, thereby keeping the MySQL database and the data of the distributed database in real time synchronization. And after the data of the MySQL database and the data of the distributed database are completely consistent, switching the database connected with the application program to the distributed database. However, in the current data verification method, writing into the MySQL database is stopped, then a verification code is calculated for the full-table data of the MySQL database, and a verification code is calculated for the full-table data of the distributed database, and the data is verified through the two verification codes. Thus, the current data verification method has two problems:

1. writing into the MySQL database is stopped, and even if the business system stops external service, the user experience is reduced;

To ensure that the data in the MySQL database is no longer changing, the application's write to the MySQL database needs to be stopped. However, stopping the application, i.e. stopping the external services of the business system, the internet or financial business generally requires 24 hours of uninterrupted service, and thus the problem reduces the user experience.

2. And the time for calculating the check code of the whole table is long, so that the data check efficiency is reduced. For example:

the number of records of the full table of the MySQL database is 10000000:

select count(*)from sbtest；

+----------+

|count(*)|

+----------+

|10000000|

+----------+

The current data verification method is as follows: the unique check code generated by the table is 2526835448;

checksum table sbtest；

+--------------------+------------+

|Table|Checksum|

+--------------------+------------+

|ha_test3306.sbtest|2526835448|

+--------------------+------------+

1 row in set(9.11sec)。

The Checksum table function is used for reading the whole table row by row, calculating the Checksum and generating a unique check code corresponding to the whole table. Because the check code calculation time is long, the scheme requires that the whole database is stopped during the check period of the MySQL database, so that the service system supported by the MySQL database needs to be stopped, the problem has great influence on financial services which need 24 hours uninterrupted service, the check efficiency is reduced, and the user experience is reduced.

In order to solve the problems, the method and the device carry out fragment verification on the table data, shorten the calculation time of the data verification value, carry out fragment comparison verification on the table data in the second database through the fragment data verification value of each fragment table data in the first database, avoid the influence of full-table data verification on the external service of the database, and improve the data verification efficiency. Specifically, standard table data in a first database is obtained, wherein the first database is a conventional Relational Database (RDBMS) having characteristics of ACID, namely atomicity (Atomicity), consistency (Consistency), isolation (Isolation) and persistence (Durability), but poor lateral expansibility. In this embodiment, the non-relational database (NoSQL) does not have the ACID characteristic of the conventional relational database, but has characteristics of infinite lateral expansion, for example: HBase, hive or MongoDB, etc. Whereas a distributed database has the characteristics of both relational and non-relational databases, for example: voltDB, scaleDB, tiDB, etc. The standard data table is a data table storing all data of the first database. And then in the MySQL database, the standard table data are segmented according to the preset data quantity. For example, the main key of the MySQL database full table is a number increasing from 1, and the full table has 1000 ten thousand data volume, so that the full table can be divided into 1000 pieces of 1 ten thousand data volume of the fragment table data, namely, the fragment table data are respectively:

ID>＝1 and ID<＝10000

ID>＝10000+1 and ID<20000

…

ID > =999 ten thousand+1 and ID < = 1000 ten thousand

ID >1000 ten thousand.

Step S20, each data identifier in the fragment table data is obtained, and a fragment data check value corresponding to the fragment table data is calculated;

In this embodiment, a data identifier corresponding to each piece of data in each piece of table data is obtained, and each piece of data check value corresponding to each piece of table data is calculated according to a preset calculation mode. The preset calculation mode may be to calculate the fragment data check value corresponding to each fragment table data for each data of the fragment table data according to a method such as a message digest algorithm MD5, a hash algorithm, or a preset calculation function.

And step S30, performing data slicing on the table data to be checked in the second database to be checked according to each data identifier in the sliced table data and the sliced data check value corresponding to the sliced table data, and performing data check on the sliced table data to be checked.

In this embodiment, a configuration instruction, that is, session binlog_format, is set on the MySQL database as a state, where binlog is a binary log of the MySQL database, and is used to record the information of SQL statements (except for data query statements) that a user operates on the database. The MySQL database is replicated in three main ways: SQL statement-based replication (SBR), row-based replication (row-based replication, RBR), and mixed-mode replication (mixed-based replication, MBR). Correspondingly, there are three synchronization formats of binlog: STATEMENT, ROW and MIXED. Wherein:

① STATEMENT mode (SBR)

In this mode, each sql statement in the MySQL database that would modify the data is recorded into binlog.

The advantage of this mode is: the data change of each sql statement and each row does not need to be recorded, the binlog quantity is reduced, the throughput IO is saved, and the performance is improved.

The disadvantages of this mode are: in some cases, data inconsistencies in master-slave devices, such as sleep () function, last_insert_id (), and user-defined functions (udf), are prone to problems.

② ROW mode (RBR)

In this mode, the context information of each sql statement is not recorded, only which piece of data is modified, and the modified final recorded information is recorded.

The advantage of this mode is: the problem that the calling and triggering of the storage process, the function or the trigger cannot be copied correctly cannot occur under certain specific conditions.

The disadvantages of this mode are: a large number of logs, especially when alter tables (for adding, modifying or deleting columns in an existing table), are generated, the amount of log data is expanded.

③ MIXED Mode (MBR)

The above two modes are mixed, the common copy uses STATEMENT mode to store binlog, the ROW mode is used for the operation that STATEMENT mode can not copy to store binlog, mySQL database can select log storage mode according to executed SQL statement.

The data synchronization variables of the MySQL database and the distributed database comprise two ranges of actions, namely a global blobal and a current session: blobal denotes that all SQL statements use this schema, session denotes that this schema is used only for the current SQL statement under the current session. The configuration instruction is set as session binlog_format as state, so that the subsequent check SQL statement on the MySQL database is originally recorded in binlog and can be executed on the distributed database as it is. Specifically, corresponding identifiers corresponding to the fragment table data are generated in the MySQL database according to the data identifiers in the fragment table data, so that data corresponding to the fragment table data in the MySQL database are searched in the second database according to the corresponding identifiers of the fragment table data. Wherein the second database is a distributed database. And when the corresponding data is found in the distributed database, carrying out data verification on the found corresponding data in the distributed database according to the fragment data verification value corresponding to each fragment table data until the data verification in the distributed database is completed according to all fragment table data in the MySQL database, thereby carrying out fragment verification on the data in the second database. The embodiment provides a service data verification method, which comprises the steps of obtaining standard table data in a first database, and slicing the standard table data according to a preset data amount to generate various sliced table data corresponding to the standard table data; acquiring each data identifier in the fragment table data, and calculating a fragment data check value corresponding to the fragment table data; and according to each data identifier in the fragment table data and the fragment data verification value corresponding to the fragment table data, performing data fragmentation on the to-be-verified table data in the second database to be verified, and performing data verification on the fragmented to-be-verified table data. By the method, the table data are subjected to the fragment verification, so that the calculation time of the data verification value is shortened, the table data in the second database are subjected to fragment comparison verification through the fragment data verification value of each fragment table data in the first database, the influence of the whole table data verification on the external service of the database is avoided, the data verification efficiency is improved, and the technical problems of long time consumption and low verification efficiency existing in the conventional business data verification method are solved. In addition, through the scheme, the database is subjected to the fragment verification, and when certain fragment table data is verified, other fragment table data can continue to provide services, so that the data verification efficiency is improved, the condition that the whole database is subjected to data verification and the writing operation of an application program to the database is stopped is avoided, and the online data verification without shutdown is realized.

Further, based on the first embodiment of the method for checking service data of the present invention, a second embodiment of the method for checking service data of the present invention is provided.

In this embodiment, the step S30 specifically includes:

In order to prevent the changing of the data of the fragment table being checked in the MySQL database during the process of checking the fragment data, in this embodiment, pessimistic locks are added to the data fragments in the MySQL database to prevent the data fragments from being changed by the application program. Specifically, the current checking fragment table data on the MySQL database is obtained and used as the current fragment data. The current sharded data is then added to a pessimistic lock on the MySQL database. For example: select from tbName where ID > =1 and ID < =10000 for update. And the data synchronization third party component analyzes the binlog of the calculated check value in the state format, and places the related execution SQL statement on the MySQL database into the distributed database for synchronous execution, namely, firstly judging whether related fragment data corresponding to the current fragment data exists in the table data to be checked of the distributed database. If so, the corresponding relevant fragment data is divided in the data with the check list according to each data identifier in the current fragment data. And then, according to the same verification calculation mode, calculating a related data verification value corresponding to the related sliced data, and comparing the related data verification value with the sliced data verification value corresponding to the current sliced data, thereby completing the data verification of the sliced data of the table to be verified.

Further, after the step of performing data verification on the relevant sliced data according to the relevant data verification value and the sliced data verification value corresponding to the current sliced data to complete the data verification of the table data to be verified after slicing, the method further includes:

In this embodiment, after the current sliced data in the MySQL database and the related sliced data in the distributed database are subjected to slicing verification, pessimistic locks on the current sliced data in the MySQL database are released through other components, so that the current sliced data in the MySQL database can be restored to external services. And data locking is carried out for each data fragment, so that all data in the MySQL database is prevented from being locked for a long time, the MySQL database is prevented from being deactivated, and the user experience is improved. And then repeating the data verification process of the current sliced data and the related sliced data until the verification of all the sliced data in the MySQL database is completed.

Further, the method for verifying the service data further comprises the following steps:

Typically, the time taken for a fragment to lock to release the lock is short, e.g., within 1 second. However, in special cases, such as problems of low synchronization efficiency, the intermediate step of locking to release the lock is liable to take a long time, thereby resulting in that the locked fragments cannot be modified by the application program for a long time. To solve the above-described problem, in the present embodiment, by setting a lock timeout time, for example, 2 seconds, exceeding 2 seconds, the pessimistic lock of the fragment table data is automatically released, and at the same time, the data verification of the related fragment data is stopped. In a specific embodiment, after a preset time interval, the verification process is repeatedly started, and after the number of times of forcibly releasing the lock reaches the preset number of times, a failed reminding message is directly reported.

Further, in order to improve the verification efficiency of the service data, a third embodiment of the verification method of the service data is provided based on the second embodiment of the verification method of the service data of the present invention.

In this embodiment, the step 20 specifically includes:

The step of converting the self-character string corresponding to the field value of each row into a unique check code through a bitwise exclusive or function as a piece data check value corresponding to the piece table data specifically includes:

In this embodiment, each corresponding fragment data check value is calculated for each fragment table data on the MySQL database, and each fragment data check value and the corresponding fragment data identifier are stored.

The specific verification calculation steps are as follows:

splicing all field values of each row of the data of a fragment table by calculating a character string function, namely CONCAT _WS function in real time;

transmitting the spliced field value into a cyclic redundancy check function CRC32 function to generate a unique 32-bit check number for the spliced field value;

And transmitting the generated unique 32-BIT check number into a data type conversion function CAST function, transmitting the data with the converted type into a bitwise exclusive OR function BIT_XOR function, generating a unique check code, and taking the unique check code as a sliced data check value of the sliced table data. Wherein CONCAT _ws function is CONCAT With Separator, a special form of CONCAT () function. () The first parameter in the list is a separator of other parameters, and the separator is arranged between two character strings to be connected, and the separator can be one character string or the other parameters. If the separator is NULL, the result is NULL. The function ignores NULL values after any delimiter parameters (i.e., ignores all NULL), but CONCAT _ws () does not ignore any empty string. The CRC32 () function is to calculate a "redundancy check", and CRC32 represents a check value that will yield a 32bit (8-bit hexadecimal number). Since each bit of the source data block participates in the calculation when the CRC32 generates the check value, a different CRC32 value is obtained even if only one bit of the data block is changed.

The CAST (value as UNSIGNED) function is used to convert value to an unsigned integer. The BIT XOR () function, BIT-wise XOR function BIT XOR (expr), returns the exclusive or value bitwise XOR of all BITs in expr with a calculation accuracy of 64 BITs (BIGINT), and if no matching row is found, the function returns 0.

The specific process is as follows:

1. as shown in the following table, assume that there are 5 rows of data:

ID	col_1	col_2	col_n
				1	2	3	4
2	4	5	6
				3	4	5	6
4	5	6	7
				5	6	7	8

2. All field values for each row of data are combined into one string using the CONCAT _ws function using the-separator symbol, respectively, namely:

1-2-3-4；

2-4-5-6；

3-4-5-6；

4-5-6-7；

5-6-7-8。

3. Generating unique check values corresponding to the character strings of each row by using a CRC32 function for the character strings of each row synthesized in the step 2, and converting the unique check values into unsigned positive check values by using a CAST function:

2252345062；

3530292293；

1948055537；

4187647488；

2301651138。

4. And converting the plurality of check values generated in the step 3 into one check value, namely a unique check value corresponding to the fragment table data, namely 1352467344, through a BIT_XOR function.

Based on the above process, calculating the data checking value of the sliced data corresponding to all the sliced table data in the first database and the data checking value of the sliced data corresponding to the table data to be checked in the second database, and comparing the sliced data checking value of all the sliced table data in the first database with the sliced data checking value corresponding to the second database to finish the data sliced checking.

The invention also provides a device for verifying the service data, which comprises:

Further, the table data verification module specifically includes:

Further, the table data verification module further includes:

and the fragment data unlocking unit is used for releasing pessimistic lock of the current fragment data in the first database when detecting the data verification completion instruction of the related fragment data.

Further, the service data verification device further includes:

and the forced unlocking module is used for releasing the pessimistic lock of the current fragment data and stopping the data verification of the related fragment data when the locking time interval of the pessimistic lock exceeds the preset time.

Further, the check value calculation module specifically includes:

Further, the second check value calculation unit is further configured to:

Further, the table data verification module specifically includes:

the synchronous slicing unit is used for carrying out data slicing on the table data to be checked in the second database to be checked according to each data identifier in the table data and the slicing data check value corresponding to the table data;

And the concurrency checking unit is used for acquiring the currently idle checking thread and checking the concurrency data of the fragmented to-be-checked table data through the currently idle checking threads.

The method executed by each program module may refer to each embodiment of the method for verifying service data according to the present invention, which is not described herein again.

The invention also provides a computer readable storage medium.

The computer readable storage medium of the present invention stores a service data verification program, which when executed by a processor, implements the steps of the service data verification method described above.

The method implemented when the verification program of the service data running on the processor is executed may refer to various embodiments of the method for verifying service data of the present invention, which are not described herein again.

It should be noted that, in this document, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or system that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or system. Without further limitation, an element defined by the phrase "comprising one … …" does not exclude the presence of other like elements in a process, method, article, or system that comprises the element.

The foregoing embodiment numbers of the present invention are merely for the purpose of description, and do not represent the advantages or disadvantages of the embodiments.

From the above description of the embodiments, it will be clear to those skilled in the art that the above-described embodiment method may be implemented by means of software plus a necessary general hardware platform, but of course may also be implemented by means of hardware, but in many cases the former is a preferred embodiment. Based on such understanding, the technical solution of the present invention may be embodied essentially or in a part contributing to the prior art in the form of a software product stored in a storage medium (e.g. ROM/RAM, magnetic disk, optical disk) as described above, comprising instructions for causing a terminal device (which may be a mobile phone, a computer, a server, an air conditioner, or a network device, etc.) to perform the method according to the embodiments of the present invention.

The foregoing description is only of the preferred embodiments of the present invention, and is not intended to limit the scope of the invention, but rather is intended to cover any equivalents of the structures or equivalent processes disclosed herein or in the alternative, which may be employed directly or indirectly in other related arts.

Claims

1. The service data verification method is characterized by comprising the following steps of:

According to each data identifier in the fragment table data and the fragment data verification value corresponding to the fragment table data, performing data fragmentation on the to-be-verified table data in the second database to be verified, and performing data verification on the fragmented to-be-verified table data; the step of performing data slicing on the table data to be checked in the second database to be checked according to each data identifier in the sliced table data and the sliced data check value corresponding to the sliced table data, and performing data check on the sliced table data to be checked specifically includes: obtaining the fragment table data in the first database as current fragment data, and adding pessimistic locks to the current fragment data in the first database according to each data identifier in the current fragment data; judging whether related fragment data corresponding to the current fragment data exists in the to-be-checked table data or not; if the related fragment data exists in the to-be-checked table data, calculating a related data check value corresponding to the related fragment data; performing data verification on the related sliced data according to the related data verification value and the sliced data verification value corresponding to the current sliced data so as to finish data verification of the table data to be verified after slicing;

the step of performing data verification on the relevant sliced data according to the relevant data verification value and the sliced data verification value corresponding to the current sliced data to complete the data verification of the table data to be verified after slicing further includes: and when detecting a data verification completion instruction of the related sliced data, releasing pessimistic lock of the current sliced data in the first database.

2. The method for verifying service data according to claim 1, wherein the method for verifying service data further comprises:

3. The method for checking service data according to claim 1, wherein the step of obtaining each data identifier in the fragment table data and calculating a fragment data check value corresponding to the fragment table data specifically comprises:

4. The method for checking service data according to claim 3, wherein the step of calculating the fragment data check value corresponding to the fragment table data specifically includes:

and converting the character strings corresponding to the field values of each row into unique check codes through a bitwise exclusive OR function, and taking the unique check codes as the check values of the fragment data corresponding to the fragment table data.

5. The method for verifying service data as defined in claim 4, wherein the step of converting the self-string corresponding to the field value of each row into a unique verification code by a bitwise exclusive or function as the fragment data verification value corresponding to the fragment table data specifically comprises:

6. The method for checking service data according to any one of claims 1 to 5, wherein the step of performing data slicing on the table data to be checked in the second database to be checked according to each data identifier in the sliced table data and the sliced data check value corresponding to the sliced table data, and performing data checking on the sliced table data to be checked specifically includes:

7. A device for verifying service data, wherein the device for verifying service data comprises:

The table data verification module is used for carrying out data slicing on the table data to be verified in the second database to be verified according to each data identifier in the sliced table data and the sliced data verification value corresponding to the sliced table data, and carrying out data verification on the sliced table data to be verified; the table data verification module specifically comprises: the system comprises a sliced data locking unit, a pessimistic lock adding unit and a data processing unit, wherein the sliced data locking unit is used for acquiring sliced table data in the first database as current sliced data, and adding pessimistic locks to the current sliced data in the first database according to each data identifier in the current sliced data; the fragment data judging unit is used for judging whether related fragment data corresponding to the current fragment data exists in the to-be-checked table data; the first check value calculation unit is used for calculating a relevant data check value corresponding to the relevant fragment data if the relevant fragment data exists in the to-be-checked table data; the slicing data verification unit is used for carrying out data verification on the relevant slicing data according to the relevant data verification value and the slicing data verification value corresponding to the current slicing data so as to finish the data verification of the table data to be verified after slicing; the slicing data checking unit is used for performing data checking on the relevant slicing data according to the relevant data checking value and the slicing data checking value corresponding to the current slicing data so as to finish data checking of the table data to be checked after slicing, and is also used for releasing pessimistic lock of the current slicing data in the first database when detecting a data checking completion instruction of the relevant slicing data.

8. The apparatus for verifying service data according to claim 7, wherein the verification value calculating module specifically comprises:

9. The apparatus for verifying traffic data according to claim 8, wherein the second verification value calculating unit is further configured to:

10. The apparatus for verifying traffic data according to claim 9, wherein the second verification value calculating unit is further configured to:

11. A service data verification device, wherein the service data verification device comprises: memory, a processor and a verification program of service data stored on the memory and executable on the processor, which when executed by the processor, implements the steps of the method of verifying service data according to any one of claims 1 to 6.

12. A computer-readable storage medium, wherein a verification program of service data is stored on the computer-readable storage medium, which when executed by a processor, implements the steps of the service data verification method according to any one of claims 1 to 6.