WO2020207008A1

WO2020207008A1 - Data verification method, apparatus, electronic device and storage medium

Info

Publication number: WO2020207008A1
Application number: PCT/CN2019/116968
Authority: WO
Inventors: 袁文涛
Original assignee: 平安普惠企业管理有限公司
Priority date: 2019-04-12
Filing date: 2019-11-11
Publication date: 2020-10-15
Also published as: CN110083615A

Abstract

A data verification method, an apparatus, an electronic device and a storage medium, wherein the method comprises: obtaining a destination data table, a source data table and attribute information of the destination data table (S101), determining a destination verification strategy for consistency verification on the destination data table according to the attribute information of the destination data table (S102), obtaining a write data status of the source data table (S103), if the write data status of the source data table is a locked state, verifying whether the destination data table is consistent with the source data table according to the destination verification strategy (S104). Consistency verification on the destination data table and the source data table can be automatically performed, the resources can be saved, and the verification efficiency can be improved.

Description

Data verification method, device, electronic equipment and storage medium

This application claims the priority of a Chinese patent application filed with the Chinese Patent Office, the application number is 2019102974538, and the application name is "a data verification method, device, electronic equipment, and storage medium" on April 12, 2019. The entire content of the application is approved The reference is incorporated in this application.

Technical field

This application relates to the field of computer technology, and in particular to a data verification method, device, electronic equipment, and storage medium.

Background technique

With the advent of the big data era, the application and processing of big data become more frequent, and the comparison of big data consistency has become a key application in big data. For example, when the data management system is replaced, it is necessary to determine whether the big data before and after the replacement is consistent; whether the calculation results before and after the algorithm optimization are consistent; whether the data is consistent before and after the data migration, and so on. Currently, manual comparison is mainly used to verify whether the target data is consistent with the source data, which requires a lot of time and manpower, and the efficiency is low.

Summary of the invention

The embodiments of the present application provide a data verification method, device, electronic equipment, and storage medium, which can automatically verify the consistency of the target data table and the source data table, save resources, and improve verification efficiency.

In the first aspect, an embodiment of the present application provides a data verification method, which includes:

Obtain the target data table, the source data table, and the attribute information of the target data table. The attribute information of the target data table includes any one or more of the following: the data volume of the target data table, Type, the number of times the target data table is queried, and the time when the target data table is established;

Determining a target verification strategy for consistency verification of the target data table according to the attribute information of the target data table, the target verification strategy including a verification strategy based on data content or a verification strategy based on statistical information;

Acquire the write data state of the source data table, the write data state includes a locked state and an unlocked state, the locked state is used to indicate that writing data to the source data table is suspended, and the unlocked state Can instruct to write data to the source data table;

If the write data state of the source data table is the locked state, verify whether the target data table is consistent with the source data table according to the target verification strategy.

In a second aspect, an embodiment of the present application provides a data verification device, which includes:

The acquisition module is used to acquire the target data table, the source data table, and the attribute information of the target data table. The attribute information of the target data table includes any one or more of the following: the data volume of the target data table, The type of the target data table, the number of times the target data table is queried, and the time when the target data table is established;

A determining module, configured to determine a target verification strategy for the consistency verification of the target data table according to the attribute information of the target data table, the target verification strategy including a verification strategy based on data content or a verification strategy based on statistical information;

The acquisition module is further configured to acquire the data write status of the source data table, the data write status includes a locked state and an unlocked state, and the locked state is used to indicate that writing to the source data table is suspended Data, the unlocked state may indicate writing data to the source data table;

The verification module is configured to verify whether the target data table is consistent with the source data table according to the target verification strategy if the write data state of the source data table is locked.

In a third aspect, an embodiment of the present application provides an electronic device, the electronic device includes: a processor, adapted to implement one or more instructions; and,

A computer non-volatile readable storage medium, the computer non-volatile readable storage medium stores one or more instructions, and the one or more instructions are suitable for being loaded by the processor and executed as steps:

In a fourth aspect, embodiments of the present application provide a computer non-volatile readable storage medium, the computer non-volatile readable storage medium stores one or more instructions, and the one or more instructions are suitable for Loaded by the processor and execute the following steps:

In the embodiment of the present application, the electronic device can select a target verification strategy for verifying the consistency of the target data table according to the attribute information of the target data table, and verify according to the target verification strategy when the write data status of the source data table is locked. Whether the target data table is consistent with the source data table; the consistency verification strategy can be dynamically selected according to the attribute information of the target data table to improve the flexibility of consistency verification. Automatically verify the consistency of the target data table according to the target verification strategy, without manual participation, and improve the efficiency of consistency verification. In addition, when the write data status of the source data table is locked, the consistency verification of the target data table can avoid the interference caused by writing data to the source data table and improve the efficiency of consistency verification on the target data table.

Description of the drawings

In order to more clearly describe the technical solutions of the embodiments of the present application, the following will briefly introduce the accompanying drawings used in the description of the embodiments. Obviously, the accompanying drawings in the following description are some embodiments of the present application. Ordinary technicians can obtain other drawings based on these drawings without creative work.

FIG. 1 is a schematic flowchart of a data verification method provided by an embodiment of the present application;

2 is a schematic flowchart of another data verification method provided by an embodiment of the present application;

FIG. 3 is a schematic structural diagram of a data verification device provided by an embodiment of the present application;

Fig. 4 is a schematic structural diagram of an electronic device provided by an embodiment of the present application.

detailed description

The technical solutions in the embodiments of the present application will be described clearly and completely in conjunction with the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, rather than all of them. Based on the embodiments in this application, all other embodiments obtained by those of ordinary skill in the art without creative work shall fall within the protection scope of this application.

Based on the problem of low efficiency of data consistency verification methods in the prior art, embodiments of this application provide a data verification method, which can be executed by an electronic device, which can refer to a smart terminal, a server, or a computer, etc. A device that can be used for storage. The specific implementation process of this method includes: the electronic device can obtain the target data table to be verified, and the attribute information of the target data table, and determine the target for consistency verification of the target data table according to the attribute information of the target table Verification strategy, according to the target verification strategy to verify whether the target data table is consistent with the source data table; the consistency verification strategy can be flexibly selected according to the attribute information of the target data table, and the consistency verification of the target data table is automatically performed according to the selected consistency verification strategy , Improve the efficiency of verification; do not require manual participation, save manpower, to reduce the cost of data verification.

Please refer to FIG. 1, which is a schematic flowchart of a data verification method provided by an embodiment of the present application. The method of the embodiment of the present application may be executed by the above-mentioned electronic device. In this embodiment, the data verification method includes the following steps.

S101. Obtain a target data table, a source data table, and attribute information of the target data table. The attribute information of the target data table includes any one or more of the following: the amount of data in the target data table, the type of the target data table, The number of times the target data table was queried and the time when the target data table was created.

In one embodiment, the management database system usually consists of one management node and multiple processing nodes. The management node includes all data tables in the cluster database system. The electronic device in this embodiment of the application is the processing node. In order to process data services, the electronic device can copy the source data table from the management node to obtain the target data table, and store the target data table in the local database. In the process of copying the source data table, the data table is tampered with, which results in inconsistencies between the target data table and the source data table, which affects the normal processing of data services. Therefore, in this scenario, the source data table and the target data table need to be verified for consistency. The electronic device can obtain the source data table from the main management node, and obtain the target data table and the attribute information of the target data table from the local database.

In another embodiment, in a scenario where the database management system needs to be replaced or upgraded, or when the storage space of the database management system is insufficient, the source data table in the database management needs to be backed up to obtain the backup data (ie The target data table), because in the process of backing up the source data table, the target data table is inconsistent with the source data table due to equipment performance or operational errors. Therefore, in this scenario, it is necessary to verify the consistency of the source data table and the target data table. The electronic device can obtain the source data table from the original database management system, and obtain the target data table and the target data table from the database management system after the replacement or upgrade. The attribute information of the target data table.

In one embodiment, the historical consistency verification identifier of the data table in the data management system is acquired. The historical consistency verification identifier refers to the identifier of the data table that has been verified for consistency. According to the historical consistency verification identifier, the data The management system obtains the data table that has not been verified for consistency, and uses the data table that has not been verified for consistency as the target data table.

The data management system can include multiple data tables. In order to avoid consistency verification of all data tables, the electronic device can perform consistency verification on newly added data tables in the database management system to save resources. Specifically, the electronic device can obtain the historical consistency verification flag of the data table in the data management system, and obtain the data table that has not been verified for consistency from the data management system according to the historical consistency verification flag, that is, the data table that has not been verified for consistency The data table can be a newly-added data table, and the electronic device can use the data table that has not been verified for consistency as the target data table, so as to perform consistency verification on the newly-added data table. Wherein, the historical consistency verification identifier refers to the identifier of the data table that has been verified for consistency, and the identifier of the data table may be the name, number, or storage address of the data table, etc.

S102: Determine a target verification strategy for consistency verification of the target data table according to the attribute information of the target data table, where the target verification strategy includes a verification strategy based on data content or a verification strategy based on statistical information.

In order to verify the flexibility of the target data table, the electronic device can determine the target verification strategy for the consistency verification of the target data table according to the attribute information of the target data table, where the target verification strategy may include a verification strategy based on data content Or the verification strategy based on statistical information, the verification strategy based on data content needs to verify the data content of the data table, that is, the verification strategy based on data content can be a more sophisticated verification strategy; the verification strategy based on statistical information needs to verify the data table The statistical information is verified, that is, the verification strategy based on the statistical information can be a rough verification strategy.

S103. Obtain the data write status of the source data table.

In order to avoid the inconsistency between the target data table and the source data table due to writing data to the source data table in the process of verifying the consistency of the target data table, the electronic device can obtain the write data status of the target data table. The data state can be an unlocked state and a locked state. The unlocked state can mean that data can be written to the source data table, and the locked state can mean that data writing to the source data table is suspended.

In one embodiment, step S103 includes: obtaining log data about the source data table, and determining the number of times that data is written to the source data table in the current time period according to the log data in the source data table; If the number of times is less than the preset number, it is determined that the data write status of the source data table is locked; if the number of times of writing data to the source data table in the current time period is less than or equal to the preset number of times, it is determined that the data write status of the source data table is not Locked state. For example, the current time period is from 2:00 to 3:00 at night. According to the log data of the source data table, the total number of times to write data to the source data table is determined to be 100 times. Only one time is to write data to the source data table in the current time period. Therefore, it is determined that the probability of writing data to the source data table in the current time period is low, and it can be determined that the data writing state of the source data table is the locked state.

In one embodiment, the database management system can include the data writing function of the source data table. When the electronic device writes data to the source data table, the data writing function of the source data table can be turned on; when the consistency of the target data table needs to be detected When it is not, you can turn off the data writing function of the source data table. Therefore, the electronic device can detect the on or off state of the data writing function of the source data table, and determine the data writing state of the source data table. For example, if it is detected that the data writing function of the source data table is turned on, it is determined that the data writing state of the source data table is unlocked; if it is detected that the data writing function of the source data table is off, it is determined that the source The write data status of the data table is locked.

S104: If the write data state of the source data table is the locked state, verify whether the target data table is consistent with the source data table according to the target verification strategy.

In the embodiment of the present application, if the write data state of the source data table is locked, it can be verified whether the target data table is consistent with the source data table according to the target verification strategy.

In one embodiment, after step S104, it further includes: generating a verification report according to the verification result, and returning the verification report, so that the user can understand the verification result in real time. The verification report may include the verification time, the result of whether the target data table is consistent with the source data table, and the data content of the inconsistent data when the target data table is inconsistent with the source data table, and so on.

In one embodiment, if the amount of data in the database management system storing the target data table is greater than the preset data amount, the number of times each data table in the database management system has been queried is acquired, and the number of queries is less than the preset number of times. The data table and the corresponding source data table are deleted.

In order to alleviate the pressure of the database management system to store data, if the amount of data in the database management system storing the target data table is greater than the preset amount of data, it indicates that the storage space of the database management system is insufficient, and the electronic device can obtain each data in the database management system. The number of times that the data table is queried. If the number of queries of the data table is less than the preset number, it indicates that the importance of the data table is low, and the data table with the number of queries less than the preset number and the corresponding source data table can be deleted; If the number of queries is greater than or equal to the preset number, it indicates that the importance of the data table is high, and the data table can be retained.

Please refer to FIG. 2, which is a schematic flowchart of another data verification method provided by an embodiment of the present application. The method of the embodiment of the present application may be executed by the above-mentioned electronic device. The data verification method includes the following steps.

S201. Obtain a target data table, a source data table, and attribute information of the target data table, where the attribute information of the target data table includes any one or more of the following: the amount of data in the target data table, the type of the target data table, The number of times the target data table was queried and the time when the target data table was created.

S202: Determine a target verification strategy for consistency verification of the target data table according to the attribute information of the target data table, where the target verification strategy includes a verification strategy based on data content or a verification strategy based on statistical information.

In one embodiment, the attribute information of the target data table includes the type of the target data table, and step S202 includes: obtaining the importance level of the target data table according to the type of the target data table, if the importance level of the target data table is less than Preset level, the statistical information-based verification strategy is determined as the target verification strategy for the consistency verification of the target data table, and if the importance level of the target data table is greater than or equal to the preset level, the verification strategy based on the data content The verification strategy is determined as the target verification strategy for the consistency verification of the target data table.

Due to different types of data tables, users have inconsistencies in their importance. For data tables with higher importance, a more refined verification strategy can be used to verify the data tables to improve the accuracy of the verification; for data tables with lower importance , A rough verification strategy can be used to verify the data table to improve the efficiency of the verification; therefore, the verification strategy can be selected according to the type of the target data table. Specifically, the electronic device can determine the type of the target data table according to the data content in the target data table or the name of the target data table, and obtain the importance level of the target data table according to the type of the target data table. If the importance level is less than the preset level, it indicates that the importance of the target data table is relatively low, and the statistical information-based verification strategy can be determined as the target verification strategy for the consistency verification of the target data table, that is, a rough verification strategy is selected; If the importance level of the target data table is greater than or equal to the preset level, it indicates that the importance of the target data table is relatively high, and the verification strategy based on the data content can be determined as the target verification for the consistency verification of the target data table Strategy, that is, choose a sophisticated verification strategy.

For example, if the target data table type is an audit data table, the audit data table is usually related to the user’s finances and is of higher importance. Therefore, you can set a higher importance level for the target data table and select a data content-based The verification strategy verifies the consistency of the target data table; if the type of the target data table is an employee data table, the importance of the target data table is relatively low, and a lower importance level can be set for the target data table and can be selected The verification strategy based on statistical information verifies the consistency of the target data table.

For another example, if the type of the target data table is a sensitive data table including account numbers and passwords, the importance of the target data table is relatively high, and a higher importance level can be set for the target data table, and data-based The content verification strategy verifies the consistency of the target data table; if the type of the target data table is a data table including a list of hospitals in a certain area, the importance of the target data table is relatively low, and a lower value can be set for the target data table , And you can choose a verification strategy based on statistical information to verify the consistency of the target data table.

In another embodiment, the attribute information of the target data table includes the number of times the target data table is queried, and step S202 includes: obtaining the importance level of the target data table according to the number of times the target data table is queried. If the importance level of the table is less than the preset level, the statistical information-based verification strategy is determined as the target verification strategy for the consistency verification of the target data table. If the importance level of the target data table is greater than or equal to the preset level, then The verification strategy based on the data content is determined as the target verification strategy for the consistency verification of the target data table.

If the target data is queried more times, it indicates that the importance of the target data table is higher, and a more refined verification strategy can be used to verify the data table to improve the accuracy of the verification; on the contrary, if the target data table is queried If the number of times is less, it indicates that the importance of the target data table is low. A rough verification strategy can be used to verify the data table to improve the efficiency of verification; therefore, the verification strategy can be selected according to the number of times the target data table is queried. Specifically, the electronic device may determine the number of times the target data table is queried according to the log data of the target data table, and obtain the importance level of the target data table according to the number of times the target data table is queried; if the importance level of the target data table is If it is less than the preset level, it indicates that the importance of the target data table is relatively low, and the statistical information-based verification strategy is determined as the target verification strategy for the consistency verification of the target data table, that is, a relatively rough verification strategy is selected; If the importance level of the target data table is greater than or equal to the preset level, the importance of the target data table is relatively high, and the verification strategy based on the data content can be determined as the target verification strategy for the consistency verification of the target data table, namely Choose a sophisticated verification strategy.

In another embodiment, the attribute information of the target data table includes the number of times the target data table is queried and the type of the target data table, and step S202 includes: the electronic device may query the target data table according to the number of times the target data table is queried and the target data table. The type of the data table sets the first weight and the second weight for the target data table, and obtains the corresponding value of the target data table type, the number of times the target data table is queried, the corresponding value of the target data table type, Perform a weighted summation of the first weight and the second weight to obtain the weight of the target data table, and set the importance level of the target data table according to the weight of the target data table. If the importance level of the target data table is less than the preset level , The statistical information-based verification strategy is determined as the target verification strategy for the consistency verification of the target data table, and if the importance level of the target data table is greater than or equal to the preset level, the verification strategy based on the data content Determine the target verification strategy for the consistency verification of the target data table.

For example, the type of the target data table is an audit data table, the corresponding value of the target data table type is 3, the number of times the target data table is queried is 10, and the target data is set according to the number of times the target data table is queried The first weight value of the target data table is 5, and the second weight value of the target data table is set to 4 according to the type of the target data table. The number of times the target data table is queried, the corresponding value of the target data table type, and the first weight value Perform a weighted summation with the second weight, that is, 5*10+3*4=22, then the weight of the target data table is 22, and the importance level of the target data table can be set according to the weight of the target data table, namely The greater the weight, the greater the importance level. If the importance level of the target data table is less than the preset level, the statistical information-based verification strategy is determined as the target verification strategy for the consistency verification of the target data table. If the importance level of the table is greater than or equal to the preset level, the verification strategy based on the data content is determined as the target verification strategy for the consistency verification of the target data table.

In another embodiment, the attribute information of the target data table includes the data amount of the target data table, and step S202 includes: if the data amount of the target data table is greater than a preset value, determining the verification strategy based on statistical information It is the target verification strategy for the consistency verification of the target data table; if the data volume of the target data table is less than or equal to the preset value, the verification strategy based on the data content is determined as the consistency verification of the target data table Target verification strategy.

The more data included in the data table, the longer the time required to verify the data table, and the less data included in the data table, the shorter the time required to verify the data table; at the same time, verification based on statistical information The verification efficiency of the strategy is higher than that of the verification strategy based on the data content. Therefore, in order to improve the efficiency of data sheet verification, a verification strategy can be selected according to the number of data sheets included. Specifically, the electronic device can obtain the data volume of the target data table, and if the data volume of the target data table is greater than the preset value, the statistical information-based verification strategy is determined as the target verification strategy for the consistency verification of the target data table , That is, select a verification strategy with higher verification efficiency to improve verification efficiency; if the data amount of the target data table is less than or equal to the preset value, it indicates that the amount of data in the target data table itself is small, so it can be based on the data content The verification strategy is determined as the target verification strategy for the consistency verification of the target data table, and the accuracy of the verification of the target data table is improved.

In another embodiment, the attribute information of the target data table includes the time when the target data table is established, and step S202 includes: if the time interval between the system time and the time when the target data table is established is greater than the preset time interval, then Determine the statistical information-based verification strategy as the target verification strategy for the consistency verification of the target data table; if the time interval between the system time and the time when the target data table is established is less than or equal to the preset time interval, then The verification strategy based on the data content is determined as the target verification strategy for the consistency verification of the target data table.

The electronic device can select a verification strategy according to the establishment time of the target data table. Specifically, if the time interval between the system time and the establishment time of the target data table is greater than the preset time interval, it indicates the probability that the target data table has been verified Therefore, the statistical information-based verification strategy can be determined as the target verification strategy for the consistency verification of the target data table to improve the verification efficiency; if the time interval between the system time and the time when the target data table is established Less than or equal to the preset time interval, it indicates that the probability that the target data table has been verified is low. Therefore, the verification strategy based on the data content can be determined as the target verification strategy for the consistency verification of the target data table. Improve the accuracy of verification.

S203: Acquire the data writing status of the source data table.

S204: If the write data state of the source data table is the locked state, verify whether the target data table is consistent with the source data table according to the verification strategy based on the data content.

In one embodiment, step S204 includes: comparing the data content of the target data table with the data content of the source data table, and if the data content of the target data table is the same as the data content of the source data table, determining There is consistency between the target data table and the source data table; if the data content of the target data table is different from the data content of the source data table, it is determined that the target data table is inconsistent with the source data table.

When the target verification strategy is a verification strategy based on data content, the electronic device can compare the data content of the target data table with the data content of the source data table, that is, the correspondence between each row of the target data table and the source data table Compare the row data. If the data content of the target data table is the same as the data content of the source data table, that is, each row of data in the target data table is the same as the corresponding row data of the source data table, indicating that the target data table and the source data The tables are the same, that is, it is determined that there is consistency between the target data table and the source data table; if the data content of the target data table is different from the data content of the source data table, that is, at least one row of data in the target data table is The corresponding row data of the source data table is different, indicating that the data content of the target data table has been tampered with, and it is determined that the target data table is inconsistent with the source data table.

In another embodiment, step S204 includes: the electronic device may encrypt the data content of the target data table to obtain the first hash value, and obtain the second hash value obtained by encrypting the data content of the source data table , Compare the first hash value with the second hash value, if the two are the same, determine the consistency between the target data table and the source data table; if the two are not the same, determine the target data The table is inconsistent with the source data table.

In order to improve the efficiency of verification and prevent users from tampering with the data content of the source data table, the electronic device can compare the hash value of the data content of the target data table with the hash value of the data content of the source data table, if the two are the same , It is determined that there is consistency between the target data table and the source data table; if the two are not the same, it is determined that the target data table is inconsistent with the source data table.

S205: If the write data state of the source data table is the locked state, verify whether the target data table is consistent with the source data table according to a verification strategy based on statistical information.

In one embodiment, step S205 includes: obtaining statistical information of the target data table and the source data table, and the statistical information includes any one or more of the following: data amount, field number, row number, and column number. If the statistical information of the target data table is the same as the attribute information of the source data table, it is determined that there is consistency between the target data table and the source data table; if the statistical information of the target data table is different from the attribute information of the source data table If they are the same, it is determined that the target data table is inconsistent with the source data table.

When the target verification strategy is a verification strategy based on statistical information, the electronic device can obtain the statistical information of the target data table and the source data table. If the statistical information of the target data table is the same as the attribute information of the source data table, then Determine the consistency between the target data table and the source data table, that is, determine that the data content of the target data table is the same as the data content of the source data table; if the statistical information of the target data table is the same as the attribute of the source data table If the information is not the same, it indicates that the data content of the target data table is inconsistent with the data content of the source data table, and it is determined that the target data table is inconsistent with the source data table.

In the embodiment of the present application, the electronic device may select a target verification strategy for performing consistency verification on the target data table according to the attribute information of the target data table. If the target verification strategy is a verification strategy based on data content, and the source data table is written If the data state is locked, a verification strategy based on data content can be used to verify whether the target data table is consistent with the source data table; if the target verification strategy is a verification strategy based on statistical information, and the write data state of the source data table is If the state is locked, a statistical information-based verification strategy can be used to verify whether the target data table is consistent with the source data table; the consistency verification strategy can be dynamically selected according to the attribute information of the target data table to improve the flexibility of consistency verification. Automatically verify the consistency of the target data table according to the target verification strategy, without manual participation, and improve the efficiency of consistency verification. In addition, when the write data status of the source data table is locked, the consistency verification of the target data table can avoid the interference caused by writing data to the source data table and improve the efficiency of consistency verification on the target data table.

Refer to FIG. 3, which is a schematic structural diagram of a data verification apparatus provided by an embodiment of the present application. The apparatus of the embodiment of the present application may be set in the aforementioned electronic device. In this embodiment, the device includes:

The obtaining module 301 is configured to obtain a target data table, a source data table, and attribute information of the target data table. The attribute information of the target data table includes any one or more of the following: the data volume of the target data table, The type of the target data table, the number of times the target data table is queried, and the time when the target data table is established;

The determining module 302 is configured to determine a target verification strategy for consistency verification of the target data table according to the attribute information of the target data table, the target verification strategy including a verification strategy based on data content or a verification strategy based on statistical information;

The acquiring module 301 is also used to acquire the write data state of the source data table, the write data state includes a locked state and an unlocked state, and the locked state is used to indicate a pause to the source data table. Writing data, the unlocked state may indicate writing data to the source data table;

The verification module 303 is configured to verify whether the target data table is consistent with the source data table according to the target verification strategy if the write data state of the source data table is the locked state.

Optionally, the attribute information of the target data table includes the type of the target data table and/or the number of times the target data table is queried. The determining module 302 is specifically configured to obtain the attribute information of the target data table. The importance level of the target data table; if the importance level of the target data table is less than a preset level, the statistical information-based verification strategy is determined as the target verification strategy for the consistency verification of the target data table; if If the importance level of the target data table is greater than or equal to the preset level, the verification strategy based on the data content is determined as the target verification strategy for the consistency verification of the target data table.

Optionally, the attribute information of the target data table includes the data volume of the target data table or the time when the target data table is established, and the determining module 302 is specifically configured to: if the data volume of the target data table is greater than a preset Value, or the time interval between the system time and the time when the target data table is established is greater than the preset time interval, then the statistical information-based verification strategy is determined as the target verification strategy for the consistency verification of the target data table ; If the data volume of the target data table is less than or equal to the preset value, or the time interval between the system time and the time when the target data table is established is less than or equal to the preset time interval, then the The verification strategy based on the data content is determined as the target verification strategy for the consistency verification of the target data table.

Optionally, the verification module 303 is specifically configured to compare the data content of the target data table with the data content of the source data table; if the data content of the target data table is compared with the data content of the source data table If the content is the same, it is determined that there is consistency between the target data table and the source data table; if the data content of the target data table is different from the data content of the source data table, the target data table is determined Inconsistent with the source data table.

Optionally, the verification module 303 is specifically configured to obtain statistical information of the target data table and the source data table, where the statistical information includes any one or more of the following: data amount, field number, row number, and column If the statistical information of the target data table is the same as the statistical information of the source data table, it is determined that there is consistency between the target data table and the source data table; if the statistical information of the target data table If the statistical information is different from the source data table, it is determined that the target data table is inconsistent with the source data table.

Optionally, the acquiring module 301 is specifically configured to acquire the historical consistency verification identifier of the data table in the data management system. The historical consistency verification identifier refers to the identifier of the data table that has been verified for consistency; according to the history The consistency verification identifier obtains a data table that has not been verified for consistency from the data management system; and uses the data table that has not been verified for consistency as a target data table.

Optionally, the deleting module 304 is configured to, if the amount of data in the database management system storing the target data table is greater than the preset data amount, obtain the number of times each data table in the database management system has been queried; The data tables whose times are less than the preset times and the corresponding source data tables are deleted.

Refer to FIG. 4, which is a schematic structural diagram of an electronic device provided by an embodiment of the present application. As shown in the figure, the electronic device in this embodiment may include: one or more processors 401; one or more input devices 402 , One or more output devices 403 and memory 404. The aforementioned processor 401, input device 402, output device 403, and memory 404 are connected via a bus 405.

The processor 401 may be a central processing unit (Central Processing Unit, CPU), and the processor may also be other general-purpose processors, digital signal processors (Digital Signal Processors, DSP), application specific integrated circuits (ASICs). ), Field-Programmable Gate Array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or the processor may also be any conventional processor or the like.

The input device 402 may include a touch panel, a fingerprint sensor (used to collect user fingerprint information and fingerprint orientation information), a microphone, etc., and the output device 403 may include a display (LCD, etc.), a speaker, and the like.

The memory 404 may include a read-only memory and a random access memory, and provides instructions and data to the processor 401. A part of the memory 404 may also include a non-volatile random access memory. The memory 404 is used to store a computer program. The computer program includes program instructions. The processor 401 is used to execute the program instructions stored in the memory 404 to execute a A data verification method that is used to perform the following operations:

Optionally, the processor is used to call the program instructions stored in the memory to perform the following operations:

Acquiring the importance level of the target data table according to the attribute information of the target data table;

If the importance level of the target data table is less than the preset level, determining the verification strategy based on statistical information as the target verification strategy for the consistency verification of the target data table;

If the importance level of the target data table is greater than or equal to the preset level, the verification strategy based on the data content is determined as the target verification strategy for the consistency verification of the target data table.

If the data volume of the target data table is greater than the preset value, or the time interval between the system time and the time when the target data table is established is greater than the preset time interval, the statistical information-based verification strategy is determined to be correct The target verification strategy for the consistency verification of the target data table;

If the data amount of the target data table is less than or equal to the preset value, or the time interval between the system time and the time when the target data table is established is less than or equal to the preset time interval, then the The verification strategy of the data content is determined as the target verification strategy for the consistency verification of the target data table.

Comparing the data content of the target data table with the data content of the source data table;

If the data content of the target data table is the same as the data content of the source data table, it is determined that there is consistency between the target data table and the source data table;

If the data content of the target data table is different from the data content of the source data table, it is determined that the target data table is inconsistent with the source data table.

Acquire statistical information of the target data table and the source data table, where the statistical information includes any one or more of the following: data amount, field number, row number, and column number;

If the statistical information of the target data table is the same as the statistical information of the source data table, it is determined that there is consistency between the target data table and the source data table;

If the statistical information of the target data table is different from the statistical information of the source data table, it is determined that the target data table is inconsistent with the source data table.

Acquiring the historical consistency verification identifier of the data table in the data management system, where the historical consistency verification identifier refers to the identifier of the data table that has been verified for consistency;

Obtaining a data table that has not been verified for consistency from the data management system according to the historical consistency verification identifier;

The data table that has not been verified for consistency is used as the target data table.

If the amount of data in the database management system storing the target data table is greater than the preset data amount, acquiring the number of times each data table in the database management system is queried;

Delete the data table and the corresponding source data table whose query times are less than the preset times.

The processor 401, input device 402, and output device 403 described in the embodiments of this application can execute the implementations described in the first and second embodiments of the data verification method provided in the embodiments of this application, and can also execute The implementation of the electronic device described in the embodiment of the application will not be repeated here.

The embodiment of the present application also provides a computer non-volatile readable storage medium, the computer non-volatile readable storage medium stores a computer program, the computer program includes program instructions, and the program instructions are processed The data verification method shown in the embodiment of FIG. 1 and FIG. 2 of the present application is implemented when the device is executed.

The computer non-volatile readable storage medium may be the internal storage unit of the medical management device described in any of the foregoing embodiments, such as a hard disk or memory of the control device. The computer non-volatile readable storage medium may also be an external storage device of the control device, such as a plug-in hard disk equipped on the control device, a Smart Media Card (SMC), a secure digital ( Secure Digital, SD card, Flash Card, etc. Further, the computer non-volatile readable storage medium may also include both an internal storage unit of the control device and an external storage device. The computer non-volatile readable storage medium is used to store the computer program and other programs and data required by the control device. The computer non-volatile readable storage medium can also be used to temporarily store data that has been output or will be output.

A person of ordinary skill in the art may realize that the units and algorithm steps of the examples described in the embodiments disclosed herein can be implemented by electronic hardware, computer software, or a combination of the two, in order to clearly illustrate the hardware and software Interchangeability. In the above description, the composition and steps of each example have been generally described in terms of function. Whether these functions are executed by hardware or software depends on the specific application and design constraint conditions of the technical solution. Professionals and technicians can use different methods for each specific application to implement the described functions, but such implementation should not be considered beyond the scope of this application. Those skilled in the art can clearly understand that, for the convenience and conciseness of the description, the specific working process of the control device and unit described above can refer to the corresponding process in the foregoing method embodiment, which will not be repeated here.

In the several embodiments provided in this application, it should be understood that the disclosed control device and method can be implemented in other ways. For example, the device embodiments described above are illustrative. For example, the division of the units may be a logical function division, and there may be other divisions in actual implementation, for example, multiple units or components may be combined or may be Integrate into another system, or some features can be ignored or not implemented. The above are only specific implementations of this application, but the protection scope of this application is not limited to this. Anyone familiar with the technical field can easily think of various equivalents within the technical scope disclosed in this application. Modifications or replacements, these modifications or replacements shall be covered within the protection scope of this application. Therefore, the protection scope of this application shall be subject to the protection scope of the claims.

Claims

A data verification method, characterized in that it comprises:

Obtain the target data table, the source data table, and the attribute information of the target data table. The attribute information of the target data table includes any one or more of the following: the data volume of the target data table, Type, the number of times the target data table is queried, and the time when the target data table is established;

Determining a target verification strategy for consistency verification of the target data table according to the attribute information of the target data table, the target verification strategy including a verification strategy based on data content or a verification strategy based on statistical information;

Acquire the write data state of the source data table, the write data state includes a locked state and an unlocked state, the locked state is used to indicate that writing data to the source data table is suspended, and the unlocked state Can instruct to write data to the source data table;

If the write data state of the source data table is the locked state, verify whether the target data table is consistent with the source data table according to the target verification strategy.
The method according to claim 1, wherein the attribute information of the target data table includes the type of the target data table and/or the number of times the target data table is queried, and the target data table The attribute information of determines the target verification strategy for the consistency verification of the target data table, including:

Acquiring the importance level of the target data table according to the attribute information of the target data table;

If the importance level of the target data table is less than the preset level, determining the verification strategy based on statistical information as the target verification strategy for the consistency verification of the target data table;

If the importance level of the target data table is greater than or equal to the preset level, the verification strategy based on the data content is determined as the target verification strategy for the consistency verification of the target data table.
The method according to claim 1, wherein the attribute information of the target data table includes the amount of data of the target data table or the time when the target data table is established, and the attribute information of the target data table is The information determines the target verification strategy for the consistency verification of the target data table, including:

If the data volume of the target data table is greater than the preset value, or the time interval between the system time and the time when the target data table is established is greater than the preset time interval, the statistical information-based verification strategy is determined to be correct The target verification strategy for the consistency verification of the target data table;

If the data amount of the target data table is less than or equal to the preset value, or the time interval between the system time and the time when the target data table is established is less than or equal to the preset time interval, then the The verification strategy of the data content is determined as the target verification strategy for the consistency verification of the target data table.
The method according to any one of claims 1-3, wherein the target verification strategy includes a verification strategy based on data content, and the target data table and the source data are verified according to the target verification strategy. Whether the tables are consistent, including:

Comparing the data content of the target data table with the data content of the source data table;

If the data content of the target data table is the same as the data content of the source data table, it is determined that there is consistency between the target data table and the source data table;

If the data content of the target data table is different from the data content of the source data table, it is determined that the target data table is inconsistent with the source data table.
The method according to any one of claims 1-3, wherein the target verification strategy comprises a verification strategy based on statistical information, and the target data table and the source data are verified according to the target verification strategy Whether the tables are consistent, including:

Acquire statistical information of the target data table and the source data table, where the statistical information includes any one or more of the following: data amount, field number, row number, and column number;

If the statistical information of the target data table is the same as the statistical information of the source data table, it is determined that there is consistency between the target data table and the source data table;

If the statistical information of the target data table is different from the statistical information of the source data table, it is determined that the target data table is inconsistent with the source data table.
The method according to claim 1, wherein said acquiring a target data table comprises:

Acquiring the historical consistency verification identifier of the data table in the data management system, where the historical consistency verification identifier refers to the identifier of the data table that has been verified for consistency;

Obtaining a data table that has not been verified for consistency from the data management system according to the historical consistency verification identifier;

The data table that has not been verified for consistency is used as the target data table.
The method of claim 1, wherein the method further comprises:

If the amount of data in the database management system storing the target data table is greater than the preset data amount, acquiring the number of times each data table in the database management system is queried;

Delete the data table and the corresponding source data table whose query times are less than the preset times.
The method according to claim 1, wherein said obtaining the write data status of the source data table comprises:

Acquiring log data about the source data table;

Determining the number of times of writing data to the source data table in the current time period according to the log data of the source data table;

If the number of times of writing data to the source data table in the current time period is less than the preset number of times, determining that the data writing state of the source data table is a locked state;

If the number of times of writing data to the source data table in the current time period is greater than or equal to the preset number of times, it is determined that the data writing state of the source data table is an unlocked state.
The method according to claim 1, wherein said obtaining the write data status of the source data table comprises:

If it is detected that the data writing function of the source data table is in an on state, it is determined that the data writing state of the source data table is an unlocked state;

If it is detected that the data writing function of the source data table is in a closed state, it is determined that the data writing state of the source data table is a locked state.
The method of claim 1, wherein the method further comprises:

Encrypting the data content of the target data table to obtain a first hash value;

The second hash value obtained by encrypting the data content of the source data table;

Comparing the first hash value with the second hash value;

If the first hash value is the same as the second hash value, it is determined that there is consistency between the target data table and the source data table.
The method of claim 1, wherein the method further comprises:

Generating a verification report according to the verification result, the verification report including verification time and result information on whether the target data table is consistent with the source data table;

Return the verification report.
A data verification device is characterized in that it comprises:

The acquisition module is used to acquire the target data table, the source data table, and the attribute information of the target data table. The attribute information of the target data table includes any one or more of the following: the data volume of the target data table, The type of the target data table, the number of times the target data table is queried, and the time when the target data table is established;

A determining module, configured to determine a target verification strategy for verifying the consistency of the target data table according to the attribute information of the target data table, the target verification strategy including a verification strategy based on data content or a verification strategy based on statistical information;

The acquisition module is further configured to acquire the data write status of the source data table, the data write status includes a locked state and an unlocked state, and the locked state is used to indicate that writing to the source data table is suspended Data, the unlocked state is used to indicate to write data to the source data table;

The verification module is configured to verify whether the target data table is consistent with the source data table according to the target verification strategy if the write data state of the source data table is locked.
The device according to claim 12, wherein the attribute information of the target data table includes the type of the target data table and/or the number of times the target data table is queried, and the determining module is specifically configured to The importance level of the target data table is obtained according to the attribute information of the target data table; if the importance level of the target data table is less than a preset level, the statistical information-based verification strategy is determined to be the target data The target verification strategy for table consistency verification; if the importance level of the target data table is greater than or equal to the preset level, the data content-based verification strategy is determined as the target of the target data table consistency verification Verification strategy.
The device according to claim 12, wherein the attribute information of the target data table includes the amount of data of the target data table or the time when the target data table is established, and the determining module is specifically configured to: If the amount of data in the target data table is greater than a preset value, or the time interval between the system time and the time when the target data table is established is greater than the preset time interval, the statistical information-based verification strategy is determined to be Target verification strategy for consistency verification of target data tables;

If the data amount of the target data table is less than or equal to the preset value, or the time interval between the system time and the time when the target data table is established is less than or equal to the preset time interval, then the The verification strategy of the data content is determined as the target verification strategy for the consistency verification of the target data table.
The device according to any one of claims 12-14, wherein the verification module is specifically configured to compare the data content of the target data table with the data content of the source data table; If the data content of the target data table is the same as the data content of the source data table, it is determined that there is consistency between the target data table and the source data table;

If the data content of the target data table is different from the data content of the source data table, it is determined that the target data table is inconsistent with the source data table.
The device according to any one of claims 12-14, wherein the verification module is specifically configured to obtain statistical information of the target data table and the source data table, and the statistical information includes any of the following Item or multiple items: the amount of data, the number of fields, the number of rows, and the number of columns; if the statistical information of the target data table is the same as the statistical information of the source data table, the target data table and the source data table are determined If the statistical information of the target data table is different from the statistical information of the source data table, it is determined that the target data table is inconsistent with the source data table.
The device according to claim 12, wherein the acquiring module is specifically configured to acquire a historical consistency verification identifier of a data table in a data management system, and the historical consistency verification identifier refers to the consistency verification. The identifier of the data table; the data table that has not been verified for consistency is obtained from the data management system according to the historical consistency verification identifier; the data table that has not been verified for consistency is used as the target data table.
The device according to claim 12, wherein the device further comprises:

The deleting module is used to obtain the number of times each data table in the database management system has been queried if the data volume in the database management system storing the target data table is greater than the preset data volume; and reduce the number of queries to be less than the preset number The data table and the corresponding source data table are deleted.
An electronic device, characterized in that it comprises:

Processor, suitable for implementing one or more instructions; and,

A computer non-volatile readable storage medium, the computer non-volatile readable storage medium stores one or more instructions, and the one or more instructions are suitable for being loaded by the processor and executed as in claim 1 -11 Any of the data verification methods.
A computer nonvolatile readable storage medium, wherein the computer nonvolatile readable storage medium stores one or more instructions, and the one or more instructions are suitable for being loaded and executed by a processor The data verification method according to any one of claims 1-11.