WO2021159639A1

WO2021159639A1 - Checking method, apparatus, and device for data migration, and storage medium

Info

Publication number: WO2021159639A1
Application number: PCT/CN2020/093187
Authority: WO
Inventors: 兰东平
Original assignee: 平安科技（深圳）有限公司
Priority date: 2020-02-12
Filing date: 2020-05-29
Publication date: 2021-08-19
Also published as: CN111290998A

Abstract

A checking method, apparatus, and device for data migration, and a storage medium. The method comprises: when a data migration instruction is detected, obtaining a file to be migrated before migration in a source and a file to be verified after migration in a cloud (S10); calculating sub-standard digest hash values corresponding to sub-files to be migrated in the file to be migrated, and calculating, on the basis of the sub-standard digest hash values and the sizes of the sub-files to be migrated, a total standard digest hash value corresponding to the file to be migrated (S20); calculating sub-digest hash values to be verified corresponding to sub-files to be verified in the file to be verified, and calculating, on the basis of the sub-digest hash values to be verified and the sizes of the sub-files to be verified, a total digest hash value to be verified corresponding to the file to be verified (S30); and checking the file to be verified according to the total standard digest hash value and the total digest hash value to be verified, if the total standard digest hash value and the total digest hash value to be verified are the same, determining that the file to be verified is successfully migrated, and generating a corresponding file migration success prompt message (S40). The method reduces the number of file comparisons, improves the file migration efficiency, and reduces the migration cost.

Description

Proofreading method, device, equipment and storage medium of migrated data

Cross-references to related applications

This application affirms that it enjoys the priority of the Chinese patent application with the application number CN202010091715.8 filed on February 12, 2020, and the title of "The proofreading method, device, equipment and storage medium of migrated data", and the overall content of the Chinese patent application Incorporated in this application by reference.

Technical field

This application relates to the field of data processing technology, and in particular to a proofreading method, device, device, and computer-readable storage medium for migrating data.

Background technique

With the promotion of cloud computing, more and more users need to migrate data to the cloud for storage, especially object storage. The amount of migrated data is very large, reaching terabytes or even petabytes. In the process of uploading data to object storage, there may be errors such as missed migration, wrong migration, and loss of data packets during network transmission. In order to ensure the completeness of the data, data reconciliation must be done. The traditional method of data reconciliation often compares the data before and after the upload one by one. In different scenarios, the method of comparison is also different. If you compare the file size and MD5 (Message-Digest Algorithm) one by one, once a file is found to be incorrect, you need to re-upload the original file and repeat the above data proofreading steps again until all are completed. The inventor realizes that although the above-mentioned method of file-by-file comparison can ensure the accuracy, because data calibration is performed on a file-by-file basis, the file size and MD5 must be transmitted through the network for each data calibration, which not only reduces the efficiency of data calibration, but also increases data calibration. cost. Therefore, how to solve the technical problems of low efficiency of data proofreading and high cost of data proofreading has become a technical problem to be solved urgently at present.

Summary of the invention

The main purpose of this application is to provide a proofreading method, device, device, and computer-readable storage medium for migrating data, aiming to solve the existing technical problems of low data proofreading efficiency and high data proofing cost.

In order to achieve the above objective, this application provides a proofreading method for migrated data, and the proofreading method for migrated data includes the following steps:

When the data migration instruction is detected, the files to be migrated before migration and the files to be verified after migration in the cloud are obtained from the source;

Calculate the sub-standard digest hash value corresponding to each sub-file to be migrated in the file to be migrated, and calculate the total standard digest hash value corresponding to the file to be migrated based on the hash value of each sub-standard digest and the size of each sub-file to be migrated Column value

Calculate the sub-to-be-verified digest hash value corresponding to each sub-file to be verified in the file to be verified, and calculate the to-be-verified sub-file based on the hash value of each sub-to-be-verified digest and the size of each sub-file to be verified The total hash value of the digest to be verified corresponding to the verification file;

The document to be verified is collated according to the total standard digest hash value and the total digest hash value to be verified, if the total standard digest hash value and the total digest hash value to be verified If they are the same, it is determined that the file to be verified is successfully migrated, and a corresponding file migration successful reminder message is generated.

In addition, in order to achieve the above objective, the present application also provides a proofreading device for migrated data, and the proofreading device for migrated data includes:

The migration file determination module is used to obtain the files to be migrated before migration and the files to be verified after migration in the cloud when the data migration instruction is detected;

The first MD5 calculation module is used to calculate the sub-standard digest hash value corresponding to each sub-file to be migrated in the file to be migrated, and calculate the to-be-migrated sub-file based on the hash value of each sub-standard digest and the size of each sub-file to be migrated. The total standard digest hash value corresponding to the migration file;

The second MD5 calculation module is used to calculate the sub-to-be-verified digest hash value corresponding to each sub-file to be verified in the file to be verified, and based on the hash value of each sub-to-be-verified digest and each to-be-verified Sub-file size, calculating the total hash value of the digest to be verified corresponding to the file to be verified;

The migration data proofreading module is used to proofread the file to be verified according to the total standard digest hash value and the total digest hash value to be verified. If the total standard digest hash value is compared with the total If the hash value of the digest to be verified is the same, it is determined that the file to be verified is successfully migrated, and a corresponding file migration success reminder message is generated.

In addition, in order to achieve the above-mentioned object, the present application also provides a proofreading device for migrated data. The proofreading device for migrated data includes a processor, a memory, and migrated data that is stored on the memory and can be executed by the processor. The proofreading program of the migration data, wherein when the proofreading program of the migration data is executed by the processor, the following steps of the proofreading method of the migration data are implemented:

In addition, in order to achieve the above object, this application also provides a computer-readable storage medium on which a proofreading program for migration data is stored, wherein the proofreading program for migration data is executed by a processor. The above-mentioned proofreading method of migration data has the following steps: when a data migration instruction is detected, the file to be migrated before migration and the file to be verified after migration in the cloud are obtained from the source end;

This application provides a proofreading method for migration data, by obtaining the files to be migrated before migration and the files to be verified after migration in the cloud when the data migration instruction is detected; calculating each of the files to be migrated The sub-standard summary hash value corresponding to the sub-file to be migrated is calculated, and the total standard summary hash value corresponding to the file to be migrated is calculated based on the hash value of each sub-standard summary and the size of each sub-file to be migrated; Verify the hash value of the sub-to-be-verified digest corresponding to each sub-file to be verified, and calculate the corresponding sub-file to be verified based on the hash value of each sub-to-be-verified digest and the size of each sub-file to be verified The total hash value of the digest to be verified; the document to be verified is collated according to the hash value of the total standard digest and the hash value of the total digest to be verified. If the total hash value of the digest to be verified is the same, it is determined that the file to be verified is successfully migrated, and a corresponding file migration success reminder message is generated. In the above manner, this application uses the hash value of each sub-standard digest of each sub-file to be migrated in the file to be migrated in the source and the value of each sub-file to be verified in the file to be verified after migration in the cloud. The standard MD5 value of each sub-file to be verified is respectively spliced and calculated to generate the total standard digest hash value corresponding to the file to be migrated and the total digest hash value to be verified corresponding to the file to be verified, and through direct comparison The total standard summary hash value and the total pending summary hash value are used to determine whether the file migration is abnormal, reduce the number of file comparisons, improve the efficiency of file migration, reduce migration costs, and solve the inefficiency of existing data proofreading. The technical problem of the high cost of data proofreading.

Summary of the invention

technical problem

The solution to the problem

The beneficial effects of the invention

Brief description of the drawings

Description of the drawings

FIG. 1 is a schematic diagram of the hardware structure of a proofreading device for migrating data involved in a solution of an embodiment of the application;

2 is a schematic flowchart of a first embodiment of a method for proofreading migrated data in an application;

FIG. 3 is a schematic flowchart of a second embodiment of a method for proofreading migrated data in an application;

4 is a schematic flowchart of a third embodiment of a method for proofreading migrated data in an application;

FIG. 5 is a schematic diagram of the functional modules of the first embodiment of the proofreading device for migrating data according to this application.

The realization, functional characteristics, and advantages of the purpose of this application will be further described in conjunction with the embodiments and with reference to the accompanying drawings.

Detailed ways

It should be understood that the specific embodiments described here are only used to explain the present application, and are not used to limit the present application.

The proofreading method of migrated data involved in the embodiments of this application is mainly applied to proofreading equipment for migrated data. The proofreading equipment for migrated data may be devices with display and processing functions such as PCs, portable computers, and mobile terminals.

Referring to FIG. 1, FIG. 1 is a schematic diagram of the hardware structure of the proofreading device for migrating data involved in the solution of the embodiment of the application. In the embodiment of the present application, the proofreading device for migrating data may include a processor 1001 (for example, a CPU), a communication bus 1002, a user interface 1003, a network interface 1004, and a memory 1005. Among them, the communication bus 1002 is used to realize the connection and communication between these components; the user interface 1003 may include a display (Display), an input unit such as a keyboard (Keyboard); the network interface 1004 may optionally include a standard wired interface, a wireless interface (Such as WI-FI interface); the memory 1005 can be a high-speed RAM memory, or a non-volatile memory (non-volatile memory), such as a disk memory. The memory 1005 can optionally be a storage device independent of the aforementioned processor 1001 .

Those skilled in the art can understand that the hardware structure shown in FIG. 1 does not constitute a limitation on the proofreading device for migrating data, and may include more or less components than shown in the figure, or a combination of certain components, or different components Layout.

Continuing to refer to FIG. 1, the memory 1005 as a computer-readable storage medium in FIG. 1 may include an operating system, a network communication module, and a proofreading program for migrating data.

In FIG. 1, the network communication module is mainly used to connect to the server and perform data communication with the server; and the processor 1001 can call the proofreading program of the migration data stored in the memory 1005, and execute the proofreading method of the migration data provided in the embodiment of the application .

The embodiment of the present application provides a proofreading method for migrated data.

Referring to FIG. 2, FIG. 2 is a schematic flowchart of a first embodiment of a method for proofreading migrated data in this application.

In this embodiment, the proofreading method of the migration data includes the following steps:

Step S10, when the data migration instruction is detected, the files to be migrated before migration and the files to be verified after migration in the cloud are obtained from the source end;

The traditional method of data reconciliation often compares the data before and after the upload one by one. In different scenarios, the method of comparison is also different. If you compare the file size and MD5 (Message-Digest Algorithm) one by one, once a file is found to be incorrect, you need to re-upload the original file and repeat the above data proofreading steps again until all are completed. Although the above-mentioned file-by-file comparison method can ensure accuracy, the file size and MD5 must be transmitted through the network for each file-by-file data calibration, which not only has low efficiency in data calibration, but also increases the cost of data calibration. In order to solve the above problems, this application adopts the standard MD5 value of each sub-file of each sub-file to be migrated in the file to be migrated in the source end, that is, the sub-standard digest hash value and the file to be verified after migration in the cloud. The MD5 value of each sub-file to be verified standard of each sub-file to be verified is respectively spliced and calculated to generate the total standard MD5 corresponding to the file to be migrated, that is, the total standard digest hash value and the total standard MD5 value corresponding to the file to be verified. Verify MD5, which is the total hash value of the digest to be verified, and determine the file migration by directly comparing the total standard MD5, which is the total standard digest hash value and the total to be verified MD5, which is the digest hash value of the total to be verified Whether there is an abnormality, the number of file comparisons is reduced, the file migration efficiency is improved, and the migration cost is reduced.

Wherein, before the step S10, it further includes:

According to a preset arrangement rule, the sub-files to be migrated in the source end are arranged in order, and the arranged sub-files to be migrated are sequentially uploaded to the cloud.

Wherein, according to the lexicographic order of the file name, the sub-files to be migrated in the source end are sequentially arranged, and the arranged sub-files to be migrated are sequentially uploaded to the cloud.

Specifically, MD5 (Message-Digest Algorithm, MD5 message digest algorithm) is a widely used cryptographic hash function that can generate a 128-bit (16-byte) hash value to ensure Information transmission is complete and consistent. There are many ways to arrange the files to be migrated. For example, if the files to be migrated are in various directories, that is, the file storage paths are different, they are sorted in lexicographical order of individual file names. For example, file file1 is ranked before file2. According to the sequence of the arranged files, the migration procedure of the files to be migrated is executed, that is, the files to be migrated are uploaded to the cloud one by one on demand from the source for storage.

Step S20: Calculate the hash value of the sub-standard summary corresponding to each sub-file to be migrated in the file to be migrated, and calculate the total corresponding to the file to be migrated based on the hash value of each sub-standard summary and the size of each sub-file to be migrated. Standard digest hash value;

In this embodiment, during the upload process, the source end calculates the sub-file standard MD5 corresponding to each sub-file to be migrated in the file to be migrated, that is, the sub-standard digest hash value, and saves each sub-file to be migrated sub-file Corresponding standard MD5. Wherein, the standard MD5 corresponding to the sub-file to be migrated may be that the MD5 is calculated only based on the size of the file to be migrated, or it may be that the MD5 is calculated according to the size of the file to be migrated, and then according to the size of the file to be migrated and The corresponding MD5 is further calculated to calculate the standard MD5 and the MD5 to be verified.

Step S30: Calculate the sub-to-be-verified digest hash value corresponding to each sub-file to be verified in the file to be verified, and calculate based on the hash value of each sub-to-be-verified digest and the size of each sub-file to be verified The total hash value of the digest to be verified corresponding to the file to be verified;

In this embodiment, the file to be verified migrated to the cloud is calculated with reference to the calculation step of step S20, and the sub-to-be-verified digest hash value corresponding to each sub-file to be verified in the file to be verified is calculated. And save the MD5 to be verified corresponding to each sub-file to be verified.

Step S40, the document to be verified is collated according to the total standard digest hash value and the total digest hash value to be verified, if the total standard digest hash value and the total digest hash value to be verified If the hash value is the same, it is determined that the file to be verified is successfully migrated, and a corresponding file migration successful reminder message is generated.

In this embodiment, the total standard MD5 of the source and the cloud is directly compared, that is, the total standard digest hash value and the total MD5 to be verified, that is, the total digest hash value to be verified. If the two values are exactly the same , It proves that all migrated correctly. Otherwise, the sub-file verification is further performed.

This embodiment provides a proofreading method for migrated data. When a data migration instruction is detected, the file to be migrated before migration and the file to be verified after migration in the cloud are obtained from the source end; and the file to be migrated is calculated Each sub-standard digest hash value corresponding to each sub-file to be migrated is calculated, and the total standard digest hash value corresponding to the file to be migrated is calculated based on the digest hash value of each sub-standard and the size of each sub-file to be migrated; The hash value of the sub-to-be-verified summary corresponding to each sub-file to be verified in the verification file is calculated, and the corresponding to the file to be verified is calculated based on the hash value of each sub-to-be-verified summary and the size of each sub-file to be verified The total digest hash value to be verified; the document to be verified is collated according to the total standard digest hash value and the total digest hash value to be verified, if the total standard digest hash value is equal to If the total hash value of the digest to be verified is the same, it is determined that the file to be verified is successfully migrated, and a corresponding file migration success reminder message is generated. In the above manner, this application uses the hash value of each sub-standard digest of each sub-file to be migrated in the file to be migrated in the source and the value of each sub-file to be verified in the file to be verified after migration in the cloud. The standard MD5 value of each sub-file to be verified is respectively spliced and calculated to generate the total standard digest hash value corresponding to the file to be migrated and the total digest hash value to be verified corresponding to the file to be verified, and through direct comparison The total standard summary hash value and the total pending summary hash value are used to determine whether the file migration is abnormal, reduce the number of file comparisons, improve the efficiency of file migration, reduce migration costs, and solve the inefficiency of existing data proofreading. The technical problem of the high cost of data proofreading.

Referring to FIG. 3, FIG. 3 is a schematic flowchart of a second embodiment of a method for proofreading migrated data in this application.

Based on the embodiment shown in FIG. 2 above, in this embodiment, the step S20 specifically includes:

Step S21: Calculate the sub-standard digest hash value corresponding to each sub-file to be migrated in the file to be migrated, and set the size of the sub-file to be migrated corresponding to the sub-file to be migrated and the hash value of the sub-standard digest as The sub-standard meta information corresponding to the sub-file to be migrated;

Step S22, according to the arrangement order of the sub-files, the preset number of sub-standard meta-information corresponding to the sub-files to be migrated are sequentially spliced to generate standard batch meta-information;

Step S23: Calculate the batch standard digest hash value corresponding to the standard batch meta information, and calculate the total standard digest hash value corresponding to the file to be migrated based on the batch standard digest hash value.

Wherein, the step S23 specifically includes:

Calculate the batch standard digest hash value corresponding to the standard batch meta-information, and concatenate each batch standard digest hash value for a preset number of times to calculate a file standard MD5 as the corresponding file to be migrated The total standard digest hash value.

In this embodiment, the processing process of the MD5 algorithm is: processing the file to be migrated, setting the initial value, cyclic processing, and splicing the result. The MD5 algorithm is specifically described as follows:

Step 1: Process the files to be migrated;

First, we calculate the remainder of the length (bit) of the file to be migrated to 512. If it is not equal to 448, the file to be migrated needs to be filled so that the length of the file to be migrated is equal to 448 for the remainder of 512. The filling method is to fill the first bit with 1 and the remaining bits with 0. After filling, the length of the file to be migrated is 512*N+448.

Then, use the remaining position (512-448=64 bits) to record the true length of the file to be migrated, and add the binary value of the length to the end. The length of the file to be migrated after such processing is 512*(N+1).

Step 2: Set the initial value;

The length of the MD5 hash result is 128 bits, divided into one group for each 32 bits, a total of 4 groups. These 4 sets of results are obtained through continuous evolution of 4 initial values A, B, C, and D. In the official implementation of MD5, the initial values of A, B, C, and D are as follows (hexadecimal):

A=0x01234567, B=0x89ABCDEF, C=0xFEDCBA98, D=0x76543210

The third step: cycle processing;

A, B, C, and D are the four groups of hash values. Each cycle will cause the old ABCD to produce a new ABCD. How many cycles do you go through? Determined by the length of the file to be migrated after processing.

Assuming that the length of the file to be migrated after processing is M, the number of main loops=M/512, and each main loop contains 512/32*4=64 sub-loops.

The fourth step: splicing results;

The four values A, B, C and D finally produced by the cycle processing are spliced together and converted into a string.

The MD5 calculated for each file is used as the meta-information of the file, and the preset number of meta-information is spliced one by one in the file sort order according to the preset splicing unit. For example, the meta information of every 1000 files is spliced into a total file (according to the specific number of files to be migrated, it can be adjusted appropriately. The minimum is 10, the maximum is 1000), the total file, that is, the accumulated meta information is added, and the total is further calculated Standard MD5, that is, the total standard digest hash value and the total MD5 to be verified, that is, the total digest hash value to be verified, namely MD5_1000. It is described in detail as follows.

The first file file1, MD5 value is MD5_file1;

The second file file2, MD5 value is MD5_file2;

The third file file3, MD5 value is MD5_file3;

Wait until 1000 files.

The splicing meta information of MD5_1000 is: MD5_file1MD5_file2MD5_file3...MD5_file1000. Then, the MD5_1000 of the 1000 files are gradually spliced and accumulated, and the MD5_1000 is calculated every 1000 MD5_1000 to generate MD5_1000_1000. After the above repeated steps, the source and the cloud will finally generate an MD5 value calculated after a finite number of splicing and accumulation. It is assumed to be MD5_1000_1000_1000 (corresponding to 1 million file migration scenarios), that is, the general standard MD5, that is, the general standard digest hash value and MD5 always to be verified, that is, the digest hash value of the total to be verified.

Further, when the data migration of all files to be migrated is completed, the data reconciliation operation is started. At this time, there are only two results: all the data is migrated correctly (high probability), and there is an error in the data migration process.

Further, the step S30 specifically includes:

Calculate the hash value of the sub-file to be verified corresponding to each sub-file to be verified in the file to be verified, and hash the size of the sub-file to be verified corresponding to the sub-file to be verified and the digest of the sub-file to be verified. The column value is set as the sub-to-be-verified meta information corresponding to the sub-file to be verified;

According to the arrangement order of the sub-files, sequentially splicing the sub-to-be-verified meta information corresponding to the preset number of the sub-files to be verified to generate batches of sub-to-be-verified meta information;

Calculate the batch-to-be-verified MD5 corresponding to the batch-to-be-verified meta-information, that is, the batch-to-be-verified digest hash value, based on the batch-to-be-verified MD5, that is, the batch-to-be-verified digest hash The MD5 of each batch to be verified (that is, the digest hash value of the batch to be verified) is sequentially spliced for a preset number of times to calculate the MD5 of a file to be verified, that is, the digest hash value of the file to be verified, as the all The total to-be-verified MD5 corresponding to the file to be verified is the hash value of the total to-be-verified digest.

In this embodiment, the total to-be-verified digest hash value corresponding to the file to be verified is calculated through the above steps, that is, the total-to-be-verified MD5.

Referring to FIG. 4, FIG. 4 is a schematic flowchart of a third embodiment of a method for proofreading migrated data in this application.

Based on the embodiment shown in FIG. 3, in this embodiment, the step S40 specifically includes:

Step S41, judging whether the total standard digest hash value is the same as the total digest hash value to be verified, so as to perform file verification on the files to be verified after migration in the cloud;

Step S42: If the total standard digest hash value is different from the total digest hash value to be verified, compare the batch standard digest hash value with the corresponding batch digest hash value to be verified Yes, to verify the sub-files in batches with migration exceptions;

Step S43: If the total standard digest hash value is the same as the total digest hash value to be verified, it is determined that the file to be verified is successfully migrated, and a corresponding file migration success reminder message is generated.

In this embodiment, by directly comparing the file to be migrated at the source end and the file to be verified MD5_1000_1000_1000 in the cloud, if the two values are exactly the same, it is proved that all are migrated correctly. At this time, there was only one request for reconciliation. If the two values are not the same, you need to quickly find the error file. The process of finding the wrong file needs to be reversed. That is, after comparing the differences in MD5_1000_1000_1000, go to the next level and compare 1000 MD5_1000_1000 to find out the different values between the source and the cloud. In order to improve efficiency, these 1000 MD5_1000_1000 can be reconciled in batches and sent to the reconciliation program for comparison. In this cycle, you can quickly find files with different MD5 values, that is, files that have been migrated incorrectly during the migration process. And quickly retransmit.

As a result, under the premise of ensuring the correct rate, quick reconciliation can be achieved; at the same time, if there is a file migration error, it can also be quickly found and completed; the network cost is very high, so network transmission can be greatly reduced. reduce costs.

For example, when the number of files to be migrated is 1 million.

Traditionally, the MD5 of the source and cloud files needs to be compared 1 million times.

Use this efficient approach, divided into several situations:

1. If the migration is completely correct, a comparison is required;

2. If there is a file error in the migration process, you need to compare MD5_1000+N times 1000 times to find the wrong file. (N＜=1000)

3. In the same way, if two files fail during the migration process, you need to compare 1000+2N times to find the wrong file. (N＜=1000)

4. In the same way, if there are more file failures in the migration process, the number of comparisons will be more. About n files fail, then compare 1000+nN times.

Generally, the probability of file migration errors is small, and the first three are more likely to appear. That is, the efficiency is increased by nearly 1000 times. Moreover, most of the comparisons are on the migration server, which only needs to go through a limited number of network transmissions. .

In addition, the embodiment of the present application also provides a proofreading device for migrated data.

Referring to FIG. 5, FIG. 5 is a schematic diagram of the functional modules of the first embodiment of the proofreading device for migrating data in this application.

In this embodiment, the proofreading device for migration data includes:

The migration file determination module 10 is used to obtain the files to be migrated before the migration and the files to be verified after the migration in the cloud when the data migration instruction is detected;

The first MD5 calculation module 20 is configured to calculate the sub-standard digest hash value corresponding to each sub-file to be migrated in the file to be migrated, and calculate the hash value of each sub-standard digest hash value and the size of each sub-file to be migrated The total standard digest hash value corresponding to the file to be migrated;

The second MD5 calculation module 30 is configured to calculate the sub-to-be-verified digest hash value corresponding to each sub-file to be verified in the file to be verified, and based on the hash value of each sub-to-be-verified digest and each sub-file to be verified Verify the size of the sub-file, and calculate the total hash value of the digest to be verified corresponding to the file to be verified;

The migration data proofreading module 40 is configured to proofread the document to be verified according to the total standard digest hash value and the total digest hash value to be verified. If the total standard digest hash value is equal to the If the total hash value of the digest to be verified is the same, it is determined that the file to be verified is successfully migrated, and a corresponding file migration success reminder message is generated.

Further, the proofreading device for the migrated data further includes:

The file order module is used to arrange the sub-files to be migrated in the source terminal in order according to preset arranging rules, and upload the arranged sub-files to be migrated to the cloud in order.

Further, the first MD5 calculation module 20 specifically includes:

The first standard calculation unit is used to calculate the hash value of the sub-standard summary corresponding to each sub-file to be migrated in the file to be migrated, and to compare the size of the sub-file to be migrated corresponding to the sub-file to be migrated and the sub-standard summary The hash value is set as the sub-standard meta information corresponding to the sub-file to be migrated;

The second standard calculation unit is configured to sequentially splice the predetermined number of substandard meta information corresponding to the sub files to be migrated according to the arrangement order of the sub files to generate standard batch meta information;

The third standard calculation unit is used to calculate the batch standard digest hash value corresponding to the batch meta information of the standard, and calculate the total standard digest corresponding to the file to be migrated based on the batch standard digest hash value Hash value.

Further, the third standard calculation unit is also used for:

Further, the second MD5 calculation module 30 specifically includes:

The first to-be-verified calculation unit is used to calculate the sub-to-be-verified digest hash value corresponding to each of the to-be-verified sub-files in the to-be-verified file, and to compare the to-be-verified sub-file corresponding to the syndrome to be verified The size and the hash value of the sub-to-be-verified digest are set to the sub-to-be-verified meta information corresponding to the sub-file to be verified;

The second to-be-verified calculation unit is configured to sequentially splice the sub-to-be-verified meta information corresponding to the preset number of the sub-files to be verified according to the arrangement order of the sub-files to generate batches of sub-to-be Check meta information;

The third to-be-verified calculation unit is used to calculate the batch-to-be-verified digest hash value corresponding to the batch of sub-to-be-verified meta-information, and based on the batch-to-be-verified digest hash value, and to compare each sub-to-be-verified digest hash value The batch of digest hash values to be verified are sequentially spliced for a preset number of times, and a digest hash value of a file to be verified is calculated as the total digest hash value to be verified corresponding to the file to be verified.

Further, the migration data proofreading module 40 specifically includes:

The first MD5 proofreading unit is configured to determine whether the total standard digest hash value is the same as the total digest hash value to be verified, so as to perform file proofreading on the files to be verified after migration in the cloud;

The second MD5 proofreading unit is configured to, if the hash value of the total standard digest is different from the hash value of the total digest to be verified, compare the hash value of the batch of standard digests with the corresponding batch of digests to be verified The hash value is compared to verify the batch of sub-files with migration abnormalities;

The migration success reminder unit is configured to, if the total standard digest hash value is the same as the total digest hash value to be verified, determine that the file to be verified is successfully migrated, and generate a corresponding file migration success reminder message.

Among them, each module in the above-mentioned migrating data proofreading device corresponds to each step in the above-mentioned migrating data proofreading method embodiment, and its functions and implementation processes will not be repeated here.

In addition, the embodiments of the present application also provide a computer-readable storage medium. The computer-readable storage medium may be non-volatile or volatile.

The computer-readable storage medium of the present application stores a proofreading program for migrated data, where the proofreading program for migrated data is executed by a processor to implement the steps of the above-mentioned proofreading method for migrated data.

Among them, the method implemented when the proofreading program of migrated data is executed can refer to the various embodiments of the proofreading method for migrated data of this application, which will not be repeated here.

It should be noted that in this article, the terms "include", "include" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or system including a series of elements not only includes those elements, It also includes other elements that are not explicitly listed, or elements inherent to the process, method, article, or system. Without more restrictions, the element defined by the sentence "including a..." does not exclude the existence of other identical elements in the process, method, article, or system that includes the element.

The serial numbers of the foregoing embodiments of the present application are for description only, and do not represent the superiority or inferiority of the embodiments.

Through the description of the above implementation manners, those skilled in the art can clearly understand that the above-mentioned embodiment method can be implemented by means of software plus the necessary general hardware platform, of course, it can also be implemented by hardware, but in many cases the former is better.的实施方式。 Based on this understanding, the technical solution of this application essentially or the part that contributes to the existing technology can be embodied in the form of a software product, and the computer software product is stored in a storage medium (such as ROM/RAM) as described above. , Magnetic disks, optical disks), including several instructions to make a terminal device (which can be a mobile phone, a computer, a server, an air conditioner, or a network device, etc.) execute the method described in each embodiment of the present application.

The above are only the preferred embodiments of the application, and do not limit the scope of the patent for this application. Any equivalent structure or equivalent process transformation made using the content of the description and drawings of the application, or directly or indirectly applied to other related technical fields , The same reason is included in the scope of patent protection of this application.

Claims

A proofreading method for migrated data, wherein the proofreading method for migrated data includes the following steps:

When the data migration instruction is detected, the files to be migrated before migration and the files to be verified after migration in the cloud are obtained from the source;

Calculate the standard sub-digest hash value of each sub-file to be migrated in the file to be migrated, and calculate the standard sub-digest hash value corresponding to the file to be migrated based on the sub-digest hash value of each standard and the size of each sub-file to be migrated Total digest hash value;

Calculate the hash value of the sub-digest to be verified corresponding to each sub-file to be verified in the file to be verified, and calculate the hash value of each sub-digest to be verified and the size of each sub-file to be verified The hash value of the total digest to be verified corresponding to the file to be verified;

The document to be verified is collated according to the total standard digest hash value and the total digest hash value to be verified, if the total standard digest hash value and the total digest hash value to be verified If they are the same, it is determined that the file to be verified is successfully migrated, and a corresponding file migration successful reminder message is generated.
The method for proofreading migrated data according to claim 1, wherein, when the data migration instruction is detected, before the step of acquiring the files to be migrated before migration and the files to be verified after migration in the cloud in the source end, Also includes:

According to a preset arrangement rule, the sub-files to be migrated in the source end are arranged in order, and the arranged sub-files to be migrated are sequentially uploaded to the cloud.
The method for proofreading migration data according to claim 2, wherein the calculation of the sub-standard digest hash value corresponding to each sub-file to be migrated in the file to be migrated is based on the hash value of each sub-standard digest and each sub-file to be migrated. The steps of migrating the size of the sub-file and calculating the total standard digest hash value corresponding to the file to be migrated specifically include:

Calculate the sub-standard summary hash value corresponding to each sub-file to be migrated in the file to be migrated, and set the size of the sub-file to be migrated corresponding to the sub-file to be migrated and the hash value of the sub-standard summary as the Sub-standard meta-information corresponding to the migration sub-file;

According to the arrangement sequence of the sub-files to be migrated, the preset number of sub-standard meta-information corresponding to the sub-files to be migrated are sequentially spliced to generate standard batch meta-information;

Calculate the batch standard digest hash value corresponding to the batch meta-information of the standard, and calculate the standard total digest hash value of the standard corresponding to the file to be migrated based on the standard batch digest hash value.
The verification method of migration data according to claim 3, wherein the hash value of the sub-to-be-verified digest corresponding to each sub-file to be verified in the to-be-verified file is calculated, and the hash value of the sub-to-be-verified digest is hashed based on each sub-file to be verified. The column value and the size of each sub-file to be verified, and the step of calculating the total digest hash value to be verified corresponding to the file to be verified specifically includes:

Calculate the hash value of the sub-file to be verified corresponding to each sub-file to be verified in the file to be verified, and hash the size of the sub-file to be verified corresponding to the sub-file to be verified and the digest of the sub-file to be verified. The column value is set as the sub-to-be-verified meta information corresponding to the sub-file to be verified;

According to the arrangement order of the sub-files, sequentially splicing the sub-to-be-verified meta information corresponding to the preset number of the sub-files to be verified to generate batches of sub-to-be-verified meta information;

Calculate the hash values of the digests to be verified in batches corresponding to the meta-information of the sub-to-be-verified batches, based on the hash values of the digests to be verified in the batches, and sequentially concatenate the hash values of the digests to be verified in each batch For a preset number of times, the digest hash value of a file to be verified is calculated as the total digest hash value to be verified corresponding to the file to be verified.
The method for proofreading migration data according to claim 4, wherein the document to be verified is proofread according to the total standard digest hash value and the total digest hash value to be verified, if the If the total standard digest hash value is the same as the total digest hash value to be verified, it is determined that the file to be verified is successfully migrated, and the steps of generating a corresponding file migration success reminder message specifically include:

Judging whether the total standard digest hash value is the same as the total digest hash value to be verified, so as to perform file proofreading on the files to be verified after migration in the cloud;

If the total standard digest hash value is different from the total digest hash value to be verified, the batch standard digest hash value is compared with the corresponding batch digest hash value to be verified to Verify the sub-files in batches with migration exceptions;

If the total standard digest hash value is the same as the total digest hash value to be verified, it is determined that the file to be verified is successfully migrated, and a corresponding file migration success reminder message is generated.
The method for proofreading migrated data according to any one of claims 1 to 5, wherein, according to a preset arrangement rule, the sub-files to be migrated in the source end are arranged in order, and each of the arranged sub-files The steps of sequentially uploading the sub-files to be migrated to the cloud specifically include:

According to the lexicographic order of the file names, the sub-files to be migrated in the source end are sequentially arranged, and the arranged sub-files to be migrated are sequentially uploaded to the cloud.
A proofreading device for migrated data, wherein the proofreading device for migrated data includes:

The migration file determination module is used to obtain the files to be migrated before migration and the files to be verified after migration in the cloud when the data migration instruction is detected;

The first MD5 calculation module is used to calculate the sub-standard digest hash value corresponding to each sub-file to be migrated in the file to be migrated, and calculate the to-be-migrated sub-file based on the hash value of each sub-standard digest and the size of each sub-file to be migrated. The total standard digest hash value corresponding to the migration file;

The second MD5 calculation module is used to calculate the sub-to-be-verified digest hash value corresponding to each sub-file to be verified in the file to be verified, and based on the hash value of each sub-to-be-verified digest and each to-be-verified Sub-file size, calculating the total hash value of the digest to be verified corresponding to the file to be verified;

The migration data proofreading module is used to proofread the file to be verified according to the total standard digest hash value and the total digest hash value to be verified. If the total standard digest hash value is compared with the total If the hash value of the digest to be verified is the same, it is determined that the file to be verified is successfully migrated, and a corresponding file migration success reminder message is generated.
7. The proofreading device for migration data according to claim 7, wherein the proofreading device for migration data further comprises a migration file splicing module, and the migration file splicing module is used for:

According to a preset arrangement rule, arrange the sub-files to be migrated in the source end in order, and upload the arranged sub-files to be migrated to the cloud in order;

Calculate the sub-standard summary hash value corresponding to each sub-file to be migrated in the file to be migrated, and set the size of the sub-file to be migrated corresponding to the sub-file to be migrated and the hash value of the sub-standard summary as the Sub-standard meta-information corresponding to the migration sub-file;

According to the arrangement order of the sub-files to be migrated, the preset number of sub-standard meta-information corresponding to the sub-files to be migrated are sequentially spliced to generate standard batch meta-information;

Calculate the batch standard digest hash value corresponding to the standard batch meta information, and calculate the total standard digest hash value corresponding to the file to be migrated based on the batch standard digest hash value.
A proofreading device for migrating data, wherein the proofreading device for migrating data includes a processor, a memory, and a proofreading program for migrating data stored on the memory and executable by the processor, wherein the migrating data When the proofreading program of is executed by the processor, the following steps of the proofreading method of migrated data are realized:

When the data migration instruction is detected, the files to be migrated before migration and the files to be verified after migration in the cloud are obtained from the source;

Calculate the standard sub-digest hash value corresponding to each sub-file to be migrated in the file to be migrated, and calculate the standard sub-digest hash value corresponding to the file to be migrated based on the sub-digest hash value of each standard and the size of each sub-file to be migrated Total digest hash value;

Calculate the hash value of the sub-digest to be verified corresponding to each sub-file to be verified in the file to be verified, and calculate the hash value of each sub-digest to be verified and the size of each sub-file to be verified The hash value of the total digest to be verified corresponding to the file to be verified;

The document to be verified is collated according to the total standard digest hash value and the total digest hash value to be verified, if the total standard digest hash value and the total digest hash value to be verified If they are the same, it is determined that the file to be verified is successfully migrated, and a corresponding file migration successful reminder message is generated.
9. The proofreading device for migrating data according to claim 9, wherein when the data migration instruction is detected, before the step of acquiring the files to be migrated before migration and the files to be verified after migration in the cloud in the source end, Also includes:

According to a preset arrangement rule, the sub-files to be migrated in the source end are arranged in order, and the arranged sub-files to be migrated are sequentially uploaded to the cloud.
The proofreading device for migration data according to claim 10, wherein the calculation of the sub-standard digest hash value corresponding to each sub-file to be migrated in the file to be migrated is based on the hash value of each sub-standard digest and each sub-file to be migrated. The steps of migrating the size of the sub-file and calculating the total standard digest hash value corresponding to the file to be migrated specifically include:

Calculate the sub-standard summary hash value corresponding to each sub-file to be migrated in the file to be migrated, and set the size of the sub-file to be migrated corresponding to the sub-file to be migrated and the hash value of the sub-standard summary as the Sub-standard meta-information corresponding to the migration sub-file;

According to the arrangement order of the sub-files to be migrated, the preset number of sub-standard meta-information corresponding to the sub-files to be migrated are sequentially spliced to generate standard batch meta-information;

Calculate the batch standard digest hash value corresponding to the batch meta-information of the standard, and calculate the standard total digest hash value of the standard corresponding to the file to be migrated based on the standard batch digest hash value.
The verification device for migration data according to claim 11, wherein the hash value of the sub-to-be-verified digest corresponding to each sub-file to be verified in the file to be verified is calculated, and the hash value is calculated based on each sub-to-be-verified digest. The column value and the size of each sub-file to be verified, and the step of calculating the total digest hash value to be verified corresponding to the file to be verified specifically includes:

Calculate the hash value of the sub-file to be verified corresponding to each sub-file to be verified in the file to be verified, and hash the size of the sub-file to be verified corresponding to the sub-file to be verified and the digest of the sub-file to be verified. The column value is set as the sub-to-be-verified meta information corresponding to the sub-file to be verified;

According to the arrangement order of the sub-files, sequentially splicing the sub-to-be-verified meta information corresponding to the preset number of the sub-files to be verified to generate batches of sub-to-be-verified meta information;

Calculate the hash values of the digests to be verified in batches corresponding to the meta-information of the sub-to-be-verified batches, based on the hash values of the digests to be verified in the batches, and sequentially concatenate the hash values of the digests to be verified in each batch For a preset number of times, the digest hash value of a file to be verified is calculated as the total digest hash value to be verified corresponding to the file to be verified.
The proofreading device for migration data according to claim 12, wherein the document to be verified is proofread according to the total standard digest hash value and the total digest hash value to be verified, if the If the total standard digest hash value is the same as the total digest hash value to be verified, it is determined that the file to be verified is successfully migrated, and the steps of generating a corresponding file migration success reminder message specifically include:

Judging whether the total standard digest hash value is the same as the total digest hash value to be verified, so as to perform file proofreading on the files to be verified after migration in the cloud;

If the total standard digest hash value is different from the total digest hash value to be verified, the batch standard digest hash value is compared with the corresponding batch digest hash value to be verified to Verify the sub-files in batches with migration exceptions;

If the total standard digest hash value is the same as the total digest hash value to be verified, it is determined that the file to be verified is successfully migrated, and a corresponding file migration success reminder message is generated.
The proofreading device for migrated data according to any one of claims 9 to 13, wherein the sub-files to be migrated in the source end are sequentially arranged according to a preset arrangement rule, and each of the arranged sub-files The steps of sequentially uploading the sub-files to be migrated to the cloud specifically include:

According to the lexicographic order of the file names, the sub-files to be migrated in the source end are sequentially arranged, and the arranged sub-files to be migrated are sequentially uploaded to the cloud.
A computer-readable storage medium, wherein a proofreading program for migration data is stored on the computer-readable storage medium, and when the proofreading program for migration data is executed by a processor, the following steps of the proofreading method for migration data are implemented:

When the data migration instruction is detected, the files to be migrated before migration and the files to be verified after migration in the cloud are obtained from the source;

Calculate the standard sub-digest hash value corresponding to each sub-file to be migrated in the file to be migrated, and calculate the standard sub-digest hash value corresponding to the file to be migrated based on the sub-digest hash value of each standard and the size of each sub-file to be migrated Total digest hash value;

Calculate the hash value of the sub-digest to be verified corresponding to each sub-file to be verified in the file to be verified, and calculate the hash value of each sub-digest to be verified and the size of each sub-file to be verified The hash value of the total digest to be verified corresponding to the file to be verified;

The document to be verified is collated according to the total standard digest hash value and the total digest hash value to be verified, if the total standard digest hash value and the total digest hash value to be verified If they are the same, it is determined that the file to be verified is successfully migrated, and a corresponding file migration successful reminder message is generated.
15. The computer-readable storage medium according to claim 15, wherein, when the data migration instruction is detected, before the step of obtaining the file to be migrated before migration and the file to be verified after migration in the cloud in the source end, Also includes:

According to a preset arrangement rule, the sub-files to be migrated in the source end are arranged in order, and the arranged sub-files to be migrated are sequentially uploaded to the cloud.
The computer-readable storage medium according to claim 16, wherein the calculation of the sub-standard digest hash value corresponding to each sub-file to be migrated in the file to be migrated is based on the hash value of each sub-standard digest and each sub-file to be migrated. The steps of migrating the size of the sub-file and calculating the total standard digest hash value corresponding to the file to be migrated specifically include:

Calculate the sub-standard summary hash value corresponding to each sub-file to be migrated in the file to be migrated, and set the size of the sub-file to be migrated corresponding to the sub-file to be migrated and the hash value of the sub-standard summary as the Sub-standard meta-information corresponding to the migration sub-file;

According to the arrangement order of the sub-files to be migrated, the preset number of sub-standard meta-information corresponding to the sub-files to be migrated are sequentially spliced to generate standard batch meta-information;

Calculate the batch standard digest hash value corresponding to the batch meta-information of the standard, and calculate the standard total digest hash value of the standard corresponding to the file to be migrated based on the standard batch digest hash value.
The computer-readable storage medium according to claim 17, wherein the hash value of the sub-to-be-verified digest corresponding to each sub-file to be verified in the file to be verified is calculated, and the hash value of the sub-to-be-verified digest is hashed based on each sub-file to be verified. The column value and the size of each sub-file to be verified, and the step of calculating the total digest hash value to be verified corresponding to the file to be verified specifically includes:

Calculate the hash value of the sub-file to be verified corresponding to each sub-file to be verified in the file to be verified, and hash the size of the sub-file to be verified corresponding to the sub-file to be verified and the digest of the sub-file to be verified. The column value is set as the sub-to-be-verified meta information corresponding to the sub-file to be verified;

According to the arrangement order of the sub-files, sequentially splicing the sub-to-be-verified meta information corresponding to the preset number of the sub-files to be verified to generate batches of sub-to-be-verified meta information;

Calculate the hash values of the digests to be verified in batches corresponding to the meta-information of the sub-to-be-verified batches, based on the hash values of the digests to be verified in the batches, and sequentially concatenate the hash values of the digests to be verified in each batch For a preset number of times, the digest hash value of a file to be verified is calculated as the total digest hash value to be verified corresponding to the file to be verified.
18. The computer-readable storage medium of claim 18, wherein the document to be verified is collated according to the total standard digest hash value and the total digest hash value to be verified, if the If the total standard digest hash value is the same as the total digest hash value to be verified, it is determined that the file to be verified is successfully migrated, and the steps of generating a corresponding file migration success reminder message specifically include:

Judging whether the total standard digest hash value is the same as the total digest hash value to be verified, so as to perform file proofreading on the files to be verified after migration in the cloud;

If the total standard digest hash value is different from the total digest hash value to be verified, the batch standard digest hash value is compared with the corresponding batch digest hash value to be verified to Verify the sub-files in batches with migration exceptions;

If the total standard digest hash value is the same as the total digest hash value to be verified, it is determined that the file to be verified is successfully migrated, and a corresponding file migration success reminder message is generated.
The computer-readable storage medium according to any one of claims 15 to 19, wherein:

The step of sequentially arranging the sub-files to be migrated in the source end according to a preset arrangement rule, and uploading the arranged sub-files to be migrated to the cloud in order specifically includes:

According to the lexicographic order of the file names, the sub-files to be migrated in the source end are sequentially arranged, and the arranged sub-files to be migrated are sequentially uploaded to the cloud.