CN111125063A - Method and device for rapidly verifying data migration among clusters - Google Patents
Method and device for rapidly verifying data migration among clusters Download PDFInfo
- Publication number
- CN111125063A CN111125063A CN201911327676.0A CN201911327676A CN111125063A CN 111125063 A CN111125063 A CN 111125063A CN 201911327676 A CN201911327676 A CN 201911327676A CN 111125063 A CN111125063 A CN 111125063A
- Authority
- CN
- China
- Prior art keywords
- data
- clusters
- computers
- migration
- check
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/21—Design, administration or maintenance of databases
- G06F16/214—Database migration support
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/22—Indexing; Data structures therefor; Storage structures
- G06F16/2282—Tablespace storage structures; Management thereof
Landscapes
- Engineering & Computer Science (AREA)
- Databases & Information Systems (AREA)
- Theoretical Computer Science (AREA)
- Data Mining & Analysis (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Software Systems (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The invention discloses a method and a device for rapidly verifying data migration among clusters. The method for rapidly verifying data migration among clusters comprises the following steps: acquiring a proofreading data table of a related cluster of data migration; selectively storing the data in the proofreading data table of the related cluster of the data migration in a proofreading data model; transmitting the data content in the verification type data model to a plurality of computers; writing the results fed back by the plurality of computers into a result library; and displaying the verification result in the result library through a web page or a corresponding display file. The invention can avoid cutting the data table, greatly accelerates the checking process of the accuracy of the migration data, and can directly check certain specific data columns in the data table according to the requirements of users.
Description
Technical Field
The invention relates to the technical field of computer clusters, in particular to a method and a device for quickly verifying data migration between clusters.
Background
During use of computer clusters, data migration between computer clusters is often required. The amount of data migrated is often massive, which requires checking after migration. The sender's computer cluster and the recipient's computer cluster each have a data table associated with the data migration, and the check is also made by checking whether the contents of the data tables are consistent. However, the mass data migration also brings mass data amount in the data table, and the data amount of many GB is often not directly verifiable. The data table needs to be cut, but how to select an appropriate cutting point is also a problem in the industry. There is no direct way to customize certain columns of data in a data table. It is also inefficient to use a single device for single threaded verification of the data sheet. How to properly solve the above problems is an urgent issue to be solved in the industry.
Disclosure of Invention
The invention provides a method and a device for rapidly verifying data migration among clusters, which are used for avoiding cutting a data table, greatly accelerating the verification process of the accuracy of migrated data, and directly verifying certain specific data columns in the data table according to the requirements of users.
According to a first aspect of the embodiments of the present invention, a method for quickly verifying data migration between clusters is provided, including:
acquiring a proofreading data table of a related cluster of data migration;
selectively storing the data in the proofreading data table of the related cluster of the data migration in a proofreading data model;
transmitting the data content in the verification type data model to a plurality of computers;
writing the results fed back by the plurality of computers into a result library;
and displaying the verification result in the result library through a web page or a corresponding display file.
In one embodiment, the obtaining the collation data table of the related cluster of data migration includes:
analyzing the number of sender clusters and the number of receiver clusters related to data migration, wherein the number of the sender clusters and the number of the receiver clusters can be multiple;
and acquiring a proofreading data table of each sender cluster and each receiver cluster of the data migration.
In one embodiment, the data in the check data table of the related cluster to which the data is migrated is selectively stored in a check-type data model, and the method includes:
according to a preset filter, performing filtering reading on the proofreading data table;
and storing the data read in a filtering mode in a preset check type data model.
In one embodiment, the sending the data content in the verification-type data model to a plurality of computers includes:
converting the check data model after data storage into a plurality of distributed memory type data sets;
and sending the plurality of distributed memory type data sets to a plurality of computers, and carrying out verification operation by the computer at the receiving end.
In one embodiment, said writing the results of said plurality of computer feedbacks to a results library comprises:
collecting results of the check operations of the plurality of computers;
and splicing the results of the check operation of the plurality of computers and writing the results into a result library.
According to a second aspect of the embodiments of the present invention, an apparatus for fast checking data migration between clusters is provided, including:
the acquisition module is used for acquiring a proofreading data table of a related cluster of data migration;
the storage module is used for selectively storing the data in the check data table of the related cluster of the data migration in a check data model;
the sending module is used for sending the data content in the verification type data model to a plurality of computers;
the writing module is used for writing the results fed back by the plurality of computers into a result library;
and the display module is used for displaying the verification result in the result library through a web page or a corresponding display file.
In one embodiment, the obtaining module includes:
the analysis submodule is used for analyzing the number of the sender clusters and the number of the receiver clusters related to data migration, wherein the number of the sender clusters and the number of the receiver clusters can be multiple;
and the obtaining submodule is used for obtaining the proofreading data tables of each sender cluster and each receiver cluster of the data migration.
In one embodiment, the storage module includes:
the filtering submodule is used for filtering and reading the proofreading data table according to a preset filter;
and the storage submodule is used for storing the data read in a filtering mode in a preset check type data model.
In one embodiment, the sending module includes:
the conversion submodule is used for converting the check data model stored with the data into a plurality of distributed memory type data sets;
and the sending submodule is used for sending the plurality of distributed memory type data sets to a plurality of computers and attaching an instruction for enabling the computer at the receiving end to carry out checking operation.
In one embodiment, the write module includes:
a collection submodule for collecting results of the check operations of the plurality of computers;
and the splicing submodule is used for splicing the results of the check operation of the plurality of computers and writing the results into a result library.
Additional features and advantages of the invention will be set forth in the description which follows, and in part will be obvious from the description, or may be learned by practice of the invention. The objectives and other advantages of the invention will be realized and attained by the structure particularly pointed out in the written description and claims hereof as well as the appended drawings.
The technical solution of the present invention is further described in detail by the accompanying drawings and embodiments.
Drawings
The accompanying drawings, which are included to provide a further understanding of the invention and are incorporated in and constitute a part of this specification, illustrate embodiments of the invention and together with the description serve to explain the principles of the invention and not to limit the invention. In the drawings:
FIG. 1 is a flow chart illustrating a method for fast check of data migration between clusters in accordance with an exemplary embodiment of the present invention;
FIG. 2 is a flowchart illustrating step S11 of a method for fast checking data migration between clusters according to an exemplary embodiment of the present invention;
FIG. 3 is a flowchart illustrating step S12 of a method for fast checking data migration between clusters according to an exemplary embodiment of the present invention;
FIG. 4 is a flowchart illustrating step S13 of a method for fast checking data migration between clusters according to an exemplary embodiment of the present invention;
FIG. 5 is a flowchart illustrating step S14 of a method for fast checking data migration between clusters according to an exemplary embodiment of the present invention;
FIG. 6 is a block diagram illustrating an apparatus for fast checking data migration between clusters in accordance with an exemplary embodiment of the present invention;
FIG. 7 is a block diagram of an acquisition module 61 of an apparatus for fast checking data migration between clusters according to an exemplary embodiment of the present invention;
FIG. 8 is a block diagram illustrating a storage module 62 of an apparatus for fast checking data migration between clusters, according to an exemplary embodiment of the present invention;
FIG. 9 is a block diagram illustrating a sending module 63 of an apparatus for fast checking data migration between clusters according to an exemplary embodiment of the present invention;
fig. 10 is a block diagram illustrating a write module 64 of an apparatus for fast checking data migration between clusters according to an exemplary embodiment of the invention.
Detailed Description
The preferred embodiments of the present invention will be described in conjunction with the accompanying drawings, and it will be understood that they are described herein for the purpose of illustration and explanation and not limitation.
Fig. 1 is a flowchart illustrating a method for fast checking data migration between clusters according to an exemplary embodiment, where the method for fast checking data migration between clusters, as shown in fig. 1, includes the following steps S11-S14:
in step S11, a collation data table of the relevant cluster for data migration is acquired;
in step S12, selectively storing the data in the check data table of the related cluster of the data migration in a check data model;
in step S13, transmitting the data content in the verification-type data model to a plurality of computers;
in step S14, writing the results of the plurality of computer feedbacks into a result library;
and displaying the verification result in the result library through a web page or a corresponding display file.
In one embodiment, data migration between computer clusters is often required during use of the computer clusters. The amount of data migrated is often massive, which requires checking after migration. The sender's computer cluster and the recipient's computer cluster each have a data table associated with the data migration, and the check is also made by checking whether the contents of the data tables are consistent. However, the mass data migration also brings mass data amount in the data table, and the data amount of many GB is often not directly verifiable. The data table needs to be cut, but how to select an appropriate cutting point is also a problem in the industry. There is no direct way to customize certain columns of data in a data table. It is also inefficient to use a single device for single threaded verification of the data sheet. The above problems can be properly solved in this embodiment.
And acquiring a proofreading data table of the related cluster of the data migration. Analyzing the number of sender clusters and the number of receiver clusters related to data migration, wherein the number of the sender clusters and the number of the receiver clusters can be multiple; and acquiring a proofreading data table of each sender cluster and each receiver cluster of the data migration.
And selectively storing the data in the check data table of the related cluster of the data migration in a check type data model. According to a preset filter, filtering and reading the proofreading data table; and storing the data read in a filtering mode in a preset check type data model.
And transmitting the data content in the verification type data model to a plurality of computers. Converting the check data model after data storage into a plurality of distributed memory type data sets; and sending the plurality of distributed memory type data sets to a plurality of computers, and carrying out verification operation by the computer at the receiving end.
And writing the results fed back by the plurality of computers into a result library. Wherein the results of the check operations of the plurality of computers are collected; and splicing the results of the check operation of the plurality of computers and writing the results into a result library.
And displaying the verification result in the result library through a web page or a corresponding display file.
The technical scheme in the embodiment can avoid cutting the data table, greatly accelerates the checking process of the accuracy of the migration data, and can directly check certain specific data columns in the data table according to the user requirements.
In one embodiment, as shown in FIG. 2, step S11 includes the following steps S21-S22:
analyzing the number of sender clusters and receiver clusters related to data migration in step S21, where the number of the sender clusters and the number of the receiver clusters may be multiple;
in step S22, a collation data table for each sender cluster and each receiver cluster of the data migration is acquired.
In one embodiment, the number of the sender clusters of the data migration may be n, and the number of the clusters of the receiver of the data migration may also be m, where n and m are positive integers. In each data migration process, a data table is respectively arranged in the receiver cluster and the sender cluster, and any one or more of a data name, a data column, a data format and a data formula of the sending data are recorded in the data table of the receiver cluster and the data table of the sender cluster.
In one embodiment, as shown in FIG. 3, step S12 includes the following steps S31-S32:
in step S31, the collation data table is read in a filtering manner according to a preset filter;
in step S32, the filter-read data is stored in a preset check-type data model.
In one embodiment, the preset filter may set those data columns of the data in the data migration to be filtered out, so that the data table can be simplified. And data columns which are not interesting to the user can be removed according to the use purpose of the user, so that a large amount of computing resources are saved for the subsequent processing process, and the processing process is accelerated. When a preset checking type data model is established, the data model is already set to comprise the data columns, the data format and the data formula corresponding to the data columns, and the like.
In one embodiment, as shown in FIG. 4, step S13 includes the following steps S41-S42:
in step S41, converting the stored check data model into a plurality of distributed memory type data sets;
in step S42, the distributed memory type data sets are sent to a plurality of computers, and a command for the receiving computer to perform a check operation is given.
In one embodiment, the data content in the verification type data model is converted into the distributed memory type data set by the characteristics of fast processing of large-scale data through a Spark calculation engine. In order to use a plurality of computers for cooperative operation, the distributed memory type data set is divided into a plurality of small distributed memory type data sets, so that the subsequent use of a plurality of computers for completing the check operation of the distributed memory type data sets of the respective parts is facilitated.
In one embodiment, as shown in FIG. 5, step S14 includes the following steps S51-S52:
in step S51, collecting results of the check operations of the plurality of computers;
in step S52, the results of the check operations of the plurality of computers are concatenated and written into a result library.
In one embodiment, the feedback results of the plurality of computers are collected after each computer completes the check operation of the respective portion of the distributed memory type data set. And splicing the feedback results according to the sequence of the numbers of the distributed memory type data sets received by each computer, and writing the spliced results into a result library, so that the results can be conveniently displayed in the subsequent process.
In one embodiment, FIG. 6 is a block diagram illustrating an apparatus for fast check of data migration between clusters, according to an example embodiment. As shown in fig. 6, the apparatus includes an obtaining module 61, a storing module 62, a sending module 63, a writing module 64, and a presentation module 65.
The obtaining module 61 is configured to obtain a collation data table of a relevant cluster for data migration;
the storage module 62 is configured to selectively store the data in the check data table of the relevant cluster of the data migration in a check-type data model;
the sending module 63 is configured to send the data content in the verification-type data model to a plurality of computers;
the writing module 64 is configured to write the results fed back by the plurality of computers into a result library;
the display module 65 is configured to display the verification result in the result library through a web page or a corresponding display file.
As shown in fig. 7, the acquisition module 61 includes an analysis submodule 71 and an acquisition submodule 72.
The analysis sub-module 71 is configured to analyze the number of sender clusters and the number of receiver clusters related to data migration, where the number of the sender clusters and the number of the receiver clusters may be multiple;
the obtaining sub-module 72 is configured to obtain a check data table of each sender cluster and each receiver cluster of the data migration.
As shown in fig. 8, the storage module 62 includes a filtering submodule 81 and a storage submodule 82.
The filtering submodule 81 is configured to filter and read the calibration data table according to a preset filter;
the storage sub-module 82 is configured to store the data read in a filtering manner in a preset check type data model.
As shown in fig. 9, the sending module 63 includes a conversion sub-module 91 and a sending sub-module 92.
The transformation module 91 is configured to transform the check data model after storing the data into a plurality of distributed memory type data sets;
the sending submodule 92 is configured to send the plurality of distributed memory type data sets to a plurality of computers, and attach an instruction for causing the computer at the receiving end to perform a check operation.
As shown in fig. 10, the write module 64 includes a gather sub-module 101 and a splice sub-module 102.
The collecting submodule 101 is configured to collect results of the check operations of the plurality of computers;
the splicing submodule 102 is configured to splice the results of the check operations of the multiple computers and write the results into a result library.
As will be appreciated by one skilled in the art, embodiments of the present invention may be provided as a method, system, or computer program product. Accordingly, the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present invention may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, optical storage, and the like) having computer-usable program code embodied therein.
The present invention is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each flow and/or block of the flow diagrams and/or block diagrams, and combinations of flows and/or blocks in the flow diagrams and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
It will be apparent to those skilled in the art that various changes and modifications may be made in the present invention without departing from the spirit and scope of the invention. Thus, if such modifications and variations of the present invention fall within the scope of the claims of the present invention and their equivalents, the present invention is also intended to include such modifications and variations.
Claims (10)
1. A method for rapidly verifying data migration between clusters is characterized by comprising the following steps:
acquiring a proofreading data table of a related cluster of data migration;
selectively storing the data in the proofreading data table of the related cluster of the data migration in a proofreading data model;
transmitting the data content in the verification type data model to a plurality of computers;
writing the results fed back by the plurality of computers into a result library;
and displaying the verification result in the result library through a web page or a corresponding display file.
2. The method of claim 1, wherein the obtaining the collation data table for the dependent cluster of data migrations comprises:
analyzing the number of sender clusters and the number of receiver clusters related to data migration, wherein the number of the sender clusters and the number of the receiver clusters can be multiple;
and acquiring a proofreading data table of each sender cluster and each receiver cluster of the data migration.
3. The method of claim 1, wherein selectively storing data in the collation data table of the dependent cluster from which the data is migrated in a collation data model comprises:
according to a preset filter, performing filtering reading on the proofreading data table;
and storing the data read in a filtering mode in a preset check type data model.
4. The method of claim 1, wherein said transmitting data content in said verification-type data model to a plurality of computers comprises:
converting the check data model after data storage into a plurality of distributed memory type data sets;
and sending the plurality of distributed memory type data sets to a plurality of computers, and carrying out verification operation by the computer at the receiving end.
5. The method of claim 1, wherein writing the results of the plurality of computer feedbacks to a results library comprises:
collecting results of the check operations of the plurality of computers;
and splicing the results of the check operation of the plurality of computers and writing the results into a result library.
6. An apparatus for fast checking data migration between clusters, comprising:
the acquisition module is used for acquiring a proofreading data table of a related cluster of data migration;
the storage module is used for selectively storing the data in the check data table of the related cluster of the data migration in a check data model;
the sending module is used for sending the data content in the verification type data model to a plurality of computers;
the writing module is used for writing the results fed back by the plurality of computers into a result library;
and the display module is used for displaying the verification result in the result library through a web page or a corresponding display file.
7. The apparatus of claim 6, wherein the obtaining module comprises:
the analysis submodule is used for analyzing the number of the sender clusters and the number of the receiver clusters related to data migration, wherein the number of the sender clusters and the number of the receiver clusters can be multiple;
and the obtaining submodule is used for obtaining the proofreading data tables of each sender cluster and each receiver cluster of the data migration.
8. The apparatus of claim 6, wherein the storage module comprises:
the filtering submodule is used for filtering and reading the proofreading data table according to a preset filter;
and the storage submodule is used for storing the data read in a filtering mode in a preset check type data model.
9. The apparatus of claim 6, wherein the sending module comprises:
the conversion submodule is used for converting the check data model stored with the data into a plurality of distributed memory type data sets;
and the sending submodule is used for sending the plurality of distributed memory type data sets to a plurality of computers and attaching an instruction for enabling the computer at the receiving end to carry out checking operation.
10. The apparatus of claim 6, wherein the write module comprises:
a collection submodule for collecting results of the check operations of the plurality of computers;
and the splicing submodule is used for splicing the results of the check operation of the plurality of computers and writing the results into a result library.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911327676.0A CN111125063B (en) | 2019-12-20 | 2019-12-20 | Method and device for rapidly checking data migration among clusters |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911327676.0A CN111125063B (en) | 2019-12-20 | 2019-12-20 | Method and device for rapidly checking data migration among clusters |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN111125063A true CN111125063A (en) | 2020-05-08 |
| CN111125063B CN111125063B (en) | 2023-09-26 |
Family
ID=70500745
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201911327676.0A Active CN111125063B (en) | 2019-12-20 | 2019-12-20 | Method and device for rapidly checking data migration among clusters |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111125063B (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111680004A (en) * | 2020-06-08 | 2020-09-18 | 中国银行股份有限公司 | Method and device for checking migration accuracy of unstructured image file |
Citations (14)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20150142858A1 (en) * | 2013-11-15 | 2015-05-21 | Workday, Inc. | Identifying and formatting data for data migration |
| CN105989044A (en) * | 2015-02-04 | 2016-10-05 | 阿里巴巴集团控股有限公司 | Database verification method and system |
| CN106055670A (en) * | 2016-06-06 | 2016-10-26 | 中国工商银行股份有限公司 | Inter-system data migration method and device |
| US20170097839A1 (en) * | 2015-10-01 | 2017-04-06 | International Business Machines Corporation | Risk-appropriate validation for live operating system migration |
| WO2017071337A1 (en) * | 2015-10-26 | 2017-05-04 | 中兴通讯股份有限公司 | Database table data management method, apparatus and system |
| CN107766572A (en) * | 2017-11-13 | 2018-03-06 | 北京国信宏数科技有限责任公司 | Distributed extraction and visual analysis method and system based on economic field data |
| CN108073688A (en) * | 2017-11-20 | 2018-05-25 | 苏宁云商集团股份有限公司 | A kind of method and device of Data Migration |
| CN108241632A (en) * | 2016-12-23 | 2018-07-03 | 航天星图科技（北京）有限公司 | A kind of data verification method of data base-oriented Data Migration |
| US20180232174A1 (en) * | 2017-02-15 | 2018-08-16 | Beijing Baidu Netcom Science And Technology Co., Ltd. | Data Migration Between Cloud Storage Systems |
| CN108595664A (en) * | 2018-04-28 | 2018-09-28 | 尚谷科技（天津）有限公司 | A kind of agricultural data monitoring method under hadoop environment |
| US20190155801A1 (en) * | 2017-08-16 | 2019-05-23 | Walmart Apollo, Llc | Systems and methods for distributed data validation |
| US20190266134A1 (en) * | 2017-01-11 | 2019-08-29 | Tencent Technology (Shenzhen) Company Limited | Data migration method, apparatus, and storage medium |
| CN110209521A (en) * | 2019-02-22 | 2019-09-06 | 腾讯科技（深圳）有限公司 | Data verification method, device, computer readable storage medium and computer equipment |
| CN110543483A (en) * | 2019-08-30 | 2019-12-06 | 北京百分点信息科技有限公司 | Data auditing method and device and electronic equipment |
-
2019
- 2019-12-20 CN CN201911327676.0A patent/CN111125063B/en active Active
Patent Citations (14)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20150142858A1 (en) * | 2013-11-15 | 2015-05-21 | Workday, Inc. | Identifying and formatting data for data migration |
| CN105989044A (en) * | 2015-02-04 | 2016-10-05 | 阿里巴巴集团控股有限公司 | Database verification method and system |
| US20170097839A1 (en) * | 2015-10-01 | 2017-04-06 | International Business Machines Corporation | Risk-appropriate validation for live operating system migration |
| WO2017071337A1 (en) * | 2015-10-26 | 2017-05-04 | 中兴通讯股份有限公司 | Database table data management method, apparatus and system |
| CN106055670A (en) * | 2016-06-06 | 2016-10-26 | 中国工商银行股份有限公司 | Inter-system data migration method and device |
| CN108241632A (en) * | 2016-12-23 | 2018-07-03 | 航天星图科技（北京）有限公司 | A kind of data verification method of data base-oriented Data Migration |
| US20190266134A1 (en) * | 2017-01-11 | 2019-08-29 | Tencent Technology (Shenzhen) Company Limited | Data migration method, apparatus, and storage medium |
| US20180232174A1 (en) * | 2017-02-15 | 2018-08-16 | Beijing Baidu Netcom Science And Technology Co., Ltd. | Data Migration Between Cloud Storage Systems |
| US20190155801A1 (en) * | 2017-08-16 | 2019-05-23 | Walmart Apollo, Llc | Systems and methods for distributed data validation |
| CN107766572A (en) * | 2017-11-13 | 2018-03-06 | 北京国信宏数科技有限责任公司 | Distributed extraction and visual analysis method and system based on economic field data |
| CN108073688A (en) * | 2017-11-20 | 2018-05-25 | 苏宁云商集团股份有限公司 | A kind of method and device of Data Migration |
| CN108595664A (en) * | 2018-04-28 | 2018-09-28 | 尚谷科技（天津）有限公司 | A kind of agricultural data monitoring method under hadoop environment |
| CN110209521A (en) * | 2019-02-22 | 2019-09-06 | 腾讯科技（深圳）有限公司 | Data verification method, device, computer readable storage medium and computer equipment |
| CN110543483A (en) * | 2019-08-30 | 2019-12-06 | 北京百分点信息科技有限公司 | Data auditing method and device and electronic equipment |
Non-Patent Citations (1)
| Title |
|---|
| 邱景;李宜卓;: "基于Spark的大规模软件完整性校验行为识别框架", 软件导刊, no. 04, pages 52 - 55 * |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111680004A (en) * | 2020-06-08 | 2020-09-18 | 中国银行股份有限公司 | Method and device for checking migration accuracy of unstructured image file |
| CN111680004B (en) * | 2020-06-08 | 2023-09-22 | 中国银行股份有限公司 | Method and device for checking migration accuracy of unstructured image file |
Also Published As
| Publication number | Publication date |
|---|---|
| CN111125063B (en) | 2023-09-26 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN102314460B (en) | Data analysis method and system and servers | |
| CN107992492B (en) | Data block storage method, data block reading method, data block storage device, data block reading device and block chain | |
| CN111367976A (en) | Method and device for exporting EXCEL file data based on JAVA reflection mechanism | |
| EP3796182A1 (en) | Data processing method and apparatus, and computer-readable storage medium | |
| CN113688288B (en) | Data association analysis method, device, computer equipment and storage medium | |
| CN111061696A (en) | Method and device for analyzing transaction message log | |
| CN111125063B (en) | Method and device for rapidly checking data migration among clusters | |
| CN111221698A (en) | Task data acquisition method and device | |
| CN108133017A (en) | A kind of multi-data source acquisition configuration method and device | |
| CN110727565B (en) | Network equipment platform information collection method and system | |
| CN115757174A (en) | Database difference detection method and device | |
| CN115509637A (en) | Form-based intelligent filling method, system, equipment and medium | |
| WO2021129849A1 (en) | Log processing method, apparatus and device, and storage medium | |
| CN111143156B (en) | Big data platform garbage task acquisition system, method and computer system | |
| CN113377681A (en) | Test case data processing method and device, electronic equipment and storage medium | |
| CN114281761A (en) | Data file loading method and device, computer equipment and storage medium | |
| CN103034730A (en) | Data processing method and system based on Android platform | |
| CN110955709B (en) | Data processing method and device and electronic equipment | |
| KR20130068633A (en) | Apparatus and method for visualizing data | |
| CN105930323A (en) | File generating method and apparatus | |
| CN106815106B (en) | Host interface test method, device, test terminal and system | |
| CN111651531A (en) | Data import method, device, equipment and computer storage medium | |
| CN111090436B (en) | Jenkins compiling result summarizing method, device and equipment | |
| CN111143329A (en) | Data processing method and device | |
| CN111107154B (en) | Data reporting method and device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |