CN113220660A - Data migration method, device and equipment and readable storage medium - Google Patents
Data migration method, device and equipment and readable storage medium Download PDFInfo
- Publication number
- CN113220660A CN113220660A CN202110406594.6A CN202110406594A CN113220660A CN 113220660 A CN113220660 A CN 113220660A CN 202110406594 A CN202110406594 A CN 202110406594A CN 113220660 A CN113220660 A CN 113220660A
- Authority
- CN
- China
- Prior art keywords
- migration
- data
- internet
- subdata
- things
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/21—Design, administration or maintenance of databases
- G06F16/214—Database migration support
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/21—Design, administration or maintenance of databases
- G06F16/219—Managing data history or versioning
Landscapes
- Engineering & Computer Science (AREA)
- Databases & Information Systems (AREA)
- Theoretical Computer Science (AREA)
- Data Mining & Analysis (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The disclosure relates to a data migration method, a data migration device, data migration equipment and a readable storage medium, and relates to the technical field of Internet of things. The method comprises the following steps: acquiring data information corresponding to data to be migrated; determining migration requirements based on the data information; determining a grouping rule based on the migration requirement, wherein the grouping rule is used for dividing the data into at least two groups of subdata; and controlling the first Internet of things equipment to migrate the subdata to the second Internet of things equipment according to the grouping rule. According to the method, the data information is determined, and the grouping rule is determined based on the data information, so that the first Internet of things device carries out data migration to the second Internet of things device according to the grouping rule of the data. In the migration process, the data is divided into a plurality of groups of subdata according to the information of the data, the data migration is carried out in the subdata form, so that the data volume of each migration is small, the migration processes of the subdata are mutually independent, and the success rate of the data migration is improved.
Description
Technical Field
The present disclosure relates to the field of internet of things technologies, and in particular, to a data migration method, apparatus, device, and readable storage medium.
Background
In the scene of the internet of things, different devices have different system versions. In different devices corresponding to different versions, data synchronization between the two devices needs to be realized through a data migration method.
In the related art, data migration is generally performed by an unmanaged data migration scheme. An unmanaged solution is data migration through scripting or open source tools.
However, the data migration process has high requirements for the size of the data volume and the migration conditions, and the success rate of data migration by the prior art is low.
Disclosure of Invention
The disclosure relates to a data migration method, a data migration device, a data migration apparatus and a readable storage medium, which can reduce the risk of failure of data migration. The technical scheme is as follows:
in one aspect, a data migration method is provided, and the method includes:
acquiring data information corresponding to data to be migrated, wherein the data is used for migrating from first internet of things equipment configured with a first system version to second internet of things equipment configured with a second system version;
determining migration requirements corresponding to the data based on the data information;
determining a grouping rule based on billions of requirements, wherein the grouping rule is used for dividing data into at least two groups of subdata;
and controlling the first Internet of things equipment to migrate the subdata to the second Internet of things equipment according to the grouping rule.
In another aspect, a data migration apparatus is provided, the apparatus including:
the system comprises an acquisition module, a storage module and a processing module, wherein the acquisition module is used for acquiring data information corresponding to data to be migrated, and the data is used for migrating from first internet of things equipment configured with a first system version to second internet of things equipment configured with a second system version;
the determining module is used for determining migration requirements corresponding to the data based on the data information;
the determining module is further used for determining a grouping rule based on the migration requirement, and the grouping rule is used for dividing the data into at least two groups of subdata;
and the control module is used for controlling the first Internet of things equipment to migrate the subdata to the second Internet of things equipment according to the grouping rule.
In another aspect, a computer device is provided, which includes a processor and a memory, where at least one instruction, at least one program, a code set, or a set of instructions is stored in the memory, and the at least one instruction, the at least one program, the code set, or the set of instructions is loaded and executed by the processor to implement the data migration method provided in the embodiments of the present disclosure.
In another aspect, a computer-readable storage medium is provided, in which at least one instruction, at least one program, a set of codes, or a set of instructions is stored, and the at least one instruction, the at least one program, the set of codes, or the set of instructions is loaded and executed by a processor to implement the data migration method provided in the embodiments of the present disclosure.
In another aspect, a computer program product or computer program is provided, the computer program product or computer program comprising computer instructions stored in a computer readable storage medium. The processor of the computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions to cause the computer device to perform the data migration method described in any of the above embodiments.
The beneficial effect that technical scheme that this disclosure provided brought includes at least:
the data migration method includes the steps that data information corresponding to first internet-of-things equipment serving as source equipment of data migration, second internet-of-things equipment serving as target equipment of the data migration and data information corresponding to data needing to be migrated are determined, and migration requirements and grouping rules are determined based on the data information, so that the first internet-of-things equipment conducts data migration to the second internet-of-things equipment according to the grouping rules of the data. In the migration process, the data is divided into a plurality of groups of subdata according to the information of the data, the data migration is carried out in the subdata form, so that the data volume of each migration is small, the migration processes of the subdata are mutually independent, and the success rate of the data migration is improved.
Drawings
In order to more clearly illustrate the technical solutions in the embodiments of the present disclosure, the drawings needed to be used in the description of the embodiments are briefly introduced below, and it is obvious that the drawings in the following description are only some embodiments of the present disclosure, and it is obvious for those skilled in the art to obtain other drawings based on the drawings without creative efforts.
Fig. 1 is a schematic diagram illustrating an internet of things scenario provided by an exemplary embodiment of the present application;
FIG. 2 illustrates a flow chart of a data migration method provided by an exemplary embodiment of the present application;
FIG. 3 illustrates a flow chart for determining migration results provided by an exemplary embodiment of the present application;
FIG. 4 is a diagram illustrating a migration list stored in a server according to an exemplary embodiment of the present application;
FIG. 5 illustrates a flow chart of a method of data migration provided by another exemplary embodiment of the present application;
fig. 6 shows a schematic structural diagram of a first internet of things device and a second internet of things device in a corresponding online migration scheme;
fig. 7 shows a schematic structural diagram of a first internet of things device and a second internet of things device when an offline migration scheme is adopted;
FIG. 8 illustrates a process diagram for data migration provided by an exemplary embodiment of the present application;
FIG. 9 illustrates a schematic diagram of a data migration apparatus provided by an exemplary embodiment of the present application;
FIG. 10 illustrates a schematic diagram of a data migration apparatus provided in another exemplary embodiment of the present application;
fig. 11 shows a block diagram of a server provided by an exemplary embodiment of the present disclosure.
Detailed Description
To make the objects, technical solutions and advantages of the present disclosure more apparent, embodiments of the present disclosure will be described in detail with reference to the accompanying drawings.
First, terms referred to in the embodiments of the present disclosure are briefly described:
the Internet of Things (IoT) refers to a huge network formed by combining various information sensing devices, such as radio frequency identification devices, infrared sensors, global positioning systems, laser scanners, and the like, with the Internet, and aims to connect all articles with the network, so as to facilitate identification and management of users or other systems.
In the scenario of the internet of things, at least two pieces of internet of things devices are generally included, and at least one server is used for realizing data interaction of all the pieces of internet of things devices in the scenario. Fig. 1 is a schematic diagram illustrating an internet of things scenario provided in an exemplary embodiment of the present application. In the internet of things scenario 100, a first internet of things device 110, a second internet of things device 120, and a server 130 are included. The first internet of things device 110 corresponds to a first system version, the second internet of things device 120 corresponds to a second system version, and the server 130 is configured to implement data interaction between the first internet of things device 110 and the second internet of things device 120. In an example, during an operation process, the first internet of things device 110 and the second internet of things device 120 both send generated data to the server 130 for storage, where the server 130 stores a first system version identifier of a first system version corresponding to the first internet of things device 110 and a second system version identifier of a second system version corresponding to the second internet of things device 120.
In this application embodiment, the internet of things device can be implemented as at least one of a terminal, a server or a cloud server, and the specific form of the internet of things device is not limited in this application embodiment.
Data migration is a process that is applied among a plurality of internet of things devices and migrates data that is providing online services from one internet of things device to another internet of things device. According to the network connection state in the migration process, the data migration mode comprises an off-line migration mode and an on-line migration mode; according to the manual participation state in the migration process, the data migration mode comprises an unmanaged mode, a semi-managed mode and a full managed mode.
Fig. 2 shows a flowchart of a data migration method provided in an exemplary embodiment of the present application, which is described by taking as an example that the method is applied to a server in a scene of an internet of things, and the method includes:
In the embodiment of the application, the server is a server in the scene of the internet of things. In one example, the server is implemented as an entity server that establishes network connection relationships with both the first internet-of-things device and the second internet-of-things device; in another example, the server may be implemented as a cloud server that establishes communication network connections with both the first internet-of-things device and the second internet-of-things device.
Correspondingly, the first internet of things device and the second internet of things device are devices which are in communication connection with the server. In the embodiment of the present application, the communication connection may be implemented as a communication connection based on a long-distance network communication technology, and may also be implemented as a communication connection based on a short-distance network communication technology. In one example, the first internet of things device and the second internet of things device are implemented as two smart home devices, and the server is implemented as a smart home central control device connected with the two smart home devices through a bluetooth network; in another example, the first and second internet-of-things devices are implemented as two electricity meters. The server is implemented as a cloud server. The first Internet of things device and the second Internet of things device are in communication connection with the cloud server through the narrow-band Internet of things technology, and data interaction can be carried out through the cloud server.
In this embodiment, the first internet of things device corresponds to a first system version, and the second internet of things device corresponds to a second system version. Optionally, the first system version is different from the second system version. In one example, the first internet of things device and the second internet of things device are established with a communication connection through a narrowband internet of things technology, and during the updating process of the system version by the server, the first internet of things device is awakened, and the system version is updated in response to the receiving of the new system version. The second networked device is not woken up, i.e. no update of the system version is made. At this time, a difference in system version exists between the first internet of things device and the second internet of things device. In another example, the first internet of things device is implemented as a water meter, and the second internet of things device is implemented as a smart home device, and at this time, the first internet of things device and the second internet of things device are configured with different series of system versions. In another example, the system version of the first internet of things device is the same as the system version of the second internet of things device. The embodiment of the application does not limit the relationship between the system version of the first internet of things device and the system version of the second internet of things device.
In this embodiment of the application, data needs to be migrated from the first internet of things device to the second internet of things device, where the data is data stored in the first internet of things device. In the migration process, the first internet of things device can still use the data.
In the embodiment of the application, the migration requirement is determined based on the data information, and in the data migration process, the first internet of things device and the second internet of things device have requirements for the migration process. Optionally, the migration requirement may be determined by data information. In an example, the data information indicates that the size of the data is 100G, and the migration requirement includes a requirement that the migration device needs to have 100G of memory.
In the embodiment of the application, the data corresponds to data information. The data information corresponding to the data is used for characterizing the characteristics of the data. In one example, the data information indicates a size of the data, and a data size correspondence table is stored in the server. The server determines a grouping rule from the data size corresponding table according to the size of the data; in another example, the data information includes grouping requirements of the data, and the server determines the grouping rule for the data according to the grouping requirements of the data.
In the present application, the device storing data divides the data into at least two groups of sub data according to a grouping rule. In one example, the total size of the data is 1000G, and the grouping rule indicates that the size of each group of sub-data is 10G, then 100 groups of sub-data can be finally obtained, and the size of each group of sub-data is 10G.
Optionally, the content in each group of sub data is not repeated.
Optionally, when the first system version is different from the second system version, the data needs to be adapted to the difference between the first system version and the second system version for format conversion while being grouped.
And 204, controlling the first internet of things device to migrate the subdata to the second internet of things device according to the grouping rule.
In this embodiment of the application, since the data migration is the migration of data from the second first internet of things device to the second internet of things device, the server cannot directly migrate the data to the second internet of things device, and the first internet of things device needs to be instructed to migrate the data to the second internet of things device in the form of subdata.
In one example, the number of the sub data is 100 groups, and the server instructs the first internet of things device to simultaneously migrate 100 groups of sub data to the second internet of things device; in another example, the number of the sub-data is 100, and the server instructs the first internet of things device to sequentially migrate 100 sub-data to the second internet of things device; in another example, the number of the sub data is 100 groups, and the server instructs the first internet of things device to migrate 100 groups of sub data into the second internet of things device 10 times, and migrate 10 groups at a time; in another example, the number of the sub data is 100, and the server instructs the first internet of things device to migrate 100 sub data groups into the second internet of things device 10 times, and the number of the sub data groups migrated each time is uncertain. In the embodiment of the present application, the process and the method for the first internet of things device to perform packet transmission on the sub data are not limited.
In summary, in the method provided in this embodiment, by determining the first internet of things device serving as the source device of data migration, the second internet of things device serving as the target device of data migration, and the data information corresponding to the data to be migrated, and determining the migration requirement and the grouping rule based on the data information, the first internet of things device performs data migration to the second internet of things device according to the grouping rule of the data. In the migration process, the data is divided into a plurality of groups of subdata according to the information of the data, the data migration is carried out in the subdata form, so that the data volume of each migration is small, the migration processes of the subdata are mutually independent, and the success rate of the data migration is improved.
After the data migration is completed, the server can perform success verification on the data migration through data reading of the second networked device, and execute a corresponding compensation step when the migration fails. Fig. 3 shows a flowchart for determining a migration result according to an exemplary embodiment of the present application, which is described by taking as an example that the method is applied to a server in a scenario of internet of things, and includes:
In the embodiment of the application, after the sub-data is migrated from the first internet of things device to the second internet of things device, the second internet of things device sends a migration completion prompt to the server, and the migration completion prompt is used for reminding the server that the data migration process is completed and indicating the server to verify the migration result of the sub-data.
In the embodiment of the application, the sub-data corresponds to a sub-data identifier, and the sub-data identifier may indicate a position of the sub-data in the original data. In one example, if the number of the sub-data is 100, then according to the position of the sub-data group in the original data, the sub-data is labeled with 1 to 100, which is used as the sub-data group identifier.
In one example, the server checks the integrity of the data, and when the integrity check of the data indicates that the data is complete, the migration is determined to be successful. When the migration is determined to be successful, the server may indicate that the data migration is complete.
In another example, the server detects the integrity of the data, and when the integrity detection of the data indicates that the data is incomplete, i.e., it is determined that the migration is not completely successful, the server needs to determine the child data with which the migration failed. To further determine the location in the data of the child data for which migration was unsuccessful. Alternatively, the location may be decided by a sub data identification corresponding to the sub data.
In the embodiment of the application, the first internet of things device corresponds to a first system version, the second internet of things device corresponds to a second system version, and the first system version is different from the second system version.
Step 302, in response to that the migration result is a migration success result, recording the subdata identifications in a migration success list, where the migration success list includes the subdata identifications corresponding to the subdata with successful migration.
In the embodiment of the present application, the migration result indicates a migration result corresponding to each group of child data. The migration result comprises a migration success result and a migration failure result, wherein the migration success result indicates that the subdata is completely migrated from the first internet of things device to the second internet of things device; the migration failure result indicates that the sub data is not completely migrated from the first internet of things device to the second internet of things device.
And in response to the migration result being a successful migration result, the server may determine that the subdata has been completely and accurately migrated from the first internet of things device to the second internet of things device.
Alternatively, the migration failure list and the migration success list in step 302 may be merged into the same list. Fig. 4 is a diagram illustrating a migration list stored in a server according to an exemplary embodiment of the present application. The migration list 400 includes a migration success list 410 and a migration failure list 420. And the server checks the integrity of the subdata according to the acquired data information and the sequence corresponding to the subdata identifier and judges the migration result of the subdata. As shown in fig. 4, the migration result of the sub data group with the sub data identifiers 3 and 58 is the migration failure result, and the migration result of the other sub data groups is the migration success result. At this time, the sub data migration failure of the sub data id 3 and the sub data id 58 is indicated.
And step 304, determining the subdata corresponding to the subdata identifier in the migration failure list.
In the application, after the sub data migration fails, the server determines the sub data corresponding to the sub data according to the sub data identifier. In one example, the server determines a storage location of the subdata from the first internet of things device according to the data information of the data, and then determines the subdata corresponding to the subdata identifier.
And 305, migrating the subdata from the first internet of things device to the second internet of things device again.
After determining the subdata, the server instructs the first internet of things device to migrate the subdata to the second internet of things device again.
Optionally, after migrating the sub-data to the second networked device again, the server repeatedly performs the detection process for the integrity of the sub-data. When the migration result indicated by the subdata re-migrated to the second networked device is updated to a migration success result, determining that the migration is successful; and when the subdata re-migrated to the second networking equipment indicates that the migration result is not updated and is still the migration failure result, determining that the migration is unsuccessful, and indicating the first networking equipment to migrate the subdata which is not successfully migrated to the second networking equipment again.
And until the migration result corresponding to each subdata is a successful migration result, the server determines that the first Internet of things device successfully migrates the data to the second Internet of things device.
In summary, the method provided in this embodiment detects the migration result for each group of sub-data, and when the data migration is not completely successful, can quickly determine the reason why the migration is not completely successful, and provide a corresponding solution, thereby improving the efficiency of data migration.
After the reason for the incomplete migration is determined, the sub-data which is not successfully migrated is migrated again, and all the data is not migrated again, so that the efficiency of data migration is further improved.
Fig. 5 shows a flowchart of a data migration method provided in an exemplary embodiment of the present application, please refer to fig. 5, which is described as being applied to a server in a scenario of internet of things, and the method includes:
In the embodiment of the present application, the data information is information corresponding to data to be migrated. In one example, the data information may include a size of the data, a content summary of the data, and a format of the data; in another example, the data information further includes requirements for data migration, including at least one of a length of time requirement for data migration, an environmental requirement for data migration, and a cost requirement for data migration.
In the embodiment of the application, the first internet of things device corresponds to a first system version, the second internet of things device corresponds to a second system version, and the first system version is different from the second system version.
In the embodiment of the application, part of the migration requirements can be directly obtained from the data information. In one example, the data information includes a duration requirement for data migration, and the duration can be directly used as duration data in the migration requirement. The migration requirement acquisition mode is not limited in the application. In the embodiment of the present application, the migration requirement includes at least one of a duration requirement, a cost requirement, and a tool requirement. The time length requirement indicates that migration time length of the data is estimated to obtain estimated time length; the cost requirement indicates that the migration cost of the data is estimated to obtain the migration cost; the tool requirement indicates the type of the migration tool obtained by performing migration tool pre-estimation on the data.
At step 503, grouping rules are determined based on the migration requirements.
In the present application, the device storing data divides the data into at least two groups of sub data according to a grouping rule. In one example, the total size of the data is 1000G, and the grouping rule indicates that the size of each group of sub-data is 10G, then 100 groups of sub-data can be finally obtained, and the size of each group of sub-data is 10G.
In an example of the embodiment of the application, a server stores preset candidate grouping rules, and selects a grouping rule from the preset candidate grouping rules based on a migration requirement; in another example, the server creates new grouping rules based on migration requirements.
Optionally, the content in each group of sub data is not repeated.
In an embodiment of the present application, the migration scheme includes an online migration scheme and an offline migration scheme. The online migration scheme is a scheme for realizing data migration through communication connection between the first internet of things device and the second internet of things device. The offline migration scheme is a scheme that data are firstly written into other storage devices from the first internet of things device, and then are mounted on the second internet of things device through the other storage devices and are written into the second internet of things device, so that data migration is achieved.
In one example, the migration requirements are: the number of the devices is 1000, the number of the single-device measuring points is 50, the measuring point frequency is in the second level, the migration time is 90 days, the data of the single-device measuring points is 30 bytes, and the network bandwidth is 20 MB. The data volume was 100 x 50 x 86400 x 90 x 30/1024/1024/1024 x 10862G, and the migration time required was 10862G/20MB x 6.3 days.
In the embodiment of the present application, in response to the migration scheme being the online migration scheme, steps 505 to 506 are performed; in response to the migration scheme being the offline migration scheme, steps 507 to 508 are performed.
And 505, responding to the migration scheme being an online migration scheme, and configuring the communication network based on the migration requirement.
When the migration scheme is an online migration scheme, the server configures a communication network established between the first internet of things device and the second internet of things device according to the migration requirement. In one example, if the total migration duration indicated in the migration requirement is 20 hours, the server configures the communication network to be on for 21 hours. In another example, the migration requirement indicates that the configuration of the communication network is performed by cellular mobile data, and the server configures the cellular mobile network as the communication network according to the indication.
After the server establishes the network connection between the first communication network and the second communication network, the server indicates the first internet of things device to send data to the second internet of things device through the communication network.
In this embodiment of the application, the data is sent to the second internet of things device in the form of a data group, so the server may instruct the first internet of things device to send the sub-data in different forms. In one example, the server instructs the first internet of things device to send all the subdata to the second internet of things device simultaneously; in another example, the server instructs the first internet of things device to send the sub-data to the second internet of things device once, and after the previous sub-data is sent, the next sub-data is sent at an interval of 30 seconds.
Referring to fig. 6, corresponding to the online migration scheme, the first internet of things device 610 has a source database 611 and a migration tool 612, where the migration tool 612 includes a parameter configuration module 613, configured to perform parameter configuration according to data that needs to be migrated; a task splitting module 614, configured to split data into sub-data; an agent module 615 for coordinating the work among the modules; a progress recording module 616, configured to record progress of data packaging and sending; a file database 617 for storing data; a message sending module 618, configured to send data. After passing through the server configured public network 620, the data is sent to the second networked device 630. The second networking device 630 includes a message queue module 631, configured to sort the received sub data; the write-in service module 632 is configured to write the received sub data into a second version system corresponding to the second networking device; a target database 633 for receiving data; a dependency module 634 for checking dependency between the data and the environment.
In step 507, in response to the migration solution being the offline migration solution, the device status of the migration tool is determined.
When the migration scheme is an offline migration scheme, the server determines the equipment state of the migration tool according to the migration requirement. Optionally, the server selects a tool to be migrated according to the migration tool type determined by the migration requirement.
Referring to fig. 7, corresponding to the offline migration scheme, the first internet of things device 710 includes a database 711 and a migration tool 712, and the migration tool 712 includes a parameter configuration module 713, configured to perform parameter configuration according to data that needs to be migrated; a task splitting module 714, configured to split the data into sub-data; and the agent module 715 is used for allocating the work among the modules. After the data is packed into the migration device 720, the migration device 720 migrates the data into the second networked device 730 in a physical transportation manner, and the second networked device 730 includes a message queue module 731 for sorting the received subdata; the write service module 732 is configured to write the received sub data into a second version system corresponding to the second networking device; a target database 733 for receiving data; a dependency module 734 for checking dependencies between the data and the environment.
After performing the online migration scheme as in steps 505 to 506 or performing the offline migration scheme as in steps 507 to 508, the server performs step 509.
In step 509, the sub data in the second networked device is read.
In this step, the server reads the sub-data in the second networked device to determine the integrity of the data migrated to the second networked device.
Optionally, the server determines the sub-data in the second networked device by obtaining the data information in the second networked device.
At step 510, the integrity of the child data in the second networked device is verified.
In the embodiment of the application, the server firstly determines the subdata identifier corresponding to the subdata, and then performs integrity verification on the subdata once according to the subdata identifier.
Optionally, the server determines a migration result of the sub-data according to the verification result of the integrity of the sub-data, and writes the migration result into the migration result list.
And step 511, responding to the completeness of the subdata, and determining that the subdata is successfully migrated.
And in response to the incomplete subdata, the server determines the subdata according to the subdata identifier, and instructs the first internet of things device to send the subdata to the second internet of things device again after the subdata identifier is determined.
In summary, in the method provided in this embodiment, by determining the first internet of things device serving as the source device of data migration, the second internet of things device serving as the target device of data migration, and the data information corresponding to the data to be migrated, and determining the migration requirement and the grouping rule based on the data information, the first internet of things device performs data migration to the second internet of things device according to the grouping rule of the data. In the migration process, the data is divided into a plurality of groups of subdata according to the information of the data, the data migration is carried out in the subdata form, so that the data volume of each migration is small, the migration processes of the subdata are mutually independent, and the success rate of the data migration is improved.
The data migration environment and the data migration mode are determined by the mode of determining the migration environment according to the corresponding data, the device states of the first internet of things device and the second internet of things device and the setting of the migration condition corresponding to the migration requirement, and the success rate of data migration is further improved.
Fig. 8 is a schematic diagram illustrating a data migration process according to an exemplary embodiment of the present application. The process is applied to the environment of the Internet of things, and comprises the following steps:
The implementation environment of the embodiment of the application comprises a server, a first Internet of things device and a second Internet of things device.
Optionally, the process is a process of testing a migration tool that needs to be used in the offline migration process. Through the test process, tool data in the migration requirement can be obtained. Optionally, the process is performed by the server instructing the migration tool and obtaining the function and performance test results.
The process is a dependency check process, and is a check on the degree of adaptability of the data to the second system version corresponding to the second networked device in the data migration process. Optionally, the process is performed by the server instructing the second networked device and obtaining the environment to check the result.
The process is that the server acquires the data information and generates the grouping rule based on the data information. In the process, the configuration file is a file which is sent to the first internet of things device by the server and indicates the grouping rule.
The process is a process for determining the migration requirement according to the grouping rule and the data information. In the embodiment of the present application, the migration requirement includes a migration time and a migration cost.
And step 805, determining a migration scheme according to the evaluation result.
In the embodiment of the present application, the migration scheme includes an online migration scheme and an offline migration scheme. Among other things, offline migration schemes require the use of migration tools.
In response to the migration scheme being the online migration scheme, steps 806 through 807 are performed.
Step 806, network security and proxy configuration.
The process is a process in which the server configures the communication network in response to the migration scheme being the online migration scheme. Through the proxy configuration, a communication network connection is established between the first internet of things device and the second internet of things device.
In step 807, the migration tool is executed at the source end to query data, and write into the target environment through the network.
The process is a process of sending the data from the first internet of things device to the second internet of things device in the form of subdata.
In response to the migration scheme being the offline migration scheme, steps 808-810 are performed.
The process is a process of determining a migration tool, optionally, data corresponding to the migration tool is stored in the first internet of things device, and the migration tool can be determined by querying the data.
Optionally, a connection relationship is established between the first internet of things device and the migration tool.
In the embodiment of the application, the first internet of things device migrates the data to the migration tool, or the first internet of things device mirrors the data and sends the data to the migration tool. And then, the connection between the first internet of things device and the migration tool is released, and the connection between the migration tool and the second internet of things device is established.
The process is a process in which the migration tool writes data into a database in the second networked device, and after the data is written, the second networked device receives the data.
The process is a process of verifying the integrity of the subdata identification data after the subdata migration is finished.
In summary, in the method provided in this embodiment, by determining the first internet of things device serving as the source device of data migration, the second internet of things device serving as the target device of data migration, and the data information corresponding to the data to be migrated, and determining the migration requirement and the grouping rule based on the data information, the first internet of things device performs data migration to the second internet of things device according to the grouping rule of the data. In the migration process, the data is divided into a plurality of groups of subdata according to the information of the data, the data migration is carried out in the subdata form, so that the data volume of each migration is small, the migration processes of the subdata are mutually independent, and the success rate of the data migration is improved.
Fig. 9 is a schematic diagram illustrating a data migration apparatus according to an exemplary embodiment of the present application, please refer to fig. 9, where the apparatus includes:
an obtaining module 901, configured to obtain data information corresponding to data to be migrated, where the data is used to migrate from a first internet of things device configured with a first system version to a second internet of things device configured with a second system version;
a determining module 902, configured to determine, based on the data information, a migration requirement corresponding to the data;
the determining module 902 is further configured to determine a grouping rule based on the migration requirement, where the grouping rule is used to divide the data into at least two groups of sub data;
the control module 903 is configured to control the first internet of things device to migrate the sub data to the second internet of things device according to the grouping rule.
In an optional embodiment, the child data corresponds to a child data identifier;
the determining module 902 is further configured to determine, in response to the sub data being migrated from the first internet of things device to the second internet of things device, a migration result of the sub data, where the migration result includes a migration success result and a migration failure result;
the apparatus further includes a recording module 904, configured to record a sub data identifier in a migration success list in response to that the migration result is a migration success result, where the migration success list includes the sub data identifier corresponding to the sub data that was successfully migrated.
In an optional embodiment, the recording module 904 is further configured to record the sub data identifier in a migration failure list, where the migration failure list includes the sub data identifier corresponding to the sub data with which migration has failed.
In an optional embodiment, the determining module 902 is further configured to determine sub data corresponding to the sub data identifier in the migration failure list;
and the control module 903 is configured to control the first internet of things device to migrate the sub-data to the second internet of things device again.
In an alternative embodiment, the migration requirement comprises at least one of a duration requirement, a cost requirement, and a tool requirement;
the time length requirement indicates that migration time length of the data is estimated to obtain estimated time length;
the cost requirement indicates that the migration cost of the data is estimated to obtain the migration cost;
the tool requirement indicates the type of the migration tool obtained by performing migration tool pre-estimation on the data.
In an optional embodiment, before controlling the first internet of things device to migrate the sub data to the second internet of things device according to the grouping rule, the determining module 902 is further configured to determine a migration scheme based on the migration requirement and the grouping rule, where the migration scheme includes an online migration scheme and an offline migration scheme.
In an optional embodiment, the apparatus further includes a configuration module 905 configured to configure the communication network based on the migration requirement;
the control module 903 is further configured to control the first internet of things device to send the sub data from the first internet of things device to the second internet of things device according to a grouping rule through a communication network;
the device further comprises a reading module 906, which is used for reading the subdata in the second networking equipment;
the apparatus further includes a verification module 907 for verifying the integrity of the sub data in the second networked device;
the determining module 902 is further configured to determine that the sub data migration is successful in response to the completeness of the sub data.
In an optional embodiment, the determining module 902 is further configured to determine a device status of a migration tool, where the migration tool is an electronic device used for migrating data from a first internet of things device to a second internet of things device in an offline migration scheme;
the apparatus further includes an indication module 908, configured to indicate, in response to an indication that the migration tool has a capability of performing data migration according to the device status, that the first internet of things device writes the sub-data into the migration tool according to a grouping rule;
a reading module 906, configured to read the sub data in the second networked device;
a determining module 902, configured to determine that the sub data migration is successful in response to the completeness of the sub data.
In an optional embodiment, the apparatus further includes a selecting module 909 for selecting a grouping rule based on migration requirement from the preset candidate grouping rules;
or the like, or, alternatively,
the apparatus further includes a creation module 910 for creating a new grouping rule based on the migration requirement.
In summary, the apparatus provided in the embodiment of the present application determines, as a source device of data migration, a first internet of things device, a second internet of things device of a target device of data migration, and data information corresponding to data to be migrated, and determines a grouping rule according to the data information, so that the first internet of things device performs data migration to the second internet of things device according to the grouping rule of the data. In the migration process, the data is divided into a plurality of groups of subdata according to the information of the data, the data migration is carried out in the subdata form, so that the data volume of each migration is small, the migration processes of the subdata are mutually independent, and the success rate of the data migration is improved.
It should be noted that: the data migration apparatus provided in the foregoing embodiment is only illustrated by dividing the functional modules, and in practical applications, the functions may be allocated to different functional modules according to needs, that is, the internal structure of the device is divided into different functional modules, so as to complete all or part of the functions described above.
The present disclosure also provides a server, which includes a processor and a memory, where the memory stores at least one instruction, and the at least one instruction is loaded and executed by the processor to implement the data migration method provided by the foregoing method embodiments. It should be noted that the server may be a server as provided in fig. 11 below.
Referring to fig. 11, a schematic structural diagram of a server according to an exemplary embodiment of the present disclosure is shown. Specifically, the method comprises the following steps: the server 1100 includes a Central Processing Unit (CPU) 1101, a system Memory 1104 including a Random Access Memory (RAM) 1102 and a Read-Only Memory (ROM) 1103, and a system bus 1105 connecting the system Memory 104 and the CPU 1101. The server 1100 also includes a basic Input/Output (I/O) System 106, which facilitates the transfer of information between devices within the computer, and a mass storage device 1107 for storing an operating System 1111, application programs 1114 and other program modules 1115.
The basic input/output system 1106 includes a display 1108 for displaying information and an input device 1109 such as a mouse, keyboard, etc. for user input of information. Wherein the display 1108 and the input device 1109 are connected to the central processing unit 1101 through an input output controller 1110 connected to the system bus 1105. The basic input/output system 1106 may also include an input/output controller 1110 for receiving and processing input from a number of other devices, such as a keyboard, mouse, or electronic stylus. Similarly, input-output controller 1110 also provides output to a display screen, a printer, or other type of output device.
The mass storage device 1107 is connected to the central processing unit 1101 through a mass storage controller (not shown) that is connected to the system bus 1105. The mass storage device 1107 and its associated computer-readable media provide non-volatile storage for the server 1100. That is, the mass storage device 1107 may include a computer-readable medium (not shown) such as a hard disk or CD-ROM drive.
Without loss of generality, computer readable media may comprise computer storage media and communication media. Computer storage media includes volatile and nonvolatile, removable and non-removable media implemented in any method or technology for storage of information such as computer readable instructions, data structures, program modules or other data. Computer storage media includes RAM, ROM, Erasable Programmable Read-Only Memory (EPROM), Electrically Erasable Programmable Read-Only Memory (EEPROM), flash Memory or other solid state Memory technology, CD-ROM, Digital Versatile Disks (DVD), or other optical storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices. Of course, those skilled in the art will appreciate that computer storage media is not limited to the foregoing. The system memory 1104 and mass storage device 1107 described above may be collectively referred to as memory.
The memory stores one or more programs configured to be executed by the one or more central processing units 1101, the one or more programs containing instructions for implementing the data migration method described above, and the central processing unit 1101 executes the one or more programs to implement the data migration method provided by the various method embodiments described above.
The server 1100 may also operate in accordance with various embodiments of the application through remote computers connected to a network, such as the internet. That is, the server 1100 may connect to the network 1112 through the network interface unit 1113 connected to the system bus 1105, or may connect to other types of networks or remote computer systems (not shown) using the network interface unit 1111.
The memory further includes one or more programs, the one or more programs are stored in the memory, and the one or more programs include steps executed by the server for performing the data migration method provided by the embodiment of the application.
Embodiments of the present application further provide a computer-readable storage medium, where at least one instruction, at least one program, a code set, or a set of instructions is stored in the computer-readable storage medium, and the at least one instruction, the at least one program, the code set, or the set of instructions is loaded and executed by a processor to implement the data migration method.
The present application also provides a computer program product or computer program comprising computer instructions stored in a computer readable storage medium. The processor of the computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions to cause the computer device to perform the data migration method described in any of the above embodiments.
Those skilled in the art will appreciate that all or part of the steps in the methods of the above embodiments may be implemented by hardware related to instructions of a program, which may be stored in a computer readable storage medium, which may be a computer readable storage medium contained in a memory of the above embodiments; or it may be a separate computer-readable storage medium not incorporated in the terminal. The computer readable storage medium has stored therein at least one instruction, at least one program, set of codes, or set of instructions that is loaded and executed by a processor to implement the above-described data migration method.
Optionally, the computer-readable storage medium may include: a Read Only Memory (ROM), a Random Access Memory (RAM), a Solid State Drive (SSD), or an optical disc. The Random Access Memory may include a resistive Random Access Memory (ReRAM) and a Dynamic Random Access Memory (DRAM). The above-mentioned serial numbers of the embodiments of the present disclosure are merely for description and do not represent the merits of the embodiments.
It will be understood by those skilled in the art that all or part of the steps of implementing the above embodiments may be implemented by hardware, or may be implemented by a program instructing relevant hardware, and the program may be stored in a computer-readable storage medium, and the above-mentioned storage medium may be a read-only memory, a magnetic disk or an optical disk, etc.
The foregoing is considered as illustrative of the embodiments of the disclosure and is not to be construed as limiting thereof, and any modifications, equivalents, improvements and the like made within the spirit and principle of the disclosure are intended to be included within the scope of the disclosure.
Claims (12)
1. A data migration method is applied to a server, and the method comprises the following steps:
acquiring data information corresponding to data to be migrated, wherein the data is used for migrating from first internet of things equipment configured with a first system version to second internet of things equipment configured with a second system version;
determining migration requirements corresponding to the data based on the data information;
determining a grouping rule based on the migration requirement, wherein the grouping rule is used for dividing the data into at least two groups of subdata;
and controlling the first Internet of things device to migrate the subdata to the second Internet of things device according to the grouping rule.
2. The method of claim 1, wherein the child data corresponds to a child data identifier;
the method further comprises the following steps:
responding to the sub data transferred from the first internet of things device to a second internet of things device, and determining a transfer result of the sub data, wherein the transfer result comprises a transfer success result and a transfer failure result;
and recording the subdata identifications in a migration success list in response to the migration result being a migration success result, wherein the migration success list comprises the subdata identifications corresponding to the subdata with successful migration.
3. The method of claim 2, wherein in response to the migration result being a migration failure result, the method further comprises:
recording the subdata identifications in a migration failure list, wherein the migration failure list comprises the subdata identifications corresponding to the subdata with migration failure.
4. The method of claim 3, wherein after migrating the child data from the first IOT device to the second IOT device, further comprising:
determining subdata corresponding to the subdata identification in the migration failure list;
and controlling the first Internet of things equipment to migrate the subdata to the second Internet of things equipment again.
5. The method of any of claims 1 to 4, wherein the migration requirements include at least one of duration requirements, cost requirements, tool requirements;
the time length demand indication carries out migration time length estimation on the data to obtain estimated time length;
the cost requirement indicates migration cost prediction of the data to obtain migration cost;
and the tool requirement indicates the type of the migration tool obtained by performing migration tool pre-estimation on the data.
6. The method according to any one of claims 1 to 4, wherein before controlling the first IOT device to migrate the sub data to the second IOT device according to the grouping rule, the method further comprises:
determining a migration scheme based on the migration requirement and the grouping rule, wherein the migration scheme comprises an online migration scheme and an offline migration scheme.
7. The method of claim 6, wherein in response to the migration scheme being an online migration scheme, the controlling the first IOT device to migrate the child data into the second IOT device according to the grouping rules comprises:
configuring a communication network based on the migration needs;
controlling the first Internet of things device to transmit the subdata from the first Internet of things device to the second Internet of things device according to the grouping rule through the communication network;
reading subdata in the second networking equipment;
verifying the integrity of the subdata in the second networked device;
and responding to the completeness of the subdata, and determining that the subdata is successfully migrated.
8. The method of claim 6, wherein in response to the migration scheme being an offline migration scheme, the controlling the first IOT device to migrate the child data into the second IOT device according to the grouping rules comprises:
determining a device state of a migration tool, wherein the migration tool is an electronic device used for migrating the data from the first internet of things device to the second internet of things device under the offline migration scheme;
in response to the device status indicating that the migration tool has the capability of data migration, indicating the first internet of things device to write the subdata into the migration tool according to the grouping rule;
reading the subdata in the second networking equipment;
and responding to the completeness of the subdata, and determining that the subdata is successfully migrated.
9. The method according to any one of claims 1 to 4, wherein the determining a grouping rule based on the migration requirement comprises:
selecting the grouping rule from preset candidate grouping rules based on the migration requirement;
or the like, or, alternatively,
creating a new grouping rule based on the migration requirement.
10. An apparatus for data migration, the apparatus comprising:
the system comprises an acquisition module, a storage module and a processing module, wherein the acquisition module is used for acquiring data information corresponding to data to be migrated, and the data is used for migrating from first internet of things equipment configured with a first system version to second internet of things equipment configured with a second system version;
the determining module is further used for determining migration requirements corresponding to the data based on the data information;
the determining module is further configured to determine a grouping rule based on the data information, where the grouping rule is used to divide the data into at least two groups of sub data;
and the control module is used for controlling the first Internet of things device to migrate the subdata to the second Internet of things device according to the grouping rule.
11. A computer device comprising a processor and a memory, the memory having stored therein at least one instruction, at least one program, set of codes, or set of instructions, which is loaded and executed by the processor to implement a data migration method as claimed in any one of claims 1 to 9.
12. A computer readable storage medium having stored therein at least one instruction, at least one program, set of codes, or set of instructions, which is loaded and executed by a processor to implement a data migration method according to any one of claims 1 to 9.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110406594.6A CN113220660A (en) | 2021-04-15 | 2021-04-15 | Data migration method, device and equipment and readable storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110406594.6A CN113220660A (en) | 2021-04-15 | 2021-04-15 | Data migration method, device and equipment and readable storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN113220660A true CN113220660A (en) | 2021-08-06 |
Family
ID=77087449
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110406594.6A Pending CN113220660A (en) | 2021-04-15 | 2021-04-15 | Data migration method, device and equipment and readable storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113220660A (en) |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113805804A (en) * | 2021-08-27 | 2021-12-17 | 济南浪潮数据技术有限公司 | Method, device, equipment and medium for data offline migration |
| CN114553958A (en) * | 2022-01-04 | 2022-05-27 | 深圳绿米联创科技有限公司 | Data migration processing method and device, electronic equipment and storage medium |
| CN114827172A (en) * | 2022-03-30 | 2022-07-29 | 阿里云计算有限公司 | Data migration method and device based on Internet of things and electronic equipment |
| CN117610970A (en) * | 2024-01-04 | 2024-02-27 | 成都开元精创信息技术有限公司 | Intelligent evaluation method and system for data migration work |
Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20090228527A1 (en) * | 2008-03-05 | 2009-09-10 | Jinhu Wang | System and method for providing data migration services |
| CN109933287A (en) * | 2019-03-05 | 2019-06-25 | 中国联合网络通信集团有限公司 | Data migration method, device and equipment based on Internet of Things |
| CN110674108A (en) * | 2019-08-30 | 2020-01-10 | 中国人民财产保险股份有限公司 | Data processing method and device |
| CN111008188A (en) * | 2019-10-29 | 2020-04-14 | 平安科技（深圳）有限公司 | Data migration method and device, computer equipment and storage medium |
| CN111143331A (en) * | 2019-12-30 | 2020-05-12 | 广州酷狗计算机科技有限公司 | Data migration method and device and computer storage medium |
| CN111723079A (en) * | 2020-06-28 | 2020-09-29 | 深圳壹账通智能科技有限公司 | Data migration method and device, computer equipment and storage medium |
| CN112083889A (en) * | 2020-09-14 | 2020-12-15 | 腾讯科技（深圳）有限公司 | Data migration method, device, equipment and readable storage medium |
-
2021
- 2021-04-15 CN CN202110406594.6A patent/CN113220660A/en active Pending
Patent Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20090228527A1 (en) * | 2008-03-05 | 2009-09-10 | Jinhu Wang | System and method for providing data migration services |
| CN109933287A (en) * | 2019-03-05 | 2019-06-25 | 中国联合网络通信集团有限公司 | Data migration method, device and equipment based on Internet of Things |
| CN110674108A (en) * | 2019-08-30 | 2020-01-10 | 中国人民财产保险股份有限公司 | Data processing method and device |
| CN111008188A (en) * | 2019-10-29 | 2020-04-14 | 平安科技（深圳）有限公司 | Data migration method and device, computer equipment and storage medium |
| CN111143331A (en) * | 2019-12-30 | 2020-05-12 | 广州酷狗计算机科技有限公司 | Data migration method and device and computer storage medium |
| CN111723079A (en) * | 2020-06-28 | 2020-09-29 | 深圳壹账通智能科技有限公司 | Data migration method and device, computer equipment and storage medium |
| CN112083889A (en) * | 2020-09-14 | 2020-12-15 | 腾讯科技（深圳）有限公司 | Data migration method, device, equipment and readable storage medium |
Cited By (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113805804A (en) * | 2021-08-27 | 2021-12-17 | 济南浪潮数据技术有限公司 | Method, device, equipment and medium for data offline migration |
| CN113805804B (en) * | 2021-08-27 | 2024-02-13 | 济南浪潮数据技术有限公司 | Method, device, equipment and medium for offline migration of data |
| CN114553958A (en) * | 2022-01-04 | 2022-05-27 | 深圳绿米联创科技有限公司 | Data migration processing method and device, electronic equipment and storage medium |
| CN114827172A (en) * | 2022-03-30 | 2022-07-29 | 阿里云计算有限公司 | Data migration method and device based on Internet of things and electronic equipment |
| CN117610970A (en) * | 2024-01-04 | 2024-02-27 | 成都开元精创信息技术有限公司 | Intelligent evaluation method and system for data migration work |
| CN117610970B (en) * | 2024-01-04 | 2024-04-02 | 成都开元精创信息技术有限公司 | Intelligent evaluation method and system for data migration work |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN113220660A (en) | Data migration method, device and equipment and readable storage medium | |
| JP2023525393A (en) | Method and apparatus for updating gateway resources and IOT control platform | |
| CN112988485B (en) | Simulation test method and device for electric power Internet of things equipment | |
| CN106250192A (en) | The method for upgrading software of host computer and system | |
| CN112822260B (en) | File transmission method and device, electronic equipment and storage medium | |
| JP2018521411A (en) | Node configuration management in the network | |
| CN107070752B (en) | Testing method and testing system for long connection capacity | |
| CN111124286A (en) | Libcloud-based multi-cloud management implementation method | |
| CN111897846A (en) | Cache data verification method, device, equipment and storage medium | |
| CN111800468A (en) | Cloud-based multi-cluster management method, device, medium and electronic equipment | |
| CN111506358B (en) | Method and device for updating container configuration | |
| CN111651352B (en) | Warehouse code merging method and device | |
| CN110990356A (en) | Real-time automatic capacity expansion method and system for logical mirror image | |
| CN114465937A (en) | Network card testing method, device, server, medium, and computer program product | |
| CN111147226B (en) | Data storage method, device and storage medium | |
| CN116361153A (en) | Method and device for testing firmware codes, electronic equipment and storage medium | |
| CN113342500B (en) | Task execution method, device, equipment and storage medium | |
| CN106790521B (en) | System and method for distributed networking by using node equipment based on FTP | |
| CN112084006B (en) | Mirror image package processing method and device and electronic equipment | |
| CN111651189A (en) | Product delivery method and device of continuous integration system and electronic equipment | |
| CN116319837B (en) | File synchronization method, device and equipment supporting multiple protocols and storage medium | |
| CN111614743B (en) | File detection method and device, computer equipment and storage medium | |
| CN114237754B (en) | Data loading method and device, electronic equipment and storage medium | |
| CN117370038A (en) | Method, storage medium and server for Redis data mock | |
| CN118113552A (en) | Fault detection method and server |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |