CN113986119A - Simplified data reading method and system for accelerating cloud migration speed - Google Patents
Simplified data reading method and system for accelerating cloud migration speed Download PDFInfo
- Publication number
- CN113986119A CN113986119A CN202111162949.8A CN202111162949A CN113986119A CN 113986119 A CN113986119 A CN 113986119A CN 202111162949 A CN202111162949 A CN 202111162949A CN 113986119 A CN113986119 A CN 113986119A
- Authority
- CN
- China
- Prior art keywords
- migration
- data
- reading
- target
- cloud
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Withdrawn
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0602—Interfaces specially adapted for storage systems specifically adapted to achieve a particular effect
- G06F3/061—Improving I/O performance
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0628—Interfaces specially adapted for storage systems making use of a particular technique
- G06F3/0646—Horizontal data movement in storage systems, i.e. moving data in between storage devices or systems
- G06F3/0647—Migration mechanisms
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0668—Interfaces specially adapted for storage systems adopting a particular infrastructure
- G06F3/067—Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS]
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Human Computer Interaction (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The invention discloses a simplified data reading method and a system for accelerating cloud migration speed, wherein the method comprises the following steps: step S1: migration environment configuration and detection; step S2: configuration and monitoring of migration tasks; step S3: compact data reading and transmission. The invention can analyze the data block of the hard disk file system in the machine to be migrated of the user, quickly and completely read the effective data in the data block, and automatically skip the empty data block. The method has the advantages that the fast reading speed of the storage level according to the block index data is realized, the small data volume of the file level according to the effective data block reading is realized, and the migration speed can be greatly improved in the cloud migration design method and the cloud migration design system.
Description
Technical Field
The invention relates to the technical field of computer networks, in particular to a method and a system for reading simplified data for accelerating cloud migration speed.
Background
With the business construction of the information system, the enterprise data center is provided with a plurality of physical servers, which mainly comprise a database system, a business system, a network security system and the like. Due to the requirements of site change of the machine room, upgrading of a service system and the like, relocation work needs to be carried out on a machine room server.
From 2008 to the present, cloud computing is generally accepted by enterprises and organizations due to the characteristics of elasticity, flexibility, safety, low cost and the like, and the rapid development from concepts to falling to the ground and blooming is realized. For enterprises, the cloud-up advantages are obvious, such as cost reduction, rapid market entry, efficiency improvement and the like; for government departments, whether online service systems of all levels of governments are uniformly migrated to a cloud platform or local governments help enterprises in governed areas to cloud, it is common knowledge that the cloud is normal and the cloud is not exceptional. However, even with the profound knowledge of the "trend that cloud computing has become irreversible as IT infrastructure," going "to the cloud is still frustrating and difficult for many users.
The migration of mass data is a complex and systematic project, and not only data center applications, service systems and various data need to be migrated to a cloud platform, but also a plurality of challenges exist during the migration: the production environment is complex and diversified; the requirement on bandwidth is high; migration time cannot be guaranteed; technical professional development and operation and maintenance personnel lack. Particularly, when a large number of servers are migrated to the cloud, because the data size is large and the file system formats are various, if the migration data contains both valid data blocks and invalid data blocks according to the mode of migrating the storage device, the migration data size is too large; if the file migration mode is adopted, the index of the massive files can cause the influence of slow data reading speed and the like. There is a need for a method and system that can read only valid data and increase the speed of reading data.
Disclosure of Invention
In view of the above, the first aspect of the present invention is to provide a method for reading reduced data for accelerating cloud migration speed; it is an object of a second aspect of the invention to provide a system for streamlining data reading for speeding up cloud migration.
The purpose of the first aspect of the invention is realized by the following technical scheme:
the simplified data reading method for accelerating cloud migration speed comprises the following steps:
step S1: migration environment configuration and detection
Preparing a machine environment to be migrated and migrated, installing a migration client in the machine to be migrated, installing a migration target client in a migration target server, installing a management server on a third-party server, and judging whether the environment of the target server has a migration condition or not;
step S2: migration task configuration and monitoring
Starting a management platform, and configuring migration task parameters for the paired migration and the platform to be migrated;
step S3: reduced data reading and transmission
And after the migration task is started, reading the effective data blocks according to the sequence of the file system, and sending the effective data blocks to a migration target environment through a network.
Specifically, in step S1, it is determined whether the environment of the target server has the conditions for migration, including target disk size, operating system matching, and target drive matching, where the operating systems involved in the operating system matching include WINDOWS system, LINUX system, and macOS system.
Specifically, in step S2, the configuration of the parameters includes the disk configuration of the migration target, the selection of the migration target server, the selection of the migration end condition, and the selection of the cold migration and the hot migration.
In particular, in the step S3, it is necessary to determine in advance the hard disk size of the migration source server, the partition information, and the file system format (e.g., NTFS, FAT32, EXT3, EXT4, etc.) in each partition, and such file system must be identifiable for the method and system, and already include the method of parsing, jumping, reading, etc. of the data format of such file system.
The second aspect of the invention is realized by the following technical scheme:
a reduced data reading system for accelerating cloud migration speed comprises a server, a memory, a processor and a computer program which is stored on the memory and can run on the processor, wherein the processor executes the computer program to realize the method.
The invention has the beneficial effects that:
(1) simplified reading data non-stop service
Compared with the traditional migration mode or manual data migration, the time of user service interruption is uncontrollable, the invention adopts a unique synchronization and verification algorithm to support online incremental thermal migration, firstly migrates the whole disk of data in full, and then migrates new incremental data generated in service operation without interruption, and easily realizes that the service does not need to be stopped when the source end data is read.
(2) Streamlined data read greatly improves migration efficiency
The invention can analyze the data block of the hard disk file system in the machine to be migrated of the user, quickly and completely read the effective data in the data block, and automatically skip the empty data block. The method has the advantages that the fast reading speed of the storage level according to the block index data is realized, the small data volume of the file level according to the effective data block reading is realized, and the migration speed can be greatly improved in the cloud migration design method and the cloud migration design system.
(3) Streamlined data read improving migration success rate
The simplified data reading technology can completely read all effective data blocks in the disk at one time according to the data reading capacity of the storage level block data, contains all disk guide information, partition information, operating system data, application program data, production data and the like, can completely migrate all data of a source host to a target host according to the data distribution condition of the source disk, and greatly improves the migration success rate. .
(4) Cloud platform seamless compatibility
The cloud platform and the virtualization platform which are compatible with domestic and foreign mainstream are seamless, the file system which is compatible with domestic and foreign mainstream is compatible, the special requirements of the service carried on the source host can be successfully avoided, one migration client meets the requirements of any service type host, and the cloud migration compatibility is improved.
Additional advantages, objects, and features of the invention will be set forth in part in the description which follows and in part will become apparent to those having ordinary skill in the art upon examination of the following or may be learned from practice of the invention. The objectives and other advantages of the present invention will be realized and attained by the structure particularly pointed out in the written description and claims hereof as well as the appended drawings.
Drawings
In order to make the objects, technical solutions and advantages of the present invention more apparent, the present invention will be further described in detail with reference to the accompanying drawings, in which:
FIG. 1 is a schematic structural view of the present invention;
Detailed Description
Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings. It should be understood that the preferred embodiments are illustrative of the invention only and are not limiting upon the scope of the invention.
As shown in the figure, the simplified data reading method for accelerating the cloud migration speed includes the following steps:
step S1: migration environment configuration and detection
Preparing a machine environment to be migrated and migrated, installing a migration client in the machine to be migrated, installing a migration target client in a migration target server, installing a management server on a third-party server, and judging whether the environment of the target server has a migration condition or not; the conditions for judging whether the environment of the target server has migration include target disk size, operating system matching and target drive matching. The operating systems involved in operating system matching include WINDOWS systems, LINUX systems, and macOS systems.
Step S2: migration task configuration and monitoring
Starting a management platform, and configuring migration task parameters for the paired migration and the platform to be migrated; in this embodiment, the configuration of the parameter includes a disk configuration of a migration target, a selection of a migration target server, a selection of a migration end condition, and a selection of a cold migration and a hot migration.
Step S3: reduced data reading and transmission
Firstly, judging hard disk partition information in a machine to be migrated, and judging a file system format in each partition. And then, according to a file system data block distribution control table, reading the effective simplified data blocks according to the data block indexes, and skipping blank or invalid data. And the read simplified data blocks are data packets and are sent to the migration target environment through the network.
The method of the invention can read the effective data in the hard disk of the machine to be migrated of the user according to the file system data block index, automatically skip the invalid or blank data block, and migrate the data block to the target environment. All data are read without service interruption, and the service of the user is minimally influenced. The migration mode is mainly a point-to-point migration mode, the machine to be migrated directly falls into the data disk of the target machine, and data is only transmitted through a network and is not written into the data disk of a third-party machine.
And the migration task is established in a migration management server WEB management center, and the configuration and monitoring of the migration task are supported. The migration source end is registered to the management center through the installation of the migration client, the migration target end is registered to the management center through the starting of the migration target end mirror image, and the migration source end and the migration target end are respectively selected to generate migration tasks. Meanwhile, the task strategy can be configured, and the task strategy comprises migration data encryption, compression, speed limit and the like. In the migration process, client information monitoring including disk and connection state detection, local environment detection and drive information detection can be performed. The migration task console performs centralized management and control on current task state display, current task operation and incremental migration. The migration failure process may repair the machine that failed the migration again.
The system for reading the simplified data for accelerating the cloud migration speed is based on a general PC server, is provided with a large-capacity hard disk and a high-speed network card, can read the whole machine data (including all data such as a partition format, an operating system, an application program, a database, production data and the like) of a source environment, can be independently stored in equipment and transported offline, can be released in a specified target end environment (a physical environment or a cloud environment) to realize the cross-network migration, the computer room relocation and the cloud migration of the whole machine data. The invention can be applied to the following scenarios:
1. migrating physical machines of different brands to physical machines;
2. migrating a physical machine to a virtual machine;
3. cross-brand, cross-underlying architecture virtualization inter-migration;
4. migration from the physical machine and the virtual machine to the cloud platform;
5. mutual migration among the cross-public cloud manufacturers;
6. migrating a public cloud to a physical machine, a virtual machine private cloud and the like;
7. inter-network private migration between private networks.
It should be recognized that embodiments of the present invention can be realized and implemented by computer hardware, a combination of hardware and software, or by computer instructions stored in a non-transitory computer readable memory. The methods may be implemented in a computer program using standard programming techniques, including a non-transitory computer-readable storage medium configured with the computer program, where the storage medium so configured causes a computer to operate in a specific and predefined manner, according to the methods and figures described in the detailed description. Each program may be implemented in a high level procedural or object oriented programming language to communicate with a computer system. However, the program(s) can be implemented in assembly or machine language, if desired. In any case, the language may be a compiled or interpreted language. Furthermore, the program can be run on a programmed application specific integrated circuit for this purpose.
Further, the operations of processes described herein can be performed in any suitable order unless otherwise indicated herein or otherwise clearly contradicted by context. The processes described herein (or variations and/or combinations thereof) may be performed under the control of one or more computer systems configured with executable instructions, and may be implemented as code (e.g., executable instructions, one or more computer programs, or one or more applications) collectively executed on one or more processors, by hardware, or combinations thereof. The computer program includes a plurality of instructions executable by one or more processors.
Further, the method may be implemented in any type of computing platform operatively connected to a suitable interface, including but not limited to a personal computer, mini computer, mainframe, workstation, networked or distributed computing environment, separate or integrated computer platform, or in communication with a charged particle tool or other imaging device, and the like. Aspects of the invention may be embodied in machine-readable code stored on a non-transitory storage medium or device, whether removable or integrated into a computing platform, such as a hard disk, optically read and/or write storage medium, RAM, ROM, or the like, such that it may be read by a programmable computer, which when read by the storage medium or device, is operative to configure and operate the computer to perform the procedures described herein. Further, the machine-readable code, or portions thereof, may be transmitted over a wired or wireless network. The invention described herein includes these and other different types of non-transitory computer-readable storage media when such media include instructions or programs that implement the steps described above in conjunction with a microprocessor or other data processor. The present invention also includes a computer when programmed according to the methods and techniques described herein.
Finally, the above embodiments are only intended to illustrate the technical solutions of the present invention and not to limit the present invention, and although the present invention has been described in detail with reference to the preferred embodiments, it will be understood by those skilled in the art that modifications or equivalent substitutions may be made on the technical solutions of the present invention without departing from the spirit and scope of the technical solutions, and all of them should be covered by the claims of the present invention.
Claims (8)
1. A simplified data reading method for accelerating cloud migration speed is characterized in that: the method comprises the following steps:
step S1: migration environment configuration and detection
Preparing a machine environment to be migrated and migrated, installing a migration client in the machine to be migrated, starting a migration target mirror image in the migration target environment, installing a management server on a third-party server, and judging whether the environment of the target server has a migration condition or not;
step S2: migration task configuration and monitoring
Starting a management platform, and configuring migration task parameters for the paired migration and the platform to be migrated;
step S3: reduced data reading and transmission
And after the migration task is started, reading the effective simplified data blocks according to the sequence of the file system, and sending the effective simplified data blocks to a migration target environment through a network.
2. The method and system for reading reduced data for accelerating cloud migration according to claim 1, wherein: in step S1, it is determined whether the environment of the target server has migration conditions including target disk size, operating system matching, and target drive matching.
3. The method for implementing reduced data reading for accelerating cloud migration speed according to claim 1, wherein: in step S2, the configuration of the parameters includes the disk configuration of the migration target.
4. The method for reading reduced data for accelerating cloud migration according to claim 3, wherein: the configuration of the parameters further comprises the selection of a migration target server, the selection of a migration ending condition and the selection of cold migration and hot migration.
5. The method for reading reduced data for accelerating cloud migration according to claim 1, wherein: in step S3, the hard disk partition information in the machine to be migrated is first determined, and the file system format is determined for each partition. And then, according to a file system data block distribution control table, reading the effective simplified data blocks according to the data block indexes, and skipping blank or invalid data.
6. The method for reading reduced data for accelerating cloud migration according to claim 5, wherein: and the read simplified data blocks are data packets and are sent to the migration target environment through the network.
7. The method for reading reduced data for accelerating cloud migration according to claim 2, wherein: the operating systems involved in operating system matching include WINDOWS systems, LINUX systems, and macOS systems.
8. A system for facilitating thin data reading for cloud migration speed, comprising a server, a memory, a processor, and a computer program stored on the memory and capable of running on the processor, wherein: the processor, when executing the computer program, implements the method of any of claims 1-7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111162949.8A CN113986119A (en) | 2021-09-30 | 2021-09-30 | Simplified data reading method and system for accelerating cloud migration speed |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111162949.8A CN113986119A (en) | 2021-09-30 | 2021-09-30 | Simplified data reading method and system for accelerating cloud migration speed |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN113986119A true CN113986119A (en) | 2022-01-28 |
Family
ID=79737481
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202111162949.8A Withdrawn CN113986119A (en) | 2021-09-30 | 2021-09-30 | Simplified data reading method and system for accelerating cloud migration speed |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113986119A (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114489517A (en) * | 2022-02-14 | 2022-05-13 | 苏州浪潮智能科技有限公司 | Offline accelerated migration method, system, equipment and storage medium |
| CN116932464A (en) * | 2023-09-14 | 2023-10-24 | 中移(苏州)软件技术有限公司 | Data processing method, device, electronic equipment and storage medium |
-
2021
- 2021-09-30 CN CN202111162949.8A patent/CN113986119A/en not_active Withdrawn
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114489517A (en) * | 2022-02-14 | 2022-05-13 | 苏州浪潮智能科技有限公司 | Offline accelerated migration method, system, equipment and storage medium |
| CN114489517B (en) * | 2022-02-14 | 2023-09-08 | 苏州浪潮智能科技有限公司 | Offline acceleration migration method, system, equipment and storage medium |
| CN116932464A (en) * | 2023-09-14 | 2023-10-24 | 中移(苏州)软件技术有限公司 | Data processing method, device, electronic equipment and storage medium |
| CN116932464B (en) * | 2023-09-14 | 2023-11-28 | 中移(苏州)软件技术有限公司 | Data processing method, device, electronic equipment and storage medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US10613788B2 (en) | Data migration between cloud storage systems | |
| EP2972746B1 (en) | Storage unit selection for virtualized storage units | |
| US8429645B2 (en) | Method for optimizing migration of software applications to address needs | |
| US20180367468A1 (en) | Remote Service for Executing Resource Allocation Analyses for Distributed Computer Systems | |
| US9128765B2 (en) | Assigning restored virtual machine based on past application usage of requesting user | |
| CN113986119A (en) | Simplified data reading method and system for accelerating cloud migration speed | |
| US9571415B2 (en) | System and method for performing customized resource allocation analyses for distributed computer systems | |
| CN104360892B (en) | Create the system and method for virtual machine | |
| CN109510856A (en) | By the target device in storage area network to the mark of alternately Major Members port | |
| KR101357135B1 (en) | Apparatus for Collecting Log Information | |
| US20140156851A1 (en) | Server facility, method for providing copying service for system of server facility, and copying program for system of sever facility | |
| WO2021190659A1 (en) | System data acquisition method and apparatus, and medium and electronic device | |
| CN111124615A (en) | Virtual machine migration method, device, equipment and computer readable storage medium | |
| GB2518894A (en) | A method and a system for operating programs on a computer cluster | |
| WO2024021703A1 (en) | Server control method, server, and storage medium | |
| CN114362983A (en) | Firewall policy management method and device, computer equipment and storage medium | |
| CN113312656B (en) | Data rotation method, device, equipment and system | |
| CN113489754A (en) | Method and system for realizing cross-network migration of mass complete machine data | |
| US11537488B2 (en) | Portable power management profile for an information handling system | |
| CN105162833A (en) | Client computer management system and method applied to diskless workstation | |
| US20210073676A1 (en) | Model improvement support system | |
| EP3316518A1 (en) | Method and device for upgrading virtual network element, and computer storage medium | |
| US10325096B1 (en) | Modifying a portion of a read-only file system | |
| CN105790975A (en) | Service processing operation execution method and device | |
| CN109189424A (en) | Firmware upgrading method and device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| WW01 | Invention patent application withdrawn after publication |
Application publication date: 20220128 |
|
| WW01 | Invention patent application withdrawn after publication |