CN112491953B - Implementation method, system, equipment and medium supporting continuous transmission of cloud platform mirror image data - Google Patents
Implementation method, system, equipment and medium supporting continuous transmission of cloud platform mirror image data Download PDFInfo
- Publication number
- CN112491953B CN112491953B CN202011129915.4A CN202011129915A CN112491953B CN 112491953 B CN112491953 B CN 112491953B CN 202011129915 A CN202011129915 A CN 202011129915A CN 112491953 B CN112491953 B CN 112491953B
- Authority
- CN
- China
- Prior art keywords
- mirror image
- image data
- cloud platform
- data
- offset
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 230000005540 biological transmission Effects 0.000 title claims abstract description 80
- 238000000034 method Methods 0.000 title claims abstract description 35
- 230000000903 blocking effect Effects 0.000 claims description 9
- 238000004140 cleaning Methods 0.000 claims description 6
- 210000004907 gland Anatomy 0.000 claims description 5
- 238000010586 diagram Methods 0.000 description 4
- 238000012986 modification Methods 0.000 description 3
- 230000004048 modification Effects 0.000 description 3
- 230000003287 optical effect Effects 0.000 description 3
- 238000004590 computer program Methods 0.000 description 2
- 230000000694 effects Effects 0.000 description 2
- 239000013307 optical fiber Substances 0.000 description 2
- 230000000644 propagated effect Effects 0.000 description 2
- 241000109539 Conchita Species 0.000 description 1
- 238000003491 array Methods 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000002093 peripheral effect Effects 0.000 description 1
- 239000004065 semiconductor Substances 0.000 description 1
- 239000002699 waste material Substances 0.000 description 1
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
- H04L67/10—Protocols in which an application is distributed across nodes in the network
- H04L67/1095—Replication or mirroring of data, e.g. scheduling or transport for data synchronisation between network nodes
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
- H04L67/06—Protocols specially adapted for file transfer, e.g. file transfer protocol [FTP]
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
- H04L67/10—Protocols in which an application is distributed across nodes in the network
- H04L67/104—Peer-to-peer [P2P] networks
- H04L67/1074—Peer-to-peer [P2P] networks for supporting data block transmission mechanisms
- H04L67/1078—Resource delivery mechanisms
- H04L67/1085—Resource delivery mechanisms involving dynamic management of active down- or uploading connections
Landscapes
- Engineering & Computer Science (AREA)
- Computer Networks & Wireless Communication (AREA)
- Signal Processing (AREA)
- Information Transfer Between Computers (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The invention provides a method, a system, equipment and a storage medium for realizing the continuous transmission of mirror image data of a cloud platform, and belongs to the technical field of data transmission. The method comprises the steps that a user uploads mirror image data again and issues a task to a cloud platform; the cloud platform inquires in the work service of the cloud platform according to the ID of the mirror image data uploaded last time by the user, and judges whether the currently uploaded mirror image data is data continuous transmission or not; utilizing the pane service to inquire and determine a continuous transmission point of the currently uploaded mirror image data; the cloud platform receives mirror image data uploaded by a user, and calculates an MD5 value and a mirror image offset of the mirror image data; and storing the mirror image data to a glance service back end, and deleting the mirror image offset and the temporary MD5 value of the mirror image data. The implementation system comprises modules corresponding to the steps of the method.
Description
Technical Field
The invention provides a method, a system, equipment and a storage medium for realizing the continuous transmission of mirror image data of a cloud platform, and belongs to the technical field of data transmission.
Background
When a user creates a cloud host by using a cloud platform daily, a system using a mirror image as the cloud host is often needed, such as windows, centros or Linux. Before a user uses a mirror image file to create a cloud host, an administrator needs to upload a mirror image to the storage back end of a cloud platform and manage mirror image data through a work service, the mirror image data files of general systems are large, the mirror image data uploading failure can be caused by poor networks or power failure of platform services and the like in the data uploading process, the administrator needs to upload the mirror image data again, the same problem can occur when the mirror image file uploading failure is caused by the network, and the cloud platform user is extremely unfriendly to the user experience when the mirror image data uploading failure is repeated.
Disclosure of Invention
The invention provides a realization method, a system, equipment and a storage medium for supporting the continuous transmission of mirror image data of a cloud platform, which support a user to continuously upload data which is transmitted to a work rear end last time under the condition that the efficiency of data transmission of the cloud platform is low even the transmission is disconnected due to bad network, and avoid the problems of time waste and unfriendly experience of the user caused by repeatedly uploading mirror image data. The technical scheme is as follows:
an implementation method for supporting continuous transmission of mirror image data of a cloud platform comprises the following steps:
the user uploads the mirror image data again and issues a task to the cloud platform;
the cloud platform inquires in the work service of the cloud platform according to the ID of the mirror image data uploaded last time by the user, and judges whether the currently uploaded mirror image data is data continuous transmission or not; if the judgment result is that the currently uploaded mirror image data is data continuous transmission, executing a continuous transmission point determining step; if the judgment result is that the currently uploaded mirror image data is not data continuous transmission, directly executing the step of receiving the data;
determining a continuous transmission point: determining a continuous transmission point of the currently uploaded mirror image data by using the work service query;
receiving data: the cloud platform receives mirror image data uploaded by a user, and calculates an MD5 value and a mirror image offset of the mirror image data;
and storing the mirror image data to a liance service back end, and deleting the mirror image offset and the temporary MD5 value of the mirror image data.
Further, determining a continuous transmission point of the currently uploaded mirror image data by using the work service query, comprising:
acquiring mirror image offset uploaded by mirror image data last time and MD5 value of mirror image data after uploading by using the work service query;
determining a continuous transmission point of currently uploaded mirror image data through the mirror image offset and the MD5 value;
and the cloud platform uploads the mirror image data to the company service according to the continuous transmission point.
Further, the cloud platform receives the mirror image data uploaded by the user, and calculates an MD5 value and a mirror image offset of the mirror image data, including:
the cloud platform receives mirror image data uploaded by a user by using a work service;
the received mirror image data is read in a blocking mode to obtain a plurality of mirror image blocks, and an MD5 value of each mirror image block is calculated;
writing the data corresponding to each mirror image block into the storage back end of the pane service, and calculating the offset value of the mirror image data according to the size of the data written into the mirror image data and the size of the data written into the mirror image block at the storage back end, wherein the offset value is the mirror image offset;
and the gland service back end updates the MD5 value and the mirror offset of the mirror data and stores the updated MD5 value and the mirror offset into the database.
Further, storing the mirror image data to a glance service backend, and deleting the mirror image offset and the temporary MD5 value of the mirror image data, including:
writing each piece of mirror image data into the storage back end of the work service;
uploading mirror image data until the uploading is finished, and completely writing the mirror image data into the storage back end of the work service;
and clearing the mirror image offset and the temporary MD5 value of the mirror image data, and finishing uploading the current mirror image data to the cloud platform.
An implementation system supporting continuous transmission of mirror image data of a cloud platform comprises:
the cloud platform interface module is used for uploading mirror image data again for the user and issuing a task to the cloud platform to provide an interface;
the query judging module is used for the cloud platform to query in the work service of the cloud platform according to the ID of the mirror image data uploaded last time by the user and judging whether the currently uploaded mirror image data is continuously transmitted; if the judgment result is that the currently uploaded mirror image data is data continuous transmission, starting a continuous transmission point determining module; if the judgment result is that the currently uploaded mirror image data is not data continuous transmission, starting a data receiving module;
the continuous transmission point determining module is used for determining a continuous transmission point of the currently uploaded mirror image data by utilizing the liance service query;
the data receiving module is used for receiving mirror image data uploaded by a user by the cloud platform and calculating an MD5 value and a mirror image offset of the mirror image data;
and the storage clearing module is used for storing the mirror image data to a glance service back end and deleting the mirror image offset and the temporary MD5 value of the mirror image data.
Further, the continuation point determining module includes:
the acquisition module is used for acquiring the mirror image offset uploaded by the mirror image data at the last time and the MD5 value of the mirror image data after uploading by utilizing the work service query;
the continuous transmission point acquisition module is used for determining a continuous transmission point of the currently uploaded mirror image data through the mirror image offset and the MD5 value;
and the uploading data module is used for controlling the cloud platform to upload the mirror image data to the company service according to the continuous transmission point.
Further, the data receiving module comprises:
the receiving module is used for receiving mirror image data uploaded by a user by the cloud platform through a work service;
the blocking module is used for reading the received mirror image data in a blocking mode to obtain a plurality of mirror image blocks and calculating an MD5 value of each mirror image block;
the offset acquisition module is used for writing the data corresponding to each mirror image block into the storage back end of the work service, and calculating the offset value of the mirror image data according to the size of the data written into the mirror image block and the size of the data written into the storage back end mirror image block at this time, wherein the offset value is the mirror image offset;
and the storage module is used for updating the MD5 value and the mirror image offset of the mirror image data by the rear end of the work service and storing the updated MD5 value and the mirror image offset into a database.
Further, the memory clearing module comprises:
the back-end storage module is used for uploading mirror image data until the uploading is finished, and the mirror image data is completely written into the storage back end of the work service;
and the cleaning module is used for cleaning the mirror image offset and the temporary MD5 value of the mirror image data and finishing the uploading of the current mirror image data to the cloud platform.
An apparatus for supporting cloud platform mirror image data continuous transmission, the apparatus comprising:
a processor;
a memory configured to store processor-executable instructions;
wherein the processor is configured to: when executed, the method realizes the realization method of any one of the above items.
A non-transitory computer-readable storage medium, comprising: the storage medium stores computer-executable instructions which, when executed by the processor, implement any of the implementation methods described above.
The invention has the beneficial effects that:
the invention provides a realization method, a system, equipment and a storage medium for supporting the mirror image data continuous transmission of a cloud platform, aiming at the situation that a user does not need to upload the whole mirror image data again under the condition of daily mirror image data uploading failure, the mirror image data uploaded to the cloud platform last time can be continuously uploaded, and meanwhile, the mirror image data of the user can be completely uploaded to the cloud platform. The implementation method, the system, the equipment and the storage medium for supporting the mirror image data continuous transmission of the cloud platform can effectively avoid the problem that the user needs to repeatedly upload the whole amount of mirror image data after the mirror image is unsuccessfully uploaded due to network and other reasons when the user uploads the mirror image data, can save the mirror image data uploaded to the cloud platform by the user to a certain extent, reduce the network pressure between the user network and the cloud platform, and particularly can greatly relieve the network pressure under the condition of poor network.
Drawings
FIG. 1 is a flow chart of the method of the present invention;
FIG. 2 is a first schematic diagram of a method of the present invention;
FIG. 3 is a second schematic diagram of the method of the present invention;
FIG. 4 is a system block diagram of the system of the invention;
FIG. 5 is a schematic view of the apparatus of the present invention;
FIG. 6 is a schematic diagram of a storage medium according to the present invention.
Detailed Description
The preferred embodiments of the present invention will be described in conjunction with the accompanying drawings, and it should be understood that they are presented herein only to illustrate and explain the present invention and not to limit the present invention.
The invention provides a realization method, a system, equipment and a storage medium for supporting the mirror image data continuous transmission of a cloud platform, which support a user to continuously transmit the mirror image data which is not uploaded last time to the cloud platform, and prevent the user from repeatedly uploading the mirror image data due to service disconnection when the user uploads part of the mirror image to the rear end of a gland.
The embodiment of the invention provides an implementation method for supporting continuous transmission of cloud platform mirror image data, as shown in fig. 1 to 3, when a user uploads a mirror image to a cloud platform, a cloud platform administrator needs to upload a system and other mirror image files to the cloud platform so as to create a cloud host for the platform user to use, and in the process of uploading the mirror image file to the cloud platform, because the mirror image file is large, the communication quality between a platform user network and a platform internal network is poor, so that data transmission is disconnected when a part of the mirror image file is uploaded, under the condition, the implementation method is used for carrying out mirror image data transmission, and specifically comprises the following steps:
the implementation method comprises the following steps:
s1, the user uploads the mirror image data again and issues a task to the cloud platform;
s2, the cloud platform inquires in the work service of the cloud platform according to the ID of the mirror image data uploaded last time by the user, and judges whether the currently uploaded mirror image data is continuously transmitted; if the judgment result is that the currently uploaded mirror image data is data continuous transmission, executing step S3; if the judgment result is that the currently uploaded mirror image data is not the data continuous transmission, directly executing the step S4;
s3, determining a continuous transmission point of the currently uploaded mirror image data by using the work service query;
s4, the cloud platform receives mirror image data uploaded by a user, and calculates an MD5 value and a mirror image offset of the mirror image data;
and S5, storing the mirror image data to a glance service back end, and deleting the mirror image offset and the temporary MD5 value of the mirror image data.
The step S3 of determining a continuous transmission point of the currently uploaded mirror image data by using the work service query includes:
s301, inquiring mirror image properties information from the liance service by using a mirror image ID, and acquiring an offset value uploaded by mirror image data at the last time and an MD5 value of the mirror image data after uploading; wherein, the offset value is the mirror image offset;
s302, determining a continuous transmission point of the currently uploaded mirror image data through the mirror image offset and the MD5 value;
and S303, the cloud platform uploads the mirror image data to the work service according to the continuous transmission point.
S4, the cloud platform receives the mirror image data uploaded by the user, and calculates the MD5 value and the mirror image offset of the mirror image data, including:
s401, the cloud platform receives mirror image data uploaded by a user by means of a work service;
s402, reading the received mirror image data in a blocking mode to obtain a plurality of mirror image blocks, and calculating an MD5 value of each mirror image block; the size of the data obtained by the mirror image block can be 8M;
s403, writing the data corresponding to each mirror image block into the storage back end of the glance service, and calculating the offset value of the mirror image data according to the size of the data written into the mirror image data and the size of the data written into the storage back end mirror image block at this time, wherein the offset value is the mirror image offset;
s404, the liance service back end updates the MD5 value and the mirror offset of the mirror data, and stores the updated MD5 value and the mirror offset into a database.
S5, storing the mirror image data to the rear end of the weather service, and deleting the mirror image offset and the temporary MD5 value of the mirror image data, including:
s501, writing each piece of mirror image data into a storage back end of a work service;
s502, uploading mirror image data until uploading is completed, and completely writing the mirror image data into a storage back end of a work service;
s503, clearing the mirror image offset and the temporary MD5 value of the mirror image data, and finishing uploading of the current mirror image data to the cloud platform.
The working principle of the technical scheme is as follows: the implementation method for supporting the mirror image data continuous transmission of the cloud platform provided by the invention supports a user to continuously upload the mirror image data when the mirror image uploading task is interrupted due to a certain reason when the cloud platform user uploads a part of the mirror image data; then the cloud platform confirms the mirror image data continuous transmission point by inquiring the offset position of the last mirror image data transmission and the data MD5 value uploaded to the storage back end; then, uploading mirror images to a storage back end by using a cloud platform company service, writing the residual mirror image data into the storage back end, calculating the offset and MD5 value of the uploaded mirror images, and storing the offset and MD5 value in a shared storage (such as a shared storage back end for multi-instance company service query, such as NFS or a database); and finally, after the mirror image data are completely uploaded to the cloud platform, the offset of the mirror image data and the temporary MD5 value are cleared, and the uploading of the mirror image data of the cloud platform is completed.
The effect of the above technical scheme is as follows: aiming at the situation that the user does not need to upload the whole mirror image data again under the condition that the mirror image data is unsuccessfully uploaded by the user in daily life, the mirror image data uploaded to the cloud platform last time can be continuously uploaded, and meanwhile, the mirror image data of the user is completely uploaded to the cloud platform. The problem that the user needs to repeatedly upload the full amount of mirror image data after mirror image uploading fails due to network and other reasons when mirror image data are uploaded can be effectively solved, the situation that the user uploads the mirror image data to the cloud platform to reduce the network pressure in the user network and the cloud platform can be saved to a certain extent, and especially under the condition that the network is poor, the network pressure can be greatly relieved.
The embodiment of the invention provides an implementation system supporting continuous transmission of cloud platform mirror image data, as shown in fig. 4, the implementation system includes:
the cloud platform interface module is used for uploading mirror image data again for the user and issuing a task to the cloud platform to provide an interface;
the query judging module is used for the cloud platform to query in the work service of the cloud platform according to the ID of the mirror image data uploaded last time by the user and judging whether the currently uploaded mirror image data is continuously transmitted; if the judgment result is that the currently uploaded mirror image data is data continuous transmission, starting a continuous transmission point determining module; if the judgment result is that the currently uploaded mirror image data is not data continuous transmission, starting a data receiving module;
the continuous transmission point determining module is used for determining a continuous transmission point of the currently uploaded mirror image data by utilizing the liance service query;
the data receiving module is used for receiving mirror image data uploaded by a user by the cloud platform and calculating an MD5 value and a mirror image offset of the mirror image data;
and the storage clearing module is used for storing the mirror image data to a glance service back end and deleting the mirror image offset and the temporary MD5 value of the mirror image data.
Wherein, the continuous transmission point determining module comprises:
the acquisition module is used for acquiring the mirror image offset uploaded by the mirror image data at the last time and the MD5 value of the mirror image data after uploading by utilizing the work service query;
the continuous transmission point acquisition module is used for determining a continuous transmission point of the currently uploaded mirror image data through the mirror image offset and the MD5 value;
and the uploading data module is used for controlling the cloud platform to upload the mirror image data to the company service according to the continuous transmission point.
The data receiving module comprises:
the receiving module is used for receiving mirror image data uploaded by a user by the cloud platform through a company service;
the blocking module is used for reading the received mirror image data in a blocking mode to obtain a plurality of mirror image blocks and calculating an MD5 value of each mirror image block;
the offset acquisition module is used for writing the data corresponding to each mirror image block into the storage back end of the work service, and calculating the offset value of the mirror image data according to the size of the data written into the mirror image block and the size of the data written into the storage back end mirror image block at this time, wherein the offset value is the mirror image offset;
and the storage module is used for updating the MD5 value and the mirror image offset of the mirror image data by the rear end of the work service and storing the updated MD5 value and the mirror image offset into a database.
The memory clearing module comprises:
the back-end storage module is used for uploading the mirror image data until the uploading is finished, and completely writing the mirror image data into the storage back end of the weather service;
and the cleaning module is used for cleaning the mirror image offset and the temporary MD5 value of the mirror image data and finishing the uploading of the current mirror image data to the cloud platform.
The working principle of the technical scheme is as follows: a user issues a cloud platform mirror image creation request, a part of A1 of the mirror image A is stored in the rear end of the cloud platform due to the fact that the mirror image file is too large or the network environment is not good, and the rest part of the mirror image A2 is not uploaded to the storage rear end of the cloud platform (assuming that A is A1+ A2), and the user mirror image creation request is disconnected; the realization system supporting the mirror image data continuous transmission of the cloud platform controls the cloud platform to judge that the mirror image uploading is not finished according to the task state of the mirror image A, the network disconnection and the like, and an interface can prompt a user to continuously create the mirror image; the user clicks to continue to create the mirror image A; the cloud platform receives a request for continuously creating the mirror image A, and acquires an uploading offset _ latest of the mirror image A and an md5 value of a last uploaded mirror image data block from the work service. And the realization system controls the cloud platform to calculate a continuous transmission point of the mirror image data according to the offset and the md5 value of the mirror image block uploaded last time, and then the continuous transmission point starts to transmit the mirror image residual data to the angle. The wipe service receives the mirror data and updates the md5 value (obtained by hashing the mirror block data) and the mirror offset of offset _ latest + img _ block, which are newly written into the mirror a metadata of the storage back end. And (4) the gland confirms that the mirror image data is completely written into the storage back end of the cloud platform, and simultaneously, the last mirror image block md5 value and the offset in the mirror image A metadata are cleared.
The effect of the above technical scheme is as follows: aiming at the situation that the user does not need to upload the whole mirror image data again under the condition that the mirror image data is unsuccessfully uploaded by the user in daily life, the mirror image data uploaded to the cloud platform last time can be continuously uploaded, and meanwhile, the mirror image data of the user is completely uploaded to the cloud platform. The problem that the user needs to repeatedly upload the full amount of mirror image data after mirror image uploading fails due to network and other reasons when mirror image data are uploaded can be effectively solved, the situation that the user uploads the mirror image data to the cloud platform to reduce the network pressure in the user network and the cloud platform can be saved to a certain extent, and especially under the condition that the network is poor, the network pressure can be greatly relieved.
An apparatus for supporting cloud platform mirror image data continuous transmission, the apparatus comprising:
a processor;
a memory configured to store processor-executable instructions;
wherein the processor is configured to: when executed, the method realizes the realization method of any one of the above items. As shown in fig. 5. The device comprises at least one memory unit 11, at least one processor 12 and a bus 13 connecting the different platform systems.
The storage unit 11 may include readable media in the form of volatile storage units, such as a random access memory unit (RAM)111 and/or a cache memory unit 112, and may further include a read only memory unit (ROM) 113.
The storage unit 11 further stores a program product 2, and the program product 2 may be executed by the processor 12, so that the processor 12 executes the steps of the implementation method supporting the cloud platform mirror image data continuous transmission in this embodiment (as shown in fig. 1). The storage unit 11 may also include a program/utility 114 having a set (at least one) of program modules 115, such program modules including but not limited to: an operating system, one or more application programs, other program modules, and program data, each of which or some combination thereof may comprise an implementation of a network environment.
Accordingly, the processor 12 may execute the program product 2 described above, as well as execute the program/utility 114.
A device 1 that supports cloud platform mirrored data retransmission may also communicate with one or more external devices 14, such as a keyboard, pointing device, bluetooth device, etc., and may also communicate with one or more devices capable of interacting with the device 1, and/or any device (e.g., router, modem, etc.) that enables the device 1 to communicate with one or more other computing devices. Such communication may be via an input/output (I/O) interface 15. Also, the device 1 may communicate with one or more networks (e.g., a Local Area Network (LAN), a Wide Area Network (WAN), and/or a public network, such as the internet) via the network adapter 16. The network adapter 16 may communicate with other modules of the device 1 via the bus 13. It should be understood that although not shown in fig. 5, other hardware and/or software modules may be used in conjunction with the device 1, including but not limited to: microcode, device drivers, redundant processors, external disk drive arrays, RAID systems, tape drives, and data backup storage platforms, to name a few.
A non-transitory computer readable storage medium, as shown in fig. 6, comprising: the storage medium stores computer-executable instructions that, when executed by the processor, implement any of the methods described above. Specifically, referring to fig. 4, an embodiment of the present invention further provides a computer-readable storage medium for supporting cloud platform mirror image data retransmission, where the computer-readable storage medium is used to store a computer program, and when the computer program is executed, the steps of the implementation method for supporting cloud platform mirror image data retransmission in the embodiment of the present invention are implemented (as shown in fig. 1). Fig. 6 shows a program product 2 provided by the present embodiment for implementing the above method, which may employ a portable compact disc read only memory (CD-ROM) and include program codes, and may be run on a terminal device, such as a personal computer. However, the program product 2 of the present invention is not limited thereto, and in the present invention, the readable storage medium may be any tangible medium that can contain or store a program for use by or in connection with an instruction execution system, apparatus, or device. Program product 2 may employ any combination of one or more readable media. The readable medium may be a readable signal medium or a readable storage medium. A readable storage medium may be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the foregoing. More specific examples (a non-exhaustive list) of the readable storage medium include: an electrical connection having one or more wires, a portable disk, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.
A computer readable storage medium may include a propagated data signal with readable program code embodied therein, for example, in baseband or as part of a carrier wave. Such a propagated data signal may take many forms, including, but not limited to, electro-magnetic, optical, or any suitable combination thereof. A readable storage medium may also be any readable medium that is not a readable storage medium and that can communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device. Program code embodied on a readable storage medium may be transmitted using any appropriate medium, including but not limited to wireless, wireline, optical fiber cable, RF, etc., or any suitable combination of the foregoing. Program code for carrying out operations for aspects of the present invention may be written in any combination of one or more programming languages, including an object oriented programming language such as Java, C + + or the like and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code may execute entirely on the user's computing device, partly on the user's device, as a stand-alone software package, partly on the user's computing device and partly on a remote computing device, or entirely on the remote computing device or server. In the case of a remote computing device, the remote computing device may be connected to the user computing device through any kind of network, including a Local Area Network (LAN) or a Wide Area Network (WAN), or may be connected to an external computing device (e.g., through the internet using an internet service provider).
It will be apparent to those skilled in the art that various changes and modifications may be made in the present invention without departing from the spirit and scope of the invention. Thus, if such modifications and variations of the present invention fall within the scope of the claims of the present invention and their equivalents, the present invention is also intended to include such modifications and variations.
Claims (10)
1. An implementation method for supporting continuous transmission of mirror image data of a cloud platform is characterized by comprising the following steps:
the user uploads the mirror image data again and issues a task to the cloud platform;
the cloud platform inquires in the work service of the cloud platform according to the ID of the mirror image data uploaded last time by the user, and judges whether the currently uploaded mirror image data is data continuous transmission or not; if the judgment result is that the currently uploaded mirror image data is data continuous transmission, executing a continuous transmission point determining step; if the judgment result is that the currently uploaded mirror image data is not data continuous transmission, directly executing the step of receiving the data;
determining a continuous transmission point: determining a continuous transmission point of the currently uploaded mirror image data by using the work service query;
receiving data: the cloud platform receives mirror image data uploaded by a user, and calculates an MD5 value and a mirror image offset of the mirror image data;
and storing the mirror image data to a liance service back end, and deleting the mirror image offset and the temporary MD5 value of the mirror image data.
2. The method as claimed in claim 1, wherein determining a retransmission point of the currently uploaded mirror data using the liance service query comprises:
acquiring mirror image offset uploaded by mirror image data last time and MD5 value of mirror image data after uploading by using the work service query;
determining a continuous transmission point of currently uploaded mirror image data through the mirror image offset and the MD5 value;
and the cloud platform uploads the mirror image data to the company service according to the continuous transmission point.
3. The implementation method of claim 1, wherein the cloud platform receives the mirror image data uploaded by the user, and calculates an MD5 value and a mirror image offset of the mirror image data, including:
the cloud platform receives mirror image data uploaded by a user by using a work service;
the received mirror image data is read in a blocking mode to obtain a plurality of mirror image blocks, and an MD5 value of each mirror image block is calculated;
writing the data corresponding to each mirror image block into the storage back end of the work service, and calculating the offset value of the mirror image data according to the size of the data written into the mirror image data and the size of the data written into the storage back end mirror image block at this time, wherein the offset value is the mirror image offset;
and the gland service back end updates the MD5 value and the mirror offset of the mirror data and stores the updated MD5 value and the mirror offset into the database.
4. The method as claimed in claim 1, wherein storing the mirror data in a liance service backend, and deleting the mirror offset and the temporary MD5 value of the mirror data comprises:
writing each piece of mirror image data into the storage back end of the work service;
uploading mirror image data until the uploading is finished, and completely writing the mirror image data into the storage back end of the work service;
and clearing the mirror image offset and the temporary MD5 value of the mirror image data, and finishing uploading the current mirror image data to the cloud platform.
5. An implementation system supporting continuous transmission of mirror image data of a cloud platform is characterized by comprising:
the cloud platform interface module is used for uploading mirror image data again for the user and issuing a task to the cloud platform to provide an interface;
the query judging module is used for the cloud platform to query in the work service of the cloud platform according to the ID of the mirror image data uploaded last time by the user and judging whether the currently uploaded mirror image data is continuously transmitted; if the judgment result is that the currently uploaded mirror image data is data continuous transmission, starting a continuous transmission point determining module; if the judgment result is that the currently uploaded mirror image data is not the data continuous transmission, starting a data receiving module;
the continuous transmission point determining module is used for determining a continuous transmission point of the currently uploaded mirror image data by utilizing the pane service query;
the data receiving module is used for receiving mirror image data uploaded by a user by the cloud platform and calculating an MD5 value and a mirror image offset of the mirror image data;
and the storage clearing module is used for storing the mirror image data to the rear end of the gland service and deleting the mirror image offset and the temporary MD5 value of the mirror image data.
6. The system of claim 5, wherein the retransmission point determining module comprises:
the acquisition module is used for acquiring the mirror image offset uploaded by the mirror image data at the last time and the MD5 value of the mirror image data after uploading by utilizing the work service query;
the continuous transmission point acquisition module is used for determining a continuous transmission point of the currently uploaded mirror image data through the mirror image offset and the MD5 value;
and the uploading data module is used for controlling the cloud platform to upload the mirror image data to the company service according to the continuous transmission point.
7. The system of claim 5, wherein the means for receiving data comprises:
the receiving module is used for receiving mirror image data uploaded by a user by the cloud platform through a work service;
the blocking module is used for reading the received mirror image data in a blocking mode to obtain a plurality of mirror image blocks and calculating an MD5 value of each mirror image block;
the offset acquisition module is used for writing the data corresponding to each mirror image block into the storage back end of the weather service, and calculating the offset value of the mirror image data according to the size of the data written into the mirror image data and the size of the data written into the mirror image block at the storage back end, wherein the offset value is the mirror image offset;
and the storage module is used for updating the MD5 value and the mirror image offset of the mirror image data by the rear end of the work service and storing the updated MD5 value and the mirror image offset into a database.
8. The system of claim 5, wherein the memory clearing module comprises:
the back-end storage module is used for uploading mirror image data until the uploading is finished, and the mirror image data is completely written into the storage back end of the work service;
and the cleaning module is used for cleaning the mirror image offset and the temporary MD5 value of the mirror image data and finishing the uploading of the current mirror image data to the cloud platform.
9. An apparatus for supporting cloud platform mirror image data continuous transmission, the apparatus comprising:
a processor;
a memory configured to store executable instructions;
wherein the processor is configured to: when the executable instructions are executed, the implementation method of any one of claims 1-4 is realized.
10. A non-transitory computer-readable storage medium, comprising: the storage medium stores computer-executable instructions that, when executed by the processor, implement the method of any of claims 1-4.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011129915.4A CN112491953B (en) | 2020-10-21 | 2020-10-21 | Implementation method, system, equipment and medium supporting continuous transmission of cloud platform mirror image data |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011129915.4A CN112491953B (en) | 2020-10-21 | 2020-10-21 | Implementation method, system, equipment and medium supporting continuous transmission of cloud platform mirror image data |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN112491953A CN112491953A (en) | 2021-03-12 |
| CN112491953B true CN112491953B (en) | 2022-06-14 |
Family
ID=74927106
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202011129915.4A Active CN112491953B (en) | 2020-10-21 | 2020-10-21 | Implementation method, system, equipment and medium supporting continuous transmission of cloud platform mirror image data |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112491953B (en) |
Families Citing this family (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115277723B (en) * | 2022-07-19 | 2024-06-18 | 国能信控互联技术有限公司 | Breakpoint continuous transmission method and system of edge collection history module based on buffer event |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107888695A (en) * | 2017-11-23 | 2018-04-06 | 新华三云计算技术有限公司 | Cloud platform creates the method and cloud platform equipment of mirror image |
| CN109104451A (en) * | 2017-06-21 | 2018-12-28 | 阿里巴巴集团控股有限公司 | The pre-heating mean and node of the method for down loading and node of Docker mirror image, Docker mirror image |
| CN109783193A (en) * | 2018-12-27 | 2019-05-21 | 南京优速网络科技有限公司 | A kind of mirror image file management method that can be multiplexed in multiple domain cloud platform |
| CN110336883A (en) * | 2019-07-09 | 2019-10-15 | 深信服科技股份有限公司 | Mirror image data transmission method, device, equipment and computer readable storage medium |
| CN111221556A (en) * | 2020-01-09 | 2020-06-02 | 联陆智能交通科技(上海)有限公司 | Method and system for remotely upgrading RSU mirror image |
-
2020
- 2020-10-21 CN CN202011129915.4A patent/CN112491953B/en active Active
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109104451A (en) * | 2017-06-21 | 2018-12-28 | 阿里巴巴集团控股有限公司 | The pre-heating mean and node of the method for down loading and node of Docker mirror image, Docker mirror image |
| CN107888695A (en) * | 2017-11-23 | 2018-04-06 | 新华三云计算技术有限公司 | Cloud platform creates the method and cloud platform equipment of mirror image |
| CN109783193A (en) * | 2018-12-27 | 2019-05-21 | 南京优速网络科技有限公司 | A kind of mirror image file management method that can be multiplexed in multiple domain cloud platform |
| CN110336883A (en) * | 2019-07-09 | 2019-10-15 | 深信服科技股份有限公司 | Mirror image data transmission method, device, equipment and computer readable storage medium |
| CN111221556A (en) * | 2020-01-09 | 2020-06-02 | 联陆智能交通科技(上海)有限公司 | Method and system for remotely upgrading RSU mirror image |
Also Published As
| Publication number | Publication date |
|---|---|
| CN112491953A (en) | 2021-03-12 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN107122379B (en) | Collecting transaction data associated with a locally stored data file | |
| CN110891057B (en) | Protocol conversion method, device, equipment and storage medium | |
| WO2021051563A1 (en) | Target data archiving method and apparatus, and electronic device and computer non-volatile readable storage medium | |
| JP2016537703A (en) | Improving backup system performance | |
| CN111382123B (en) | File storage method, device, equipment and storage medium | |
| CN104252466A (en) | Stream computing processing method, equipment and system | |
| CN113760509B (en) | Timeout management method and device and electronic equipment | |
| CN112905537B (en) | File processing method and device, electronic equipment and storage medium | |
| CN111818145B (en) | File transmission method, device, system, equipment and storage medium | |
| US10802922B2 (en) | Accelerated deduplication block replication | |
| CN112491953B (en) | Implementation method, system, equipment and medium supporting continuous transmission of cloud platform mirror image data | |
| US10574751B2 (en) | Identifying data for deduplication in a network storage environment | |
| KR20210040864A (en) | File directory traversal method, apparatus, device, and medium | |
| KR20210042865A (en) | Data processing method and apparatus | |
| US9146816B2 (en) | Managing system image backup | |
| CN112925584A (en) | Scene-based file configuration method, device, storage medium, and program product | |
| US10402373B1 (en) | Filesystem redirection | |
| US11169714B1 (en) | Efficient file replication | |
| CN110290196B (en) | Content distribution method and device | |
| CN111309740A (en) | Data processing method, device, equipment and storage medium | |
| CN111770208A (en) | Control method and device for Internet of things equipment | |
| CN111124291A (en) | Data storage processing method and device of distributed storage system and electronic equipment | |
| CN113076175B (en) | Memory sharing method and device for virtual machine | |
| US11176097B2 (en) | Accelerated deduplication block replication | |
| CN113360689A (en) | Image retrieval system, method, related device and computer program product |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |