WO2021184992A1

WO2021184992A1 - Mirror image file uploading method, related device and computer storage medium

Info

Publication number: WO2021184992A1
Application number: PCT/CN2021/075236
Authority: WO
Inventors: 孙思东; 申思
Original assignee: 华为技术有限公司
Priority date: 2020-03-16
Filing date: 2021-02-04
Publication date: 2021-09-23
Also published as: CN113411363A

Abstract

Provided in the present application are a mirror image file uploading method, a related device and a computer storage medium. The method comprises: acquiring a target mirror image file; performing slicing on the target mirror image file to obtain slices S1, S2,..., Sm, and the total number m of slices, wherein m is a positive integer; and sending data packets P1, P2,..., Pn and the total number m of slices to an upload agent device, wherein the data packet P1 includes the slice S1 and a slice sequence number I1, the data packet P2 includes the slice S2 and a slice sequence number I2, …, and the data packet Pn includes the slice Sn and a slice sequence number In, n is a positive integer, and n ≤ m.

Description

Method for uploading mirror file, related equipment and computer storage medium

Technical field

This application relates to the field of virtualization technology, and in particular to a method for uploading image files, related equipment, and computer storage media.

Background technique

In the application scenario of cloud services, the mirroring service is presented in the form of a portal, so the user needs to complete the registration and upload of the mirroring through a browser. In the process of uploading files in the browser, the problem of file upload interruption or upload failure often occurs due to poor network environment, and the browser does not support file uploading in fragments, so when the user uploads a large image file through the browser and does not When it is successfully uploaded to the server, the large image file needs to be re-uploaded, which greatly reduces the upload efficiency of the large image file.

At present, for public cloud mirroring services, users need to open commercial object storage services and upload mirror files through dedicated tools (for example, Packer, OBS Browser). This method requires users to download and learn the tools before they can Upload the image file, thereby reducing the ease of use. For private cloud mirroring services, enterprises need to deploy dedicated object storage services. Therefore, this method will consume a higher cost and reduce the ease of use to a certain extent.

Summary of the invention

The embodiment of the application discloses a method for uploading a mirror file, related equipment, and a computer storage medium, which can effectively solve the problem that the browser does not support uploading large mirror files and the upload efficiency is low.

In the first aspect, this application provides a method for uploading image files, which is applied to the client, including:

Obtain the target image file;

Fragmenting the target mirror file to obtain the fragments S ₁ , S ₂ ,..., S _m and the total fragment m, where m is a positive integer;

Send data packets P ₁ , P ₂ ,..., P _n and the total number of fragments m to the upload agent device, where the data packet P ₁ includes fragment S ₁ and fragment sequence number I ₁ , data The packet P ₂ contains the fragment S ₂ and the fragment sequence number I ₂ ,..., the data packet P _n contains the fragment S _n and the fragment sequence number I _n , n is a positive integer, and n≤m.

In some possible designs, when n is less than m, the method further includes: when n is less than m, the method further includes: sending a resumable upload request to the upload proxy device to obtain The fragment sequence number I _n , wherein the breakpoint resumable transmission request is used to request the fragment sequence number corresponding to the fragment where the breakpoint occurred; according to the fragment sequence number I _n , the data packet P _{n+1 is} sent to The upload agent device.

In some possible designs, the sending of the data packets P ₁ , P ₂ ,..., P _n to the upload agent device includes: using 1 thread to transfer the data packets P ₁ , P ₂ ,..., P _l send to the upload agent device in parallel; use the l threads to send data packets P _l+1 , P _l+2 ,..., P _2×l to the upload agent device in parallel; ...; use The l threads send data packets P _n-l+1 , P _n-l+2 ,..., P _n to the upload agent device in parallel, where 1≤l≤n/2, and l is positive Integer.

It can be seen that by using the above method, the target image file to be uploaded can be fragmented in the client, and then the fragments of the target image file can be sent to the upload agent device. Therefore, when the upload of the target image file is interrupted or failed during the upload process, the client can re-upload the target image file from the failed segment, thereby improving the upload efficiency of the target image file. Moreover, the client divides the target mirror file into fragments, and completes the upload of the target mirror file by uploading the fragments, thereby improving the upload success rate of the target mirror file. In addition, the client can also send segments of the target image file to the upload agent device in parallel, which can greatly improve the upload efficiency of the target image file.

In the second aspect, this application provides another method for uploading image files, which is applied to uploading agent equipment, including:

Receive the total number of fragments m and data packets P ₁ , P ₂ ,..., P _n sent by the client, where the data packet P ₁ contains the fragment S ₁ and the fragment sequence number I ₁ , and the data packet P ₂ contains the fragment The fragment S ₂ and the fragment sequence number I ₂ ,..., the data packet P _n contains the fragment S _n and the fragment sequence number I _n , and the fragments S ₁ , S ₂ ,..., S _m are the target mirror images of the client The file is obtained by fragmentation, m and _n are both positive integers, and n≤m;

The total number of fragments m and the fragments S ₁ , S ₂ ,..., S _{n are} sent to the object storage device.

In some possible designs, when n is less than m, after receiving the total number of fragments m sent by the client and the data packets P ₁ , P ₂ ,..., P _n , the method further includes: receiving The data packets P _n+2 , P _n+3 ,..., P _m sent by the client, and the fragments S _n+2 , S _n+3 ,..., S _{m are} sent to the object A storage device, wherein the data packet P _n+2 contains the fragment S _n+2 and the fragment sequence number I _n+2 , and the data packet P _n+3 contains the fragment S _n+3 and the fragment sequence number I _n+3 ,... , The data packet P _m includes the fragment S _m and the fragment sequence number I _m ; receiving the resumable transmission request sent by the client; according to the resuming transmission request, the fragment corresponding to the fragment where the breakpoint occurred The sequence number I _{n is} sent to the client; the data packet P _n+1 sent by the client according to the fragment sequence number I _{n is} received, and the data packet P _{n+1 is} sent to the object storage device, For the object storage device to _{restore the target image file according to the fragments S 1} , S ₂ ,..., S _m , wherein the data packet P _n+1 includes the fragment S _n+1 and the fragment The slice number I _n+1 .

In some possible designs, when n is less than m, the method further includes: receiving a resumable transmission request sent by the client; and according to the resuming transmission request, fragments of breakpoints will occur fragment corresponding to a number I _n the client; client receives the remaining data packets from said sheet number data packet I _n P _{n + 1} based on the resume start points and the The fragments corresponding to the remaining data packets are sent to the object storage device, so that the object storage device restores the target image file _{according to the fragments S 1} , S ₂ ,..., S _{m, where:} The remaining data packets include data packets P _n+1 , P _n+2 ,..., P _m , and the data packet P _n+1 includes a fragment S _n+1 and a fragment sequence number I _n+1 , The data packet P _n+2 includes a fragment S _n+2 and a fragment sequence number I _n+2 , ..., the data packet P _m includes a fragment S _m and a fragment sequence number _Im .

In some possible designs, the method further comprising: transmitting the data packet P is received in the client _1, the call management device to create the mirror image file corresponding to the target image identifier, and stores the image identification .

In some possible designs, the method further includes: invoking the image management device to refresh the image status corresponding to the target image file.

In the above method, after the upload proxy device receives the fragments of the target mirror file sent by the client, it forwards the fragments to the object storage device, and the object storage device restores the fragments to the target mirror file, thereby realizing the target mirror file Upload. At the same time, when the upload of the target image file is interrupted or failed during the upload process, the upload proxy device can send the fragment sequence number corresponding to the fragment that has the breakpoint to the client, so that the client can be based on the fragment that has the breakpoint. The corresponding segment sequence number continues to upload the target mirror file, thereby improving the upload efficiency and upload success rate of the mirror file.

In the third aspect, this application provides a client, including:

The obtaining unit is used to obtain the target image file;

The fragmentation processing unit is configured to fragment the target mirror file to obtain fragments S ₁ , S ₂ ,..., S _m and a total fragment m, where m is a positive integer;

The sending unit is configured to send data packets P ₁ , P ₂ ,..., P _n and the total number of fragments m to the upload agent device, where the data packet P ₁ includes a fragment S ₁ and a fragment The sequence number I ₁ , the data packet P ₂ contains the fragment S ₂ and the fragment sequence number I ₂ ,..., the data packet P _n contains the fragment S _n and the fragment sequence number I _n , n is a positive integer, and n≤m.

In some possible designs, when n is less than m, the sending unit is further configured to: send a resumable upload request to the upload proxy device to obtain the fragment sequence number I _n , where the The breakpoint resume request is used to request the fragment sequence number corresponding to the fragment where the breakpoint occurred; according to the fragment sequence number I _n , the data packet P _{n+1 is} sent to the upload agent device.

In some possible designs, the sending unit is specifically configured to: use 1 thread to send data packets P ₁ , P ₂ ,..., P ₁ to the upload agent device in parallel; use the 1 thread , The data packets P _l+1 , P _l+2 ,..., P _{2×l are} sent to the upload agent device in parallel; ...; using the l threads, the data packets P ₁ , P ₂ ,. .., P _{n is} sent to the upload agent device in parallel, where 1≤l≤n/2, and l is a positive integer.

In the foregoing client, the fragmentation processing unit divides the target mirror file into fragments, so as to realize that the client uploads the target mirror file in fragments. When the upload of the target image file is interrupted or failed during the upload process, the above-mentioned client can realize the resumable upload of the image file, which not only improves the upload efficiency of the target image file, but also improves the upload success rate of the target image file. . At the same time, the sending unit can also implement concurrent uploading of fragments. Therefore, when the above-mentioned client is used to upload the mirror file, the upload efficiency of the target mirror file can be greatly improved.

In a fourth aspect, this application provides an upload proxy device, which is characterized in that it includes:

The acquiring unit is used to receive the total number of fragments m and data packets P ₁ , P ₂ ,..., P _n sent by the client, where the data packet P ₁ includes the fragment S ₁ and the fragment sequence number I ₁ , and the data The packet P ₂ contains the fragment S ₂ and the fragment sequence number I ₂ ,..., the data packet P _n contains the fragment S _n and the fragment sequence number I _n , the fragments S ₁ , S ₂ ,..., S _m are said The client fragments the target image file, m and _n are both positive integers, and n≤m;

The sending unit is configured to send the total number of fragments m and the fragments S ₁ , S ₂ ,..., S _n to the object storage device.

In some possible designs, when n is less than m, after the acquiring unit receives the total number of fragments m sent by the client and the data packets P ₁ , P ₂ ,..., P _n , the acquiring The unit is also used to receive data packets P _n+2 , P _n+3 ,...,P _m sent by the client, and to fragment S _n+2 ,S _n+3 ,...,S _m Sent to the object storage device, where the data packet P _n+2 contains the fragment S _n+2 and the fragment sequence number I _n+2 , and the data packet P _n+3 contains the fragment S _n+3 and the fragment sequence number I _n+3 ,..., the data packet P _m includes the fragment S _m and the fragment sequence number I _m ; the acquiring unit is also used to receive a resumable transmission request sent by the client; the sending unit is also used to the HTTP request, the breakpoint will occur fragment corresponding to fragment I _n number sent to the client; the obtaining unit is further configured to receive a client I _n accordance with the transmission sequence number fragment packet P _{n + 1,} and the packet P _{+ 1} to _n transmitting the object storage device, for the object storage device according to the slice _{_{S 1, S 2, ...,}} S m Restore the target image file, where the data packet P _n+1 includes a fragment S _n+1 and a fragment sequence number I _n+1 .

In some possible designs, when n is less than m, the acquiring unit is further configured to receive a resumable transmission request sent by the client; and the sending unit is further configured to resume transmission according to the resumable transmission request. , the fragment corresponding to the occurrence of a breakpoint fragment I _n number sent to the client; the obtaining unit is further configured to receive the client according to the fragment ID from the packet I _n P _{n + 1} Start resuming the remaining data packets, and send the fragments corresponding to the remaining data packets to the object storage device for the object storage device according to the fragments S ₁ , S ₂ , ...,S _m restores the target image file, wherein the remaining data packets include data packets P _n+1 , P _n+2 ,..., P _m , and the data packets P _n+1 Contains a fragment S _n+1 and a fragment sequence number I _n+1 , the data packet P _n+2 contains a fragment S _n+2 and a fragment sequence number I _n+2 ,..., the data packet P _m contains a fragment S _m and the fragment sequence number I _m .

In some possible designs, the apparatus further comprises a call progress recording unit and a unit, the unit calls for the data packet P is sent in the client ₁ receives the call to create the target image management apparatus The mirror ID corresponding to the mirror file; the progress recording unit is used to save the mirror ID.

In some possible designs, the calling unit is further configured to: call the mirror management device to refresh the mirror state corresponding to the target mirror file.

It can be seen that after the acquiring unit receives the fragments of the target mirror file sent by the client, the sending unit forwards the fragments to the object storage device, so that the object storage device can be restored to the target mirror file according to the fragments, thereby realizing the target mirroring Upload of files. Moreover, when the upload of the target image file is interrupted or failed during the upload process, the sending unit can send the segment sequence number corresponding to the segment where the breakpoint occurred to the client, so that the client can correspond to the segment where the breakpoint occurred Continue to upload the target image file with the segment serial number of the file, thereby improving the upload efficiency and upload success rate of the image file.

In a fifth aspect, this application provides a mirror file upload system, characterized in that the system includes the client described in the third aspect, the upload proxy device described in the fourth aspect, an object storage device, and a mirroring service device, in,

The object storage device is used to receive the fragments S ₁ , S ₂ ,..., S _m sent by the upload agent device, and to convert the fragments S ₁ , S ₂ ,..., S _m Restore to the target image file;

The image management device is used to create an image identifier corresponding to the target image file;

The mirror management device is also used to refresh the mirror status of the target mirror file.

In the above-mentioned image file upload system, the client divides the image file into fragments and sends the fragments to the upload proxy device. The upload proxy device forwards the fragments to the object storage device, and the object storage device restores the fragments. It is the target mirror file to complete the upload of the target mirror file. At the same time, during the uploading of the target image file, the image management device refreshes the image status of the target image file to determine whether the target image file is successfully uploaded. Therefore, the above-mentioned mirror file upload system can realize the resumable upload of the target mirror file and the concurrent upload of the target mirror files, which not only improves the upload efficiency of the target mirror file, but also improves the upload success rate of the target mirror file.

In a sixth aspect, the present application provides a client, including a processor and a memory, and the processor executes the code in the memory to implement part or all of the steps described in the first aspect.

In a seventh aspect, the present application provides an upload proxy device, including a processor and a memory, and the processor executes the code in the memory to implement part or all of the steps described in the second aspect.

In an eighth aspect, this application provides a computer storage medium storing computer instructions; the computer instructions are used to implement part or all of the steps described in the first aspect.

In a ninth aspect, this application provides a computer storage medium storing computer instructions; the computer instructions are used to implement part or all of the steps described in the second aspect.

Description of the drawings

In order to explain the technical solutions of the embodiments of the present application more clearly, the following will briefly introduce the drawings needed in the description of the embodiments. Obviously, the drawings in the following description are some embodiments of the present application. For those of ordinary skill in the art, without creative work, other drawings can be obtained from these drawings.

Figure 1 is a schematic structural diagram of a cloud system provided by this application;

Figure 2 is a schematic structural diagram of a specific cloud system provided by this application;

Figure 3 is a schematic diagram of the structure of a mirror file upload system provided by this application;

Figure 4 is a schematic diagram of a mirror upload interface provided by this application;

Figure 5 is a schematic diagram of a process for uploading a mirror file provided by this application;

Fig. 6 is a schematic diagram of a process for uploading a mirror file provided by this application;

FIG. 7 is a schematic structural diagram of a client provided by this application;

Fig. 8 is a schematic structural diagram of an upload proxy device provided by the present application;

FIG. 9 is a schematic structural diagram of another client provided by this application;

Fig. 10 is a schematic structural diagram of another upload proxy device provided by the present application.

Detailed ways

Refer to Figure 1, which shows a schematic structural diagram of a cloud system. The owner of the cloud system deploys the cloud computing infrastructure by himself, namely deploys computing resources (for example, servers) 110, deploys storage resources (for example, storage) 120, deploys network resources (for example, network cards) 130, and so on. The owner of the public cloud (for example, an operator) virtualizes the computing resources, storage resources, and network resources of the cloud computing infrastructure, and provides corresponding services for cloud users (for example, users) to use. Among them, operators can provide the following three services for users: cloud computing infrastructure as a service (infrastructure as a service, IaaS), platform as a service (platform as a service, PaaS), and software as a service (software as a service, SaaS).

The service IaaS provides to users is the utilization of cloud computing infrastructure, including processing, storage, network, and other basic computing resources, enabling users to deploy and run any software, including operating systems and applications. Users do not manage or control any cloud computing infrastructure, but can control the choice of operating system, storage space, deployment applications, and may also gain control of restricted network components (for example, firewalls, load balancers, etc.).

The service provided by PaaS to users is to deploy applications developed or acquired by users using development languages and tools provided by vendors (for example, Java, python, Net, etc.) to cloud computing infrastructure. Users do not need to manage or control the underlying cloud computing infrastructure, including networks, servers, operating systems, storage, etc., but users can control the deployed applications and may also control the configuration of the hosting environment for running applications.

The services provided by SaaS to users are applications run by operators on cloud computing infrastructure. Users can access applications on cloud computing infrastructure on various devices through client interfaces (for example, browsers). Users do not need to manage or control any cloud computing infrastructure, including networks, servers, operating systems, storage, etc.

Therefore, users can use the services provided by operators to run their own businesses, such as deep learning, artificial intelligence, and big data.

Refer to Figure 2, which shows a schematic diagram of a specific cloud system. The cloud system of the present application includes: a cloud management platform 210 and an infrastructure 220. Among them, the cloud management platform 210 provides cloud services for users by calling the infrastructure 220. The cloud management platform 210 may be OpenStack, CloudStack, Hadoop, AWS (Amazon Web Service), etc., which is not specifically limited here.

The cloud management platform 210 may include a portal 211, a computing service 212, a storage service 213, a network service 214, and a management service 215. Among them, the portal211 can provide users with access to cloud systems or cloud services. The computing service 212 may be a component for deploying virtual machines and the like. The storage service 213 may be used to provide components such as distributed and persistent virtual object storage. The network service 214 may be a component for providing network topology management of network nodes. The management service 215 may be a component for controlling other services.

Taking the cloud management platform 210 as OpenStack as an example, the computing service 212 may be Nova, the storage service 213 may be Swift, Cinder, the network service 214 may be Neutron, and the management service 215 may be Glance, Nova, Horizon, Neutron, Keystone, and so on. Among them, Glance is mainly responsible for the management of the metadata of various images provided in the cloud system, and provides users with image management services, which can specifically realize the registration, search and retrieval of virtual machine images, so as to facilitate users to choose virtual machines And install the operating system. Moreover, Glance's back-end storage supports docking with multiple storage types, such as Swift, simple storage service (S3), or container storage, etc. Swift is the default object storage solution in OpenStack. Compared with other storage solutions (such as S3, container storage, etc.), Swift is easier to build on low-cost, standard hardware storage infrastructure, and Swift supports large object files (such as , Static large files and dynamic large files) upload in fragments.

Therefore, this application uses Swift as the back-end storage of Glance to store image files uploaded through the client browser. For details, refer to FIG. 3, which shows a mirror file upload system related to this application. The image file upload system may specifically include a client 310, an upload proxy device 320, an object storage device 330, and an image management device 340. In this application, the object storage device 330 is a device deployed with Swift, and the image management device 340 is a device deployed with Glance. Therefore, the object storage device 330 is referred to as Swift and the image management device 340 is referred to as Glance hereinafter. The upload proxy device 320, the object storage device 330, and the image management device 340 may be deployed on one computing device, or may be deployed on different computing devices, which are not specifically limited here.

In this application, when a user uploads a mirror file through the client 310, the mirror file is segmented at the client 310 first, and then the segments of the mirror file are sent to the upload proxy device 320, and the upload proxy device 320 receives the mirror file. After the file is fragmented, it is forwarded to Swift330, and Glance340 is called to refresh the status of the image. It can be seen that the system can be used to upload the mirror file in fragments through the client. That is to say, when the upload of the mirror file is interrupted or failed during the upload process, the client can continue to upload the image from the breakpoint. Mirror file, without re-uploading the complete mirror file, which greatly improves the transmission efficiency of the mirror file and the upload success rate.

Hereinafter, the application will be described in detail based on the mirror file upload system shown in FIG. 3.

In a specific embodiment of the present application, before the client sends the image file to be uploaded to the upload agent device, the client may receive the login request input by the user through the browser, and send the login request to the cloud management platform. Among them, the login request carries the account number and password, and the login request is used to request to log in to the mirror management system, which is provided by Glance. Then, the cloud management platform determines the login permission of the client according to the login request. If the client has the login permission, the mirror upload interface is displayed in the browser interface, as shown in Figure 4. Taking FIG. 4 as an example, the image file to be uploaded includes a first image file, a second image file, and a third image file. Taking the first image file as an example, the method for uploading the image file provided in this application is described in detail below, which can be specifically divided into the following two situations.

First, introduce the situation that the uploading process of the first image file did not occur interruption or upload failure. As shown in FIG. 5, FIG. 5 shows a schematic flowchart of a method for uploading a mirror file provided by the present application. The mirror file upload method of this embodiment includes but is not limited to the following steps:

S101. The client obtains the first image file.

In a specific embodiment of the present application, after obtaining the first image file, the client determines whether the size of the first image file is greater than a preset threshold. If the size of the first image file is greater than the preset threshold, execute S102; if the size of the first image file is less than or equal to the preset threshold, the client sends the first image file to the upload agent device, and then the upload agent device forwards it Store for Swift. When Swift receives the first image file, the upload agent device calls the Glance interface to refresh the image status of the first image file.

In a more specific embodiment, the client receives the dragging instruction input by the user through the browser, and obtains the first image file according to the dragging instruction. Among them, the image upload interface provides a file reading interface, and the drag instruction is used to drag the first image file to the file reading interface. The client determines the size of the first image file based on the file reading interface, and if the size of the first image file is greater than the preset threshold, execute S102. In addition, the client can also receive other types of instructions (for example, selection instructions) input by the user through the browser, so as to obtain the first image file.

S102. The client performs a fragmentation operation on the first mirror file according to the preset fragment size, and obtains the fragments S ₁ , S ₂ ,..., S _{m of the} first mirror file and the total fragment m, where m Is a positive integer.

In a specific embodiment of this application, the client performs a fragmentation operation on the first mirror file according to a preset fragment size to obtain fragments S ₁ , S ₂ ,..., S _m , and add to _{fragment S 1} The fragment sequence number I ₁ , the fragment S ₂ adds the fragment sequence number I ₂ , ..., the fragment S _m adds the fragment sequence number I _m , thereby obtaining data packets P ₁ , P ₂ ,..., P _m . Among them, the data packet P ₁ contains the fragment S ₁ and the fragment sequence number I ₁ , the data packet P ₂ contains the fragment S ₂ and the fragment sequence number I ₂ ,..., the data packet P _m contains the fragment S _m and the fragment sequence number I _m . Among them, the fragment sequence numbers I ₁ , I ₂ ,..., I _{m are} used to indicate _{the sequence of the fragments S 1} , S ₂ ,..., S _m , that is, the fragment S ₁ is the first Fragment, the second fragment of the fragment S ₂ ,..., the m-th fragment of the fragment S _m.

It should be noted that the data packets P ₁ , P ₂ ,..., P _m may also carry information such as the file name of the first image file and the size of the corresponding fragment, which is not specifically limited here.

S103. The client sends the data packets P ₁ , P ₂ ,..., P _m and the total number of fragments m to the upload agent device.

In a possible embodiment, the client sends the data packets P ₁ , P ₂ ,..., P _m to the upload agent device in sequence.

In another possible embodiment, the client uses l threads to simultaneously send data packets P ₁ , P ₂ ,..., P _l to the upload proxy device, and when the upload proxy device receives the data packets P ₁ , P ₂ After ,...,P _l , the client uses the above l threads to simultaneously send the data packets P _l+1 ,P _l+2 ,...,P _2×l to the upload agent device until the client sends the data The packages P ₁ , P ₂ ,..., P _{m are} all sent to the upload agent device. Among them, 1≤l≤m/2, and l is a positive integer. It can be seen that the number of uploads and file data flow can be effectively reduced by means of sharding and concurrency.

In the specific embodiment of this application, when the client sends a data packet to the upload proxy device for the first time (for example, data packet _P1 or data packets P ₁ , P ₂ ,..., P _l ), the upload proxy device The Glance interface is called to create mirror metadata, so as to obtain the identity document (ID) corresponding to the mirror metadata. Then, the upload agent device creates an empty object file named after the mirror ID according to the mirror ID. Among them, the image metadata is used to describe the first image, which can specifically include the image ID of the first image, the size of the first image, the image status of the first image, and the image format of the first image (for example, qcow2 format, vhd format) and many more. The image ID is used to identify the first image.

S104. The upload agent device sends the total number of fragments m and the fragments D ₁ , D ₂ ,..., D _m to Swift.

In a possible embodiment, if the client sends data packets P ₁ , P ₂ ,..., P _m to the upload proxy device in turn, the upload proxy device will segment D ₁ , D ₂ ,... , D _m is sent to Swift in turn.

Specifically, the proxy device receives the upload data packet P after _1, the packet P ₁ and the image ID, which is divided sheets S ₁ is added to give _a target segment prefix name N D _1, D ₁ and then sent to the segment Swift. Wherein the object name prefix N _{₁ 1} corresponding to the segment ID and segment number includes mirror image D, D ₁ corresponding to the segment number is the segment number fragment I _1. When the segments are received Swift D _1, the proxy device stores uploaded image segment ID and segment D ₁ D ₁ corresponding to the segment number in the segment record uploading. Also, the upload agent apparatus, after receiving the data packet P _2, the data packet P ₂ and the mirror ID, which is added slice S ₂ N ₂ to give the object name prefix segment D _2, D ₂ and then sent to the segment Swift. Wherein the object name prefix N _{₂ 2} corresponding to the segment ID and segment number includes mirror image D, D ₂ corresponding to the segment number is the segment number fragment I _2. When the segments are received Swift D _2, the proxy device stores uploaded image segment ID and segment D ₂ D ₂ corresponding to the segment number in the segment record uploading. …. After receiving the data packet P _m , the upload agent device adds the object prefix name N _m _{to the segment S m} according to the data packet P _m and the mirror ID to obtain the segment D _m , and then sends the segment D _m to Swift. Wherein the object includes mirror image prefix name N _m D _m corresponding to the segment ID and segment number, segment D _m corresponding to the segment number is fragment number I _m. When the segments are received Swift D _m, the proxy device stores upload mirror segment D _m D _m corresponding to the segment ID and segment number of the segment in the upload history.

Among them, the object prefix name can be specifically expressed as the mirror ID-segment serial number, the format of the mirror ID can be set to the default storage format of Glance, such as xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx, and the segment serial number can also be set to the default Glance storage format. Storage format, and the segment sequence number corresponds to the segment sequence number one to one. For example _{, the segment sequence number corresponding to the segment sequence number I 15} (00015) is 00015, so the object prefix name N ₁₅ can be expressed as xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx -00015. The multipart upload record can be seen in Table 1. From Table 1, it can be seen that _{the object prefix name corresponding to the segment S 1} is xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx-00001, and the object prefix name corresponding to the _{segment S 2 is xxxxxxxx-} xxxx-xxxx-xxxx-xxxxxxxxxxxx-00002,..., _{the object prefix name corresponding to the fragment S m} is xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx-m.

Table 1. Multipart upload record

In another possible embodiment, if the client sends data packets P ₁ , P ₂ ,..., P _l to the upload proxy device in parallel, P _l+1 , P _l+2 ,..., P _{2×l is} sent to the upload agent device in parallel,..., the data packets P _m-l+1 ,P _m-l+2 ,...,P _{m are} sent to the upload agent device in parallel, then the upload agent device will be divided into pieces D ₁ ,D ₂ ,...,D _{l are} sent to Swift in parallel, and the segments D _l+1 ,D _l+2 ,...,D _{2×l are} sent to Swift in parallel,..., the segments D _{m -l+1} ,D _m-l+2 ,...,D _{m are} sent to Swift in parallel.

Specifically, after the upload agent device receives the data packets P ₁ , P ₂ ,..., P _l , according to the data packets P ₁ , P ₂ ,..., P _l and the mirror ID, they are respectively fragments S ₁ , S ₂ ,...,S _l add object prefix names N ₁ ,N ₂ ,...,N _{l to} get segments D ₁ ,D ₂ ,...,D _l , and then segment D ₁ ,D ₂ ,...,D _{l is} sent to Swift in parallel. Among them, the object prefix name N ₁ includes the mirror ID and _{the segment sequence number corresponding to the segment D 1} , and the segment sequence number corresponding to the segment D _{1 is} the segment sequence number I ₁ ; the object prefix name N ₂ includes the mirror ID and the segment D ₂ The corresponding segment sequence number, the segment sequence number corresponding to segment D _{2 is} the segment sequence number I ₂ ; ...; the object prefix name N _l includes the mirror ID and the segment sequence number corresponding to the segment _{D m,} _{and the segment corresponding to the segment D l} The segment sequence number is the fragment sequence number I _l . If Swift receives segments D ₁ , D ₂ ,..., D _l , the upload agent device will save the mirror IDs and points of _{segments D 1} , D ₂ ,..., D _{l in the segment upload record.} Segment number corresponding to segment D ₁ , D ₂ ,..., D _l. It should be understood that the upload proxy device sends the fragments S _l+1 , S _l+2 ,..., S _2×l to Swift in parallel,..., the fragments S _m-l+1 ,S _{m-l The process of sending +2} ,...,S _m to Swift in parallel is similar to the process of sending the _{fragments S 1} , S ₂ ,..., S _l to Swift in parallel by the uploading agent device, and will not be repeated here.

It can be seen that the above steps implement the process from the input stream of the upload proxy device to the output stream of Swift, that is, read the image file from the upload proxy device and write the image file to Swift, thus eliminating the need for local caching. In this case, uploading the image file to Swift not only improves the upload efficiency of the image file, but also saves storage space.

S105. Swift merges the fragments S ₁ , S ₂ ,..., S _m into the first image file according to the object prefix names N ₁ , N ₂ ,..., N _m.

In the particular application of the present embodiment, Swift according to the received segment D _1, to give the object points sheets S ₁ and the name prefix N _1; Swift according to the received segment D _2, to give the object slice S ₂ and the prefix name N _2; ...; Swift D _m in accordance with the received segments, and obtain a target fragment prefix name S _m N _m. Moreover, Swift _{can also obtain the mirror IDs of the segments S 1} , S ₂ ,..., S _m and the corresponding segment sequence numbers according to the object prefix names of the _{segments D 1} , D ₂ ,...D _{m, thereby} It is determined that the fragments S ₁ , S ₂ ,..., S _m all belong to the fragments of the first mirror file. When Swift receives all the parts, that is, when the upload proxy device determines that the number of parts recorded in the multipart upload record is the same as the total number of fragments m, the upload proxy device sends the object file named with the mirror ID to Swift and sends Invoke the Swift interface to make Swift merge the fragments S ₁ , S ₂ ,..., S _{m of the} first image file according to the segment number _{according to the object prefix names N 1} , N ₂ ,..., N _m, Thus, the first image file is obtained.

S106. The upload proxy device obtains the upload progress of the first image file according to the multipart upload record and the total number of fragments m, and then calls the Glance interface to refresh the image status of the first image.

Among them, the mirror state includes the queued state, the saving state, the active state, and the killed state, and so on. The queued state indicates that the image file is waiting to be uploaded; the saving state indicates that the image file is being uploaded; the active state indicates that the image file is uploaded and the image can be used at this time; the killed state indicates that an error occurred during the upload of the image file.

In a specific embodiment of the present application, the upload agent device determines the upload progress of the first mirror file according to the multipart upload record, and then calls the Glance interface to refresh the mirror status corresponding to the first mirror file. If the image ID of any one of the segments and its sequence number are not stored in the multipart upload record, it means that the upload progress of the first image file is 0, and the status of the first image is queued at this time. If the multipart upload record stores the image IDs of k segments and the corresponding k segment sequence numbers, it means that the upload progress of the first image file is k/m, and the state of the first image is the saving state at this time. If the multipart upload record stores m segment mirror IDs and their corresponding m segment sequence numbers, it means that the upload progress of the first mirror file is 100%, and the state of the first mirror at this time is the active state. If the upload process is interrupted or the upload fails, the status of the first image is killed at this time.

In the specific embodiment of this application, after Swift _{merges the fragments S 1} , S ₂ ,..., S _m into the first image file, the upload proxy device calls the Glance interface to refresh the image state of the first image, and determines the first image file. A mirror is in the available state, so that the registration of the first mirror is completed.

For example, suppose that the size of the first image file is 130 GB, and the preset fragment size is 50 megabytes (MB). After the file reading interface receives the first image file, the client processes the first image file into 2663 fragments according to the preset fragment size, that is, fragments S ₁ , S ₂ ,..., S ₂₆₆₃ , Among them, the total number of fragments is 2,663, the size of fragments S ₁ , S ₂ ,..., S ₂₆₆₂ are all 50 MB, and _{the size of fragment S 2663} is 20 MB. The client adds the fragment sequence numbers for the fragments S ₁ , S ₂ ,..., S ₂₆₆₃ in turn, and obtains data packets P ₁ , P ₂ ,..., P ₂₆₆₃ , where the fragment sequence number of the _{fragment S 1 is I} ₁ referred to as 00001, slice number S fragment denoted as I ₂ ₂ 00002, ...., slice S fragment number ₂₆₆₃ ₂₆₆₃ is referred to as the I 02663. The client sends the data packet P ₁ and the total number of fragments to the upload proxy device. The upload proxy device receives the data packet P ₁ and calls the Glance interface to create mirror metadata, thereby obtaining the mirror ID (xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx) ), and then create a target object file named after the mirror ID based on the mirror ID. After that, the upload agent device adds the object prefix name N ₁ (xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx-00001) to the obtained segment _{S 1} _{according to the data packet P 1} and the mirror ID to obtain the _{segment D 1} , and then Segment D _{1 is} forwarded to Swift. When Swift receives segment D ₁ , the upload agent device saves the mirror ID and segment serial number (00001) corresponding _{to segment D 1 in the segment upload record.} Similarly, the client sends the data packet P ₂ to the upload proxy device, and the upload proxy device adds the object prefix name N ₂ (xxxxxxxx-xxxx-xxxx-xxxx) to the _{received segment S 2} _{according to the data packet P 2} and the mirror ID. -xxxxxxxxxxxx-00002) Obtain segment D ₂ and forward segment D ₂ to Swift. When Swift receives segment D ₂ , the upload proxy device will upload the corresponding mirror ID and segment serial number of _{segment D 2 (00002} ) Is saved in the multipart upload record; ...; the client sends the data packet P ₂₆₆₃ to the upload proxy device, and the upload proxy device adds the object prefix name N _{2663 to the} _{received segment S 2663} _{according to the data packet P 2663} and the mirror ID (xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx-02663) Get segment D ₂₆₆₃ , and forward segment D ₂₆₆₃ to Swift. When Swift receives segment D ₂₆₆₃ , the upload proxy device will upload segment D ₂₆₆₃ to the corresponding The mirror ID and part number (02663) are saved in the part upload record. At this point, the upload proxy device sends the target object file to Swift, and will call the Swift interface, so that Swift divides S ₁ , S ₂ ,... _{according to the object prefix name N 1} , N ₂ ,..., N _2663. , S _{2663 is} merged into the first image file. In addition, during the uploading process of the first image file, the upload agent device also periodically calls the Glance interface to refresh the image status of the first image according to the multipart upload record and the total number of fragments. After Swift _{merges the fragments S 1} , S ₂ ,..., S ₂₆₆₃ into the first image file, the upload proxy device refreshes the image status of the first image by calling the Glance interface, and obtains that the image status of the first image is available. , So as to complete the registration of the first mirror.

In the above method, when there is no upload interruption or upload failure of the image file during the upload process, the client divides the image file into fragments, and sends the fragments of the image file to the upload agent device, and then the upload agent device The segment corresponding to the segment is forwarded to Swift, so that Swift can merge the segments into a mirror file based on the received segment and the object prefix name. It can be seen that by using the above method, it is possible to use the fragments of the mirror file to replace the mirror file for upload, thereby improving the upload success rate of the mirror file. In addition, the client can also send the fragments of the mirror file to the upload proxy device in parallel, and then the upload proxy device sends the fragments to Swift in parallel, thereby improving the upload efficiency of the mirror file.

When the client sends the fragments of the first image file to the upload proxy device, the upload proxy device may not be able to completely receive all the fragments S ₁ , S _{2 of the} first image file due to network reasons or abnormalities of the upload proxy device, etc. ...,S _m . Then, next, taking the upload interruption of the first image file during the upload process or the upload failure of a single segment as an example, the method for uploading the image file provided in this application will be introduced. As shown in FIG. 6, FIG. 6 shows a schematic flowchart of a method for uploading a mirror file provided by the present application. The mirror file upload method of this embodiment includes but is not limited to the following steps:

S201: The client sends a resumable upload request to the upload proxy device.

Wherein, the resumable upload request carries the image ID corresponding to the first image file, and the resume resumable upload request is used to request the segment sequence number corresponding to the segment at the breakpoint when the upload is interrupted or the upload fails.

In the specific embodiment of the present application, when the client uploads the fragments of the first mirror file, the upload of a single fragment fails, for example, the upload agent device does not receive the fragment S _n+1 . Or, when the client uploads the fragments of the first image file, the upload is interrupted, that is, the upload proxy device does not receive the fragments S _n+1 , S _n+2 ,..., S _m . Among them, 1≤n<m, and n is a positive integer.

In a possible embodiment, when the client sends a data packet P _n+1 to the upload agent device, the network is interrupted, the current network environment does not support the upload of the data packet, the upload agent device is abnormal, etc., so that the upload agent device The data packet P _{n+1 is} not received. When the client sends a data packet P _n+2 to the upload proxy device, the network environment or the upload proxy device returns to normal, and the upload proxy device will be able to receive the data packets P _n+2 , P _n+3 ,... ,P _m , in other words, the upload agent device receives the data packets P ₁ , P ₂ ,...,P _n ,P _n+2 ,P _n+3 ,...,P _m .

In another possible embodiment, when the client sends a data packet P _n+1 to the upload agent device, the network is interrupted, the current network environment does not support the upload of the data packet, the upload agent device is abnormal, etc., so that the upload agent The device cannot receive the remaining data packets (P _n+1 ,P _n+2 ,...,P _m _{) starting from the data packet P n+1} . At this time, the upload proxy device only receives the data packet P ₁ , P ₂ ,...,P _n .

_{S202. The upload proxy device sends the segment sequence number I n} corresponding to the segment at the breakpoint to the client according to the request for resuming the transfer at the breakpoint.

In the specific embodiment of this application, after the upload agent device receives the resumable upload request, according to the image ID corresponding to the first image file, it calls the Glance interface to refresh the image status of the first image, thereby obtaining a file upload token (token ). _{Then, the upload proxy device sends the segment sequence number I n} (segment sequence number) corresponding to the segment (segment) where the breakpoint occurred in the multipart upload record to the client according to the file upload token, so that the client can The sequence number I _n starts from the data packet P _n+1 and continues to transmit the remaining data packets.

S203, the client transmits upload failures I _n corresponding fragment packet to upload the proxy apparatus according to the fragment number, the proxy device transmits the upload by uploading fragment corresponding to the failed segments to Swift.

In one possible embodiment, the client sends only upload proxy device according to the slice packet number I _n P _{n + 1.} Among them, the data packet P _n+1 includes a fragment S _n+1 and a fragment sequence number I _n+1 . Upload proxy device after receiving the data packet P _{n + 1,} the data packet P _{n + 1} and the image file ID of the first mirror, to slice the object S _{+ 1} to add a prefix name _{_n} N _n _{+ 1} D _{n +} segment obtained ₁ , and then send segment D _n+1 to Swift. Wherein the object name prefix N _{n + 1} and the segment ID includes mirror image D _{n + 1} corresponding to the segment number, segment D _{n + 1} corresponding to the segment number is the slice number I _{n + 1.} When the segments are received Swift D _{n + 1,} the proxy device upload saved image D _{n +} segment ID and segment D _n ₁ _{+ 1} in a segment corresponding to the segment number of the record uploading.

In another possible embodiment, the client sequentially sends data packets P _n+1 , P _n+2 ,..., P _m _{to the upload proxy device according to the fragment sequence number I n} . Among them, the data packet P _n+1 includes the fragment S _n+1 and the fragment sequence number I _n+1 , and the data packet P _n+2 includes the fragment S _n+2 and the fragment sequence number I _n+2 ,..., the data packet P _m includes the fragment S _m and the fragment sequence number I _m . Upload proxy device after receiving the data packet P _{n + 1,} the data packet P _{n + 1} and the image ID, add a prefix name for the object slice S _{_{n + 1}} N _{n + 1} D to give the segment _{n + 1,} then the points Segment D _{n+1 is} sent to Swift. Wherein the object prefix name N _{n + 1} and the segment ID includes mirror image D _{n + 1} corresponding to the segment number, segment D _{n + 1} corresponding to the segment number is the slice number I _{n + 1.} When the segments are received Swift D _{n + 1,} the proxy device upload saved image D _{n +} segment ID and segment D _n ₁ _{+ 1} in a segment corresponding to the segment number of the record uploading. Upload proxy device after receiving the data packet P _{n + 2,} the data packet P _{n + 2} and the image ID, add a prefix name for the object slice S _{_{n + 2}} N _{n + 2} to give D segment _{n + 2,} then the sub- Segment D _{n+2 is} sent to Swift. Wherein the object name prefix N _{n + 2} includes mirror image ID and segment D _{n + 2} corresponding to the segment number, segment D _{n + 2} corresponding to the segment number is the slice number I _{n + 2.} When the segments are received Swift D _{n + 2,} the proxy device upload saved image D _{n +} segment ID and segment D _n ₂ _{+ 2} corresponding to the segment number in the segment uploaded records. …. After receiving the data packet P _m , the upload agent device adds the object prefix name N _m _{to the segment S m} according to the data packet P _m and the mirror ID to obtain the segment D _m , and then sends the segment D _m to Swift. Wherein the object includes mirror image prefix name N _m D _m corresponding to the segment ID and segment number, segment D _m corresponding to the segment number is fragment number I _m. When the segments are received Swift D _m, the proxy device stores upload mirror segment D _m D _m corresponding to the segment ID and segment number of the segment in the upload history.

In another possible embodiment, the client uses 1 thread to send the data packets P _n+1 , P _n+2 ,..., P _n+1 to the upload agent device in parallel until the upload agent device receives Data packet P _m . After the upload agent device receives the data packets P _n+1 ,P _n+2 ,...,P _n+l , according to the data packets P _n+1 ,P _n+2 ,...,P _n+l and mirroring ID, respectively add the object prefix name N _n+1 ,N _n+2 ,...,N _n+l _{to the fragments S n+1} , S _n+2 ,..., S _n+l to get the score Segment D _n+1 , D _n+2 ,..., D _n+l , and then send the segments D _n+1 , D _n+2 ,..., D _n+l to Swift in parallel. Wherein the object name prefix N _{n + 1} and the segment ID includes mirror image D _{n + 1} corresponding to the segment number, segment D _{n + 1} corresponding to the segment number is the slice number I _{n + 1;} target name prefix N _{n +2} includes the mirror ID and _{the segment number corresponding to the segment D n+2} , the segment number corresponding to the segment D _{n+2 is} the segment number I _n+2 ; ...; the object prefix name N _n+1 includes the mirror ID The segment sequence number corresponding to the segment D _n+1 , and the segment sequence number corresponding to the segment D _{n+1 is} the segment sequence number I _n+1 . If Swift receives the segments D _n+1 , D _n+2 ,..., D _n+l , the upload agent device will save the segments D _n+1 , D _n+2 , in the segment upload record respectively. .., D _{n + l} mirrored segment ID and _{_{D n + 1, D n +}} 2, ..., D n + l corresponding to the segment number.

S204. Swift _{merges the fragments S 1} , S ₂ ,..., S _m into the first image file.

In a possible embodiment, before the client sends a resumable upload request to the upload proxy device, the segments D ₁ , D ₂ ,..., D _n , D _n+2 , D _{n are saved in Swift +3} ,...,D _m . Among them, the segments D ₁ , D ₂ ,..., D _n , D _n+2 , D _n+3 ,..., D _m are determined by the upload agent device according to the data packets P ₁ , P ₂ ,... ,P _n ,P _n+2 ,P _n+3 ,...,P _m and the mirror ID, which are respectively the fragments S ₁ , S ₂ ,..., S _n , S _n+2 , S _n+3 ,...,S _{m is obtained after} adding object prefix names N ₁ ,N ₂ ,...,N _n ,N _n+2 ,N _n+3 ,...,N _m . When Swift receives the segment D _n , Swift according to the segment D ₁ , D ₂ ,..., D _n , D _n+1 , D _n+2 ,..., D _m object prefix name N ₁ , N ₂ ,...,N _n ,N _n+1 ,N _n+2 ,...,N _m , merge the slices S ₁ , S ₂ ,..., S _m into the first image file.

_{In another possible embodiment, the segments D 1} , D ₂ ,..., D _n are saved in Swift before the client sends a resumable upload request to the upload proxy device. Among them, the segments D ₁ , D ₂ ,..., D _n _{are the segments S 1} , S ₂ , and S 2 according to the data packets P ₁ , P ₂ ,..., P _n and the mirror ID by the upload agent device. ...,S _{n is obtained after} adding object prefix names N ₁ ,N ₂ ,...,N _n . When Swift receives the segments D _n+1 , D _n+2 ,...,D _m , Swift according to the segments D ₁ , D ₂ ,..., D _n , D _n+1 , D _n+2 ,...,D _m object prefix name N ₁ ,N ₂ ,...,N _n ,N _n+1 ,N _n+2 ,...,N _m , S ₁ ,S ₂ , ..., S _{m is} merged into the first image file.

S205. The upload agent device obtains the upload progress of the first image file according to the multipart upload record and the total number of fragments m, and then calls the Glance interface to refresh the image status of the first image.

In the specific embodiment of this application, after Swift _{merges the fragments S 1} , S ₂ ,..., S _m into the first image file, the proxy component calls the Glance interface to refresh the image status of the first image, and determines the first image file. The mirror is in a usable state, thereby completing the registration of the first mirror. In addition, during the process of resuming the uploading of the failed fragments, the upload proxy device can also call the Glance interface to refresh the mirror status of the first mirror.

For example, suppose that the size of the first image file is 500GB, and the preset fragment size is 200MB. After the file reading interface receives the first image file, the client browser processes the first image file into 2560 fragments according to the preset fragment size, that is, fragments S ₁ , S ₂ ,...,S ₂₅₆₀ . Among them, the total number of fragments is 2560, and _{the sizes of fragments S 1} , S ₂ ,..., S ₂₅₆₀ are all 200 MB. The browser adds the sequence numbers of the fragments to the fragments S ₁ , S ₂ ,..., S ₂₅₆₀ in turn to obtain data packets P ₁ , P ₂ ,..., P ₂₅₆₀ . Among them, the fragment serial number I _{1 of the} _{fragment S 1} is recorded as 00001, the fragment serial number I _{2 of the} _{fragment S 2} is recorded as 00002,..., and the fragment serial number I _{2560 of the} _{fragment S 2560} is recorded as 02560. The client sends the data packet P ₁ and the total number of fragments to the upload proxy device. The upload proxy device receives the data packet P ₁ and calls the Glance interface to create mirror metadata, thereby obtaining the mirror ID (xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx) ), and then create a target object file named after the mirror ID based on the mirror ID. After the upload agent device receives the data packet P ₁ , it adds the object prefix name N ₁ (xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx-00001) to the segment _{S 1} according to the data packet P ₁ and the mirror ID _{, thereby obtaining the segment D 1} , And send segment D ₁ to Swift. When Swift receives segment D ₁ , the upload proxy device saves _{the mirror ID corresponding to segment D 1} and its segment serial number (00001) in the segment upload record; the client will The data packet P _{2 is} sent to the upload agent device. After the upload agent device receives the data packet P ₂ , it adds the object prefix name N ₂ (xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx) to the segment _{S 2} according to the data packet P ₂ and the mirror ID -00002), so as to obtain segment D ₂ and send segment D ₂ to Swift. When Swift receives segment D ₂ , the upload agent device saves the mirror ID corresponding to _{segment D 2 and its corresponding image ID in the segment upload record.} Segment sequence number (00002); ...; the client sends the data packet P ₂₀₀₀ to the upload agent device. After the upload agent device receives the data packet P ₂₀₀₀ , it adds the object prefix name for the _{segment S 2000} according to the data packet P ₂₀₀₀ and the mirror ID N ₂₀₀₀ (xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx-02000) to obtain segment D ₂₀₀₀ and send segment D ₂₀₀₀ to Swift. When Swift receives segment D ₂₀₀₀ , the upload agent device uploads records in segment _{The mirror ID corresponding to segment D 2000} and its segment number (02000) are stored in it. When the client sends the data packet P ₂₀₀₁ to the upload agent device, the data packet P ₂₀₀₁ and the remaining data packets cannot be sent to the upload agent device due to a network interruption. When the network is restored, the client sends a resumable upload request to the upload agent device. The upload proxy device sends the fragment sequence number I ₂₀₀₀ (02000) to the client according to the resumable upload request. Then, the client sends a data packet P ₂₀₀₁ to the upload proxy device according to the fragment sequence number I _{2000. After} the upload proxy device receives the data packet P ₂₀₀₁ , it adds the object prefix name N _{2001 to the} _{fragment S 2001} according to the data packet P ₂₀₀₁ and the mirror ID. (xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx-02001), to get segment D ₂₀₀₁ , and send segment D ₂₀₀₁ to Swift, when Swift receives segment D ₂₀₀₁ , the upload agent device saves it in the segment upload record The mirror ID corresponding to segment D ₂₀₀₁ and its segment serial number (02001); ...; the client sends a data packet P ₂₅₆₀ to the upload proxy device, and the upload proxy device receives the data packet P ₂₅₆₀ according to the data packet P ₂₅₆₀ and the mirror ID S ₂₅₆₀ is added to slice the object name prefix _{N 2560 (xxxxxxxxxxxx-xxxxxxxx-xxxxxxxxxxxx} -02560), to obtain a D segment _2560, and ₂₅₆₀ to send Swift D segment, the segment is received when Swift D ₂₅₆₀ , The upload agent device saves _{the mirror ID corresponding to the segment D 2560} and its segment serial number (02560) in the segment upload record. At this point, the upload proxy device calls the Swift interface, so that Swift merges the fragments S ₁ , S ₂ ,..., S ₂₅₆₀ into the first image file _{according to the object prefix names N 1} , N ₂ ,..., N ₂₅₆₀ . In addition, during the uploading process of the first image file, the upload agent device also periodically calls the Glance interface to refresh the image status of the first image according to the multipart upload record and the total number of fragments. After Swift _{merges the fragments S 1} , S ₂ ,..., S ₂₅₆₀ into the first image file, the upload proxy device refreshes the first image's image status by calling the Glance interface, and gets the first image's image status as available. , So as to complete the registration of the first mirror.

It should be understood that when the size of the first image file is less than or equal to the preset threshold, and the upload of the first image file is interrupted or failed during the upload process, the browser will re-upload the first image file to the upload proxy device, and then The upload proxy device forwards it to Swift for storage. When Swift receives the first image file again, the upload agent device calls the Glance interface to refresh the image status of the first image file.

It should be noted that the preset fragment size may be 50MB, 200MB, 60MB, 150MB, etc. in the foregoing embodiment, and there is no specific limitation here.

For the sake of simplicity, this embodiment does not provide data packets P ₁ , P ₂ ,...,P _n-1 ,P _n+1 ,...,P _m P ₁ ,P ₂ ,...,P _m , The definition and acquisition process of data packets P ₁ , P ₂ ,..., P _n-1 _{, segment D 1} , D ₂ ,..., D _n-1 , D _n+1 ,..., D _{The definition and acquisition process of m} , segment D ₁ , D ₂ ,..., D _n-1 are described in detail, and Swift does not merge the segments S ₁ , S ₂ ,..., S _m into the first image file. The specific process of the upload agent device calling the Glance interface to refresh the mirror status is described in detail, and the multipart upload record has not been described in detail. For details, please refer to S101-S106 and related embodiments, which will not be repeated here. . This embodiment only introduces an application scenario where the upload of a single part fails, and the similar above-mentioned method can also be applied to an application scenario where the upload of multiple parts fails, which will not be repeated here. In addition, in the above two embodiments, only the upload method of the first image file is described in detail. In actual applications, the upload method of the second image file, the third image file, etc. is similar to the upload process of the first image file. I won't expand the details here again.

In the above method, when the upload is interrupted or the upload fails during the upload process of the image file, the client sends a breakpoint resuming upload request to the upload proxy device to obtain the fragment sequence number corresponding to the fragment where the breakpoint occurred, thereby This allows the client to resend the failed upload fragments to the upload proxy device, thereby completing the image file upload process. It can be seen that the use of the above method can realize the resumable upload of the mirror file, and improve the upload success rate and upload efficiency of the mirror file. In addition, the client can also send the fragments of the mirror file to the upload proxy device in parallel, and then the upload proxy device sends the fragments to Swift in parallel, thereby improving the upload efficiency of the mirror file.

With reference to the foregoing Figures 1 to 6 and related embodiments, the following will take the uploading process of the first image file as an example to describe in detail the related devices involved in the embodiments of the present application.

In this embodiment of the application, the client 310 may be an entity on the user side for receiving or transmitting signals, such as a new generation UE (gUE), which may specifically include mobile phones, tablets, and personal digital assistants. (personal digital assistant, PDA), mobile internet device (mobile internet device, MID), notebook computer, smart wearable device (such as smart watch, smart bracelet), etc., which are not limited in the embodiment of this application. Please refer to FIG. 7 for details. FIG. 7 is a schematic structural diagram of a client provided in this application. The client 310 may include an acquiring unit 410, a fragment processing unit 420, and a sending unit 430. in,

The obtaining unit 410 is used to obtain the first image file.

The fragmentation processing unit 420 is configured to perform a fragmentation operation on the first mirror file according to the preset fragment size to obtain the fragments S ₁ , S ₂ ,..., S _{m of the} first mirror file and the total fragment m, Among them, m is a positive integer.

The sending unit 430 is configured to send the data packets P ₁ , P ₂ ,..., P _m and the total number of fragments m to the upload agent device.

In a specific embodiment of the present application, the obtaining unit 410 is further configured to determine whether the size of the first image file is greater than a preset threshold after obtaining the first image file, and if the size of the first image file is greater than the preset threshold, The first mirror file 311 is sent to the fragmentation processing unit 420; if the size of the first mirror file is less than or equal to the preset threshold, the first mirror file is sent to the sending unit 430, and then the first mirror file is sent by the sending unit 430 Sent to the upload agent device.

In a more specific embodiment, the acquiring unit 410 may receive a drag instruction input by the user and acquire the first image file according to the drag instruction; the acquiring unit 410 may also receive a selection instruction input by the user, and according to the selection instruction Obtaining the first image file, etc., is not specifically limited here.

In a specific embodiment of the present application, the fragmentation processing unit 420 is specifically configured to perform a fragmentation operation on the first mirror file according to a preset fragment size to obtain fragments S ₁ , S ₂ ,..., S _m , and is added sheets S ₁ sub-fragment No. I _1, S ₂ is added slice slice number I _2, ..., S _m slice add fragment number I _m, to thereby obtain the data packet P _1, P _2, ..., P _m . Among them, the data packet P ₁ contains the fragment S ₁ and the fragment sequence number I ₁ , the data packet P ₂ contains the fragment S ₂ and the fragment sequence number I ₂ ,..., the data packet P _m contains the fragment S _m and the fragment sequence number I _m . Among them, the fragment sequence numbers I ₁ , I ₂ ,..., I _{m are} used to indicate _{the sequence of the fragments S 1} , S ₂ ,..., S _m , that is, the fragment S ₁ is the first Fragment, the second fragment of the fragment S ₂ ,..., the m-th fragment of the fragment S _m.

In a possible embodiment, when the upload is not interrupted or the upload fails during the uploading process of the first image file, the sending unit 430 may send the data packets P ₁ , P ₂ ,..., P _m to the uploader in sequence. Agent equipment. Alternatively, the sending unit 430 uses l threads to simultaneously send the data packets P ₁ , P ₂ ,..., P _l to the upload proxy device, and when the upload proxy device receives the data packets P ₁ , P ₂ ,..., P _{After l} , the sending unit 430 uses the above l threads to send the data packets P _l+1 , P _l+2 ,..., P _2×l to the upload agent device at the same time, until the sending unit 430 sends the data packets P ₁ , P ₂ ,..., P _{m are} all sent to the upload agent device. Among them, 1≤l≤m/2, and l is a positive integer. It can be seen that the number of uploads and file data flow can be effectively reduced by means of sharding and concurrency.

In another possible embodiment, when the first slice of a single image file occurs during the upload process (e.g., data packets corresponding to P _{n + 1} corresponding to the fragment S _{n + 1)} fails to upload, the transmission unit 430 It is also used to send a resumable upload request to the upload agent device to obtain the fragment sequence number I _n , where the resumable upload request carries the mirror ID corresponding to the first mirror file, and the resume resumable upload request is used for request generation The fragment sequence number corresponding to the fragment of the breakpoint; according to the fragment sequence number I _n , the data packet P _{n+1 is} sent to the upload agent device. It should be understood that the failure to upload multiple fragments of the first image file during the upload process is similar to the failure to upload a single fragment, and will not be repeated here.

In another possible embodiment, when the uploading of the first image file is interrupted during the uploading process (for example, the interruption occurs at the data packet P _n+1 ), the sending unit 430 is also used to send the upload proxy device Send a breakpoint resume request to obtain the fragment sequence number I _n . Among them, the resumable transmission request carries the mirror ID corresponding to the first mirror file, and the resumable transmission request is used to request the fragment sequence number corresponding to the fragment where the breakpoint occurred; according to the fragment sequence number I _n , the data packet P _n+1 ,P _n+2 ,...,P _{m are} sent to the upload agent device. Wherein, the sending unit 430 may send the data packets P _n+1 , P _n+2 ,..., P _m to the upload agent device in turn, and may also use the concurrent upload method to send the data packets P _n+1 , P _{n+ 2} ,...,P _{m is} sent to the upload agent device, please refer to S203 for details.

In the above client, the fragmentation processing unit divides the mirror file into fragments, so that the client can upload the mirror file in fragments. Moreover, the sending unit can also implement concurrent uploading of fragments. Therefore, when the above-mentioned client is used to upload the mirror file, the upload efficiency of the target mirror file can be greatly improved. In addition, when the upload of the mirror file is interrupted or failed during the upload process, the sending unit sends a resumable upload request to the upload proxy device to obtain the segment sequence number corresponding to the segment where the breakpoint occurred, so that the sending unit can The segment sequence number corresponding to the segment of the breakpoint sends the data packet corresponding to the segment that failed to upload to the upload agent device. It can be seen that when using the above-mentioned client to upload the mirror file, the resumable upload of the mirror file and the concurrent upload of the mirror file can be realized, which not only improves the upload efficiency of the mirror file, but also improves the upload success rate of the mirror file.

The upload proxy device provided in this application can be a software device or a hardware device. When the upload proxy device is a software device, the upload proxy device can be deployed separately in any environment (cloud environment, edge environment, terminal computing equipment, etc.) On the computing device, it can also be deployed on the same computing device as Swift and Glance; when the upload proxy device is a hardware device, the upload proxy device can be a computing device in any environment. The unit modules inside the upload agent device can also be divided into multiple categories. Each module can be a software module, a hardware module, or a part of a software module and a part of a hardware module, which is not limited in this application. FIG. 8 is an exemplary division manner. As shown in FIG. 8, the upload proxy device 320 may include an acquiring unit 510, a sending unit 520, a calling unit 530, and a progress recording unit 540. The functions of each functional unit are introduced below.

The acquiring unit 510 is configured to receive the total number of fragments m and data packets P ₁ , P ₂ ,..., P _m sent by the client, where the data packet P ₁ includes the fragment S ₁ and the fragment sequence number I ₁ , and the data The packet P ₂ contains the fragment S ₂ and the fragment sequence number I ₂ ,..., the data packet P _m contains the fragment S _m and the fragment sequence number I _m , and the fragments S ₁ , S ₂ ,..., S _m are clients It is obtained by fragmenting the target image file, and m is a positive integer.

The sending unit 520 is configured to send the total number of fragments m and the fragments S ₁ , S ₂ ,..., S _m to Swift.

Calling unit 530 is used when the data packet P is sent by a client _1, a first call Glance create a mirror image file corresponding to the ID.

The progress recording unit 540 is used to save the image ID of the first image file, the image size of the first image file, the size of the fragments S ₁ , S ₂ ,..., S _m , and the fragment sequence numbers I ₁ , I ₂ ,... ..,I _m .

In a specific embodiment of the present application, before the sending unit 520 sends the fragments S ₁ , S ₂ ,..., S _m to Swift, the sending unit 520 is also used to pair the fragments S ₁ , S ₂ ,... ,S _m adds the corresponding object prefix name to obtain the segments D ₁ , D ₂ ,..., D _m , and then sends the segments D ₁ , D ₂ ,..., D _m to Swift. Wherein the sub-sheets S ₁ object prefix name of the corresponding N ₁ of includes mirror image ID and the fragment number I _1, slice S ₂ corresponding to the object prefix name N ₂ includes mirror image ID and slice number I _2, ..., slice S _m The corresponding object prefix name N _m includes the mirror ID and the fragment sequence number I _m .

In one possible embodiment, when the first upload interrupted or failed to upload the image file does not occur during upload, the transmitting unit 520 for the sub-object prefix added sheets S ₁ N ₁ to give the name of the segment D _1, and then the points The segment D _{1 is} sent to Swift. If Swift receives the segment D ₁ , the mirror ID of the _{segment D 1 and} _{the segment sequence number I 1} are stored in the progress recording unit 540; the sending unit 520 adds an object prefix to the _{segment S 2} Name N ₂ gets segment D ₂ , and then sends segment D ₂ to Swift. If Swift receives segment D ₂ , it saves the mirror ID of _{segment D 2} _{and segment number I 2} in the progress recording unit 540 ；...; The sending unit 520 adds the object prefix name N _m _{to the segment S m to} obtain the segment D _m , and then sends the segment D _m to Swift, and if Swift receives the segment D _m , it is saved in the progress recording unit 540 The segment D _m mirror ID and the _{segment sequence number I m} .

In another possible embodiment, when the uploading process of the first image file is not interrupted or the uploading fails, the sending unit 520 adds object prefixes to the _{segments S 1} , S ₂ ,..., S _{1 respectively.} Name N ₁ ,N ₂ ,...,N _{l to} get the segments D ₁ , D ₂ ,...,D _l , and then send the segments D ₁ , D ₂ ,..., D _l to Swift in parallel. If Swift receives the segments D ₁ , D ₂ ,..., D _l , the mirror IDs of the _{segments D 1} , D ₂ ,..., D _l _{and the segment D 1} are respectively saved in the progress recording unit 540 ,D ₂ ,...,D _l corresponding to the fragment sequence numbers I ₁ ,I ₂ ,...,I _l . It should be understood that the upload proxy device sends the fragments S _l+1 , S _l+2 ,..., S _2×l to Swift in parallel,..., the fragments S _m-l+1 ,S _{m-l The process of sending +2} ,...,S _m to Swift in parallel is similar to the process of sending the _{fragments S 1} , S ₂ ,..., S _l to Swift in parallel by the uploading agent device, and will not be repeated here.

In another possible embodiment, when the first slice of a single image file occurs during the upload process (e.g., data packets corresponding to P _{n + 1} corresponding to the fragment S _{n + 1)} upload failures, i.e. acquiring unit When 510 only receives the data packets P ₁ , P ₂ ,..., P _n , P _n+2 , P _n+3 ,..., P _m sent by the client, the acquiring unit 510 is also used to receive the client The sending unit 520 is also used to send the segment sequence number I _n saved in the progress recording unit 540 to the client for the sent request for resuming the transmission. After that, the acquiring unit 510 is further configured to receive the data packet P _n+1 _{sent by the client according to the fragment sequence number I n} , and the sending unit 520 is further configured to add the object prefix name N _{n+1 to the} _{fragment S n+1 to} obtain the fragment Segment D _n+1 and send segment D _n+1 to Swift. If Swift receives the segment D _n+1 , it saves the mirror ID of the _{segment D n+1 and} _{the segment sequence number I n+1} in the progress recording unit 540. It should be understood that the failure to upload multiple fragments of the first image file during the upload process is similar to the failure to upload a single fragment, and will not be repeated here.

In another possible embodiment, when the uploading of the first image file is interrupted during the uploading process (for example, the interruption occurs at the data packet P _n+1 ), that is, the obtaining unit 510 only receives the transmission from the client. When the data packets P ₁ , P ₂ ,..., P _n are included, the acquiring unit 510 is also used to receive a resumable transmission request sent by the client, and the sending unit 520 is also used to transfer the fragments stored in the progress recording unit 540 The sequence number I _{n is} sent to the client. After that, the acquisition unit 510 for further receiving the remaining data packet fragment in accordance with the client ID from the packet I _n P _{n + 1} starts continuous transmission _{_{(P n + 1, P n}} + 2, ..., P m ), the sending unit 520 is also used to add object prefix names N _n+1 , N _n+2 ,..., N _{m to the} _{segments S n+1} , S _n+2 ,..., S _{m to} obtain the points Segment D _n+1 , D _n+2 ,..., D _m , and send segments D _n+1 , D _n+2 ,..., D _m to Swift. If Swift receives the segments D _n+1 , D _n+2 ,..., D _m _{, the segments D n+1} , D _n+2 ,..., D _m are respectively saved in the progress recording unit 540 Corresponding mirror IDs and fragment sequence numbers I _n+1 , I _n+2 ,...,I _m .

In the specific embodiment of the present application, after Swift receives the segment D _m , that is, the progress recording unit 540 saves the mirror IDs of the _{segments D 1} , D ₂ ,..., D _{m and the corresponding segment sequence numbers.} I _1, I _2, ..., then I _m, call unit 530 is further configured to restore the first call Swift image file.

In the specific embodiment of the present application, the calling unit 530 is also used to call Glance to refresh the mirroring state of the first mirror after _{Swift merges the fragments S 1} , S ₂ , ..., S _{m into the first mirror file.} It is determined that the first image is in an available state, so that the registration of the first image is completed. In addition, the calling unit 530 may also call Glance in real time during the uploading process of the first mirror file to refresh the mirror status of the first mirror, which is not specifically limited here.

For the sake of simplicity, this embodiment does not elaborate on the specific process of the upload proxy device calling Swift to restore the first image file, nor the specific process of the upload proxy device calling Glance to refresh the image status of the first image. The specific process of uploading the proxy device to send the segment corresponding to the segment has not been elaborated in detail. For details, please refer to Figure 5-Figure 6 and some or all of the steps of S103-S106 and S202-S205, which will not be repeated here.

It can be seen that the acquisition unit of the upload proxy device receives the data packet sent by the client, and can obtain the fragments of the mirror file, and then the sending unit forwards the fragments corresponding to the fragments to Swift, so that Swift can obtain the fragments according to the fragments. Object prefix names and fragments, so that the fragments can be restored to mirror files without local caching, and the target mirror file can be uploaded. Moreover, when the upload of the target image file is interrupted or failed during the upload process, the sending unit can send the segment sequence number corresponding to the segment where the breakpoint occurred to the client, so that the client can correspond to the segment where the breakpoint occurred Continue to upload the failed fragments in the mirror file with the serial number of the fragment, thereby greatly improving the upload efficiency and upload success rate of the mirror file.

As shown in Figure 9, Figure 9 shows a schematic structural diagram of another client provided by this application. The client 310 may specifically include a processor 610, a communication interface 620, and a memory 630. Wherein, the processor 610, the communication interface 620, and the memory 630 are coupled through the bus 640.

The processor 610 may be a central processing unit (CPU), a general-purpose processor, a digital signal processor (digital signal processor, DSP), an application specific integrated circuit (ASIC), a field programmable gate array ( field programmable gate array (FPGA) or other programmable logic device (programmable logic device, PLD), transistor logic device, hardware component, or any combination thereof. The processor 610 may implement or execute various exemplary methods described in conjunction with the disclosure of the present application. Specifically, the processor 610 reads the program code stored in the memory 630, and cooperates with the communication interface 620 to execute part or all of the steps of S101-S103, S201, and S203.

The communication interface 620 can be a wired interface or a wireless interface for communicating with other modules or devices. The wired interface can be an Ethernet interface, a controller area network interface, a local interconnect network (LIN), and a FlexRay interface. The interface can be a cellular network interface or a wireless local area network interface. Specifically, the communication interface 620 may be connected to the upload proxy device 320 and so on.

The memory 630 may include volatile memory, such as random access memory (random access memory, RAM); the memory 630 may also include non-volatile memory, such as read only memory (ROM), flash memory, and hard disk. (hard disk drive, HDD) or solid state drive (solid state drive, SSD), the storage 630 may also include a combination of the above types of storage. The memory 630 may store program codes and program data. Wherein, the program code is composed of the codes of some or all of the units in the client shown in FIG. The program data is the data generated by the client in the process of running the program shown in Figure 7, for example, the fragments S ₁ , S ₂ ,..., S _m , and the data packets P ₁ , P ₂ ,..., P _m , the size of the image file, the number of fragments, and so on.

The bus 640 may be a controller area network (CAN) or other internal buses. The bus 640 can be divided into an address bus, a data bus, a control bus, and so on. For ease of representation, only one thick line is used in FIG. 9 to represent it, but it does not mean that there is only one bus or one type of bus.

As shown in Fig. 10, Fig. 10 shows a schematic structural diagram of another upload proxy device provided by the present application. The upload proxy device 320 may specifically include a processor 710, a communication interface 720, and a memory 730. Wherein, the processor 710, the communication interface 720, and the memory 730 are coupled through a bus 740.

The processor 710 may be a CPU, a general-purpose processor, DSP, ASIC, FPGA or other PLD, transistor logic device, hardware component, or any combination thereof. The processor 710 may implement or execute various exemplary methods described in conjunction with the disclosure of the present application. Specifically, the processor 710 reads the program code stored in the memory 730, and cooperates with the communication interface 720 to execute part or all of the steps of S103-S106 and S202-S205.

The communication interface 720 can be a wired interface or a wireless interface for communicating with other modules or devices. The wired interface can be an Ethernet interface, a controller area network interface, LIN and FlexRay interface, and the wireless interface can be a cellular network interface or use a wireless LAN Interface, etc. Specifically, the communication interface 720 may be connected to the network device 750, and the network device 750 may include the client 310, Swift330, Glance340, and so on.

The memory 730 may include volatile memory, such as RAM; the memory 730 may also include non-volatile memory, such as ROM, flash memory, HDD, or SSD, and the memory 730 may also include a combination of the foregoing types of memory. The memory 730 may store program codes and program data. The program code is composed of the codes of some or all of the units in the upload proxy device shown in FIG. 8, for example, the code of the acquiring unit 510, the code of the sending unit 520, the code of the calling unit 530, and so on. The program data is the data generated by the upload device in the process of running the program shown in Figure 8, for example, segment D ₁ , D ₂ ,..., D _m , object prefix name N ₁ , N ₂ ,..., N _m , upload records in parts, etc.

The bus 740 may be CAN or other internal buses. The bus 740 can be divided into an address bus, a data bus, a control bus, and so on. For ease of representation, only one thick line is used to represent in FIG. 10, but it does not mean that there is only one bus or one type of bus.

This application also provides a computer storage medium. The computer storage medium stores computer instructions. The computer instructions are executed by hardware (such as a processor, etc.) to implement part or all of the steps (such as , S101-103, S201, S203).

This application also provides a computer storage medium. The computer storage medium stores computer instructions. The computer instructions are executed by hardware (such as a processor, etc.) to implement part or all of the steps performed by the upload agent device 320 in this application ( For example, S103-S106, S202-S205).

In the above-mentioned embodiments, it may be implemented in whole or in part by software, hardware, firmware, or any combination thereof. When implemented by software, it can be implemented wholly or partly in the form of a computer instruction product. The above-mentioned computer instruction product includes one or more computer instructions. When the foregoing computer instructions are loaded and executed on a computer, the processes or functions described in the embodiments of the present application are generated in whole or in part. The above-mentioned computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The aforementioned computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the aforementioned computer instructions can be transmitted from a website, computer, server, or data center through a wired (For example, coaxial cable, optical fiber, digital subscriber line) or wireless (for example, infrared, wireless, microwave, etc.) to transmit to another website site, computer, server or data center. The foregoing computer-readable storage medium may be any available medium that can be accessed by a computer or a data storage device such as a server or data center integrated with one or more available media. The above-mentioned usable medium may be a magnetic medium (such as a floppy disk, a storage disk, a magnetic tape), an optical medium (such as a DVD), or a semiconductor medium (such as an SSD), etc. In the above-mentioned embodiments, the description of each embodiment has its own emphasis. For parts that are not described in detail in an embodiment, reference may be made to related descriptions of other embodiments.

In the several embodiments provided in this application, it should be understood that the disclosed device may also be implemented in other ways. For example, the device embodiments described above are only illustrative. For example, the division of the units is only a logical function division, and there may be other divisions in actual implementation, for example, multiple units or components can be combined or integrated. To another system, or some features can be ignored or not implemented. In addition, the displayed or discussed indirect coupling or direct coupling or communication connection between each other may be through some interfaces, indirect coupling or communication connection between devices or units, and may be in electrical or other forms.

The units described above as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the objectives of the solutions in the embodiments of the present application.

In addition, the functional units in the embodiments of the present application may be integrated into one processing unit, or each unit may exist alone physically, or two or more units may be integrated into one unit. The above-mentioned integrated unit can be implemented in the form of hardware or software functional unit.

If the above integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer readable storage medium. Based on this understanding, the technical solution of the present application essentially or the part that contributes to the existing technology or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium. A number of instructions are included to enable a computer device (which may be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the methods described in the various embodiments of the present application. The aforementioned storage media may include, for example, various media capable of storing program codes and data files, such as a U disk, a mobile hard disk, a read-only memory, a random access memory, a magnetic disk, or an optical disk.

The above are only specific implementations of this application, but the protection scope of this application is not limited to this. Anyone familiar with the technical field can easily think of various equivalents within the technical scope disclosed in this application. Modifications or replacements, these modifications or replacements shall be covered within the protection scope of this application. Therefore, the protection scope of this application shall be subject to the protection scope of the claims.

Claims

A method for uploading a mirror file, applied to a client, is characterized in that it includes:

Obtain the target image file;

Fragmenting the target mirror file to obtain the fragments S 1 , S 2 ,..., S m and the total fragment m, where m is a positive integer;

Send data packets P 1 , P 2 ,..., P n and the total number of fragments m to the upload agent device, where the data packet P 1 includes fragment S 1 and fragment sequence number I 1 , data The packet P 2 contains the fragment S 2 and the fragment sequence number I 2 ,..., the data packet P n contains the fragment S n and the fragment sequence number I n , n is a positive integer, and n≤m.
The method according to claim 1, characterized in that, in the case where n is smaller than m, the method further comprising: sending the upload request to the HTTP proxy device to obtain the I the fragment number n, Wherein, the request for resuming transmission with a breakpoint is used to request the fragment sequence number corresponding to the fragment where the breakpoint occurred;

According to the fragment sequence number I n , the data packet P n+1 is sent to the upload proxy device.
The method according to claim 2, wherein the sending the data packets P 1 , P 2 ,..., P n to the upload agent device comprises:

Using l threads, the data packets P 1 , P 2 ,..., P l are sent to the upload agent device in parallel; using the l threads, the data packets P l+1 , P l+2 ,. .., P 2×l is sent to the upload agent device in parallel; ...; Using the l threads, the data packets P n-l+1 , P n-l+2 ,..., P n are sent in parallel For the upload agent device, where 1≤l≤n/2, and l is a positive integer.
A method for uploading a mirror file, which is applied to an upload agent device, and is characterized in that it includes:

Receive the total number of fragments m and data packets P 1 , P 2 ,..., P n sent by the client, where the data packet P 1 contains the fragment S 1 and the fragment sequence number I 1 , and the data packet P 2 contains the fragment The fragment S 2 and the fragment sequence number I 2 ,..., the data packet P n contains the fragment S n and the fragment sequence number I n , and the fragments S 1 , S 2 ,..., S m are the target mirror images of the client The file is obtained by fragmentation, m and n are both positive integers, and n≤m;

The total number of fragments m and the fragments S 1 , S 2 ,..., S n are sent to the object storage device.
The method according to claim 4, wherein when n is less than m, after receiving the total number of fragments m sent by the client and the data packets P 1 , P 2 ,..., P n , The method also includes:

Receive data packets P n+2 , P n+3 ,..., P m sent by the client, and send the fragments S n+2 , S n+3 ,..., S m to the An object storage device, where the data packet P n+2 contains the fragment S n+2 and the fragment sequence number I n+2 , and the data packet P n+3 contains the fragment S n+3 and the fragment sequence number I n+3 , ..., the data packet P m includes the fragment S m and the fragment sequence number I m ;

Receiving a resumable transmission request sent by the client;

Sending the fragment sequence number I n corresponding to the fragment where the breakpoint occurred to the client according to the request for resuming the transmission of the breakpoint;

Receive the data packet P n+1 sent by the client according to the fragment sequence number I n , and send the data packet P n+1 to the object storage device, so that the object storage device can according to the The fragments S 1 , S 2 ,..., S m restore the target image file, where the data packet P n+1 includes a fragment S n+1 and a fragment sequence number I n+1 .
The method according to claim 4, characterized in that, when n is less than m, the method further comprises:

Receiving a resumable transmission request sent by the client;

Sending the fragment sequence number I n corresponding to the fragment where the breakpoint occurred to the client according to the request for resuming the transmission of the breakpoint;

Receive the remaining data packets that the client continues to transmit from the data packet P n+1 according to the fragment sequence number I n , and send the fragments corresponding to the remaining data packets to the object Storage device for the object storage device to restore the target image file according to the fragments S 1 , S 2 ,..., S m , wherein the remaining data packets include data packets P n+1 ,P n+2 ,...,P m , the data packet P n+1 includes a fragment S n+1 and a fragment sequence number I n+1 , and the data packet P n+2 includes a fragment S n +2 and the fragment sequence number I n+2 ,..., the data packet P m includes the fragment S m and the fragment sequence number Im .
The method according to claim 5 or 6, wherein the method further comprises:

When receiving the packet P 1 is transmitted to the client, image management device creates a call to the target image corresponding to the image file identifier, and stores the image identification.
The method according to any one of claims 4 to 7, wherein the method further comprises: invoking the image management device to refresh the image status corresponding to the target image file.
A client, characterized in that it includes:

The obtaining unit is used to obtain the target image file;

The fragmentation processing unit is configured to fragment the target mirror file to obtain fragments S 1 , S 2 ,..., S m and a total fragment m, where m is a positive integer;

The sending unit is configured to send data packets P 1 , P 2 ,..., P n and the total number of fragments m to the upload agent device, where the data packet P 1 includes a fragment S 1 and a fragment The sequence number I 1 , the data packet P 2 contains the fragment S 2 and the fragment sequence number I 2 ,..., the data packet P n contains the fragment S n and the fragment sequence number I n , n is a positive integer, and n≤m.
The client according to claim 9, wherein when n is less than m, the sending unit is further configured to:

Sends the upload request HTTP proxy device to obtain the fragment number I n, wherein, said HTTP request ID fragmentation occurred fragment corresponding breakpoints for requesting;

According to the fragment sequence number I n , the data packet P n+1 is sent to the upload proxy device.
The client according to claim 10, wherein the sending unit is specifically configured to:

Using l threads, the data packets P 1 , P 2 ,..., P l are sent to the upload agent device in parallel; using the l threads, the data packets P l+1 , P l+2 ,. .., P 2×l is sent to the upload agent device in parallel; ...; using the l threads, data packets P 1 , P 2 ,..., P n are sent to the upload agent device in parallel, wherein , 1≤l≤n/2, and l is a positive integer.
An upload proxy device, characterized in that it comprises:

The acquiring unit is used to receive the total number of fragments m and data packets P 1 , P 2 ,..., P n sent by the client, where the data packet P 1 includes the fragment S 1 and the fragment sequence number I 1 , and the data The packet P 2 contains the fragment S 2 and the fragment sequence number I 2 ,..., the data packet P n contains the fragment S n and the fragment sequence number I n , the fragments S 1 , S 2 ,..., S m are said The client fragments the target image file, m and n are both positive integers, and n≤m;

The sending unit is configured to send the total number of fragments m and the fragments S 1 , S 2 ,..., S n to the object storage device.
The device according to claim 12, characterized in that, in the case that n is less than m, the acquiring unit receives the total number of fragments m sent by the client and the data packets P 1 , P 2 ,..., P After n,

The acquiring unit is also used to receive data packets P n+2 , P n+3 ,...,P m sent by the client, and to fragment S n+2 , S n+3 ,... , S m is sent to the object storage device, where the data packet P n+2 contains the fragment S n+2 and the fragment sequence number I n+2 , and the data packet P n+3 contains the fragment S n+3 and the fragment The slice sequence number I n+3 ,..., the data packet P m includes the slice S m and the slice sequence number I m ;

The acquiring unit is further configured to receive a resumable transmission request sent by the client;

The sending unit is further configured to send the fragment sequence number I n corresponding to the fragment where the breakpoint occurs to the client according to the breakpoint resuming transmission request;

The acquiring unit is further configured to receive a data packet P n+1 sent by the client according to the fragment sequence number I n , and send the data packet P n+1 to the object storage device for the The object storage device restores the target image file according to the fragments S 1 , S 2 ,..., S m , wherein the data packet P n+1 includes a fragment S n+1 and a fragment sequence number I n+1 .
The device according to claim 12, wherein when n is less than m,

The acquiring unit is further configured to receive a resumable transmission request sent by the client;

The sending unit is further configured to send the fragment sequence number I n corresponding to the fragment where the breakpoint occurs to the client according to the breakpoint resuming transmission request;

The obtaining unit is further configured to receive the client based on the remaining sub-packets from said sheet number data packet I n P n + 1 starts resume, and the rest of the packet corresponds The fragments are sent to the object storage device, so that the object storage device restores the target image file according to the fragments S 1 , S 2 ,..., S m , wherein the remaining data packets Including data packets P n+1 , P n+2 ,..., P m , the data packet P n+1 includes a fragment S n+1 and a fragment sequence number I n+1 , the data packet P n +2 contains the fragment S n+2 and the fragment sequence number I n+2 ,..., the data packet P m contains the fragment S m and the fragment sequence number I m .
The device according to claim 13 or 14, wherein the device further comprises a calling unit and a progress recording unit,

The calling unit for receiving said data packet sent by the client when P 1, calls the image management device creates the image file corresponding to the target image identifier;

The progress recording unit is used to save the mirror image identifier.
The device according to any one of claims 12 to 15, wherein the calling unit is further configured to: call the mirror management device to refresh the mirror status corresponding to the target mirror file.
A mirror file upload system, characterized in that the system includes the client according to any one of claims 9 to 11, the upload agent device according to any one of claims 12 to 16, and object storage Equipment and mirroring service equipment, of which,

The object storage device is used to receive the fragments S 1 , S 2 ,..., S m sent by the upload proxy device, and to convert the fragments S 1 , S 2 ,..., S m Restore to the target image file;

The image management device is used to create an image identifier corresponding to the target image file;

The mirror management device is also used to refresh the mirror status of the target mirror file.
A client is characterized by comprising a processor and a memory, and the processor executes the code in the memory to implement the method according to any one of claims 1 to 3.
An upload proxy device, which is characterized by comprising a processor and a memory, and the processor executes the code in the memory to implement the method according to any one of claims 4 to 8.
A computer storage medium, characterized in that it stores computer instructions; the computer instructions are used to implement the method according to any one of claims 1 to 3.
A computer storage medium, characterized in that it stores computer instructions; the computer instructions are used to implement the method according to any one of claims 4 to 8.