CN107872489B

CN107872489B - File slice uploading method and device and cloud storage system

Info

Publication number: CN107872489B
Application number: CN201610858544.0A
Authority: CN
Inventors: 卢洲
Original assignee: Hangzhou Hikvision Digital Technology Co Ltd
Current assignee: Hangzhou Hikvision Digital Technology Co Ltd
Priority date: 2016-09-28
Filing date: 2016-09-28
Publication date: 2020-06-05
Anticipated expiration: 2036-09-28
Also published as: CN107872489A; WO2018059222A1

Abstract

The embodiment of the invention provides a file slice uploading method and device and a cloud storage system. Wherein, the method comprises the following steps: the method comprises the steps that a metadata server receives a distribution request of a target file to be uploaded, which is sent by a client; determining an available slicing server corresponding to a target file with a file identifier according to the current load information of the slicing server; determining available object storage equipment corresponding to a target file with a file identifier according to the current load information of the available slicing server and the object storage equipment; the device identifier of the available slicing server is sent to the client, so that the client sends the slicing file of the target file to the available slicing server based on the device identifier, and the available slicing server stores the first slicing file of the target file to the available object storage device. According to the scheme, the file can still be uploaded normally when the server is overloaded or a severe network environment occurs.

Description

File slice uploading method and device and cloud storage system

Technical Field

The invention relates to the technical field of computers and internet data processing, in particular to a file slice uploading method and device and a cloud storage system.

Background

With the continuous improvement of internet speed and the rapid popularization of mobile internet and intelligent terminals, the cloud storage system has become the current trend of internet development. From the aspect of application architecture, the difference between a cloud storage system and a traditional distributed storage system is not obvious, more essential difference is reflected on an internal software architecture, the cloud storage system is based on a large number of servers and storage devices, a large-scale storage cluster is constructed, storage capacity is provided, capacity expansion can be carried out on line, and therefore the overall cost for constructing the large-capacity storage system is far lower than that of the traditional storage architecture. As the number of mobile users increases, the data volume of the cloud storage system also increases, and therefore, the file uploading technology of the cloud storage system is just a new challenge for the development of the cloud storage system.

And the client uploads the file to the server or a corresponding storage device for storage. In the existing file uploading technology, a large amount of resources, such as a CPU (Central processing unit), a port, a flow, a memory, and a disk I/O (input/output port) of a server, are occupied when a file is uploaded, and once the load of the server is too large, too large time delay or packet loss is easily generated when the file is uploaded, thereby causing file uploading failure; meanwhile, the network environment is severe, which is also an important factor causing file uploading failure.

Disclosure of Invention

The embodiment of the invention aims to provide a file slice uploading method and device and a cloud storage system, so that a file can be uploaded normally when a server is overloaded or a severe network environment occurs. The specific technical scheme is as follows:

in a first aspect, an embodiment of the present invention provides a file slice uploading method, which is applied to a metadata server in a cloud storage system, where the cloud storage system further includes multiple slice servers and multiple object storage devices, and the method includes:

receiving an allocation request of a target file to be uploaded, which is sent by a client, wherein the allocation request carries: file identifiers of the target files, and identification signals of available slicing servers and available object storage devices;

determining an available slicing server corresponding to the target file with the file identifier according to the current load information of the slicing server;

determining available object storage equipment corresponding to the target file with the file identifier according to the available slicing server and the current load information of the object storage equipment;

and sending a device identifier of the available slicing server to the client, so that the client sends the slicing file of the target file to the available slicing server based on the device identifier, so that the available slicing server stores a first slicing file of the target file to the available object storage device, wherein the first slicing file is obtained by performing original data calculation on the slicing file of the target file.

Optionally, before the step of receiving a distribution request of a target file to be uploaded, the method further includes:

receiving an initialization request of the target file sent by the client;

generating an initial identifier, and sending the initial identifier to the client, so that the client takes the received initial identifier as a file identifier of the target file.

Optionally, after the step of determining an available object storage device corresponding to the target file with the file identifier, the method further includes:

sending an object identifier of the available object storage device to the available slice server;

accordingly, the step of sending the device identifier of the available slicing server to the client, so that the client sends the slicing file of the target file to the available slicing server based on the device identifier, so that the available slicing server stores the first slicing file of the target file to the available object storage device, comprises:

sending a device identifier of the available slicing server to the client to cause the client to send the slicing file of the target file to the available slicing server based on the device identifier, thereby causing the available slicing server to store a first slicing file of the target file to the available object storage device based on the received object identifier.

Optionally, the step of sending a device identifier of the available slicing server to the client, so that the client sends the slicing file of the target file to the available slicing server based on the device identifier, so that the available slicing server stores the first slicing file of the target file to the available object storage device, includes:

sending a device identifier of the available slicing server to the client, so that the client sends the slicing file of the target file to the available slicing server based on the device identifier, so that the available slicing server requests the metadata server for the object identifier of the available object storage device, and stores a first slicing file of the target file to the available object storage device based on the requested object identifier.

Optionally, the determining, according to the current load information of the slicing server, an available slicing server corresponding to the target file with the file identifier includes:

acquiring load values of the plurality of slice servers, wherein the load values are at least one of the utilization rate of a central processing unit, the utilization rate of a memory and the utilization rate of input/output equipment;

judging whether a first slice server exists in the plurality of slice servers, wherein the load value of the first slice server is smaller than a preset first load threshold value;

if so, determining an available slicing server corresponding to the target file with the file identifier from the first slicing server;

if not, determining an available slicing server corresponding to the target file with the file identifier from the plurality of slicing servers.

Optionally, the determining, from the first slice server, an available slice server corresponding to the target file with the file identifier includes:

determining an available slice server corresponding to the target file with the file identifier from the first slice server by adopting a rotation method;

alternatively, the first and second electrodes may be,

and determining an available slicing server corresponding to the target file with the file identifier from the first slicing server by adopting a random method.

Optionally, the determining, from the plurality of slice servers, an available slice server corresponding to the target file with the file identifier includes:

determining an available slicing server corresponding to the target file with the file identifier from the plurality of slicing servers by adopting a rotation method;

alternatively, the first and second electrodes may be,

and determining an available slicing server corresponding to the target file with the file identifier from the plurality of slicing servers by adopting a random method.

Optionally, the determining, according to the current load information of the available slicing server and the object storage device, an available object storage device corresponding to the target file having the file identifier includes:

acquiring the IP addresses of the network protocol of the available slice server and the IP addresses of the object storage devices;

judging whether a first object storage device exists in the plurality of object storage devices, wherein the IP address of the first object storage device is the same as the IP address of the available slicing server;

if so, determining that the first object storage device is an available object storage device;

if not, determining the available object storage device corresponding to the target file with the file identifier from the plurality of object storage devices based on the current load information of the plurality of object storage devices.

Optionally, the determining, from the plurality of object storage devices, an available object storage device corresponding to the target file with the file identifier based on the current load information of the plurality of object storage devices includes:

acquiring load values of the plurality of object storage devices;

judging whether a second object storage device exists in the object storage devices, wherein the load value of the second object storage device is smaller than a preset second load threshold value;

if yes, determining available object storage equipment corresponding to the target file with the file identifier from the second object storage equipment;

if not, acquiring the distribution times of the object storage devices;

judging whether a third object storage device exists in the plurality of object storage devices, wherein the distribution frequency of the third object storage device is greater than a preset distribution frequency threshold value;

if so, determining the available object storage device corresponding to the target file with the file identifier from the third object storage device.

Optionally, the determining, from the second object storage device, an available object storage device corresponding to the target file with the file identifier includes:

determining available object storage equipment corresponding to the target file with the file identifier from the second object storage equipment by adopting a rotation method;

alternatively, the first and second electrodes may be,

and determining the available object storage device corresponding to the target file with the file identifier from the second object storage device by adopting a random method.

Optionally, the original data calculation method is an erasure code EC algorithm.

Optionally, after the step of sending the device identifier of the available slicing server to the client, so that the client sends the slicing file of the target file to the available slicing server based on the device identifier, so that the available slicing server stores the first slicing file of the target file to the available object storage device, the method further includes:

storing first metadata sent by the available slice server into a metadata table, wherein the first metadata is metadata of the first slice file, the metadata includes a disk serial number, the disk serial number is a serial number of a disk storing the first slice file, and the disk is determined by the available object storage device according to a memory size of the first slice file.

Optionally, the file slice uploading method further includes:

after the first metadata is successfully stored, sending a storage success instruction corresponding to the first metadata to the available slicing server, so that the available slicing server sends an uploading ending instruction of the target file to the client after receiving the storage success instruction of the first metadata of each first slicing file;

receiving an aggregation request of the target file sent by the client, wherein the aggregation request carries: the file identifier of the target file and an identification signal requesting to aggregate the first metadata are sent by the client after receiving an uploading ending instruction of the target file;

aggregating first metadata corresponding to the target file in the metadata table based on the file identifier of the target file.

Optionally, the file slice uploading method further includes:

receiving a termination request sent by the client, wherein the termination request carries: the file identifier of the target file and an identification signal requesting termination of uploading of the slice file are sent, wherein the termination request is sent by the client after receiving an uploading end instruction of the target file;

and deleting the file identifier after judging that the aggregation is finished and the first metadata corresponding to the target file in the metadata table.

In a second aspect, an embodiment of the present invention further provides a file slice uploading apparatus, which is applied to a metadata server in a cloud storage system, where the cloud storage system further includes a plurality of slice servers and a plurality of object storage devices, and the apparatus includes:

a first receiving module, configured to receive an allocation request of a target file to be uploaded, where the allocation request carries: file identifiers of the target files, and identification signals of available slicing servers and available object storage devices;

a first determining module, configured to determine, according to current load information of the slicing server, an available slicing server corresponding to the target file with the file identifier;

a second determining module, configured to determine, according to the available slicing server and current load information of the object storage device, an available object storage device corresponding to the target file having the file identifier;

a first sending module, configured to send a device identifier of the available slicing server to the client, so that the client sends the slicing file of the target file to the available slicing server based on the device identifier, so that the available slicing server stores a first slicing file of the target file in the available object storage device, where the first slicing file is obtained by performing an original data calculation method on the slicing file of the target file.

In a third aspect, an embodiment of the present invention further provides a cloud storage system, where the system includes a metadata server, multiple slice servers, and multiple object storage devices;

the metadata server is to: receiving an allocation request of a target file to be uploaded, which is sent by a client, wherein the allocation request carries: file identifiers of the target files, and identification signals of available slicing servers and available object storage devices; determining an available slicing server corresponding to the target file with the file identifier according to the current load information of the slicing server; determining available object storage equipment corresponding to the target file with the file identifier according to the available slicing server and the current load information of the object storage equipment; sending a device identifier of the available slicing server to the client, so that the client sends the slicing file of the target file to the available slicing server based on the device identifier;

the slicing server is configured to: acquiring and sending the current load information of the slice server and the current load information of the object storage device corresponding to the slice server to the metadata server; and when receiving the slice file of the target file sent by the client, storing a first slice file of the target file to the available object storage device, wherein the first slice file is obtained by executing an original data calculation method on the slice file of the target file. According to the file slice uploading method, the file slice uploading device and the cloud storage system, the slice server and the object storage equipment can be reasonably distributed through the metadata server according to the load information of the slice server and the object storage equipment, the target file is uploaded, and when the server load is overlarge, the situation that the file uploading fails is effectively avoided by selecting the reasonable slice server and the reasonable object storage equipment; in addition, by slicing the target file to be uploaded, if the sliced file is lost in a severe network environment, uploading of other sliced files is not influenced; and executing an original data calculation method in the file uploading process to ensure that the file stored by the object storage equipment is a recoverable target file to be uploaded.

Drawings

In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below, it is obvious that the drawings in the following description are only some embodiments of the present invention, and for those skilled in the art, other drawings can be obtained according to the drawings without creative efforts.

Fig. 1 is a first flowchart of a file slice uploading method according to an embodiment of the present invention;

fig. 2 is a second flowchart of a file slice uploading method according to an embodiment of the present invention;

FIG. 3 is a third flowchart of a file slice uploading method according to an embodiment of the present invention;

fig. 4 is a fourth flowchart of a file slice uploading method according to an embodiment of the present invention;

fig. 5 is a fifth flowchart of a file slice uploading method according to an embodiment of the present invention;

fig. 6 is a first structural diagram of a file slice uploading device according to an embodiment of the present invention;

fig. 7 is a second structural diagram of a file slice uploading device according to an embodiment of the present invention;

fig. 8 is a schematic structural diagram of a third file slice uploading device according to an embodiment of the present invention;

fig. 9 is a fourth structural diagram of a file slice uploading device according to an embodiment of the present invention;

fig. 10 is a schematic structural diagram of a fifth file slice uploading device according to an embodiment of the present invention;

fig. 11 is a schematic structural diagram of a cloud storage system according to an embodiment of the present invention.

Detailed Description

The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.

In order to ensure that a file can be uploaded normally when the server is overloaded or a severe network environment occurs, the embodiment of the invention provides a file slice uploading method and device and a cloud storage system.

First, a file slice uploading method provided by an embodiment of the present invention is described below.

It should be noted that an execution subject of the file slice uploading method provided by the embodiment of the present invention is a metadata server in a cloud storage system. The functional software for implementing the file slice uploading method provided by the embodiment is software arranged in a metadata server.

As shown in fig. 1, a file slice uploading method provided by an embodiment of the present invention may include the following steps:

s101, receiving a distribution request of a target file to be uploaded, which is sent by a client.

It should be noted that, before uploading a target file, a client needs to send an allocation request to a metadata server to apply for an available slice server and an available object storage device; in addition, at the same time, the client may send multiple allocation requests to multiple target files at the same time, and in order to ensure that the slice servers and the object storage devices allocated by the metadata server correspond to the target files, the allocation requests need to carry file identifiers of the target files; the allocation request may be an HTTP (Hyper text Transfer Protocol) message, and the metadata server provides a service to the client in the form of HTTP REST API (Hyper text Transfer Protocol Representational State Transfer application programming Interface). The client may be dedicated client software on a desktop computer or a mobile phone, or may also be browser software on the desktop computer or the mobile phone, and the like, which is reasonable.

Moreover, it can be understood that the file identifier is identification information that can uniquely determine the target file in the server, for example: the name of the target file, or the number assigned to the target file by the metadata server, etc.

It is emphasized that, in the cloud storage system, the metadata server is used for uniformly managing the metadata of the business application and the storage system, the slicing server is used for data calculation, and the object storage device is used for storing files. The target file to be uploaded may be a video file, a picture file, or a voice file, etc.

S102, determining an available slicing server corresponding to the target file with the file identifier according to the current load information of the slicing server.

In order to eliminate the influence of overlarge server load on file uploading, load information of a slice server needs to be obtained; the method comprises the steps that a slicing server collects and sends current load information of the slicing server and current load information of object storage equipment corresponding to the slicing server to a metadata server; the metadata server receives the load information. The load information of the slicing server may be a load size of the slicing server or a load occupation ratio of the slicing server, but is not limited thereto.

It should be noted that the cloud storage system includes a plurality of slice servers, and load information of each slice server is determined. The metadata server selects a plurality of slice servers from all slice servers by adopting different selection methods according to the current load information of each slice server; the selection method can be a rotation method, a random method or the like, and an available slicing server is selected.

It can be understood that there are various specific implementations of the step of determining the available slicing server corresponding to the target file with the file identifier according to the current load information of the slicing server. For clarity of the layout of the scheme, a specific implementation manner of the step of determining an available slicing server corresponding to the target file with the file identifier according to the current load information of the slicing server is described later.

S103, determining the available object storage device corresponding to the target file with the file identifier according to the current load information of the available slicing server and the object storage device.

In order to ensure that the file is normally uploaded, after an available slicing server is determined, because the device for storing the file is an object storage device and the object storage device has a correspondence with the slicing server, the available object storage device corresponding to the target file with the file identifier can be determined according to the available slicing server and the current load information of the object storage device. The load information of the target storage device is the load size of the target storage device or the load occupation ratio of the target storage device, but is not limited thereto.

It is emphasized that a plurality of object storage devices are included in the cloud storage system, and the object storage devices may have a correspondence with the slicing server, where the correspondence specifically refers to: the object storage device may be located within the slicing server, or the object storage device may be associated with the slicing server, and so on. If the object storage device is disposed in the slicing server, the object storage device and the slicing server have the same IP (Internet Protocol) address and the same load information.

It can be understood that there are various specific implementation manners of the step of determining the available object storage device corresponding to the target file with the file identifier according to the available slicing server and the current load information of the object storage device. For clarity of the layout of the scheme, the step of determining the available object storage device corresponding to the target file with the file identifier according to the available slice server and the current load information of the object storage device is described in the following by way of example.

And S104, sending the device identifier of the available slicing server to the client, so that the client sends the slicing file of the target file to the available slicing server based on the device identifier, and the available slicing server stores the first slicing file of the target file to the available object storage device.

The metadata server obtains the device identifier of the available slicing server after determining the available slicing server, the metadata server sends the device identifier to the client, and the client can determine which available slicing servers to send the target file after receiving the device identifier; the method comprises the steps that a client side slices a target file according to a preset slicing strategy in the process of sending the target file, and then uploads the sliced file to a slicing server; after receiving the slicing file, the slicing server calculates the original data calculation method for the slicing file to obtain a first slicing file so as to ensure that the first slicing file can recover an original target file, and then sends the first slicing file to available object storage equipment; after receiving the first slice file, the available object storage device stores the first slice file into different disks according to a preset disk selection method.

It should be noted that the device identifier is identification information that can uniquely determine an available slicing server in the server and the client, for example: IP addresses of available slicing servers, or numbers assigned by the metadata server to the available slicing servers, etc. The slicing method of the target file and the selection method of the disk belong to the prior art, and are not described in detail here.

By applying the embodiment of the invention, the metadata server reasonably distributes the slice servers and the object storage devices according to the load information of the slice servers and the object storage devices, uploads the target file, and effectively avoids the situation of file uploading failure by selecting the reasonable slice servers and the object storage devices when the server load is overlarge; in addition, by slicing the target file to be uploaded, if the sliced file is lost in a severe network environment, uploading of other sliced files is not influenced; and executing an original data calculation method in the file uploading process to ensure that the file stored by the object storage equipment is a recoverable target file to be uploaded.

As shown in fig. 2, the file slice uploading method provided in the embodiment of the present invention may further include, before the step of receiving an allocation request of a target file to be uploaded, sent by a client, the following steps:

s201, receiving an initialization request of a target file sent by a client.

It should be noted that, before sending the allocation request and uploading the target file, the client may send an initialization request to the metadata server, so as to apply for the metadata server to perform an initialization operation before uploading the target file; the initialization operation includes: storing basic information of a target file, version information of a created target file and an initial identifier of the created target file; the basic information of the target file can be attribute information of the target file or description information of the target file; the initialization request may be an HTTP message.

S202, generating an initial identifier and sending the initial identifier to the client so that the client takes the received initial identifier as the file identifier of the target file.

After the initial identifier of the target file is created, the metadata server sends the initial identifier to the client; after receiving the initial identifier, the client uses the initial identifier as a file identifier of the target file, and is used for carrying the file identifier in subsequent steps of applying for allocation and uploading of the slice file, so that the situation that the metadata server allocates a non-corresponding slice server or object storage device is avoided.

It should be noted that, when the client receives the initial identifier, the client indicates that the initialization operation of the metadata server is completed, that is, the metadata server has stored the basic information of the target file, the version information of the created target file, and the initial identifier of the target file. The metadata server stores basic information of the target file and is used for quickly indexing metadata of a slice file corresponding to the target file in the subsequent steps; and the version information of the target file is sent to the slicing server after being created, and is used by the slicing server when data calculation is carried out.

By applying the embodiment, the metadata server reasonably distributes the slice servers and the object storage devices according to the load information of the slice servers and the object storage devices, uploads the target file, and effectively avoids the situation of file uploading failure by selecting the reasonable slice servers and the object storage devices when the server load is overlarge; by slicing the target file to be uploaded, if the sliced file is lost in a severe network environment, uploading of other sliced files is not influenced; executing an original data calculation method in the file uploading process to ensure that the file stored by the object storage equipment is a recoverable target file to be uploaded; before the client sends the allocation request, the client applies for initialization, the metadata server generates an initial identifier corresponding to the target file, and sends the initial identifier to the client as the file identifier of the target file, so that the situation that the metadata server allocates a non-corresponding slice server or object storage equipment can be avoided, and the file identifiers are more convenient and occupy less memory due to uniform numbering of the metadata servers.

Optionally, in a specific implementation manner, in order to enable the slicing server to send the first slicing file to the available object storage devices, the file slicing uploading method according to the embodiment of the present invention may further include, after the step of determining the available object storage devices corresponding to the target files having the file identifiers, that:

object identifiers of the available object storage devices are sent to the available slice servers.

Accordingly, the step of sending a device identifier of an available slicing server to the client, so that the client sends the slicing file of the target file to the available slicing server based on the device identifier, so that the available slicing server stores the first slicing file of the target file to the available object storage device, comprises:

sending a device identifier of the available slicing server to the client, such that the client sends the slicing file of the target file to the available slicing server based on the device identifier, thereby causing the available slicing server to store the first slicing file of the target file to the available object storage device based on the received object identifier.

The method comprises the steps that after an available object storage device is distributed and determined by a metadata server, an object identifier of the available object storage device is obtained, the metadata server sends the object identifier to an available slicing server, and the available slicing server can determine which available object storage devices can store a first slicing file after receiving the object identifier; after receiving the first slice file, the available object storage device stores the first slice file into different disks according to a preset disk selection method.

It should be noted that the object identifier is identification information that can uniquely determine an available object storage device in the server and the storage device, for example: IP addresses of the available object storage devices, or numbers assigned by the metadata server to the available object storage devices, etc.

By applying the embodiment, when determining the available object storage device, the metadata server sends the object identifier of the available object storage device to the available slicing server, so that the available slicing server sends the first slicing file corresponding to the target file to the available object storage device, and the accuracy of file uploading is ensured.

Optionally, in a specific implementation, in addition to the metadata server sending the object identifier of the available object storage device to the available slicing server, the available slicing server may request the object identifier of the available object storage device from the metadata server after receiving the slicing file. For this implementation, the step of sending a device identifier of an available slicing server to the client, so that the client sends the slicing file of the target file to the available slicing server based on the device identifier, so that the available slicing server stores the first slicing file of the target file to the available object storage device, may include:

and sending the device identifier of the available slicing server to the client, so that the client sends the slicing file of the target file to the available slicing server based on the device identifier, so that the available slicing server requests the object identifier of the available object storage device from the metadata server, and stores the first slicing file of the target file to the available object storage device based on the requested object identifier.

By applying the embodiment, when determining the available object storage device, the metadata server stores or generates the object identifier of the available object storage device, the available slicing server applies to the metadata server for storing the first slicing file after calculating the slicing file to obtain the first slicing file, and the metadata server sends the object identifier to the available slicing server, so that the available slicing server sends the first slicing file corresponding to the target file to the available object storage device, and the accuracy of file uploading is ensured.

Optionally, in a specific implementation manner, in order to determine an available slicing server, the reasonably allocating the slicing server, where determining, according to current load information of the slicing server, the available slicing server corresponding to the target file having the file identifier may include:

acquiring load values of a plurality of slice servers, wherein the load values are at least one of the utilization rate of a central processing unit, the utilization rate of a memory and the utilization rate of input/output equipment;

judging whether a first slicing server exists in the plurality of slicing servers, wherein the load value of the first slicing server is smaller than a preset first load threshold value;

if yes, determining an available slicing server corresponding to the target file with the file identifier from the first slicing server;

The preset first load threshold is set according to the actual load condition of the slicing server, and if the load value of the slicing server is smaller than the preset first load threshold, the success rate of sending the slicing file of the target file to the slicing server is higher, the slicing file can be sent to the slicing server; if the load value of the slicing server is larger than the preset first load threshold value, the load of the slicing server is over large, and the slicing file is not suitable to be sent to the slicing server.

When determining an available slice server corresponding to a target file having a file identifier, the selection may be performed by using a random method, a round robin method, or another selection method in each of the plurality of slice servers or the plurality of first slice servers.

Optionally, in a specific implementation manner, in order to determine an available object storage device and reasonably allocate the object storage device, the determining, according to the available slicing server and the current load information of the object storage device, the available object storage device corresponding to the target file having the file identifier may include:

acquiring a network protocol IP address of an available slicing server and IP addresses of a plurality of object storage devices;

judging whether a first object storage device exists in the plurality of object storage devices, wherein the IP address of the first object storage device is the same as the IP address of an available slicing server;

If an object storage device with the same IP address as that of the available slice server exists in the object storage devices, it is indicated that the load information of the object storage device is the same as that of the available slice server, the object storage device can be directly determined to be the available object storage device, and the process of judging the load information by the object storage device is simplified; if an object storage device with the same IP address as that of the available slicing server does not exist in the plurality of object storage devices, the load information of the object storage device needs to be judged, and then the available object storage device is determined.

Optionally, in a specific implementation manner, when there is no object storage device with the same IP address as that of an available slice server among the multiple object storage devices, in order to determine an available object storage device, and reasonably perform allocation of the object storage device, the determining, from the multiple object storage devices, an available object storage device corresponding to a target file with a file identifier based on current load information of the multiple object storage devices may include:

acquiring load values of a plurality of object storage devices;

judging whether a second object storage device exists in the plurality of object storage devices, wherein the load value of the second object storage device is smaller than a preset second load threshold value;

if not, acquiring the distribution times of the plurality of object storage devices;

The preset second load threshold is set according to the actual load condition of the object storage device, and if the load value of the object storage device is smaller than the preset second load threshold, which indicates that the success rate of sending the first slice file of the target file to the object storage device is higher, the first slice file can be sent to the object storage device; if the load value of the object storage device is greater than the preset second load threshold, the allocation can be performed according to the allocation times of the object storage device, the preset allocation times threshold can be set according to historical data, the greater the allocated times, the higher the reliability of the first slice file of the object storage device storage target file is, the object storage device can be selected as the first slice file of the available object storage device storage target file.

It should be noted that, when determining the available object storage device corresponding to the target file with the file identifier, in the second object storage device or the third object storage device, a random method, a round robin method, or another selection method may be used for selection.

Optionally, in a specific implementation manner, in order to recover an original target file when a slice file of the target file is lost, so as to ensure reliability of file storage, the original data calculation method is an EC (Erasure Code) algorithm.

The EC algorithm has the specific principle that m check files are added to n original slice files, and the original slice files can be restored through any n files in n + m files. The EC algorithm comprises two processes of encoding and decoding, wherein the original n slice files are changed into n + m files which are encoded, the n + m files can be stored in different positions of the available slice files in a dispersing way, if any file smaller than m fails, the rest data can still be restored, and the restoring process is decoding. The specific process of the EC algorithm belongs to the prior art, and is not described herein.

It should be noted that only the original slice file and the check file are reserved through the EC algorithm, and when the slice file is lost, the original slice file can be restored through the EC algorithm, so that the reliability of storing the slice file is ensured, and meanwhile, the storage resources and the storage cost are reduced.

As shown in fig. 3, after the step of sending the device identifier of the available slicing server to the client, so that the client sends the slicing file of the target file to the available slicing server based on the device identifier, so that the available slicing server stores the first slicing file of the target file in the available object storage device, the file slicing uploading method provided by the embodiment of the present invention may further include:

s105, storing the first metadata sent by the available slicing server to a metadata table.

The first metadata is metadata of the first slice file; in the embodiment of the present invention, the metadata includes a disk serial number, where the disk serial number is a serial number of a disk storing the first slice file, and the disk is determined by an available object storage device according to a memory size of the first slice file.

It should be noted that the metadata table in the metadata server is used to record information that the first slice file is stored in the available object storage device, where the information includes a slice file identifier and a disk serial number of the first slice file, and when the target file needs to be called, the first slice file stored in the available object storage device can be indexed according to the slice file identifier and the disk serial number by applying to the metadata server, and the calling is performed through the index.

By applying the embodiment, the metadata server stores the first metadata to the metadata table, so that the subsequent aggregation and calling of the target file are facilitated.

As shown in fig. 4, after storing the first metadata sent by the available slice server in the metadata table, the file slice uploading method according to the embodiment of the present invention may further include the following steps:

and S106, sending a storage success instruction corresponding to the first metadata to the available slicing servers, so that the available slicing servers send an uploading ending instruction of the target file to the client after receiving the storage success instruction of the first metadata of each first slicing file.

The store successful instruction may store successful status information for the first metadata, or any instruction signal indicating that the store was successful. After the first metadata is successfully stored, the target file is uploaded and ended, and an uploading ending instruction is sent to the client so that the client executes the subsequent steps; the upload complete command may be status information that the target file has completed uploading, or any command signal indicating that the uploading has been completed.

S107, receiving an aggregation request of the target file sent by the client.

After receiving the uploading completion instruction, the client needs to send an aggregation request to the metadata server, and the aggregation request is used for applying for first metadata of a target file corresponding to an aggregation file identifier; in addition, at the same time, the client may send multiple aggregation requests to multiple target files at the same time, and in order to ensure that the first metadata aggregated by the metadata server corresponds to the target files, the aggregation requests need to carry file identifiers of the target files; the aggregated request may be an HTTP message.

S108, aggregating the first metadata corresponding to the target file in the metadata table based on the file identifier of the target file.

The first metadata corresponding to the target file in the aggregated metadata table may be the first metadata stored in an adjacent entry in the data table, or may be the first metadata stored in an entry in the data table, which is reasonable.

By applying the embodiment, the metadata information of the first sliced file is aggregated through the metadata server, and the file calling speed is increased when the file is called.

As shown in fig. 5, the file slice uploading method provided in the embodiment of the present invention may further include:

s109, receiving a termination request sent by the client.

After receiving the uploading completion instruction, the client needs to send a termination request to the metadata server for applying for metadata to release the remaining space; in addition, at the same time, the client may send multiple termination requests to multiple target files at the same time, and in order to ensure that the information deleted by the metadata server corresponds to the target files, the termination requests need to carry file identifiers of the target files; the termination request may be an HTTP message.

S110, judging whether the first metadata corresponding to the target file is aggregated. If yes, executing S111; otherwise, waiting until the aggregation is finished.

S111, deleting the file identifier of the target file.

It should be noted that, when the remaining space of the metadata server is released, the basic information of the target file should be deleted in addition to the file identifier of the target file.

By applying the embodiment, after the target file is uploaded, the residual space of the metadata server is effectively released, and the space waste of the metadata server is avoided.

The file slice uploading method provided by the embodiment of the invention is described below with reference to specific application examples.

Assuming that the client is browser software on a desktop computer, the file slice uploading process is as follows:

a user selects a video file A to upload through a browser on a desktop computer, when an upload key is clicked, a metadata server detects an upload instruction, namely, an initialization request is obtained, the metadata server allocates an initial identifier 0001 as a file identifier of the video file A, and then the file identifier 0001 is sent to the browser.

When the browser receives a file identifier 0001, namely the initialization completion flag is obtained, the browser sends an allocation request to the metadata server, and when the metadata server receives the allocation request, the metadata server allocates a slice server 1, a slice server 2, a slice server 3, an object storage device 1, an object storage device 2 and an object storage device 3 according to load information of the slice server and the object storage device, and the slice server and the object storage device are used for uploading and storing a video file A, wherein the IP address of the slice server 1 is 168.1.1.1, the IP address of the slice server 2 is 202.1.1.1, the IP address of the slice server 3 is 155.1.1, the IP address of the object storage device 1 is 168.1.1.1, the IP address of the object storage device 2 is 202.1.1.26, and the IP address of the object storage device 3 is 155.1.1.125; the metadata server transmits the IP addresses of the slicing server 1, the slicing server 2, and the slicing server 3 to the browser, and transmits the IP addresses of the object storage device 1, the object storage device 2, and the object storage device 3 to the slicing server.

The method comprises the steps that when the browser receives IP addresses of a slicing server 1, a slicing server 2 and a slicing server 3, the browser starts to upload a video file A, in the uploading process, the video file A is sliced according to a slicing method to obtain a slicing file A1, a slicing file A2 and a slicing file A3, the slicing file A1, the slicing file A2, the slicing file A3 are uploaded to the slicing server 1, the slicing server 2 and the slicing server 3 respectively, the slicing server 1, the slicing server 2 and the slicing server 3 respectively store the received slicing files into a strip cache, EC calculation is carried out, and a slicing file B1, a slicing file B2 and a slicing file B3 are obtained respectively; then, the slicing server 1, the slicing server 2 and the slicing server 3 respectively send the slicing file B1, the slicing file B2 and the slicing file B3 to the object storage device 1, the object storage device 2 and the object storage device 3, the object storage device 1, the object storage device 2 and the object storage device 3 select proper disks to store the slicing file B1, the slicing file B2 and the slicing file B3, and the serial numbers of the disks are returned to the slicing server; the slicing server sends the slicing file A1, the slicing file A2, the slicing file A3 and corresponding disk serial numbers to a metadata server, and then stores the slicing file A1, the slicing file A2, the slicing file A3 and the corresponding disk serial numbers to a metadata table; after the storage is finished, the metadata server sends a storage finishing instruction to the slicing server, and the slicing server further sends an uploading result to a browser of the desktop computer.

Compared with the prior art, the metadata server reasonably distributes the slicing servers and the object storage devices according to the load information of the slicing servers and the object storage devices, uploads the target file, and effectively avoids the situation of file uploading failure by selecting the reasonable slicing servers and the object storage devices when the server load is overlarge; in addition, by slicing the target file to be uploaded, if the sliced file is lost in a severe network environment, uploading of other sliced files is not influenced; and executing an original data calculation method in the file uploading process to ensure that the file stored by the object storage equipment is a recoverable target file to be uploaded.

Corresponding to the foregoing method embodiment, an embodiment of the present invention provides a file slice uploading apparatus, and as shown in fig. 6, the apparatus may include:

a first receiving module 610, configured to receive an allocation request of a target file to be uploaded, where the allocation request carries: file identifiers of the target files, and identification signals of available slicing servers and available object storage devices;

a first determining module 620, configured to determine, according to current load information of the slicing server, an available slicing server corresponding to the target file with the file identifier;

a second determining module 630, configured to determine, according to the available slicing servers and current load information of the object storage devices, available object storage devices corresponding to the target files with the file identifiers;

a first sending module 640, configured to send a device identifier of the available slice server to the client, so that the client sends the slice file of the target file to the available slice server based on the device identifier, so that the available slice server stores a first slice file of the target file to the available object storage device, where the first slice file is a slice file obtained by performing an original data computation on the slice file of the target file.

Further, on the basis of the first receiving module 610, the first determining module 620, the second determining module 630 and the first sending module 640, as shown in fig. 7, the file slice uploading apparatus provided in the embodiment of the present invention may further include:

a second receiving module 650, configured to receive an initialization request of the target file sent by the client;

a second sending module 660, configured to generate an initial identifier, and send the initial identifier to the client, so that the client uses the received initial identifier as a file identifier of the target file.

By applying the embodiment, the metadata server reasonably distributes the slice servers and the object storage devices according to the load information of the slice servers and the object storage devices, uploads the target file, and effectively avoids the situation of file uploading failure by selecting the reasonable slice servers and the object storage devices when the server load is overlarge; by slicing the target file to be uploaded, if the sliced file is lost in a severe network environment, uploading of other sliced files is not influenced; executing an original data calculation method in the file uploading process to ensure that the file stored by the object storage equipment is a recoverable target file to be uploaded; before the client sends the allocation request, the client applies for initialization, the metadata server generates an initial identifier corresponding to the target file, and sends the initial identifier to the client as the file identifier of the target file, so that the situation that the metadata is not allocated to the corresponding slice server or object storage equipment can be avoided, and the file identifiers are more convenient and occupy less memory due to the uniform numbering of the metadata servers.

Optionally, in order to enable the slicing server to send the first slice file to the available object storage device, the file slice uploading apparatus according to the embodiment of the present invention may further include:

a third sending module, configured to send the object identifier of the available object storage device to the available slicing server after the second determining module 630 determines the available object storage device corresponding to the target file with the file identifier;

accordingly, the first sending module 640 is specifically configured to send the device identifier of the available slicing server to the client, so that the client sends the slicing file of the target file to the available slicing server based on the device identifier, so that the available slicing server stores the first slicing file of the target file to the available object storage device based on the received object identifier.

Optionally, the first sending module 640 is specifically configured to send a device identifier of the available slicing server to the client, so that the client sends the slicing file of the target file to the available slicing server based on the device identifier, so that the available slicing server requests the metadata server for an object identifier of the available object storage device, and stores the first slicing file of the target file to the available object storage device based on the requested object identifier.

Optionally, the first determining module 620 may include:

the first obtaining submodule is used for obtaining load values of the plurality of slice servers, wherein the load values are at least one of the utilization rate of a central processing unit, the utilization rate of a memory and the utilization rate of input/output equipment;

the first judgment submodule is used for judging whether a first slice server exists in the plurality of slice servers, wherein the load value of the first slice server is smaller than a preset first load threshold value;

a first determining submodule, configured to determine, if an output of the first determining submodule is yes, an available slice server corresponding to the target file having the file identifier from the first slice server;

and the second determining submodule is used for determining an available slicing server corresponding to the target file with the file identifier from the plurality of slicing servers if the output of the first determining submodule is negative.

Optionally, the first determining sub-module may include:

a first determining unit, configured to determine, from the first slice servers, an available slice server corresponding to the target file with the file identifier by using a round robin method;

alternatively, the first and second electrodes may be,

a second determining unit, configured to determine, from the first slice servers, an available slice server corresponding to the target file with the file identifier by using a random method.

Optionally, the second determining sub-module may include:

a third determining unit, configured to determine, by using a round robin method, an available slice server corresponding to the target file having the file identifier from the plurality of slice servers;

alternatively, the first and second electrodes may be,

a fourth determining unit, configured to determine, by using a random method, an available slice server corresponding to the target file having the file identifier from the plurality of slice servers.

Optionally, the second determining module 630 may include:

a second obtaining submodule, configured to obtain a network protocol IP address of the available slice server and IP addresses of the plurality of object storage devices;

a second determining submodule, configured to determine whether a first object storage device exists in the plurality of object storage devices, where an IP address of the first object storage device is the same as an IP address of the available slice server;

a third determining submodule, configured to determine that the first object storage device is an available object storage device if an output of the second determining submodule is yes;

a fourth determining sub-module, configured to determine, if the output of the second determining sub-module is negative, an available object storage device corresponding to the target file with the file identifier from the multiple object storage devices based on current load information of the multiple object storage devices.

Optionally, the fourth determining sub-module may include:

an acquisition unit configured to acquire load values of the plurality of object storage devices;

the first judging unit is used for judging whether a second object storage device exists in the plurality of object storage devices, wherein the load value of the second object storage device is smaller than a preset second load threshold value;

a fifth determining unit, configured to determine, if the output of the first determining unit is yes, an available object storage device corresponding to the target file having the file identifier from the second object storage device;

a sixth determining unit configured to determine that the allocation times of the plurality of object storage devices are acquired if the output of the first determining unit is negative;

a second determining unit, configured to determine whether a third object storage device exists in the plurality of object storage devices, where a distribution frequency of the third object storage device is greater than a preset distribution frequency threshold;

a seventh determining unit, configured to determine, if the output of the second determining unit is yes, an available object storage device corresponding to the target file with the file identifier from the third object storage device.

Optionally, the fifth determining unit may include:

a first determining subunit, configured to determine, by using a round robin method, an available object storage device corresponding to the target file having the file identifier from the second object storage device;

alternatively, the first and second electrodes may be,

and the second determining subunit is configured to determine, from the second object storage device, an available object storage device corresponding to the target file with the file identifier by using a random method.

Optionally, in the first sending module 640, the original data calculation method is an erasure code EC algorithm.

Further, on the basis of the first receiving module 610, the first determining module 620, the second determining module 630 and the first sending module 640, as shown in fig. 8, the file slice uploading apparatus provided in the embodiment of the present invention may further include:

a storage module 670, configured to send, by the first sending module 640, a device identifier of the available slice server to the client, so that the client sends the slice file of the target file to the available slice server based on the device identifier, so that after the available slice server stores the first slice file of the target file in the available object storage device, the first metadata sent by the available slice server is stored in a metadata table, where the first metadata is metadata of the first slice file, the metadata includes a disk serial number, and the disk serial number is a serial number of a disk storing the first slice file, and the disk is determined by the available object storage device according to a memory size of the first slice file.

Further, on the basis of the first receiving module 610, the first determining module 620, the second determining module 630, the first sending module 640, and the storing module 670, as shown in fig. 9, an apparatus for uploading a file slice according to an embodiment of the present invention may further include:

a fourth sending module 680, configured to send, after the first metadata is successfully stored, a storage success instruction corresponding to the first metadata to the available slicing server, so that after the storage success instruction of the first metadata of each first slicing file is received by the available slicing server, send an upload end instruction of the target file to the client;

a third receiving module 690, configured to receive an aggregation request of the target file sent by the client, where the aggregation request carries: the file identifier of the target file and an identification signal requesting to aggregate the first metadata are sent by the client after receiving an uploading ending instruction of the target file;

an aggregating module 700, configured to aggregate, based on the file identifier of the target file, first metadata corresponding to the target file in the metadata table.

Further, on the basis of the first receiving module 610, the first determining module 620, the second determining module 630, the first sending module 640, the storing module 670, the fourth sending module 680, the third receiving module 690 and the aggregating module 700, as shown in fig. 10, the file slice uploading apparatus provided in the embodiment of the present invention may further include:

a fourth receiving module 710, configured to receive a termination request sent by the client, where the termination request carries: the file identifier of the target file and an identification signal requesting termination of uploading of the slice file are sent, wherein the termination request is sent by the client after receiving an uploading end instruction of the target file;

a deleting module 720, configured to delete the file identifier after determining that the aggregation is completed on the first metadata corresponding to the target file in the metadata table.

It is understood that, in another embodiment of the present invention, the file slice uploading device may include: a first receiving module 610, a first determining module 620, a second determining module 630, a first transmitting module 640, a second receiving module 650, a second transmitting module 660, a storing module 670, a fourth transmitting module 680, a third receiving module 690, an aggregating module 700, a fourth receiving module 710, and a deleting module 720.

In response to the above embodiments of the file slice uploading method and apparatus, as shown in fig. 11, an embodiment of the present invention further provides a cloud storage system, where the cloud storage system includes a metadata server 1110, a plurality of slice servers 1120, and a plurality of object storage devices 1130;

the slicing server is configured to: acquiring and sending the current load information of the slice server and the current load information of the object storage device corresponding to the slice server to the metadata server; and when receiving the slice file of the target file sent by the client, storing a first slice file of the target file to the available object storage device, wherein the first slice file is obtained by executing an original data calculation method on the slice file of the target file.

In the embodiment of the invention, the metadata server reasonably distributes the slicing server and the object storage equipment according to the load information of the slicing server and the object storage equipment, uploads the target file, and effectively avoids the situation of file uploading failure by selecting the reasonable slicing server and the object storage equipment when the server load is overlarge; in addition, by slicing the target file to be uploaded, if the sliced file is lost in a severe network environment, uploading of other sliced files is not influenced; and executing an original data calculation method in the file uploading process to ensure that the file stored by the object storage equipment is a recoverable target file to be uploaded.

Optionally, the metadata server is further configured to receive, before the step of receiving a distribution request of a target file to be uploaded, sent by a client, an initialization request of the target file sent by the client;

Optionally, the metadata server is further configured to send, after determining an available object storage device corresponding to the target file with the file identifier, an object identifier of the available object storage device to the available slicing server;

accordingly, the slicing server is specifically configured to store a first slice file of the target file to the available object storage device based on the received object identifier.

Optionally, the slicing server is specifically configured to request the metadata server for an object identifier of the available object storage device, and store the first sliced file of the target file to the available object storage device based on the requested object identifier.

Optionally, the metadata server is specifically configured to:

alternatively, the first and second electrodes may be,

Optionally, the metadata server is specifically configured to:

alternatively, the first and second electrodes may be,

Optionally, the metadata server is specifically configured to:

acquiring load values of the plurality of object storage devices;

if not, acquiring the distribution times of the object storage devices;

Optionally, the metadata server is specifically configured to:

alternatively, the first and second electrodes may be,

Optionally, the metadata server is further configured to store, after the available slicing server stores the first slicing file of the target file in the available object storage device, first metadata sent by the available slicing server in a metadata table, where the first metadata is metadata of the first slicing file, the metadata includes a disk serial number, the disk serial number is a serial number of a disk storing the first slicing file, and the disk is determined by the available object storage device according to a memory size of the first slicing file.

Optionally, the metadata server is further configured to: after the first metadata is successfully stored, sending a storage success instruction corresponding to the first metadata to the available slicing server; receiving an aggregation request of the target file sent by the client, wherein the aggregation request carries: the file identifier of the target file and an identification signal requesting to aggregate the first metadata are sent by the client after receiving an uploading ending instruction of the target file; aggregating first metadata corresponding to the target file in the metadata table based on the file identifier of the target file;

correspondingly, the slicing server is further configured to send an upload ending instruction of the target file to the client after receiving a storage success instruction of the first metadata of each first slicing file.

Optionally, the metadata server is further configured to:

It is noted that, herein, relational terms such as first and second, and the like may be used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Also, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising an … …" does not exclude the presence of other identical elements in a process, method, article, or apparatus that comprises the element.

All the embodiments in the present specification are described in a related manner, and the same and similar parts among the embodiments may be referred to each other, and each embodiment focuses on the differences from the other embodiments. In particular, for the system embodiment, since it is substantially similar to the method embodiment, the description is simple, and for the relevant points, reference may be made to the partial description of the method embodiment.

The above description is only for the preferred embodiment of the present invention, and is not intended to limit the scope of the present invention. Any modification, equivalent replacement, or improvement made within the spirit and principle of the present invention shall fall within the protection scope of the present invention.

Claims

1. A file slice uploading method is applied to a metadata server in a cloud storage system, the cloud storage system further comprises a plurality of slice servers and a plurality of object storage devices, and the method comprises the following steps:

sending a device identifier of the available slicing server to the client, so that the client sends the slicing file of the target file to the available slicing server based on the device identifier, so that the available slicing server stores a first slicing file of the target file to the available object storage device, wherein the first slicing file is obtained by the available slicing server performing original data calculation on the slicing file of the target file; the available object storage device is used for storing the first slice file.

2. The file slice uploading method according to claim 1, wherein the step of receiving an allocation request of a target file to be uploaded sent by a client is preceded by the method further comprising:

receiving an initialization request of the target file sent by the client;

3. The file slice uploading method of claim 1, wherein after the step of determining the available object storage device corresponding to the target file having the file identifier, the method further comprises:

4. The file slice uploading method according to claim 1, wherein the step of sending a device identifier of the available slice server to the client, so that the client sends the slice file of the target file to the available slice server based on the device identifier, so that the available slice server stores the first slice file of the target file to the available object storage device, comprises:

5. The file slice uploading method according to claim 1, wherein the determining an available slice server corresponding to the target file with the file identifier according to the current load information of the slice server includes:

6. The file slice uploading method according to claim 5, wherein the determining, from the first slice server, an available slice server corresponding to the target file having the file identifier comprises:

alternatively, the first and second electrodes may be,

7. The file slice uploading method according to claim 5, wherein the determining, from the plurality of slice servers, an available slice server to which the target file with the file identifier corresponds comprises:

alternatively, the first and second electrodes may be,

8. The file slice uploading method according to claim 1, wherein the determining, according to the current load information of the available slice servers and the object storage devices, the available object storage devices corresponding to the target files having the file identifiers comprises:

9. The file slice uploading method according to claim 8, wherein the determining, from the plurality of object storage devices, an available object storage device corresponding to the target file having the file identifier based on current load information of the plurality of object storage devices comprises:

acquiring load values of the plurality of object storage devices;

if not, acquiring the distribution times of the object storage devices;

10. The file slice uploading method according to claim 9, wherein the determining, from the second object storage device, an available object storage device corresponding to the target file with the file identifier comprises:

alternatively, the first and second electrodes may be,

11. The file slice uploading method of claim 1, wherein the original data calculation method is an erasure code EC algorithm.

12. The file slice uploading method of claim 1, wherein after the step of sending a device identifier of the available slice server to the client to cause the client to send the slice file of the target file to the available slice server based on the device identifier to cause the available slice server to store the first slice file of the target file to the available object storage device, the method further comprises:

13. The file slice uploading method according to claim 12, further comprising:

14. The file slice uploading method according to claim 13, further comprising:

15. An apparatus for uploading a file slice, the apparatus being applied to a metadata server in a cloud storage system, the cloud storage system further including a plurality of slice servers and a plurality of object storage devices, the apparatus comprising:

a first sending module, configured to send a device identifier of the available slicing server to the client, so that the client sends the slicing file of the target file to the available slicing server based on the device identifier, so that the available slicing server stores a first slicing file of the target file to the available object storage device, where the first slicing file is a slicing file obtained by the available slicing server performing original data computation on the slicing file of the target file; the available object storage device is used for storing the first slice file.

16. A cloud storage system is characterized in that the system comprises a metadata server, a plurality of slice servers and a plurality of object storage devices;

the slicing server is configured to: acquiring and sending the current load information of the slice server and the current load information of the object storage device corresponding to the slice server to the metadata server; when receiving a slice file of the target file sent by the client, storing a first slice file of the target file to the available object storage device, wherein the first slice file is obtained by the available slice server executing an original data calculation method on the slice file of the target file; the available object storage device is used for storing the first slice file.