CN112698783A - Object storage method, device and system - Google Patents
Object storage method, device and system Download PDFInfo
- Publication number
- CN112698783A CN112698783A CN201911015492.0A CN201911015492A CN112698783A CN 112698783 A CN112698783 A CN 112698783A CN 201911015492 A CN201911015492 A CN 201911015492A CN 112698783 A CN112698783 A CN 112698783A
- Authority
- CN
- China
- Prior art keywords
- storage
- server
- storage server
- stored
- target
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 81
- 238000004891 communication Methods 0.000 claims description 10
- 238000010586 diagram Methods 0.000 description 5
- 230000008569 process Effects 0.000 description 5
- 230000000694 effects Effects 0.000 description 4
- 230000003993 interaction Effects 0.000 description 4
- 238000005516 engineering process Methods 0.000 description 2
- 230000006870 function Effects 0.000 description 2
- 238000013507 mapping Methods 0.000 description 2
- 238000012545 processing Methods 0.000 description 2
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000005540 biological transmission Effects 0.000 description 1
- 238000004590 computer program Methods 0.000 description 1
- 238000000802 evaporation-induced self-assembly Methods 0.000 description 1
- 230000014509 gene expression Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 230000004044 response Effects 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0602—Interfaces specially adapted for storage systems specifically adapted to achieve a particular effect
- G06F3/061—Improving I/O performance
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0628—Interfaces specially adapted for storage systems making use of a particular technique
- G06F3/0638—Organizing or formatting or addressing of data
- G06F3/064—Management of blocks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0628—Interfaces specially adapted for storage systems making use of a particular technique
- G06F3/0638—Organizing or formatting or addressing of data
- G06F3/0643—Management of files
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0668—Interfaces specially adapted for storage systems adopting a particular infrastructure
- G06F3/067—Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS]
Abstract
The invention provides an object storage method, device and system, relating to the technical field of storage, wherein the method comprises the steps of receiving a storage request of an object to be stored; selecting a target storage server from the alternative storage servers according to a preset rule; and storing the object to be stored in the target storage server. In the object storage operation, the control module in the object storage system is bypassed, and the uploading request of the user is directly forwarded to the storage server, so that the object storage is realized, the control resource can be saved, and the writing efficiency is improved.
Description
Technical Field
The present invention relates to the field of storage technologies, and in particular, to a method, an apparatus, and a system for storing an object.
Background
A distributed object storage system typically includes a user agent module, a control module, and a storage module. The user agent module is used for receiving and forwarding user requests (such as uploading, downloading and deleting of object data) to the corresponding storage module. The storage module is used for storing object data uploaded by a user and generally comprises a plurality of storage servers, wherein each storage server is physically divided into a plurality of fixed-length block files, each block file is provided with a plurality of copies, and each copy is located in a different storage server. In addition, only one master copy is selected from the multiple copies of the block file, and the rest are all slave copies. In addition, the control module is mainly used for managing the mapping between the copies of the block files to the storage servers.
At present, a common object storage method is: when the user agent module receives an object uploading request, the user agent module firstly inquires the control module, the control module selects a non-full block file in the storage module, and returns a master copy address and a slave copy address of the block file; and then, the user agent module sends the data to the master copy, the master copy forwards the data to the slave copies after the master copy succeeds in writing, and a writing success prompt is returned to the user agent module after all the slave copies succeed in writing. For the object storage mode, interaction with the control module is needed during each uploading operation, the control module selects the written block file, and if the interaction amount is large, the control module needs to process one by one, so that limited control resources are occupied more, and the writing efficiency is influenced.
Disclosure of Invention
In view of this, an object of the present invention is to provide an object storage method, apparatus and system, which can bypass a control device in an object storage system and directly forward an upload request of a user to a storage server to implement storage of an object, thereby saving control resources and improving write efficiency.
In a first aspect, an embodiment of the present invention provides an object storage method, where the method is applied to a user agent server in a distributed object storage system; the user agent server is connected with the storage server in the system in a communication way; the method comprises the following steps: receiving a storage request of an object to be stored; selecting a target storage server from the alternative storage servers according to a preset rule; and storing the object to be stored in the target storage server.
In a preferred embodiment of the present invention, the predetermined rule includes a weight of the storage server; the step of selecting a target storage server from the alternative storage servers according to the preset rule includes: and determining the storage server with the largest weight as the target storage server in the alternative storage servers.
In a preferred embodiment of the present invention, the weight of the storage server includes a remaining storage space of the storage server.
In a preferred embodiment of the present invention, the alternative storage servers are pre-stored in a storage server list; the storage server list is generated by the following method: acquiring the IP address and the residual storage space of each storage server in the object storage system according to a preset time interval; and saving the IP address and the residual storage space as a storage server list.
In a preferred embodiment of the present invention, the step of storing the object to be stored in the target storage server includes: randomly selecting a block file with a main role as a main and auxiliary file from the block files of the target storage server; and writing the object to be stored into the primary copy file.
In a preferred embodiment of the present invention, after the step of writing the object to be stored into the main and auxiliary files, the method further includes: and if the object to be stored is successfully written into the main copy file, forwarding the object to be stored to the slave copy file corresponding to the main copy file so as to write the object to be stored into the slave copy file.
In a preferred embodiment of the present invention, the slave copy file is located on a storage server other than the target storage server, and the number of the slave copy files is at least one.
In a second aspect, an embodiment of the present invention further provides an object storage apparatus, where the apparatus is applied to a user agent server in a distributed object storage system; the user agent server is connected with the storage server in the system in a communication way; the device includes: the storage request receiving module is used for receiving a storage request of an object to be stored; the target storage server selection module is used for selecting a target storage server from the alternative storage servers according to a preset rule; and the target storage server storing module is used for storing the object to be stored into the target storage server.
In a preferred embodiment of the present invention, the predetermined rule includes a weight of the storage server; the target storage server selection module is further configured to: and determining the storage server with the largest weight as the target storage server in the alternative storage servers.
In a preferred embodiment of the present invention, the target storage server storing module is further configured to: randomly selecting a block file with a main role as a main and auxiliary file from the block files of the target storage server; and writing the object to be stored into the primary copy file.
In a third aspect, an embodiment of the present invention further provides an object storage system, which includes a processor and a memory, where the memory stores computer-executable instructions that can be executed by the processor, and the processor executes the computer-executable instructions to implement the object storage method.
In a fourth aspect, embodiments of the present invention also provide a computer-readable storage medium storing computer-executable instructions that, when invoked and executed by a processor, cause the processor to implement the object storage method described above.
The embodiment of the invention has the following beneficial effects:
the embodiment of the invention provides an object storage method, device and system, which comprises the steps of firstly receiving a storage request of an object to be stored; then, selecting a target storage server from the alternative storage servers according to a preset rule; and storing the object to be stored in the target storage server. The method bypasses the control equipment in the object storage system, and directly forwards the uploading request of the user to the storage server, so as to realize the storage of the object, reduce the control burden of the control equipment in the uploading operation, further save the control resource and improve the writing efficiency.
Additional features and advantages of the disclosure will be set forth in the description which follows, or in part may be learned by the practice of the above-described techniques of the disclosure, or may be learned by practice of the disclosure.
In order to make the aforementioned objects, features and advantages of the present disclosure more comprehensible, preferred embodiments accompanied with figures are described in detail below.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below, and it is obvious that the drawings in the following description are some embodiments of the present invention, and other drawings can be obtained by those skilled in the art without creative efforts.
Fig. 1 is a schematic view of an application scenario of object storage according to an embodiment of the present invention;
fig. 2 is a schematic flowchart of an object storage method according to an embodiment of the present invention;
fig. 3 is a schematic flowchart of another object storage method according to an embodiment of the present invention;
fig. 4 is a schematic flowchart of another object storage method according to an embodiment of the present invention;
fig. 5 is a schematic flowchart of another object storage method according to an embodiment of the present invention;
FIG. 6 is a schematic structural diagram of an object storage device according to an embodiment of the present invention;
fig. 7 is a schematic structural diagram of an object storage system according to an embodiment of the present invention.
Icon: 100-a distributed object storage system; 110-a user agent server; 120-a control device; 130-a storage device; 200-a client; 61-storage request receiving module; 62-target storage server selection module; 63-target storage server storing module; 71-a processor; 72-a memory; 73-bus; 74 — communication interface.
Detailed Description
To make the objects, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings, and it is apparent that the described embodiments are some, but not all embodiments of the present invention. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
The distributed storage system is to store data on a plurality of independent storage servers in a distributed manner, and to perform communication and data transmission among the independent devices through a network. In a distributed object storage system, all data of a user is understood by the system as one object, wherein one object may be a picture, a video file, an audio file, or a Word document, etc.
Referring to fig. 1, which is a schematic diagram illustrating an application scenario of object storage, it can be seen from fig. 1 that the distributed object storage system 100 includes a user agent server 110, a control device 120, and a storage device 130, which are connected to each other. Wherein, the client 200 interacts with the distributed object storage system 100 through the user agent server 110, and realizes uploading of the object data to the storage device 130 in the distributed object storage system 100, or downloading of the object data from the storage device 130 to the client 200, and so on.
In the distributed object storage system 100 shown in FIG. 1, the storage appliance 130 is comprised of a plurality of storage servers, where each storage server is physically partitioned into a plurality of fixed-length block files (chunk files), each block file has three replicas, including one master replica and two slave replicas, and each replica is located on a different storage server. In actual operation, the user agent server 110 is configured to receive and forward a data operation request of the client 200, the storage device 130 is configured to store object data uploaded by the client 200, and the control device 120 is configured to manage mapping between a copy of the block file and the storage server. Moreover, in the existing object storage technology, each time of the upload operation, the user agent server 110 needs to interact with the control device 120, and the control device 120 selects the block file to be written, and if the interaction amount is large, the control device 120 needs to process one by one, which occupies more limited control resources and affects the writing efficiency.
Based on this, the object storage method, device and system provided by the embodiments of the present invention may be applied to various object storage scenarios, such as data backup, data archiving, cloud disk storage, and the like.
To facilitate understanding of the embodiment, a detailed description will be first given of an object storage method disclosed in the embodiment of the present invention.
Fig. 2 is a flow chart illustrating an object storage method, which is applied to a user agent server in a distributed object storage system, and the user agent server is communicatively connected to a storage server in the system. As can be seen from fig. 2, the object storage method includes the steps of:
step S202: a storage request for an object to be stored is received.
Here, the object to be stored may be a picture, text, video, audio, and the like. The storage request is generally issued by a user, for example, the user may issue a storage request through a client to request to store the object to be stored, where the client may be a mobile phone, a notebook computer, a computer, or the like.
In one embodiment, if the object storage method is applied to the distributed object storage system shown in fig. 1, the storage request is received by a user agent server in the storage system. The user agent server is used for receiving and forwarding a data operation request of a user.
Step S204: and selecting a target storage server from the alternative storage servers according to a preset rule.
The number of the above-mentioned alternative storage servers may be one or more, and usually, more than one storage server is arranged in one object storage system. And the storage server is used for storing the object data uploaded by the user. In this embodiment, the target storage server represents a server selected to store the object to be stored.
After receiving the storage request, a target storage server needs to be selected from the alternative storage servers, wherein a rule is preset to select the target storage server. By the preset rule, the target storage server can be definitely screened out. For example, the preset rule may be to designate a certain server as a target storage server, and assume that the alternative storage servers include: server a, server B, and server C, the target storage server may be designated as server B.
For another example, the preset rule may be alternately stored in sequence, and the above alternative storage server still includes: taking server a, server B and server C as examples, arranging in the order of a-B-C, if the last object to be stored is stored in server B, the current target server is server C, and the next target server is server a, and circulating in this logic.
In other possible embodiments, the preset rule may be other settings, and may be flexibly set according to actual needs. In summary, according to the preset rule, the target storage server can be explicitly selected from the alternative storage servers.
In addition, still taking the application scenario shown in fig. 1 as an example, the operation of selecting the target storage server in step S204 and the operation of receiving the storage request in step S202 are all implemented in the user agent server, and the control device is not required to select the target storage server.
Step S206: and storing the object to be stored in the target storage server.
Through the steps, the target storage server is selected, so that the object to be stored can be stored in the target storage server. Taking the application scenario shown in fig. 1 as an example, after selecting a target storage server, the user agent server forwards the object to be stored to the target storage server, and the target storage server performs a storage operation on the object to be stored.
Compared with the prior art, in the object storage method provided by this embodiment, in response to the upload operation of the user, the control device in the object storage system is completely bypassed, the control device is not required to participate in selecting the storage server of the object to be stored, but the user agent server selects the target storage server according to the preset rule, and directly forwards the upload request of the user to the target storage server, so as to store the object to be stored, thereby reducing the control burden of the control device in the upload operation. For the control device in the object storage system, since only a master server and a slave server are generally arranged, the control burden of the control device is reduced, and the limited control resources are released.
The object storage method provided by the embodiment of the invention comprises the steps of firstly receiving a storage request of an object to be stored; then, selecting a target storage server from the alternative storage servers according to a preset rule; and storing the object to be stored in the target storage server. The method bypasses the control equipment in the object storage system, and directly forwards the uploading request of the user to the storage server to realize the storage of the object, thereby saving the control resource and improving the writing efficiency.
On the basis of the object storage method shown in fig. 2, this embodiment further provides another object storage method, where the method mainly describes a specific implementation process of selecting a target storage server from alternative storage servers according to a preset rule, as shown in fig. 3, which is a flowchart of the method and can be seen from fig. 3, the method includes the following steps:
step S302: a storage request for an object to be stored is received.
Step S304: and determining the storage server with the largest weight as the target storage server in the alternative storage servers.
In this embodiment, the preset rule includes a weight size of the storage server. In at least one possible embodiment, the weight of the storage server may be the remaining storage space, that is, the weight of the storage server is determined according to the size of the remaining storage space of the storage server. In this embodiment, the larger the remaining storage space, e.g., the larger the disk remaining space, the greater the weight determined for the storage server. In this way, among the candidate storage servers, the storage server having the largest remaining storage space has the largest corresponding weight, and is thus determined as the target storage server.
The alternative storage servers may be stored in a storage server list in advance. Wherein the storage server list may be generated by the following steps (10) - (11):
(10) and acquiring the IP address and the residual storage space of each storage server in the object storage system according to a preset time interval.
Here, taking the application scenario shown in fig. 1 as an example, the IP address and the remaining storage space of each storage server in the object storage system may be obtained through interaction between the user agent server and the control device. Here, the preset time interval may be set according to actual needs, and may be several minutes, or a longer or shorter time interval. For example, it may be set that the user agent server and the control device interact once every 5 minutes to obtain the IP address and the remaining storage space of each storage server in the current object storage system.
(11) And saving the IP address and the residual storage space as a storage server list.
In this embodiment, the acquired IP address and the remaining storage space of each storage server are saved in a list, and the storage server list is saved in the memory.
In another possible embodiment, the weight of the storage server may also be a failure occurrence rate of the server, and the storage server with the smaller failure occurrence rate is considered to have the larger weight, so that the smaller failure occurrence rate, that is, the storage server with the more stable operation, has the larger weight. Thus, in this embodiment, the storage server with the smallest failure occurrence rate is determined as the target storage server because the weight corresponding to the storage server is the largest.
In another possible embodiment, the weight of the storage server may also be the access activity of the storage server, for example, the storage server with higher access activity may be set, and the corresponding weight is larger, so that in this embodiment, the storage server with the highest access activity corresponds to the largest weight and is thus determined as the target storage server.
Step S306: and storing the object to be stored in the target storage server.
After the target storage server is selected, the object to be stored can be stored in the target storage server.
In this way, the object storage method provided by this embodiment provides multiple implementation modes of the preset rule, and realizes selecting the target storage server from the alternative storage servers.
On the basis of the object storage method shown in fig. 2, this embodiment further provides another object storage method, where the method mainly describes a specific implementation process of storing an object to be stored in the target storage server, as shown in fig. 4, which is a flowchart of the method, as can be seen from fig. 4, the method includes the following steps:
step S402: a storage request for an object to be stored is received.
Step S404: and selecting a target storage server from the alternative storage servers according to a preset rule.
Step S406: and randomly selecting one block file with the main role as a main and auxiliary file from the block files of the target storage server.
Here, a block file, i.e., a block file, is usually physically divided into a plurality of block files of fixed length for each storage server, wherein the block file is generally of GB class, and each block file has a unique block identifier (block id).
Typically, there are multiple copies of each tile file, each copy located on a different storage server, and the data of each copy is identical. Furthermore, each copy usually includes multiple objects, where for a small file, a file is an object, and for a large file, the file needs to be split into multiple small objects at the business layer. Within the block file, there is one object identification (object id) for each object, and there is no gap between each object.
In addition, for a block file, only one copy of all the copies of the block file has the master role, namely the master copy, and the other copies have the slave roles and are slave copies. In actual operation, when a user uploads an object, a master copy of a selected block file needs to be written in, and after the master copy is successfully written in, a slave copy of the block file is written in.
In this embodiment, after the target storage server is selected, one tile file with a main role is randomly selected from the tile files of the target storage server as a main and auxiliary file. Here, among the plurality of block files on each storage server, some of the block files have a master role and some of the block files have a slave role. In one possible implementation, a block file may be selected from the block files having a main role in the local block file list of the target storage server by randomly selecting the block file, and the selected block file may be used as the main and auxiliary files.
Step S408: and writing the object to be stored into the primary copy file.
And then writing the object to be stored into the randomly selected primary copy file. In one embodiment, the object to be stored may be additionally written at the end of the block file.
Here, it is assumed that the tile file of the target storage server has only one copy, and the copy is already determined as the primary and secondary files through step S406, so that the uploading operation is completed after the object to be stored is written into the primary and secondary files.
In the object storage method provided by this embodiment, one tile file is selected as a primary and secondary file from the tile files with a primary role in the target storage server in a random manner, and the object to be stored is stored in the primary and secondary files. According to the method, the control equipment does not need to select the target storage server or select the master copy file, the control equipment is completely bypassed, the object is uploaded and stored, and therefore precious control resources are saved.
On the basis of the object storage method shown in fig. 4, this embodiment further provides another object storage method, where the method mainly introduces a specific implementation process of writing the object to be stored into the secondary copy file, as shown in fig. 5, which is a flowchart of the method, as can be seen from fig. 5, the method includes the following steps:
step S502: a storage request for an object to be stored is received.
Step S504: and selecting a target storage server from the alternative storage servers according to a preset rule.
Step S506: and randomly selecting one block file with the main role as a main and auxiliary file from the block files of the target storage server.
Step S508: and writing the object to be stored into the primary copy file.
Step S510: and judging whether the object to be stored is successfully written into the main and auxiliary files. If yes, go to step S512; if not, step S514 is performed.
In this embodiment, the tile file of the target storage server corresponds to at least two copies, and after the main and auxiliary files are determined in step S506, the other copies are slave copy files, where the number of the slave copy files is at least one, and the slave copy files are located on storage servers other than the target storage server.
After the operation of writing the object to be stored into the main and auxiliary files in step S508, it is determined whether the object to be stored is successfully written into the main and auxiliary files: if so, forwarding the object to be stored to a slave copy file corresponding to the master copy file so as to write the object to be stored into the slave copy file; and if not, returning write failure information to the user.
Step S512: and forwarding the object to be stored to the slave copy file corresponding to the master copy file so as to write the object to be stored into the slave copy file.
Here, the storage server where the master copy file is located forwards the object to be stored to the storage server where the slave copy file is located. And if a plurality of slave copy files exist, forwarding the object to be stored to the storage server where each slave copy file is located. In one embodiment, after receiving the object to be stored from the storage server where the replica file is located, the storage server also additionally writes the storage object at the end of the block file.
In at least one possible implementation, when an object to be stored is successfully written into all the slave replica files, information of successful writing is returned to a user, and a unique identification of the object in the storage system is returned, wherein the unique identification comprises a block identification (block id) and an object identification (object id).
Step S514: and returning write failure information to the user.
If the object to be stored is not successfully written into the main and auxiliary files, the storage operation fails, at this time, write failure information is returned to the user, for example, "write failure", "write error" or "Failed" information can be returned to the user, so that the user can take corresponding operations in time.
The object storage method provided by this embodiment introduces an implementation manner of writing an object to be stored into a master-slave file and a slave-slave file, which is closer to an application scenario of actual uploading operation.
Corresponding to the object storage method, this embodiment further provides an object storage device, as shown in fig. 6, which is a schematic structural diagram of the object storage device, as can be seen from fig. 6, the device includes a storage request receiving module 61, a target storage server selecting module 62, and a target storage server storing module 63, which are connected in sequence, where functions of the modules are as follows:
a storage request receiving module 61, configured to receive a storage request of an object to be stored;
a target storage server selection module 62, configured to select a target storage server from the candidate storage servers according to a preset rule;
and a target storage server storing module 63, configured to store the object to be stored in the target storage server.
The object storage device firstly receives a storage request of an object to be stored; then, selecting a target storage server from the alternative storage servers according to a preset rule; and storing the object to be stored in the target storage server. The device bypasses the control equipment in the object storage system, and directly forwards the uploading request of the user to the storage server, so that the object is stored, the control burden of the control equipment in the uploading operation is reduced, the control resource is saved, and the writing efficiency is improved.
In one possible implementation, the preset rule includes a weight size of the storage server; the target storage server selection module 62 is further configured to: and determining the storage server with the largest weight as the target storage server in the alternative storage servers.
In another possible implementation manner, the target storage server storing module 63 is further configured to: randomly selecting a block file with a main role as a main and auxiliary file from the block files of the target storage server; and writing the object to be stored into the primary copy file.
In another possible embodiment, the preset rule includes a weight size of the storage server; the target storage server selecting module 62 is further configured to: and determining the storage server with the largest weight as the target storage server in the alternative storage servers.
In another possible embodiment, the weight of the storage server includes a remaining storage space of the storage server.
In another possible implementation, the alternative storage servers are pre-stored in a storage server list; the target storage server selecting module 62 is further configured to: acquiring the IP address and the residual storage space of each storage server in the object storage system according to a preset time interval; and saving the IP address and the residual storage space as a storage server list.
In another possible implementation, the apparatus further includes a write-from-copy file module configured to: and if the object to be stored is successfully written into the main copy file, forwarding the object to be stored to the slave copy file corresponding to the main copy file so as to write the object to be stored into the slave copy file.
In another possible embodiment, the slave copy file is located on a storage server other than the target storage server, and the number of the slave copy files is at least one.
The object storage device provided in the embodiment of the present invention has the same implementation principle and technical effect as those of the foregoing object storage method embodiment, and for brief description, reference may be made to corresponding contents in the foregoing object storage method embodiment for the part of the embodiment of the object storage device that is not mentioned.
An embodiment of the present invention further provides an object storage system, as shown in fig. 7, which is a schematic structural diagram of the object storage system, where the object storage system includes a processor 71 and a memory 72, the memory 72 stores machine executable instructions that can be executed by the processor 71, and the processor 71 executes the machine executable instructions to implement the object storage method.
In the embodiment shown in fig. 7, the object storage system further comprises a bus 73 and a communication interface 74, wherein the processor 71, the communication interface 74 and the memory 72 are connected by the bus.
The Memory 72 may include a high-speed Random Access Memory (RAM) and may also include a non-volatile Memory (non-volatile Memory), such as at least one disk Memory. The communication connection between the network element of the system and at least one other network element is realized through at least one communication interface 74 (which may be wired or wireless), and the internet, a wide area network, a local network, a metropolitan area network, etc. may be used. The bus may be an ISA bus, PCI bus, EISA bus, or the like. The bus may be divided into an address bus, a data bus, a control bus, etc. For ease of illustration, only one double-headed arrow is shown in FIG. 7, but this does not indicate only one bus or one type of bus.
The processor 71 may be an integrated circuit chip having signal processing capabilities. In implementation, the steps of the above method may be performed by integrated logic circuits of hardware or instructions in the form of software in the processor 71. The Processor 71 may be a general-purpose Processor, and includes a Central Processing Unit (CPU), a Network Processor (NP), and the like; the device can also be a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), a Field-Programmable Gate Array (FPGA), or other Programmable logic devices, discrete Gate or transistor logic devices, discrete hardware components. The various methods, steps and logic blocks disclosed in the embodiments of the present invention may be implemented or performed. A general purpose processor may be a microprocessor or the processor may be any conventional processor or the like. The steps of the method disclosed in connection with the embodiments of the present invention may be directly implemented by a hardware decoding processor, or implemented by a combination of hardware and software modules in the decoding processor. The software module may be located in ram, flash memory, rom, prom, or eprom, registers, etc. storage media as is well known in the art. The storage medium is located in a memory, and the processor 71 reads information in the memory 72 and completes the steps of the object storage method of the foregoing embodiment in combination with hardware thereof.
Embodiments of the present invention further provide a machine-readable storage medium, where the machine-readable storage medium stores machine-executable instructions, and when the machine-executable instructions are called and executed by a processor, the machine-executable instructions cause the processor to implement the object storage method, and specific implementation may refer to the foregoing embodiments of the object storage method, and is not described herein again.
The object storage method, the object storage device, and the computer program product of the object storage system provided in the embodiments of the present invention include a computer-readable storage medium storing a program code, where instructions included in the program code may be used to execute the object storage method described in the foregoing method embodiments, and specific implementations may refer to the method embodiments and are not described herein again.
Unless specifically stated otherwise, the relative steps, numerical expressions, and values of the components and steps set forth in these embodiments do not limit the scope of the present invention.
The functions, if implemented in the form of software functional units and sold or used as a stand-alone product, may be stored in a non-volatile computer-readable storage medium executable by a processor. Based on such understanding, the technical solution of the present invention may be embodied in the form of a software product, which is stored in a storage medium and includes instructions for causing a computer device (which may be a personal computer, a server, or a network device) to execute all or part of the steps of the method according to the embodiments of the present invention. And the aforementioned storage medium includes: a U-disk, a removable hard disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a magnetic disk or an optical disk, and other various media capable of storing program codes.
Finally, it should be noted that: the above-mentioned embodiments are only specific embodiments of the present invention, which are used for illustrating the technical solutions of the present invention and not for limiting the same, and the protection scope of the present invention is not limited thereto, although the present invention is described in detail with reference to the foregoing embodiments, those skilled in the art should understand that: any person skilled in the art can modify or easily conceive the technical solutions described in the foregoing embodiments or equivalent substitutes for some technical features within the technical scope of the present disclosure; such modifications, changes or substitutions do not depart from the spirit and scope of the embodiments of the present invention, and they should be construed as being included therein. Therefore, the protection scope of the present invention shall be subject to the protection scope of the appended claims.
Claims (10)
1. An object storage method is characterized in that the method is applied to a user agent server in a distributed object storage system; the user agent server is in communication connection with a storage server in the system; the method comprises the following steps:
receiving a storage request of an object to be stored;
selecting a target storage server from the alternative storage servers according to a preset rule;
and storing the object to be stored in the target storage server.
2. The object storage method according to claim 1, wherein the preset rule includes a weight size of the storage server;
the step of selecting a target storage server from the alternative storage servers according to a preset rule includes:
and determining the storage server with the largest weight as the target storage server in the alternative storage servers.
3. The object storage method of claim 2, wherein the weight of the storage server comprises a remaining storage space of the storage server.
4. The object storage method according to claim 3, wherein the alternative storage servers are pre-stored in a storage server list;
the storage server list is generated by the following method:
acquiring the IP address and the residual storage space of each storage server in the object storage system according to a preset time interval;
and saving the IP address and the residual storage space as the storage server list.
5. The object storage method according to claim 1, wherein the step of storing the object to be stored in the target storage server comprises:
randomly selecting a block file with a main role as a main and auxiliary file from the block files of the target storage server;
and writing the object to be stored into the main and auxiliary files.
6. The object storage method according to claim 5, wherein after the step of writing the object to be stored into the main-auxiliary file, the method further comprises:
and if the object to be stored is successfully written into the main copy file and the auxiliary copy file, forwarding the object to be stored to the auxiliary copy file corresponding to the main copy file so as to write the object to be stored into the auxiliary copy file.
7. The object storage method according to claim 6, wherein the slave replica file is located on a storage server other than the target storage server, and the number of the slave replica files is at least one.
8. An object storage apparatus, wherein the apparatus is applied to a user agent server in a distributed object storage system; the user agent server is in communication connection with a storage server in the system; the device comprises:
the storage request receiving module is used for receiving a storage request of an object to be stored;
the target storage server selection module is used for selecting a target storage server from the alternative storage servers according to a preset rule;
and the target storage server storing module is used for storing the object to be stored into the target storage server.
9. An object storage system comprising a processor and a memory, the memory storing computer-executable instructions executable by the processor, the processor executing the computer-executable instructions to implement the object storage method of any one of claims 1 to 7.
10. A computer-readable storage medium having stored thereon computer-executable instructions that, when invoked and executed by a processor, cause the processor to implement the object storage method of any of claims 1 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911015492.0A CN112698783A (en) | 2019-10-22 | 2019-10-22 | Object storage method, device and system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911015492.0A CN112698783A (en) | 2019-10-22 | 2019-10-22 | Object storage method, device and system |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN112698783A true CN112698783A (en) | 2021-04-23 |
Family
ID=75505370
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201911015492.0A Pending CN112698783A (en) | 2019-10-22 | 2019-10-22 | Object storage method, device and system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112698783A (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113311999A (en) * | 2021-06-01 | 2021-08-27 | 中国电信股份有限公司河南分公司 | Object storage device and method based on intelligent campus middleboxes |
| CN113806300A (en) * | 2021-09-23 | 2021-12-17 | 北京百度网讯科技有限公司 | Data storage method, system, device, equipment and storage medium |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102882983A (en) * | 2012-10-22 | 2013-01-16 | 南京云创存储科技有限公司 | Rapid data memory method for improving concurrent visiting performance in cloud memory system |
| CN106649401A (en) * | 2015-11-03 | 2017-05-10 | 阿里巴巴集团控股有限公司 | Data writing method and device of distributed file system |
| CN110188084A (en) * | 2019-05-16 | 2019-08-30 | 浙江讯盟科技有限公司 | A kind of distributed file storage system and file memory method |
-
2019
- 2019-10-22 CN CN201911015492.0A patent/CN112698783A/en active Pending
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102882983A (en) * | 2012-10-22 | 2013-01-16 | 南京云创存储科技有限公司 | Rapid data memory method for improving concurrent visiting performance in cloud memory system |
| CN106649401A (en) * | 2015-11-03 | 2017-05-10 | 阿里巴巴集团控股有限公司 | Data writing method and device of distributed file system |
| CN110188084A (en) * | 2019-05-16 | 2019-08-30 | 浙江讯盟科技有限公司 | A kind of distributed file storage system and file memory method |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113311999A (en) * | 2021-06-01 | 2021-08-27 | 中国电信股份有限公司河南分公司 | Object storage device and method based on intelligent campus middleboxes |
| CN113806300A (en) * | 2021-09-23 | 2021-12-17 | 北京百度网讯科技有限公司 | Data storage method, system, device, equipment and storage medium |
| CN113806300B (en) * | 2021-09-23 | 2023-08-01 | 北京百度网讯科技有限公司 | Data storage method, system, device, equipment and storage medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| JP6876806B2 (en) | Blockchain Consensus Formation Methods and Devices | |
| JP6605078B2 (en) | Information processing method, information processing apparatus, server, and computer-readable recording medium | |
| CN109995813B (en) | Partition expansion method, data storage method and device | |
| CN109656873B (en) | Block chain-based data archiving method and device and terminal equipment | |
| US20150149658A1 (en) | Software upgrade of routers | |
| WO2016155635A1 (en) | Data processing method and device | |
| CN106031127B (en) | Method and apparatus for management of applications | |
| WO2018121456A1 (en) | Data storage method, server and storage system | |
| CN107179940B (en) | Task execution method and device | |
| US10771358B2 (en) | Data acquisition device, data acquisition method and storage medium | |
| US20170185454A1 (en) | Method and Electronic Device for Determining Resource Consumption of Task | |
| CN105786539B (en) | File downloading method and device | |
| CN112698783A (en) | Object storage method, device and system | |
| CN110989934B (en) | Block chain link point data storage method, block chain system and block chain node | |
| CN111694639A (en) | Method and device for updating address of process container and electronic equipment | |
| CN113467719B (en) | Data writing method and device | |
| US10379743B1 (en) | Offloaded delete operations | |
| CN112631994A (en) | Data migration method and system | |
| CN111046004B (en) | Data file storage method, device, equipment and storage medium | |
| CN112764897B (en) | Task request processing method, device and system and computer readable storage medium | |
| CN111405313B (en) | Method and system for storing streaming media data | |
| CN114138475A (en) | Data transmission load balancing method, device, equipment and storage medium | |
| WO2016176045A1 (en) | System, method, and apparatus for updating data in a distributed storage system | |
| CN109151016B (en) | Flow forwarding method and device, service system, computing device and storage medium | |
| KR101858970B1 (en) | Method and apparatus for sharing common editing documents |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |