WO2022141021A1 - 一种文件存储方法、装置、系统以及设备 - Google Patents

一种文件存储方法、装置、系统以及设备 Download PDF

Info

Publication number
WO2022141021A1
WO2022141021A1 PCT/CN2020/140642 CN2020140642W WO2022141021A1 WO 2022141021 A1 WO2022141021 A1 WO 2022141021A1 CN 2020140642 W CN2020140642 W CN 2020140642W WO 2022141021 A1 WO2022141021 A1 WO 2022141021A1
Authority
WO
WIPO (PCT)
Prior art keywords
file
storage
server
target
stored
Prior art date
Application number
PCT/CN2020/140642
Other languages
English (en)
French (fr)
Inventor
李伟
蔡亮
匡立中
张帅
王熠泽
Original Assignee
杭州趣链科技有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 杭州趣链科技有限公司 filed Critical 杭州趣链科技有限公司
Priority to PCT/CN2020/140642 priority Critical patent/WO2022141021A1/zh
Publication of WO2022141021A1 publication Critical patent/WO2022141021A1/zh

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/10—File systems; File servers
    • G06F16/13—File access structures, e.g. distributed indices
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/10—File systems; File servers
    • G06F16/17—Details of further file system functions
    • G06F16/172—Caching, prefetching or hoarding of files
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/10—File systems; File servers
    • G06F16/17—Details of further file system functions
    • G06F16/178—Techniques for file synchronisation in file systems
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/10—File systems; File servers
    • G06F16/18—File system types
    • G06F16/182—Distributed file systems

Definitions

  • the present application relates to the field of storage, and in particular, to a file storage method, apparatus, system, and device.
  • a file storage system can generally be configured for use by users such as consumers or internal personnel of the company. Users can upload files to the server of the file storage system on the client side for storage.
  • the file storage system may include a directory server and a storage server.
  • the directory server centrally processes the storage requests initiated by the client, allocates a storage server that can store the current file, and then uses the directory server to centrally process the storage requests initiated by the client.
  • the storage service stores the files uploaded by the client, and the file query and download links are similar. In this way, the file storage service of the file storage system is realized based on the index service provided by the directory server.
  • the present application provides a file storage method, device, system and device.
  • the present application provides a file storage method, the method comprising:
  • the obtained files to be stored are sent to the target storage server for storage through the target directory server, and index data between the files to be stored and the target storage server is established, wherein the target storage server is allocated by the target server from the set of storage servers,
  • a collection of storage servers includes different storage servers.
  • the present application provides a file storage device, the method comprising:
  • a determining unit configured to determine a file to be stored, wherein the file to be stored is a file requested by the client to be stored in the file storage system;
  • an allocation unit configured to allocate a corresponding target directory server for the file to be stored in the directory server set, wherein the directory server set includes different directory servers;
  • a sending unit configured to send the obtained files to be stored to a target storage server for storage through the target directory server, wherein the target storage server is allocated by the target server from a set of storage servers, and the set of storage servers includes different storage servers;
  • the establishment unit is used to establish index data between the file to be stored and the target storage server.
  • the present application further provides a file storage system, the system includes a set of directory servers and a set of storage servers, the set of target servers includes different directory servers, and the set of storage servers includes different storage servers, the system is used to execute the first aspect of the present application Or the method provided by any possible implementation manner of the first aspect of the present application.
  • the present application also provides a file storage device, comprising a processor and a memory, wherein a computer program is stored in the memory, and the processor executes the first aspect of the present application or any of the first aspect of the present application when the processor calls the computer program in the memory.
  • a file storage device comprising a processor and a memory, wherein a computer program is stored in the memory, and the processor executes the first aspect of the present application or any of the first aspect of the present application when the processor calls the computer program in the memory.
  • the present application also provides a computer-readable storage medium, where the computer-readable storage medium stores a plurality of instructions, and the instructions are suitable for the processor to load, so as to execute the first aspect of the present application or any of the first aspect of the present application.
  • a possible implementation provides the method.
  • the directory server set is the to-be-stored file. Allocate the corresponding target directory server, and then send the obtained file to be stored to the target storage server for storage through the target directory server, and establish index data between the file to be stored and the target storage server, wherein the target storage server is stored by the target server. It is allocated from the storage server set.
  • the selected target directory server is a directory server suitable for processing the indexing service of the current file to be stored, it is guaranteed that the file storage system can realize efficient indexing service. In this way, the situation of low processing efficiency of the directory server in the prior art is avoided, a more efficient file storage service is provided, and a better user experience is provided.
  • Fig. 1 is a kind of architectural schematic diagram of the file storage system of the application
  • Fig. 2 is a kind of schematic flow chart of the application file storage method
  • FIG. 3 is a schematic flowchart of allocating a target directory server for the present application
  • FIG. 4 is a schematic flowchart of the application for processing files to be stored
  • FIG. 5 is a schematic structural diagram of a file storage device of the application.
  • FIG. 6 is a schematic structural diagram of a file storage device of the present application.
  • the file storage method, device and computer-readable storage medium provided by this application can be applied to file storage devices.
  • By building a file storage system with a multi-directory server architecture it is ensured that the file storage system can achieve efficient indexing services, thus providing more efficient indexing services. Efficient file storage service.
  • the execution body may be a file storage device, or different types of file storage devices such as a server device, a physical host, or a user equipment (User Equipment, UE) integrated with the device, wherein the device may It is implemented in hardware or software.
  • the UE can be a terminal device such as a smart phone, a tablet computer, a notebook computer, a handheld computer, a desktop computer or a Personal Digital Assistant (PDA).
  • PDA Personal Digital Assistant
  • the device can also be configured in the form of a device cluster. .
  • the file storage device does not specifically refer to the storage server allocated by the file storage system for the current file to be stored and used to store the file to be stored, but refers to the device applying the file storage method of the present application.
  • the built file storage system includes two parts: indexing service and storage service, and corresponding to the indexing service, the relative Compared with the single directory server architecture in the existing file storage system, the present application introduces a multi-directory server architecture, so that a target service server suitable for processing can be allocated to the current file to be stored according to the actual situation, so as to achieve a more efficient load Balance mechanism.
  • multiple directory servers and multiple storage servers can form a corresponding server set, and for the storage server sets corresponding to different target servers, there may be independent and overlapping (As shown in FIG. 1 , the storage server set A and the storage server set B, and the storage service set C and the storage server set D are in two overlapping states), including or as a whole corresponding relationship.
  • FIG. 1 the number of directory servers and storage servers shown in FIG. 1 is only an example and should not be construed as a limitation.
  • FIG. 2 shows a schematic flowchart of the file storage method of the present application.
  • the file storage method provided by the present application may specifically include the following steps:
  • Step S201 determining a file to be stored, wherein the file to be stored is a file requested by a client to be stored in a file storage system;
  • Step S202 assigning a corresponding target directory server for the file to be stored in the directory server set, wherein the directory server set includes different directory servers;
  • Step S203 sending the obtained file to be stored to the target storage server for storage through the target directory server, and establishing index data between the file to be stored and the target storage server, wherein the target storage server is selected from the storage server set by the target server. It is assigned that the set of storage servers includes different storage servers.
  • the index data of wherein, the target storage server is allocated by the target server from the storage server set, in the storage process of this file, because the selected target directory server is suitable for processing the directory server of the index service of the current file to be stored, Therefore, it is ensured that the file storage system can realize an efficient indexing service, thus avoiding the situation of low processing efficiency of the directory server in the prior art, providing a more efficient file storage service, and having a better user experience.
  • this application describes the provided file storage method from the perspective of a file storage system.
  • the file storage method can not only be executed by a node in the file storage system, but also more likely What’s more, it is carried out by different nodes in the file storage system in a cooperative manner, and it also involves signaling interaction between different nodes to realize the cooperation between nodes.
  • the content of the steps of applying for the file storage method will generally not be repeated.
  • the client requests the files to be stored in the file storage system, mainly corresponding to two application scenarios.
  • the client has reported the storage request and carried the file to be stored in the request information of the storage request.
  • the user can log in to the user account and directly upload and store the file.
  • the file to be stored may be stored in the node that receives the storage request in the file storage system, such as a directory server or other nodes.
  • the other is that the client first reports the storage request, and after the file storage system allocates the target directory server and the target storage server, under the scheduling of the directory server, the client is instructed to upload the file to be stored to the target storage server, or the target
  • the directory server receives the to-be-stored file uploaded by the client, and forwards the to-be-stored file to the target storage server for storage.
  • determining the file to be stored in step S201 may refer to determining the file to be stored that the client will initiate a storage operation. Receiving and storing; or it may also refer to the files to be stored that have been uploaded by the client to the file storage system, and the subsequent storage policies are continued to be deployed for the files to be stored that are already in the system.
  • the file to be stored can be a file of different file types such as a document, a table, a picture, a video, an audio or a compressed package, and the file type can be adjusted according to actual needs, which is not limited here, and the file storage method of the present application is particularly It is aimed at files to be stored with larger file sizes, such as high-definition video files.
  • the file storage system can allocate an adapted target directory server or even a target storage server to it, so as to realize an adapted high-quality indexing service.
  • the allocation strategy of the target directory server can be specifically set according to different adaptation indicators.
  • the file storage system can configure directory servers in different places. After determining the files to be stored, the directory closest to the client that initiated the files to be stored can be selected. The server is used as the target directory server, or the directory server closest to the node that stores the file to be stored in the current system is selected as the target directory server;
  • the current computing resources and/or current storage resources of each directory server in the target server set may be calculated, and the amount of idle resources or the amount of free resources may be calculated.
  • the directory server with the highest proportion of idle resources is used as the target directory server;
  • the directory server with the highest transmission speed with the client that initiates the file to be stored can be selected from the directory server set as the target directory server, or the directory server with the highest transmission speed in the current system can be selected.
  • the directory server with the highest transmission speed between the nodes storing the files to be stored is used as the target directory server;
  • the allocation of the target directory server may include the following steps S3101 and S302:
  • Step S301 through the transmission test of the test data packet, evaluate the network transmission quality between the different directory servers and the client respectively;
  • a test data package can be configured, and the data package is used to test the transmission speed between the client and the directory server.
  • the data package can be the data package configured for the test , or data packets that may be transmitted between the client and the directory server in other services.
  • the file storage system can trigger the client to send the test data package to each directory server in the system, or the client side involving the application program of the file storage system can also trigger the sending of the test data to each directory server in the system by itself Bag.
  • the duration of the transmission process that is, the transmission duration
  • the reference factors that affect the network transmission quality can be compared, and the evaluation values and weights of different reference factors can be used to evaluate the current customer.
  • the network transmission quality between the client and different directory servers is used as the quantitative information of the transmission speed.
  • Step S302 selecting the directory server corresponding to the best network transmission quality as the target directory server.
  • the directory server corresponding to the best network transmission quality can be used as the target directory server, so as to achieve efficient indexing services and thus efficient file storage services.
  • the allocation strategy of the target directory server can not only adopt the above-mentioned allocation strategy, but also can configure other allocation strategies corresponding to actual conditions, which is not specifically limited here.
  • the acquired files to be stored may be sent to the target storage server for storage through the target directory server.
  • the directory server it is preset with a range of storage servers (storage server set) that can be indexed. After determining the to-be-stored files that it needs to process, the target directory server can assign targets for the to-be-stored files from the storage server set A storage server, where the target storage server stores the files to be stored.
  • the storage server with the most idle storage resources or the highest ratio of idle storage resources is generally preferred as the allocation result.
  • other allocation strategies are not excluded, which are not limited here. .
  • the processing of files to be stored by the target directory server may specifically include the following steps S401 to S403:
  • Step S401 fragment the obtained file to be stored by the target directory server to obtain a plurality of fragmented files
  • the target directory server can receive the file to be stored uploaded by the client, or retrieve the file to be stored from the node storing the file to be stored in the system, and can classify the file to be stored. or, the client has performed fragmentation processing locally, and then uploads it.
  • Fragmentation can be understood as division.
  • the file to be stored is decomposed into multiple sub-files, which are called fragmented files. After these fragmented files are combined, the original file to be stored can be obtained.
  • these sharded files are obtained by slicing the files to be stored in different sharding ranges, and these sharding ranges may be independent of each other or may overlap.
  • the range of shards can be configured by random, preset, etc., and if the overlapping mechanism of shard ranges is introduced, it is worth noting that due to the duplication of file contents between shard files, the occurrence of files can be further reduced. Content loss caused by abnormal situations such as loss, ie. If the A segment file is lost, since the B segment file contains the X file content of the A segment file, the content of the X file is obviously due to the existence of the B segment file, and no content loss is caused.
  • Step S402 sending a plurality of fragmented files to a plurality of sub-target storage servers for distributed storage;
  • the file storage system of the present application can also disperse the target storage servers storing the files to be stored into multiple storage servers (sub-storage servers), and each sub-storage service can store one or more copies. Fragmented file.
  • the target directory server may initiate distributed storage based on the communication connection between itself and a plurality of allocated sub-target storage services, and distribute the fragmented files to these sub-target storage servers.
  • the distributed storage mechanism not only the transmission speed of the files to be stored to the target storage server can be improved, but also the data security of the files to be stored in the file storage server can be further improved, and the data loss caused by the abnormality of individual storage servers can be reduced.
  • Step S403 establishing index data between multiple fragmented files and multiple sub-target storage servers.
  • the indexing service provided by the directory server can be realized through index data.
  • the index data records the correspondence between different files to be stored and different storage servers. In this way, when the current to-be-stored file is determined After the target storage service of the file is completed, the storage relationship can be reflected in the index data. If there is a query or download of the file to be stored later, the query or download request can be responded to according to the index data.
  • the index data may be configured in the form of a data table, or may be configured in other data forms, which are not specifically limited herein.
  • the storage relationship can be stored in the index data, for example, the storage relationship can be updated into the total index data, or a new index data can be created. The establishment of the current index data is reached.
  • the network transmission quality between the original target directory server and the client that initiates the query and download will vary with actual conditions. If the impact changes, for example, the network environment connected to the client is updated, the location of the client is changed, etc. At this time, the original target directory server is not the directory with the best network transmission quality between the directory server set and the current client. The server, obviously, at this time, in response to the query of the current client, the downloaded directory server can be allocated to the directory server corresponding to the best network transmission quality.
  • the directory server that receives the query and download request may not be the directory server that originally processed the files to be stored.
  • the present application can pre-store the index indicating the storage relationship between the file to be stored and the target storage server
  • the data is synchronized to other directory servers in the directory server set, so that it can be called directly in the future to ensure the high processing efficiency of the index service.
  • the directory server is abnormal or data is lost, the disaster recovery capability of the file storage system is improved.
  • this application can also introduce blockchain technology, or in other words, the file storage system is built on the basis of the blockchain platform.
  • the directory server and the storage server are both blocks.
  • the blockchain node device in the chain platform at this time, the file to be stored is locally uploaded as the data to be uploaded, so as to achieve the purpose of storing the file to be stored in the file storage system.
  • the client it may also be the blockchain node device in the blockchain platform or even included in the file storage system.
  • the client can perform fragmentation processing of the file to be stored.
  • it may also be a region The equipment outside the blockchain platform will not be described in detail here.
  • the hash value is calculated by the Hash algorithm.
  • the originator of the data sends data, it can calculate the hash value of the data through the hash algorithm, and upload the hash value to the chain (for example, to the directory server). or storage server), when the receiver of the data receives the data, it can calculate the hash value of the data through the hash algorithm.
  • the hash value calculated by the hash algorithm is unique.
  • the recipient of the data compares and finds that the hash value calculated by itself is consistent with the hash value on the chain, it can be considered that the data is in the transmission process. It has not been tampered with, has its authenticity, and is trusted data.
  • the specific calculation process of the hash value is generally to convert the corresponding data into binary data, and then use the SHA-256 function to convert the binary data into a hexadecimal string with a length of 64.
  • blockchain technology also has other data processing mechanisms to ensure data security, which will not be repeated here. Due to the transparent rules and data security features of blockchain technology, Therefore, building the file storage system provided by this application based on the blockchain platform obviously has higher practical value, not only because the blockchain working mechanism can prevent data from being tampered with during data transmission and ensure data security, but also because The system architecture and working mode disclosed in the above content of this application have higher processing efficiency and system disaster tolerance.
  • the file storage system can also provide users with query and download services, wherein, it is easy to understand that, in practical applications, the download services may include query services.
  • the download process is generally as follows:
  • the download request is used to request to download the file to be stored from the file storage system
  • the download service of the file to be stored is provided to the client that initiates the download request through the target storage server.
  • the file storage system can be configured with nodes open to users for users to initiate download requests, such as a directory server or other nodes.
  • download requests such as a directory server or other nodes.
  • a download request it can be determined that the file to be stored was originally processed through the previously configured index data.
  • the target directory server and then the target directory service responds to the download request.
  • the client since the client may have communicated with the target directory server that previously processed the files to be stored, the client can also directly initiate a download to the target directory server according to historical records (such as historical communication addresses, historical server identifiers, etc.). ask.
  • historical records such as historical communication addresses, historical server identifiers, etc.
  • the target directory server can determine the corresponding target storage server according to the preconfigured index data, and then the target storage server provides the download service to the client.
  • index data can be synchronized between directory servers. Therefore, the download request can be sent to any server that stores index data.
  • the directory service responds; alternatively, the node that receives the download request can query the directory server for the target storage server to respond, or, according to the download request, the client can feed back index data, and the client can send the index data to the target according to the download request.
  • the storage server initiates the download service.
  • Providing a download service to the client through the target storage server can be understood not only as the target storage server directly transmits the files to be stored to the client, but also as the indirect transmission of the files to be stored to the client through the forwarding of the directory server.
  • the sharded files of the mentioned sub-target storage servers are aggregated or even combined by the directory server to obtain the files to be stored, and then transmitted to the client, or each sub-target storage server directly transmits the stored sub-files to be stored to the client.
  • the client combines to obtain the original to-be-stored file.
  • a file storage system based on a blockchain platform is used as an example, combined with the sharding mechanism mentioned above, to provide an example of the processing flow of the storage service involved in the file to be stored.
  • the client requests file upload
  • the directory server shards the files uploaded by the client (obtained from the original file shards) and stores them in the storage server to form redundant backups of sharded files (one shard can be stored in multiple storage servers). Calculate the hash value of the original file and file shards, and store the certificate on the chain;
  • each storage server After each storage server receives the file fragment, it calculates the fragment hash value and compares it with the hash value of the corresponding file fragment on the chain to ensure that the received fragment files are true and consistent;
  • the directory server establishes an index directory, and stores the index directory on the chain;
  • the original directory server synchronizes the index directory to other directory servers or even storage servers, and the other directory servers compare the received hash value of the index directory with the hash value of the index directory stored on the chain to ensure the authenticity and consistency ;
  • the client initiates a download request
  • the directory server provides the index directory to the client
  • the client compares the obtained hash value of the index directory with the hash value of the index directory stored on the chain to ensure the authenticity and consistency.
  • the client obtains the sharded files from the storage server according to the index directory, each sharded file is hashed on the client side, and the obtained hash value is compared with the hash value on the chain to ensure the authenticity of the sharded file. Consistent; if there are inconsistencies in individual shard files, you can obtain the backup of the same shard file on another storage server according to the index directory, and compare the hash values again until the true consistency;
  • the client assembles the original file according to the fragmented file, calculates the hash value, and compares it with the hash value of the original file on the chain to ensure the authenticity and consistency.
  • the present application also provides a file storage device.
  • FIG. 5 is a schematic structural diagram of a file storage device of the present application.
  • the file storage device 500 is applied to a file storage system, and the file storage device 500 may specifically include the following structure:
  • a determining unit 501 configured to determine a file to be stored, wherein the file to be stored is a file requested by a client to be stored in a file storage system;
  • an allocating unit 502 configured to allocate a corresponding target directory server for the file to be stored in the directory server set, wherein the directory server set includes different directory servers;
  • the sending unit 503 is configured to send the obtained file to be stored to the target storage server for storage through the target directory server, wherein the target storage server is allocated by the target server from the storage server set, and the storage server set includes different storage servers;
  • the establishing unit 504 is configured to establish index data between the file to be stored and the target storage server.
  • the allocating unit 502 is specifically configured to:
  • the directory server corresponding to the best network transmission quality is selected as the target directory server.
  • the target storage server includes multiple sub-target storage servers, and the sending unit 503 is specifically configured to:
  • the obtained to-be-stored files are fragmented through the target directory server to obtain multiple fragmented files;
  • the apparatus further includes a synchronization unit 505 for:
  • the index data between the file to be stored and the target storage server is synchronized to other directory servers in the directory server set through the target directory server.
  • the sending unit 503 is specifically configured to:
  • the directory server receives the file to be stored uploaded by the client
  • the file storage system is built based on a blockchain platform, and different directory servers and different storage servers are different blockchain node devices in the blockchain platform.
  • the apparatus further includes a downloading unit 506 for:
  • the target storage server for storing the file to be stored is determined by the index data
  • the download service of the file to be stored is provided to the client that initiates the download request through the target storage server.
  • FIG. 6 shows a schematic structural diagram of the file storage device of the present application.
  • the file storage device of the present application includes a processor 601, a memory 602, and an input and output device 603.
  • the processor 601 When the processor 601 is used to execute the computer program stored in the memory 602, it implements each step of the file storage method in any of the embodiments corresponding to FIG. 2 to FIG. 4; or, the processor 601 is used to implement the computer program stored in the memory 602.
  • FIG. 5 corresponds to the functions of each unit in the embodiment, and the memory 602 is used for storing the computer program required by the processor 601 to execute the file storage method in any of the embodiments corresponding to FIG. 2 to FIG. 4 .
  • the computer program may be divided into one or more modules/units, and the one or more modules/units are stored in the memory 602 and executed by the processor 601 to complete the present application.
  • One or more modules/units may be a series of computer program instruction segments capable of performing specific functions, and the instruction segments are used to describe the execution process of the computer program in a computer apparatus.
  • the file storage device may include, but is not limited to, the processor 601 , the memory 602 , and the input and output device 603 .
  • the file storage device may also include a network access device, a bus, etc., and the processor 601, the memory 602, the input and output device 603, and the network access device are connected through the bus.
  • the processor 601 may be a central processing unit (Central Processing Unit, CPU), and may also be other general-purpose processors, digital signal processors (Digital Signal Processors, DSPs), application specific integrated circuits (Application Specific Integrated Circuits, ASICs), off-the-shelf processors. Field-Programmable Gate Array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc.
  • the general-purpose processor can be a microprocessor or the processor can also be any conventional processor, etc.
  • the processor is the control center of the file storage device, and uses various interfaces and lines to connect various parts of the entire device.
  • the memory 602 can be used to store computer programs and/or modules, and the processor 601 implements various functions of the computer device by running or executing the computer programs and/or modules stored in the memory 602 and calling data stored in the memory 602.
  • the memory 602 may mainly include a stored program area and a stored data area, wherein the stored program area may store an operating system, an application program required for at least one function (eg, etc.; the stored data area may store data created according to the use of the file storage device) etc.
  • the memory may include high-speed random access memory, and may also include non-volatile memory such as hard disk, internal memory, plug-in hard disk, Smart Media Card (SMC), Secure Digital (SD) ) card, flash card (Flash Card), at least one magnetic disk storage device, flash memory device, or other volatile solid state storage device.
  • non-volatile memory such as hard disk, internal memory, plug-in hard disk, Smart Media Card (SMC), Secure Digital (SD) ) card, flash card (Flash Card), at least one magnetic disk storage device, flash memory device, or other volatile solid state storage device.
  • processor 601 When the processor 601 is configured to execute the computer program stored in the memory 602, the following functions can be specifically implemented:
  • the obtained files to be stored are sent to the target storage server for storage through the target directory server, and index data between the files to be stored and the target storage server is established, wherein the target storage server is allocated by the target server from the set of storage servers,
  • a collection of storage servers includes different storage servers.
  • the present application also provides a file storage system, the system includes a set of directory servers and a set of storage servers, the set of target servers includes different directory servers, and the set of storage servers includes different storage servers, the system is used to perform any implementation corresponding to FIG. 2 to FIG. 4 .
  • the target server and the storage server in the example reference may be made to the description of the hardware structure of the file storage device in the embodiment corresponding to FIG. 6 .
  • FIG. 2 to FIG. 4 corresponding to the file storage in any embodiment.
  • the description of the method will not be repeated here.
  • the present application provides a computer-readable storage medium, in which a plurality of instructions are stored, and the instructions can be loaded by a processor to execute the file storage method in any of the embodiments corresponding to FIG. 2 to FIG. 4 of the present application.
  • a processor to execute the file storage method in any of the embodiments corresponding to FIG. 2 to FIG. 4 of the present application.
  • the specific operation refer to the description of the file storage method in any of the embodiments corresponding to FIG. 2 to FIG. 4 , which will not be repeated here.
  • the computer-readable storage medium may include: a read-only memory (Read Only Memory, ROM), a random access memory (Random Access Memory, RAM), a magnetic disk or an optical disk, and the like.
  • ROM Read Only Memory
  • RAM Random Access Memory
  • any implementation corresponding to FIGS. 2 to 4 of the present application can be implemented.
  • the beneficial effects that can be achieved by the file storage method in the example are detailed in the foregoing description, and are not repeated here.

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Data Mining & Analysis (AREA)
  • Databases & Information Systems (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)

Abstract

本申请提供了一种文件存储方法、装置、系统以及设备，通过搭建多目录服务器架构的文件存储系统，保障文件存储系统可实现高效的索引服务，如此提供更为高效的文件存储服务。方法应用于文件存储系统，包括: 确定待存储文件，其中，待存储文件为客户端请求存储于文件存储系统的文件; 在目录服务器集合中为待存储文件分配对应的目标目录服务器，其中，目录服务器集合包括文件存储系统预设的不同目录服务器; 通过目标目录服务器将获取到的待存储文件发送至目标存储服务器进行存储，并建立待存储文件与目标存储服务器之间的索引数据，其中，目标存储服务器由目标服务器从存储服务器集合中分配得到，存储服务器集合包括不同存储服务器。

Description

一种文件存储方法、装置、系统以及设备 技术领域
本申请涉及存储领域，尤其涉及一种文件存储方法、装置、系统以及设备。
背景技术
为便于文件的存储管理，一般可配置一文件存储系统，以供消费者或者公司内部人员等用户使用，用户可在客户端侧将文件上传至文件存储系统的服务器进行存储。
而在服务器侧，为方便工作的分层处理，文件存储系统具体可包括目录服务器和存储服务器，由目录服务器来集中处理客户端发起的存储请求，分配可存储当前文件的存储服务器，再由该存储服务存储客户端上传的文件，文件的查询、下载环节与此类似。如此基于目录服务器所提供的索引服务，实现文件存储系统的文件存储服务。
而在现有的相关技术的研究过程中，发明人发现，随着文件存储系统提供的文件存储服务，随着如今大文件的出现频次逐步提高，目录服务器正逐渐不堪重负，时常出现处理效率低下的情况。
技术问题
本申请提供了一种文件存储方法、装置、系统以及设备，通过搭建多目录服务器架构的文件存储系统，保障文件存储系统可实现高效的索引服务，如此避免现有技术中目录服务器出现处理效率低下的情况，提供更为高效的文件存储服务。
技术解决方案
第一方面，本申请提供了一种文件存储方法，方法包括：
确定待存储文件，其中，待存储文件为客户端请求存储于文件存储系统的文件；
在目录服务器集合中为待存储文件分配对应的目标目录服务器，其中，目录服务器集合包括不同目录服务器；
通过目标目录服务器将获取到的待存储文件发送至目标存储服务器进行 存储，并建立待存储文件与目标存储服务器之间的索引数据，其中，目标存储服务器由目标服务器从存储服务器集合中分配得到，存储服务器集合包括不同存储服务器。
第二方面，本申请提供了一种文件存储装置，方法包括：
确定单元，用于确定待存储文件，其中，待存储文件为客户端请求存储于文件存储系统的文件；
分配单元，用于在目录服务器集合中为待存储文件分配对应的目标目录服务器，其中，目录服务器集合包括不同目录服务器；
发送单元，用于通过目标目录服务器将获取到的待存储文件发送至目标存储服务器进行存储，其中，目标存储服务器由目标服务器从存储服务器集合中分配得到，存储服务器集合包括不同存储服务器；
建立单元，用于建立待存储文件与目标存储服务器之间的索引数据。
第三方面，本申请还提供了一种文件存储系统，系统包括目录服务器集合以及存储服务器集合，目标服务器集合包括不同目录服务器，存储服务器集合包括不同存储服务器，系统用于执行本申请第一方面或者本申请第一方面任一种可能的实现方式提供的方法。
第四方面，本申请还提供了一种文件存储设备，包括处理器和存储器，存储器中存储有计算机程序，处理器调用存储器中的计算机程序时执行本申请第一方面或者本申请第一方面任一种可能的实现方式提供的方法。
第五方面，本申请还提供了一种计算机可读存储介质，计算机可读存储介质存储有多条指令，指令适于处理器进行加载，以执行本申请第一方面或者本申请第一方面任一种可能的实现方式提供的方法。
有益效果
针对于文件存储场景，本申请所搭建的文件存储系统中，其部署了多目录服务器架构，当确定客户端请求存储于文件存储系统的待存储文件后，在目录服务器集合中为该待存储文件分配对应的目标目录服务器，再通过目标目录服务器将获取到的待存储文件发送至目标存储服务器进行存储，并建立待存储文件与目标存储服务器之间的索引数据，其中，目标存储服务器由目标服务器从存储服务器集合中分配得到，在该文件的存储流程中，由于所选择的目标目录 服务器为适于处理当前待存储文件的索引服务的目录服务器，因此保障文件存储系统可实现高效的索引服务，如此避免现有技术中目录服务器出现处理效率低下的情况，提供更为高效的文件存储服务，具有更佳的用户体验。
附图说明
图1为本申请文件存储系统的一种架构示意图；
图2为本申请文件存储方法的一种流程示意图；
图3为本申请分配目标目录服务器的一种流程示意图；
图4为本申请处理待存储文件的一种流程示意图；
图5为本申请文件存储装置的一种结构示意图；
图6为本申请文件存储设备的一种结构示意图。
本发明的实施方式
首先，在介绍本申请之前，先介绍下本申请关于应用背景的相关内容。
本申请提供的文件存储方法、装置以及计算机可读存储介质，可应用于文件存储设备上，通过搭建多目录服务器架构的文件存储系统，保障文件存储系统可实现高效的索引服务，如此提供更为高效的文件存储服务。
本申请提及的文件存储方法，其执行主体可以为文件存储装置，或者集成了该装置的服务器设备、物理主机或者用户设备(User Equipment，UE)等不同类型的文件存储设备，其中，装置可以采用硬件或者软件的方式实现，UE具体可以为智能手机、平板电脑、笔记本电脑、掌上电脑、台式电脑或者个人数字助理(Personal Digital Assistant，PDA)等终端设备，设备还可以以设备集群的形式配置。
其中，可以理解的是，文件存储设备，并不是特指文件存储系统为当前的待存储文件所分配的，用于存储待存储文件的存储服务器，而是指应用本申请文件存储方法的设备。
参阅图1示出的本申请文件存储系统的一种架构示意图，在本申请中，所搭建的文件存储系统，按照功能区分，包括了索引服务以及存储服务两部分，而对应于索引服务，相比于现有的文件存储系统中单一的目录服务器架构，本申请引入了多目录服务器架构，如此可根据实际情况为当前的待存储文件分配 适于处理的目标服务服务器，实现更为高效的负载均衡机制。
其中如图1所示，在文件存储系统中，多个的目录服务器以及多个的存储服务器可形成对应的服务器集合，而对于不同的目标服务器所对应的存储服务器集合，可存在相互独立、重叠(如图1中存储服务器集合A与存储服务器集合B，存储服务集合C与存储服务器集合D为两种重叠状态)、包含或者为一整体的对应关系。
此外，图1所示的目录服务器以及存储服务器的数量仅为举例，不应作为限定。
下面，开始介绍本申请提供的文件存储方法。
图2示出了本申请文件存储方法的一种流程示意图，如图2示出的，本申请提供的文件存储方法，具体可包括如下步骤：
步骤S201，确定待存储文件，其中，待存储文件为客户端请求存储于文件存储系统的文件；
步骤S202，在目录服务器集合中为待存储文件分配对应的目标目录服务器，其中，目录服务器集合包括不同目录服务器；
步骤S203，通过目标目录服务器将获取到的待存储文件发送至目标存储服务器进行存储，并建立待存储文件与目标存储服务器之间的索引数据，其中，目标存储服务器由目标服务器从存储服务器集合中分配得到，存储服务器集合包括不同存储服务器。
从上述图2所示实施例可看出，针对于文件存储场景，本申请所搭建的文件存储系统中，其部署了多目录服务器架构，当确定客户端请求存储于文件存储系统的待存储文件后，在目录服务器集合中为该待存储文件分配对应的目标目录服务器，再通过目标目录服务器将获取到的待存储文件发送至目标存储服务器进行存储，并建立待存储文件与目标存储服务器之间的索引数据，其中，目标存储服务器由目标服务器从存储服务器集合中分配得到，在该文件的存储流程中，由于所选择的目标目录服务器为适于处理当前待存储文件的索引服务的目录服务器，因此保障文件存储系统可实现高效的索引服务，如此避免现有技术中目录服务器出现处理效率低下的情况，提供更为高效的文件存储服务， 具有更佳的用户体验。
以下结合图1所示文件存储系统的架构，继续对图2所示的各步骤以及在实际应用中可能采用的具体实施方式进行详细阐述：
首先可以理解的是，本申请是以文件存储系统的角度来说明所提供的文件存储方法的，在实际应用中，该文件存储方法，不仅可由文件存储系统中的一个节点来执行，更有可能的是，由文件存储系统中的不同节点以协作的方式进行，而其中还涉及到不同节点之间的信令交互，以实现节点之间的协作，该信令交互的内容，除了涉及到本申请文件存储方法的步骤内容，一般不再展开赘述。
在实际应用中，确定客户端请求存储于文件存储系统的待存储文件，主要对应两种应用场景。
一种，是客户端已上报存储请求，并将该待存储文件携带在了该存储请求的请求信息中，例如一些网盘服务中，用户可登录用户账号并直接将文件进行上传及其存储，此时，待存储文件可保存在文件存储系统中接收到存储请求的节点，例如某一目录服务器或者其他节点。
另一种，则是客户端先上报存储请求，待文件存储系统分配目标目录服务器以及目标存储服务器后，在目录服务器的调度下，指示客户端将待存储文件上传至目标存储服务器，或者由目标目录服务器接收客户端上传的待存储文件，并将待存储文件转发至目标存储服务器进行存储。
如此可看出，步骤S201中确定待存储文件，可能指的是确定客户端将要发起存储操作的待存储文件，当文件存储系统确定该待存储文件的存储策略后，再进行该待存储文件的接收以及存储；或者也可能指的是客户端已经上传至文件存储系统的待存储文件，将已经处于系统中的待存储文件继续部署后续的存储策略。
该待存储文件，其可以为文档、表格、图片、视频、音频或者压缩包等不同文件类型的文件，其文件类型具体可随实际需要调整，在此不做限定，而本申请文件存储方法尤其针对的是较大文件大小的待存储文件，例如高清视频文件。
在确定当前的待存储文件后，文件存储系统则可为其分配适配的目标目录服务器甚至目标存储服务器，以实现适配的高质量索引服务。
在本申请的多目录服务器架构中，目标目录服务器的分配策略，具体可根据不同的适配指标进行设置。
例如，若文件存储系统面对的是较大地区范围的用户，则文件存储系统可在不同的地方配置目录服务器，当确定待存储文件后，可选择与发起待存储文件的客户端最近的目录服务器作为目标目录服务器，或者选择与当前系统中存储了待存储文件的节点最近的目录服务器作为目标目录服务器；
又例如，若文件存储系统考虑了计算资源、存储资源对目录索引服务的影响，则可计算目标服务器集合中的每个目录服务器的当前计算资源和/或当前存储资源，并将空闲资源量或者空闲资源比例最高的目录服务器作为目标目录服务器；
又例如，若文件存储系统考虑的是传输速度，则可从目录服务器集合中挑选出与发起待存储文件的客户端之间具有最高传输速度的目录服务器作为目标目录服务器，或者选择与当前系统中存储了待存储文件的节点之间具有最高传输速度的目录服务器作为目标目录服务器；
以客户端还未上传待存储文件的存储场景为例，参阅图3示出的本申请分配目标目录服务器的一种流程示意图，目标目录服务器的分配，可包括下面的步骤S3101以及步骤S302：
步骤S301，通过测试数据包的传输测试，评估不同目录服务器分别与客户端之间的网络传输质量；
对应于以传输速度为目标目录服务器的分配因素，可配置一测试数据包，该数据包用于测试客户端与目录服务器之间的传输速度，该数据包，可以为针对测试所配置的数据包，或者也可以为客户端与目录服务器之间可能在其他业务中传输的数据包。
文件存储系统，可触发客户端向系统的每个目录服务器发送该测试数据包，或者，客户端侧涉及到文件存储系统的应用程序，也可自行触发向系统的每个目录服务器发送该测试数据包。
每一次测试数据包的发送，通过采集客户端侧的发送时间点以及目录服务 器的采集时间点，则可确定传输过程所用时长，即传输时长。
其次，在传输过程中，有可能还会出现网络波动、丢包等其他的影响因素，因此，可对应影响到网络传输质量的参考因素，结合不同参考因素的评估值及其权重，评估当前客户端与不同目录服务器之间的网络传输质量，作为传输速度的量化信息。
步骤S302，将最佳网络传输质量对应的目录服务器选定为目标目录服务器。
在评估了客户端与不同目录服务器之间的网络传输质量后，则可将最佳网络传输质量对应的目录服务器作为目标目录服务器，如此，达到高效的索引服务，进而达到高效的文件存储服务。
当然，在实际应用中，目标目录服务器的分配策略，不仅可采用上述提及的分配策略，对应于实际条件，也可配置其他的分配策略，具体在此不做限定。
当分配了目标目录服务器后，则可通过目标目录服务器将获取到的待存储文件发送至目标存储服务器进行存储。
对于目录服务器，其预设有可进行索引的存储服务器范围(存储服务器集合)，在确定了自身所需处理的待存储文件后，目标目录服务器则可从存储服务器集合中为待存储文件分配目标存储服务器，由该目标存储服务器进行待存储文件的存储。
在该目标存储服务器的分配处理中，可以理解，一般优先以具有最多空闲存储资源或者最高空闲存储资源比例的存储服务器作为分配结果，当然，也不排除其他的分配策略，具体在此不做限定。
此外，可以理解的是，对于目标存储服务器，其在实际应用中，可能还存在两个甚至多个的情况，以实现在多个存储服务器都存储了待存储文件的情况，达到提高系统容灾能力的效果，避免单个存储服务器故障对系统的文件存储服务的影响。
与此同时，为进一步提高存储服务的响应效率，在本申请中，还引入了待存储文件的分片机制。
参阅图4示出的本申请处理待存储文件的一种流程示意图，在本申请中，通过目标目录服务器实现的对待存储文件的处理，具体可包括下面的步骤 S401至步骤S403：
步骤S401，通过目标目录服务器将获取到的待存储文件进行分片，得到多个分片文件；
可以理解，目标目录服务器获取到待存储文件后，例如目标目录服务器可接收客户端上传的待存储文件，或者从系统中存储有待存储文件的节点调取待存储文件，可将待存储文件进行分片；或者，客户端在本地已进行分片处理，再进行上传。
分片，可以理解为分割，将待存储文件分解成多份的子文件，称为分片文件，这些分片文件进行组合后，则可得到原来的待存储文件。
应当理解的是，这些分片文件，待存储文件在不同的分片范围下分片得到的，而这些分片范围，可能是相互独立，也可能存在重叠的情况。
其分片范围，可采用随机、预置等方式进行配置，而若引入分片范围的重叠机制，值得注意的是，还可因为分片文件之间存在文件内容重复的情况，进一步减少发生文件丢失等异常情况时所导致的内容损失，即。若A分片文件丢失，由于B分片文件存在A分片文件的X文件内容，则该X文件内容显然由于B分片文件的存在，并未造成内容损失。
步骤S402，将多个分片文件分别发送至多个子目标存储服务器进行分布式存储；
对待存储文件进行分片类似的，本申请文件存储系统也可将存储待待存储文件的目标存储服务器分散为多个存储服务器(子存储服务器)，每个子存储服务可存储一份或者多份的分片文件。
在该情况下，目标目录服务器则可基于自身与多个分配的子目标存储服务之间的通信连接，发起分布式存储，将分片文件分发至这些子目标存储服务器上。
通过该分布式存储机制，不仅可提高待存储文件发送至目标存储服务器的传输速度，此外，也可进一步提高待存储文件在文件存储服务器的数据安全性，减少个别存储服务器异常引起的数据损失。
步骤S403，建立多个分片文件与多个子目标存储服务器之间的索引数据。
目录服务器，可以理解的，其提供的索引服务，可通过索引数据实现，该 索引数据中，记载了不同的待存储文件与不同的存储服务器之间的对应关系，如此，当确定了当前待存储文件的目标存储服务后，则可在索引数据中体现出该存储关系，后续若出现了待存储文件的查询或者下载时，则可根据该索引数据，响应查询或者下载需求。
具体的，索引数据可通过数据表的方式进行配置，或者也可采用其他的数据形式进行配置，具体在此不做限定。
与此同时，当应用了分片机制以及多目标存储服务器机制时，则可将该存储关系存储于索引数据中，例如，将该存储关系更新进总的索引数据中，或者新建一索引数据，达到当前索引数据的建立。
进一步的，考虑到后续的查询、下载场景，在一些应用场景中，针对于某一待存储文件，原先的目标目录服务器与发起查询、下载的客户端之间的网络传输质量会随实际条件的影响发生变化，例如客户端所连接的网络环境更新、客户端的位置更等情况，此时，原来的目标目录服务器，并不是目录服务器集合中与当前客户端之间具有最佳网络传输质量的目录服务器，显然，此时，响应于当前客户端的查询、下载的目录服务器可分配至最佳网络传输质量对应的目录服务器。
或者，考虑到接收到查询、下载请求的目录服务器，可能不是原先处理待存储文件的目录服务器。
此时，为节省当前目录服务器从原先处理待存储文件的目录服务器处调取相关的索引数据的所需成本，本申请则可预先将指示待存储文件与目标存储服务器之间的存储关系的索引数据同步至目录服务器集合中的其他目录服务器上，如此，后续则可直接调用，保障索引服务的高处理效率，同时，在该同步机制下，也可进一步提高索引数据的数据安全性，避免单个目录服务器异常或者数据丢失的情况发生，提高文件存储系统的容灾能力。
后续的，为进一步提高数据安全，本申请还可引入区块链技术，或者说，文件存储系统是在区块链平台的基础上搭建得到的，对此，目录服务器以及存储服务器都是区块链平台中的区块链节点设备，此时，本地将待存储文件作为待上链数据进行上链，达到将待存储文件存储至文件存储系统的目的。
此外，对于所称的客户端，也可能是区块链平台中的区块链节点设备甚至 包含于文件存储系统中，例如可由客户端进行待存储文件的分片处理，当然，也可能是区块链平台外的设备，具体在此不再赘述。
在该应用场景下，容易理解的是，基于区块链技术，待存储文件、子待存储文件、索引数据等上述内容涉及的数据，在区块链平台中进行传输时，可涉及到基于哈希(Hash)算法计算哈希值的处理，数据的发起方在发送数据时，可通过哈希算法计算该数据的哈希值，并将该哈希值进行上链(例如上链至目录服务器或者存储服务器)，数据的接收方接收到该数据时，可通过哈希算法计算该数据的哈希值，由于哈希算法是一种只可加密不可解密的算法，因此可以认为某一数据通过哈希算法所计算得到的哈希值是具有唯一性的，此时，若数据的接收方比对发现自身计算的哈希值与链上的哈希值一致，则可认为数据在传输过程中未经过篡改、具有其真实性，为可信数据。
举例而言，其哈希值的具体计算过程，一般为将相应数据转换为二进制数据，再将二进制数据使用SHA-256函数，转换为一个长度为64的十六进制字符串。
当然，除了哈希算法实现的校验机制，区块链技术还存在其他保证数据安全的数据处理机制，具体在此不再赘述，而由于区块链技术存在的规则透明、数据安全等特点，因此基于区块链平台搭建本申请所提供的文件存储系统，显然具有更高的实用价值，不仅由于区块链工作机制可防止数据传输过程中数据遭到篡改、保证了数据安全性，且由于本申请在上述内容揭露的系统架构以及工作模式，具有更高的处理效率以及系统容灾能力。
而在完成待存储文件的存储后，文件存储系统还可为用户提供查询、下载服务，其中，容易理解的是，在实际应用中，下载服务可包含查询服务。
以文件存储系统提供的文件下载服务为例，其下载流程一般为：
获取下载请求，其中，下载请求用于请求从文件存储系统中下载待存储文件；
通过索引数据确定存储待存储文件的目标存储服务器；
通过目标存储服务器向发起下载请求的客户端提供待存储文件的下载服务。
具体的，文件存储系统可配置向用户开放的节点，以供用户发起下载请求， 例如目录服务器或者其他节点，而当获取到下载请求时，通过原先配置的索引数据，可确定原先处理待存储文件的目标目录服务器，再由该目标目录服务对该下载请求进行响应。
当然，由于客户端之前可能已经与原先处理待存储文件的目标目录服务器进行通信后，因此，客户端也可直接根据历史记录(例如历史通信地址、历史服务器标识等)，向目标目录服务器发起下载请求。
目标目录服务器在响应下载请求的过程中，可根据预先配置的索引数据确定对应的目标存储服务器，再由该目标存储服务器向客户端提供下载服务。
需要说明的是，该下载场景仅为举例，在实际应用中，如前文提及到，索引数据是可在目录服务器之间同步的，因此，可将下载请求交由任意一个存储有索引数据的目录服务进行响应；或者，也可由接收到下载请求的节点向目录服务器查询目标存储服务器，以进行响应，或者，也可根据下载请求，向客户端反馈索引数据，由客户端根据索引数据向目标存储服务器发起下载服务。
而通过目标存储服务器向客户端提供下载服务，不仅可以理解为目标存储服务器向客户端直接传输待存储文件，也可理解为通过目录服务器的转发，向客户端间接传输待存储文件，例如将上述提及的各子目标存储服务器的分片文件由目录服务器进行汇总甚至组合得到待存储文件，再传输至客户端，或者各子目标存储服务器将存储的子待存储文件直接传输至客户端，由客户端组合得到原来的待存储文件。
下面，为便于说明，则以基于区块链平台搭建得到的文件存储系统为例，结合上述提及的分片机制，提供一待存储文件所涉及存储服务的处理流程实例。
1.客户端请求文件上传；
2.测试网络，选择属于最佳上传节点的目录服务器；
3.目录服务器将客户端上传的文件分片(原文件分片得到)，存储至存储服务器，形成分片文件的冗余备份(一个分片可在多个存储服务器中存储)，同时目录服务器将原文件、文件分片均计算哈希值，并上链存证；
4.各存储服务器接收到文件分片后，计算分片哈希值，并与链上的对应文件分片的哈希值进行比对，确保接收到的分片文件真实一致；
5.目录服务器建立索引目录，将索引目录上链存证；
6.原目录服务器将索引目录同步至其他目录服务器甚至存储服务器中，其他目录服务器将接收到的索引目录的哈希值与链上存证的索引目录的哈希值进行比对，确保真实一致；
7.客户端发起下载请求；
8.选取目录服务器，目录服务器提供索引目录给客户端；
9.客户端把获取到的索引目录的哈希值与链上存证的索引目录的哈希值进行比对，确保真实一致。
10.客户端根据索引目录从存储服务器获取分片文件，每个分片文件在客户端进行哈希计算，得到的哈希值与链上的哈希值进行比对，确保分片文件的真实一致；若存在个别分片文件不一致，则可根据索引目录获取同一个分片文件在另一个存储服务器上的备份，再次进行哈希值的比对，直至真实一致；
11.客户端根据分片文件组合出原文件，计算哈希值，并与链上原文件的哈希值进行比对，确保真实一致。
以上是本申请文件存储方法的说明，为便于更好的实施本申请提供的文件存储方法，本申请还提供了文件存储装置。
参阅图5，图5为本申请文件存储装置的一种结构示意图，在本申请中，文件存储装置500应用于文件存储系统，文件存储装置500具体可包括如下结构：
确定单元501，用于确定待存储文件，其中，待存储文件为客户端请求存储于文件存储系统的文件；
分配单元502，用于在目录服务器集合中为待存储文件分配对应的目标目录服务器，其中，目录服务器集合包括不同目录服务器；
发送单元503，用于通过目标目录服务器将获取到的待存储文件发送至目标存储服务器进行存储，其中，目标存储服务器由目标服务器从存储服务器集合中分配得到，存储服务器集合包括不同存储服务器；
建立单元504，用于建立待存储文件与目标存储服务器之间的索引数据。
在一种示例性的实现方式中，分配单元502，具体用于：
通过测试数据包的传输测试，评估不同目录服务器分别与客户端之间的网络传输质量；
将最佳网络传输质量对应的目录服务器选定为目标目录服务器。
在又一种示例性的实现方式中，目标存储服务器包括多个子目标存储服务器，发送单元503，具体用于：
通过目标目录服务器将获取到的待存储文件进行分片，得到多个分片文件；
将多个分片文件分别发送至多个子目标存储服务器进行分布式存储；
建立单元，具体用于：
建立多个分片文件与多个子目标存储服务器之间的索引数据。
在又一种示例性的实现方式中，装置还包括同步单元505，用于：
通过目标目录服务器将待存储文件与目标存储服务器之间的索引数据，同步至目录服务器集合中的其他目录服务器。
在又一种示例性的实现方式中，发送单元503，具体用于：
由目录服务器接收客户端上传的待存储文件；
通过目标服务器将待存储文件发送至目标存储服务器进行存储。
在又一种示例性的实现方式中，文件存储系统是基于区块链平台搭建的，不同目录服务器以及不同存储服务器皆为区块链平台中的不同区块链节点设备。
在又一种示例性的实现方式中，装置还包括下载单元506，用于：
接收下载请求，其中，下载请求用于请求从文件存储系统中下载待存储文件；
由索引数据确定存储待存储文件的目标存储服务器；
通过目标存储服务器向发起下载请求的客户端提供待存储文件的下载服务。
本申请还提供了文件存储设备，参阅图6，图6示出了本申请文件存储设备的一种结构示意图，具体的，本申请文件存储设备包括处理器601、存储器602以及输入输出设备603，处理器601用于执行存储器602中存储的计算机 程序时实现如图2至图4对应任意实施例中文件存储方法的各步骤；或者，处理器601用于执行存储器602中存储的计算机程序时实现如图5对应实施例中各单元的功能，存储器602用于存储处理器601执行上述图2至图4对应任意实施例中文件存储方法所需的计算机程序。
示例性的，计算机程序可以被分割成一个或多个模块/单元，一个或者多个模块/单元被存储在存储器602中，并由处理器601执行，以完成本申请。一个或多个模块/单元可以是能够完成特定功能的一系列计算机程序指令段，该指令段用于描述计算机程序在计算机装置中的执行过程。
文件存储设备可包括，但不仅限于处理器601、存储器602、输入输出设备603。本领域技术人员可以理解，示意仅仅是文件存储设备的示例，并不构成对文件存储设备的限定，可以包括比图示更多或更少的部件，或者组合某些部件，或者不同的部件，例如文件存储设备还可以包括网络接入设备、总线等，处理器601、存储器602、输入输出设备603以及网络接入设备等通过总线相连。
处理器601可以是中央处理单元(Central Processing Unit，CPU)，还可以是其他通用处理器、数字信号处理器(Digital Signal Processor，DSP)、专用集成电路(Application Specific Integrated Circuit，ASIC)、现成可编程门阵列(Field-Programmable Gate Array，FPGA)或者其他可编程逻辑器件、分立门或者晶体管逻辑器件、分立硬件组件等。通用处理器可以是微处理器或者该处理器也可以是任何常规的处理器等，处理器是文件存储设备的控制中心，利用各种接口和线路连接整个设备的各个部分。
存储器602可用于存储计算机程序和/或模块，处理器601通过运行或执行存储在存储器602内的计算机程序和/或模块，以及调用存储在存储器602内的数据，实现计算机装置的各种功能。存储器602可主要包括存储程序区和存储数据区，其中，存储程序区可存储操作系统、至少一个功能所需的应用程序(比如等；存储数据区可存储根据文件存储设备的使用所创建的数据等。此外，存储器可以包括高速随机存取存储器，还可以包括非易失性存储器，例如硬盘、内存、插接式硬盘，智能存储卡(Smart Media Card，SMC)，安全数字(Secure Digital，SD)卡，闪存卡(Flash Card)、至少一个磁盘存储器件、 闪存器件、或其他易失性固态存储器件。
处理器601用于执行存储器602中存储的计算机程序时，具体可实现以下功能：
确定待存储文件，其中，待存储文件为客户端请求存储于文件存储系统的文件；
在目录服务器集合中为待存储文件分配对应的目标目录服务器，其中，目录服务器集合包括不同目录服务器；
通过目标目录服务器将获取到的待存储文件发送至目标存储服务器进行存储，并建立待存储文件与目标存储服务器之间的索引数据，其中，目标存储服务器由目标服务器从存储服务器集合中分配得到，存储服务器集合包括不同存储服务器。
本申请还提供一种文件存储系统，该系统包括目录服务器集合以及存储服务器集合，目标服务器集合包括不同目录服务器，存储服务器集合包括不同存储服务器，系统用于执行如图2至图4对应任意实施例中文件存储方法，目标服务器以及存储服务器的硬件结构说明，可参阅图6对应实施例中有关文件存储设备的硬件结构的说明内容。
所属领域的技术人员可以清楚地了解到，为描述的方便和简洁，上述描述的文件存储装置、设备及其相应单元的具体工作过程，可以参考如图2至图4对应任意实施例中文件存储方法的说明，具体在此不再赘述。
本领域普通技术人员可以理解，上述实施例的各种方法中的全部或部分步骤可以通过指令来完成，或通过指令控制相关的硬件来完成，该指令可以存储于一计算机可读存储介质中，并由处理器进行加载和执行。
为此，本申请提供一种计算机可读存储介质，其中存储有多条指令，该指令能够被处理器进行加载，以执行本申请如图2至图4对应任意实施例中文件存储方法中的步骤，具体操作可参考如图2至图4对应任意实施例中文件存储方法的说明，在此不再赘述。
其中，该计算机可读存储介质可以包括：只读存储器(Read Only Memory，ROM)、随机存取记忆体(Random Access Memory，RAM)、磁盘或光盘等。
由于该计算机可读存储介质中所存储的指令，可以执行本申请如图2至图 4对应任意实施例中文件存储方法中的步骤，因此，可以实现本申请如图2至图4对应任意实施例中文件存储方法所能实现的有益效果，详见前面的说明，在此不再赘述。
以上对本申请提供的文件存储方法、装置、设备以及计算机可读存储介质进行了详细介绍，本文中应用了具体个例对本申请的原理及实施方式进行了阐述，以上实施例的说明只是用于帮助理解本申请的方法及其核心思想；同时，对于本领域的技术人员，依据本申请的思想，在具体实施方式及应用范围上均会有改变之处，综上所述，本说明书内容不应理解为对本申请的限制。

Claims (17)

  1. 一种文件存储方法，其特征在于，所述方法包括：
    确定待存储文件，其中，所述待存储文件为客户端请求存储于所述文件存储系统的文件；
    在目录服务器集合中为所述待存储文件分配对应的目标目录服务器，其中，所述目录服务器集合包括不同目录服务器；
    通过所述目标目录服务器将获取到的所述待存储文件发送至目标存储服务器进行存储，并建立所述待存储文件与所述目标存储服务器之间的索引数据，其中，所述目标存储服务器由所述目标服务器从存储服务器集合中分配得到，所述存储服务器集合包括不同存储服务器。
  2. 根据权利要求1所述的方法，其特征在于，所述在目录服务器集合中为所述待存储文件分配对应的目标目录服务器，包括：
    通过测试数据包的传输测试，评估所述不同目录服务器分别与所述客户端之间的网络传输质量；
    将最佳网络传输质量对应的目录服务器选定为所述目标目录服务器。
  3. 根据权利要求1所述的方法，其特征在于，所述目标存储服务器包括多个子目标存储服务器，所述通过所述目标目录服务器将获取到的所述待存储文件发送至目标存储服务器进行存储，并建立所述待存储文件与所述目标存储服务器之间的索引数据，包括：
    通过所述目标目录服务器将获取到的所述待存储文件进行分片，得到多个分片文件；
    将多个所述分片文件分别发送至多个所述子目标存储服务器进行分布式存储；
    建立多个所述分片文件与多个所述子目标存储服务器之间的索引数据。
  4. 根据权利要求1所述的方法，其特征在于，所述方法还包括：
    通过所述目标目录服务器将所述待存储文件与所述目标存储服务器之间的索引数据，同步至所述目录服务器集合中的其他目录服务器。
  5. 根据权利要求1所述的方法，其特征在于，所述通过所述目标目录服 务器将获取到的所述待存储文件发送至目标存储服务器进行存储，包括：
    由所述目录服务器接收所述客户端上传的所述待存储文件；
    通过所述目标服务器将所述待存储文件发送至所述目标存储服务器进行存储。
  6. 根据权利要求1所述的方法，其特征在于，所述文件存储系统是基于区块链平台搭建的，所述不同目录服务器以及所述不同存储服务器皆为所述区块链平台中的不同区块链节点设备。
  7. 根据权利要求1所述的方法，其特征在于，所述方法还包括：
    接收下载请求，其中，所述下载请求用于请求从所述文件存储系统中下载所述待存储文件；
    由所述索引数据确定存储所述待存储文件的目标存储服务器；
    通过所述目标存储服务器向发起所述下载请求的客户端提供所述待存储文件的下载服务。
  8. 一种文件存储装置，其特征在于，所述方法包括：
    确定单元，用于确定待存储文件，其中，所述待存储文件为客户端请求存储于所述文件存储系统的文件；
    分配单元，用于在目录服务器集合中为所述待存储文件分配对应的目标目录服务器，其中，所述目录服务器集合包括不同目录服务器；
    发送单元，用于通过所述目标目录服务器将获取到的所述待存储文件发送至目标存储服务器进行存储，其中，所述目标存储服务器由所述目标服务器从存储服务器集合中分配得到，所述存储服务器集合包括不同存储服务器；
    建立单元，用于建立所述待存储文件与所述目标存储服务器之间的索引数据。
  9. 根据权利要求8所述的装置，其特征在于，所述分配单元，具体用于：
    通过测试数据包的传输测试，评估所述不同目录服务器分别与所述客户端之间的网络传输质量；
    将最佳网络传输质量对应的目录服务器选定为所述目标目录服务器。
  10. 根据权利要求8所述的装置，其特征在于，所述目标存储服务器包括多个子目标存储服务器，所述发送单元，具体用于：
    通过所述目标目录服务器将获取到的所述待存储文件进行分片，得到多个分片文件；
    将多个所述分片文件分别发送至多个所述子目标存储服务器进行分布式存储；
    所述建立单元，具体用于：
    建立多个所述分片文件与多个所述子目标存储服务器之间的索引数据。
  11. 根据权利要求8所述的装置，其特征在于，所述装置还包括同步单元，用于：
    通过所述目标目录服务器将所述待存储文件与所述目标存储服务器之间的索引数据，同步至所述目录服务器集合中的其他目录服务器。
  12. 根据权利要求8所述的装置，其特征在于，所述发送单元，具体用于：
    由所述目录服务器接收所述客户端上传的所述待存储文件；
    通过所述目标服务器将所述待存储文件发送至所述目标存储服务器进行存储。
  13. 根据权利要求8所述的装置，其特征在于，所述文件存储系统是基于区块链平台搭建的，所述不同目录服务器以及所述不同存储服务器皆为所述区块链平台中的不同区块链节点设备。
  14. 根据权利要求8所述的装置，其特征在于，所述装置还包括下载单元，用于：
    接收下载请求，其中，所述下载请求用于请求从所述文件存储系统中下载所述待存储文件；
    由所述索引数据确定存储所述待存储文件的目标存储服务器；
    通过所述目标存储服务器向发起所述下载请求的客户端提供所述待存储文件的下载服务。
  15. 一种文件存储系统，其特征在于，所述系统包括目录服务器集合以及存储服务器集合，所述目标服务器集合包括不同目录服务器，所述存储服务器集合包括不同存储服务器，所述系统用于执行如权利要求1至7任一项所述的方法。
  16. 一种文件存储设备，其特征在于，包括处理器和存储器，所述存储器 中存储有计算机程序，所述处理器调用所述存储器中的计算机程序时执行如权利要求1至7任一项中所述的方法。
  17. 一种计算机可读存储介质，其特征在于，所述计算机可读存储介质存储有多条指令，所述指令适于处理器进行加载，以执行权利要求1至7任一项所述的方法。
PCT/CN2020/140642 2020-12-29 2020-12-29 一种文件存储方法、装置、系统以及设备 WO2022141021A1 (zh)

Priority Applications (1)

Application Number Priority Date Filing Date Title
PCT/CN2020/140642 WO2022141021A1 (zh) 2020-12-29 2020-12-29 一种文件存储方法、装置、系统以及设备

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
PCT/CN2020/140642 WO2022141021A1 (zh) 2020-12-29 2020-12-29 一种文件存储方法、装置、系统以及设备

Publications (1)

Publication Number Publication Date
WO2022141021A1 true WO2022141021A1 (zh) 2022-07-07

Family

ID=82259906

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2020/140642 WO2022141021A1 (zh) 2020-12-29 2020-12-29 一种文件存储方法、装置、系统以及设备

Country Status (1)

Country Link
WO (1) WO2022141021A1 (zh)

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN115826865A (zh) * 2022-12-07 2023-03-21 安世数擎(杭州)信息科技服务有限公司 一种适用于大规模跨域机群文件的管理系统及方法
CN116107978A (zh) * 2023-04-12 2023-05-12 北京尽微致广信息技术有限公司 一种文件导出方法及装置、存储介质及电子设备
CN116628728A (zh) * 2023-07-24 2023-08-22 江苏华存电子科技有限公司 一种基于特征感知的数据存储分析方法及系统

Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN104731516A (zh) * 2013-12-18 2015-06-24 腾讯科技（深圳）有限公司 一种存取文件的方法、装置及分布式存储系统
CN109302346A (zh) * 2018-10-25 2019-02-01 网宿科技股份有限公司 一种传输数据流量的方法和装置
CN111049884A (zh) * 2019-11-18 2020-04-21 武汉方始科技有限公司 一种分布式大文件存储系统及文件上传和下载方法

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN104731516A (zh) * 2013-12-18 2015-06-24 腾讯科技（深圳）有限公司 一种存取文件的方法、装置及分布式存储系统
CN109302346A (zh) * 2018-10-25 2019-02-01 网宿科技股份有限公司 一种传输数据流量的方法和装置
CN111049884A (zh) * 2019-11-18 2020-04-21 武汉方始科技有限公司 一种分布式大文件存储系统及文件上传和下载方法

Cited By (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN115826865A (zh) * 2022-12-07 2023-03-21 安世数擎(杭州)信息科技服务有限公司 一种适用于大规模跨域机群文件的管理系统及方法
CN116107978A (zh) * 2023-04-12 2023-05-12 北京尽微致广信息技术有限公司 一种文件导出方法及装置、存储介质及电子设备
CN116628728A (zh) * 2023-07-24 2023-08-22 江苏华存电子科技有限公司 一种基于特征感知的数据存储分析方法及系统
CN116628728B (zh) * 2023-07-24 2023-11-14 江苏华存电子科技有限公司 一种基于特征感知的数据存储分析方法及系统

Similar Documents

Publication Publication Date Title
WO2022141021A1 (zh) 一种文件存储方法、装置、系统以及设备
US11297140B2 (en) Point of presence based data uploading
CN107332876B (zh) 区块链状态的同步方法及装置
CN112650721A (zh) 一种文件存储方法、装置、系统以及设备
US10681127B2 (en) File upload method and system
TWI528191B (zh) File Handling Method Based on Cloud Storage, System and Server Cluster System
WO2019075978A1 (zh) 数据传输方法、装置、计算机设备和存储介质
US10129152B2 (en) Setting method, server device and service chain system
US10333780B2 (en) Method, apparatus and computer program product for updating load balancer configuration data
KR20120018178A (ko) 객체 저장부들의 네트워크상의 스웜-기반의 동기화
WO2020253634A1 (zh) 一种数据处理方法、系统及装置
CN111464661B (zh) 负载均衡方法、装置、代理设备、缓存设备及服务节点
WO2017215646A1 (zh) 数据传输方法和装置
US10129358B2 (en) Partitioned serialized caching and delivery of large files
US20140143339A1 (en) Method, apparatus, and system for resource sharing
CN112261094A (zh) 一种报文处理方法及代理服务器
WO2016180188A1 (zh) 分发建链方法、装置和系统
US11394800B2 (en) Systems and methods for remote network topology discovery
US20230102617A1 (en) Repeat transaction verification method, apparatus, and device, and medium
US20210368006A1 (en) Request response method, device, and system applied to bit torrent system
US11444998B2 (en) Bit rate reduction processing method for data file, and server
US20160323371A1 (en) Methods and apparatus for load balancing in a network
US10897506B2 (en) Managing port connections
EP3369008B1 (en) Randomized peer-to-peer synchronization of shared content items
CN111064785B (zh) 资源包下载方法、装置和系统

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 20967371

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 20967371

Country of ref document: EP

Kind code of ref document: A1