CN108733822A - A kind of file memory method, device, electronic equipment and storage medium - Google Patents

A kind of file memory method, device, electronic equipment and storage medium Download PDF

Info

Publication number
CN108733822A
CN108733822A CN201810495187.5A CN201810495187A CN108733822A CN 108733822 A CN108733822 A CN 108733822A CN 201810495187 A CN201810495187 A CN 201810495187A CN 108733822 A CN108733822 A CN 108733822A
Authority
CN
China
Prior art keywords
file
unit
data block
storage unit
file management
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN201810495187.5A
Other languages
Chinese (zh)
Inventor
夏正伟
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Beijing Hundred Enlightenment Technology Co Ltd
Original Assignee
Beijing Hundred Enlightenment Technology Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Beijing Hundred Enlightenment Technology Co Ltd filed Critical Beijing Hundred Enlightenment Technology Co Ltd
Priority to CN201810495187.5A priority Critical patent/CN108733822A/en
Publication of CN108733822A publication Critical patent/CN108733822A/en
Pending legal-status Critical Current

Links

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L67/00—Network arrangements or protocols for supporting network services or applications
    • H04L67/01—Protocols
    • H04L67/06—Protocols specially adapted for file transfer, e.g. file transfer protocol [FTP]

Landscapes

  • Engineering & Computer Science (AREA)
  • Computer Networks & Wireless Communication (AREA)
  • Signal Processing (AREA)
  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)

Abstract

A kind of file memory method of offer of the embodiment of the present invention, device, electronic equipment and storage medium, method are applied to the customer unit in one-of-a-kind system, and one-of-a-kind system includes multiple customer units, multiple file storage units and multiple first file management units；Method includes：One file upload request is sent to first object file management unit, first object file management unit is any one in multiple first file management units；When the instruction for receiving the transmission of first object file management unit allows the first response message that file uploads, the first data upload requests are sent to first object file management unit；When receiving the second response message of first object file management unit transmission, the first data block is uploaded in first object file storage unit；The information of the first object file storage unit for storing the first data block is carried in second response message.The embodiment of the present invention realizes multi-user's write-in and reduces request time delay.

Description

A kind of file memory method, device, electronic equipment and storage medium
Technical field
The present embodiments relate to technical field of memory more particularly to a kind of file memory method, device, electronic equipment and Storage medium.
Background technology
Hadoop distributed file systems (Hadoop Distributed File System, HDFS) are designed to suitable Operate in the distributed file system on common hardware (commodity hardware).HDFS has the characteristics of high fault tolerance, And it is designed to be deployed on cheap hardware.In addition, HDFS is capable of providing the data that high-throughput carrys out access application, It is suitble to the application program of super large data set.But HDFS exists simultaneously following deficiency while having the advantages that above：
First, being not suitable for low latency data access.HDFS is for handling large data set analysis task, in order to reach high number According to handling capacity, high latency may result in.
Second, not supporting multi-user to be written.Only there are one writers for a file in HDFS, and write operation can only be in text Part end is completed, i.e., can only execute additional operation.
In conclusion HDFS has that time delay is higher and multi-user cannot be supported to be written in the prior art.
Invention content
A kind of file memory method of offer of the embodiment of the present invention, device, electronic equipment and storage medium, to solve existing skill The problem of HDFS time delays are higher in art and multi-user cannot be supported to be written.
In view of the above-mentioned problems, in a first aspect, the embodiment of the present invention provides a kind of file memory method, applied to one-of-a-kind system In a customer unit, wherein the one-of-a-kind system includes multiple customer units, is deposited for carrying out multiple files of file storage Storage unit and for storing metadata and for carrying out multiple first file management units of file management；The method packet It includes：
One file upload request is sent to first object file management unit, wherein the first object file management list Member is any one in the multiple first file management unit；
Text is allowed according to the instruction that the file upload request is sent when receiving the first object file management unit When the first response message that part uploads, according to first response message, asked to first object file management unit transmission Seek the first data upload requests for uploading the first data block；
When the second sound for receiving the first object file management unit and being sent according to first data upload requests When answering message, according to second response message, first data block is uploaded in first object file storage unit；Its In,
The first object file storage unit for storing first data block is carried in second response message Information, the first object file storage unit is at least one of the multiple file storage unit.
Second aspect, the embodiment of the present invention provide a kind of file memory method, the first object being applied in one-of-a-kind system File management unit, wherein the one-of-a-kind system includes multiple customer units, stored for carrying out multiple files of file storage Unit and for storing metadata and for carrying out multiple first file management units of file management, the first object text Part administrative unit is any one in the multiple first file management unit；The method includes：
Receive the file upload request of the first customer unit transmission；
When detecting that permission file uploads according to the file upload request, sends one to first customer unit and refer to Show the first response message for allowing file to upload, is uploaded so that first customer unit is sent according to first response message First data upload requests of the first data block；
According to first data upload requests, the second response message is sent to first customer unit, so that described First customer unit is uploaded to first object file storage unit according to second response message, by first data block On；Wherein,
The first object file storage unit for storing first data block is carried in second response message Information, the first object file storage unit is at least one of the multiple file storage unit.
The third aspect, the embodiment of the present invention provide a kind of file storage device, and described device is applied in one-of-a-kind system One customer unit, wherein the one-of-a-kind system includes multiple customer units, stores list for carrying out multiple files of file storage Multiple first file management units first and for storing metadata and for carrying out file management；Described device includes：
First sending module, for a file upload request to be sent to first object file management unit, wherein described First object file management unit is any one in the multiple first file management unit；
Second sending module receives the first object file management unit according to the file upload request for working as When the instruction of transmission allows the first response message that file uploads, according to first response message, to first object text Part administrative unit sends the first data upload requests that request uploads the first data block；
First uploading module is uploaded for that ought receive the first object file management unit according to first data When asking the second response message sent, according to second response message, first data block is uploaded to first object In file storage unit；Wherein,
The first object file storage unit for storing first data block is carried in second response message Information, the first object file storage unit is at least one of the multiple file storage unit.
Fourth aspect, the embodiment of the present invention also provide a kind of electronic equipment, including memory, processor and are stored in storage On device and the computer program that can run on a processor, the processor realize first aspect when executing the computer program The step of described file memory method.
5th aspect, the embodiment of the present invention also provide a kind of non-transient computer readable storage medium, are stored thereon with meter Calculation machine program, when which is executed by processor realize first aspect described in file memory method the step of.
6th aspect, the embodiment of the present invention provide a kind of file storage device, and described device is applied in one-of-a-kind system First object file management unit, wherein the one-of-a-kind system includes multiple customer units, for carrying out the multiple of file storage File storage unit and for storing metadata and for carrying out multiple first file management units of file management, described the One file destination administrative unit is any one in the multiple first file management unit；Described device includes：
First receiving module, the file upload request for receiving the transmission of the first customer unit；
First sending module, for when detected according to the file upload request allow file to upload when, to described the One customer unit sends the first response message that an instruction allows file to upload, so that first customer unit is according to described the One response message sends the first data upload requests for uploading the first data block；
Second sending module, for according to first data upload requests, second to be sent to first customer unit Response message, so that first customer unit is uploaded to first according to second response message, by first data block In file destination storage unit；Wherein,
The first object file storage unit for storing first data block is carried in second response message Information, the first object file storage unit is at least one of the multiple file storage unit.
7th aspect, the embodiment of the present invention provides a kind of electronic equipment, including memory, processor and is stored in memory Computer program that is upper and can running on a processor, the processor realize second aspect institute when executing the computer program The step of file memory method stated.
Eighth aspect, the embodiment of the present invention provide a kind of non-transient computer readable storage medium, are stored thereon with calculating Machine program, when which is executed by processor realize second aspect described in file memory method the step of.
File memory method, device, electronic equipment and storage medium provided in an embodiment of the present invention, by one-of-a-kind system It is middle that multiple customer units, multiple file storage units for carrying out file storage are set and for storing metadata and being used for Carry out multiple first file management units of file management so that file upload request can be sent to the first mesh by customer unit File management unit is marked, which can be any one in multiple first file management units, And it is receiving the first response message of first object file management unit transmission and is carrying for storing the first data block First object file storage unit information the second response message when, the first data block is uploaded to first object file and is deposited On storage unit, realizing customer unit can ask to the transmission file upload of any one in multiple first file management units It asks, avoids multiple customer units and be merely able to send the process of file upload request to a file management unit, to realize Multiple customer units can send file upload request to arbitrary first file management unit respectively, i.e., different first files Administrative unit can handle file upload request simultaneously, file be written to realize multi-user, and reduce customer unit Request pressure, reduce file propagation delay time.
Description of the drawings
In order to more clearly explain the embodiment of the invention or the technical proposal in the existing technology, to embodiment or will show below There is attached drawing needed in technology description to be briefly described, it should be apparent that, the accompanying drawings in the following description is this hair Some bright embodiments for those of ordinary skill in the art without creative efforts, can be with root Other attached drawings are obtained according to these attached drawings.
Fig. 1 shows the step flow charts for the file memory method for being applied to customer unit in the embodiment of the present invention；
Fig. 2 indicates that the step of being applied to the file memory method of first object file management unit in the embodiment of the present invention flows Cheng Tu；
Fig. 3 indicates the module frame chart for being applied to the file storage device of customer unit in the embodiment of the present invention；
Fig. 4 indicates to realize the electronic equipment of the file memory method applied to customer unit in the embodiment of the present invention Module frame chart；
Fig. 5 indicates the module frame for being applied to the file storage device of first object file management unit in the embodiment of the present invention Figure；
Fig. 6 indicates to realize the file memory method applied to first object file management unit in the embodiment of the present invention Electronic equipment module frame chart.
Specific implementation mode
In order to make the object, technical scheme and advantages of the embodiment of the invention clearer, below in conjunction with the embodiment of the present invention In attached drawing, technical scheme in the embodiment of the invention is clearly and completely described, it is clear that described embodiment is A part of the embodiment of the present invention, instead of all the embodiments.Based on the embodiments of the present invention, those of ordinary skill in the art The every other embodiment obtained without creative efforts, shall fall within the protection scope of the present invention.
As shown in Figure 1, being the file memory method for the customer unit being applied in the embodiment of the present invention in one-of-a-kind system Flow chart of steps.
Wherein, one-of-a-kind system include multiple customer units, multiple file storage units for carrying out file storage and Multiple first file management units for storing metadata and for carrying out file management.
Specifically, multiple first file management units are by the metadata that is stored, to institute in multiple file storage units The file data of storage is managed.
Wherein, metadata is also known as broker data, relaying data mainly describe data attribute to describe the data of data Information, for support such as indicate storage location, historical data, resource lookup, file record function.Certainly, metadata can It is to include file store path, i.e., corresponding between file storage unit and the data block being stored in respective file storage unit Correspondence etc. between file data and fileinfo that relationship, data block are stored.
Certainly, it should be noted that multiple refer at least two, and customer unit, file are not limited specifically herein The particular number of storage unit and the first file management unit.
Wherein, the method includes：
Step 101：One file upload request is sent to first object file management unit.
In this step, specifically, when any one customer unit needs in one-of-a-kind system are written in one-of-a-kind system When file, a file upload request can be sent to first object file management unit, so that first object file management Unit can inquire whether file upload can be carried out according to this document upload request.
Specifically, store path can be carried in file upload request, so that first object file management unit It can be whether there is by detecting the store path, detect whether that file is allowed to upload.Certainly, when first object file management list In the presence of member detects the store path carried in file upload request, judgement allows file to upload.
In addition, specifically, the first object file management unit be multiple first file management units in any one, I.e. this it is not specific limit first object file management unit as in multiple first file management units it is specific which.
Certainly, the metadata all same that each first file management unit is stored in multiple first file storage units, So that each first file management unit is when as first object file management unit, it can be according to the member stored Data handle file upload request, the availability of each first file management unit are ensure that, to realize client File upload request can be sent to any one first file management unit by unit, i.e., multiple first file management units are equal The file upload request that can be sent to customer unit is handled, and to slow down the request pressure of customer unit, is reduced Time delay is written in file, and realizes multi-user while file is written.
Step 102：Text is allowed according to the instruction that file upload request is sent when receiving first object file management unit When the first response message that part uploads, according to the first response message, sends request to first object file management unit and upload the First data upload requests of one data block.
It in this step, can basis specifically, when first object file management unit receives file upload request This document upload request detects whether that file is allowed to upload.When first object file management unit detects that operating file uploads When, i.e., when first object file management unit inquires the store path of file, sending instruction to customer unit allows on file The first response message passed.
At this point, specifically, customer unit sends to first object file management unit and asks according to first response message The first data upload requests of the first data block are uploaded, so that first object file management unit can be according to the first data Upload request is that the first data block distributes file storage unit.
Specifically, first object file management unit when receiving the first data upload requests, can be the first data Block distributes file storage unit.Certainly, customer unit can specify the number of copies of the first data block, if customer unit is not specified Number of copies, then it is 3 to give tacit consent to number of copies, i.e., needs to distribute three file storage units for the first data block at this time, to ensure the The safety of one data block.
In addition, first object file management unit for the first data block distribute file storage unit after, can be to visitor Family unit sends the second response message, and the first object file for storing the first data block is carried in second response message The information of storage unit, certainly, the first object file storage unit are at least one of multiple file storage units.
Step 103：When the second sound for receiving first object file management unit and being sent according to the first data upload requests When answering message, according to the second response message, the first data block is uploaded in first object file storage unit.
In this step, specifically, the second response message is sent to customer unit in first object file management unit Later, customer unit, according to second response message, the first data block can be uploaded when receiving second response message To first object file storage unit, to realize the write-in of the first data block.
Specifically, the first object file carried in the second response message for storing first data block stores list The information of member, the first object file storage unit are at least one of the multiple file storage unit.
This is illustrated below.
For example, it is assumed that the number of copies of the first data block is 3, then first object file storage unit is first object file pipe Wherein three file storage units in multiple file storage units that reason unit is distributed need to upload the first data block To three file storage units, to ensure the storage security of the first data block.It is assumed to be depositing for the first data block A distribution It is the first object file storage unit a chosen from multiple file storage units, first object file storage unit b to store up address With first object file storage unit c, then need the first data block A being uploaded to first object file storage unit a, respectively In one file destination storage unit b and first object file storage unit c, so that a first object file is deposited wherein When the problems such as storage unit fails, the first data block A can be obtained from other first object file storage units, to protect The storage security of data is demonstrate,proved.
The present embodiment in one-of-a-kind system by being arranged multiple customer units, multiple file storage units and multiple first File management unit so that file upload request can be sent to first object file management unit by customer unit, this first File destination administrative unit can be any one in multiple first file management units, and receive first object file The first response message and carry first object file storage unit for storing the first data block that administrative unit is sent Information the second response message when, the first data block is uploaded in first object file storage unit, realize client list Member can send file upload request to any one in multiple first file management units, single to realize multiple clients Member can send file upload request to arbitrary first file management unit respectively, i.e., different first file management units can File upload request is handled simultaneously, to realize multi-user while file is written, and reduces the request pressure of customer unit Power reduces file propagation delay time.
Further, the first data block is uploaded to first object file and deposited by customer unit according to the second response message When on storage unit, it is single that the first data block can be uploaded to the storage of any one first object file according to the second response message In member, the first data block is copied to other first object files by the first object file storage unit by receiving the first data block In storage unit.
It is connected specifically, customer unit is only established with any one first object file storage unit, the reception first number Data transmission is copied into other first object file storage units according to the first object file storage unit of block.
For example, it is assumed that the number of copies of the first data block A is 3, first object file storage unit includes first object file Storage unit a, first object file storage unit b and when first object file storage unit c, customer unit can be only with first File destination storage unit a, which is established, to be connected and carries out data stream type transmission, and then first object file storage unit a passes data Defeated to copy on first object file storage unit b, data transmission is copied to first object by first object file storage unit b On file storage unit c, to realize the entire transmission process of the first data block.
In addition, further, one-of-a-kind system further includes the second file management unit for backing up metadata；Work as institute at this time It is described according to second response message when stating the file size of file and being less than or equal to a data block length, by described the After one data block is uploaded in first object file storage unit, the method further includes：One file is uploaded into the logical of completion Know that message is sent to the first object file management unit, so that the first object file management unit is according to the notice The metadata of the file is written in message, and by the metadata replication in addition to the first object file management unit In other first file management units and second file management unit.
Specifically, when the file size of file is less than or equal to a data block length, customer unit is in the first data After block uploads, that is, the upload to this document is completed, a file can be uploaded the notice completed and disappeared by customer unit at this time Breath be sent to first object file management unit so that first object file management unit can more new metadata, that is, exist The metadata that file is written in first object file management unit provides the foundation for the inquiry reading of later file.Meanwhile being The consistency of the metadata stored in guarantee all first file management units, first object file management unit can will The metadata of this document is simultaneously on replication synchronization to other first file management units；In addition, in order to ensure first number of this document According to storage security, first object file management unit can also by the metadata of this document simultaneously replication synchronization to be used for it is standby In second file management unit of part metadata, so that when the first file management unit fails, it can be from the second file The metadata of file is obtained in administrative unit, and then ensure that the storage security of the metadata of file.
Certainly, when the file size of file is more than data block length, according to second response message, by described the After one data block is uploaded in first object file storage unit, the method further includes：Include by the file successively Other data blocks are uploaded in the second file destination storage unit corresponding with other described data blocks；Wherein described second target File storage unit is at least one of the multiple file storage unit.
Specifically, file data is first cached to local, when data cached by customer unit before starting transmission data block Length is more than a data block length, i.e., when the file size of file is more than data block length, customer unit needs will write-in text Multiple data blocks of number of packages evidence are uploaded successively, and need successively by other data blocks that file includes be uploaded to and other In the corresponding second file destination storage unit of data block, completed until entire file uploads.It is passed below with second data block Defeated process illustrates.
In customer unit according to second response message, first data block is uploaded to the storage of first object file After on unit, customer unit sends the second data upload that request uploads the second data block to first object file management unit Request；When customer unit receives that the first object file management unit is sent according to second data upload requests When three response messages, according to the third response message, second data block is uploaded to the second file destination storage unit On；Wherein, the second file destination storage unit for storing second data block is carried in the third response message Information, the second file destination storage unit is at least one of the multiple file storage unit.
In this way, by the way that the data block included by file to be transmitted separately in different file storage units, further protect The storage security of file data is demonstrate,proved.
In addition, further, it should be noted that when the file does not write full integer number of data blocks, in not The data block for writing full state is additionally operable to storage alternative document data.In this way, increasing the utilization rate of data block, realizing can be high Effect storage large amount of small documents, optimizes the storage of small documents.It is explained below.
Assuming that the file size of the first file is 100M, the data block length of a data block is 128M, then the first file A full data block can not be write, i.e. the data block with the first file there remains 28M spare spaces, if there are the second texts at this time Part, and the file size of the second file is 28M, then is written in the remaining spaces 28M in the data block at this with the first file Second file.In this way so that each data block is in full state of writing, rather than is drawn to data block according to different files Point, realize data block share, to increase the utilization rate of data block, realize can efficient storage large amount of small documents, Optimize the storage of small documents.
In addition, further, the method further includes：The request message that one request is read to file is sent to the second target File management unit, wherein the second file destination administrative unit is any one in the multiple first file management unit It is a；Receive the metamessage for the file that the second file destination administrative unit is sent according to the request message, wherein institute State the information that metamessage includes the third file destination storage unit for all data blocks for being stored with the file, and each data Block is corresponding at least one third file destination storage unit；According to the metamessage, respectively from each data block it is corresponding to Corresponding data block is downloaded in any one third file destination storage in a few third file destination storage unit；Its In, in the data block downloading process of the file, processing is merged to the data block for having completed to download, until the text All data merged blocks of part are completed.
Specifically, when customer unit needs to read file data, one can be sent to the second file destination administrative unit The request message of file is read in request.Wherein, by the metadata of the file stored in multiple first file management units Consistency, therefore the second file destination administrative unit can be any one in multiple first file management units, that is, exist It is specially which that this, which does not limit the second file destination administrative unit specifically,.
In addition, specifically, documentary store path can be carried in the request message.At this point, when the second file destination , can be according to the store path of file when administrative unit receives request message, by query metadata, whether inquiry this document In the presence of if there are this document, obtaining the metamessage of this document, that is, obtain the third target for storing documentary all data blocks The information of file storage unit finally should wherein each data block is corresponding at least one third file destination storage unit The metamessage of file is sent to customer unit.
At this point, customer unit is after receiving metamessage, it can be respectively from each data block corresponding at least one Corresponding data block is downloaded in any one third file destination storage in three file destination storage units.Specifically, in order to Ensure that the integrality of file needs to merge the data block of download in data block downloading process, until entire file closes And it completes.
In this way, being sent to by the way that multiple first file management units are arranged, and by the request message of a request reading file Any one in multiple first file management units, avoiding multiple customer units can only send to a file management unit The problem of request message, reduces file download time delay to alleviate the request pressure of customer unit.
The embodiment of the present invention by the way that multiple customer units are arranged in one-of-a-kind system, multiple texts for carrying out file storage Part storage unit and for storing metadata and for carrying out multiple first file management units of file management so that client File upload request can be sent to first object file management unit by unit, which can be Any one in multiple first file management units, and in the first response for receiving the transmission of first object file management unit Message and when carrying the second response message of information of first object file storage unit for storing the first data block, First data block is uploaded in first object file storage unit, realizing customer unit can be to multiple first file management Any one in unit sends file upload request, avoids multiple customer units and is merely able to send out to a file management unit The process for sending file upload request can be sent out to arbitrary first file management unit respectively to realize multiple customer units File upload request is sent, i.e., different first file management units can handle file upload request simultaneously, multi-purpose to realize File is written in family, and reduces the request pressure of customer unit, reduces file propagation delay time.
In addition, as shown in Fig. 2, to be applied to the first object file management unit in one-of-a-kind system in the embodiment of the present invention File memory method step flow chart.
Wherein, one-of-a-kind system include multiple customer units, multiple file storage units for carrying out file storage and Multiple first file management units for storing metadata and for carrying out file management, the first object file management list Member is any one in the multiple first file management unit.
Specifically, the metadata that each first file management unit is stored in multiple first file storage units is homogeneous It together, can be according to being stored so that each first file management unit is when as first object file management unit Metadata, file upload request is handled, the availability of each first file management unit is ensure that, to realize Each first file management unit can receive the file upload request of customer unit transmission, i.e., multiple first file managements The file upload request that unit can send customer unit is handled, to slow down the request pressure of customer unit, Reduce file write-in time delay, and realizes multi-user while file is written.
Specifically, multiple first file management units are by the metadata that is stored, to institute in multiple file storage units The file data of storage is managed.
Wherein, metadata is also known as broker data, relaying data mainly describe data attribute to describe the data of data Information, for support such as indicate storage location, historical data, resource lookup, file record function.Certainly, metadata can It is to include file store path, i.e., corresponding between file storage unit and the data block being stored in respective file storage unit Correspondence etc. between file data and fileinfo that relationship, data block are stored.
Certainly, it should be noted that multiple refer at least two, and customer unit, file are not limited specifically herein The particular number of storage unit and the first file management unit.
Wherein, the method includes：
Step 201：Receive the file upload request of the first customer unit transmission.
In this step, specifically, when any one customer unit needs in one-of-a-kind system are written in one-of-a-kind system When file, a file upload request can be sent to first object file management unit, at this time first object file management unit Receive the file upload request of the first customer unit transmission.
Specifically, store path can be carried in file upload request, so that first object file management unit It can be whether there is by detecting the store path, detect whether that file is allowed to upload.Certainly, when first object file management list In the presence of member detects the store path carried in file upload request, judgement allows file to upload.
In addition, specifically, it should be noted that the first customer unit can be any one in multiple customer units It is a, i.e., the first customer unit specifically is not limited as which of multiple customer units herein.
In addition, specifically, it should be noted that the first object file management unit is multiple first file managements Any one in unit, i.e., specifically do not limit first object file management unit as multiple first file management units herein In it is specific which so that each first file management unit is when as first object file management unit, equal energy It is enough that file upload request is handled according to the metadata stored, it ensure that the available of each first file management unit Property, it can receive the file upload request of customer unit transmission to realize each first file management unit, i.e., it is more The file upload request that a first file management unit can send customer unit is handled, to slow down client's list The request pressure of member reduces file write-in time delay, and realizes multi-user while file is written.
Step 202：When detecting that permission file uploads according to file upload request, sends one to the first customer unit and refer to Show the first response message for allowing file to upload.
In this step, specifically, allowing when first object file management unit is detected according to file upload request When transmitting file, the first response message that an instruction allows file to upload can be sent to the first customer unit, so that described first Customer unit sends the first data upload requests for uploading the first data block according to first response message.
Certainly, the store path less than file is inquired according to file upload request when first object file management unit, i.e., Detect that then feeding back one to the first customer unit does not allow the message of file upload when not allowing file to upload.
Step 203：According to the first data upload requests, the second response message is sent to the first customer unit.
In this step, specifically, when first object file management unit receives the first of the first customer unit transmission Can be that the first data block distributes file storage unit when data upload requests.Certainly, the first customer unit can specify first The number of copies of data block, if the first customer unit does not specify number of copies, it is 3 to give tacit consent to number of copies, i.e., needs to be first at this time Data block distributes three file storage units, to ensure the safety of the first data block.
At this point, first object file management unit is for the first data block after distributing file storage unit, it can be to the One customer unit sends the second response message, so that first customer unit is according to second response message, by described the One data block is uploaded in first object file storage unit；Wherein, it is carried in second response message for storing The information of the first object file storage unit of the first data block is stated, the first object file storage unit is the multiple text At least one of part storage unit.
For example, it is assumed that the number of copies of the first data block is 3, first object file management unit is that the distribution of the first data block is more Wherein three file storage units in a file storage unit are first object file storage unit, so that the first client First data block can be uploaded in three file storage units by unit, to ensure the storage security of the first data block.
In this way, the present embodiment is by regarding any one in multiple first file management units as first object file pipe Unit is managed, and the first object file management unit can receive the file upload request of the first customer unit transmission, and examined When measuring permission file upload, sending instruction to the first customer unit allows the first response message of file upload, and is receiving To the first customer unit send the first data upload requests when, to the first customer unit transmission carry for store first number According to the information of the first object file storage unit of block, so that the first data block can be uploaded to by the first customer unit In one file destination storage unit, first object file management list can be used as by realizing each first file management unit Member receives the file upload request that customer unit is sent, handles file upload request, to realize the first text of difference Part administrative unit can receive the file upload request of different clients unit transmission simultaneously, and then realize multi-user while being written File, and the request pressure of customer unit is reduced, reduce file propagation delay time.
Further, the one-of-a-kind system further includes the second file management unit for backing up metadata；The method Further include：It receives the file that first customer unit is sent and uploads the notification message completed；It is written according to the notification message The metadata of the file, and by the metadata replication of the file to its in addition to the first object file management unit In his the first file management unit and second file management unit.
Specifically, when the first customer unit is after file uploads, a file can be uploaded the notification message completed It is sent to first object file management unit, first object file management unit receives the notice that the first customer unit is sent at this time Message, and according to the notification message more new metadata, i.e., the metadata of file is written in first object file management unit, from And be later file inquiry reading provide the foundation.Meanwhile in order to ensure to be stored in all first file management units The consistency of metadata, first object file management unit can by the metadata of this document simultaneously replication synchronization to other first In file management unit；In addition, the storage security of the metadata in order to ensure this document, first object file management unit is also Can by the metadata of this document simultaneously replication synchronization to the second file management unit for backing up metadata so that When the first file management unit fails, the metadata of file can be obtained from the second file management unit, and then ensure that The storage security of the metadata of file.
In addition, further, the request that first object file management unit can receive the transmission of the second customer unit is read The request message of file；According to the request message, the metamessage of the file is sent to second customer unit, so that Second customer unit is according to the metamessage, respectively from the corresponding at least one third file destination storage of each data block Corresponding data block is downloaded in any one third file destination storage in unit；Wherein, the metamessage includes being stored with The information of the third file destination storage unit of all data blocks of the file, and each data block is corresponding at least one Three file destination storage units；In the data block downloading process of the file, the data block for having completed to download is closed And handle, until all data merged blocks of the file are completed.
Specifically, the second customer unit can be any one in multiple customer units.In addition, when the second customer unit When needing to read file data, the request message that file is read in a request can be sent to first object file management unit.
In addition, specifically, documentary store path can be carried in the request message.At this point, working as first object file , can be according to the store path of file when administrative unit receives request message, by query metadata, whether inquiry this document In the presence of if there are this document, obtaining the metamessage of this document, that is, obtain the third target for storing documentary all data blocks The information of file storage unit finally should wherein each data block is corresponding at least one third file destination storage unit The metamessage of file is sent to the second customer unit.
At this point, the second customer unit is after receiving metamessage, it can be respectively from each data block corresponding at least one Corresponding data block is downloaded in any one third file destination storage in a third file destination storage unit.Specifically, It in order to ensure the integrality of file, in data block downloading process, needs to merge the data block of download, until entire text Part, which merges, to be completed.
In this way, by the way that multiple first file management units are arranged, and any one first file management unit can connect Receive request read file request message, avoid only there are one file management unit can receive request message the problem of, from And the request pressure of multiple customer units is alleviated, reduce file download time delay.
In addition, further, first object file management unit is stored with the first of first data block detecting When the quantity of file destination storage unit is less than predetermined threshold value, required amount of the is obtained from the multiple file storage unit Four file destination storage units so that the first object file storage unit by first data block be copied to it is described needed for In 4th file destination storage unit of quantity；Wherein, requirement is that the predetermined threshold value is deposited with the first object file Difference between the quantity of storage.
In this way, default by being less than in the quantity for detecting the first object file storage unit for being stored with the first data block When threshold value, required amount of 4th file destination storage unit is obtained from multiple file storage units, makes first object file First data block is copied in required amount of 4th file destination storage unit by storage unit, ensure that the first data block Storage quantity, and then ensure that the storage security of the first data block.
In addition, further, when the file does not write full integer number of data blocks, in not writing the data block of full state also For storing alternative document data.In this way, increase the utilization rate of data block, realize can efficient storage large amount of small documents, Optimize the storage of small documents.
In this way, the present embodiment is by regarding any one in multiple first file management units as first object file pipe Unit is managed, and the first object file management unit can receive the file upload request of the first customer unit transmission, and examined When measuring permission file upload, sending instruction to the first customer unit allows the first response message of file upload, and is receiving To the first customer unit send the first data upload requests when, to the first customer unit transmission carry for store first number According to the information of the first object file storage unit of block, so that the first data block can be uploaded to by the first customer unit In one file destination storage unit, first object file management list can be used as by realizing each first file management unit Member receives the file upload request that customer unit is sent, handles file upload request, to realize the first text of difference Part administrative unit can receive the file upload request of different clients unit transmission simultaneously, and then realize multi-user while being written File, and the request pressure of customer unit is reduced, reduce file propagation delay time.
In addition, as shown in figure 3, the module frame of the file storage device to be applied to customer unit in the embodiment of the present invention Figure.Described device is applied to the customer unit in one-of-a-kind system, wherein the one-of-a-kind system includes multiple customer units, use In the multiple file storage units for carrying out file storage and for storing metadata and for carrying out file management multiple the One file management unit；Described device includes：
First sending module 301, for a file upload request to be sent to first object file management unit, wherein institute It is any one in the multiple first file management unit to state first object file management unit；
Second sending module 302 is uploaded for that ought receive the first object file management unit according to the file When the instruction that request is sent allows the first response message that file uploads, according to first response message, to first mesh It marks file management unit and sends the first data upload requests that request uploads the first data block；
First uploading module 303 receives the first object file management unit according to first data for working as When the second response message that upload request is sent, according to second response message, first data block is uploaded to first In file destination storage unit；Wherein,
The first object file storage unit for storing first data block is carried in second response message Information, the first object file storage unit is at least one of the multiple file storage unit.
File storage device provided in an embodiment of the present invention applied to customer unit, will by the first sending module 301 One file upload request is sent to first object file management unit, and wherein first object file management unit is multiple first texts Any one in part administrative unit；In the instruction for receiving first object file management unit and being sent according to file upload request When the first response message for allowing file to upload, asked to the transmission of first object file management unit by the second sending module 302 Seek the first data upload requests for uploading the first data block；When receiving first object file management unit according in the first data When passing the second response message that request is sent, the first data block is uploaded to by first object file by the first uploading module 303 In storage unit；Wherein, the first object file storage unit for storing the first data block is carried in the second response message Information, realize customer unit can to any one in multiple first file management units send file upload request, Multiple customer units are avoided to be merely able to send the process of file upload request to a file management unit, it is more to realize A customer unit can send file upload request to arbitrary first file management unit respectively, i.e., different first file managements Unit can handle file upload request simultaneously, file be written to realize multi-user, and reduce asking for customer unit Pressure is sought, file propagation delay time is reduced.
Optionally, first uploading module 303 is used for, according to second response message, by first data block It is uploaded in any one first object file storage unit, the first object file by receiving first data block stores single Member copies to first data block in other first object file storage units.
Optionally, the one-of-a-kind system further includes the second file management unit for backing up metadata；
When the file size of the file is less than or equal to a data block length, described device further includes：
Third sending module is sent to the first object file management for a file to be uploaded the notification message completed Unit, so that the first object file management unit is written the metadata of the file according to the notification message, and by institute Metadata replication is stated to other first file management units and described the in addition to the first object file management unit In two file management units.
Optionally, it when the file size of the file is more than data block length, according to second response message, incites somebody to action After first data block is uploaded in first object file storage unit, described device further includes：
Second uploading module, for being successively uploaded to other data blocks that the file includes and other described data blocks In corresponding second file destination storage unit；The wherein described second file destination storage unit is that the storage of the multiple file is single At least one of member.
Optionally, it when the file does not write full integer number of data blocks, is additionally operable to deposit in the data block for not writing full state Store up alternative document data.
Optionally, described device further includes：
4th sending module, it is single that the request message for a request to be read to file is sent to the management of the second file destination Member, wherein the second file destination administrative unit is any one in the multiple first file management unit；
Receiving module, the text sent according to the request message for receiving the second file destination administrative unit The metamessage of part, wherein the metamessage includes the third file destination storage unit for all data blocks for being stored with the file Information, and each data block is corresponding at least one third file destination storage unit；
Download module is used for according to the metamessage, respectively from the corresponding at least one third target text of each data block Corresponding data block is downloaded in any one third file destination storage in part storage unit；Wherein,
In the data block downloading process of the file, processing is merged to the data block for having completed to download, until All data merged blocks of the file are completed.
File storage device provided in an embodiment of the present invention, by the way that multiple customer units are arranged in one-of-a-kind system, are used for Carry out multiple file storage units of file storage and for storing metadata and for carrying out multiple the first of file management File management unit so that file upload request can be sent to first object file management unit by customer unit, this first File destination administrative unit can be any one in multiple first file management units, and receive first object file The first response message and carry first object file storage unit for storing the first data block that administrative unit is sent Information the second response message when, the first data block is uploaded in first object file storage unit, realize client list Member can send file upload request to any one in multiple first file management units, avoid multiple customer units only The process that file upload request can be sent to file management unit, to realize multiple customer units can respectively to Arbitrary first file management unit sends file upload request, i.e., different first file management units can handle file simultaneously Upload request is written file to realize multi-user, and reduces the request pressure of customer unit, reduces file transmission Time delay.
It should be noted that in the embodiment of the present invention hardware processor (hardware processor) can be passed through It realizes related function module, and identical technique effect can be reached, do not repeated herein.
In another embodiment of the present invention, a kind of electronic equipment is provided, as shown in figure 4, the electronic equipment includes It memory (memory) 401, processor (processor) 402 and is stored on memory 401 and can be on the processor 402 The computer program of operation.Wherein, the memory 401, processor 402 complete mutual communication by bus 403.It is described Processor 402 is used to call the program instruction in the memory 401, to execute following method：One file upload request is sent out It send to first object file management unit, wherein the first object file management unit is the multiple first file management list Any one in member；When the instruction for receiving the first object file management unit and being sent according to the file upload request When the first response message for allowing file to upload, according to first response message, to the first object file management unit Send the first data upload requests that request uploads the first data block；When receive the first object file management unit according to When the second response message that first data upload requests are sent, according to second response message, by first data Block is uploaded in first object file storage unit；Wherein, it is carried in second response message for storing described first The information of the first object file storage unit of data block, the first object file storage unit store for the multiple file At least one of unit.
Electronic equipment provided in an embodiment of the present invention, the specific steps in executable file storage method, and can reach Identical technique effect, is herein no longer specifically introduced this.
In addition, the program instruction in above-mentioned memory 401 can be realized and be used as by the form of SFU software functional unit Independent product sale in use, can be stored in a computer read/write memory medium.Based on this understanding, originally Substantially the part of the part that contributes to existing technology or the technical solution can be in other words for the technical solution of invention The form of software product embodies, which is stored in a storage medium, including some instructions to So that a computer equipment (can be personal computer, server or the network equipment etc.) executes each implementation of the present invention The all or part of step of example the method.And storage medium above-mentioned includes：USB flash disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), magnetic disc or CD etc. it is various The medium of program code can be stored.
In another embodiment of the present invention, a kind of non-transient computer readable storage medium is provided, is stored thereon with Computer program, to execute following method when which is executed by processor：One file upload request is sent to One file destination administrative unit, wherein the first object file management unit is in the multiple first file management unit Any one；Text is allowed according to the instruction that the file upload request is sent when receiving the first object file management unit When the first response message that part uploads, according to first response message, asked to first object file management unit transmission Seek the first data upload requests for uploading the first data block；When receiving the first object file management unit according to described When the second response message that one data upload requests are sent, according to second response message, first data block is uploaded To first object file storage unit；Wherein, it is carried in second response message for storing first data block First object file storage unit information, the first object file storage unit be the multiple file storage unit in It is at least one.
Non-transient computer readable storage medium provided in an embodiment of the present invention is specific in executable file storage method Step, and identical technique effect can be reached, no longer this is specifically introduced herein.
In another embodiment of the present invention, a kind of computer program product, the computer program product packet are provided The computer program being stored in non-transient computer readable storage medium is included, the computer program includes program instruction, when To execute following method when described program instruction is computer-executed：One file upload request is sent to first object file pipe Unit is managed, wherein the first object file management unit is any one in the multiple first file management unit；When Receive the first object file management unit allows that file uploads according to the instruction that the file upload request is sent the When one response message, according to first response message, sends request to the first object file management unit and upload first First data upload requests of data block；It is uploaded according to first data when receiving the first object file management unit When asking the second response message sent, according to second response message, first data block is uploaded to first object In file storage unit；Wherein, the first object for storing first data block is carried in second response message The information of file storage unit, the first object file storage unit are at least one in the multiple file storage unit It is a.
Computer program product provided in an embodiment of the present invention, the specific steps in executable file storage method, and energy Enough reach identical technique effect, no longer this is specifically introduced herein.
In addition, as shown in figure 5, to be applied to the file storage dress of first object file management unit in the embodiment of the present invention The module frame chart set.Described device is applied to the first object file management unit in one-of-a-kind system, wherein the one-of-a-kind system Multiple file storage units including multiple customer units, for carrying out file storage and for store metadata and for into Multiple first file management units of row file management, the first object file management unit are the multiple first file pipe Manage any one in unit；Described device includes：
First receiving module 501, the file upload request for receiving the transmission of the first customer unit；
First sending module 502, for when detected according to the file upload request allow file to upload when, to described First customer unit sends the first response message that an instruction allows file to upload, so that first customer unit is according to First response message sends the first data upload requests for uploading the first data block；
Second sending module 503 sends for according to first data upload requests to first customer unit Two response messages, so that first customer unit is uploaded to according to second response message, by first data block In one file destination storage unit；Wherein,
The first object file storage unit for storing first data block is carried in second response message Information, the first object file storage unit is at least one of the multiple file storage unit.
File storage device provided in an embodiment of the present invention receives the first customer unit hair by the first receiving module 501 The file upload request sent；Send that an instruction allows that file uploads to the first customer unit by the first sending module 502 the One response message, so that the first data upload that the first customer unit sends the first data block of upload according to the first response message is asked It asks；By the second sending module 503 according to the first data upload requests, the second response message is sent to the first customer unit, with Make the first customer unit according to the second response message, the first data block is uploaded in first object file storage unit, realizes Each first file management unit can be used as first object file management unit, receive on the file that customer unit is sent Request is passed, file upload request is handled, difference can be received simultaneously to realize different first file management units The file upload request that customer unit is sent, and then realize multi-user while file is written, and reduce customer unit Pressure is asked, file propagation delay time is reduced.
Optionally, the one-of-a-kind system further includes the second file management unit for backing up metadata；Described device is also Including:
Second receiving module uploads the notification message completed for receiving the file that first customer unit is sent；
Replication module, the metadata for the file to be written according to the notification message, and by first number of the file According to other first file management units and second file being copied in addition to the first object file management unit In administrative unit.
Optionally, described device further includes：
The request message of file is read in third receiving module, the request for receiving the transmission of the second customer unit；
Third sending module, for according to the request message, the metamessage of the file to be sent to second visitor Family unit, so that second customer unit is according to the metamessage, respectively from the corresponding at least one third of each data block Corresponding data block is downloaded in any one third file destination storage in file destination storage unit；Wherein,
The metamessage includes the information of the third file destination storage unit for all data blocks for being stored with the file, And each data block is corresponding at least one third file destination storage unit；
In the data block downloading process of the file, processing is merged to the data block for having completed to download, until All data merged blocks of the file are completed.
Optionally, described device further includes：
Acquisition module, for when the quantity for detecting the first object file storage unit for being stored with first data block When less than predetermined threshold value, required amount of 4th file destination storage unit is obtained from the multiple file storage unit, with Make the first object file storage unit that first data block is copied to required amount of 4th file destination to deposit On storage unit；Wherein, requirement is the difference between the predetermined threshold value and the quantity of first object file storage.
Optionally, it when the file does not write full integer number of data blocks, is additionally operable to deposit in the data block for not writing full state Store up alternative document data.
File storage device provided in this embodiment, by the way that any one in multiple first file management units to be used as First object file management unit, and the first object file management unit can receive on the file of the first customer unit transmission Request is passed, and when detecting that permission file uploads, sending instruction to the first customer unit allows the first response of file upload Message, and when receiving the first data upload requests of the first customer unit transmission, carried to the transmission of the first customer unit Information for the first object file storage unit for storing the first data block, so that the first customer unit can be by first Data block is uploaded in first object file storage unit, and the first mesh can be used as by realizing each first file management unit File management unit is marked, the file upload request that customer unit is sent is received, file upload request is handled, to realize Different first file management units can receive the file upload request of different clients unit transmission simultaneously, and then realize more File is written simultaneously in user, and reduces the request pressure of customer unit, reduces file propagation delay time.
It should be noted that in the embodiment of the present invention hardware processor (hardware processor) can be passed through It realizes related function module, and identical technique effect can be reached, do not repeated herein.
In another embodiment of the present invention, a kind of electronic equipment is provided, as shown in fig. 6, the electronic equipment includes It memory (memory) 601, processor (processor) 602 and is stored on memory 601 and can be on processor 602 The computer program of operation.Wherein, the memory 601, processor 602 complete mutual communication by bus 603.It is described Processor 602 is used to call the program instruction in the memory 601, to execute following method：Receive the first customer unit hair The file upload request sent；When detecting that permission file uploads according to the file upload request, to first client list Member sends the first response message that an instruction allows file to upload, so that first customer unit disappears according to first response Breath sends the first data upload requests for uploading the first data block；According to first data upload requests, to first visitor Family unit sends the second response message, so that first customer unit is counted according to second response message by described first It is uploaded in first object file storage unit according to block；Wherein, it is carried in second response message for storing described The information of the first object file storage unit of one data block, the first object file storage unit are that the multiple file is deposited At least one of storage unit.
Electronic equipment provided in an embodiment of the present invention, the specific steps in executable file storage method, and can reach Identical technique effect, is herein no longer specifically introduced this.
In addition, the program instruction in above-mentioned memory 601 can be realized and be used as by the form of SFU software functional unit Independent product sale in use, can be stored in a computer read/write memory medium.Based on this understanding, originally Substantially the part of the part that contributes to existing technology or the technical solution can be in other words for the technical solution of invention The form of software product embodies, which is stored in a storage medium, including some instructions to So that a computer equipment (can be personal computer, server or the network equipment etc.) executes each implementation of the present invention The all or part of step of example the method.And storage medium above-mentioned includes：USB flash disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), magnetic disc or CD etc. it is various The medium of program code can be stored.
In another embodiment of the present invention, a kind of non-transient computer readable storage medium is provided, is stored thereon with Computer program, to execute following method when which is executed by processor：Receive the text of the first customer unit transmission Part upload request；When detecting that permission file uploads according to the file upload request, sent to first customer unit One instruction allows the first response message that file uploads, so that first customer unit is sent according to first response message Upload the first data upload requests of the first data block；According to first data upload requests, to first customer unit The second response message is sent, so that first customer unit is according to second response message, it will be in first data block It reaches in first object file storage unit；Wherein, it is carried in second response message for storing first data The information of the first object file storage unit of block, the first object file storage unit are the multiple file storage unit At least one of.
Non-transient computer readable storage medium provided in an embodiment of the present invention is specific in executable file storage method Step, and identical technique effect can be reached, no longer this is specifically introduced herein.
In another embodiment of the present invention, a kind of computer program product, the computer program product packet are provided The computer program being stored in non-transient computer readable storage medium is included, the computer program includes program instruction, when To execute following method when described program instruction is computer-executed：Receive the file upload request of the first customer unit transmission； When detecting that permission file uploads according to the file upload request, it is literary to send an instruction permission to first customer unit The first response message that part uploads, so that first customer unit is sent according to first response message uploads the first data First data upload requests of block；According to first data upload requests, the second response is sent to first customer unit Message, so that first customer unit is uploaded to first object according to second response message, by first data block In file storage unit；Wherein, the first object for storing first data block is carried in second response message The information of file storage unit, the first object file storage unit are at least one in the multiple file storage unit It is a.
Computer program product provided in an embodiment of the present invention, the specific steps in executable file storage method, and energy Enough reach identical technique effect, no longer this is specifically introduced herein.
The apparatus embodiments described above are merely exemplary, wherein the unit illustrated as separating component can It is physically separated with being or may not be, the component shown as unit may or may not be physics list Member, you can be located at a place, or may be distributed over multiple network units.It can be selected according to the actual needs In some or all of module achieve the purpose of the solution of this embodiment.Those of ordinary skill in the art are not paying creativeness Labour in the case of, you can to understand and implement.
Through the above description of the embodiments, those skilled in the art can be understood that each embodiment can It is realized by the mode of software plus required general hardware platform, naturally it is also possible to pass through hardware.Based on this understanding, on Stating technical solution, substantially the part that contributes to existing technology can be expressed in the form of software products in other words, should Computer software product can store in a computer-readable storage medium, such as ROM/RAM, magnetic disc, CD, including several fingers It enables and using so that a computer equipment (can be personal computer, server or the network equipment etc.) executes each implementation Method described in certain parts of example or embodiment.
Finally it should be noted that：The above embodiments are merely illustrative of the technical solutions of the present invention, rather than its limitations；Although Present invention has been described in detail with reference to the aforementioned embodiments, it will be understood by those of ordinary skill in the art that：It still may be used With technical scheme described in the above embodiments is modified or equivalent replacement of some of the technical features； And these modifications or replacements, various embodiments of the present invention technical solution that it does not separate the essence of the corresponding technical solution spirit and Range.

Claims (17)

1. a kind of file memory method, which is characterized in that the method is applied to the customer unit in one-of-a-kind system, wherein The one-of-a-kind system includes multiple customer units, multiple file storage units for carrying out file storage and for storing member Data and multiple first file management units for being used to carry out file management；The method includes：
One file upload request is sent to first object file management unit, wherein the first object file management unit is Any one in the multiple first file management unit；
Allow on file when receiving the instruction that the first object file management unit is sent according to the file upload request When the first response message passed, according to first response message, sent in request to the first object file management unit Pass the first data upload requests of the first data block；
Disappeared according to the second response that first data upload requests are sent when receiving the first object file management unit When breath, according to second response message, first data block is uploaded in first object file storage unit；Wherein,
The letter of the first object file storage unit for storing first data block is carried in second response message Breath, the first object file storage unit are at least one of the multiple file storage unit.
2. according to the method described in claim 1, it is characterized in that, described according to second response message, by described first Data block is uploaded in first object file storage unit, including：
According to second response message, first data block is uploaded to any one first object file storage unit On, first data block is copied to other the first mesh by the first object file storage unit by receiving first data block It marks in file storage unit.
3. according to the method described in claim 1, it is characterized in that, the one-of-a-kind system further includes for backing up metadata Two file management units；
It is described according to second response message when the file size of the file is less than or equal to a data block length, After first data block is uploaded in first object file storage unit, the method further includes：
One file is uploaded into the notification message completed and is sent to the first object file management unit, so that the first object File management unit is written the metadata of the file according to the notification message, and by the metadata replication to except described the In other first file management units and second file management unit except one file destination administrative unit.
4. according to the method described in claim 1, it is characterized in that, the file size when the file is more than data block length When, according to second response message, after first data block is uploaded in first object file storage unit, institute The method of stating further includes：
Other data blocks that the file includes the second file destination corresponding with other described data blocks is uploaded to successively to deposit On storage unit；The wherein described second file destination storage unit is at least one of the multiple file storage unit.
5. according to the method described in claim 1, it is characterized in that, when the file does not write full integer number of data blocks, it is in The data block for not writing full state is additionally operable to storage alternative document data.
6. according to the method described in claim 1, it is characterized in that, the method further includes：
The request message that one request is read to file is sent to the second file destination administrative unit, wherein second file destination Administrative unit is any one in the multiple first file management unit；
Receive the metamessage for the file that the second file destination administrative unit is sent according to the request message, wherein institute State the information that metamessage includes the third file destination storage unit for all data blocks for being stored with the file, and each data Block is corresponding at least one third file destination storage unit；
According to the metamessage, respectively from arbitrary in the corresponding at least one third file destination storage unit of each data block Corresponding data block is downloaded in one third file destination storage；Wherein,
In the data block downloading process of the file, processing is merged to the data block for having completed to download, until described All data merged blocks of file are completed.
7. a kind of file memory method, which is characterized in that the method is applied to the first object file management in one-of-a-kind system Unit, wherein the one-of-a-kind system include multiple customer units, multiple file storage units for carrying out file storage and Multiple first file management units for storing metadata and for carrying out file management, the first object file management list Member is any one in the multiple first file management unit；The method includes：
Receive the file upload request of the first customer unit transmission；
When detecting that permission file uploads according to the file upload request, sends an instruction to first customer unit and permit Perhaps the first response message that file uploads, so that first customer unit is sent according to first response message uploads first First data upload requests of data block；
According to first data upload requests, the second response message is sent to first customer unit, so that described first First data block is uploaded in first object file storage unit by customer unit according to second response message；Its In,
The letter of the first object file storage unit for storing first data block is carried in second response message Breath, the first object file storage unit are at least one of the multiple file storage unit.
8. the method according to the description of claim 7 is characterized in that the one-of-a-kind system further includes for backing up metadata Two file management units；The method further includes:
It receives the file that first customer unit is sent and uploads the notification message completed；
The metadata of the file is written according to the notification message, and by the metadata replication of the file to except described first In other first file management units and second file management unit except file destination administrative unit.
9. the method according to the description of claim 7 is characterized in that the method further includes：
The request message of file is read in the request for receiving the transmission of the second customer unit；
According to the request message, the metamessage of the file is sent to second customer unit, so that second visitor Family unit is according to the metamessage, respectively from appointing in the corresponding at least one third file destination storage unit of each data block Corresponding data block is downloaded in one third file destination storage of meaning；Wherein,
The metamessage includes the information of the third file destination storage unit for all data blocks for being stored with the file, and every One data block is corresponding at least one third file destination storage unit；
In the data block downloading process of the file, processing is merged to the data block for having completed to download, until described All data merged blocks of file are completed.
10. the method according to the description of claim 7 is characterized in that the method further includes：
When detecting that the quantity for the first object file storage unit for being stored with first data block is less than predetermined threshold value, from Required amount of 4th file destination storage unit is obtained in the multiple file storage unit, so that the first object file First data block is copied in required amount of 4th file destination storage unit by storage unit；Wherein, required Quantity is the difference between the predetermined threshold value and the quantity of first object file storage.
11. the method according to the description of claim 7 is characterized in that when the file does not write full integer number of data blocks, it is in The data block for not writing full state is additionally operable to storage alternative document data.
12. a kind of file storage device, which is characterized in that described device is applied to the customer unit in one-of-a-kind system, wherein The one-of-a-kind system includes multiple customer units, multiple file storage units for carrying out file storage and for storing member Data and multiple first file management units for being used to carry out file management；Described device includes：
First sending module, for a file upload request to be sent to first object file management unit, wherein described first File destination administrative unit is any one in the multiple first file management unit；
Second sending module is sent for that ought receive the first object file management unit according to the file upload request Instruction allow file upload the first response message when, according to first response message, to the first object file pipe It manages unit and sends the first data upload requests that request uploads the first data block；
First uploading module receives the first object file management unit according to first data upload requests for working as When the second response message sent, according to second response message, first data block is uploaded to first object file In storage unit；Wherein,
The letter of the first object file storage unit for storing first data block is carried in second response message Breath, the first object file storage unit are at least one of the multiple file storage unit.
13. a kind of electronic equipment, including memory, processor and storage are on a memory and the calculating that can run on a processor Machine program, which is characterized in that the processor is realized when executing the computer program such as any one of claim 1 to 6 institute The step of file memory method stated.
14. a kind of non-transient computer readable storage medium, is stored thereon with computer program, which is characterized in that the computer It is realized such as the step of claim 1 to 6 any one of them file memory method when program is executed by processor.
15. a kind of file storage device, which is characterized in that described device is applied to the first object file management in one-of-a-kind system Unit, wherein the one-of-a-kind system include multiple customer units, multiple file storage units for carrying out file storage and Multiple first file management units for storing metadata and for carrying out file management, the first object file management list Member is any one in the multiple first file management unit；Described device includes：
First receiving module, the file upload request for receiving the transmission of the first customer unit；
First sending module, for when according to the file upload request detect allow file upload when, to it is described first visitor Family unit sends the first response message that an instruction allows file to upload, so that first customer unit is according to first sound Message is answered to send the first data upload requests for uploading the first data block；
Second sending module, for according to first data upload requests, the second response to be sent to first customer unit Message, so that first customer unit is uploaded to first object according to second response message, by first data block In file storage unit；Wherein,
The letter of the first object file storage unit for storing first data block is carried in second response message Breath, the first object file storage unit are at least one of the multiple file storage unit.
16. a kind of electronic equipment, including memory, processor and storage are on a memory and the calculating that can run on a processor Machine program, which is characterized in that the processor is realized when executing the computer program such as any one of claim 7 to 11 institute The step of file memory method stated.
17. a kind of non-transient computer readable storage medium, is stored thereon with computer program, which is characterized in that the computer It is realized such as the step of claim 7 to 11 any one of them file memory method when program is executed by processor.
CN201810495187.5A 2018-05-22 2018-05-22 A kind of file memory method, device, electronic equipment and storage medium Pending CN108733822A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201810495187.5A CN108733822A (en) 2018-05-22 2018-05-22 A kind of file memory method, device, electronic equipment and storage medium

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201810495187.5A CN108733822A (en) 2018-05-22 2018-05-22 A kind of file memory method, device, electronic equipment and storage medium

Publications (1)

Publication Number Publication Date
CN108733822A true CN108733822A (en) 2018-11-02

Family

ID=63937902

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201810495187.5A Pending CN108733822A (en) 2018-05-22 2018-05-22 A kind of file memory method, device, electronic equipment and storage medium

Country Status (1)

Country Link
CN (1) CN108733822A (en)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112351111A (en) * 2020-12-30 2021-02-09 常州微亿智造科技有限公司 Near-real-time progress monitoring method for uploading industrial quality inspection file to server terminal

Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102694860A (en) * 2012-05-25 2012-09-26 北京邦诺存储科技有限公司 Method, equipment and system for data processing of cloud storage
CN103399941A (en) * 2013-08-13 2013-11-20 广州中国科学院软件应用技术研究所 Distributed file processing method, device and system

Patent Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102694860A (en) * 2012-05-25 2012-09-26 北京邦诺存储科技有限公司 Method, equipment and system for data processing of cloud storage
CN103399941A (en) * 2013-08-13 2013-11-20 广州中国科学院软件应用技术研究所 Distributed file processing method, device and system

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112351111A (en) * 2020-12-30 2021-02-09 常州微亿智造科技有限公司 Near-real-time progress monitoring method for uploading industrial quality inspection file to server terminal
CN112351111B (en) * 2020-12-30 2023-01-03 常州微亿智造科技有限公司 Near-real-time progress monitoring method for uploading industrial quality inspection file to server terminal

Similar Documents

Publication Publication Date Title
CN103890729B (en) computerized method, storage server and management server
CN103198090B (en) For optimizing the method and system of the storage distribution in virtual desktop environment
CN105824810B (en) A kind of Hadoop group system and data processing method
CN104503708B (en) The method and device of data hash storage
US8250171B2 (en) Content delivery apparatus, content delivery method, and content delivery program
CN109614377A (en) File delet method, device, equipment and the storage medium of distributed file system
CN106446159B (en) A kind of method of storage file, the first virtual machine and name node
CN108139941A (en) Dynamic resource allocation based on network flow control
CN104765661B (en) The multinode hot spare method of Metadata Service node in a kind of cloud storage service
CN104461768B (en) copy storage device and copy storage method
CN105373613A (en) Policy-based storage structure distribution
CN109240617A (en) Distributed memory system write request processing method, device, equipment and storage medium
CN104811473B (en) A kind of method, system and management system for creating virtual non-volatile storage medium
CN109582213A (en) Data reconstruction method and device, data-storage system
CN105912698A (en) Deletion method and system of data file in disk
CN105653209A (en) Object storage data transmitting method and device
EP3384384A1 (en) Methods and devices for acquiring data using virtual machine and host machine
CN108255989A (en) Picture storage method, device, terminal device and computer storage media
CN109271367A (en) Distributed file system multinode snapshot rollback method and system
CN104715044B (en) A kind of distributed system and its data manipulation method
CN103248636B (en) The system and method downloaded offline
CN109597903A (en) Image file processing apparatus and method, document storage system and storage medium
CN105978744B (en) A kind of resource allocation methods, apparatus and system
CN105306570A (en) Storage method of cluster data
CN107220388A (en) A kind of method and apparatus of upper transmitting file

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
RJ01 Rejection of invention patent application after publication

Application publication date: 20181102

RJ01 Rejection of invention patent application after publication