CN104239125A - Object processing method, distributive file system and client device - Google Patents
Object processing method, distributive file system and client device Download PDFInfo
- Publication number
- CN104239125A CN104239125A CN201410508764.1A CN201410508764A CN104239125A CN 104239125 A CN104239125 A CN 104239125A CN 201410508764 A CN201410508764 A CN 201410508764A CN 104239125 A CN104239125 A CN 104239125A
- Authority
- CN
- China
- Prior art keywords
- storage server
- information
- request
- identification information
- process parameter
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Abstract
The embodiment of the invention provides an object processing method, a distributive file system and a client device. A virtual machine can be installed in each storage server, an HTTP (Hyper Text Transfer Protocol) request containing processing parameter information transmitted by a client is received through an API (Application Program Interface) server, an address of one storage server stored with a to-be-processed object is acquired from a metadata server, and identification information and processing parameter information of the to-be-processed object are transmitted to the storage server located at the address. Since each storage server is provided with one virtual machine, the object on the corresponding storage server can be directly processed by the virtual machine which is installed on the storage server storing the object according to the processing parameter information, the downloading of the object is not needed, and more convenience and rapidness can be realized. By adopting the object processing method, the object can be rapidly processed. Meanwhile, the method also adequately utilizes the computing capacity of the storage servers, so that the utilization rate of the storage servers is increased.
Description
Technical field
The present invention relates to distributed file system technology field, particularly relate to object processing method, distributed file system and client device.
Background technology
Along with the development of internet, the object of the propagation in internet also gets more and more, as video, picture etc.Now, many websites are all stored object by distributed file system, are stored in by object in multiple storage servers of distributed file system.Meanwhile, the distributed file system of prior art adopts the communication mode based on http protocol to communicate with client device.
When needs process object, client device sends the HTTP request carrying object identity to distributed file system.Corresponding object in storage server is sent in client device according to HTTP request by distributed file system.Prior art is convenient to position produced problem by carrying out communication based on the communication mode of http protocol, easily determines that produced problem is the problem of client device or the problem of distributed file system.But this kind of communication mode is when processing stored object, just need object to be downloaded from the storage server of distributed file system object to be dealt with in client device.After treating object download, could process object.
Because object download is by the time more for cost, therefore existing object handles mode cannot realize the fast processing to object.
Summary of the invention
The object of the embodiment of the present invention is to provide a kind of object processing method, distributed file system and client device, to realize the fast processing of object.
For achieving the above object, the embodiment of the invention discloses a kind of object processing method, be applied in distributed file system, described distributed file system is connected with client device communications, meta data server, application programming interfaces API service device and multiple storage server storing object is included in described distributed file system, each described storage server is all provided with a virtual machine, described method comprises:
Described API service device receives the HTTP request that client device sends;
When the type of described HTTP request is object handles request, the identification information of the object in described request is sent to described meta data server by described API service device, wherein, identification information and the process parameter information of object to be dealt with is carried in described object handles request;
Described API service device receives the address information storing the storage server of the object with described identification information that described meta data server returns;
The identification information of described object and process parameter information are sent in the storage server with described address information by described API service device;
The virtual machine be arranged in the storage server with described address information processes the object with described identification information according to described process parameter information.
Optionally, the identification information of described object and process parameter information are sent to after in the storage server with described address information at described API service device, described method also comprises:
The storage server with described address information determines to have the store path of the object of described identification information according to described identification information;
There is the internet protocol address of storage server by this storage server of described address information, described store path and described process parameter information encapsulate, generate pending message;
Described pending message is inserted in message queue by the storage server with described address information;
The described virtual machine be arranged in the storage server with described address information processes the object with described identification information according to described process parameter information, comprising:
The virtual machine be arranged in the storage server with described address information judges that whether the IP address in the pending message of the new insertion in described message queue is identical with the IP address of the storage server at this virtual machine place, if so, then according to the process parameter information in the pending message of described new insertion, the object being stored in described store path is processed.
Optionally, also comprise:
The result processed object is sent in external unit by the virtual machine be arranged in the storage server with described address information.
Optionally, when the described type in described HTTP request is object handles request, the identification information of the object in described request is sent to described meta data server by described API service device, comprising:
The type of described HTTP request determined by described API service device according to the request header of described HTTP request;
When the type determining described HTTP request is object handles request, the identification information of the object in described request is sent to described meta data server.
Optionally, the identification information of described object and process parameter information are sent to after in the storage server with described address information at described API service device, described method also comprises:
Described API service device receives the storage server with described address information and described pending message is inserted the message returned after in message queue and to fall in lines response, and response of described message being fallen in lines returns to described client device.
A kind of object processing method, be applied in client device, described client device and distributed file system communicate to connect, meta data server, application programming interfaces API service device and multiple storage server storing object is included in described distributed file system, each described storage server is all provided with a virtual machine, described method comprises:
Described client device sends the HTTP request of identification information and the process parameter information carrying object to be dealt with to described API service device, to make:
Described in the identification of described API service device, the type of HTTP request is object handles request, the identification information of the object in described request is sent to described meta data server, receive the address information storing the storage server of the object with described identification information that described meta data server returns, the identification information of described object and process parameter information are sent in the storage server with described address information, and then the virtual machine making to be arranged in the storage server with described address information processes the object with described identification information according to described process parameter information.
A kind of distributed file system, described distributed file system is connected with client device communications, described distributed file system comprises: meta data server, application programming interfaces API service device and multiple storage server storing object, each described storage server includes the virtual machine unit running virtual machine, described API service device comprises: request reception unit, mark transmitting element, title receiving element and information transmitting unit
Described request receiving element, for receiving the HTTP request that client device sends;
Described mark transmitting element, for when the type of described HTTP request is object handles request, the identification information of the object in described request is sent to described meta data server, wherein, in described object handles request, carries identification information and the process parameter information of object to be dealt with;
Described title receiving element, for receiving the address information storing the storage server of the object with described identification information that described meta data server returns;
Described information transmitting unit, for being sent in the storage server with described address information by the identification information of described object and process parameter information;
There is the virtual machine unit in the storage server of described address information, for processing the object with described identification information according to described process parameter information.
Optionally, also comprise in described storage server: path determining unit, Information encapsulation unit and message inserting unit,
There is the described path determining unit in the storage server of described address information, for the identification information of described object and process parameter information being sent to after in the storage server with described address information in described information transmitting unit, determine the store path of the object with described identification information according to described identification information;
There is the described Information encapsulation unit in the storage server of described address information, encapsulating for this being had the internet protocol address of the storage server of described address information, described store path and described process parameter information, generating pending message;
There is the described message inserting unit in the storage server of described address information, for inserting in message queue by described pending message;
There is the described virtual machine unit in the storage server of described address information, comprising: IP address judgment sub-unit and object handles subelement,
Described IP address judgment sub-unit, whether identical with the IP address of the storage server at this virtual machine unit place for judging the IP address in the pending message of the new insertion in described message queue, if so, then trigger described object handles subelement;
Described object handles subelement, for processing the object being stored in described store path according to the process parameter information in the pending message of described new insertion.
Optionally, there is the described virtual machine unit in the storage server of described address information, also comprise: result returns unit, for the result processed object is sent in external unit.
Optionally, described mark transmitting element, comprising: request type determination subelement and mark send subelement,
Described request type determination unit, for determining the type of described HTTP request according to the request header of described HTTP request;
Described mark sends subelement, for when the type that described request type determination unit determines described HTTP request is object handles request, the identification information of the object in described request is sent to described meta data server.
Optionally, described API service device also comprises: response pass unit, for the identification information of described object and process parameter information being sent to after in the storage server with described address information in described information transmitting unit, receive the storage server with described address information described pending message to be inserted the message returned after in message queue and to fall in lines response, and response of described message being fallen in lines returns to described client device.
A kind of client device, described client device and distributed file system communicate to connect, meta data server, application programming interfaces API service device and multiple storage server storing object is included in described distributed file system, each described storage server is all provided with a virtual machine, described client device comprises: request transmitting unit, for sending the HTTP request of identification information and the process parameter information carrying object to be dealt with to described API service device, to make:
Described in the identification of described API service device, the type of HTTP request is object handles request, the identification information of the object in described request is sent to described meta data server, receive the address information storing the storage server of the object with described identification information that described meta data server returns, the identification information of described object and process parameter information are sent in the storage server with described address information, and then the virtual machine making to be arranged in the storage server with described address information processes the object with described identification information according to described process parameter information.
The object processing method that the embodiment of the present invention provides, distributed file system and client device, a virtual machine can be all installed in each storage server, receive the HTTP request including process parameter information of client transmission by API service device and from meta data server, obtain the storage server address that handling object is wanted in storage to some extent, the identification information of object to be dealt with and process parameter information are sent to the storage server being arranged in this address.Owing to being all provided with a virtual machine in each storage server, therefore store the virtual machine that the storage server of this object is installed just directly to process this object on this storage server according to process parameter information, and no longer need to carry out object download, convenient and quick.Therefore present invention achieves the fast processing of object.Meanwhile, the present invention also takes full advantage of the computing power of storage server, and the utilization factor of storage server is improved.
Certainly, arbitrary product of the present invention is implemented or method must not necessarily need to reach above-described all advantages simultaneously.
Accompanying drawing explanation
In order to be illustrated more clearly in the embodiment of the present invention or technical scheme of the prior art, be briefly described to the accompanying drawing used required in embodiment or description of the prior art below, apparently, accompanying drawing in the following describes is only some embodiments of the present invention, for those of ordinary skill in the art, under the prerequisite not paying creative work, other accompanying drawing can also be obtained according to these accompanying drawings.
The process flow diagram of a kind of object processing method that Fig. 1 provides for the embodiment of the present invention;
The process flow diagram of the another kind of object processing method that Fig. 2 provides for the embodiment of the present invention;
The structural representation of a kind of distributed file system that Fig. 3 provides for the embodiment of the present invention;
The structural representation of API service device in a kind of distributed file system that Fig. 4 provides for the embodiment of the present invention;
The structural representation of storage server in a kind of distributed file system that Fig. 5 provides for the embodiment of the present invention.
Embodiment
Below in conjunction with the accompanying drawing in the embodiment of the present invention, be clearly and completely described the technical scheme in the embodiment of the present invention, obviously, described embodiment is only the present invention's part embodiment, instead of whole embodiments.Based on the embodiment in the present invention, those of ordinary skill in the art, not making the every other embodiment obtained under creative work prerequisite, belong to the scope of protection of the invention.
As shown in Figure 1, embodiments provide a kind of object processing method, be applied in distributed file system, described distributed file system is connected with client device communications, meta data server, application programming interfaces API service device and multiple storage server storing object is included in described distributed file system, each described storage server is all provided with a virtual machine, the method can comprise:
S100, described API service device receive the HTTP request that client device sends;
API (Application Programming Interface, application programming interface) be the calling interface that operating system leaves application program for, application program makes operating system go the order of executive utility by the API of call operation system.
HTTP request refers to the request message from client to server end, can comprise: use agreement and process parameter information etc. to the identifier of the requesting method of resource, resource, resource.
Wherein, API service device, meta data server, storage server can be arranged in identical or different physical equipment.
The metadata information needed for various distributed file system can be preserved, as the list object etc. that storage server address information, storage server store in meta data server.Wherein, metadata information is the information for data of description, is the descriptive information to data and resource.Optionally, the meta data server in the present invention can have multiple stage.
S200, when the type of described HTTP request is object handles request, the identification information of the object in described request is sent to described meta data server by described API service device, wherein, identification information and the process parameter information of object to be dealt with is carried in described object handles request;
Wherein, process parameter information can comprise the various process parameters of carrying out needed for object handles, as: when needs carry out transcoding to video, the process parameter information in object handles request can comprise target bit rate.Concrete process parameter information has various forms, in actual applications, suitable process parameter information can be selected as required to add in HTTP request.Object in the present invention can be video, picture, audio frequency, text etc.Wherein, the storage address information of the rear object of process in object handles request, can also be carried, store so that the object after process is sent to corresponding memory address.This memory address can be the memory address in storage server, also can be the memory address in external unit.
Wherein, S200 can comprise:
The type of described HTTP request determined by described API service device according to the request header of described HTTP request;
When the type determining described HTTP request is object handles request, the identification information of the object in described request is sent to described meta data server.
Wherein, the mark of request type can be included in request header, mark in the request header corresponding with object handles request can set as required and revise, as: setting X-QUICK-VIDEO-PROCESS is the mark of request type corresponding to object handles request.Like this, just can identify according to the mark of request type in request is the request of which kind of type.
S300, described API service device receive the address information storing the storage server of the object with described identification information that described meta data server returns;
Optionally, when have at least two storage servers store the object with described identification information time, the address information that meta data server returns can for storing the address list of the storage server of the object with described identification information.Be understandable that, in some cases, distributed file system can repeatedly back up and storage backup a certain object.Like this, the security of this object just improves greatly, should not lose.
The identification information of described object and process parameter information are sent in the storage server with described address information by S400, described API service device;
S500, the virtual machine be arranged in the storage server with described address information process the object with described identification information according to described process parameter information.
Virtual machine (Virtual Machine) be by software simulation have complete hardware system function, the complete computer operated in a completely isolated environment.By virtual machine, logically by virtual machine and hardware environment isolation, both can realize the process of virtual machine to task, original layout of the hardware at virtual machine place can not have been disturbed again.Virtual machine and storage server can by bind-mount mechanism, ensure that the object storage file directory path seen in virtual machine and in storage server is consistent, and virtual machine can obtain the configuration information of the storage server at its place, such as IP address etc.Because virtual machine technique is prior art, conveniently, repeat no more.
Further, the object after processing can carry out storing or being sent in external unit by storage server.As: the object after process is stored in this storage server by the storage server being arranged on address information as described in having, or, be arranged on the storage server with described address information and the object after process is sent in external unit stores.
When being arranged on the storage server with described address information and being stored in this storage server by the object after process, the present invention can reduce by a transmitting procedure again relative to prior art.Object is downloading to after client device processes by prior art from certain storage server, also need the object after by process to re-send to this storage server and store, this just needs twice object transfer process.And under the solution of the present invention, after virtual machine on this storage server processes the object being stored in this storage server, can being directly stored in this storage server, without the need to carrying out object transfer, therefore being also improved the preservation speed of the rear object of process.
A kind of object processing method that the embodiment of the present invention provides, a virtual machine can be all installed in each storage server, receive the HTTP request including process parameter information of client transmission by API service device and from meta data server, obtain the storage server address that handling object is wanted in storage to some extent, the identification information of object to be dealt with and process parameter information are sent to the storage server being arranged in this address.Owing to being all provided with a virtual machine in each storage server, therefore store the virtual machine that the storage server of this object is installed just directly to process this object on this storage server according to process parameter information, and no longer need to carry out object download, convenient and quick.Therefore present invention achieves the fast processing of object.Meanwhile, the present invention also takes full advantage of the computing power of storage server, and the utilization factor of storage server is improved.
As shown in Figure 2, in the another kind of object processing method that the embodiment of the present invention provides, after S400, can also comprise: S410 to S430, wherein:
S410, the storage server with described address information determine to have the store path of the object of described identification information according to described identification information;
S420, the internet protocol address of storage server by this storage server with described address information, described store path and described process parameter information encapsulate, and generate pending message;
Described pending message is inserted in message queue by S430, the storage server with described address information;
As shown in Figure 2, the S500 in embodiment illustrated in fig. 1 can comprise:
S510, the virtual machine be arranged in the storage server with described address information judge that whether the IP address in the pending message of the new insertion in described message queue is identical with the IP address of the storage server at this virtual machine place, if so, then S520 is performed;
S520, the virtual machine be arranged in the storage server with described address information process the object being stored in described store path according to the process parameter information in the pending message of described new insertion.
Be understandable that, unified management can be carried out to various task by the mode of message queue.
On Fig. 1 or basis embodiment illustrated in fig. 2, the another kind of object processing method that the embodiment of the present invention provides, can also comprise:
The result processed object is sent in external unit by the virtual machine be arranged in the storage server with described address information.
Wherein, external unit can be database facility etc.Result is sent to external unit, the function such as monitoring or result storage of external unit can be realized.
On Fig. 1 or basis embodiment illustrated in fig. 2, the another kind of object processing method that the embodiment of the present invention provides, after S400, can also comprise:
Described API service device receives the storage server with described address information and described pending message is inserted the message returned after in message queue and to fall in lines response, and response of described message being fallen in lines returns to described client device.
Being fallen in lines by message, just can to inform that its HTTP request of client device has been added into message queue in time medium pending in response, thus make client device know process progress.Certainly, if the address information storing the storage server of the object with described identification information that meta data server returns is for empty, then 404 message can be returned to client device by API service device.The implication of 404 message being indicative is not for finding asked object.
Corresponding with said method embodiment, present invention also offers a kind of distributed file system.
A kind of distributed file system that the embodiment of the present invention provides, communicate to connect with client device 010, as shown in Figure 3, this distributed file system 011 can comprise: meta data server 001, application programming interfaces API service device 002 and multiple storage server 003 storing object, each described storage server 003 includes the virtual machine unit running virtual machine, as shown in Figure 4, API service device 002 can comprise: request reception unit 100, mark transmitting element 200, title receiving element 300 and information transmitting unit 400
Described request receiving element 100, for receiving the HTTP request that client device 010 sends;
Described mark transmitting element 200, for when the type of described HTTP request is object handles request, the identification information of the object in described request is sent to described meta data server 001, wherein, identification information and the process parameter information of object to be dealt with is carried in described object handles request;
Wherein, described mark transmitting element 200, can comprise: request type determination subelement and mark send subelement,
Described request type determination unit, for determining the type of described HTTP request according to the request header of described HTTP request;
Described mark sends subelement, for when the type that described request type determination unit determines described HTTP request is object handles request, the identification information of the object in described request is sent to described meta data server.
Described title receiving element 300, for receiving the address information storing the storage server of the object with described identification information that described meta data server 001 returns;
Described information transmitting unit 400, for being sent in the storage server with described address information by the identification information of described object and process parameter information;
There is the virtual machine unit in the storage server 003 of described address information, for processing the object with described identification information according to described process parameter information.
A kind of distributed file system that the embodiment of the present invention provides, a virtual machine can be all installed in each storage server, receive the HTTP request including process parameter information of client transmission by API service device and from meta data server, obtain the storage server address that handling object is wanted in storage to some extent, the identification information of object to be dealt with and process parameter information are sent to the storage server being arranged in this address.Owing to being all provided with a virtual machine in each storage server, therefore store the virtual machine that the storage server of this object is installed just directly to process this object on this storage server according to process parameter information, and no longer need to carry out object download, convenient and quick.Therefore present invention achieves the fast processing of object.Meanwhile, the present invention also takes full advantage of the computing power of storage server, and the utilization factor of storage server is improved.
In an alternative embodiment of the invention, API service device 002 can also comprise: response pass unit, for the identification information of described object and process parameter information being sent to after in the storage server with described address information in described information transmitting unit 400, receive the storage server with described address information described pending message to be inserted the message returned after in message queue and to fall in lines response, and response of described message being fallen in lines returns to described client device 010.
As shown in Figure 5, in the another kind of distributed file system that the embodiment of the present invention provides, in storage server 003 except virtual machine unit 004, can also comprise: path determining unit 005, Information encapsulation unit 006 and message inserting unit 007,
There is the described path determining unit 005 in the storage server of described address information, for the identification information of described object and process parameter information being sent to after in the storage server with described address information in described information transmitting unit, determine the store path of the object with described identification information according to described identification information;
There is the described Information encapsulation unit 006 in the storage server of described address information, encapsulating for this being had the internet protocol address of the storage server of described address information, described store path and described process parameter information, generating pending message;
There is the described message inserting unit 007 in the storage server of described address information, for inserting in message queue by described pending message;
There is the described virtual machine unit 004 in the storage server of described address information, can comprise: IP address judgment sub-unit 004a and object handles subelement 004b,
Described IP address judgment sub-unit 004a, whether identical with the IP address of the storage server at this virtual machine unit place for judging the IP address in the pending message of the new insertion in described message queue, if so, then described object handles subelement 004b is triggered;
Described object handles subelement 004b, for processing the object being stored in described store path according to the process parameter information in the pending message of described new insertion.
Wherein, there is the described virtual machine unit in the storage server of described address information, can also comprise: result returns unit, for the result processed object is sent in external unit.
Present invention also offers another kind of object processing method, be applied in client device, as shown in Figure 3, described client establishes 010 to communicate to connect with distributed file system 011, as shown in Figure 3, include meta data server 001, application programming interfaces API service device 002 and multiple storage server 003 storing object in described distributed file system 011, each described storage server 003 is all provided with a virtual machine, and the method can comprise:
Described client device 010 sends the HTTP request of identification information and the process parameter information carrying object to be dealt with to described API service device 002, to make:
Described API service device 002 identifies that the type of described HTTP request is object handles request, the identification information of the object in described request is sent to described meta data server 001, receive the address information of the storage server 003 storing the object with described identification information that described meta data server 001 returns, the identification information of described object and process parameter information being sent to has in the storage server 003 of described address information, and then make to be arranged on the virtual machine had in the storage server 003 of described address information and according to described process parameter information, the object with described identification information is processed.
A kind of object processing method that the embodiment of the present invention provides, process parameter information is added in HTTP request, so just making to store institute wants virtual machine that the storage server of handling object is installed directly can process this object on this storage server according to processing parameter information, and no longer need to carry out object download, convenient and quick.Therefore present invention achieves the fast processing of object.
Accordingly, as shown in Figure 3, present invention also offers a kind of client device 010, described client device 010 communicates to connect with distributed file system 011, meta data server 001 is included in described distributed file system 011, application programming interfaces API service device 002 and multiple storage server 003 storing object, each described storage server 003 is all provided with a virtual machine, described client device 010 comprises: request transmitting unit, for sending the HTTP request of identification information and the process parameter information carrying object to be dealt with to described API service device 002, to make:
Described API service device 002 identifies that the type of described HTTP request is object handles request, the identification information of the object in described request is sent to described meta data server 001, receive the address information of the storage server 003 storing the object with described identification information that described meta data server 001 returns, the identification information of described object and process parameter information being sent to has in the storage server 003 of described address information, and then make to be arranged on the virtual machine had in the storage server 003 of described address information and according to described process parameter information, the object with described identification information is processed.
A kind of client device that the embodiment of the present invention provides, process parameter information is added in HTTP request, so just making to store institute wants virtual machine that the storage server of handling object is installed directly can process this object on this storage server according to processing parameter information, and no longer need to carry out object download, convenient and quick.Therefore present invention achieves the fast processing of object.
It should be noted that, in this article, the such as relational terms of first and second grades and so on is only used for an entity or operation to separate with another entity or operational zone, and not necessarily requires or imply the relation that there is any this reality between these entities or operation or sequentially.And, term " comprises ", " comprising " or its any other variant are intended to contain comprising of nonexcludability, thus make to comprise the process of a series of key element, method, article or equipment and not only comprise those key elements, but also comprise other key elements clearly do not listed, or also comprise by the intrinsic key element of this process, method, article or equipment.When not more restrictions, the key element limited by statement " comprising ... ", and be not precluded within process, method, article or the equipment comprising described key element and also there is other identical element.
Each embodiment in this instructions all adopts relevant mode to describe, between each embodiment identical similar part mutually see, what each embodiment stressed is the difference with other embodiments.Especially, for system embodiment, because it is substantially similar to embodiment of the method, so description is fairly simple, relevant part illustrates see the part of embodiment of the method.
The foregoing is only preferred embodiment of the present invention, be not intended to limit protection scope of the present invention.All any amendments done within the spirit and principles in the present invention, equivalent replacement, improvement etc., be all included in protection scope of the present invention.
Claims (12)
1. an object processing method, it is characterized in that, be applied in distributed file system, described distributed file system is connected with client device communications, meta data server, application programming interfaces API service device and multiple storage server storing object is included in described distributed file system, each described storage server is all provided with a virtual machine, described method comprises:
Described API service device receives the HTTP request that client device sends;
When the type of described HTTP request is object handles request, the identification information of the object in described request is sent to described meta data server by described API service device, wherein, identification information and the process parameter information of object to be dealt with is carried in described object handles request;
Described API service device receives the address information storing the storage server of the object with described identification information that described meta data server returns;
The identification information of described object and process parameter information are sent in the storage server with described address information by described API service device;
The virtual machine be arranged in the storage server with described address information processes the object with described identification information according to described process parameter information.
2. method according to claim 1, is characterized in that, the identification information of described object and process parameter information are sent to after in the storage server with described address information at described API service device, described method also comprises:
The storage server with described address information determines to have the store path of the object of described identification information according to described identification information;
There is the internet protocol address of storage server by this storage server of described address information, described store path and described process parameter information encapsulate, generate pending message;
Described pending message is inserted in message queue by the storage server with described address information;
The described virtual machine be arranged in the storage server with described address information processes the object with described identification information according to described process parameter information, comprising:
The virtual machine be arranged in the storage server with described address information judges that whether the IP address in the pending message of the new insertion in described message queue is identical with the IP address of the storage server at this virtual machine place, if so, then according to the process parameter information in the pending message of described new insertion, the object being stored in described store path is processed.
3. method according to claim 1 and 2, is characterized in that, also comprises:
The result processed object is sent in external unit by the virtual machine be arranged in the storage server with described address information.
4. method according to claim 1 and 2, is characterized in that, when the described type in described HTTP request is object handles request, the identification information of the object in described request is sent to described meta data server by described API service device, comprising:
The type of described HTTP request determined by described API service device according to the request header of described HTTP request;
When the type determining described HTTP request is object handles request, the identification information of the object in described request is sent to described meta data server.
5. method according to claim 2, is characterized in that, the identification information of described object and process parameter information are sent to after in the storage server with described address information at described API service device, described method also comprises:
Described API service device receives the storage server with described address information and described pending message is inserted the message returned after in message queue and to fall in lines response, and response of described message being fallen in lines returns to described client device.
6. an object processing method, it is characterized in that, be applied in client device, described client device and distributed file system communicate to connect, meta data server, application programming interfaces API service device and multiple storage server storing object is included in described distributed file system, each described storage server is all provided with a virtual machine, described method comprises:
Described client device sends the HTTP request of identification information and the process parameter information carrying object to be dealt with to described API service device, to make:
Described in the identification of described API service device, the type of HTTP request is object handles request, the identification information of the object in described request is sent to described meta data server, receive the address information storing the storage server of the object with described identification information that described meta data server returns, the identification information of described object and process parameter information are sent in the storage server with described address information, and then the virtual machine making to be arranged in the storage server with described address information processes the object with described identification information according to described process parameter information.
7. a distributed file system, it is characterized in that, described distributed file system is connected with client device communications, described distributed file system comprises: meta data server, application programming interfaces API service device and multiple storage server storing object, each described storage server includes the virtual machine unit running virtual machine, described API service device comprises: request reception unit, mark transmitting element, title receiving element and information transmitting unit
Described request receiving element, for receiving the HTTP request that client device sends;
Described mark transmitting element, for when the type of described HTTP request is object handles request, the identification information of the object in described request is sent to described meta data server, wherein, in described object handles request, carries identification information and the process parameter information of object to be dealt with;
Described title receiving element, for receiving the address information storing the storage server of the object with described identification information that described meta data server returns;
Described information transmitting unit, for being sent in the storage server with described address information by the identification information of described object and process parameter information;
There is the virtual machine unit in the storage server of described address information, for processing the object with described identification information according to described process parameter information.
8. system according to claim 7, is characterized in that, also comprises in described storage server: path determining unit, Information encapsulation unit and message inserting unit,
There is the described path determining unit in the storage server of described address information, for the identification information of described object and process parameter information being sent to after in the storage server with described address information in described information transmitting unit, determine the store path of the object with described identification information according to described identification information;
There is the described Information encapsulation unit in the storage server of described address information, encapsulating for this being had the internet protocol address of the storage server of described address information, described store path and described process parameter information, generating pending message;
There is the described message inserting unit in the storage server of described address information, for inserting in message queue by described pending message;
There is the described virtual machine unit in the storage server of described address information, comprising: IP address judgment sub-unit and object handles subelement,
Described IP address judgment sub-unit, whether identical with the IP address of the storage server at this virtual machine unit place for judging the IP address in the pending message of the new insertion in described message queue, if so, then trigger described object handles subelement;
Described object handles subelement, for processing the object being stored in described store path according to the process parameter information in the pending message of described new insertion.
9. the system according to claim 7 or 8, is characterized in that, has the described virtual machine unit in the storage server of described address information, also comprises: result returns unit, for the result processed object being sent in external unit.
10. the system according to claim 7 or 8, is characterized in that, described mark transmitting element, comprising: request type determination subelement and mark send subelement,
Described request type determination unit, for determining the type of described HTTP request according to the request header of described HTTP request;
Described mark sends subelement, for when the type that described request type determination unit determines described HTTP request is object handles request, the identification information of the object in described request is sent to described meta data server.
11. systems according to claim 7 or 8, it is characterized in that, described API service device also comprises: response pass unit, for the identification information of described object and process parameter information being sent to after in the storage server with described address information in described information transmitting unit, receive the storage server with described address information described pending message to be inserted the message returned after in message queue and to fall in lines response, and response of described message being fallen in lines returns to described client device.
12. 1 kinds of client devices, it is characterized in that, described client device and distributed file system communicate to connect, meta data server, application programming interfaces API service device and multiple storage server storing object is included in described distributed file system, each described storage server is all provided with a virtual machine, described client device comprises: request transmitting unit, for sending the HTTP request of identification information and the process parameter information carrying object to be dealt with to described API service device, to make:
Described in the identification of described API service device, the type of HTTP request is object handles request, the identification information of the object in described request is sent to described meta data server, receive the address information storing the storage server of the object with described identification information that described meta data server returns, the identification information of described object and process parameter information are sent in the storage server with described address information, and then the virtual machine making to be arranged in the storage server with described address information processes the object with described identification information according to described process parameter information.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201410508764.1A CN104239125B (en) | 2014-09-28 | 2014-09-28 | Object processing method, distributed file system and client device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201410508764.1A CN104239125B (en) | 2014-09-28 | 2014-09-28 | Object processing method, distributed file system and client device |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN104239125A true CN104239125A (en) | 2014-12-24 |
| CN104239125B CN104239125B (en) | 2018-04-10 |
Family
ID=52227259
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201410508764.1A Active CN104239125B (en) | 2014-09-28 | 2014-09-28 | Object processing method, distributed file system and client device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN104239125B (en) |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105701179A (en) * | 2016-01-06 | 2016-06-22 | 南京斯坦德云科技股份有限公司 | Windows access method of distributed file system based on UniWhale |
| CN106254544A (en) * | 2016-09-28 | 2016-12-21 | 乐视控股（北京）有限公司 | Data access method and device |
| CN107844542A (en) * | 2017-10-26 | 2018-03-27 | 山东浪潮通软信息科技有限公司 | A kind of distributed document storage method and device |
| CN109428861A (en) * | 2017-08-29 | 2019-03-05 | 阿里巴巴集团控股有限公司 | Network communication method and equipment |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1619522A (en) * | 2003-11-17 | 2005-05-25 | 联想(北京)有限公司 | Computer group file service system and its input output treatment method |
| US20110219205A1 (en) * | 2010-03-05 | 2011-09-08 | Solidfire, Inc. | Distributed Data Storage System Providing De-duplication of Data Using Block Identifiers |
| CN102932472A (en) * | 2012-11-13 | 2013-02-13 | 深圳市路友网络科技有限公司 | Data processing method and communication system |
| US20130332418A1 (en) * | 2012-06-08 | 2013-12-12 | Electronics And Telecommunications Research Institute | Method of managing data in asymmetric cluster file system |
| CN103944994A (en) * | 2014-04-25 | 2014-07-23 | 中国联合网络通信集团有限公司 | Distributed resource obtaining method and device |
-
2014
- 2014-09-28 CN CN201410508764.1A patent/CN104239125B/en active Active
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1619522A (en) * | 2003-11-17 | 2005-05-25 | 联想(北京)有限公司 | Computer group file service system and its input output treatment method |
| US20110219205A1 (en) * | 2010-03-05 | 2011-09-08 | Solidfire, Inc. | Distributed Data Storage System Providing De-duplication of Data Using Block Identifiers |
| US20130332418A1 (en) * | 2012-06-08 | 2013-12-12 | Electronics And Telecommunications Research Institute | Method of managing data in asymmetric cluster file system |
| CN102932472A (en) * | 2012-11-13 | 2013-02-13 | 深圳市路友网络科技有限公司 | Data processing method and communication system |
| CN103944994A (en) * | 2014-04-25 | 2014-07-23 | 中国联合网络通信集团有限公司 | Distributed resource obtaining method and device |
Cited By (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105701179A (en) * | 2016-01-06 | 2016-06-22 | 南京斯坦德云科技股份有限公司 | Windows access method of distributed file system based on UniWhale |
| CN105701179B (en) * | 2016-01-06 | 2018-12-18 | 南京斯坦德云科技股份有限公司 | The form access method of distributed file system based on UniWhale |
| CN106254544A (en) * | 2016-09-28 | 2016-12-21 | 乐视控股（北京）有限公司 | Data access method and device |
| CN109428861A (en) * | 2017-08-29 | 2019-03-05 | 阿里巴巴集团控股有限公司 | Network communication method and equipment |
| CN107844542A (en) * | 2017-10-26 | 2018-03-27 | 山东浪潮通软信息科技有限公司 | A kind of distributed document storage method and device |
Also Published As
| Publication number | Publication date |
|---|---|
| CN104239125B (en) | 2018-04-10 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110703980B (en) | File transmission method and device | |
| EP3057281B1 (en) | M2m application remote registration method, device, system and storage medium | |
| CN111343048B (en) | Pressure testing method and device, computer device and storage medium | |
| CN108377247B (en) | Message pushing method and device | |
| CN108833578B (en) | Method and related equipment for duplex communication based on FIFO named pipeline | |
| CN104239125A (en) | Object processing method, distributive file system and client device | |
| CN109522462B (en) | Cloud query method, device, equipment and storage medium based on block chain | |
| CN104462033A (en) | Method and device for realizing editing operation among devices | |
| CN112445700A (en) | Test method and device | |
| CN111491037A (en) | Communication method with object storage server through SFTP data stream | |
| US20030140146A1 (en) | Method and system for interconnecting a Web server with a wireless portable communications device | |
| CN103024056B (en) | Data processing method, load equalizer and cloud system | |
| CN114390044A (en) | File uploading method, system, device and storage medium | |
| CN110808974A (en) | Data acquisition method and device, computer device and storage medium | |
| CN104113510A (en) | Virtual desktop system and message data transmitting method thereof | |
| CN111917810B (en) | Cloud communication method and device, user equipment and network equipment | |
| CN104821939B (en) | It is a kind of to be used to implement computer and the method and apparatus of mobile equipment room asynchronous communication | |
| EP3232631B1 (en) | Content sharing method and server | |
| US10412147B2 (en) | Data processing system, center apparatus, and program | |
| CN110245192A (en) | Identity information querying method and system | |
| CN101561955A (en) | Method for managing data sharing in personal data bank and network side device | |
| CN113746794B (en) | Account processing method and device, electronic equipment and computer storage medium | |
| CN104519109A (en) | Method and device for sharing file among multiple devices | |
| CN108573162A (en) | data copy system, method and device | |
| CN113434541A (en) | Method and device for sending query result, storage medium and electronic device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |