The content of the invention
It is a primary object of the present invention to provide a kind of distributed file system and its data processing method and device, it is intended to
Improve the stability and efficiency of data storage resource.
To achieve the above object, the Distributed File System Data processing method that the present invention is provided, the distributed document
System data processing method is comprised the following steps:
Receive the data resource that distributed type file system client side sends;
The processing unit to be accessed of the distributed type file system client side according to received data resource judgment;
Received data resource is processed;
By the data resource storage after treatment to respective memory regions.
Preferably, the data resource is stream medium data resource.
Preferably, the step of data resource that the reception distributed type file system client side sends, includes:
Receive the data block ID and namespace information of the data resource that distributed type file system client side sends.
Preferably, the data resource storage by after treatment includes to before the step of respective stored node:
Data after treatment are first cached to pre-stored equipment;
The data Cun Chudao respective memory regions that will be stored in pre-stored equipment.
Additionally, to achieve the above object, the present invention also provides a kind of Distributed File System Data processing unit, described point
Cloth file data processing unit includes:
Receiver module, the data resource for receiving distributed type file system client side transmission;
Judge module, to be accessed for the distributed type file system client side according to received data resource judgment
Destination object;
Processing module, for processing received data resource;
Memory module, for the data resource storage after treatment to be arrived into corresponding target storage domain.
Preferably, the receiver module includes:
Receiving unit, the data block ID and name for receiving the data resource of distributed type file system client side transmission is empty
Between information.
Preferably, the memory module includes:
Buffer unit, for the data after treatment first to be cached to pre-stored equipment;
Memory cell, for the data Cun Chudao respective memory regions that will be stored in pre-stored equipment.
Additionally, to achieve the above object, the present invention also provides a kind of distributed file system, the distributed file system
Including:
Distributed type file system client side, for transmitting data resource to the distributed file system node;
Distributed file system node, the data resource for processing the distributed type file system client side transmission, and
The distributed file system node also includes RTSP gateways;
The distributed type file system client side is connected with the distributed file system node signal;The distributed text
Part system node includes the Distributed File System Data processing unit as described in power is upper.
Preferably, distributed file system node and at least one RTSP gateways described at least one, and each are set
Can be in communication with each other between the distributed file system node, can be equally in communication with each other between each described RTSP gateway.
Preferably, the RTSP gateways are connected with least one distributed type file system client side, and the RTSP nets
Pass can simultaneously receive the data resource of at least one distributed type file system client side transmission.
The Distributed File System Data processing method of the embodiment of the present invention is by receiving distributed type file system client side
The data of transmission;The distributed type file system client side destination object to be accessed is judged according to received data;It is right
Received data are processed;By the corresponding target storage nodes of data Cun Chudao after treatment.By by client
The data of transmission are stored using the processing mode of distributed file system, can be with convenient, efficient and stabilization data storage
Resource.
Specific embodiment
It should be appreciated that the specific embodiments described herein are merely illustrative of the present invention, it is not intended to limit the present invention.
A kind of Distributed File System Data processing method of present invention offer, reference picture 1, in one embodiment, the distribution
Formula file system data processing method includes:
Step S10, receives the data resource that distributed type file system client side sends;
Specifically, the distributed type file system client side in the present embodiment includes that setting for RTSP data can be provided
It is standby, including but not limited to high-definition camera, computer, Set Top Box etc..The distributed type file system client side sends to be needed
To distributed file system, the distributed file system includes RTSP gateways and distributed field system to the data resource for the treatment of
System node, the data resource is sent to the RTSP gateways by RTSP, Real-time Transport Protocol.Wherein, the RTSP agreements are real-time
Flow host-host protocol, is an application layer protocol in ICP/IP protocol system, is the multimedia string for controlling sound or image
Stream protocol, is widely used in monitoring, net cast, program request.
It should be noted that the data resource is stream medium data resource.The rear end of existing streaming media video system
Storage is used mostly centralized architecture, if running into failure cannot ensure reliability of service.The present embodiment can deposit distribution
Storage resource is integrated with stream medium data resource, convenient, efficiently storage media data resource, is embodied larger reality and is answered
With value.
Preferably, in the present embodiment, it is desirable to which the distributed type file system client side not only sends the data to be stored money
Source will also simultaneously send the information such as the corresponding data block ID of data resource to back end.Client can be by pre-setting
Data store strategy determines the data block ID and corresponding namespace information of the data to be stored in distributed file system.
Need to store the data resource and the data block ID and namespace information simultaneously.
Preferably, the distributed file system receives the data money transmitted by the distributed type file system client side
Source, can prejudge the size of received data resource.If the size of data resource is less than preset value, can be by
Data resource is stored to cloud server;If data resource is more than preset value, can by data resource cutting for it is multiple from
Data, and store to local file system.Specifically can as needed be configured, not be limited herein.
Step S20, the distributed type file system client side target pair to be accessed is judged according to received data
As;
Received data resource is analyzed, the received data resource destination object to be accessed is judged,
Whether i.e. received data resource has the preservation address of determination, if the analysis according to the data resource for receiving can be obtained clearly
Know that the data resource needs the address for preserving, then preserved after directly data resource is processed to corresponding address.
Step S30, is processed received data resource;
Step S40, by the data resource storage after treatment to corresponding target storage domain.
Specifically, the RTSP gateways can be understood as be the distributed file system node a part, it is specific to use
In the data for processing the distributed type file system client side transmission.
After RTSP, Real-time Transport Protocol are sent to the RTSP gateways, the RTSP gateways are to being received for the data resource
To data processed, the data resource is further sent to the corresponding target storage of distributed file system after treatment
Domain.
The Distributed File System Data processing method of the present embodiment is sent by receiving distributed type file system client side
Data;The distributed type file system client side destination object unit to be accessed is judged according to received data;It is right
Received data are processed, and by the corresponding target storage nodes of data Cun Chudao after treatment.By by client
The data for sending are held to be stored using the processing mode of distributed file system, can be with convenient, efficient data storage resource.
Further, reference picture 2, based on Distributed File System Data processing method first embodiment of the present invention, at this
In invention Distributed File System Data processing method second embodiment, above-mentioned steps S40 includes:
Step S41, the data resource after treatment is first cached to pre-stored equipment;
Step S42, the data resource storage that will be stored in pre-stored equipment to corresponding target storage domain.
In the present embodiment, the data resource crossed by RTSP gateway processes is stored to before corresponding memory node, also
Solid state hard disc can first be used as pre-stored equipment, first by treated data buffer storage to pre-stored equipment, then by data
Store in distributed file system in corresponding memory node.
Can mitigate the storage of distributed file system memory node using treated data are first written into pre-stored equipment
Pressure, while the data in pre-stored equipment caching can be such that data do not lose as a kind of interim storage, make data storage
Process is more stablized.
The present invention also provides a kind of Distributed File System Data processing unit, reference picture 3, in one embodiment, this hair
The Distributed File System Data processing unit of bright offer includes:
Receiver module 10, the data resource for receiving distributed type file system client side transmission;
Specifically, the distributed type file system client side in the present embodiment includes that setting for RTSP data can be provided
It is standby, including but not limited to high-definition camera, computer, Set Top Box etc..The distributed type file system client side sends to be needed
To distributed file system, the distributed file system includes RTSP gateways and distributed field system to the data resource for the treatment of
System node, the data resource is sent to the RTSP gateways by RTSP, Real-time Transport Protocol.Wherein, the RTSP agreements are real-time
Flow host-host protocol, is an application layer protocol in ICP/IP protocol system, is the multimedia string for controlling sound or image
Stream protocol, is widely used in monitoring, net cast, program request.
It should be noted that the data resource is stream medium data resource.The rear end of existing streaming media video system
Storage is used mostly centralized architecture, if running into failure cannot ensure reliability of service.The present embodiment can deposit distribution
Storage resource is integrated with stream medium data resource, convenient, efficiently storage media data resource, is embodied larger reality and is answered
With value.
Preferably, in the present embodiment, it is desirable to which the distributed type file system client side not only sends the data to be stored money
Source will also simultaneously send the information such as the corresponding data block ID of data resource to back end.Client can be by pre-setting
Data store strategy determines the data block ID and corresponding namespace information of the data to be stored in distributed file system.
Need to store the data resource and the data block ID and namespace information simultaneously.
Preferably, the distributed file system receives the data money transmitted by the distributed type file system client side
Source, can prejudge the size of received data resource.If the size of data resource is less than preset value, can be by
Data resource is stored to cloud server;If data resource is more than preset value, can by data resource cutting for it is multiple from
Data, and store to local file system.Specifically can as needed be configured, not be limited herein.
Judge module 20, for judging what the distributed type file system client side to be accessed according to received data
Destination object;
Received data resource is analyzed, the received data resource destination object to be accessed is judged,
Whether i.e. received data resource has the preservation address of determination, if the analysis according to the data resource for receiving can be obtained clearly
Know that the data resource needs the address for preserving, then preserved after directly data resource is processed to corresponding address.
Processing module 30, for processing received data resource;
Memory module 40, for the data resource storage after treatment to be arrived into corresponding target storage domain.
Specifically, the RTSP gateways can be understood as be the distributed file system node a part, it is specific to use
In the data for processing the distributed type file system client side transmission.
After RTSP, Real-time Transport Protocol are sent to the RTSP gateways, the RTSP gateways are to being received for the data resource
To data processed, the data resource is further sent to the corresponding target storage of distributed file system after treatment
Domain.
The Distributed File System Data processing method of the present embodiment is sent by receiving distributed type file system client side
Data;The distributed type file system client side destination object to be accessed is judged according to received data;To being connect
The data for receiving are processed, and by the corresponding target storage nodes of data Cun Chudao after treatment.Sent out by by client
The data sent are stored using the processing mode of distributed file system, can be with convenient, efficient data storage resource.
Further, reference picture 4, based on Distributed File System Data processing unit first embodiment of the present invention, at this
In invention Distributed File System Data processing unit second embodiment, above-mentioned memory module 40 includes:
Buffer unit 41, for the data resource after treatment first to be cached to pre-stored equipment;
Respective memory regions are arrived in memory cell 42, the data resource storage for that will be stored in pre-stored equipment.
In the present embodiment, before the data storage that will be crossed by RTSP gateway processes to corresponding storage region, can be with
Solid state hard disc is first used as pre-stored equipment, first by treated data buffer storage to pre-stored equipment, then by data storage
To in distributed file system in corresponding storage region.It is understood that described storage region is to deposit data resource
In storage to the storage medium of distributed file system, described storage medium can be mechanical hard disk etc..
Can mitigate the storage of distributed file system memory node using treated data are first written into pre-stored equipment
Pressure, while the data in pre-stored equipment caching can be such that data do not lose as a kind of interim storage, make data storage
Process is more stablized.
The present invention also provides a kind of distributed file system, reference picture 5, in one embodiment, the distribution that the present invention is provided
Formula file system includes:
Distributed type file system client side, for transmitting data resource to the distributed file system node;
Distributed file system node, the data resource for processing the distributed type file system client side transmission, and
The distributed file system node also includes RTSP gateways;
The distributed type file system client side is connected with the distributed file system node signal;The distributed text
Part system node includes Distributed File System Data processing unit as described above.
It should be noted that setting distributed file system node and at least one described at least one in the present embodiment
Can be in communication with each other between RTSP gateways, and each described distributed file system node, it is same between each described RTSP gateway
Sample can be in communication with each other.
Preferably, the RTSP gateways can be in communication with each other and cooperate, there is provided including but not limited to:Active and standby, load
The mode of operations such as equilibrium, when distributed file system node or RTSP gateways cannot normal work when, can will be connected to failure
The client of node is automatically assigned on normal RTSP gateways;Or when the workload of certain RTSP gateway is higher, will be new
Client connection is assigned to other RTSP nodes, so as to improve the availability and fault-tolerance of whole framework.
Wherein, the distributed file system node includes that any energy provides file system mode of operation in a distributed manner
Equipment, including but not limited to:X86-based, ARM IA frame serverPCs.
Preferably, the RTSP gateways are connected with least one distributed type file system client side, and the RTSP nets
Pass can simultaneously at least one distributed type file system client side provides service.
It should be noted that the distributed file system node also includes solid state hard disc, the solid state hard disc is used to delay
Deposit the data that the RTSP gateway processes are crossed.After the data resource that the RTSP gateways will be received is processed, Ke Yixian
By in data resource caching and solid state hard disc.Specifically, solid state hard disc on each distributed file system node include but
It is not limited to monolithic to use, it is also possible to use disk array (Redundant Arrays of Independent Disks, RAID)
Performance, extension capacity are improved etc. prior art.
The present embodiment can be entered the storage of scalability and high reliability and stream medium data by the distributed system
Row is integrated, if a distributed file system node paralysis, existing client end can be automatically handed over other distributed documents
RTSP gateways on system node, such distributed file system still can provide service for client, thus, even if part
Server apparatus are gone wrong, and influence will not be also brought on client device, can be with convenient, efficiently and stably storage media number
According to resource, larger actual application value is embodied.
The preferred embodiments of the present invention are these are only, the scope of the claims of the invention is not thereby limited, it is every to utilize this hair
Equivalent structure or equivalent flow conversion that bright specification and accompanying drawing content are made, or directly or indirectly it is used in other related skills
Art field, is included within the scope of the present invention.