CN107885829A - Object storage docking Hadoop renaming method, apparatus, equipment and storage medium - Google Patents
Object storage docking Hadoop renaming method, apparatus, equipment and storage medium Download PDFInfo
- Publication number
- CN107885829A CN107885829A CN201711091477.5A CN201711091477A CN107885829A CN 107885829 A CN107885829 A CN 107885829A CN 201711091477 A CN201711091477 A CN 201711091477A CN 107885829 A CN107885829 A CN 107885829A
- Authority
- CN
- China
- Prior art keywords
- renaming
- metadata
- new
- data
- hadoop
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/10—File systems; File servers
- G06F16/18—File system types
- G06F16/182—Distributed file systems
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/30—Information retrieval; Database structures therefor; File system structures therefor of unstructured textual data
- G06F16/38—Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually
Abstract
This application discloses renaming method, apparatus, equipment and the storage medium that object stores docking Hadoop, this method includes obtaining the metadata for treating renaming data；It is newname by the name modifications of the metadata for treating renaming data, obtains new metadata；In the new metadata, the mapping of renaming data is treated described in addition, obtains new data corresponding with the new metadata, the device includes acquiring unit, and the metadata of renaming data is treated for obtaining；Unit is changed, for being newname by the name modifications of the metadata for treating renaming data, obtains new metadata；Adding device, in the new metadata, the mapping of renaming data to be treated described in addition, obtain new data corresponding with the new metadata.Above-mentioned object storage docking Hadoop renaming method, apparatus, equipment and storage medium, it is possible to increase the efficiency of renaming operation, lift the performance and robustness of storage system.
Description
Technical field
The invention belongs to technical field of distributed memory, and the renaming side for docking Hadoop is stored more particularly to object
Method, device, equipment and storage medium.
Background technology
Ceph distributed memory systems support block storage, object storage and file system simultaneously, with big data application
Constantly rise, the unstructured data on internet constantly increases, and object storage is non-structural suitable for storage picture video etc.
Change data, and scalability is strong, can adapt to the growth of mass data, therefore ceph objects are stored after being stored as Hadoop
End is increasingly valued by people.
During Ceph objects store docking Hadoop, Hadoop operates to the renaming of data storage, deposits at present
One larger the problem of, i.e., it is to delete original object when Ceph objects are stored in the renaming operation for receiving Hadoop
Remove, then re-create a new object, and when the size of object is larger, the efficiency of such renaming operation is extremely low
Under.
The content of the invention
To solve the above problems, the invention provides object store docking Hadoop renaming method, apparatus, equipment and
Storage medium, it is possible to increase the efficiency of renaming operation, lift the performance and robustness of storage system.
Object storage docking Hadoop provided by the invention renaming method, including：
Obtain the metadata for treating renaming data；
It is newname by the name modifications of the metadata for treating renaming data, obtains new metadata；
In the new metadata, the mapping of renaming data is treated described in addition, is obtained and the new metadata pair
The new data answered.
Preferably, in the renaming method that above-mentioned object stores docking Hadoop, obtained and the new member described
After new data corresponding to data, in addition to：
Receive the instruction that user obtains the new data；
Search out new metadata corresponding with the new data；
The new data corresponding with the new metadata are searched out, and the new data searched out are returned
To user.
Object storage docking Hadoop provided by the invention renaming device, including：
Acquiring unit, the metadata of renaming data is treated for obtaining；
Unit is changed, for being newname by the name modifications of the metadata for treating renaming data, obtains new member
Data；
Adding device, in the new metadata, the mapping of renaming data to be treated described in addition, obtain with it is described
New data corresponding to new metadata.
Preferably, in the renaming device that above-mentioned object stores docking Hadoop, in addition to：
Receiving unit, the instruction of the new data is obtained for receiving user；
First finds unit, for searching out new metadata corresponding with the new data；
Second finds unit, for searching out the new data corresponding with the new metadata, and will search out
The new data return to user.
Object storage docking Hadoop provided by the invention renaming equipment, including：
Memory, for storing computer program；
Processor, realize that Hadoop is docked in object storage as described in any one above during for performing the computer program
Renaming method the step of.
Storage medium provided by the invention, computer program, the computer program quilt are stored with the storage medium
The step of object storage docking Hadoop renaming method as described in any one above is realized during computing device.
By foregoing description, above-mentioned object storage docking Hadoop provided by the invention renaming method, apparatus,
Equipment and storage medium, because this method includes obtaining the metadata for treating renaming data；By the member for treating renaming data
The name modifications of data are newname, obtain new metadata；In the new metadata, renaming data are treated described in addition
Mapping, obtain new data corresponding with the new metadata, therefore this need not just delete original data, so as to
The efficiency of renaming operation is improved, lifts the performance and robustness of storage system.
Brief description of the drawings
In order to illustrate more clearly about the embodiment of the present invention or technical scheme of the prior art, below will be to embodiment or existing
There is the required accompanying drawing used in technology description to be briefly described, it should be apparent that, drawings in the following description are only this
The embodiment of invention, for those of ordinary skill in the art, on the premise of not paying creative work, can also basis
The accompanying drawing of offer obtains other accompanying drawings.
Fig. 1 is the schematic diagram of the renaming method for the first object storage docking Hadoop that the embodiment of the present application provides；
Fig. 2 is the schematic diagram of the renaming device for the first object storage docking Hadoop that the embodiment of the present application provides；
Fig. 3 is the schematic diagram of the renaming equipment for the object storage docking Hadoop that the embodiment of the present application provides.
Embodiment
The core concept of the present invention is to provide object storage docking Hadoop renaming method, apparatus, equipment and deposited
Storage media, it is possible to increase the efficiency of renaming operation, lift the performance and robustness of storage system.
Below in conjunction with the accompanying drawing in the embodiment of the present invention, the technical scheme in the embodiment of the present invention is carried out clear, complete
Site preparation describes, it is clear that described embodiment is only part of the embodiment of the present invention, rather than whole embodiments.It is based on
Embodiment in the present invention, those of ordinary skill in the art are obtained every other under the premise of creative work is not made
Embodiment, belong to the scope of protection of the invention.
The renaming method for the first object storage docking Hadoop that the embodiment of the present application provides is as shown in figure 1, Fig. 1 is
The schematic diagram of the renaming method for the first object storage docking Hadoop that the embodiment of the present application provides, this method includes as follows
Step：
S1：Obtain the metadata for treating renaming data；
S2：It is newname by the name modifications of the metadata for treating renaming data, obtains new metadata；
S3：In the new metadata, the mapping of renaming data is treated described in addition, is obtained and the new metadata
Corresponding new data.
It should be noted that this method stores the scene of docking Hadoop renaming just for object, it is right in Ceph
The actual storage data of elephant separate with metadata, and Hadoop is a kind of distributed system architecture, existing heavy life
Original metadata and actual storage data are first copied during name operation, then export original metadata and actual storage number
According to, it is then based on copy and regenerates new metadata and new actual storage data, and delete original actual storage data
The consuming time is longer, and when data are bigger, such operating efficiency is lower.And in the present embodiment, to original metadata
Title be changed to newname, while retain a map information of original actual storage data wherein, without to original
Actual storage data are modified, and this makes it possible to the efficiency for improving object renaming.
Pass through foregoing description, the renaming side for the first object storage docking Hadoop that the embodiment of the present application provides
Method, due to treating the metadata of renaming data including obtaining；Name modifications by the metadata for treating renaming data are new
Title, obtain new metadata；In the new metadata, the mapping of renaming data is treated described in addition, obtain with it is described
New data corresponding to new metadata, therefore this need not just delete original data, so as to improve renaming operation
Efficiency, lift the performance and robustness of storage system.
The embodiment of the present application provide second of object storage docking Hadoop renaming method, be it is above-mentioned the first
On the basis of object storage docking Hadoop renaming method, in addition to following technical characteristic：
It is described obtain new data corresponding with the new metadata after, in addition to：
Receive the instruction that user obtains the new data；
Search out new metadata corresponding with the new data；
The new data corresponding with the new metadata are searched out, and the new data searched out are returned
To user.
As can be seen that in this case, data are not found to user and cause any adverse effect, therefore the program
There is higher efficiency relative to prior art.
The renaming device for the first object storage docking Hadoop that the embodiment of the present application provides is as shown in Fig. 2 Fig. 2 is
The schematic diagram of the renaming device for the first object storage docking Hadoop that the embodiment of the present application provides, the device include：
Acquiring unit 201, the metadata of renaming data is treated for obtaining；
Unit 202 is changed, for being newname by the name modifications of the metadata for treating renaming data, is obtained new
Metadata；
Adding device 203, in the new metadata, the mapping of renaming data to be treated described in addition, obtain with
New data corresponding to the new metadata.
It should be noted that the device stores the scene of docking Hadoop renaming just for object, it is right in Ceph
The actual storage data of elephant separate with metadata, and Hadoop is a kind of distributed system architecture, in the present embodiment,
Newname is changed to the title of original metadata, while retains a mapping letter of original actual storage data wherein
Breath, without being modified to original actual storage data, this makes it possible to the efficiency for improving object renaming.
The embodiment of the present application provide second of object storage docking Hadoop renaming device, be it is above-mentioned the first
On the basis of object storage docking Hadoop renaming device, in addition to following technical characteristic：
Also include：
Receiving unit, the instruction of the new data is obtained for receiving user；
First finds unit, for searching out new metadata corresponding with the new data；
Second finds unit, for searching out the new data corresponding with the new metadata, and will search out
The new data return to user.
As can be seen that in this case, data are not found to user and cause any adverse effect, therefore the program
There is higher efficiency relative to prior art.
The renaming equipment for the object storage docking Hadoop that the embodiment of the present application provides is as shown in figure 3, Fig. 3 is the application
The schematic diagram of the renaming equipment for the object storage docking Hadoop that embodiment provides, the equipment include：
Memory 301, for storing computer program；
Processor 302, realize during for performing the computer program and docked as any object above stores
The step of Hadoop renaming method.
The storage medium that the embodiment of the present application provides, computer program, the computer are stored with the storage medium
Realized when program is executed by processor such as the step of any object stores the renaming method for docking Hadoop above.
In summary, the present embodiment need to only change object metadata, be achieved that the renaming of object, improve object weight
The execution efficiency of naming operation, system is reduced because mass data changes the possibility to throw into question, improves storage system
Robustness.
The foregoing description of the disclosed embodiments, professional and technical personnel in the field are enable to realize or using the present invention.
A variety of modifications to these embodiments will be apparent for those skilled in the art, as defined herein
General Principle can be realized in other embodiments without departing from the spirit or scope of the present invention.Therefore, it is of the invention
The embodiments shown herein is not intended to be limited to, and is to fit to and principles disclosed herein and features of novelty phase one
The most wide scope caused.
Claims (6)
1. object storage docking Hadoop renaming method, it is characterised in that including：
Obtain the metadata for treating renaming data；
It is newname by the name modifications of the metadata for treating renaming data, obtains new metadata；
In the new metadata, the mapping of renaming data is treated described in addition, is obtained corresponding with the new metadata
New data.
2. object storage docking Hadoop according to claim 1 renaming method, it is characterised in that obtained described
After new data corresponding with the new metadata, in addition to：
Receive the instruction that user obtains the new data；
Search out new metadata corresponding with the new data；
The new data corresponding with the new metadata are searched out, and the new data searched out are returned into use
Family.
3. object storage docking Hadoop renaming device, it is characterised in that including：
Acquiring unit, the metadata of renaming data is treated for obtaining；
Unit is changed, for being newname by the name modifications of the metadata for treating renaming data, obtains new metadata；
Adding device, in the new metadata, the mapping of renaming data to be treated described in addition, obtain with it is described new
New data corresponding to metadata.
4. object storage docking Hadoop according to claim 3 renaming device, it is characterised in that also include：
Receiving unit, the instruction of the new data is obtained for receiving user；
First finds unit, for searching out new metadata corresponding with the new data；
Second finds unit, for searching out the new data corresponding with the new metadata, and the institute that will be searched out
State new data and return to user.
5. object storage docking Hadoop renaming equipment, it is characterised in that including：
Memory, for storing computer program；
Processor, realize that object storage is docked as described in any one of claim 1 to 2 during for performing the computer program
The step of Hadoop renaming method.
6. storage medium, it is characterised in that computer program is stored with the storage medium, the computer program is processed
The step of object storage docking Hadoop renaming method as described in any one of claim 1 to 2 is realized when device performs.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201711091477.5A CN107885829A (en) | 2017-11-08 | 2017-11-08 | Object storage docking Hadoop renaming method, apparatus, equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201711091477.5A CN107885829A (en) | 2017-11-08 | 2017-11-08 | Object storage docking Hadoop renaming method, apparatus, equipment and storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN107885829A true CN107885829A (en) | 2018-04-06 |
Family
ID=61779581
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201711091477.5A Pending CN107885829A (en) | 2017-11-08 | 2017-11-08 | Object storage docking Hadoop renaming method, apparatus, equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN107885829A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109756484A (en) * | 2018-12-12 | 2019-05-14 | 杭州数梦工场科技有限公司 | Control method, control device, gateway and the medium of gateway based on object storage |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103150394A (en) * | 2013-03-25 | 2013-06-12 | 中国人民解放军国防科学技术大学 | Distributed file system metadata management method facing to high-performance calculation |
| CN103229173A (en) * | 2012-12-26 | 2013-07-31 | 华为技术有限公司 | Metadata management method and system |
| CN106484820A (en) * | 2016-09-26 | 2017-03-08 | 华为技术有限公司 | A kind of renaming method, access method and device |
-
2017
- 2017-11-08 CN CN201711091477.5A patent/CN107885829A/en active Pending
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103229173A (en) * | 2012-12-26 | 2013-07-31 | 华为技术有限公司 | Metadata management method and system |
| CN103150394A (en) * | 2013-03-25 | 2013-06-12 | 中国人民解放军国防科学技术大学 | Distributed file system metadata management method facing to high-performance calculation |
| CN106484820A (en) * | 2016-09-26 | 2017-03-08 | 华为技术有限公司 | A kind of renaming method, access method and device |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109756484A (en) * | 2018-12-12 | 2019-05-14 | 杭州数梦工场科技有限公司 | Control method, control device, gateway and the medium of gateway based on object storage |
| CN109756484B (en) * | 2018-12-12 | 2021-10-15 | 杭州数梦工场科技有限公司 | Object storage-based gateway control method, control device, gateway and medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US10108543B1 (en) | Efficient physical garbage collection using a perfect hash vector | |
| CN103838830B (en) | A kind of data managing method and system of HBase data bases | |
| US10152773B2 (en) | Creating a blurred area for an image to reuse for minimizing blur operations | |
| US9063947B2 (en) | Detecting duplicative hierarchical sets of files | |
| CN103544156B (en) | File memory method and device | |
| CN106599091B (en) | RDF graph structure storage and index method based on key value storage | |
| CN102142032B (en) | Method and system for reading and writing data of distributed file system | |
| CN103139224A (en) | Network file system and method for accessing network file system | |
| US20180129736A1 (en) | System to organize search and display unstructured data | |
| CN107704203A (en) | It polymerize delet method, device, equipment and the computer-readable storage medium of big file | |
| US20160224581A1 (en) | Recursive Multi-Threaded File System Scanner For Serializing File System Metadata Exoskeleton | |
| CN110399348A (en) | File deletes method, apparatus, system and computer readable storage medium again | |
| US7483906B2 (en) | Method and system for renaming consecutive keys in a B-tree | |
| US8880553B2 (en) | Redistribute native XML index key shipping | |
| CN106844089A (en) | A kind of method and apparatus for recovering tree data storage | |
| CN106547644A (en) | Incremental backup method and equipment | |
| CN107168651A (en) | A kind of small documents polymerize storage processing method | |
| Hu et al. | Towards big linked data: a large-scale, distributed semantic data storage | |
| CN103403709B (en) | A kind of methods, devices and systems of reading and writing data | |
| CN107885829A (en) | Object storage docking Hadoop renaming method, apparatus, equipment and storage medium | |
| CN106503186A (en) | A kind of data managing method, client and system | |
| CN107729432A (en) | A kind of storage of distributed small documents, read method, device and access system | |
| KR101252375B1 (en) | Mapping management system and method for enhancing performance of deduplication in storage apparatus | |
| Lewis et al. | Towards multimedia thesaurus support for media-based navigation | |
| CN101499096A (en) | Distributed reptile cluster system |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20180406 |